Proven experience building production-grade streaming systems and handling complex event-driven data edge cases like backfills and idempotency.
Strong proficiency in Python, SQL, and modern data stack tools such as Airflow, dbt, and AWS data services.
Pragmatic mindset with the ability to bridge data engineering with AI/ML requirements, including feature generation and RAG-ready datasets.
Who are Jack & Jill?
Ok, I'll go first. I'm Jack, an AI that gets to know you on a quick call, learning what you're great at and what you want from your career. Then I help you land your dream job by finding unmissable opportunities as they come up, supporting you with applications, interview prep, and moral support.
And I'm Jill, an AI Recruiter who talks to companies to understand who they're looking to hire. Then I recruit from Jack's network, making an introduction when I spot an excellent candidate.
Step 3. Talk to Jack so he can understand your experience and ambitions.
Step 4. Jack will make sure Jill (the AI agent working for the company) considers you for this role.
Step 5. If Jill thinks you're a great fit and her client wants to meet you, they will make the introduction.
Step 6. If not, Jack will find you excellent alternatives. All for free.
We never post fake jobs
Give Jack a spin! You could land this role. If not, most people find him incredibly helpful with their job search, and we're giving his services away for free.
Responsibilities
Design and maintain high-volume batch and streaming pipelines using Kafka, SQS, and Spark for real-time AI workflows.
Develop scalable data models and transformation layers using dbt and lakehouse patterns to support ML and analytics.
Deploy and manage AWS data infrastructure using Terraform/CDK, ensuring high reliability, observability, and cost-effective performance.
Benefits
Opportunity to shape the data foundations of agentic AI systems that drive real-time, revenue-critical advertising decisions.
Join a well-funded, distributed team where you have significant autonomy to influence technical decisions and scale production infrastructure.
Potential to transition into a Tech Lead role while working with a modern stack including Spark, Kafka, and agentic coding tools.