Senior Data Engineer, Data Platform
Upload My Resume
Drop here or click to browse · PDF, DOCX, DOC, RTF, TXT
Requirements
• 5+ years of deep experience in modern, cloud-based data engineering. You have a proven track record of building and maintaining robust platforms that enable the discovery, query, and processing of large-scale datasets. • Expertise in Core Technologies (Mandatory): You possess advanced proficiency in Python and SQL and experience with Containerization (Docker, Kubernetes), Infrastructure as Code (Terraform, etc.), and Agentic Development (Cursor, Claude, etc.) • Expertise in Core Technologies (Mandatory): • Python and SQL • Containerization (Docker, Kubernetes) • Infrastructure as Code (Terraform, etc.) • Agentic Development • Data Fundamentals & Architecture: You have a deep understanding of Relational Databases (Postgres, MySql, etc.) and are proficient in working with data container files (e.g., Parquet, Avro, etc.). You are familiar with Medallion Architecture (Bronze/Silver/Gold) and know how to apply its principles to build scalable, reliable data products. • Data Fundamentals & Architecture: • data container files (e.g., Parquet, Avro, etc.) • Medallion Architecture • Cloud & Platform Operations: You are seasoned in at least one major cloud provider (GCP, AWS, or Azure). You are comfortable in a "DevOps" capacity—specifically managing the infrastructure and tools that empower other engineers to run their pipelines, rather than focusing solely on building ETL flows. • Cloud & Platform Operations: • GCP, AWS, or Azure • Hands-On: You understand the trade-offs between different architectures (Data Lake vs. Warehouse) and can make high-level decisions regarding CI/CD and system maintenance. Crucially, you are a "doer" who enjoys getting into the weeds of implementation, not only designing the blueprint. • Hands-On: • Collaborative Problem Solver: You thrive in environments with significant technical ambiguity. You can navigate complex systems and diverse technologies to deliver solutions that are easily monitored, quantified, and understood in production. • Collaborative Problem Solver: • People-First Mindset: You prioritize supporting your coworkers’ growth. We value a culture where meeting deadlines never comes at the expense of being a helpful, empathetic teammate. • People-First Mindset: • Intellectual Curiosity: While a background in drug discovery, biology, or chemistry isn't required, a drive to learn the domain and explore new technologies is essential! • Intellectual Curiosity: • Our Tech StackWhile we require the core competencies above, you will be working directly with our specific stack: • Cloud: Google Cloud Platform (GCP), GCS, Pub/Sub. • Cloud: • Tools: dbt, Prefect, Argo Workflows, BigQuery, Snowflake, Fivetran, Datastream, AI/LLM (Cursor, Claude Code, and Gemini available). • Tools: • Infrastructure: Kubernetes, PostgreSQL, CI/CD (Argo). • Infrastructure: • Nice to Haves: Experience with Search (Elasticsearch), Vector/Graph databases, or deep experience scaling within the GCP ecosystem is a significant plus.
Responsibilities
• Build, scale, and operate a data platform. You will be a member of the platform team responsible for building, operating, and tuning a data platform that allows users to discover and query across the breadth of our data at Recursion, which includes a chemistry library of billions compounds, petabytes of cellular microscopy images taken in millions of different experimental contexts, and millions of assay results, all supporting Recursion’s drug discovery. • Build, scale, and operate a data platform • Build relatability into a heterogeneous dataset. At Recursion, we generate datasets based on a wide swath of diverse biological models and treatment approaches. You'll work with biologists, chemists, and data scientists to build relatability and query-ability into these datasets so they can be used in the future to answer the sorts of questions we haven't even thought of asking yet. • Build relatability into a heterogeneous dataset. • Act as a mentor, coach, and sponsor. You will share your technical knowledge and experiences, delivering impact, learning, and growth across teams at Recursion. • Act as a mentor, coach, and sponsor. • The Team You’ll Join • You will join the Data Lake team that built and maintains our Data Lake/house. The team is responsible for relational and object storage and has the motto: all data flows to the Data Lake. The team solves the problem of making our diverse data discoverable, queryable, and relatable across datasets while we continue to add new data modalities as we grow. This will require collaboration with many different groups including teams building out reports, dashboards, and applications, teams finding and generating the required data for machine learning problems, and teams building and iterating on new data processing pipelines. • all data flows to the Data Lake
Benefits
• This position can be based at either our London or Milton Park office. Please note that we are a hybrid environment and ask that employees spend 50% of their time in the office. • London • Milton Park office