Graphcore - Lead Data Engineer

• Lead the design, build and evolution of robust data pipelines and platform services that support analytics, reporting and operational use cases across Graphcore. • Own the data engineering stack, planning and delivering improvements to reliability, scalability, maintainability, performance and security. • Build and operate Python-based batch and streaming workflows, with clear approaches to orchestration, testing, deployment, monitoring and incident resolution. • Design and implement data solutions on AWS using services such as S3, Lambda, Aurora PostgreSQL, Athena, Glue and Redshift, ensuring they are secure, resilient and cost-conscious. • Define and apply engineering standards for data quality, observability, documentation, release processes and operational support. • Partner with analysts, engineers and business stakeholders to translate requirements into trusted datasets, well-structured data models and reusable data products. • Drive improvements to platform resilience through approaches such as idempotent processing, retry and recovery mechanisms, buffering strategies and backfill or replay capabilities. • Lead technical decision-making in your area by reviewing designs and code, sharing expertise and helping to raise the quality bar for data engineering across the team. • Build and maintain CI/CD workflows and development practices that enable safe, repeatable and efficient delivery of data infrastructure and workflows. • Ensure appropriate data protection and access controls are in place, including least-privilege access, secure secrets handling and suitable database permissions. • Contribute to the development of internal tools and lightweight applications that improve access to data and support self-serve workflows. • Work across teams to identify opportunities for platform and process improvements, helping shape the direction of data engineering within the wider Data & Analytics function. • Candidate Profile • Candidate Profile • Essential • Strong experience designing, building and operating production-grade data pipelines and data platforms in Python. • Strong hands-on experience with modern data orchestration, testing, deployment and monitoring practices in a production environment. • Experience building solutions on AWS data services, including storage, processing and query technologies. • Strong understanding of data modelling, data quality, schema design and performance optimisation across relational and analytical systems. • Experience designing reliable data systems that recover gracefully from failure and operate effectively in real-world production conditions. • Experience working with batch and streaming data pipelines, including operational support, troubleshooting and continuous improvement. • Strong knowledge of security and access control principles for data platforms, including IAM, database permissions and secure handling of credentials and secrets. • Experience providing technical leadership as a senior individual contributor through design reviews, code reviews, standards-setting and mentoring of others. • Ability to work effectively with both technical and non-technical stakeholders, turning business needs into practical, scalable data solutions. • Strong communication skills, with the ability to explain technical decisions clearly and influence outcomes across teams. • Desirable • Experience with Prefect or a similar workflow orchestration platform. • Experience with streaming or data collection technologies. • Experience with PostgreSQL, Redshift, ClickHouse or similar database and warehouse technologies. • Experience with CI/CD tooling and Infrastructure as Code approaches. • Experience building lightweight internal tools or data applications using Python frameworks such as Streamlit or Flask. • Familiarity with dbt and working models that combine data engineering and analytics engineering. • Understanding of operational best practices for cloud-based data platforms, including cost optimisation and observability. • Experience working in a fast-moving product, technology or engineering-led environment.