Capco - Senior Azure Data Engineer (Databricks)
Requirements
• Proven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions. • Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks. • Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern Data Lakehouse architectures. • Experience with schema design, GDPR-compliant data solutions, DevOps tooling, and CI/CD processes. • A collaborative approach to engineering with experience using AI-enabled tools responsibly to improve productivity while maintaining appropriate human oversight, governance, and solution quality. • Development experience using Scala or Java. • Familiarity with Cloudera, Hadoop, Hive, Spark, and related distributed data ecosystems. • Experience working with sensitive data and applying data privacy regulations, including GDPR. • Curiosity to explore new technologies, including responsible AI and automation, to continuously improve engineering practices. • Passion for collaboration, knowledge sharing, and continuous learning within high-performing engineering teams.
Responsibilities
• Design and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services. • Build real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs, applying AI-enabled engineering approaches where appropriate to improve delivery efficiency. • Develop and maintain CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions, and modern DevOps practices to support reliable deployments. • Collaborate with clients and multidisciplinary teams to translate business requirements into scalable, production-ready data solutions. • Champion clean code, data lifecycle optimisation, engineering best practices, and the responsible use of approved AI tools and automation to improve collaboration and delivery.
Benefits
• Deliver high-impact technology solutions for Tier 1 financial institutions • Work in a collaborative, flat, and entrepreneurial consulting culture • Access continuous learning, training, and industry certifications • Be part of a team shaping the future of digital financial services • Help shape the future of digital transformation across FS & Energy. • We offer a competitive, people-first benefits package designed to support every aspect of your life: • Core Benefits: Discretionary bonus, competitive pension, health insurance, life insurance and critical illness cover. • Mental Health: Easy access to CareFirst, Unmind, Aviva consultations, and in-house first aiders. • Mental Health: • Family-Friendly: Maternity, adoption, shared parental leave, plus paid leave for sickness, pregnancy loss, fertility treatment, menopause, and bereavement. • Family-Friendly: • Family Care: 8 complimentary backup care sessions for emergency childcare or elder care. • Family Care: • Holiday Flexibility: 5 weeks of annual leave with the option to buy or sell holiday days based on your needs. • Holiday Flexibility: • Continuous Learning: Your growth, your way - minimum 40 hours of training annually. Take your pick; workshops, certifications, e-learning. Also, Business Coach assigned from Day One: Get one-on-one guidance to fast-track your goals and accelerate your development. • Continuous Learning: • Healthcare Access: Convenient online GP services. • Healthcare Access: • Extra Perks: Gympass(Wellhub), travel insurance, Tastecard, season ticket loans, Cycle to Work, and dental insurance. • Inclusion at Capco We’re committed to making our recruitment process accessible and straightforward for everyone. If you need any adjustments at any stage, just let us know – we’ll be happy to help. We value each person’s unique perspective and contribution. At Capco, we believe that being yourself is your greatest strength. Our #BeYourselfAtWork culture encourages individuality and collaboration – a mindset that shapes how we work with clients and each other every day. #LI-DNI • Inclusion at Capco
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT