Excella - Lead Consultant Data Engineer
Requirements
• 6+ years of experience in data engineering and related technical domains. • Designed and built scalable, extensible data pipelines for production environments with strong expertise in SQL, Python, and PySpark. • 2+ years of hands-on experience with Databricks, including development, data processing, and pipeline optimization in a cloud-based environment • Hands-on experience with big data and distributed systems, Parquet, Delta Lake, HDInsight • Extensive work with cloud-native data platforms and services, such as Azure Data Factory, Synapse, Azure Storage Accounts, Streaming Analytics, Azure Fabric • Deep understanding of data lake architecture, secure and performant data store design, and analytical platform patterns. • Proficient in orchestration tools and CI/CD workflows, infrastructure-as-code (e.g., Terraform, Azure DevOps Pipelines), and observability best practices. • Advanced Git and Azure DevOps Repos expertise and strong familiarity with version control, branching strategies, and collaborative development. • Leadership experience mentoring engineers and guiding cross-functional technical teams. • Excellent communicator, capable of translating complex technical ideas for both technical and non-technical audiences. • Agile practitioner with experience leading and delivering within Scrum/Kanban frameworks, and a continuous learning mindset. • Technically savvy, entrepreneurial spirit who thrives in environments that reward self-initiative and resourcefulness. • Advocating for adopting industry tools and practices at the right time • Appreciate the importance of schema design and can evolve an analytics schema on top of unstructured data. • Leadership and mentoring of team members on project and within Excella. • Excited to try out new technologies and produce proofs-of-concept that balance technical advancement and user experience. • Empathetic working with stakeholders, listening to them, asking the right questions, and collaboratively developing the best solutions for their needs. • Champion for data privacy and integrity, and always act in the best interest of clients.
Responsibilities
• Lead the design, development, and optimization of scalable data solutions. • Define data architecture strategies and ensure best practices are followed. • Oversee and guide the creation and automation of data pipelines and platforms. • Establish and enforce data quality and governance frameworks. • Collaborate with Architects, Product Owner, Data Scientists, and DevOps to align data solutions with business needs. • Research and evaluate emerging data technologies and methodologies. • Ensure seamless integration of data management solutions into client environments. • Develop risk mitigation strategies and implement data recovery plans. • Lead the development of data repositories, including data warehouses, data lakes, and operational data stores. • Mentor and support the development of junior and senior team members.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT