IT Labs - Lead Data Engineer
Requirements
• 7+ years of experience in data engineering within cloud or hybrid environments. • Strong experience designing, building, and maintaining production-scale ETL/ELT pipelines. • Advanced SQL skills and proficiency with data transformation techniques. • Strong Python programming skills for data processing and automation. • Hands-on experience with Snowflake, including: • Query optimization • Experience migrating from BigQuery or similar columnar data warehouse technologies. • Experience with batch and streaming data processing platforms such as Kafka, Event Hub, or equivalent. • Experience building enterprise-grade API and source-system integrations. • Experience implementing data quality and validation frameworks. • Good understanding of: • Data governance • Access control and security best practices • Strong problem-solving and communication skills. • Ability to work effectively in a collaborative, cross-functional environment. • Spark-based pipelines • Experience with Spark, Scala, or PySpark. • Familiarity with Tableau. • Understanding of HIPAA and HITRUST data handling practices. • Experience with dbt (data build tool) for transformation layer management. • Experience working within healthcare, insurance, or regulated industries. • Working Conditions • Full-time, 40h/week • Contract or B2B arrangement • We are a company that seeks the best for both our employees and clients, reaching beyond expectations in turning dreams into reality. Our way of working is rooted in our core values (Integrity, Excellence, Proactivity, Innovation, and People), with an expectation that our future colleagues will make these their second nature in their everyday work and life. We don’t ask for perfection, but we do appreciate people motivated to better themselves in every conceivable aspect.
Responsibilities
• Lead the design, development, and optimization of scalable data platforms and pipelines. • Design, build, and maintain production-grade ETL/ELT workflows for batch and near real-time data processing. • Drive the migration and modernization of data assets from BigQuery and other analytical platforms into Snowflake. • Develop robust integrations with enterprise applications, APIs, and external data providers. • Build and optimize Snowflake data models, schemas, Snowpipe ingestion processes, and query performance. • Collaborate with business stakeholders, analysts, and engineering teams to translate business requirements into scalable data solutions. • Implement data quality, validation, monitoring, and observability frameworks. • Establish and promote best practices for data governance, lineage tracking, metadata management, and security. • Support both batch and streaming data architectures using technologies such as Kafka, Event Hub, or equivalent. • Mentor other engineers and contribute to architectural decisions, technical standards, and engineering best practices. • Ensure data platforms meet performance, reliability, scalability, and regulatory requirements.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT