3Pillar - Lead Data Engineer

• Writing code and implementing the proposed solutions • Creating data pipelines, versioning and change management • Manage the complexity inherent in versioned data pipelines • Develop ETL/ELT processes to help extract and manipulate data from multiple sources. • Design, build and maintain batch or real-time data pipelines in production. • Automate data workflows such as data ingestion, aggregation, and ETL processing. • Logging and instrumentation of pipelines and services. • Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures. • Train and mentor junior engineers.