Pro members applied to this job 36 hours before you saw itGet Pro ›

soum - Data Engineer

Remote4d ago

Remote Mid WW Cloud Computing Artificial Intelligence Data Engineer SQL Python AWS GCP Azure BigQuery Snowflake Redshift Airflow Docker Kubernetes Airbyte GitHub Git Reporting Governance Kafka Data Governance Data Quality

Requirements

• Proficiency in Python and SQL for data engineering tasks. • Strong understanding of ETL/ELT processes, data warehousing, and data modeling. • Hands-on experience with cloud platforms (AWS, GCP, or Azure) and data storage solutions (BigQuery, Redshift, Snowflake, etc.). • Familiarity with data orchestration tools Airflow, Airbyte is a must. • Experience with containerization & deployment tools (Docker, Kubernetes) is a plus. • Knowledge of data governance, security, and best practices for handling sensitive data.Familiarity to work with Git and GitHub. • Dataform is a must • Strong skills in eliciting requirements from cross-functional stakeholders and translating them into actionable data engineering tasks. • Preferably total 4 years of experience • 2+ years in data engineering, building and maintaining data pipelines. • 2+ years of SQL and Python development in production environments. • Experience working in Fast-growing startup environments is a plus. • Exposure to real-time data processing frameworks (Kafka, Spark, Flink) is a plus.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines to support analytics, ML models, and business reporting. • Collaborate with data scientists and analysts to ensure data is available, clean, and optimized for downstream use. • Implement data quality checks, monitoring, and validation processes. • Design efficient ETL/ELT workflows and integrate multiple data sources (databases, APIs, third-party tools) into centralized storage (data lakes/warehouses). • Support cloud-based infrastructure for data storage and retrieval. • Monitor, troubleshoot, and optimize existing pipelines for large-scale, real-time data flows. • Apply best practices for query optimization and cost-efficient storage. • Partner with product, engineering, and business stakeholders to gather requirements. • Document workflows, schemas, and processes while supporting a culture of reliability, governance, and security.