people-data-labs - Senior Data Engineer

• Expertise with Apache Spark (Java, Scala, and/or Python-based) • Experience building scalable data processing systems (e.g., cleaning, transformation)  from the ground up. • Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar) • Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills) • Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.) • Experience with cloud computing services (AWS (preferred), GCP, Azure or similar) • Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar) • Understanding of modern data storage formats and tools (e.g., parquet, ORC, Avro, Delta Lake) • People Thrive Here Who Can • Balance high ownership and autonomy with a strong ability to collaborate • Work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities) • Demonstrate strong written communication skills on Slack/Chat and in documents • Exhibt experience in writing data design docs (pipeline design, dataflow, schema design) • Scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders • Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering • Experience working with entity data (entity resolution / record linkage) • Experience working with data acquisition / data integration • Expertise with Python and the Python data stack (e.g., numpy, pandas) • Experience with streaming platforms (e.g., Kafka) • Experience evaluating data quality and maintaining consistently high data standards across new feature releases (e.g., consistency, accuracy, validity, completeness)