Data Engineer (Part-Time)

• Data Reliability & Pipeline Monitoring • Ensure data pipelines run reliably and data is fresh, accurate, and available as expected. • Monitor, build, and respond to Daton pipeline notifications and alerts • Track data latency, freshness, and completeness across all source systems • Design, build, and maintain QC processors for all source data and custom reports • Monitor job execution, investigate failures, and perform root cause analysis at: • QC / validation level • API / source system level • Create and enhance data pipelines, onboard new platform integrations, and implement logic changes to existing pipelines • Coordinate with source system owners and vendors when issues originate upstream • Monitor alerts from source systems and custom reports • Ensure overall data reliability through proactive monitoring and validation • Optimize query performance and warehouse costs • Maintain documentation for all logic, schema, and pipeline changes, with a continuously updated change log • Data Quality & Validation • Implement and maintain automated data quality checks (source + reports) to build trust and confidence in data across the organization. • Key Activities: • Monitor and respond to data quality and test failures • Implement automated validation checks, including: • Duplicate detection • Range and boundary checks • Valid value checks • Referential integrity checks • Implement business-logic validations for key KPIs • Perform daily validation of critical metrics against source UIs (Shopify, GA4, Meta, Klaviyo, Google Ads, Loop, etc.) • Ensure KPI consistency across raw, transformed, and reporting layers • Implement anomaly detection for key tables and metrics • Cost Optimization • Optimize warehouse performance and manage costs proactively to ensure sustainable data operations. • Key Activities: • Monitor and respond to billing alerts for BigQuery, dbt, and ETL tools • Maintain cost monitoring dashboards • Implement and optimize table partitioning and clustering • Optimize incremental loads and expensive queries • Proactively flag high-cost queries via Slack • Query performance optimization (where applicable) • Source System Monitoring & (API) Integration Management • Proactively manage issues originating from upstream systems and maintain healthy integrations. • Key Activities: • Monitor and respond to source schema and data-type changes • Handle source delays caused by API limits, downtime, or auth failures • Coordinate with vendors and internal teams to resolve upstream issues • Assess business impact and classify incidents as P0/P1 when required • Security & Compliance • Ensure data access and handling align with regulatory requirements and security best practices. • Key Activities: • Maintain GDPR, CCPA, and related compliance controls • Manage RBAC and column-level security in BigQuery • Ensure PII masking and access restrictions • Respond to security incidents related to data access or credentials • Documentation & Change Management • Ensure operational continuity and knowledge transfer • Key Activities: • Maintain documentation for pipelines, tables, and business logic • Update test cases for logic or schema changes • Document incident RCA and resolutions • Maintain operational runbooks • Manage logic and schema change requests from business teams