Gather AI - Principal Data Engineer
Requirements
• 10+ years in data engineering, with 3+ years architecting data platforms for data products, analytics, or AI-driven products. • Proven experience building a greenfield data warehouse and leading an OLTP to OLAP transition, not just maintaining an existing one. • Deep expertise designing multi-layer transformation architectures and reusable frameworks that scale across multiple product areas. • Expert SQL and dbt, hands-on ELT and orchestration, and large-scale or streaming data experience. • Production experience on a major cloud (Azure preferred, AWS or GCP acceptable), plus infrastructure as code and CI/CD. • Track record with data quality, security, governance, and multi-tenancy in production environments. • Data transformation and modeling that turns raw multi-source data into refined, serving-ready datasets (raw to refined to serving). • Pipeline orchestration and workflow automation for scheduling, dependency management, and reliable execution across data flows. • Large-scale and distributed processing of high-volume batch data. • Real-time and streaming ingestion that captures and processes event data as it arrives. • Semantic and metrics-layer design that defines business metrics once and serves them consistently to every consumer. • Serving-layer optimization for fast, low-latency consumption through wide and flattened tables and pre-computed metrics. • Cloud data engineering and infrastructure automation that provisions, deploys, and operates the platform reproducibly (cloud-native, infrastructure as code, CI/CD). • Data quality, observability, and lineage that ensure trust, freshness, and end-to-end traceability. • Security, governance, and multi-tenancy including tenant isolation, access control, and resiliency. • Multimodal data integration that links structured records to unstructured image and video (drone captures) with traceability. • Ways of Working • Ways of Working • Treats data as a product for internal consumers, not just a pipeline feeding dashboards. • Comfortable making long-lead architecture calls (platform, isolation model) with incomplete consensus. • Strong cross-functional collaborator, works closely with integration/backend, ML, product, customer success teams and internal analytics consumers. • Experience modeling structured data linked to unstructured or blob data such as images, video, or sensor files • Experience with feature stores, annotation pipelines, or ML data infrastructure supporting computer vision products. • IoT, edge, or device-telemetry background • BI or presentation-layer and dashboard design experience • Warehousing, logistics, or supply-chain domain knowledge
Responsibilities
• Architect a greenfield, multi-layer data warehouse (raw, refined, serving) that separates analytical workloads from production OLTP traffic. • Deliver a governed, self-service data-access layer for internal consumers first (Product, CSM, Deployment/Operations, and Leadership) as Phase 1, ahead of customer-facing conversational analytics. • Build a semantic and metrics layer so every metric, such as "scan accuracy by site," is defined once in code and stays identical across every dashboard and product, making self-service safe from metric drift. • Own the quality bar: 99%+ availability SLA with freshness guarantees, 100% traceability, zero cross-tenant leakage, 99.5%+ pipeline success, and no data loss. • Design tenant isolation, per-tenant cost attribution, and schema and row-level RBAC to scale toward hundreds of tenants (300+ target), not today's fleet size. • Own data-ingestion correctness at the boundary with the integration/backend team, covering data contracts, schema validation, and pipeline quality, so WMS data lands in the right place, shape, and time across WMS versions. • Stand up a data catalog and lineage layer (Purview as the Azure-native fit, DataHub as the open-source alternative) so every consumer can find data, see ownership, and trace lineage when a metric looks wrong. • Prove the foundation end to end on Gather's drone product, then generalize it so each new product extends the model instead of rebuilding it • Act as the connective tissue between product and ML (3DCC, damage detection). Link structured records to unstructured drone imagery and video with full traceability, and stand up the data-infra readiness for feature stores and annotation pipelines on one trusted foundation.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT