Diligent Robotics - Founding Sr. Data Engineer

Remote - USA1mo ago

Remote Senior NA Software Oil & Gas Data Engineer dbt Snowflake Redshift SQL Fivetran

Requirements

• 5+ years of hands-on data engineering experience, including significant work in a modern data stack (cloud warehouse + dbt + version-controlled transformations). • Deep, hands-on experience with BigQuery (or another columnar cloud warehouse — Snowflake, Redshift — with willingness to ramp on BigQuery quickly). • BigQuery • Deep, hands-on experience with dbt, including project structure, testing, and modeling patterns. You can debate the tradeoffs of staging vs. intermediate vs. marts layering and have opinions about when to use incremental models. • Strong dimensional modeling fundamentals (Kimball or equivalent). You understand conformed dimensions, slowly changing dimensions, and grain. • Strong SQL. Comfortable reviewing and refactoring complex SQL written by analysts. • Experience designing ingestion pipelines from multiple source types — operational databases, SaaS APIs, log-based sources. • Excellent technical communication. You can explain tradeoffs and modeling decisions to analysts, engineers, and executives. • Comfortable being the first dedicated data engineer in an organization — setting standards, making decisions without a peer to defer to, partnering with non-data folks. • Experience with Firestore and the native Firestore-to-BigQuery integration, particularly at scale (multi-project setups). • Firestore • Experience with Airbyte (self-hosted or cloud) or comparable managed ingestion tools (Fivetran, Stitch). • Airbyte • Experience working with Elastic as a data source — extracting metrics or operational data out of Elastic indexes into downstream systems. • Elastic • Experience designing a metrics catalog or semantic layer (dbt Semantic Layer, MetricFlow, LookML, or similar). • metrics catalog

Responsibilities

• Stand up the warehouse. Design and implement our BigQuery environment from the ground up — project structure, IAM, dataset organization, naming conventions, cost controls — so it scales cleanly as we add sources and users. • Stand up the warehouse. • Build the transformation layer. Set up dbt as the home for our transformation logic. Migrate the business logic from our existing analytics environment into version-controlled, tested dbt models. • Build the transformation layer. • Model master data. Introduce dimensional models for our core business entities so data from different sources can be joined consistently. • Model master data. • Own ingestion. Configure native Firestore-to-BigQuery integration, then stand up Airbyte as our managed connector layer for SaaS sources and log exports as we broaden the warehouse. • Own ingestion. • Partner across the org. Work with source-system owners, the analyst team, and engineering teams that will eventually consume curated data from the warehouse. Help define the boundary between what the warehouse is and is not for. • Partner across the org.