Defense Unicorns - Senior FDE Data Engineer
Requirements
• Data engineering breadth • Unstructured data at scale. Production experience storing and querying large unstructured datasets using data lake architectures. Spark strongly preferred. • Unstructured data at scale. • Streaming & integration. Building and operating stream processing infrastructure (Kafka, Redpanda, Flink, or equivalent) and bridging data from heterogeneous sources into modern pipelines. • Streaming & integration. • Data warehousing. Open-source data warehousing platform experience. These environments do not support proprietary platforms, so you need to be comfortable building without them. • Data warehousing. • Pipelines & orchestration. Airflow, Dagster, Argo Workflows, or similar. Comfort building, scheduling, monitoring, and recovering production data pipelines. • Pipelines & orchestration. • Data modeling & SQL. Fluent in SQL. Comfortable designing schemas for both analytical and operational workloads. • Data modeling & SQL. • Open-source orientation. You are comfortable building on open-source tooling and contributing back to it. • Open-source orientation. • What "forward-deployed" requires • U.S. citizenship and the ability to obtain and maintain a DoD security clearance. Clearance sponsorship available for the right candidate. • Comfort being the technical face of Defense Unicorns to a mission hero. Clear communication with both technical and non-technical stakeholders. • Comfort with periodic on-site work, sometimes for days at a stretch, and equal comfort working remotely. • Bias toward delivery. Preference for shipping a working integration over perfecting a design that hasn't met a real workload. • Self-direction. You will encounter environments and problems that are not yet documented and will need to work through them independently. • Willingness to mentor junior engineers and help them build technical judgment through hands-on experience. • Data provenance, lineage, and governance. Experience with lineage tracking, data catalogs, provenance systems, or governance frameworks. Depth here will be weighted heavily. • Data provenance, lineage, and governance. • DoD or defense program experience. • Active Secret clearance (or higher). • Active Secret clearance • Lakehouse & storage: Apache Iceberg (or Delta/Hudi), object storage (Ceph/S3-compatible), Postgres (including extensions like pgvector), columnar/OLAP engines (Trino, DuckDB, ClickHouse, Spark SQL). • Change Data Capture: Debezium or similar CDC patterns. • Governance, catalog & access: REST catalogs (Iceberg REST, Polaris/Gravitino/Nessie family), ABAC/RBAC patterns, OIDC/OAuth, lineage and audit. • Kubernetes awareness. Deep K8s expertise is not required; general familiarity with deployments, operators, and how applications run on Kubernetes is valuable. • Linux fundamentals, container runtime behavior, networking, TLS, secrets management. • IaC (Terraform, Pulumi, or similar) and GitOps patterns (Flux, ArgoCD). • Familiarity with the CNCF ecosystem, including the distinction between foundation projects and single-vendor projects. • AI/ML awareness. General understanding of how data infrastructure supports model training, versioning, provenance, and AI operations. • Familiarity with Air Force or Space Force systems of record (e.g., MILPDS, ARMS) and how data flows between them. • You do not need to meet every preferred qualification. You do need to be credible across the core data engineering requirements.
Responsibilities
• Deploy and harden UDS Data Capability in the mission hero's environment — stand up the UDS Store (Iceberg, Rook/Ceph, pgvector, Postgres), wire up UDS Transit for air-gap data movement, configure UDS Govern policies (Pepr/Lula), and integrate UDS Connect (Strimzi/Kafka) where streaming or legacy connectors are required. • Deploy and harden UDS Data Capability in the mission hero's environment • Own the integration with what they already have — connect UDS Data Capability to whatever's already running: Big Bang, legacy Oracle and SQL Server, flat-file drops, SOAP/REST endpoints, message buses, existing object storage, identity providers (Keycloak, mission-side SSO). • Own the integration with what they already have • Build pipelines that move data through classification boundaries — ingestion, transformation, catalog registration, model/dataset packaging via Zarf, cross-domain transit, eventual consistency across DDIL conditions. • Build pipelines that move data through classification boundaries • Operate what you deploy — initial day-2 ownership: capacity, performance, backup/restore (Velero), observability (Vector/Loki), incident response, upgrade paths. Hand off to the mission hero's ops team once it's stable. • Operate what you deploy • Generate accreditation artifacts — STIG evidence, cATO documentation, FIPS validation notes, policy mappings. You produce the evidence the mission hero's ISSM/ISSO needs to actually run this in IL4/IL5. • Generate accreditation artifacts • Be the voice of the mission hero back to product and engineering — file the issues, write the postmortems, propose the operator improvements, push the platform team on what's actually breaking in the field. Your field experience is the highest-signal input we have. • Be the voice of the mission hero back to product and engineering • Train and transfer — leave the mission hero's team self-sufficient: runbooks, architecture docs, working sessions, knowledge transfer. • Train and transfer • Grow junior Data Engineer FDEs — pair on hard problems, review integration designs before they reach the customer, and help junior engineers build judgment faster than they would alone. You're not managing anyone; you're making the team better. • Grow junior Data Engineer FDEs • Travel Expectations: 10%-25% • Travel Expectations: • Preferred Experience and Qualifications: • Data engineering breadth • Lakehouse & storage — production experience with Apache Iceberg (or Delta/Hudi), object storage (Ceph/S3-compatible), Postgres (including extensions like pgvector), and at least one columnar/OLAP engine (Trino, DuckDB, ClickHouse, Spark SQL). • Lakehouse & storage • Streaming & integration — Kafka (preferably Strimzi on Kubernetes), Flink or equivalent stream processing, CDC patterns (Debezium), and the ability to bridge legacy systems (Oracle, SQL Server, flat files, SOAP) into modern pipelines. • Streaming & integration • Pipelines & orchestration — Airflow, Dagster, Argo Workflows, or similar; comfort building, scheduling, monitoring, and recovering production data pipelines. • Pipelines & orchestration • Governance, catalog & access — REST catalogs (Iceberg REST, Polaris/Gravitino/Nessie family), ABAC/RBAC patterns, OIDC/OAuth, lineage and audit. • Governance, catalog & access • Data modeling & SQL — fluent in SQL; comfortable designing schemas for both analytical and operational workloads. • Data modeling & SQL • You do not need to be the deepest expert in any single one of these. You do need to be credible across all of them and know which one to reach for. • Platform & infrastructure • Kubernetes in production — deployments, operators, CRDs, storage classes, networking. You don't have to write controllers from scratch, but you can read and debug them. • Linux fundamentals, container runtime behavior, networking, TLS, secrets management. • IaC (Terraform, Pulumi, or similar) and GitOps patterns (Flux, ArgoCD). • Familiarity with the CNCF ecosystem — what's a foundation project, what's a single-vendor project, why it matters. • What "forward-deployed" requires • Active DoD security clearance required — TS/SCI preferred; minimum active Secret with the ability to obtain TS/SCI. • Comfort being the technical face of Defense Unicorns to a mission hero — listening before prescribing, writing clearly, briefing technical and non-technical stakeholders, and saying "I don't know yet, let me find out" without losing the room. • Comfort with on-site work when the engagement calls for it — in SCIFs and other restricted spaces, sometimes for days or weeks — and equal comfort doing the rest of the work remotely without losing the relationship. • Bias toward delivery. You'd rather ship a working integration with rough edges than a perfect design that hasn't met a real workload. • Self-direction. The mission hero's environment will surprise you, and the answer often isn't in the documentation yet — you'll write it. • The maturity to mentor without hovering. Junior engineers grow by doing
Benefits
• Health: • Medical/Dental/Vision • Premiums are 100% Company Paid • Health Savings Account • Disability Insurance • Financial: • 401k Retirement Plan • Company Stock Options • Leave: • We offer all full-time Unicorns Flexible Time Off (FTO) plus all Federal Holidays, one week for Thanksgiving, and two weeks for Christmas and New Year’s • Paid Parental Leave • Learning: • Reimbursement for approved trainings/subscriptions • Conferences (travel, lodging, and fees) • Don’t have all the preferred experience or qualifications? Studies show that underrepresented groups like women and people of color are less likely to apply to jobs if they don't meet every requirement listed.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT