Reality Defender - Staff Data Platform Engineer
Requirements
• 8+ years building data platforms or data infrastructure, including staff- or lead-level ownership of production pipelines that generate, ingest, or deliver data at high volume. • Deep hands-on experience with distributed processing (Spark, Ray, or similar) and lakehouse platforms, ideally Databricks, on top of cloud object storage (S3, GCS, or Azure Blob). • Production experience with Databricks, Unity Catalog, Delta Lake, and Spark on AWS, including performance tuning, access controls, and operational troubleshooting. • Experience working with large unstructured media such as video, audio, or images, including the realities of large files, decoding and transcoding, and metadata at scale. • A track record of building reproducible ML datasets: versioning, lineage, data quality validation, and orchestration (Airflow, Dagster, Databricks Workflows, or similar). • Strong Python and SQL, and the judgment to own ambiguous platform problems end-to-end on a lean team. • Own infrastructure and deployment automation using Terraform and CI/CD, including AWS storage, IAM, compute, Databricks resources, and distributed processing environments. • Experience building synthetic data generation pipelines or running generative models (diffusion, voice cloning/TTS, video generation) at scale. (nice to have) • Background supporting ML research teams with training and evaluation data, especially for multimodal models. (nice to have) • Experience in security, fraud, trust & safety, or another adversarial domain, or at a Series A or B startup where scope regularly outpaces headcount. (nice to have)
Responsibilities
• Own the end-to-end data platform for AI research: continuous ingestion, preprocessing, governance, and export of image, video, audio, and text data at high volume. • Complete and scale our Databricks lakehouse and distributed processing stack (Spark, Ray, or similar), making architecture and build-vs-buy decisions that hold up as data volume grows. • Build reusable infrastructure for in-house synthetic data generation, so researchers can produce new deepfake variants across modalities on demand instead of one-off. • Make every dataset reproducible and traceable through versioning, lineage, metadata, quality checks, and stable train/eval splits, so any model result can be tied back to the exact data behind it. • Own pipeline reliability, observability, and processing cost per dataset as core metrics of the platform. • Partner with ML researchers to understand their data needs and turn them into self-serve tools that speed up model iteration. • Mentor engineers and raise the bar on data engineering practices: code review, testing, documentation, and runbooks. • Work with Security and compliance stakeholders to make sure data handling, licensing, and access controls meet the standards our enterprise and government clients expect. • Help shape the long-term data platform roadmap with engineering and research leadership. • Own a mature, self-serve data platform that researchers use to create, version, and export datasets without waiting on platform support. • Have scaled synthetic data generation across vision, audio, and multimodal research, broadening our detection coverage of new generative techniques. • Deliver reproducible evaluation datasets that let the company compare model versions with confidence. • Be setting the multi-year data platform roadmap and leading high-leverage initiatives beyond the original scope.
Benefits
• $210K – $260K • Offers Equity • Offers Bonus • Reality Defender offers the following benefits to all our employees, regardless of location: • Healthcare plans with 100% premium coverage for employees and partial coverage available for dependents • Dental and Vision plans with 100% premium coverage for employees and their dependents • Short/Long-term disability and life insurance plans with 100% premium coverage for employees • FSA/HSA and 401k programs • 20 days of PTO per year • 12 weeks of Parental Leave • Learning and Development budget • Annual company-sponsored offsite • For employees working from Reality Defender's HQ in NYC, we offer the following benefits: • Daily in-office lunch through UberEats • Happy Hours and other local events • Base salary range: 210,000 - 260,000, based on experience, level, and location. Role includes equity.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT