Reality Defender - Data Engineer

• Hands-on experience with Kubernetes and AWS, including deploying, scaling, and troubleshooting containerized workloads in production environments. • Proficiency with high-performance/distributed computing frameworks such as Spark and Ray for processing large-scale datasets. • Experience with workflow orchestration tools such as Airflow (or comparable systems like Dagster, Prefect, or Luigi) to schedule and manage complex data pipelines. • Strong programming skills in Python and SQL; experience with Golang is a plus. • Demonstrated track record building and operating large-scale data processing pipelines, ideally handling multi-terabyte or streaming datasets. • Experience working with audio or video data at scale is a strong plus (e.g., transcoding, feature extraction, or preprocessing pipelines). • Familiarity with common data transformation patterns applied to large datasets (ETL/ELT, batch and stream processing, data validation and quality checks). • Experience designing and maintaining job orchestration systems, including dependency management, retries, monitoring, and alerting for production pipelines. • Bonus: experience orchestrating machine learning workflows (training pipelines, model retraining triggers, feature stores, or MLOps tooling).