sparta-commodities - Staff Data Engineer
Requirements
• 7+ years of experience as a data or software engineer, with a strong track record of delivering production-quality data systems. • 2+ years working within a product-focused organisation, collaborating closely with cross-functional teams. • Proven experience building and scaling data-intensive pipelines and platforms in production environments. • Deep understanding of stream and batch processing frameworks - hands-on experience with Flink or Spark is essential. • Comfortable working across multiple programming languages - we primarily work with Kotlin, Python, and TypeScript. • Equally comfortable designing high-level architecture as diving deep into low-level implementation details. • Experienced in designing and deploying data infrastructure in cloud environments such as AWS or GCP. • Hands-on experience with core data infrastructure tooling such as Kafka, Redis, and/or clustered Postgres. • Familiarity with open lakehouse technologies, such as Apache Iceberg and dbt. • Experience building data infrastructure to support AI agents - including low-latency retrieval and analytical interfaces. • Experience leading teams or managing engineers, driving both delivery and technical excellence. • Exposure to complex distributed environments with demanding constraints such as high throughput, low latency, or large-scale datasets.
Responsibilities
• Play a leading role in the design, build, and ongoing evolution of scalable data pipelines and ETL frameworks that power both real-time and analytical data processing. • Take shared ownership of the health and performance of our data platform - optimising for latency, throughput, and reliability at scale. • Collaborate closely with engineering and cross-functional stakeholders to ensure data infrastructure aligns with product goals and trader-facing outcomes - acting as the connective tissue between data and backend engineering. • Drive architectural decisions across our data platform, with a focus on pipeline reliability, data quality, and scalability. • Contribute to the definition of our target architecture - shaping how we ingest, transform, store, and serve data as the platform evolves. • Mentor and support other engineers through design reviews, technical discussions, and hands-on knowledge sharing.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT