Torc Robotics - Senior, ML Engineer - VLM
Requirements
• Considered highly skilled and proficient in discipline; conducts complex, important work under minimal supervision and with wide latitude for independent judgment. • Considered highly skilled and proficient in discipline • Scope of Influence: Expected to drive alignment across team interfaces to the rest of the organization. Designs, maintains, and owns team technical solutions and drives consensus. Mentors and guides engineers within the group. • Scope of Influence: • Bachelor’s Degree in Computer Science, Robotics, Electrical Engineering, or related technical field plus competences typically acquired through 6+ years of experience; OR Master’s Degree in a related technical field plus competences typically acquired through 3+ years of experience. • Required Qualifications (some combination of the following skills): • (some combination of the following skills): • Computer Vision & Deep Learning — model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation. • Computer Vision & Deep Learning • Multimodal / VLM experience — hands-on work with vision-language models, open-vocabulary or zero-shot recognition, dense captioning, or semantic embeddings / search applied to perception data. • Model Data Curation — building targeted datasets that measurably improve downstream model performance; large-scale Parquet data processing (Databricks, Daft, Pandas, etc.). • Model Data Curation • Distributed ML & data frameworks — PyTorch, Lightning, Ray, Spark, or equivalent for training and large-scale data processing. • Distributed ML & data frameworks • Scaled MLOps & Tooling — experiment tracking, model registry, MLflow / Weights & Biases, and ML metrics, evaluation, and quality. • Scaled MLOps & Tooling • Development Tools & Eco-System (at scale) — strong Python software development, VDI and cloud-based development environments, CI systems (GitHub Actions), and Docker. • Development Tools & Eco-System (at scale) • End-to-end / VLA driving — familiarity with VLM/VLA or end-to-end driving models, trajectory and action grounding, or chain-of-causation / reasoning-trace datasets. • End-to-end / VLA driving • Auto-labeling foundation models — experience with segmentation, open-vocabulary detectors, or VLM/LLM-driven data engines for annotation and verification. • Auto-labeling foundation models • High-throughput model serving — vLLM, SGLang, or similar for batch auto-labeling and inference at scale. • High-throughput model serving • Semantic inference & retrieval — attribute mapping, semantic search, and vector databases (e.g., LanceDB) for automotive data. • Semantic inference & retrieval • AV data standards & tooling — scenario-description standards such as Pegasus layers; parsing robotics formats (ROS bags, MCAP) and optimizing columnar storage (Parquet, Arrow). • AV data standards & tooling • Cloud development & orchestration — Terraform and AWS managed services (S3, ECS, Lambda, DynamoDB, Step Functions, Athena); AWS HyperPod / Anyscale; inference orchestration. • Cloud development & orchestration • Data visualization — Foxglove, FiftyOne (51), three.js, OpenGL, or similar for dataset inspection and accessibility. • Data visualization • Evaluation & research — closed-loop / open-loop evaluation frameworks (e.g., NavSim-style planning metrics); publications in top-tier CV/AI/Robotics venues (CVPR/ECCV/ICCV, NeurIPS/ICLR/ICML, CoRL). • Evaluation & research
Responsibilities
• Own the offline dataset pipeline — design, implement, test, and deploy Cloud-based pipelines that convert logged multi-sensor data into VLM/VLA training datasets, spanning geometric labels (3D/2D detection, tracking, segmentation, depth) through semantic, scenario-level, and action/trajectory-grounded annotations. • Own the offline dataset pipeline • Build VLM-assisted auto-labeling — develop open-vocabulary detection, dense captioning, semantic enrichment, and scene/scenario description generation that move beyond closed-set bounding boxes, using foundation models to scale annotation and cut manual labeling cost. • Build VLM-assisted auto-labeling • Generate reasoning-grounded labels — produce language-grounded reasoning and chain-of-causation style annotations, temporally aligned to ego-motion and trajectories, to support VLA training and explainable driving behavior. • Generate reasoning-grounded labels • Mine and curate the long tail — surface rare, difficult, and high-uncertainty scenarios, and build curated datasets that measurably improve downstream VLM/VLA model metrics rather than simply adding volume. • Mine and curate the long tail • Close the data flywheel — define dataset schemas, quality metrics, and validation; track auto-labeling quality against model requirements; route model failures back into re-labeling and retraining loops. • Close the data flywheel • Partner with the end-to-end model team — co-define dataset specifications with VLM/VLA model developers, own the quality bar and delivery cadence, and operationalize a continuous dataset delivery loop into their training pipelines. • Partner with the end-to-end model team • Scale on cloud infrastructure — build distributed, reproducible pipelines using columnar data formats and distributed compute, with disciplined software practices, version control, and documentation. • Scale on cloud infrastructure • Lead and mentor — serve as project lead, guide less-experienced engineers, run design reviews, set coding and annotation standards, and drive alignment across team interfaces to the rest of the organization. • Lead and mentor • Stay current — track the latest advances in multimodal models, auto-labeling, and end-to-end autonomous driving, and translate relevant research into production data systems. • Stay current
Benefits
• Torc cares about our team members and we strive to provide benefits and resources to support their health, work/life balance, and future. Our culture is collaborative, energetic, and team focused. Torc offers: • A competitive compensation package that includes a bonus component and stock options • 100% paid medical, dental, and vision premiums for full-time employees • 401K plan with a 6% employer match • Flexibility in schedule and generous paid vacation (available immediately after start date) • Company-wide holiday office closures • AD+D and Life Insurance
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT