Gather AI - Annotation Program Manager
Requirements
• Annotation vendor management, including onboarding, batch management, SLA tracking, and calibration sessions • Python scripting for data preparation, automation, reporting, validation, and API integrations • Writing annotation SOPs, taxonomies, and detailed labeling guidelines • Annotation quality assurance using inter-annotator agreement (IAA), gold-set validation, and structured quality programs • Cloud storage and data handling using Azure Blob Storage or AWS S3, along with JSON and CSV data formats • Strong Jira project management and task organization across multiple concurrent annotation programs • Computer vision annotation experience including video event labeling, bounding boxes, polygon segmentation, or similar workflows • Experience with annotation platforms such as Labelbox, Scale AI, CVAT, Hasty, or V7 Darwin • Familiarity with active learning or ML-assisted annotation prioritization • 3–6 years of experience in data operations, annotation program management, ML data engineering, or a closely related role • Experience managing an external annotation vendor operating under production SLAs • Strong Python programming skills for data preparation and API integrations • Experience authoring annotation guidelines or standard operating procedures • Understanding of data privacy and confidentiality requirements when working with customer operational data • Preferred • Preferred • Experience with CloudFactory • Experience in warehouse automation, robotics, or computer vision applications • Experience with additional annotation platforms and APIs • REST APIs (CloudFactory API or equivalent annotation platform APIs) • Azure Blob Storage or AWS S3 • Success Traits • Success Traits • Highly organized and able to manage multiple concurrent workstreams • Collaborative, respectful, and an excellent cross-functional partner • Detail-oriented with a strong focus on annotation quality • Resourceful and comfortable building processes where they don't yet exist • Persistent in driving vendor accountability and meeting SLAs • Comfortable managing external teams and influencing without direct authority • Analytical, with the ability to identify root causes of annotation quality issues and implement lasting improvements
Responsibilities
• Own day-to-day management of our annotation vendor relationship, including workload planning, throughput monitoring, and SLA management. • Coordinate annotation batches, scheduling, and delivery timelines across multiple production programs. • Monitor annotation quality and continuously improve labeling accuracy through structured QA processes. • Develop, maintain, and improve annotation SOPs, taxonomies, and documentation. • Build and maintain Python-based workflows for annotation data preparation, API integrations, reporting, and validation. • Partner with ML teams to prioritize annotation work that maximizes model performance. • Track work across Jira and communicate project status, risks, and capacity needs to stakeholders. • Drive operational improvements that reduce annotation turnaround time while maintaining high quality. • First 90 Days • First 90 Days • Within your first three months, you will be expected to: • Own Annotation Operations • Take over day-to-day coordination with CloudFactory from existing program owners. • Own weekly turnaround time (TAT) reporting, batch scheduling, and vendor escalation management. • Become the primary operational contact for annotation delivery. • Become Fluent in Existing Annotation Programs • Develop working expertise in both active production annotation programs: • MHEV forklift event labeling (PICK_UP / DROP_OFF) using the internal MHEV annotation application • Drone Human-in-the-Loop (HIL) annotation pipeline supporting case counting and empty detection through the CloudFactory API • Improve Annotation Quality • Audit completed annotation batches across both programs. • Run inter-annotator agreement (IAA) analysis against gold-set datasets. • Identify common annotation errors and present recommendations to ML leads. • Lead calibration sessions with the annotation vendor as needed. • Own Documentation • Own Documentation • Take ownership of: • MHEV Event Annotation SOP • Barcode + Location Root Cause Analysis (RCA) SOP • Version control • Closing documentation gaps • Adding edge-case guidance • Ensuring all annotators are using current documentation
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT