cantina - Member of Technical Staff, Video Data & Model Evaluation
Requirements
• Hands-on experience in data strategy, dataset development, annotation, or model evaluation for generative image, video, or multimodal models. • Experience supporting model development across continued training, supervised fine-tuning, post-training, or evaluation. • Strong understanding of generative media tasks, including image and video generation, multimodal generation, and instruction-based editing. • Experience developing labeling taxonomies, annotation guidelines, captioning standards, quality thresholds, and evaluation frameworks. • Strong visual judgment and the ability to translate subjective quality expectations into measurable data and evaluation standards. • Experience managing large-scale data-production or annotation projects and collaborating with research, engineering, creative, and external partner teams. • Familiarity with automated captioning, synthetic-data generation, vision-language models, prompt design, Python, SQL, or other data and workflow automation tools is a plus.
Responsibilities
• Define quality standards for video generation across aesthetics, realism, composition, motion, temporal consistency, instruction following, subject consistency, editing preservation, and text rendering. • Evaluate model iterations, analyze failure patterns, and translate findings into actionable data, training, and prompting strategies. • Own end-to-end training data projects, including sourcing, construction, annotation, captioning, quality control, and delivery. • Develop labeling systems that support dataset segmentation, coverage analysis, quality filtering, training-data mixtures, and model performance attribution. • Design annotation and captioning standards for video generation, editing, and multi-reference tasks. • Create and evaluate synthetic-data workflows based on their stability, usability, quality, and training value. • Develop automated annotation and filtering workflows using vision-language models, system prompts, and quality-scoring models. • Build structured model evaluation processes covering test-set design, evaluation criteria, failure attribution, and reporting. • Establish feedback loops connecting model evaluation and negative-case analysis with data strategy and model iteration. • Train and calibrate annotation teams and external data partners, conduct quality audits, and improve delivery standards. • Partner with research, engineering, product, and creative teams to execute data initiatives and identify future model capabilities.
Benefits
• Competitive salary and generous company equity • Personal time off and paid holidays • Health insurance • Global travel insurance: Covers you when traveling internationally • Monthly spending stipend: $500 (~S$635) • Equipment: All equipment needed for your home office
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT