whitecircle - Multimodal ML Engineer

• Train and fine-tune large-scale multimodal models (vision-language, audio, speech) from scratch and from pretrained checkpoints • Extend models across modalities: image understanding, video temporal modeling, long-context processing, and streaming audio • Design and run experiments: architecture changes, data mixes, training recipes • Build and maintain multimodal data pipelines — from raw images, video, and audio recordings to training-ready datasets, including synthetic data generation • Train and optimize MoE architectures for efficient multimodal inference • Build alignment pipelines: SFT, DPO, GRPO, reward modeling — across modalities, not just text • Optimize models for production: quantization, distillation, batching, streaming and low-latency serving • Deploy models end-to-end: from research checkpoint to production serving • Define evaluation metrics and benchmarks that actually matter for the product: visual QA, spatial reasoning, video comprehension, speech and audio understanding • You’ll fit right in if you • 3+ years training large-scale deep learning models in multimodal domains (vision-language, audio, speech, or acoustic) • Strong PyTorch skills with hands-on distributed training experience (DeepSpeed, FSDP, or similar) • Deep experience with multimodal architectures — you understand how vision/audio encoders, projectors, and LLMs fit together (LLaVA, Qwen-VL, InternVL, Audio Flamingo, Omni Qwen, Audio Qwen, Whisper, HuBERT, Conformer, or similar) • Hands-on with RLHF/alignment for multimodal: GRPO, DPO, reward modeling — not just for text • Experience with video and/or audio sequence modeling: temporal modeling, long-context processing, efficient attention, streaming inference • Track record of shipping models to production: you've hit latency targets and optimized inference, not just reported benchmark scores • Comfortable with large-scale multimodal dataset curation: image-text pairs, video-instruction data, audio preprocessing, augmentation, synthetic data generation • Familiar with MoE architectures and their tradeoffs for multimodal workloads • Strong engineering fundamentals: clean code, version control, testing, documentation • Understanding of audio signal processing fundamentals (spectrograms, mel features, noise reduction) is a plus