Prime Intellect - Research Engineer
Requirements
• Strong software engineering skills in Python and experience building data pipelines, evaluation systems, or ML workflows used by others. • Hands-on experience working with language models or agents, with an understanding of how training data and evaluation choices affect model behavior. • Ability to inspect complex data and model outputs, identify failure modes, and turn an ambiguous quality question into a testable measurement. • A record of owning technical projects through experimentation, implementation, and validation, while collaborating closely with researchers and engineers. • Hands-on experience building technical evaluations or agent environments, with the engineering depth to inspect model traces and debug harness and scoring code. • Large-scale synthetic data generation, dataset curation, or distributed processing for model training. • Evaluation methodology, executable coding tasks, agent environments, verifiers, or reward design. • Experiment analysis, statistical testing, contamination detection, and reproducible research tooling. • Experience with RL or post-training data, including task difficulty selection and analysis of rollout or reward distributions. • Contributions to Verifiers, Prime-RL, or comparable environment and evaluation frameworks. • Work on model-based grading, long-horizon coding agents, or diagnosing reward exploitation. • We welcome depth in data generation, environments and rewards, or evaluation systems. You don’t need to have worked across every area.
Responsibilities
• Design and build challenging evaluations for coding and long-horizon agents. • Build executable environments in Verifiers, with realistic task setup and clear success criteria. • Develop and test scoring logic and verifiers, investigating reward hacks and misleading measures of progress. • Trace failures through model outputs and harness code to distinguish model limitations from problems in the environment or scoring. • Build and scale RL data pipelines that generate and curate challenging coding and agentic tasks. • Work with RL researchers to test whether changes to tasks and training data improve performance on held-out tasks.
Benefits
• Flexible work arrangements, with the option to work remotely or in-person at our offices in San Francisco. • Visa sponsorship and relocation assistance for international candidates. • Quarterly team off-sites, hackathons, conferences and learning opportunities. • Opportunity to work with a talented, hard-working and mission-driven team, united by a shared passion for leveraging technology to accelerate science and AI.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT