surge-ai - Research Engineer, Coding Evaluation & Training Data

Remote - United States1mo ago

Requirements

• 3–6+ years of professional software engineering experience building and maintaining real systems. • Strong coding ability in at least one mainstream language and comfort working in production codebases. • High “taste” for good engineering: you care about correctness, code quality, and how real engineering teams actually work. • Ability to reason about and debug technical environments, including containers, dependencies, and automated test setups. • Interest in owning projects end-to-end, including scoping, workflow design, execution, and continuous improvement. • Excellent written and verbal communication skills, with the ability to speak credibly with senior client engineers and translate fuzzy goals into actionable plans. • Excitement about AI/ML systems and the role of data, evaluation, and reward design in improving agentic coding capabilities.

Responsibilities

• Own end-to-end coding data projects, from initial scoping and pilot design through execution, iteration, and scale-up. • Design agentic training workflows and task structures that mirror real-world SWE work (e.g., refactoring, debugging, code review, large-repo navigation, tool use). • Define and iterate on rubrics, golden sets, and reward signals that capture true engineering value, not just “does it compile.” • Evaluate data and worker output with strong SWE taste, and make calls about what meets the bar for frontier training. • Design and run qualification processes for coding workers, including hands-on assessments of their coding ability. • Set up or partner on complex technical environments (e.g., containers, repos, test harnesses, sandboxes, code execution infrastructure). • Partner with technical staff at our clients to translate high-level training goals into concrete projects and technical environments. • Collaborate closely with Surge engineering, product, and operations to improve our coding data products, internal tools, and execution processes.