i.AI - Senior AI Evaluation Engineer (Data Scientist)
Requirements
• Autofill my application
Responsibilities
• Lead the design and development of evaluation pipelines and experiments across AI products • Architect product-wide evaluation suites that integrate with engineering and CI/CD workflows • Design and implement complex and exploratory evaluation methods to assess accuracy, bias, safety and overall model performance • Work closely with product, engineering and domain experts to shape evaluation approaches throughout the product lifecycle • Design and run evaluation experiments that directly inform product development and technical decisions • Analyse complex evaluation results and turn findings into clear recommendations for technical, product and senior stakeholders • Lead the communication of evaluation findings through reports, presentations and external publications • Support and review evaluation work from other engineers, helping to raise standards and develop evaluation capability across the team • You may be a strong fit if you bring some of the following: • Broad experience of evaluating generative and predictive AI systems • Extensive knowledge of experimental design and statistical hypothesis testing • Highly proficient in Python • Demonstrable experience of software development tools and practices (e.g. git, CI/CD, unit testing) • Ability to communicate technical work to a broad audience, using different media/formats e.g. technical reports, presentations, and web-based tools. • Experience implementing LLM-as-a-judge and other generative AI evaluation techniques using AI evaluation tools (e.g. pydantic evals, deepeval, inspect) • Experience using pipeline orchestration tools e.g. DVC, snakemake • Knowledge of standard quantitative research designs, methods and analysis (particularly those set out in the Magenta Book) • Experience of mentoring junior colleagues
Benefits
• Career-defining projects with outsized impact • Backing from the Prime Minister and No10 to scope and build transformative AI projects. • Unique opportunities to apply technology that could transform the public sector and impact citizens’ lives. • Talented, supportive and mission-driven colleagues. • Access to frontier models and ample compute. • Extensive operational, product, strategy, design and delivery support so you can focus on shipping. • Work with experts across national security, policy, AI research and adjacent sciences. • Growth & empowerment • A team culture and development support that prioritises personal growth. • Opportunities to own important products early and develop them in small empowered teams. • 5 days off learning and development, annual stipends for learning and development and funding for conferences and external collaborations. • Life & family* • Opportunity to work from London, Manchester or Bristol offices. • Hybrid working, flexibility for occasional remote work abroad and stipends for work-from-home equipment. • Generous annual leave - 25 days plus one additional day for each year of service • Generous paid parental leave (up to 39 weeks full pay + option for additional unpaid time). • On top of your salary, we contribute 28.97% of your base salary to your pension. • Discounts and benefits for cycling to work, dental insurance, donations and retail/gyms. • These benefits apply to direct employees. Benefits may differ for people joining through other employment arrangements such as secondments. • Salary is paid within the range shown below. As this is a GDaD role, the maximum salary includes a non-pensionable technical allowance, and successful candidates will be appointed somewhere within that range depending on assessment of skills demonstrated during the interview process. Base salary for candidates joining from outside the civil service be the minimum of the pay band. • National: £58,275 - £81,885 (maximum includes non-pensionable technical allowance of up to £15,600)London: £63,165 - £86,640 (maximum includes non-pensionable technical allowance of up to £15,600) • Selection Process • Selection Process • Appointment is conditional on successfully completing UK Government SC clearance. Prior clearance is not required—we will sponsor and support you. You should normally have been resident in the UK for 2 of the past 5 years. Employment is conditional on obtaining and maintaining the required clearance(s). More detail on clearance eligibility can be found on the UK Government website: National security vetting: clearance levels - GOV.UK • Appointment is conditional on successfully completing UK Government SC clearance. Prior clearance is not required—we will sponsor and support you. • In accordance with the Civil Service Commission rules, the following list contains all elements of the selection process. • Candidates should expect to go through some or all of the following stages once an application has been submitted: • Preliminary call - a short initial conversation with members of the team you’d be joining • Technical interview - a live online technical assessment • Behavioural interview - focused on your previous experience and how you approach different workplace situations • Senior leadership conversation - a final discussion with members of the senior leadership team • Diversity and Inclusion • The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window). As part of the application process, we monitor statistics on D&I. You can see how we process this data here: Recruitment privacy notice - GOV.UK. • £58,275 - £86,640 GBP
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT