Paloma Health - AI Engineer
Requirements
• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, or a related field, or equivalent practical experience. • A track record of building AI products in a systematic, data-driven way with at least two years of industry experience or equivalent • Hands-on experience building and evaluating LLM-powered products that reached real users – you have owned features through the full loop of build, evaluate and improve, not just experimented with prompts • Excellent analytical and problem-solving skills. • Proficiency in Python, with a strong grasp of clean, typed, tested code and good software engineering practices. • Understanding of prompt engineering principles and techniques • Experience with systematic LLM evaluation techniques – for example, golden/regression datasets, LLM-as-judge pipelines, synthetic data for evaluation, structured error analysis, human-in-the-loop review with domain experts (indicative, not exhaustive) • Strong communication skills, including the ability to translate complex technical topics into understandable language for colleagues across product and clinical domains. • Familiarity with Vertex AI/Gemini Enterprise and the wider Google Cloud Platform (e.g. Cloud Run, Pub/Sub, Cloud Storage). • Experience with an LLM observability or evaluation platform such as Langfuse, LangSmith, Braintrust or Arize Phoenix. • Understanding of MCP and integration of tools with agentic systems. • Experience grounding LLM output in data via techniques such as RAG • Experience with an LLM orchestration or agent framework such as LangGraph or Agent Development Kit. • Experience working with health data and adhering to regulatory requirements.
Responsibilities
• Cutting-edge agentic AI product development and evaluation in partnership with our expert clinicians, product managers, and software developers, always focusing on how we improve staff efficiency and the family and child experience. • Building and running the evaluation pipelines for our AI products: creating gold-standard datasets, testing every prompt and pipeline change against them, and using the results to decide what ships • Developing our internal AI ‘guild’, a collaborative team of AI engineers and internal AI experts to ensure Paloma is an AI-native organisation • As the AI team scales, you will be positioned to grow with it, into a Senior AI Engineer role and beyond. • Does this sound like you? • An innovative problem-solver with a deep belief in the potential of LLMs • You work at pace and are comfortable with the iterative nature of AI development. • You are a clear communicator and a collaborative team player who, above all, cares about the mission to make children's assessments better and accessible to more patients. • You take the responsibility of building for a clinical setting seriously and would rather get something right than ship it fast. • You have high agency, spotting what needs to be done and driving it through without waiting to be asked • You want to be part of an always-learning environment, and you will help us go further with your positive attitude. • Design, develop, and maintain AI-driven applications and agents, and integrate them into existing clinical applications • Design and run the evaluation pipelines that gate AI outputs for quality and safety, building eval sets in partnership with clinicians, product and operations, versioning every prompt and pipeline change and measuring the effect of each change before it ships • Champion evaluation-driven development across the wider engineering team, helping colleagues build the same habits • Document code and technical specifications in line with healthcare regulatory requirements.
Benefits
• You will be part of a mission-driven organisation that’s transforming access to autism and ADHD care in England. For all employees, we offer: • A strong and positive culture that prioritises team building, transparency, and career development. This includes protected time and budget for team connection and learning. • Comprehensive compensation, including: • Salary of £100,000 per year, with some flexibility based on experience. • Annual salary reviews, ensuring they are competitive • 5% employer pension contribution • Equity compensation through Share Options in Paloma Health • Optional Private Medical Insurance and Employee Assistance Programme • Professional Registration Fees Allowance for clinicians • A unique annual leave benefits package which includes: • 37 days of annual leave. This is 28 standard days, plus eight bank holidays, plus your birthday off. • Option to buy an additional 5 days per year (giving a maximum of 42 days) • Where required, 2 days per year of special leave, for events that can happen in life, whether personal or family-related • Additional benefits include: • A premium performance company laptop • NHS discounts, including the Blue Light Card (as we are an NHS provider). • Equality & Diversity
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT