Definitive Healthcare, US - Data Scientist, Applied AI
Requirements
• Bachelor’s degree in a quantitative or technical field. • 3+ years of experience in data science, machine learning, applied AI, or a related field. • Hands-on experience developing applications with LLMs and evaluating their outputs. • Strong Python, data analysis, and problem-solving skills. • Understanding of embeddings, semantic search, retrieval, RAG and evaluation metrics. • Experience with an agent framework such as PydanticAI, LangGraph, or a similar tool. • Ability to design useful test cases, interpret evaluation results, and identify limitations. • Familiarity with cloud data or ML platforms, Git, testing, and collaborative software development. • Clear communication with technical and nontechnical colleagues. • Experience with Databricks, evaluation platforms, production deployment, fine-tuning, distributed processing, or healthcare data is beneficial but not required. • Autonomy, decisions, and scope • This is an individual contributor role with no people-management responsibilities. • Works independently on defined analyses and implementation tasks, with guidance on project priorities and broader technical direction. • Makes day-to-day decisions about experiments, test cases, and implementation details within agreed project goals. • Recommends model and retrieval improvements based on results; escalates major architecture changes, sensitive-data concerns, significant cost implications, and production release decisions to senior team members or the appropriate owners. • Influences assigned projects through implementation and evidence, rather than owning the evaluation strategy or technical direction across multiple projects.
Responsibilities
• Develop AI workflows: Build and test LLM, RAG, and tool-using agent workflows for assigned use cases. • Evaluate approaches: Compare LLMs, embedding models, and retrieval methods against quality, reliability, latency, and cost, using evaluation criteria and methods set by senior team members. • Maintain evaluation assets: Create and update representative test cases, benchmark datasets, and regression checks; report results and limitations. • Improve retrieval quality: Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding, based on evaluation results. • Investigate failures: Review incorrect answers, retrieval misses, and workflow failures; implement and measure improvements. • Support production workflows: Use experimentation, tracing, and observability tools to help monitor systems and investigate performance changes. • Collaborate on delivery: Work with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions.
Benefits
• The salary range for this position is $111,000– $167,000 per year, which represents the base pay the company reasonably and in good faith expects to pay for this role. Actual pay within this range will be determined based on factors such as relevant experience, skills, and qualifications. • Depending on the position, employees may also be eligible to participate in a company bonus or commission plan. All employees are eligible for a comprehensive benefits package, including medical, dental, and vision coverage, unlimited paid time off, and participation in the company’s 401(k) plan with employer contribution. • Why we love Definitive, and why you will too! • Industry leading products • Work hard, and have fun doing it • Incredibly fast growth means limitless opportunity • Flexible and dynamic culture • Work alongside some of the most talented and dedicated teammates • Definitive Cares, our community service group, gives all of us a chance to give back • Competitive benefits package including great healthcare benefits and a 401(k) match • What our Employees are saying about us on Glassdoor: • “Great Work atmosphere, great work life balance, excellent company to work for, amazing top notch product, incredible customer service, lots of tools to help you succeed.” • Business Development Manager • “Great team. Amazing growth. Employees are treated very well.” • Research Analyst • “I have waited 36 years to work at a dream job for a dream company and I am so happy to have finally got there.” • Profile Analyst • If you don’t fit all of these qualifications, but believe you’re still a great fit, feel free to apply and tell us why in your cover letter. • If you are a California, Colorado, New York City or Washington resident and this role is a remote role, you can receive additional information about the compensation and benefits for this role, which we will provide upon request. • Definitive Hiring Philosophy
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT