Data Scientist, Applied AI

Framingham, MAfull timeRemote

Apply on Employer Site

mid

$111k–$167k

H1B: no

Definitivehc Data Scientist at Definitive Healthcare focusing on applied AI and generative AI workflows. Develops and evaluates LLM, RAG, and agent-based AI solutions.

Responsibilities

Develop AI workflows: Build and test LLM, RAG, and tool-using agent workflows for assigned use cases.
Evaluate approaches: Compare LLMs, embedding models, and retrieval methods against quality, reliability, latency, and cost, using evaluation criteria and methods set by senior team members.
Maintain evaluation assets: Create and update representative test cases, benchmark datasets, and regression checks; report results and limitations.
Improve retrieval quality: Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding, based on evaluation results.
Investigate failures: Review incorrect answers, retrieval misses, and workflow failures; implement and measure improvements.
Support production workflows: Use experimentation, tracing, and observability tools to help monitor systems and investigate performance changes.
Collaborate on delivery: Work with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions.

Qualification

Education and experience3+ years of experience in data scienceTechnical and professional skillsStrong PythonUnderstanding of embeddingsAbility to design useful test casesExperience with Databricks

Required

Education and experience
Bachelor’s degree in a quantitative or technical field.
3+ years of experience in data science, machine learning, applied AI, or a related field.
Hands-on experience developing applications with LLMs and evaluating their outputs.
Technical and professional skills
Strong Python, data analysis, and problem-solving skills.
Understanding of embeddings, semantic search, retrieval, RAG and evaluation metrics.
Experience with an agent framework such as PydanticAI, LangGraph, or a similar tool.
Ability to design useful test cases, interpret evaluation results, and identify limitations.
Familiarity with cloud data or ML platforms, Git, testing, and collaborative software development.
Clear communication with technical and nontechnical colleagues.
Experience with Databricks, evaluation platforms, production deployment, fine-tuning, distributed processing, or healthcare data is beneficial but not required.

Benefits

Dental
Vision
401(k)
PTO

Company

Definitivehc

View company page

Source: greenhouse

Active within the past hour

Remote-friendly

More roles at Definitivehc

Senior Data Scientist, Applied AI

Framingham, MA

Data Engineer

Framingham, MA

Solutions Consultant Digital Audiences

Framingham, MA

Platform Partnerships Director

Framingham, MA

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

PythonLLMRAGPydanticAILangGraphembeddings

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site

Data Scientist, Applied AI at Definitivehc · LumiJobs