Senior Data Scientist, Applied AI

Framingham, MAfull timeRemote

Apply on Employer Site

senior

$136k–$206k

H1B: no

Definitivehc Senior Data Scientist, Applied AI at Definitive Healthcare independently builds and evaluates AI solutions for assigned use cases using Python. The role includes applying evaluation frameworks, selecting technical approaches, diagnosing issues, and collaborating with engineering and product teams.

Responsibilities

Own assigned workflows: Independently build, test, and improve LLM, RAG, and tool-using agent workflows for an assigned use case, using maintainable, tested Python code.
Apply the evaluation framework: Select and apply evaluation criteria, baselines, and test datasets within the team's established evaluation approach; does not set that approach for the team.
Select technical approaches: Compare models, embeddings, and retrieval methods for the assigned use case using evidence on quality, reliability, latency, and cost; document recommendations and tradeoffs for that project.
Diagnose and resolve issues: Investigate retrieval misses, incorrect answers, and workflow failures within the assigned project; independently determine root cause and implement and validate the fix.
Maintain and extend test assets: Build and update test cases, benchmark datasets, and regression checks for the assigned use case.
Support production reliability: Monitor and investigate performance changes for workflows they own, and recommend and implement improvements within their project.
Collaborate on delivery: Communicate findings and technical tradeoffs on the assigned project to immediate engineering, product, and domain partners.

Qualification

Education and experienceBachelor's degree requiredTechnical and professional skillsStrong Python programmingStrong knowledge of LLMsAbility to weigh quality

Required

Education and experience
Bachelor's degree required; master's degree in a quantitative or technical field preferred.
5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field.
Hands-on experience building and evaluating LLM applications.
Technical and professional skills
Strong Python programming, software engineering and testing, and analytical problem-solving skills.
Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures.
Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow, for building and deploying scalable, production-grade data and AI solutions.
Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system.
Ability to weigh quality, reliability, latency, and cost when recommending an approach for their own project.
Clear communication and effective collaboration with technical and nontechnical partners.
Experience with agent frameworks and AI evaluation platforms.

Benefits

Dental
Vision
401(k)
PTO

Company

Definitivehc

View company page

Source: greenhouse

Active within the past hour

Remote-friendly

More roles at Definitivehc

Solutions Consultant Digital Audiences

Framingham, MA

Data Engineer

Framingham, MA

Platform Partnerships Director

Framingham, MA

Senior Customer Success Manager

Framingham, MA

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

PythonLLMRAGAI workflowsEvaluation frameworks

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site