Associate Director/Director, Inference Engineering

Alewife, Cambridge, MAfull timeOn Site

Apply on Employer Site

senior

$204k–$348k

H1B: no

Lila Sciences Associate Director/Director, Inference Engineering leads Lila's model serving platform, including inference APIs and deployment systems. This role defines scalable model serving balancing latency, throughput, quality, and cost while connecting research and ML to production.

Responsibilities

Lila is hiring an Associate Director/Director, Inference Engineering to lead the team responsible for Lila's model serving platform: the inference APIs, serving frameworks, request routing, batching strategies, and deployment systems that bring models reliably to production.
What You'll Be Building
Own the inference service end to end: API contracts, latency SLAs, throughput guarantees, rate limits
Drive serving architecture decisions: batching, KV cache, speculative decoding, quantization, multi-node deployment
Establish SLOs for latency, throughput, and availability, plus on-call and postmortem practices
Partner with ML, research, and product engineering to make new models production-ready
Build systems for model onboarding, deployment safety, evaluation gates, and observability
Build, mentor, and retain a high-performing engineering team, and own the serving roadmap

Qualification

Track record hiringAbility to align MLExperience with serving evaluationFamiliarity with GPU capacity planning

Required

Deep understanding of the inference stack for large language models, including serving frameworks (e.g., vLLM, TensorRT-LLM, TGI), batching strategies, memory constraints, and latency/throughput tradeoffs
Experience owning a customer- or product-facing API service end to end, including SLAs, incident response, and service reliability in fast-moving environments
Track record hiring, mentoring, and retaining senior technical talent, and scaling engineering organizations through rapid growth
Strong technical judgment across latency, throughput, reliability, cost, and delivery speed
Demonstrated ability to set technical strategy and execute as priorities change
Ability to align ML, research, product engineering, and executive stakeholders with competing priorities

Preferred

Hands-on experience with speculative decoding, quantization (AWQ, GPTQ, FP8), or model parallelism strategies
Experience with serving evaluation: output quality regression testing, A/B routing, and shadow deployments
Familiarity with GPU capacity planning, instance provisioning, utilization, and cloud cost modeling
Prior work in AI/ML startups or research-heavy organizations

Benefits

Dental
Vision
Equity/RSU
Parental Leave
Life Insurance

Company

Lila Sciences

View company page

Source: greenhouse

Active within the past hour

More roles at Lila Sciences

Co-Op, Robot Perception & State Estimation

Alewife, Cambridge, MA

Co-Op, AI Nucleic Acid Design

San Francisco, CA

Staff Forward Deployed Engineer, Physical Sciences (Level Flexible)

One Charles Park, Cambridge, MA

Staff Forward Deployed Engineer, Life Sciences

One Charles Park, Cambridge, MA

Boost Your Interview Chances

Improve Resume Match Score

Free

Your Score

9.0

Strong Applicant

Must-Have Skills for This Role

vLLMTensorRT-LLMTGIspeculative decodingquantizationmulti-node deployment

Optimize my Resume

Apply Faster with Autofill

Free

Save your profile once. LumiJobs pre-fills the employer's application form so you stop re-typing the same answers on every site.

Apply With Autofill

Or skip LumiJobs's tools and submit directly on the employer's career site.

Apply on Employer Site