Responsibilities

The work is applied, not academic. The optimizations you build land in real customer deployments, each with its own models, traffic patterns, latency targets, and cost constraints. So while performance is the heart of the role, you will also work directly with customer engineering teams to tailor deployments to their needs, take a workload from an early proof of concept to a fully monitored production service, and make sure the gains you engineer actually show up for the people running the workload.
To set expectations clearly, this is a hands-on engineering role built around coding, profiling, and low-level optimization. It also carries a customer-facing side, along with elements of product and technical solutions work, because that is where the performance work gets proven.
What You'll Be Working On:
Bring current inference techniques into production and refine them.
Design and optimize serving architectures, including prefill and decode disaggregation, request routing, and related approaches.
Work down into the serving stack, from frameworks like vLLM and SGLang to the CUDA kernels underneath, profiling and running in-depth analysis to find and fix performance problems.
Adapt and scale optimization methods across many kinds of ML models, with an emphasis on large language models.
Profile and tune deployments against clear targets for latency, throughput, and cost, and keep them dependable under real traffic.
Tailor deployments to each customer's models and constraints, partnering with their engineering teams to move a workload from an early proof of concept through to a live, well-monitored production service.
Build and support the software and product features around the inference stack in a production setting, using one or more general-purpose languages, with Python preferred given how central it is to ML work.
Experiment quickly: take fuzzy goals, shape them into clear specs and focused proofs of concept, run fast experiments to find what works, and ship well-tested results without delay.
Own delivery end to end, from the first experiment through to the optimization running in production, keeping the underlying performance goals, clear specs, and follow-through front of mind, and drafting features and product requirement documents together with other engineering and product teams.

Staff Applied AI Inference Engineer at Crusoe · LumiJobs

Staff Applied AI Inference Engineer

Responsibilities

Qualification

Preferred

Benefits

Company

Crusoe

More roles at Crusoe