Engineering Manager, Inference Router

Preferred

• Experience with large-scale LLM/inference serving and the systems around it (e.g. vLLM, TensorRT-LLM, SGLang, or comparable).
• Familiarity with prefill/decode (P/D) disaggregation, KV-cache management, and prefix/cache-aware routing.
• ML background: understanding of inference workloads, batching, and the performance characteristics of serving LLMs and multimodal models.
• Experience with autoscaling and capacity management across heterogeneous compute.
• Experience building simulation, modeling, or load-testing frameworks for distributed systems to validate behavior cost-effectively at scale.
• Strong grounding in observability (metrics, tracing, SLOs) and incident response for high-traffic production systems.
• Why This Role Matters
• The Inference Router is the control plane behind Cerebras’ core promise: the fastest AI inference in the world. The routing decisions your team makes — across data centers, across heterogeneous compute, with cache-awareness and high availability — directly determine the latency, throughput, and reliability every customer experiences. Your work keeps real-time AI applications fast and dependable, and unlocks new capabilities across enterprise and frontier AI use cases.