Software Engineer - Training/Inference (C++)

Palo Alto, CAfull timeOn Site

entry

$180k–$440k

H1B: no

xAI Software Engineer, Training/Inference (C++) at SpaceXSI designs and optimizes large-scale model serving systems end-to-end. The role includes ownership of distributed infrastructure and deep low-level optimizations to serve millions of users reliably and quickly.

Responsibilities

Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
Optimize latency and throughput of model inference under real production workloads
Build and operate high-concurrency serving systems with excellent tail latency and high reliability at the scale required to serve millions of users
Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation)
Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels
Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates
Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems

Qualification

Bachelor’s degree in computer scienceExperience with large-scaleLow-level inference optimizationsAlgorithmic inference optimizationsExperience with testing

Required

Bachelor’s degree in computer science, computer engineering, or science discipline
1+ years of experience with low-level system programming experience in C/C++ or Rust

Preferred

Experience with large-scale, high-concurrent production serving
Experience with GPU inference engines (e.g., vLLM, SGLang, Triton, TensorRT-LLM)
Strong background in system optimizations: batching, caching, load balancing, parallelism
Low-level inference optimizations: GPU kernels, code generation
Algorithmic inference optimizations: quantization, speculative decoding, distillation, low-precision numerics
Experience with testing, benchmarking, and reliability of inference services
Experience designing and implementing CI/CD infrastructure for inference

Benefits

Dental
Vision
401(k)
Equity/RSU
Life Insurance

Company

xAI

View company page

Source: greenhouse

Active within the past hour

More roles at xAI

Technical Recruiter

Memphis, TN

Mechanical Commissioning Technician

Memphis, TN

Mechanical Commissioning Lead

Memphis, TN

Manager, Materials Management - Memphis

Memphis, TN

Apply on Employer Site