Application Software Engineer

Macrohard - Memphis, TNfull time

Remote

entry

$135k–$160k

H1B: no

SpaceX Application Software Engineer at SpaceX responsible for designing and optimizing large-scale AI inference systems. Focus on building reliable, high-throughput model serving platforms with end-to-end ownership from distributed infrastructure to low-level GPU optimizations.

Responsibilities

Develop highly reliable, high-throughput inference systems that serve the best AI models internally across SpaceX
Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, global KV cache, and continuous batching
Optimize latency and throughput of model inference under real production workloads, including low-level GPU kernel work, quantization, speculative decoding, and other acceleration techniques
Build reliable, high-concurrency serving systems with 100% uptime, low tail latency, and excellent observability
Own end-to-end components such as request routing, SDK development, rate limiting, and efficient scaling for internal SpaceX AI inference platforms
Benchmark, fine-tune, and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM)
Develop custom tools for tracing, replaying, and resolving issues across the full stack — from orchestration down to GPU kernels
Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates
Collaborate across SpaceXAI teams to integrate inference capabilities into broader systems and workflows

Qualification

Bachelor's degree in computer scienceExperience in designing1+ years of experience with Rust or C++Experience with large-scaleExperience with DockerExpert knowledge of gRPC (unary

Required

Bachelor's degree in computer science, engineering, math, or scientific discipline; OR 2+ years of professional experience building software in lieu of a degree
Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems
1+ years of experience in full stack development or backend development with production systems
1+ years of experience with Rust or C++

Preferred

Experience with LLM inference engines and serving frameworks (e.g., SGLang, vLLM, Triton, TensorRT-LLM)
Deep low-level systems programming and optimizations: GPU kernels, code generation, batching, caching, parallelism, quantization, and speculative decoding
Experience with large-scale, high-concurrency production serving systems
Knowledge of service observability and reliability best practices
Experience operating commonly used databases such as PostgreSQL, ClickHouse, or MongoDB
Experience designing or building with agent SDKs and agent orchestration frameworks
Experience with Docker, Kubernetes, and containerized applications
Expert knowledge of gRPC (unary, response streaming, bi-directional streaming, REST mapping)