Forward Deployed Engineer, Inference

United Statesfull timeRemote

Apply on Employer Site

senior

$240k–$415k

H1B: yes

Cohere The Forward Deployed Engineer at Cohere architects and implements high-performance, resilient machine learning inference systems within customer environments. This role bridges core engineering and strategic partners to optimize model serving, hardware validation, and deployment reliability.

Responsibilities

As a Forward Deployed Engineer, you will:
Embed directly with customer engineering teams to architect the "connective tissue" between our AI models and their live, complex production environments.
Design advanced model serving topologies, implementing multi-tenant architectures, intelligent model routing, and cross-datacenter failover strategies for multi-cluster environments.
Optimize "AI-in-a-box" and private deployments, balancing localized SRE constraints like Kubernetes multi-cluster scale with raw model performance.
Conduct rigorous benchmarking and profiling of customer LLM workloads, root-causing performance degradation (e.g., slow generation) via metrics, dashboards, and distributed logs.
Accelerate hardware validation, collaborating with clients to run and support Cohere models on next-generation accelerators (GPUs, TPUs, AMD, and custom ASICs).
Translate complex field friction into platform upgrades, delivering prescriptive feedback and contributing directly back to core product features.
Ensure Day-1 production reliability, assisting customers in establishing sound deployment patterns, SLOs, and observability frameworks.
You may be a good fit if you have:
8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software in complex environments.
Deep technical depth in production Kubernetes, including orchestration across multi-cluster and hybrid/multi-cloud ecosystems (GCP, Azure, AWS, OCI).
Strong understanding of LLM architecture and inference dynamics, specifically how attention mechanisms, compute-bound prefill, and memory-bound decode phases dictate live performance.