Senior Software Engineer, Agents

New York - Hybrid · Remotefull time

Apply on Employer Site

Remote

senior

$160k–$210k

H1B: no

Ellipsis Health The Senior Software Engineer, Agents at Ellipsis Health builds AI agents that interact with patients, owning conversation design, prompts, tool calling, and backend services. The role includes building evaluation systems and improving agent performance using live conversation data.

Responsibilities

Design, build, and ship AI agents end to end, including conversation flows, prompts, tool and function calling, retrieval, memory, and the backend services agents depend on.
Build evaluation systems (offline test suites, simulated conversations, human review loops, and production monitoring) so we know an agent is good before and after it ships.
Improve instruction following, tool-calling reliability, latency, and cost across our LLM and speech stack.
Help design and build our next-generation voice agent platform, choosing the right models and architecture for each task based on quality, latency, and cost.
Partner with clinical, product, and customer teams to turn workflows like scheduling, refills, billing, and post-discharge check-ins into reliable agent behavior.
Dig into real conversations and failure cases, and close the loop with prompt, tool, model, or product changes.
Build internal tools to observe, monitor, and analyze agent behavior at scale to identify common failure modes and patterns.
Track what's new in models and tooling, and bring the useful parts into production quickly.

Qualification

Hands-on experience with promptingProduct senseVoice AI or real-time speech experienceSalary and Benefits

Required

5+ years of professional software engineering experience, including 1+ year building and shipping LLM-powered applications in production.
Strong proficiency in Python and/or TypeScript, comfortable across backend APIs and data, with enough frontend to ship a complete feature.
Hands-on experience with prompting, tool use and function calling, RAG, structured outputs, and agent orchestration.
Experience designing evals and using data to decide what to fix next.
Product sense: you care whether the patient got their appointment booked, not just whether the model metric moved.
Clear written and verbal communication with technical and non-technical teammates.

Preferred

Voice AI or real-time speech experience, including STT, TTS, voice activity detection and turn-taking, and telephony (Twilio, SIP, WebRTC).
Fine-tuning or post-training experience with open or hosted models.
Experience in healthcare or another regulated environment, including HIPAA, EHR, and FHIR.
Experience with workflow orchestration (e.g., Temporal).
Familiarity with LLM observability and tracing tools.
Salary and Benefits

Benefits

Dental
Vision
401(k)
Equity/RSU
Tuition

Company

Ellipsis Health

View company page

Source: ashby

Active within the past hour

Remote-friendly

More roles at Ellipsis Health

Senior Full Stack Engineer

New York - Hybrid · Remote

Senior Product Manager

New York - Hybrid · Remote

Business Operations Associate

New York - Hybrid · Remote

Senior ML Engineer, Agentic AI

San Francisco - Hybrid · Remote