Momentic Founding Engineer (AI) at Momentic to build and own AI-native automated testing platform features. Focus on integrating and tuning LLMs for real-world applications and improving testing quality.
Responsibilities
Build and take ownership of features in AI-native automated testing platform
Integrate LLMs into real-world applications using modern tech stack
Perform LLM performance tuning including prompt engineering and context management
Conduct LLM evaluations and observability
Work with supervised fine-tuning and reinforcement learning fine tuning platforms
Engineer evaluation datasets and prompt engineering
Manage LLM as judge verifiers and RL environments
Qualification
You thrive when you have autonomyMust-have qualificationsExperience with LLM performance tuning3+ years of experienceReact
Required
You dislike meetings and would much rather focus your time on building, being productive, and shipping code
You thrive when you have autonomy, own as many of the details as possible, and project manage your own work
You're in SF or you're willing to relocate, you love working in-person, and you're serious about joining us to build a culture we'll all love
Must-have qualifications
Experience with LLM performance tuning, including prompt engineering and context management strategies
Experience with LLM evaluations and observability
Experience integrating LLMs into real-world applications using a modern tech stack (Python, TypeScript, etc.)
Familiarity and experience with supervised fine-tuning (SFT) and/or reinforcement learning fine tuning (RLFT) on platforms such as fireworks, Together AI, or Baseten
Familiarity with evaluation dataset engineering, prompt engineering, LLM as a judge verifiers, and RL environments
3+ years of experience
Preferred
Experience with supervised or reinforced fine-tuning
Experience with classical machine learning techniques such as template matching, bounding box detection, and OCR
Experience running statistical experiments
React, TypeScript, Next.js, Node.js, PostgreSQL, Google Cloud, Kubernetes
