Member of Technical Staff, Agents

Required

• At least 3 years of data science or machine learning experience building evaluation systems, metrics, or quality measurement for products.
• Production-quality Python and SQL skills, including experience analyzing large datasets and building automated pipelines.
• Experience designing evaluation frameworks, datasets, measurable criteria, and appropriate metrics.
• A track record delivering production AI, machine learning, agent, or agent-evaluation systems for real users.
• Experience improving agent reliability, owning systems end to end, and measuring results in production.
• Familiarity with statistics, experiment design, and realistic end-to-end simulations; experience with fine-tuning, post-training, or reinforcement learning is valuable.
• Experience with Python, Django, React, and TypeScript; familiarity with evaluation tools such as Braintrust or Raindrop is useful.
• Demonstrated progression and sustained experience in a role, with the ability to work through ambiguity and ship quickly.