Clera Technical Program Manager at Clera to own complex, time-sensitive data and evaluation programs for AI research teams. Responsible for managing program lifecycles, improving data quality, and coordinating vendors.
Responsibilities
Own complex, time-sensitive data and evaluation programs for AI research teams and internal partners. This role works across technical program management, data operations, and vendor coordination to turn ambiguous requests into reliable programs that improve LLM and agent evaluation.
Lead data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospectives.
Turn ambiguous technical requests into clear specifications with milestones, owners, dependencies, and acceptance criteria.
Improve data quality processes using quantitative measures and qualitative review, including analysis of task coverage, difficulty, and reward distributions.
Manage external vendors through the delivery lifecycle and strengthen scalable delivery pipelines.
Improve workflows, documentation, automation, and cross-functional handoffs for data production and evaluation.
Partner with research, platform, and marketplace teams to turn findings into product improvements.
Qualification
Background in RLHF
Required
At least 3 years of experience owning complex technical programs in AI/ML, data operations, support engineering, applied research, forward-deployed engineering, or a related cross-functional field.
Experience as a Technical Program Manager or strategic program lead at a startup, with a record of bringing structure to changing or ambiguous requirements.
Strong quantitative and qualitative judgment, including the ability to assess aggregate dataset metrics and inspect individual examples.
Experience improving data quality procedures, managing external vendors, and supporting technical customers or research teams with urgent, evolving needs.
Background in RLHF, reinforcement-learning environments, agent evaluations, human-in-the-loop data pipelines, annotation workflows, or expert data collection is valuable.
Strong communication skills and comfort working independently in a fast-paced environment.
