Newtonx The Project Lead, AI Model Training at NewtonX manages client engagements end to end on data operations platform projects. They are accountable for operational execution and quality outcomes across multiple concurrent projects.
Responsibilities
NewtonX is turning the world’s largest verified expert network into the evaluation, post-training, and audit data that enterprises and AI labs need to deploy AI in production. We build high-quality, expert-generated datasets that train and evaluate frontier AI systems — spanning domain eval sprints, custom data solutions, model behavior audits, agentic RL environments, and domain QA-as-a-service.
This business is taking off, and we are expanding our Delivery team to meet it. We are hiring a Project Lead to own client engagements end to end, from scoping through delivery, on our data operations platform. You will be the client’s point of contact and the person accountable for the operational execution and the quality outcome across a portfolio of concurrent projects — many of them complex agentic-workflow evaluations.
Our commercial team wins the business. From the moment it is won, it is yours: you scope it, configure it, run it, defend its quality, and deliver it. When a client asks how it is going, you are the person who answers. When the spec turns out to be wrong at scale, you are the person who fixes it.
Location: Remote (US or Canada) or Hybrid (New York City)
Scope and set up every engagement
Translate the client’s need into a bounded statement of work. Turn a training or evaluation objective into concrete task design, a defined quality bar (defect-rate and inter-annotator agreement targets), an annotator profile, and — for agentic projects — an environment and tool strategy.
Configure the project on the platform. Stand up the multi-layer quality pipeline, task forms, grading rubrics, annotator enablement, and routing.
Author the first-draft guidelines and gold set with the ML Lead, and pressure-test them before production starts. Most delivery failures are scoping failures; your job is to catch them here.
Run production
Own quality, throughput, and cost. Track how each project is performing against its targets, diagnose what is drifting, and act before it reaches the client.
Manage the annotator pool on your projects. Assignment, layer access, qualification, and performance — working with the annotator delivery team to get the right experts on the right tasks.
Keep annotators aligned. Run project channels and calibration sessions, handle exceptions and escalations, and adjust configuration as the work reveals what the spec missed.
Qualification
3+ years of experience in data labelingAn operator who goes into the weedsClient-facing and credibleAnalytically rigorousA builderExceptional at holding many threadsAdaptable and resilientDiscreet
Required
3+ years of experience in data labeling, RLHF/SFT, or AI evaluation delivery — you have shipped real annotation or evaluation projects at a data labeling company (Scale, Surge, Mercor, Snorkel, Handshake, Centific, Labelbox, SuperAnnotate, or similar) or inside an AI lab. This is a firm requirement; we are not looking for a generalist project manager.
An operator who goes into the weeds. You are comfortable owning quality metrics, debugging a stalled queue, and rewriting a rubric that is not holding up in production.
Client-facing and credible. You can run a scoping call with a technical buyer, deliver bad news without losing the room, and translate operational mechanics into terms a client cares about.
Analytically rigorous. Data analysis skills using SQL or Python are a strong plus.
A builder. Processes here are young and some are still manual. You want to stand things up rather than inherit a finished machine.
Exceptional at holding many threads. You will run several concurrent projects with different clients, quality bars, and annotator pools.
Adaptable and resilient. Specs change, clients change their minds, and no two weeks look the same. You roll with it and find the new angle.
Discreet. This role handles annotator personal data and client-confidential information under strict data-security and confidentiality standards; all work stays within approved systems.
Valid US or Canadian work authorization required. Visa sponsorship is not available for this role.
