Before a model, feature, or workflow reaches Harvey customers, lawyers evaluate and grade its output. The Evaluation Operations function orchestrates this process, partnering closely with Harvey's Legal Product Architect team as they scope and answer relevant research questions informing key product decisions.

Compare different AI-generated outputs for the same legal task (e.g., a memo, a contract redline, a research answer) and evaluate output quality against a written rubric.

Test new product features end-to-end in a pre-release environment and provide qualitative feedback based on your legal experience.

Review and assess grading rubrics for accuracy and completion.

Build evaluation datasets that include realistic legal queries, contract markups, and/or synthetic legal documents.

Most projects run between 2 and 20 hours over multiple days. Some require feedback within 24-48 hours of the request. You may decline any project.

JD and an active license to practice in at least one U.S. jurisdiction.

3+ years of practice experience at an AmLaw 100 or comparable firm, or in an in-house legal department.

Experience in either commercial litigation or transactional law.

Experience using AI tools in professional or daily life.

Clear, concise legal writing and the ability to express your reasoning in plain language.

Availability to pick up short-notice projects and occasionally work over weekends.

Attorney, Evaluation Operations (Contract)

Responsibilities

Qualification

Required

Preferred

Company

Harvey

More roles at Harvey