Lumin AI Senior Product Manager at Lumin AI leading research, experimentation, and improvement of AI systems for an AI automation platform. Own the Intelligence and Applied AI product area to drive model capabilities and customer outcomes.
Responsibilities
As a Product Manager working on AI systems, you will play a foundational role in directing research, experimentation and rapid improvement of AI systems towards building a capable, reliable AI automation platform. The platform is used by organizations worldwide to deploy and scale executable AI automations in mission critical production environments. You will own the Intelligence and Applied AI product area, which means deciding what the team measures, how fast it can test an idea, and which model capabilities graduate from a promising result into something customers depend on every day. You are expected to have a strong proficiency in fundamentals of product management, a willingness to pick up new concepts as needed and an ability to drive technical projects in ambitious environments.
Define which internal AI benchmarks the team optimizes toward and which customer outcomes those benchmarks predict
Partner on evaluation frameworks to accelerate the speed and direction of experimentation
Decide which model improvements become shipped platform capability and which experiments to stop early
Set the bar for accuracy, generalization and failure behavior for automations running in production
Collaborate and contribute on the core product development to deliver higher platform capabilities
Work with engineering on observability and monitoring systems to safety check model behavior in critical settings
Establish how the team operates while working through early-stage questions that don't have obvious answers yet
Enable sales and customer success to speak accurately about what the AI does and does not do
Qualification
Fluency in how models are evaluatedComfort using evalsFounder and ownership mindset
Required
Proven track record of shipping AI or ML powered products in challenging projects
At least 5 years of product management experience, including 0 to 1 work
Fluency in how models are evaluated, where they fail, and what changes between a demo and a production deployment
Comfort using evals, benchmarks and model performance data as primary decision inputs
Solid fundamentals in system design and technical trade-offs, with the ability to write specifications for technical and non-technical audiences
Attention to detail and a first-principles thinking towards real world deployment of intelligent systems
Founder and ownership mindset, with experience taking products from ideation through launch and scale
