Sr Analyst, AI Analytics at Hims and Hers · LumiJobs

Responsibilities

As a Sr Analyst, AI Analytics, you will serve as a trusted analytics partner to the teams building and operating Hims & Hers' AI products (e.g., patient-facing AI chat assistants, internal AI tools for care teams). You will work cross-functionally with engineers, product managers, clinical operations, and finance to measure product performance, quantify commercial and clinical impact, and design rigorous experiments and measurement plans for AI-driven experiences. You will operate with high autonomy on your projects and report to the Sr Manager of AI Analytics.
Measure and Recommend: Deliver data-driven insight on AI product performance, with clear recommendations that inform Product roadmap and investment decisions
Run Experiments: Design and analyze tests for AI product launches and iterations (e.g., GrowthBook-based tests), including success metrics, sample-size estimation, guardrails, and results interpretation
Apply Causal Methods: Use quasi-experimental methods, including difference-in-differences, fixed-effects models, and propensity score matching, to assess AI impact on retention and clinical/operational outcomes when randomized tests aren't feasible
Build Measurement Plans: Define and validate metrics and measurement plans for new AI product use cases, from pre-launch instrumentation through post-launch readouts, translating ambiguous business questions into well-instrumented metrics within a complex, multi-source data environment
Connect Model to Business: Partner with ML/Applied AI teams to evaluate model performance in production and translate model-level signals (confidence, escalation type, resolution outcome) into product and business metrics
Quantify Commercial Impact: Analyze cost savings, handle-time efficiency, and support deflection for AI products, and size retention and churn addressability for AI initiatives
Ensure Data Integrity: Work cross-functionally with AI/ML Engineering, Analytics Engineering, and Data Engineering teams to define tracking requirements and ensure data integrity across AI product pipelines and event schemas
Go Beyond Dashboards: Use Python or R alongside SQL to develop analytical solutions (e.g., observational studies, survival/retention modeling) that go beyond what dashboards alone can answer
Enable Self-Service: Design intuitive Looker dashboards that make AI product metrics accessible to Product and Care Operations teams
Analyze Performance: Run deep-dive analyses, including user and session segmentation, to answer questions about AI product performance, identify where AI products are (and aren't) working, quantify opportunity areas, and recommend clear actions

Qualification

5+ years of experience in analyticsEducationExperimentationData PartnershipStatisticsSubscription MetricsTechnical ProficiencyCommunicationInfluenceOwnershipdbt

Required

5+ years of experience in analytics, data science, or a related quantitative field
Education: Bachelor's degree in Statistics, Mathematics, Economics, Computer Science, Engineering, or a related field
Experimentation: 3+ years of experience designing, executing, and analyzing A/B tests, ideally including tests on AI/ML-driven product experiences
Data Partnership: Experience working with data engineering and analytics engineering teams, including dbt-modeled production pipelines
Statistics: Solid grounding in experimental design and statistical inference, with exposure to causal inference techniques beyond standard A/B testing, such as quasi-experimental methods or observational study design (e.g., fixed-effects regression, survival analysis)
Subscription Metrics: Experience with subscription-based business models and metrics, including cohort analysis, retention/churn analysis, and addressability sizing
Technical Proficiency: Strong SQL skills (BigQuery preferred), working knowledge of Python or R, and advanced experience with Looker or Tableau
Communication: Ability to explain technical analyses clearly to non-technical audiences, including senior leadership
Influence: Experience turning analysis into recommendations that change product decisions
Ownership: A proactive, ownership-driven mindset with strong organizational and prioritization skills and comfort with ambiguity

Preferred

Experience evaluating conversational AI or LLM-based products (intent classification, containment/escalation, hallucination or quality monitoring)
Familiarity with telehealth concepts or subscription-based healthcare models, including clinical safety considerations in AI-assisted workflows
Experience with GrowthBook or a comparable experimentation platform
Experience with survival analysis or time-to-event modeling (e.g., lifelines, pyfixest, statsmodels)
dbt, Airflow, and Databricks experience

Sr Analyst, AI Analytics

Responsibilities

Qualification

Required

Preferred

Benefits

Company

Hims and Hers

More roles at Hims and Hers