As a Staff Analyst, AI, you will serve as a strategic analytics partner to the teams building and operating Hims & Hers' AI products (e.g., patient-facing AI chat assistants, internal AI tools for care teams). You will work cross-functionally with engineers, product managers, clinical operations, and finance to measure product performance, quantify commercial and clinical impact, and shape the experimentation and measurement strategy for AI-driven experiences. This position will report to the Manager of AI Analytics.
Provide thought leadership and strategic partnership to AI Product stakeholders, influencing roadmap and investment decisions with data-driven insight into AI product performance
Own experimentation strategy for AI product launches and iterations (e.g., GrowthBook-based tests), including test design, success metrics, sample-size estimation, guardrails, and results interpretation
Apply causal inference and quasi-experimental methods — including fixed-effects models, difference-in-differences, and propensity score matching — to assess AI impact on retention and clinical/operational outcomes when randomized tests aren't feasible
Define, validate, and own metrics frameworks and measurement plans for new and evolving AI product use cases — from pre-launch instrumentation through post-launch readouts — translating ambiguous business questions into rigorous, well-instrumented metrics within a complex, multi-source data environment
Partner with ML/Applied AI teams to evaluate model performance in production and translate model-level signals (confidence, escalation type, resolution outcome) into product and business metrics
Own commercial impact analysis for AI products — cost savings, handle-time efficiency, and support deflection — and quantify retention/churn addressability to size the impact of AI initiatives
Work cross-functionally with AI/ML Engineering, Analytics Engineering, and Data Engineering teams to define tracking requirements and ensure data integrity across AI product pipelines and event schemas
Use Python or R alongside SQL to develop scalable analytical solutions (e.g., observational studies, survival/retention modeling) that go beyond what dashboards alone can answer
Design intuitive Looker dashboards that democratize AI product metrics and elevate self-service across Product and Care Operations teams
Conduct deep-dive analyses — including user/session segmentation — to answer strategic questions about AI product performance, identify where AI products are (and aren't) working, quantify opportunity areas, and recommend clear actions
Champion data quality, documentation, and deliverable consistency across dbt models and BigQuery tables that power AI metrics, while mentoring junior analysts
Qualification
8+ years of experience in analyticsBachelor's degree in Statistics5+ years of experience designingA proactivedbt
Required
8+ years of experience in analytics, data science, or a related quantitative field
Bachelor's degree in Statistics, Mathematics, Economics, Computer Science, Engineering, or a related field
5+ years of experience designing, executing, and analyzing A/B tests, ideally including tests on AI/ML-driven product experiences
Experience partnering with ML teams to evaluate model outputs and translate them into product/business metrics (e.g., classification accuracy, confidence scores, resolution quality)
Extensive experience working with data engineering and analytics engineering teams, including dbt-modeled production pipelines
Strong knowledge of experimental design and statistical inference; experience applying causal inference techniques beyond standard A/B testing, such as quasi-experimental methods or observational study design (e.g., fixed-effects regression, survival analysis)
Experience with subscription-based business models and metrics, including cohort analysis, retention/churn analysis, and addressability sizing
Expert SQL skills (BigQuery preferred) and advanced experience with Looker or Tableau
Ability to explain technical analyses clearly to non-technical audiences, including senior leadership
Experience shaping and influencing product strategy, not just executing analyses
A proactive, ownership-driven mindset with strong organizational and prioritization skills
Preferred
Experience evaluating conversational AI or LLM-based products (intent classification, containment/escalation, hallucination or quality monitoring)
Familiarity with telehealth concepts or subscription-based healthcare models, including clinical safety considerations in AI-assisted workflows
Experience with GrowthBook or a comparable experimentation platform
Experience with survival analysis or time-to-event modeling (e.g., lifelines, pyfixest, statsmodels)
