Machine Learning Engineer - Moderation

San Francisco (United States)full time

On Site

$220k–$280k

H1B: no

Black Forest Labs Machine Learning Engineer - Moderation at Black Forest Labs develops and improves multilayered detection models to identify harmful or illegal content across multiple modalities. The role involves collaboration with infrastructure, legal, policy, and product teams to ensure effective and feasible moderation.

Responsibilities

This is a Senior / Staff IC role. Our ideal candidate is able to work across the full stack of ML engineering, collaborate cross-functionally with many teams, including legal, policy, go-to market, and infra, and has a real interest in the domains of AI safety and integrity.
Develop ML models and systems to detect harmful and/or illegal content across a variety of classes and modalities
Collaborate with our core infrastructure team to integrate these models into a large-scale production serving pipeline
Handle customer requests and feedback on our moderation behavior, and make quick adjustments in response
Collaborate with our legal, policy, and product teams to design moderation policies that are technically feasible and effective while preserving a positive user experience

Qualification

Experience in moderation/safety spaceHow We Work Together

Required

Experience deploying VLMs at scale for the classification of multimodal content
Experience building AI systems around state-of-the-art open-source VLMs
Experience tuning models for classification tasks, including building eval sets, reasoning about precision/recall at threshold, and monitoring for drift
Comfort with the full stack of ML engineering: from research and experimentation to large-scale deployment

Preferred

Comfort with the adversarial nature of generative AI moderation: prompt obfuscation, jailbreaks, output laundering, and evasion generally
Experience in moderation/safety space, preferably across a range of modalities (text, image, video, audio)
Experience optimizing VLM inference latency at scale
Experience with non-VLM solutions for safety classification (e.g., classical ML)
Domain knowledge and experience with content copyright and/or child safety moderation systems
How We Work Together
We’re a distributed team with real offices that people actually use. This role is based in San Francisco, and you’ll join us in the office at least 2 days a week, or one full week every other week.
Everything we do is grounded in four values: