Technical Operations Analyst, Automation & Evaluations
About the Team
At OpenAI, our User Safety & Risk Operations (USRO) team helps protect our products and users from abuse, fraud, safety risks, and other forms of misuse. We translate real-world user and operational signals into timely decisions, practical interventions, and improvements to our products and systems.
This role sits on the Risk Incubation & Systems (RIS) vertical in USRO, which investigates emerging risks, builds data and automation capabilities, and develops workflows that can scale into durable operational homes.
About the Role
As part of the Automation & Analytics function within RIS, you’ll partner with teams across USRO to support their data, analysis, measurement, and automation needs. You’ll build tools and workflows that help teams make informed decisions and improve how they operate. You’ll work with domain experts to turn judgment into testable criteria, then help take promising prototypes into reliable operation.
You will work hands-on with Trust & Safety domain experts, investigators, operational leads, and engineers to turn ambiguous requirements into practical systems. You’ll test and document the tools and workflows you build, and agree with partner teams who will monitor and maintain them.
Please note: This role may involve exposure to sensitive or concerning content. Strong discretion, good judgment, and resilience are essential.
In This Role, You Will
- Work with teams across USRO to define the questions they need answered and identify work that could benefit from automation.
- Use SQL and Python to investigate operational problems and turn findings into recommendations teams can act on.
- Build and maintain datasets and dashboards that teams can use for recurring analysis.
- Define metrics to assess the quality and impact of USRO’s work, and measure whether changes produce the expected improvements.
- Build scripts and API integrations to automate recurring tasks, using LLMs where they improve the workflow.
- Evaluate automated workflows against expert-reviewed examples and existing processes to understand errors and measure their practical value.
- Hand off workflows with clear ownership, maintenance instructions, and a feedback loop for reviewers.
You Might Thrive in This Role If You
- Have strong SQL skills and can use Python to analyze data and build tools that solve practical problems.
- Have worked with messy (operational) data and can assess its quality and explain how gaps affect your findings.
- Can turn a loosely defined request into a clear problem and choose a practical way to solve it.
- Have built and maintained datasets, dashboards, scripts, or API integrations that other people use in their work.
- Know how to define useful metrics and measure whether changes to a process or tool improve outcomes.
- Can explain the trade-offs involved in workflow automation, including where human judgment is needed.
- Can work with domain experts to define clear assessment criteria and work through disagreements about ambiguous cases.
- Have experience designing evaluations or quality checks and using the results to decide when to launch a tool and how to improve it.
Nice to Have
- Experience transferring prototypes or new workflows into reliable, scaled operations.