Member of Technical Staff, AI Researcher (New Stealth Startup)

About Stealth Startup

We're launching a new startup that is redefining how mission-critical organizations adopt AI. Our platform helps organizations across healthcare, government, and other regulated industries transform institutional knowledge into intelligence assets that improve the accuracy, reliability, and impact of AI in real-world workflows. We're a fast-moving team of builders, researchers, and operators working to help customers create, measure, and continuously improve AI capabilities they can own and control.

About the Role

We are seeking a Member of Technical Staff, AI Researcher to help turn complex, real-world workflows into the evaluation frameworks, reinforcement learning environments, and feedback systems needed to advance frontier AI.

This is a blended research and engineering role for someone who operates at the intersection of scientific inquiry and technical execution. You will investigate how data, evaluations, and feedback systems influence model behavior, while building the infrastructure and tools required to test and apply those ideas.

Responsibilities

Design and build evaluation frameworks, benchmarks, and methodologies for frontier large language models, AI agents, multimodal models, and reinforcement learning environments

Develop reward models, programmatic verifiers, graders, and other feedback systems that make model behavior measurable and improvable

Design and run controlled supervised fine-tuning and reinforcement learning experiments to measure the effects of datasets on model capability, generalization, and alignment

Research and define what makes evaluations representative of real-world work and operational environments

Build systems for producing high-quality human data, including expert task design, annotation methodologies, data-quality signals, and data-attribution techniques

Translate research questions into lightweight, rigorous experiments that generate actionable findings

Prototype quickly, evaluate results, diagnose failure modes, and convert findings into improved benchmarks, datasets, or technical systems

Move promising methods from experimentation into scalable, production-quality implementations

Work across research, engineering, product, and operations to advance a shared technical roadmap

Contribute to the broader technical community through benchmarks, open-source tools, research, and technical writing

This Role is a Great Fit if You:

Are equally comfortable investigating an open-ended research question and building the system needed to test it

Approach AI research with a builder mindset and focus on producing measurable, usable outcomes

Are deeply interested in how data structure, selection, attribution, and quality affect model behavior

Can design focused experiments, interpret imperfect or ambiguous results, and translate findings into technical decisions

Are eager to move quickly without sacrificing rigor, maintainability, or engineering quality

Are comfortable working across unfamiliar domains and converting real-world workflows into measurable tasks

Demonstrate high agency, sound technical judgment, and a strong sense of ownership

Communicate complex technical concepts clearly to both technical and nontechnical stakeholders

Are comfortable operating in a fast-moving environment where priorities and technical approaches may evolve

Are motivated by the purpose and real-world impact of the work, not simply by access to emerging technologies

Skills & Qualifications

5+ years of relevant experience in machine learning, artificial intelligence, computer science, data science, software engineering, or a related technical field

Demonstrated ability to operate at the intersection of AI research and software engineering

Strong Python skills and experience building reliable, scalable software

Hands-on experience with AI agents, model evaluations, reinforcement learning, post-training, or related systems

Strong understanding of frontier AI methods, including benchmarks, evaluation techniques, agent harnesses, post-training approaches, and training-data design

Experience building or working with reward models, automated verifiers, graders, or other model-feedback systems

Experience designing evaluations or experiments that measure model capability, generalization, alignment, or agent performance

Ability to build research prototypes and develop them into maintainable, production-quality systems

Strong analytical, problem-solving, written communication, and technical documentation skills

U.S. citizenship is required

An advanced degree in machine learning, artificial intelligence, computer science, data science, or a related field is preferred but not required. Candidates with equivalent research and engineering experience, including strong technical, open-source, or published output, are encouraged to apply.

Salary Range: $250,000-400,000 + equity. This represents the typical base salary range for this position based on experience, skills, and other factors.

#LI-RCP