Software Engineer, Evaluation Infrastructure
SOFTWARE ENGINEER, EVALUATION INFRASTRUCTURE
The RL infrastructure team is looking for an engineer to help develop our evaluation infrastructure.
RESPONSIBILITIES:
Build highly reliable, efficient, and easy to use infrastructure that runs all our evaluations at scale
Collaborate closely with modeling & evaluation teams on developing new evals, maintaining evaluation signal quality, and building internal tooling to support training our next-generation of models
Identify and resolve performance bottlenecks in all layers of the eval infrastructure stack, including inference, capacity fleet management, asynchronous eval orchestration, and more
BASIC QUALIFICATIONS:
Experience writing production-quality code in Python, PyTorch, JAX, Rust, or C++
PREFERRED SKILLS AND EXPERIENCE:
Experience in building, debugging, and optimizing efficiency of large-scale distributed systems
Willingness to dive deep and solve hard problems at all levels of the stack
Experience in LLM inference
Experience in GPU compute management, workload scheduling, and dynamic resource optimization
Experience in developing interfaces for comparing models and evals
COMPENSATION AND BENEFITS:
$180,000 - $440,000 USD
Base salary is just one part of our total rewards package at SpaceXSI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.