XDOF
AI & Machine LearningDirect Ashby career site

Member of Technical Staff, Evaluations

Location
Remote (US) · Hybrid
Job type
Full-time
Level
Mid-level
Listed
Today

Skills & highlights for this role

  • robot learning
  • MuJoCo
  • Python
  • benchmark design
  • statistical evaluation

Also useful

  • Isaac Sim
  • C++
  • reinforcement learning
  • real2sim

What You’ll Do

Evaluations engineers own how we measure models and what we learn from it. Sample projects include:

  • designing benchmark tasks for LLMs, VLAs, and WAMs that test real robotics capabilities, from long-horizon planning to dexterous manipulation
  • building verifiers, success detectors, and progress metrics that grade episodes reliably in simulation and on real hardware
  • building harnesses that let LLMs and coding agents control robots, directly or by planning over VLAs
  • measuring how well simulation results predict real-world performance, and improving that correlation
  • fine-tuning open-source VLAs consistently so every model gets a fair baseline
  • analyzing failures across runs to find capability gaps, and turning them into targeted data collection and RL environments
  • producing partner-facing results with rigorous statistics and clear reporting

You might be a good fit if you have

  • strong programming skills in Python or C++
  • built benchmarks, evaluation pipelines, or graders for ML models, and thought hard about what a score actually measures
  • hands-on experience with robot learning, simulation (MuJoCo, Isaac Sim, or similar), or deploying policies on real hardware
  • worked with LLM or coding agents, and can read a trace to diagnose why a model failed
  • an adversarial mindset: your first question about a metric is how it could be gamed
  • built and shipped software in fast-moving environments

Bonus points if you have

  • contributed to robotics or agent benchmarks (e.g. LIBERO, BEHAVIOR, RoboArena, Terminal-Bench)
  • experience with RL, reward modeling, or post-training
  • worked on real2sim, system identification, or sim-to-real transfer
  • experience with statistics for evaluation, like confidence intervals and variance across runs
  • familiarity with containerized evaluation frameworks such as Harbor

About XDOF

At XDOF, we’re at an inflection point. Frontier labs are racing to build general-purpose robots, and high-quality training data is the bottleneck. We’re building the foundation behind the foundation models - the data collection systems, operational capability, exabyte-scale data warehouse, and model evaluation platform - to help our partners drive the field forward.

We’re looking for an Evaluations Software Engineer to build how the field measures robot intelligence. You’ll develop the benchmarks that tell our partners how their LLMs, VLAs, and WAMs actually perform on physical tasks, in simulation and on real hardware. Examples of things you might work on: designing benchmark tasks, developing our simulation pipeline, measuring how well simulation predicts real-world performance, and integrating novel research into the evaluations workflow.

Read the original posting

From XDOF's careers page, first listed on Rolev.ai on November 17, 2026. Rolev.ai is not the employer.

How Rolev.ai helps with this application

Rolev.ai is a job search and application tool for US job seekers. For this Member of Technical Staff, Evaluations role at XDOF, it shows how your resume fits, tailors your resume to the posting, and prepares the application for you to review.

  • See how you fit

    Openings from employer career sites are ranked against your whole resume—and searchable inside Claude, ChatGPT, and Cursor via Rolev.ai MCP.

  • Tailor without inventing

    Every change to your resume is checked against your original. New experience goes in only with your yes—zero fabricated claims.

  • Draft the full kit

    Tailored and Full Match resumes, role-specific cover letters, and answers to employer screening questions are prepared in seconds.

  • 1-Click Extension Autofill

    The Chrome extension fills the employer's Greenhouse, Workday, Lever, or Ashby form in 1 click. You review it and press Submit.

Common questions

Where is the Member of Technical Staff, Evaluations job at XDOF located?

Yes, this Member of Technical Staff, Evaluations opening at XDOF is listed as Remote (US) (Hybrid). Always check XDOF's application page for specific state eligibility or relocation policies.

What qualifications are required for Member of Technical Staff, Evaluations at XDOF?

For the Member of Technical Staff, Evaluations role, XDOF highlights experience with robot learning, MuJoCo, Python, benchmark design, statistical evaluation. Rolev.ai can score your existing resume against these requirements and show any skill gaps before you apply.

How do I apply for the Member of Technical Staff, Evaluations position at XDOF?

This listing links directly to XDOF's Ashby application page. You can apply directly on XDOF's site, or use Rolev.ai to check your resume fit, generate a zero-hallucination tailored resume and cover letter for this posting, and autofill the Ashby application form in 1 click with the Rolev.ai Chrome extension. You always review the form and press Submit yourself.