Product Manager, Evals
- Location
- San Francisco, CA · On-site
- Job type
- Full-time
- Level
- Senior
- Education
- Bachelor's degree
- Listed
- 19 days ago
Skills & highlights for this role
- Vendor negotiation and relationship management
- Technical writing
- Strong numerical literacy
- Attention to detail
- Technical background (CS/EE or production engineering experience)
Also useful
- Machine learning evaluation
- Search and information retrieval
- Defining success metrics for evaluation datasets
- Partnering with ML researchers and data engineers
Evals at Exa
The Evals Team sits at the heart of our mission. We train foundational models for search. Our goal is to build systems that can instantly filter the world's knowledge to exactly what you want, no matter how complex your query. Basically, put the web into an extremely powerful database. And to do that well, we need to measure what “good search” actually means. That’s where you come in.
As a Product Manager on the Evals team, you will partner with research, product, GTM, and engineering to oversee the quality and completeness of our evaluation data. You will be responsible for ensuring our search models are being evaluated against comprehensive and high-quality datasets, across numerous domains. You’ll manage relationships with external vendors and make key strategic decisions on what data is important for our evals - and acquire it. Your work will play a key role in Exa’s mission of a perfect information society.
Who You Are
- Hustler in negotiating, securing, and managing relationships with external vendors
- Care and excellence in writing. At Exa we value human communications free of AI generated text
- High ownership and attention to detail
- Strong numerical literacy and technical background. CS/EE degree, production engineering experience, or equivalent depth earned on the job
What You Could Do
- Build specific, localized eval feedback and signal
- Write a manifesto of what perfect search means
- Define metrics and success criteria for our evaluation data suites
- Partner closely with ML researchers, data engineers, infra engineers, and product to shape the data loops that improve our search models
Logistics
- Location: This is an in-person opportunity in San Francisco.
- Visas: We're happy to sponsor international candidates (e.g., STEM OPT, OPT, H1B, O1, E3). While we cannot guarantee your visa, we have historically been successful in sponsoring candidates from all over the world. If you receive an offer, our team will work hard to get you a visa.
- Benefits: We offer premium healthcare benefits (medical, dental, vision), fertility benefits, 16 weeks of fully paid parental leave for all new parents, and a monthly wellness stipend to all of our employees.
About Exa
Exa is an applied AI lab building a search engine unlike the world has ever seen. We build massive-scale infra to crawl the entire web, train state-of-the-art embedding models to process it, and design super high performant vector databases to retrieve over it. We now power search for Cursor, Cognition, HubSpot, and over 400,000 developers and have raised $350m from Lightspeed, Benchmark, and a16z.
Our ultimate goal is to build perfect search over all the world's information, far beyond Google. If you want to build massive-scale ML systems that will define the way the new AI world consumes information, this is the place for you.
Equal opportunity and legal notices
Exa is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, creed, color, religion, sex, sexual orientation, gender identity or expression, national origin, disability, age, veteran status, marital status, pregnancy or related conditions, criminal histories consistent with applicable law, or any other basis protected by applicable law.
From Exa's careers page, first listed on Rolev.ai on October 22, 2026. Rolev.ai is not the employer.
How Rolev.ai helps with this application
Rolev.ai is a job search and application tool for US job seekers. For this Product Manager, Evals role at Exa, it shows how your resume fits, tailors your resume to the posting, and prepares the application for you to review.
See how you fit
Openings from employer career sites are ranked against your whole resume—and searchable inside Claude, ChatGPT, and Cursor via Rolev.ai MCP.
Tailor without inventing
Every change to your resume is checked against your original. New experience goes in only with your yes—zero fabricated claims.
Draft the full kit
Tailored and Full Match resumes, role-specific cover letters, and answers to employer screening questions are prepared in seconds.
1-Click Extension Autofill
The Chrome extension fills the employer's Greenhouse, Workday, Lever, or Ashby form in 1 click. You review it and press Submit.
Common questions
Where is the Product Manager, Evals job at Exa located?
This Product Manager, Evals position at Exa is located in San Francisco, CA (On-site). Always check Exa's application page for specific state eligibility or relocation policies.
What qualifications are required for Product Manager, Evals at Exa?
For the Product Manager, Evals role, Exa highlights a bachelor's degree; experience with Vendor negotiation and relationship management, Technical writing, Strong numerical literacy, Attention to detail, Technical background (CS/EE or production engineering experience). Rolev.ai can score your existing resume against these requirements and show any skill gaps before you apply.
How do I apply for the Product Manager, Evals position at Exa?
This listing links directly to Exa's Ashby application page. You can apply directly on Exa's site, or use Rolev.ai to check your resume fit, generate a zero-hallucination tailored resume and cover letter for this posting, and autofill the Ashby application form in 1 click with the Rolev.ai Chrome extension. You always review the form and press Submit yourself.