Senior Software Engineer - Model Performance
- Pay
- $220,000 – $320,000 per year
- Location
- San Francisco, CA · On-site
- Job type
- Full-time
- Level
- Senior
- Experience
- 2+ years
- Listed
- Today
Skills & highlights for this role
- CUDA
- GPU profiling
- LLM inference frameworks
Also useful
- Kubernetes
- Docker
- C++
How well does your resume fit this job?
Upload your resume to see your score for this Inference Inc. role, what matches, what's missing, and your best other matches.
Scored on our server and not saved there. The file stays in this browser so you don't have to upload it again after you sign up.
About the role
Help us make inference blazingly fast. If you love squeezing every last drop of performance out of GPUs, diving deep into CUDA kernels, and turning optimization techniques into production systems, we'd love to meet you.
About the Role
You will be responsible for making our inference stack as fast and efficient as possible. Your work spans from implementing known optimization techniques to experimenting with novel approaches, always with the goal of serving models faster and cheaper at scale.
Your north star is inference performance: latency, throughput, cost efficiency, and how quickly we can bring new model architectures into production. You'll work across the full inference stack-from CUDA kernels to serving frameworks-to find and eliminate bottlenecks. This role reports directly to the founding team. You'll have the autonomy, a large compute budget, and technical support to push the limits of what's possible in model serving.
Key Responsibilities
- Implement and productionize optimization techniques including quantization, speculative decoding, KV cache optimization, continuous batching, and LoRA serving
- Deep dive into inference frameworks (vLLM, SGLang, TensorRT-LLM) and underlying libraries to debug and improve performance
- Profile and optimize CUDA kernels and GPU utilization across our serving infrastructure
- Add support for new model architectures, ensuring they meet our performance standards before going to production
- Experiment with novel inference techniques and bring successful approaches into production
- Build tooling and benchmarks to measure and track inference performance across our fleet
- Collaborate with applied ML engineers to ensure trained models can be served efficiently
Requirements
- 2+ years of experience in ML systems, inference optimization, or GPU programming
- Strong proficiency in Python and familiarity with C++
- Hands-on experience with LLM inference frameworks (vLLM, SGLang, TensorRT-LLM, or similar)
- Deep understanding of GPU architecture and experience profiling GPU workloads
- Familiarity with LLM optimization techniques (quantization, speculative decoding, continuous batching, KV cache management)
- Experience with PyTorch and understanding of how models execute on hardware
- Track record of measurably improving system performance
Nice-to-Have
- Experience with CUDA programming
- Familiarity with serving non-LLM models (TTS, vision, embeddings)
- Experience with distributed inference and multi-GPU serving
- Contributions to open-source inference frameworks
- Experience with Docker and Kubernetes
You don't need to tick every box. Curiosity and the ability to learn quickly matter more.
Compensation
We offer competitive compensation, equity in a high-growth startup, and comprehensive benefits. The base salary range for this role is $220,000 - $320,000, plus equity and benefits, depending on experience.
About Inference.net
Inference.net trains and hosts specialized language models for companies that need frontier-quality AI at a fraction of the cost. The models we train match GPT-5 accuracy but are smaller, faster, and up to 90% cheaper. Our platform handles everything end-to-end: distillation, training, evaluation, and planet-scale hosting.
We are a well-funded ten-person team of engineers who work in-person in downtown San Francisco on difficult, high-impact engineering problems. Everyone on the team has been writing code for over 10 years, and has founded and run their own software companies. We are high-agency, adaptable, and collaborative. We value creativity alongside technical prowess and humility. We work hard, and deeply enjoy the work that we do. Most of us are in the office 4 days a week in SF; hybrid works for Bay Area candidates.
Equal opportunity and legal notices
Inference.net is an equal opportunity employer. We welcome applicants from all backgrounds and don't discriminate based on race, color, religion, gender, sexual orientation, national origin, genetics, disability, age, or veteran status.
If you're excited about making AI inference faster for everyone, we'd love to hear from you. Please send your resume and GitHub to and/or apply here on Ashby.
From Inference Inc.'s careers page, first listed on Rolev.ai on November 19, 2026. Rolev.ai is not the employer.
How Rolev.ai helps with this application
Rolev.ai is a job search and application tool for US job seekers. For this Senior Software Engineer - Model Performance role at Inference Inc., it shows how your resume fits, tailors your resume to the posting, and prepares the application for you to review.
See how you fit
Openings from employer career sites are ranked against your whole resume—and searchable inside Claude, ChatGPT, and Cursor via Rolev.ai MCP.
Tailor without inventing
Every change to your resume is checked against your original. New experience goes in only with your yes—zero fabricated claims.
Draft the full kit
Tailored and Full Match resumes, role-specific cover letters, and answers to employer screening questions are prepared in seconds.
1-Click Extension Autofill
The Chrome extension fills the employer's Greenhouse, Workday, Lever, or Ashby form in 1 click. You review it and press Submit.
Similar roles
All Software Developer jobs- Technical Lead, Data EngineeringRevance · Nashville, TN
- Senior Product Engineer, McLean, VAAppian Corporation · McLean, VA
- Staff Software Engineer, Tools - Central Technology, Unreal EcosystemRiotgames · Los Angeles, CA
- Software Engineering Intern (Winter '27)Kira · San Francisco, NY
- Fullstack Software Engineer - Embedded PaymentsXplor · Kansas City, MO · $95k – $105k/yr
- Forward Deployed Engineer, PaymentsPolygon Labs · Remote (US)
Common questions
What is the salary for the Senior Software Engineer - Model Performance role at Inference Inc.?
The Senior Software Engineer - Model Performance posting at Inference Inc. states compensation of $220,000 – $320,000 per year. It is listed as a full-time position. Final pay may depend on location, experience, and interview evaluation on Inference Inc.'s official career site.
Where is the Senior Software Engineer - Model Performance job at Inference Inc. located?
This Senior Software Engineer - Model Performance position at Inference Inc. is located in San Francisco, CA (On-site). Always check Inference Inc.'s application page for specific state eligibility or relocation policies.
What qualifications are required for Senior Software Engineer - Model Performance at Inference Inc.?
For the Senior Software Engineer - Model Performance role, Inference Inc. highlights 2+ years of relevant experience; experience with CUDA, GPU profiling, LLM inference frameworks. Rolev.ai can score your existing resume against these requirements and show any skill gaps before you apply.
How do I apply for the Senior Software Engineer - Model Performance position at Inference Inc.?
This listing links directly to Inference Inc.'s Ashby application page. You can apply directly on Inference Inc.'s site, or use Rolev.ai to check your resume fit, generate a zero-hallucination tailored resume and cover letter for this posting, and autofill the Ashby application form in 1 click with the Rolev.ai Chrome extension. You always review the form and press Submit yourself.