Software Engineer, Inference

About the role

As a Junior Software Engineer (IC1/IC2) on the Inference team, you'll own a single production service end-to-end shipping features, maintaining dashboards/alarms, and improving operational excellence. You'll collaborate with peers across runtime, scheduling, and networking to move key SLIs.

What you'll do

• Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops.
• Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
• Improve latency/throughput via micro-batching, concurrency tuning, and caching.
• Participate in on-call; lead post-incident action items for your service.
• Review peers' PRs and contribute design feedback in team reviews.

What we're looking for

• 0-3 years of software engineering experience (or equivalent) building cloud/distributed services in production.
• Proficiency in Python or Go; familiarity with gRPC/HTTP APIs and containerized deployments.
• Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
• Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
• Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).

Nice to have

• Experience with inference frameworks (Triton, vLLM, TensorRT-LLM, Ray Serve, TorchServe).
• Hands-on performance profiling/benchmarking (e.g., wrk, vegeta, perf, Nsight).
• Contributions to reliability practices: SLI/SLOs or postmortem improvements.

Benefits (highlights)

• 100% employer-paid medical, dental, and vision
• 401(k) with a generous match
• Flexible PTO and a remote-friendly culture
• Work with top talent in a culture of curiosity, ownership, and innovation

• The base salary range for this role is $120,000.00 - $160,000.00.
• The starting salary will be determined based on job-related knowledge, skills, experience, and market location.
• We strive for both market alignment and internal equity when determining compensation.
• In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility).

What We Offer

• The range we've posted represents the typical compensation range for this role.
• To determine actual compensation, we review the market rate for each candidate which can include a variety of factors.
• These include qualifications, experience, interview performance, and location.

• In addition to a competitive salary, we offer a variety of benefits to support your needs.
• The benefits below reflect our US-based offerings for full-time employees; for roles in other locations, benefits vary and are shared during the hiring process.
• These include:

• Medical, dental, and vision insurance - 100% paid for by CoreWeave
• Company-paid Life Insurance
• Voluntary supplemental life insurance
• Short and long-term disability insurance
• Flexible Spending Account
• Health Savings Account
• Tuition Reimbursement
• Ability to Participate in Employee Stock Purchase Program (ESPP)
• Mental Wellness Benefits through Spring Health
• Family-Forming support provided by Carrot
• Paid Parental Leave
• Flexible, full-service childcare support with Kinside
• 401(k) with a generous employer match
• Flexible PTO
• Catered lunch each day in our office and data center locations
• A casual work environment
• A work culture focused on innovative disruption

• California Applicants
• California Consumer Privacy Act