Sr. Software Development Engineer, Inference Team - AWS Neuron

About the role

AWS Neuron is the complete software stack for AWS Inferentia and Trainium, AWS purpose-built accelerators for cloud-scale machine learning. This senior software engineering role is part of the Machine Learning Inference Applications team and focuses on delivering high-performance model inference solutions for customer workloads running on Inferentia- and Trainium-powered instances.

The engineer will lead the development of core serving technologies within open-source frameworks (such as vLLM and SGLang) to enable customers to serve large-scale inference efficiently on AWS Neuron. The engineer will also drive excellence in inference framework optimization and engineering best practices.

The engineer will work closely with model development engineers, performance engineers, compiler engineers, and runtime engineers to ensure end-to-end model performance and deliver production-ready accuracy, scalability, and efficiency across a broad range of models and customer use cases.

Key job responsibilities

The engineer will lead the customization and optimization of open-source inference frameworks (vLLM, SGLang) on AWS Neuron-including core framework logic such as scheduling and model execution-applying state-of-the-art techniques in kernel development, parallel computation, distributed KV cache, speculative decoding, and systems engineering to deliver best-in-class LLM serving performance.

The engineer will also influence the team's technical roadmap by evaluating emerging inference research and translating it into production-ready features, and raise the bar through design leadership and mentorship. This role requires close collaboration across model development, compiler, runtime, and performance engineering teams.

Basic Qualifications

• 5+ years of non-internship professional software development experience
• 5+ years of programming with at least one software programming language experience
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Experience as a mentor, tech lead or leading an engineering team
• Fundamentals of Machine learning models, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model performance.

Preferred Qualifications

• Bachelor's degree in computer science or equivalent
• Hands-on experience with PyTorch or JAX, preferably involving developing and deploying LLMs in production on GPUs, AWS Neuron, TPUs, or other AI accelerator hardware
• Experience extending or contributing to inference frameworks such as vLLM or SGLang
• Kernel development experience (e.g., CUDA, Triton)

Additional information

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If the country/region you're applying in isn't listed, please contact your Recruiting Partner.

Pay and benefits

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave.

USA, WA, Seattle - 168,100.00 - 227,400.00 USD annually