About the position
AWS Neuron is the complete software stack for the AWS Inferentia and Trainium cloud-scale machine learning accelerators and servers that use them. This role is for a software engineer in the Machine Learning Inference Model Enablement and Generality team for AWS Neuron at Annapurna Labs. This role is responsible for development, enablement and performance tuning of a wide variety of LLM model families, including massive scale large language models like the Llama family, DeepSeek and beyond, as well as stable diffusion, vision transformers and many more. The Inference Model Enablement and Generality team works side by side with compiler engineers and runtime engineers to create, build and tune distributed inference solutions with Trainium and Inferentia. Experience optimizing LLM inference performance for both latency and throughput is highly desired. Experience with distributed inference libraries such as vLLM is a bonus.
Responsibilities
- Help lead the efforts building distributed inference support for Pytorch in the Neuron SDK.
- Tune models to ensure highest performance and maximize efficiency on AWS Trainium and Inferentia silicon and servers.
- Design and code solutions to drive efficiencies in software architecture.
- Create metrics, implement automation and other improvements, and resolve the root cause of software defects.
- Build high-impact solutions to deliver to a large customer base.
- Participate in design discussions, code review, and communicate with internal and external stakeholders.
- Work cross-functionally to help drive business decisions with technical input.
- Work in a startup-like development environment.
Requirements
- 5+ years of non-internship professional software development experience.
- 5+ years of non-internship design or architecture experience of new and existing systems.
- Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles.
- Work experience on optimizations for improving model execution.
- Experience programming with at least one software programming language.
Nice-to-haves
- 5+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
- Masters degree in computer science or equivalent.
Benefits
- Equity and sign-on payments may be provided as part of a total compensation package.
- Full range of medical, financial, and/or other benefits.
