Software Engineer, ML Performance and Scaling

About the position

Running machine learning (ML) algorithms at our scale often requires solving novel systems problems. As a Performance Engineer, you'll be responsible for identifying these problems, and then developing systems that optimize the throughput and robustness of our largest distributed systems. Strong candidates here will have a track record of solving large-scale systems problems and will be excited to grow to become an expert in ML also.

Responsibilities

• Identify systems problems related to running ML algorithms at scale.
• Develop systems that optimize throughput and robustness of distributed systems.
• Implement low-latency high-throughput sampling for large language models.
• Implement GPU kernels to adapt models to low-precision inference.
• Write a custom load-balancing algorithm to optimize serving efficiency.
• Build quantitative models of system performance.
• Design and implement a fault-tolerant distributed system with complex network topology.
• Debug kernel-level network latency spikes in a containerized environment.

Requirements

• Significant software engineering or machine learning experience, particularly at supercomputing scale.
• Results-oriented with a bias towards flexibility and impact.
• Willingness to pick up slack outside of job description.
• Enjoy pair programming.
• Desire to learn more about machine learning research.
• Care about the societal impacts of work.

Nice-to-haves

• Experience with high performance, large-scale ML systems.
• Experience with GPU/Accelerator programming.
• Knowledge of ML framework internals.
• Understanding of OS internals.
• Experience with language modeling using transformers.

Benefits

• Competitive compensation and benefits.
• Optional equity donation matching.
• Generous vacation and parental leave.
• Flexible working hours.
• Lovely office space for collaboration.