About the position
These roles support Project Sparky, an AI-powered chat assistant for iOS and Android platforms. Sparky helps users manage everyday shopping tasks-such as ordering groceries or scheduling car services-by remembering preferences and past interactions to streamline repeat activities. The project is deeply rooted in generative AI and creating the groundwork to move towards agentic AI by enabling personalized, context-aware assistance that evolves with user behavior. We are seeking a highly skilled professional with extensive experience in large-scale distributed systems and cloud-native architectures. The ideal candidate will have a proven track record in designing and developing scalable APIs and agentic workflows, as well as integrating Large Language Models (LLMs), vector databases, and inference pipelines. A strong focus on performance, security, and reliability is essential. The candidate should demonstrate organizational-level architectural leadership and excel in cross-functional collaboration.
Responsibilities
- Architect Foundational Systems: Design and document the end-to-end backend architecture for our conversational AI platform, decomposing complex requirements into scalable, fault-tolerant microservices and data pipelines.
- Innovate in the LLM Space: Pioneer robust solutions for core conversational challenges, including state management, long-term memory RAG at scale, and low-latency model inference.
- Drive Technical Strategy & Roadmap: Lead the evaluation, selection, and integration of the core technology stack, including vector databases, model serving frameworks, orchestration engines, and cloud infrastructure.
- Define Engineering Excellence: Establish the gold standard for non-functional requirements, including system reliability, security, cost-efficiency, and the extreme low-latency performance essential for conversational AI.
- Pioneer LLM Operations (LLMOps): Architect the systems and processes for robust application monitoring, CI/CD, and telemetry.
- Solve the Hardest Problems: Lead root cause analysis for the most complex, systemic issues spanning model performance, distributed systems, and data integrity.
- Mentor and Elevate: Act as a force multiplier for the engineering organization. Mentor senior engineers, lead architectural reviews, and cultivate a culture of deep technical expertise, innovation, and continuous learning.
Requirements
- Advanced degree in Computer Science, Math, or related field preferred.
- 25+ years of software development experience, with a proven track record of designing, building, and operating large-scale, distributed backend systems.
- Expert-level knowledge of architectural principles, patterns, and styles, particularly in microservices, event-driven architectures, and distributed data systems.
- Deep, hands-on experience with the technical ecosystem for modern AI applications, including LLM serving and inference optimization techniques.
- Demonstrated expertise in at least one modern programming language such as Java, Python, Go, or C++.
- Extensive experience with cloud platforms (AWS, GCP, or Azure) and container orchestration technologies (Kubernetes).
- A strategic mindset, capable of translating ambiguous business needs into concrete and scalable technical architectures.
- Strong organizational-level architectural leadership and cross-functional collaboration skills.
Nice-to-haves
- Expertise in designing scalable APIs and agentic workflows.
- Experience with the performance tuning and fine-tuning of Large Language Models.
- Experience architecting and implementing large-scale Retrieval-Augmented Generation (RAG) pipelines.
- A history of significant contributions to open-source projects in the AI/ML or distributed systems communities.
- Experience building and deploying stateful conversational AI or chatbot systems from the ground up.
Benefits
- 401(k) match
- Stock purchase plan
- Paid maternity and parental leave
- PTO
- Multiple health plans
- Incentive awards for performance
- Short-term and long-term disability
- Company discounts
- Military Leave Pay
- Adoption and surrogacy expense reimbursement
- Live Better U education benefit program
