About this role
Summary: Join the ML Infrastructure & Research team to optimize LLM serving performance, build testing frameworks, and manage GPU resources. You will collaborate with founders to deploy improved models into customer workflows and turn research experiments into production-ready tools.
Responsibilities
- Measure and improve how fast and efficiently specialized language models run in real production use, including response time, throughput, GPU usage, memory caching, batching, and model compression (quantization).
- Build testing setups, scoring tools, and reinforcement learning environments from real customer AI-agent activity, so that day-to-day usage feeds a steady cycle of model improvement.
- Work directly with the founders and business partners to put improved models into important customer workflows that have strict reliability targets.
- Turn successful research experiments into reusable building blocks that teammates, customer agents, and production systems can use.
- Build and maintain the tools that manage GPU allocation across model training and live serving.
Requirements
- 1–8 years of hands-on experience in ML infrastructure, inference optimization, or applied LLM research engineering, at a fast-growing AI startup or a leading AI lab.
- Strong skills in Python and PyTorch, plus at least some of C++/CUDA, Triton, or JAX.
- Proven experience speeding up LLM serving tools such as vLLM, SGLang, or TensorRT-LLM.
- Self-driven and comfortable taking high ownership.
- Able to work in person 5 days a week in the Bay Area.
Skills Required
- LLM inference optimization & serving (vLLM, SGLang, TensorRT-LLM)
- Python & PyTorch
- C++ / CUDA / Triton / JAX
- Reinforcement learning post-training (RLHF/RLAIF)
- Model evaluation & grader design
- Synthetic data generation
- GPU resource management
- ML infrastructure engineering