ML Engineer-Algorithm for KV Cache Optimization

Summary: Develop and optimize machine learning algorithms for next-generation KV cache systems to improve inference quality and efficiency. Collaborate with testing and systems engineering teams to integrate ML innovations into production environments.

Machine Learning Engineer (Algorithm for KV Cache Optimization)

About Tensormesh

Building the caching layer for AI production.

Tensormesh was founded by the creators of LMCache to bring production-grade context caching to AI teams. We help developers, enterprises, and industry-leading partners turn KV cache to data assets that can be used to accelerate inference, improve inference quality, and repurposing KV cache for other use cases. Our product runs on the Tensormesh platform or deploys directly into customers' own inference clusters and environments, cutting inference cost and latency, improving generation quality, and more by making KV cache a ubiquitous layer across every inference stack.

About the role

As a Machine Learning Engineer at Tensormesh, you will develop and optimize machine learning algorithms for next-generation KV cache systems. You will explore how machine learning techniques on KV cache can improve inference quality and benefit other use cases, evaluate algorithm performance through systematic experiments, and collaborate closely with testing and systems engineering teams to bring your algorithms into production.

This role sits at the intersection of machine learning research and systems engineering. You will have the opportunity to deploy your ML innovation into real systems and serve large-scale audience.

What you'll do

• Design, develop, and optimize KV-cache-based machine learning algorithms, including KV-cache-based inference acceleration, KV cache editing, generative reranking and more.
• Design and conduct experiments to evaluate algorithm performance, analyze trade-offs, and identify opportunities for improvement.
• Collaborate with the testing team to validate algorithms across different workloads, models, and evaluation metrics.
• Work closely with systems engineers to integrate ML algorithms into production KV caching systems.
• Analyze performance results and iterate on algorithms to improve cache efficiency, inference latency, throughput, and overall system performance.
• Stay current with developments in machine learning, LLMs, and inference optimization, and translate relevant research into practical solutions.

What you'll bring

• Prior hands-on machine learning experience in at least one of the following areas:
• Recommendation systems
• Information retrieval
• KV cache optimization
• Machine learning model training
• Basic understanding of LLMs, including core concepts, inference workflows, and relevant evaluation metrics.
• Strong problem-solving and analytical skills, with the ability to design experiments, interpret results, and iterate on ML algorithms.
• Solid programming skills and the ability to implement, test, and evaluate machine learning solutions.
• Ability to collaborate effectively with cross-functional teams, particularly testing and systems engineering teams.

Bonus points

• Experience deploying machine learning optimizations into production systems.
• Experience developing machine-learning-based KV cache optimization algorithms.
• Experience with LLM-related model training or fine-tuning.
• Experience with reinforcement learning (RL) or reinforcement learning-based optimization.

Location & logistics

• US-based, hybrid.