Applied AI Research Lead

Overview

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains—from doctors and lawyers to physicists and engineers—and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, poolside, Recraft, and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Israel, Czech Republic, Serbia, and more. We are headquartered in Amsterdam.

About the role

We’re looking for a hands-on AI Research Lead who will act as the bridge between our customers and our data generation teams. You will work directly with AI researchers, data scientists, and product teams at leading GenAI organizations to understand their needs, translate them into rigorous data strategies, and design the evaluation, training, or agentic workflows required to achieve their goals.

This is a hybrid research & solution design role: you’ll apply your deep expertise in LLMs and agentic systems to architect datasets, prompts, evaluation metrics, and workflows that power cutting-edge applications.

What You’ll Work On

1. Act as a customer-facing AI expert: deeply understand client research goals, model behaviors, and data needs to advise on best practices in data generation for LLMs, leveraging Toloka’s platform and ML audit capabilities.

2. Contribute thought leadership internally and externally on designing data strategies for LLM training, evaluation, and agentic workflows, including:

Defining quality criteria and evaluation metrics
Architecting prompt-based tasks, labeling methodologies, and model-in-the-loop pipelines
Identifying the right balance between expert labeling, crowdsourcing, and automation
3. Manage and grow a team of Solution Engineers – mentor them in technical execution, data methodology and client communication to turn research designs into production-grade pipelines.

4. Own project delivery end-to-end, ensuring timelines, quality, and scalability.

5. Work cross-functionally with Product, BizDev, and ML teams to support strategic accounts, pilot new data products and validate audit methods.

Company:

Toloka

Qualifications:

3+ years of experience as an AI researcher, ML scientist, or applied LLM engineer, ideally having trained or fine-tuned LLMs and/or built agentic systems. Strong understanding of LLM architectures, prompting techniques, synthetic data generation, data-centric AI, and evaluation frameworks.
Excellent ability to communicate complex technical concepts to customers and translate business goals into data and research plans.
At least 2+ years in a managerial or team lead role, overseeing technical teams or research engineers.
Strong Python skills and familiarity with common ML libraries (e.g., PyTorch, Hugging Face).
Comfortable working in fast-paced environments and collaborating cross-functionally with engineers, data scientists, and business teams.
Passionate about data quality, evaluation methodologies, and pushing the boundaries of GenAI applications.

Level of experience (years):

Mid Career (2+ years of experience)

Tagged as: Germany, Industry, Machine Learning