About this role
Summary: Design, analyze, and validate adversarial AI attack scenarios including prompt injection and data exfiltration. Author novel attack mechanisms, build grading rubrics, and develop automated validation pipelines to ensure realistic and reproducible security evaluations.
As an Adversarial AI Evaluation Scientist, you will design, analyze, and validate adversarial agent tasks: indirect prompt-injection (IPI), data-exfiltration, tool-misuse, and related attack scenarios. Ensuring each one is realistic, fair, reproducible, and correctly graded. You will author new attack scenarios and graders from scratch, pressure-test and review existing ones, and decide whether a scenario genuinely demonstrates a model vulnerability or merely an artifact of how the task was built. The role combines adversarial-security creativity with statistical analysis and LLM-assisted tooling to build strong attacks, find quality issues, analyze attack-success patterns, and recommend or automate improvements.
Responsibilities:
Create
Design and author adversarial agent tasks — IPI, data-exfiltration, tool-misuse, and related scenarios — including the threat model, attack surface, payload, and the deterministic grading logic (success/failure criteria and reward checks).
Invent novel attack mechanisms and concealment/exfiltration techniques that go beyond known patterns, and build realistic target environments (e.g. email/doc assistants, tool-using agents) in RL Gym–style task formats.
Write grading rubrics, verifiers, and deterministic checks that detect the intended harm without reward hacks, token-collapse, or evadable gates.
Analyze
Measure and interpret attack success rate (ASR@K) and its variance; verify deterministic, reproducible behavior across repeated executions.
Identify anomalies and systemic issues across large task sets. Use LLMs and AI models to analyze attack failures, identify root causes, and assist with task or grader fixes.
Judge attack quality: novelty vs. known patterns, sophistication, stealth, and coverage across harm types and target surfaces.
Review & Validate
Review adversarial tasks for realism and threat-model soundness: is the attacker’s capability plausible, can the attacker actually reach the poisoned content, and is the target system realistic?
Assess attack fairness and attribution: confirm the harm is driven by the adversary, so a careful agent could plausibly pass.
Validate grading correctness: verify that success/failure criteria, reward logic, and deterministic checks detect the intended harm, with no reward hacks, token-collapse, or evadable gates.
Scale & Collaborate
Define authoring and validation guidelines, acceptance criteria, quality thresholds, and release gates for adversarial tasks.
Build automated validation, regression, and quality-monitoring pipelines for adversarial task sets.
Work closely with red-teamers, researchers, and task authors to improve attack design, fairness, and reliability.
Must Have:
- 5+ years in an analytical, ML, security-research, or applied-research role, including hands-on work with adversarial ML, LLM security, red-teaming, or AI safety evaluation.
- Bachelor’s in Computer Science, Machine Learning, Statistics, a related quantitative field, or equivalent practical experience.
- Working knowledge of prompt injection / indirect prompt injection, jailbreaks, data exfiltration, or tool-misuse attacks against LLM-based systems or agents: enough to both construct and critique them.
- Strong Python and data-analysis skills.
- Solid grounding in statistics and experiment design: distributions, variance analysis, hypothesis testing, and anomaly detection, applied to evaluation and attack-success measurement.
- Experience building, authoring, or auditing data-driven validation, evaluation, or quality frameworks.
- Experience using LLMs for analysis, evaluation, code generation, or automated remediation.
- Strong understanding of reproducibility, deterministic systems, and automated testing.
- Sharp adversarial thinking: the creativity to design attacks that land and the rigor to spot subtle ways a test can be unfair, gameable, or mis-graded.
- Excellent analytical, problem-solving, and communication skills.
Preferred Skills That Set You Apart
- Certifications in offensive cybersecurity (e.g., OSWA, OSWE, OSCE3, SEC542, SEC522).
- Experience designing, building, or evaluating AI agents, LLM-powered applications, or agentic workflows (e.g. email/doc assistants, tool-using agents).
- Hands-on experience authoring tasks in RL Gym–style formats, Gymnasium/OpenAI Gym, or agent-evaluation harnesses.
- Experience red-teaming production AI systems or building adversarial benchmark datasets.
- Experience with automated evaluation pipelines, regression testing, and quality gates.
- Familiarity with safety/security taxonomies (harm types, attack patterns) and threat modeling.
Alice is a trust, safety, and security company built for the AI era. We safeguard the communicative technologies people use to create, collaborate, and interact- whether with each other or with machines.
In a world where AI has fundamentally changed the nature of risk, Alice provides end-to-end coverage across the entire AI lifecycle. We support frontier model labs, enterprises, and UGC platforms with a comprehensive suite of solutions: from model hardening evaluations and pre-deployment red-teaming to runtime guardrails and ongoing drift detection.
Alice is widely considered a global leader in online safety and AI security. We have some of the most forward-thinking and passionate minds in the world working to safeguard over 3 billion users across the largest AI and tech platforms.
If you're creative and driven to secure the future of AI, we want to hear from you!