Job Title: LLM Red Team Intern (Evaluation Systems)

Company Name: Elloe AI

Job Details: RemoteInternship

Job Url: https://hiring.cafe/viewjob/18pmlqvnwo52zr0y

Job Description: Posted 8mo agoLLM Red Team Intern (Evaluation Systems)@ Elloe AIView All JobsWebsiteUnited StatesRemoteInternshipResponsibilities:Red team, Evaluation design, Safety intelligenceRequirements Summary:Internship role for ML/AI researchers or engineers with experience in LLMs, eval sets, and prompt design; strong safety mindset and adversarial thinking.Technical Tools Mentioned:GPT-4, Claude, Gemini, Open models
Internship | Remote | LLM Evaluation | Reports to CTO or Safety LeadAbout ElloeElloe is the immune system for AI.We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction.Our modules (TruthChecker™, AutoRAG™, Autopsy™) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production.This role will help us break, stress-test, and harden the models used by governments and enterprises alike.About the RoleYou’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping.You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny.What You’ll Own1. Red Teaming & Risk TestingCreate prompts to trigger hallucinations, policy violations, or failure scenariosStress test Elloe-protected deployments using open and proprietary modelsDocument behavioral exploits across use cases (healthcare, compliance, gov)2. Evaluation DesignBuild truthsets and scoring rubrics tied to factuality, policy, or ethical standardsBenchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models)Collaborate with product to refine and expand our eval harnesses3. Safety IntelligenceIdentify blind spots in current detection logicRecommend scoring methods or red flag thresholds for deploymentSupport internal model comparison reports or customer safety auditsWho You AreML/AI researcher or engineer (undergrad, grad, or early career)Experience working with LLMs, eval sets, and prompt designStrong attention to detail, grounded in safety and adversarial thinkingBonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming toolsWhy This MattersThis is real-world alignment, not research theater.You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection.You’ll leave this role with:Exposure to high-stakes LLM safety deploymentsPublished frameworks or scoring methods used by enterprisesMentorship from technical founders operating at the bleeding edge of AI safetyLogistics & ApplicationStart Date: RollingDuration: 12–16 weeksCompensation: Research stipendLocation: Remote-first; flexible for global candidatesTo Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.