Note: The job is a remote job and is open to candidates in USA. Rex.zone connects AI/ML teams with professionals who build training datasets, run evaluations, and improve model quality across NLP, computer vision, and content policy evaluation. As an AI Trainer, you will evaluate and refine model outputs using structured rubrics, produce high-quality labeled data, and provide feedback aligned to RLHF-style preference signals within LLM training pipelines.

Responsibilities

• Perform RLHF-style ranking and preference judgments to improve helpfulness, harmlessness, and honesty
• Execute prompt evaluation and response grading using calibrated scoring rubrics and edge-case prompts
• Create and validate labeled datasets for NLP tasks (classification, summarization, NER)
• Conduct QA evaluation to ensure annotation guidelines compliance and consistent label quality
• Flag ambiguous instructions, propose rubric clarifications, and document decision rationales
• Identify systematic failure patterns (hallucinations, bias, refusal errors, safety gaps) and report insights
• Support content safety labeling for policy categories and sensitive topics
• Collaborate to refine evaluation frameworks, gold sets, and acceptance criteria

Skills

• Mid-Senior experience in AI training, data labeling, QA evaluation, or LLM evaluation
• Strong written communication and consistent rubric application
• High attention to detail and audit-ready documentation
• Comfort iterating with reviewer feedback while meeting throughput targets

Company Overview

• RemoExperts is a global platform where skilled professionals and tutors contribute to AI training across video, image, audio, text, code, math, and more. It was founded in 2025, and is headquartered in Palo Alto, California, US, with a workforce of 201-500 employees. Its website is https://www.remoexperts.com.

Apply To this Job