Welo Data is a global transformation partner that specializes in multilingual content transformation services. They are seeking a Generative AI Analyst to support the evaluation and quality review of AI-generated videos, focusing on identifying mismatches and visual errors in video content compared to text prompts.

Responsibilities

• Review text prompts and corresponding AI-generated video clips to identify mismatches, inconsistencies, and visual errors
• Tag specific parts of the prompt that do not match the video by identifying only the incorrect prompt text, rather than the full sentence unless needed
• Add visual or behavioral errors as annotation instances in annotation platform
• Write brief, clear descriptions for each error instance
• Avoid duplicate annotations by ensuring errors already captured as prompt mismatches are not added again as separate instances
• Evaluate video content for actions, context, motion, scene consistency, object behavior, and prompt alignment
• Apply project guidelines consistently across repetitive, detail-oriented tasks
• Use critical thinking and judgment to handle ambiguous scenarios and determine the most accurate ground truth
• Identify recurring or systematic errors across tasks and document examples for review
• Perform self-QA on completed work and correct errors before submission
• Participate in calibration sessions to align interpretation of guidelines and reduce annotator-to-annotator variance
• Incorporate feedback from quality reviews into subsequent work
• Support throughput and quality targets while maintaining accuracy at scale

Skills

• Experience in video annotation, multimedia annotation, content quality review, data labeling, computer vision labeling, Generative AI evaluation, or a closely related field
• Strong attention to detail and ability to identify subtle errors, mismatches, and inconsistencies between prompts and videos
• Solid understanding of US driving rules, road behavior, traffic scenarios, and roadway conventions
• Good reading comprehension and ability to accurately compare written prompts against visual outputs
• Strong written communication skills in English, with the ability to describe errors clearly and concisely
• Ability to follow detailed guidelines consistently and maintain high accuracy across repetitive tasks
• Critical thinking and sound judgment when reviewing ambiguous or complex scenarios
• Basic understanding of robotics behavior in real-world environments
• Comfortable working in structured annotation platforms or similar tools
• Ability to maintain focus and quality while reviewing multimedia content for extended periods
• Prior experience evaluating AI-generated video, synthetic media, autonomous driving data, robotics scenarios, or traffic-related visual content
• Familiarity with US traffic rules, road signage, lane behavior, vehicle interactions, pedestrian behavior, and common driving scenarios
• Experience with QA, audit, or second-pass review workflows, including calibration, sampling, defect tracking, or error taxonomy development
• Ability to identify recurring model failure patterns across tasks or batches

Benefits

• Remote with the option to work onsite in Santa Clara, CA area.
• Employment Type: W2 Full-Time Employee

Company Overview