whitecircle - Data Labeler

Remote$30k - $50k3w ago

Remote WW Data Analyst Quality Assurance

Requirements

• Has exceptional attention to detail • Can make consistent decisions across large volumes of data • Enjoys analysing nuanced situations where there isn't always a clear answer • Can follow guidelines while exercising good judgment • Communicates clearly and explains reasoning well • Is curious about AI and how these systems work • YOU MIGHT BE A GREAT FIT IF YOU • Have experience with content moderation, trust & safety, quality assurance, compliance, or policy enforcement • Have experience in data annotation, AI evaluation, RLHF, or model assessment • Have worked with AI tools extensively and understand their strengths and limitations • Enjoy finding edge cases and unusual model behavior • This role may involve reviewing content that is offensive, harmful, violent, sexual, or otherwise disturbing We provide tooling, and support, but candidates should be comfortable working with sensitive content when necessary

Responsibilities

• Review and evaluate AI conversations and model outputs • Assess responses for safety, quality, accuracy, policy compliance, and user intent • Identify harmful, unsafe, misleading, or low-quality behavior • Label and categorize model outputs according to internal evaluation frameworks • Moderate sensitive content and identify policy violations • Compare, rank, and score model responses • Investigate edge cases and ambiguous situations • Provide structured feedback to researchers and engineers • Help improve evaluation guidelines and annotation processes • Contribute to the datasets used to train and evaluate AI systems

Benefits

• Paid time off in line with your local regulations, no matter where you work from • All the hardware, tools, and services you need • 1. Intro call with HR (25 min) • 2. Take-home assignment • 3. Final conversation with our CEO (35 min)