handshake - AI Red Teamer (Remote)
Requirements
• Familiarity with Python or other scripting languages • Experience working with LLM APIs or evaluation tooling • Comfort with structured data annotation and rubric-based scoring • Prior work in trust and safety, content moderation, QA, or security research • Subject matter expertise in any high-risk domain (cybersecurity, chemistry, biology, medicine, law, finance, etc.)
Responsibilities
• Craft creative prompts and multi-turn scenarios to stress-test AI guardrails across diverse risk categories • Discover ways around safety filters, restrictions, and defenses using jailbreak, evasion, and prompt injection techniques • Explore edge cases to provoke disallowed, harmful, or incorrect outputs • Evaluate and score model responses against structured harm taxonomies and severity rubrics • Document experiments clearly, including what you tried, why you tried it, and what it revealed • Review and refine adversarial prompts generated by other team members • Contribute to harm taxonomy development, calibration exercises, and inter-rater reliability work • Collaborate with engineers, data scientists, and researchers to share findings and strengthen defenses • Work with potentially disturbing content on a regular basis (see Content Warning below) • Stay current on jailbreaks, attack methods, and evolving model behaviors • DESIRED CAPABILITIES • Strong hands-on experience using multiple LLMs (ChatGPT, Claude, Gemini, open-source models, etc.) • Intuition for crafting adversarial prompts; familiarity with jailbreak or evasion techniques is a strong plus • Creative, adversarial problem-solving skills • Clear and thoughtful written communication • Strong ethical judgment and the ability to separate adversarial thinking from personal values • Self-directed, collaborative, and comfortable in feedback-heavy environments • Curiosity, persistence, and comfort with frequent failure in experimentation • You treat every model response as a hypothesis to challenge • You can switch between creative free-association and rigorous documentation in the same session • You go deep into unusual interests (fandoms, niche internet cultures, gaming exploits, Wikipedia rabbit holes, etc.) • You come from a creative background: writing, visual art, improv, puzzle design, or similar • You are energized by finding the thing nobody else thought to try • You are genuinely passionate about AI and follow the space closely • CONTENT WARNING • This role involves regular and deliberate exposure to harmful content. You will encounter and intentionally generate content involving violence, self-harm, hate speech, sexually explicit material, child safety scenarios, and other categories of harmful output as part of structured adversarial testing. Candidates must be able to engage with this material professionally and sustainably. Support resources are available.
Benefits
• $32 – $95 per hour • For cash compensation, we set standard ranges for all U.S.-based roles based on function, level, and geographic location, benchmarked against similar stage growth companies. In order to be compliant with local legislation, as well as to provide greater transparency to candidates, we share salary ranges on all job postings regardless of desired hiring location. Final offer amounts are determined by multiple factors, including geographic location as well as candidate experience and expertise, and may vary from the amounts listed above. • Upload your resume here to autofill key application fields. • Drop your resume here! • Parsing your resume. Autofilling key fields... • Please share the phonetic spelling of your name to help our team pronounce it correctly. • or drag and drop here • Which AI models have you used regularly, and what have you used them for? (100 words maximum) • Yes, already based in the United States • Yes, can relocate to the United States before starting • I'd like to discuss it • Describe a time an AI model behaved unexpectedly or failed. What were you trying to do, what happened, and how did you investigate it? Personal experimentation counts. (150 words maximum) • What is an unusual topic, hobby, or community you know deeply? How could that knowledge help you spot failures other testers might miss? (150 words maximum) • AI evaluation or annotation • Adversarial testing • Writing or creative work • QA or software testing • Trust and safety • Security research • Research or analysis • Personal projects • Do you agree to allow Handshake to contact you about job opportunities for up to 24 months? • Recruiting Privacy Policy • Handshake may use Artificial Intelligence with this application. Learn more.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT