Outsmart - Lead Assessment Developer

• Master's degree (Ph.D. preferred) in educational measurement, assessment , or a related field • 3+ years managing operational, high-stakes assessment lifecycles end-to-end • Proven experience leading standard-setting studies and managing diverse SME panels • Experience managing external vendors for field tests and pilot studies • Working knowledge of validity, reliability, and fairness standards (e.g., AERA/APA/NCME) • Familiarity with cognitive complexity frameworks (DOK, Bloom's) • Strong communicator who can translate psychometric concepts for non-technical audiences • Bonus: knowledge of CTT, IRT, CDM, DIF,, AI-assisted item generation, or accreditation reviews • What You’ll Own • What You’ll Own • End-to-end assessment development operations for assigned assessments • SME network — recruitment, training, calibration, and quality • Standard-setting studies — method selection, panel facilitation, and final reporting • Vendor and external partner relationships, SLAs, and security protocols • Field testing and pilot operations • Scoring operations for constructed-response and performance-based assessments • Project timelines, risk identification, and stakeholder communication • Core assessment documentation — specs, blueprints, training materials, technical reports • What Helps You Succeed • You thrive in ambiguity and are energized by building from scratch • You balance psychometric rigor with startup pragmatism — you know when to ship and iterate • You communicate with unusual clarity across technical and non-technical audiences • You take ownership without being asked and surface risks early • You're excited about AI's role in assessment and know where validation is non-negotiable • You care deeply about fairness and the learners behind every score