Mitratech - Senior Engineering Manager, AI Quality & Governance
Requirements
• You have 8+ years of software engineering experience including meaningful experience leading engineers as a manager or technical lead. You have shipped production AI systems and have hands-on experience with the operational complexity they introduce. • Strong experience designing and operating production backend systems and APIs • Demonstrated hands-on experience building, shipping, or operating production AI systems — ideally including LLM-powered or agentic workflows • Experience with AI observability, evaluation, or debugging systems — whether through platforms such as LangSmith or LangFuse, or through internally built equivalents • Practical experience designing or operating AI guardrails: content filtering, redaction, access controls, or other controls around model or agent behavior • Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001, and the ability to engage thoughtfully with legal, compliance, and security stakeholders • A strong bias toward hands-on execution, platform thinking, and creating paved roads for other teams • Experience in legal technology, regulated SaaS, or other environments where auditability and defensibility matter • Experience with privacy-sensitive systems and PII handling • Experience participating in AI incident reviews, red-team exercises, or internal review boards for production AI systems • The Stack Context • The Stack Context • Engineering environment primarily uses Python and TypeScript • Platform operates across AWS and OCI • Observability tooling is being established — LangFuse experience relevant but not required
Responsibilities
• Own the AI evaluations and observability platform — tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool use, and user outcomes • Design and operationalize automated and human-in-the-loop evaluation strategies, integrating regression checks for quality, safety, latency, and cost into engineering and release processes • Establish the standards that define production-ready AI across the organization — evaluation criteria, release gates, incident playbooks, and long-term quality metrics • Build and operate AI guardrails and policy enforcement capabilities: content controls, PII detection and redaction, audit logging, and request- or workflow-level policy checks • Translate emerging governance and risk expectations into working engineering systems and platform controls rather than static documentation • Own platform-level SLOs and tier-two incident support for AI behavior issues, partnering with product teams who remain first-line owners for features they ship • Act as an internal authority on AI quality and governance and participate in customer-facing conversations where product quality, safety, observability, or governance posture must be explained credibly • Hire, mentor, and grow the team over time — the combined function may later evolve into separate Quality/Observability and Governance/Policy teams
Benefits
• This is a high-impact role at the center of Mitratech's AI engineering organization. You will define what production-ready AI means across the platform, build the systems that enforce it, and serve as the senior authority on AI quality, safety, and observability. You will work closely with product and platform teams, set standards that apply org-wide, and help Mitratech ship AI that customers can trust. If building that kind of foundational capability appeals to you, this role is worth a serious look. • We will disclose intended pay ranges in our job ads for US-based opportunities – This role can be performed 100% remote anywhere in the US. Anticipated Pay Range: $210K – $230K Annually USD • Anticipated Pay Range: $210K – $230K Annually USD • Total compensation includes US employee benefits and annual bonus eligibility. • Health, Dental & Vision Insurance * • 401 (k) + Employer Match * • Unlimited PTO + 11 Paid Holidays + 4 Annual Paid Global Wellness Days Off • STD, LTD & Group Life Insurance • Paid Parental Leave • FSA & HSA Options • Employee Assistance Program • Career Advancement & Professional Development Opportunities • Employee Recognition • LinkedIn Learning Platform • Mitratech is proud to be an EEOE, M/F/D/V, and we are committed to diversity both in practice and spirit at the corporate level. Mitratech participates in the Electronic Employment Verification Program. E-Verify is an Internet-based system that compares information from an employee’s I-9 to data from the U.S. Department of Homeland Security and Social Security Administration Records. To learn more, visit: everify.com
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT