OKX - Senior Product Manager, Conversational AI Chatbot & Agent Quality
Requirements
• You have hands-on experience building and operating conversational AI products in production — not just shipping agents, but owning the quality systems, data pipelines, and operational platforms that keep them reliable at scale. Ideal candidates will have background in one or more of the following areas: • Knowledge Base & Data Quality — knowledge base architecture, retrieval quality tuning, content governance, labeling pipelines, annotation guidelines, training data impact tracking, and dataset freshness management • Knowledge Base & Data Quality • Agent Evaluation & Quality Assurance — evaluation harness design, test case schemas, automated scoring rubrics (correctness, groundedness, tool-use accuracy), LLM-as-judge evaluation, regression testing for non-deterministic systems, and feedback-driven improvement loops • Agent Evaluation & Quality Assurance • Chatbot Operations & Dialogue Design — SOP-to-agent-flow translation, edge case handling, escalation path design, log-based failure triage, and metrics ownership (resolution rate, fallback rate, per-intent accuracy, CSAT) • Chatbot Operations & Dialogue Design • Agent Runtime & Observability Platforms — agent runtime product requirements, tool permission models, task configuration interfaces, developer-facing observability dashboards, failure alerting logic, and debugging workflows • Agent Runtime & Observability Platforms • Human-in-the-Loop Workflows — low-confidence case routing, reviewer task interface design, correction data capture, and feedback loop integration back into training or knowledge pipelines • Human-in-the-Loop Workflows • Chatbot & Knowledge Base (Core) • Built or rebuilt a knowledge base — defined structure, wrote/reviewed content, fixed retrieval quality, saw metrics improve • Designed SOPs that became agent flows — mapped real business processes, handled edge cases, shipped as working dialogue flows • Owned a labeling pipeline — wrote annotation guidelines, QA'd batches, tracked whether labeled data moved production metrics • Moved a metric that mattered — resolution rate, fallback rate, CSAT — and can explain exactly what changed • Agent Harness & Platform Product (Strong Plus) • Designed an agent evaluation harness: defined test case schemas, scoring rubrics, and spec'd automated evaluation pipelines with engineering • Product-designed an internal agent platform: defined requirements for agent runtime — tool permission models, task configuration interfaces, developer-facing observability dashboards, and failure debugging workflows; owned the roadmap and shipped iteratively • Closed the eval-to-improvement loop: used harness output to prioritize knowledge fixes, prompt revisions, or flow changes — not just reported scores but drove action from them • Designed human-in-the-loop review workflows: low-confidence case routing, reviewer task interfaces, correction data capture and feedback loop back into training or knowledge pipelines
Responsibilities
• Own the end-to-end product lifecycle for AI agent infrastructure: from problem discovery and requirement definition, through development and launch, to post-launch iteration • Define product vision, strategy, and roadmap for infrastructure that serves internal AI product teams as primary customers • Partner closely with engineering, AI/ML research, and business stakeholders to translate complex technical capabilities into well-scoped, shippable product requirements • Establish evaluation frameworks and quality metrics that drive continuous improvement of agent performance, reliability, and safety • Build feedback loops connecting production signals (user ratings, task success/failure, escalations) back into model and data improvement pipelines • Define platform SLAs, KPIs, and capacity plans; balance scalability and reliability with fast iteration • Stay current with the rapidly evolving AI agent landscape and translate emerging capabilities (new models, tool-use paradigms, orchestration patterns) into infrastructure opportunities • What We Look For In You • 5+ years of product management experience, with at least 2 years focused on AI/ML products or developer infrastructure • Demonstrated experience shipping AI products or platforms to production (consumer or enterprise) • Strong technical acumen: comfortable discussing model architectures, prompt engineering, API design, and system trade-offs with engineers • Solid understanding of the AI agent development lifecycle: from data collection and annotation, through model training and evaluation, to production deployment and monitoring • Data-driven mindset with experience defining KPIs, running experiments, and making decisions under uncertainty • Strong written and verbal communication in English; Mandarin Chinese is a significant plus • Nice-To-Haves • Nice-To-Haves • Hands-on experience with AI agent frameworks or platforms (e.g., Claude Skills, Anthropic MCP, LangChain / LangGraph, OpenAI Assistants API, Coze, Dify, or similar) • Experience building PaaS-level platforms: agent development environments, low-code/full-code toolchains, or multi-agent orchestration systems • Background in software engineering or technical PM with coding experience • Experience with responsible AI practices: safety evaluation, bias mitigation, hallucination detection, and guardrail design • Familiarity with the Web3 or fintech domain • Understanding of the competitive landscape: familiarity with how leading companies (both in China and globally) are approaching agent infrastructure and evaluation
Benefits
• L&D programs and education subsidy for employees' growth and development • Various team building programs and company events • Wellness and meal allowances • Comprehensive healthcare schemes for employees and dependants • More that we love to tell you along the process! • All official OKX vacancies are published on this website. While roles may appear on selected third-party platforms from time to time, information on other sites may be inaccurate or outdated. If in doubt, please apply directly through our official careers website. • If in doubt, please apply directly through our official careers website. • Information collected and processed as part of the recruitment process of any job application you choose to submit is subject to OKX's Candidate Privacy Notice.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT