Juniper Square - Lead Product Manager, Agent Runtime & Evals
Requirements
• 7+ years of product management experience in B2B or Enterprise SaaS, including 3+ years operating in an AI-oriented role. • Proven experience owning a complex product area end-to-end, from strategy and roadmap through execution and iteration, ideally in a zero-to-one context. • Hands-on understanding of how agent evaluation works: golden datasets, data labeling, automated judges, regression testing, and knowing when a human has to step in. • Background building agents and replacing human-driven processes with product and automation, with real evaluation and monitoring discipline in production. • A strong engineering partner with a product mindset. You can go deep with engineers on topics like typed contracts between agents and workflows, structured and deterministic outputs, and agent memory, and you know the right questions to ask even when you don't have the answers. This is not a TPM role: your focus is how people actually use these tools. • User empathy for non-engineering builders and operators, including how people oversee large numbers of automated agents. • Fluency with LLMs and common AI tools (e.g. Claude, Cursor). You should be able to run your own prototypes and use AI to get work done faster than a traditional PM workflow would allow. Bonus points for deep experience with managed agent platforms such as Claude Managed Agents or Workato Genies, including evals, tool and connector access, versions, and deployments. • Strong communication skills, written and verbal, with the ability to articulate product decisions clearly to engineers, designers, executives, and customers alike. • Track record of leading through influence, ideally mentoring or managing other product managers, and setting strategy that multiple teams execute against. • What success looks like • By the end of Q1: The platform runs real fund administration workflows with real automations, such as capital calls and distributions, built and deployed on Foundry. • By the end of Q1: • Following the Q1 milestone: Those workflows roll out to the entire fund administration team, fundamentally changing how they work. • Following the Q1 milestone: • Long term: An "AI factory" where AI watches how work gets done and proposes new workflows and agents, and people move from building to oversight and course correction. A new client could be up and running in a day and fully automated within the week. • Long term:
Responsibilities
• Build the agent runtime for non-engineers: Own the vision, strategy, and roadmap for a platform where business users take an agent, app, or script from idea to governed production without an engineering cycle. Define the full lifecycle: draft, test, publish, version, deploy, run, observe, improve, and retire. • Build the agent runtime for non-engineers: • Own evaluations and model governance: Define how every automation is tested before it ships. Work with engineering and our shared eval platform team on golden datasets, data labeling, and regression gates that block promotion. Decide when an automated judge is enough and when a human has to stay in the loop. Shape how evals decide which models are eligible for which agents, so a new model release becomes a product decision rather than a scramble. • Own evaluations and model governance: • Solve the "who evaluates the evaluator" problem: Agents that grade other agents need their own checks. Partner with engineering on patterns that keep quality measurable at the scale of thousands of agents, and turn them into experiences builders can understand and trust. • Solve the "who evaluates the evaluator" problem: • Shape memory, learning, and self-improvement: Define how agents keep long-lived memory, compact it, and improve from experience, with an eye on adoption and user-facing impact rather than algorithms alone. • Shape memory, learning, and self-improvement: • Run a model-agnostic platform: Own model routing, cost attribution, and pricing so teams can choose the right model for each job and see what it costs. • Run a model-agnostic platform: • Integrate with broader systems: Define how hosted agents reach data and take action across our product suite, our fund administration systems of record, and third-party tools. Own the connector, credential, and access-grant model so integrations are reusable rather than rebuilt per agent. Define agent identity and run modes, including when an agent acts with the invoking user's access and when it acts under its own service identity. • Integrate with broader systems: • Give people oversight at scale: Design the tools that let fund administration teams supervise large fleets of agents: dashboards, run history, observability through traces, metrics, and logs, kill switches, and auditability. Make it clear what is an exception and what is an issue. • Give people oversight at scale: • Define how automations bind to workflows: Partner with the workflow orchestration PM on the agent registry and typed contracts that describe what an agent needs and produces, and how workflows bind to it. Protect the safety boundary that agent-driven steps stage changes and people release them. • Define how automations bind to workflows: • Deliver configuration to business SMEs: Partner with design and engineering to build the tools SMEs use to configure, test, and tune automations themselves. Hold the line on convention over configuration so every setting earns its place. • Deliver configuration to business SMEs: • Stay ahead of the market: Track managed agent runtimes, agent frameworks, and automation platforms. Turn that view into clear build, buy, and partner calls. • Stay ahead of the market: • Be a leader: Set direction across the platform and adjacent teams, raise the bar for product craft, and be the person engineering trusts on ambiguous platform tradeoffs. • Be a leader:
Benefits
• Compensation for this position includes a base salary, equity, and a variety of benefits. The U.S. salary range for this role is $200,000 - $230,000+ USD. Actual base salaries will be based on candidate-specific factors, including experience, skillset, and location, and local minimum pay requirements as applicable. • Health, dental, and vision care for you and your family • Mental wellness coverage • Fertility and growing family support • Flex Time Off in addition to company-paid holidays • Paid family leave, medical leave, and bereavement leave policies • Retirement saving plans • Allowance to customize your work and technology setup at home • Annual professional development stipend • Your recruiter can provide additional details about compensation and benefits. • #LI-Remote #LI-NY1 #experiencedprofessional #Juniper-US #Juniper-Canada
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT