pencil - Agent Architect
Requirements
• 3+ Years of Direct GenAI Experience: You have a deep, intuitive, and technical understanding of LLMs (GPT-4, Claude, Gemini) and multimodal models (Stable Diffusion, Midjourney, Video Gen). • Systems Thinking: You don’t just write a prompt; you think about the latent space, the context window, and how one agent's output becomes another’s input. • Technical Literacy: While this isn't a "Software Engineer" role, you should be comfortable with Python, JSON structures, and API documentation. Experience with orchestration frameworks (LangChain, CrewAI, AutoGen) is a major plus. • Evaluation Obsession: You believe that if you can’t measure a prompt’s performance, you shouldn’t ship it. You are familiar with benchmarking and A/B testing AI outputs. • The "Creative/Technical" Bridge: You can sit in a room with a Creative Director and translate "make it feel more punchy" into a temperature adjustment and a few-shot prompting strategy. • You’ll Thrive Here If... • You find "hallucinations" to be a logic puzzle to be solved, not just a bug. • You are excited by the challenge of making an AI follow a 50-page brand book with 100% fidelity. • You want to build the infrastructure that defines how the next generation of advertising is created. • KPI & Success Measures • System Reliability: Reducing the "failure rate" of complex agent workflows. • Architectural Efficiency: Minimizing token usage/latency while increasing output quality. • Brand Alignment: Scoring of agent outputs against specific brand guidelines using automated evaluation tools. • Scale: Successful rollout of core agents used by the entire Pencil user base.
Responsibilities
• Agent Architecture: Design and implement multi-agent workflows, including task decomposition, state management, and tool-use (RAG, API integration, etc.). • Systematic Optimization: Move beyond "vibe-based" testing. Implement rigorous evaluation frameworks (e.g., using LLM-as-a-judge, promptfoo, or DSPy) to measure and improve agent performance at scale. • Scalable Frameworks: Develop reusable agents and prompt libraries that allow the platform to serve 1,000+ brands with unique voices simultaneously. • Cross-Functional Engineering: Partner with AI Engineering and Product teams to determine which behaviors should be handled via prompting, RAG, or fine-tuning. • Client Architecture: Act as the technical lead for complex client deployments, translating high-level creative briefs into deterministic AI workflows.
Benefits
• 25 days PTO plus public holidays, although we operate a Flexible Time Off scheme • Health insurance / private medical cover • Monthly stipend towards wellness, fitness, and learning and development • Remote - work from anywhere in your home country • Enhanced parental leave policies, whether you become a parent through birth, adoption or surrogacy • Access to our Pencil office in The Shard, London for our UK employees • Flexible working hours
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT