Docker - Senior Software Engineer, AI Gateway
Requirements
• 6+ years of backend engineering experience with production-grade systems • Deep knowledge of distributed and highly scalable systems, cloud-native infrastructure, and API design • Experience building secure, high-throughput services (e.g., gateways, proxies, load balancers, policy engines) • Fluency in Go, and/or Rust (both preferred) • Familiarity with AI/ML platforms or model serving infrastructure • A strong product mindset—you're excited about building developer-facing tools • Ownership mentality with a bias for shipping, learning, and iterating • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience • Prior experience with OpenAI, Anthropic, or similar LLM APIs • Familiarity with RAG architectures, vector databases, or agent frameworks (e.g., LangChain, AutoGPT, CrewAI) • Experience with policy engines (e.g., OPA), observability frameworks (e.g., OpenTelemetry), or API gateways • Understanding of OAuth2.1, secret management, and cloud IAM systems • Experience with Kubernetes, Docker, and microservices architecture
Responsibilities
• Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane • Build infrastructure that supports dynamic model selection/routing, auto-failover, cost-based routing, and policy enforcement • Own critical capabilities such as secure credential storage, session summarization, caching, and rate limiting • Develop APIs for developers building with OpenAI-compatible interfaces and the Model Context Protocol • Build the underlying infrastructure to support evaluation, telemetry, replay, and backtesting for agents and LLM workflows • Lead architectural decisions and mentor engineers as the team scales • Performance tuning - caching, session summarization, and rate limiting that keep the gateway fast under real production load • Contribute to roadmap planning, technical strategy, and cross-functional alignment • Key Problems You’ll Help Solve • Building a unified abstraction over fundamentally different model and tool providers without leaking their differences into the developer experience • Making credential and identity handling for tool access both airtight and fast • Giving developers real-time and historical visibility into what their agents are doing, with policy enforcement that can't be bypassed • Keeping latency low and reliability high as traffic scales from early adopters to production fleets
Benefits
• Work on what’s next: The agent era is just beginning. Help define the infrastructure stack for the next generation of software. • Work on what’s next: • Own mission-critical infrastructure: Be part of a small team building the backbone of secure, intelligent agent applications. • Own mission-critical infrastructure: • Impact at scale: Your work will touch tens of thousands of developers building AI-first applications. • Impact at scale: • Engineering excellence: Join a world-class engineering team that values simplicity, code quality, and deep technical thinking. • Engineering excellence: • Remote-first culture: Work from anywhere with a team that trusts and empowers you. • Remote-first culture: • Docker considers visa sponsorship on a case-by-case basis based on business needs. • United States: $160,900 – $260,700 + equity • United States: • ______________________________________________________________________ • Open vacancy: This posting is for an existing open role. • Open vacancy: • AI in hiring: Docker may use AI-assisted tools during our recruiting process. • AI in hiring: • Interview recordings: Candidates will be invited to opt in to interview recordings to support interviewer calibration and consistent evaluations. Recordings are optional and require explicit consent. • Interview recordings: • Remote-first by design – Work from your home, with offices in Seattle and Paris for connection and collaboration. • Remote-first by design • Flexibility that fits your life – We trust you to manage your schedule while delivering great work. • Flexibility that fits your life • Time to recharge – Generous PTO, designated quarterly Whaleness Days, and a designated end-of-year Whaleness break. • Time to recharge • Technology stipend – Equivalent to US$100 net per month to help support your work. • Technology stipend • US$100 net per month • Learning & development – Annual stipend for conferences, courses, certifications, and continued learning. • Learning & development • Parental leave – 16 weeks of paid parental leave after six months of employment. • Parental leave • Equity for all full-time employees – Share in Docker's long-term success as we continue to grow. • Equity for all full-time employees • Comprehensive benefits – Medical, retirement, and paid holidays vary by country. • Docker swag – Because representing the whale never gets old. • Docker swag
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT