Staff Software Engineer, AI Gateway

The Gateway sits at the center of modern AI applications, offering

• A model and tool routing layer with built-in security and cost optimization
• A familiar OpenAI-compatible interface and MCP server
• Unified observability and policy enforcement
• Auto-RAG, tool injection, session summarization, and more

We’re just getting started and we need exceptional engineers to help us build the backbone of the future of agent-based development.

What You’ll Do

• Set technical direction for core systems powering the AI Gateway 0 the model router, MCP gateway, and control plane - and make sure architectural decisions hold up as usage scales by 10x
• Consistently identify gaps in our infrastructure, process, or tooling before they become blockers, and drive the fix from idea to production
• Take a significant, hands-on role in defining, delivering, and implementing the Gateway's technical strategy, in close partnership with engineering leadership
• Own the hardest, highest-ambiguity problems in the system: dynamic model selection, auto-failover, cost-based routing, and policy enforcement that has to work end-to-end
• Act as the go-to expert for secure credential storage, session summarization, caching, and rate limiting - the capabilities everything else depends on
• Build the infrastructure that lets us, and our customers, evaluate, replay, and backtest agent and LLM behavior with confidence
• Mentor senior and mid-level engineers, and raise the technical bar for the team through code, design review, and example
• Partner with product, design, and leadership to shape not just what the control plane UI does, but where the product goes next

Key Problems You’ll Help Solve

• Define the unified abstraction layer across diverse model and tool providers (OpenAI, Anthropic, Google, AWS Bedrock) that the rest of the org builds on
• Architect secure, scalable identity and credential vaulting for tool and model access, trusted enough to run at enterprise scale
• Build the infrastructure for real-time and historical analytics of AI agent behavior that leadership and customers both rely on
• Guarantee that policy enforcement and logging work end-to-end - from prompt to tool to response - with no gaps an auditor (or attacker) could find
• Set the bar for developer experience through intuitive APIs and first-class observability

Required Qualifications

• 8+ years of backend engineering experience, with a track record of owning production-grade systems at significant scale
• Deep specialization in distributed systems, cloud-native infrastructure, and API design - recognized by peers and cross-functional partners as a go-to expert, not just experienced
• Experience building secure, high-throughput services (gateways, proxies, load balancers, policy engines), and a history of making judgment calls other engineers rely on
• Fluency in Go and/or Rust
• A track record of identifying systemic gaps - in architecture, process, or tooling-- and driving the fix across team boundaries, not just within your own
• A strong product mindset, with real experience shaping technical direction rather than only executing against it
• Comfortable operating with wide latitude: given a problem and significant autonomy in how to solve it, including when and who to pull in
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience

Bonus Qualifications

• Prior experience with OpenAI, Anthropic, or similar LLM APIs
• Familiarity with RAG architectures, vector databases, or agent frameworks (e.g., LangChain, AutoGPT, CrewAI)
• Experience with policy engines (e.g., OPA), observability frameworks (e.g., OpenTelemetry), or API gateways
• Understanding of OAuth 2.1, secret management, and cloud IAM systems
• Experience with Kubernetes, Docker, and microservices architecture
• A track record of setting technical standards that other teams or engineers went on to adopt

Perks & Benefits

• Remote-first by design - Work from your home, with offices in Seattle and Paris for connection and collaboration.
• Flexibility that fits your life - We trust you to manage your schedule while delivering great work.
• Time to recharge - Generous PTO, designated quarterly Whaleness Days, and a designated end-of-year Whaleness break.
• Home office support - Set up your workspace for comfort and success.
• Technology stipend - Equivalent to US$100 net per month to help support your work.
• Learning & development - Annual stipend for conferences, courses, certifications, and continued learning.
• Parental leave - 16 weeks of paid parental leave after six months of employment.
• Equity for all full-time employees - Share in Docker's long-term success as we continue to grow.
• Comprehensive benefits - Medical, retirement, and paid holidays vary by country.
• Docker swag - Because representing the whale never gets old.

#LI-REMOTE