Engineering Team Lead, Automation, Performance & Scale - k0rdent AI

About the role

We are looking for a hands-on engineering leader to own automation, non-functional verification, and performance & scale engineering for k0rdent AI - our multi-tenant control plane for enterprise GPU infrastructure. You will lead the team responsible for proving that k0rdent AI is fast, scalable, resilient, and ready to ship, and you will remain a senior individual contributor who writes code, designs test systems, and digs into failures yourself.

This is not a traditional QA management role. Your focus is engineering: building the automation frameworks, load and scale harnesses, and CI quality gates that let our engineering teams across the US, Europe, and APAC ship daily with confidence.

You will own k0rdent AI release acceptance - defining what "ready" means and making the go/no-go call with evidence - and you will raise the quality bar across development teams by bringing industry-standard practices into how they design, build, and test.

As a member of the k0rdent AI engineering leadership, you will take an active role in product roadmap planning and estimation, making sure non-functional requirements are designed in from the start rather than discovered at the end.

Main Responsibilities

Leadership & ownership

• Lead, hire, and grow the k0rdent AI automation and performance engineering team - set direction and priorities, run planning, coach engineers, and own delivery of the team's commitments.
• Own k0rdent AI acceptance verification: define release acceptance criteria and exit gates, coordinate verification across component teams, and drive evidence-based go/no-go decisions for each release, upgrade path, and hotfix.
• Drive industry-standard quality practices across development teams - shift-left testing, a healthy test pyramid, contract and integration testing, definition of done, quality gates in CI, and code review standards for test code.
• Take an active role in k0rdent AI roadmap planning and estimation: assess features for testability and non-functional risk, size verification effort, and make sure performance, scale, and reliability requirements are captured as explicit, measurable targets.
• Establish and report on quality and performance metrics - release readiness, defect escape rate, test coverage and flakiness, performance baselines and regressions - to engineering and product leadership.
• Work with product management, component development teams, SRE, and support to turn field issues and customer scenarios into automated coverage.

Hands-on engineering (individual contributor)

• Design, build, and evolve the test automation frameworks and tooling for k0rdent AI - API, end-to-end, upgrade, and multi-cluster scenarios - with a strong preference for maintainable, code-first solutions.
• Own performance and scale testing: control-plane scalability (number of clusters, tenants, objects, and API request rates), GPU workload scheduling and utilization, and AI inference/serving throughput and latency.
• Build and run non-functional verification: soak and endurance testing, resilience and failure-injection (chaos) testing, upgrade and rollback verification, backup/restore and disaster-recovery validation, and security and tenant-isolation checks.
• Build reproducible, on-demand test environments and infrastructure - from local kind clusters to multi-cluster, multi-region, and bare-metal GPU topologies - as code.
• Integrate automated suites and performance regression checks into CI/CD pipelines so every change is measured, not just tested.
• Automate test results management - collection, storage, trend analysis, and triage - and build observability and dashboards that make quality and performance visible.
• Debug and troubleshoot complex failures across the stack, trace issues into service code, and partner with component teams to drive root-cause fixes.

Qualifications & Requirements

Required Skills/Abilities

• 5+ years in software engineering, test automation, performance engineering, or SRE/platform engineering, including 2+ years leading engineers.
• Demonstrated ability to stay hands-on while leading - you still design frameworks, write production-quality code, and review your team's work.
• Strong knowledge of testing methodologies and techniques, with a proven track record of introducing modern quality practices into development teams.
• Strong test automation engineering: Python test frameworks (pytest or equivalent), API and end-to-end testing, test data and environment management, and keeping suites fast and free of flaky tests.
• Hands-on experience designing and executing performance, load, and scale tests for distributed systems or API platforms - using tools such as k6, Locust, JMeter, kube-burner, or ClusterLoader2 - including profiling, capacity analysis, and turning results into actionable engineering findings.
• Experience with resilience and non-functional verification: failure injection and chaos engineering, soak testing, and upgrade/rollback verification.
• Strong Kubernetes experience: workloads, networking, storage, RBAC, CRDs and operators, Cluster API, Helm, and cluster upgrades - able to debug from kubectl and cluster internals rather than dashboards alone.
• Strong coding ability in Python and shell scripting, and enough Go to read service code, write tooling or tests in it, trace a failure into it, and file a precise bug.
• Experience owning or defining release acceptance and go/no-go processes for complex infrastructure products.
• Strong CI/CD experience - pipelines as code, quality gates, and versioned release workflows (GitHub Actions, Jenkins, or equivalent) - and fluency with Git-based workflows.
• Experience building environments as code on Linux/UNIX - Docker, virtualization, Terraform or Ansible, GitOps (Argo CD, Flux) - and observing them with Prometheus, Grafana, and centralized logging.
• Experience contributing to roadmap planning, estimation, and prioritization alongside product and engineering leadership.
• Excellent communication with geographically distributed international teams; strong spoken and written English.

Nice to Have

• k0s or k0rdent ecosystem experience.
• GPU infrastructure on Kubernetes - device plugins, node feature discovery, scheduling and sharing of accelerators.
• AI/ML workload benchmarking - model serving and inference performance (throughput, latency, time-to-first-token), e.g., with vLLM, Triton, or KServe.
• Experience with public cloud providers (AWS, GCP, Azure) - networking, IAM, and managed Kubernetes.
• Bare-metal provisioning (Metal³ / BareMetalHost) or on-prem/OpenStack environments.
• Multi-region topologies, service mesh, or cross-cluster networking.
• Experience testing the stateful supporting stack - PostgreSQL, Temporal, Keycloak, API gateways, and message brokers.
• OpenTelemetry, distributed tracing, or formal SLO/error-budget practice.
• Security and policy testing (OPA/Kyverno), tenant-isolation validation, or pen-test remediation.
• CNCF open-source contributions.