EKS vs AKS vs GKE: I've Run Production on All Three

I’ve been running production Kubernetes workloads since 2018. Not labs, not demos, not “I followed a tutorial and deployed nginx” — actual business-critical systems with SLAs, on-call rotations, and angry stakeholders when things break.

Over the last four years, I’ve operated production clusters on all three major managed Kubernetes services: EKS at a major media conglomerate and a century-old industrial manufacturer, AKS at a Big Four consultancy, a global pharma company, the same industrial manufacturer (different division, different cloud — don’t ask), and a global marketing agency, and GKE at an AI consultancy.

Very few people have genuinely operated all three at enterprise scale. I wish that were a boast, but honestly it mostly means I’ve collected an unusually broad set of scars. Every comparison article I read is either a feature matrix copied from marketing pages or a “getting started” walkthrough that bears no resemblance to what production actually looks like. So here’s what actually hurts.

Networking: where the pain starts

Networking is where managed Kubernetes services differ most, and it’s where you’ll feel the consequences of architectural decisions made years ago by platform teams at Google, Amazon, and Microsoft.

GKE: the cleanest model

GKE’s VPC-native networking gives every pod a real, routable IP address from your VPC. No overlay network, no NAT, no surprises. Pods can talk to other GCP resources — Cloud SQL instances, Memorystore, whatever — using their pod IP directly. Firewall rules work at the pod level. Network policies work as you’d expect because the underlying network model is sane.

This isn’t an accident. Google built Kubernetes, and GKE’s networking reflects the assumptions that were baked into the design from the start. Everything just… works. I realise that’s not a very technical assessment, but after years of debugging overlay networking on other platforms, “works” is genuinely the highest praise I can give.

EKS: the IP exhaustion timebomb

EKS uses the Amazon VPC CNI plugin, which also gives pods real VPC IP addresses. In principle, this is similar to GKE’s approach. In practice, it means every pod consumes an IP address from your VPC subnet.

That sounds fine until you do the maths. Each EC2 instance pre-allocates a number of Elastic Network Interfaces (ENIs), and each ENI gets a number of secondary IPs. An m5.xlarge can run about 58 pods. An m5.large manages about 29. Those IPs come from your subnet. If you’re running a cluster with a few hundred nodes in a /24 subnet (254 usable IPs), you don’t have an IP problem — you have a “the entire VPC team needs to redesign the network” problem.

I’ve seen this at both organisations where I ran EKS. At the media conglomerate, we hit IP exhaustion in our staging environment three months in. The solution was a combination of larger subnets (which required VPC redesign), prefix delegation (which was still relatively new and had its own quirks), and a custom ENI configuration that reduced the pre-allocation. None of this is hard in isolation. All of it is tedious to retrofit.

AWS knows this is a problem. The documentation on VPC CNI custom networking is extensive, which is never a good sign — you don’t write that much documentation for something that works well out of the box.

AKS: pick your confusion

AKS gives you a choice of CNI plugins, which sounds like flexibility but in practice means confusion. As of right now, your options are:

Kubenet — the default for a long time. An overlay network where pods get IPs from a virtual address space. Simple, but pods aren’t directly routable from outside the cluster, which breaks a surprising number of enterprise networking assumptions.
Azure CNI — pods get real VPC (VNet) IPs, like EKS. Same IP exhaustion problem, arguably worse because Azure subnets in enterprise environments tend to be smaller thanks to hub-and-spoke network topologies that allocate subnets conservatively.
Azure CNI Overlay — the newer option that went GA last year. Pods get IPs from an overlay network, but the node gets a VNet IP. Best of both worlds in theory. In practice, it’s new enough that I haven’t run it at scale yet, and I’ve been bitten enough times by AKS networking that I’m cautious.

The migration path between these options is, to put it diplomatically, not straightforward. You essentially have to rebuild the cluster. Choose wisely on day one, because you’re living with it.

IAM: the identity crisis

Every cloud provider has solved “how does a pod authenticate to cloud services” differently, and the solutions range from elegant to infuriating.

GKE Workload Identity

GKE Workload Identity is the gold standard. You create a Google Service Account, bind it to a Kubernetes Service Account, and annotate the KSA. Done. The pod authenticates as the GSA. No keys, no tokens to rotate, no ambient credentials floating around. The binding is declared in Terraform (or whatever you use), it’s auditable, and it follows the principle of least privilege naturally.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  annotations:
    iam.gke.io/gcp-service-account: my-app@project-id.iam.gserviceaccount.com

That’s it. That’s the pod-side configuration. The rest is IAM bindings on the GCP side. Clean.

EKS IRSA

IAM Roles for Service Accounts (IRSA) in EKS works, but the setup is fiddly. You need to create an OIDC provider for your cluster, create an IAM role with a trust policy that references the OIDC provider and the specific service account, then annotate the Kubernetes service account with the role ARN.

The trust policy looks like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.REGION.amazonaws.com/id/CLUSTER_ID"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.REGION.amazonaws.com/id/CLUSTER_ID:sub": "system:serviceaccount:NAMESPACE:SERVICE_ACCOUNT"
        }
      }
    }
  ]
}

It’s not terrible, but every new service account needs a new role with a new trust policy, and if you get the subject claim wrong — which is easy because it’s a string with four segments — the pod silently fails to assume the role. No error, no warning, just AccessDenied from whatever AWS service you’re trying to reach. Debugging this is an exercise in patience, aws sts get-caller-identity, and quiet profanity.

AKS Pod Identity: the mess

Oh boy. AKS has had a troubled history with pod identity. The original AAD Pod Identity (v1) used a DaemonSet and a CRD-based binding system that was complex, fragile, and had well-documented race conditions where pods could briefly assume the wrong identity on startup. It was deprecated in 2022.

The replacement is Azure AD Workload Identity, which is conceptually similar to IRSA — federated credentials using OIDC. It’s better, significantly better. But it’s relatively new, and the migration from Pod Identity v1 is manual. If you’re at an enterprise that adopted AKS early, you probably have the old system, and migrating it is a project in itself.

Upgrades: where trust goes to die

Kubernetes releases a new minor version roughly every four months. Managed services trail by a bit. The question is: how much does upgrading hurt?

GKE: it mostly just happens

GKE has release channels (Rapid, Regular, Stable) that auto-upgrade your control plane and, if you opt in, your node pools. I’ve been running GKE on the Regular channel at the AI consultancy, and upgrades have been — dare I say it — boring. The control plane upgrades itself. Node pools do rolling replacements. Pods get drained and rescheduled. It works.

The “mostly” caveat is real, though. I’ve seen one auto-upgrade cause issues with a CRD that used a deprecated API version. GKE doesn’t check your workloads for compatibility before upgrading. But that’s a Kubernetes problem, not a GKE problem, and the fix was straightforward.

EKS: assemble your own upgrade

EKS upgrades the control plane for you, but that’s where the hand-holding stops. You’re responsible for upgrading:

The Amazon VPC CNI plugin
CoreDNS
kube-proxy
Any other add-ons you’ve installed

Each has its own version compatibility matrix with the cluster version. Get it wrong and things break in subtle ways. CoreDNS version mismatches, in particular, can cause intermittent DNS resolution failures that are spectacularly difficult to diagnose because they look like application bugs.

The EKS documentation has a version compatibility table that you will learn to bookmark. I’ve seen teams skip this step and spend days debugging issues that boiled down to “the VPC CNI version isn’t compatible with the new cluster version.”

Node pool upgrades are your responsibility too. You create a new node group with the updated AMI, drain the old one, delete it. Or use managed node groups, which automate some of this but still require you to trigger the upgrade and handle pod disruption budgets correctly.

AKS: hold your breath and pray

AKS upgrades are the most stressful experience of the three. The az aks upgrade command kicks off a rolling upgrade of your node pools, draining and reimaging nodes one by one (or a few at a time, depending on your surge settings).

In theory, this is fine. In practice, I’ve experienced:

Drain failures — pods with tight PodDisruptionBudgets that prevent nodes from draining, causing the upgrade to stall for hours. The Azure docs cover this, but the default timeout is generous enough that a single stuck pod can block the entire upgrade.
DNS resolution failures during the upgrade — as CoreDNS pods get rescheduled, there’s a window where DNS is degraded. If your applications don’t have DNS caching or retry logic, they fail.
Node pool version skew — if you run multiple node pools and upgrade them sequentially, you temporarily have pods running on nodes with different kubelet versions. This is technically supported (within one minor version) but it makes debugging interesting.

At the Big Four consultancy, we adopted a policy of doing AKS upgrades on Friday evenings, which tells you everything you need to know about our confidence level.

Add-ons and ecosystem: batteries included vs bring your own

GKE: the most complete experience

GKE comes with a genuinely useful set of managed add-ons: Cloud Logging and Cloud Monitoring integration out of the box, a managed ingress controller backed by Google Cloud Load Balancing, Config Connector for managing GCP resources as Kubernetes objects, and GKE-managed certificates. You can be productive on day one.

EKS: deliberately bare

EKS gives you a control plane and nodes. That’s it. You want ingress? Install and manage an AWS Load Balancer Controller. Logging? Set up Fluent Bit yourself. Monitoring? Bring your own Prometheus or pay for CloudWatch Container Insights. DNS? ExternalDNS, configured yourself.

AWS’s philosophy is clear: EKS is a building block, not a product. If you like building your own platform, this is flexibility. If you just want to run some containers, this is hours of YAML you didn’t budget for. At the media conglomerate, we spent the first three weeks just building the add-on layer before we deployed a single application workload. Three weeks.

AKS: the middle ground

AKS sits between the two. It has managed add-ons for monitoring (Azure Monitor for containers), a managed NGINX ingress controller (relatively new), and tight integration with Azure Active Directory for RBAC. It’s more opinionated than EKS but less complete than GKE. The add-ons it does have tend to be “Azure’s version of the thing” rather than the community-standard tooling, which means your operational knowledge doesn’t transfer well to other platforms.

Cost: the calculation nobody does honestly

GKE charges $0.10/hr ($72/month) for the control plane per cluster since June 2020, when the free tier ended for non-Autopilot clusters. Autopilot clusters pay per pod rather than per node, which is interesting for spiky workloads.
EKS charges $0.10/hr ($72/month) per cluster. Identical to GKE’s zonal pricing, as it happens. Compute costs are whatever EC2 costs.
AKS does not charge for the control plane. Free. But before you celebrate, the compute costs are the same as running Azure VMs, which are not cheaper than the equivalent AWS or GCP instances. The uptime SLA (99.95%) requires the paid tier, which is $0.10/hr per cluster. Sound familiar?

In practice, the control plane cost is noise. The real cost is compute, storage, and egress. And those are determined by your cloud provider pricing, not your Kubernetes service. If you’re choosing a managed Kubernetes service based on control plane pricing, you’re optimising the wrong thing.

The verdict

GKE is the best Kubernetes experience. The networking is clean, Workload Identity is elegant, upgrades are mostly painless, and the add-on ecosystem is the most complete. Google built Kubernetes, and it shows. If you’re starting greenfield and can choose your cloud, GKE is the answer.

EKS is the most flexible. It gives you the least and expects the most, but the building-block approach means you can build exactly the platform you want. If your organisation lives on AWS — and statistically, it probably does — EKS is the pragmatic choice. Just budget for the platform engineering effort.

AKS is the most frustrating. It’s not bad. It’s gotten significantly better over the last two years. But it consistently feels like it’s playing catch-up: CNI options that confuse, an identity story that’s been rewritten twice, upgrades that require courage. If your organisation is Azure-first, you’ll use AKS because that’s where your workloads live, and it’ll be fine. But “fine” is the best I can give it.

And that’s really the honest conclusion. You’ll choose your managed Kubernetes service based on where your other workloads live, not on the quality of the Kubernetes experience. The cloud provider decision was made before the Kubernetes decision, and no amount of technical comparison will change that. GKE being better won’t move your enterprise off Azure if that’s where your Active Directory, your ExpressRoute, and your three hundred other services already are.

But if someone asks me which one I’d choose, all else being equal? GKE. Every time. It’s the one where I spend the least time fighting the platform and the most time doing actual work.