14 November 2024

3 min Reading Time

TL;DR

  • Kubernetes is the standard for container orchestration – but not always the best choice.
  • Docker Swarm delivers 80% of the features at 20% of the complexity, ideal for small to mid-sized teams.
  • HashiCorp Nomad orchestrates containers, VMs, and batch jobs on a single platform.
  • AWS ECS and Google Cloud Run eliminate cluster management entirely as managed services.
  • The right choice depends on team size, workload type, and operational capacity – not hype.

Kubernetes has won the container orchestration market. But market dominance doesn’t mean it’s the right fit for every use case. For a team of five developers running ten services, Kubernetes is often overkill – with a learning curve that consumes weeks and operational complexity that ties up half an ops team. There are alternatives.

The Kubernetes Dilemma: Powerful, But Complex

Kubernetes can do almost anything: rolling deployments, auto-scaling, service discovery, secret management, network policies, custom resource definitions. This power comes at a price. A production-grade Kubernetes cluster requires deep understanding of networking (CNI plugins, ingress controllers), storage (CSI, PersistentVolumes), security (RBAC, NetworkPolicies, PodSecurityStandards), and monitoring (Prometheus, Grafana, Alertmanager).

For a platform team with three or more Kubernetes-experienced engineers, this is manageable. For a small team that also needs to ship features, it becomes a resource drain. The question shouldn’t be “Can we run Kubernetes?” but rather “Do we need Kubernetes?”

80%
of the features at 20% of the complexity for small to mid-sized teams.
20%
of the complexity for small to mid-sized teams. HashiCorp

Docker Swarm: The Pragmatic Alternative

Docker Swarm is built into Docker, requires no separate tooling, and can be set up in 15 minutes. A Swarm cluster supports rolling updates, service discovery, load balancing, and secret management – the core features that 80% of teams actually need.

Limitations include no native auto-scaling (requires manual scripts), no built-in ingress controller (workarounds like Traefik or Nginx are needed), and a shrinking community and ecosystem. Docker has effectively discontinued Swarm development – it still works, but is no longer actively evolved.

Best for: Teams running 3-20 services that need container orchestration without justifying Kubernetes-level complexity. Often used as a stepping stone before eventually migrating to Kubernetes.

HashiCorp Nomad: The Generalist

Nomad orchestrates not just containers, but also VMs, Java applications, and batch jobs. This makes it the only alternative that natively supports heterogeneous workloads. Its architecture is simpler than Kubernetes: a Nomad cluster consists of servers and clients – no etcd clusters, no API server configuration, no CNI plugins.

Nomad integrates seamlessly with the HashiCorp ecosystem: Consul for service discovery, Vault for secret management, and Terraform for infrastructure provisioning. For organizations already using HashiCorp tools, Nomad is the natural fit.

The trade-off: a smaller ecosystem than Kubernetes. Fewer third-party integrations, limited community support, and fewer engineers with Nomad experience in the job market.

Managed Services: No Cluster, No Problem

AWS ECS (Elastic Container Service) is Amazon’s native container orchestrator – fully integrated into the AWS ecosystem and simpler than EKS (Kubernetes). With Fargate as the compute backend, cluster management disappears entirely: you define tasks and services, and AWS handles the infrastructure.

Google Cloud Run takes it further: deploy containers with a single click, automatic scaling (including down to zero), and pay-per-request pricing. For HTTP-based workloads without complex service-to-service communication, Cloud Run is the simplest option on the market.

Azure Container Apps positions itself similarly to Cloud Run: serverless containers with KEDA-based auto-scaling and Dapr integration for common microservices patterns.

Decision Matrix: Which Tool Fits Which Context

Kubernetes if: > 50 Services, Multi-Cloud Requirement, large platform team, complex networking, custom controllers or operators required.

Docker Swarm if: Small team (< 5 devs), < 20 services, need quick setup, on-premises deployment, no Kubernetes expertise.

Nomad if: Heterogeneous workloads (containers + VMs + batch), HashiCorp stack already in use, want enterprise-grade simplicity.

ECS/Cloud Run if: AWS/GCP-native environment, team wants to focus on code instead of infrastructure, HTTP-based services, no multi-cloud requirement.

Frequently Asked Questions

Is Docker Swarm dead?

Not dead, but in maintenance mode. Docker now focuses on Docker Desktop and Docker Hub. Swarm remains stable and functional, but receives no new features. That’s fine for existing deployments – but for new projects, consider its long-term viability.

Can you migrate from Docker Swarm to Kubernetes?

Yes – and it’s easier than migrating from VM-based setups. Your container images remain identical. Docker Compose files can be converted to Kubernetes manifests using Kompose. The main migration effort lies in networking, storage, and deployment configuration.

Is ECS cheaper than EKS?

ECS itself is free – you only pay for compute resources (EC2 or Fargate). EKS incurs a $0.10/hour cluster fee (~$73/month) plus compute costs. For small-scale deployments, ECS is more cost-effective. At scale, the difference becomes negligible.

Why doesn’t everyone use Google Cloud Run?

Cloud Run excels for request-driven workloads like APIs and websites. For workloads requiring persistent connections (WebSockets, streaming), sophisticated service mesh capabilities, or highly specific networking requirements, Kubernetes remains the better choice. Cloud Run abstracts away too much for those use cases.

Do we even need container orchestration?

Not necessarily. For a handful of services, a simple server with Docker Compose and a reverse proxy often suffices. Container orchestration becomes valuable once you’re managing 10+ services and require rolling deployments, auto-scaling, and service discovery.

Header Image Source: Pexels / Jan van der Wolf

Also available in

A magazine by Evernine Media GmbH