Kubernetes (k8s) is wildly popular because it's the sensible next step in virtualization. Or perhaps it's not sensible, but it's definitely where a lot of things have headed. It's all about containers. So what's the difference between a container and "virtualization?" It's a bit like "fat" virtualization vs "skinny" virtualization; the difference is the hypervisor. With (full/fat) virtualization, there is a hypervisor layer between the virtual machine and the host (bare metal) OS which serves as an emulation layer allowing virtual machines of different kernels and even architecture types. For example, virtualization lets a windows machine run on a linux host. Containers, on the other hand, strip out the hypervisor layer and share the kernel with the host. Windows containers have to run on windows hosts. ARM containers have to run on ARM hosts. In that sense, it's a step backwards in flexibility, but containers are much leaner than a full virtual machine so they scale more quickly since they spin up and down in much less time than it typically takes a full (virtual) machine to boot. Containers also aren't burdened by any of the config or overhead associated with operating the virtual machine. The question of "container" vs "virtualization" is really answered by "do you want fast, or complete?" But it's also answered by "how is the application delivered?" Increasingly, that answer is kubernetes.
Alright, so great. If you're like me, you already know virtualization, and have run some containers, and want to kick the tires on kubernetes. Welp.... I hope you like reading. This is not just download an ISO and run an install sort of simplicity. Not even close! Kubernetes is a whole ecosystem unto itself, and there's plenty of variations to consider. Furthermore, kubernetes is really two layers of container clustering – the control layer and the worker/application layer, both capable of scaling load over multiple members.
There are also cool things like flatcar (recently bought by MS), which makes a thin, immutable OS layer to further streamline the container concept and get the app as close to "on metal" as it can. What's an immutable OS? A really cool concept for a container world and a fresh take on how to design the OS to best fit its purpose. Though I will say it's a bit rich that two months before MS acquired flatcar, they blogged about how CoreOS became a dead man walking after their acquisition. We'll see how well MS stewards flatcar in time, but I'm still worried "the new MS" is just out to pwn open-source by recapturing developer mindshare.
Then there's rancher, which wants to give enterprises one control layer across different cloud or premise clusters. They've also got a pretty slick looking product that, in terms of kubernetes, is almost ISO-install-easy to get up and kicking tires on, but we're talking containers, so it's docker-run rather than ISO. I was frustrated that out-of-the-box it installed a cattle orchestrator even though their docs clearly say they're k8s focused:
Rancher was originally built to work with multiple orchestrators, and it included its own orchestrator called Cattle. With the rise of Kubernetes in the marketplace, Rancher 2 exclusively deploys and manages Kubernetes clusters running anywhere, on any provider.
Did I download some previous version installer? I don't know. I thought I did their super-simple quick start, which says it's based on rancher 2, and Rancher 2.0-2.4 docs have the same orchestration disclaimer as the 2.5 docs, but somehow I ended up with cattle. Still, it was super easy and only took a couple extra steps to get a kubernetes worker hooked up to rancher, but then rancher just kicked me out to the kubernetes web UI. So what's even the point of you, rancher?!
Well, whenever I read about kubernetes from the official docs, I find myself clicking-through 4 different topics trying to tackle one thing. Want to install k8s? Sure! Easy! Wait, are you going to do that with kubeadm, kops, or kubespray? Let's choose door number 1 and go with kubeadm? OK, well... What do you want to do about HA? CGroups? Do you want to use docker, CRI-O, or container.io for the runtime? If kubernetes is one thing, it's options overload. Every question demands researching three other things before understanding how to answer it.
And that's where I'm at. Lately I've been doing everything ansible, and centralizing it in AWX, so kubespray sounds delicious. I'll just write up some inventories and run the deploy from AWX, then rehome AWX into k8s with the new AWX Operator. Right? What comes first – kubernetes or AWX? I've already got AWX pre-18, so I'll be enjoying my chicken and eggs at the same time!