In 2017, the container craze, led by Docker, got even crazier. The Docker ecosystem has become a really wide term and defines all kinds of software, so we should probably start with the most popular member, Kubernetes. The fact that the most popular member isn’t Docker is something even Docker itself has finally resigned itself to. Kubernetes has cemented its lead over Swarm, and according to container visibility biz SysDig, out of those using orchestration with Docker containers, 43 percent favored Kubernetes, 9 percent choose Mesos, and 7 percent opted for Swarm. The popularity of Kubernetes can also be demonstrated by the fact that it has more than 27,000 GitHub stars compared to about 4,800 for Swarm and 3,400 for Mesos. Let’s take a look at the year just ended and some of the biggest events in the Docker ecosystem.
Kubernetes on Docker EE
Kubernetes on Docker EE is probably why it isn’t really surprising that Docker announced native support for Kubernetes at DockerCon Europe in Copenhagen on October 17. And this means the next version of Docker EE will seamlessly integrate Kubernetes into the platform. Though Docker has made it clear that no one is giving up on Swarm or its development, by offering native Kubernetes support for the first time, it is acknowledging that people are using it in “unignorable” numbers. To make it look like this is was all part of the plan and Kubernetes is just part of the stack, the new architecture will let you select an orchestration runtime.
Non-Googlers head a Kubernetes release
Kubernetes 1.6 was the first time a release team was primarily staffed by non-Googlers and was a milestone in terms of Kubernetes being a true community project and not a tool with Google’s vested interest buried inside. This also shows the strength and commitment of the community to be able to take over from Google and actually head a release themselves.
Kubernetes as a Service
The fact that Kubernetes is a success in spite of it being really hard to use just goes to show the lengths enterprise customers will go to avoid using a proprietary product like Swarm. In fact, they would rather pay someone to make Kubernetes easier for them than pay for something proprietary, and this is demonstrated by the rise of an entire industry, which is coming to be to known as “Kubernetes as a Service.” Some examples are CoreOs, Platform9, Kismatic, Heptio, Diamanti, and Hypernetes.
We also can’t possibly talk about Kubernetes and not talk about its home, the Cloud Native Computing Foundation (CNCF). Apart from the initial two projects Kubernetes and rkt, the CNCF has since taken on an additional 10, the most recent two being the Envoy service mesh and the Jaeger microservice debugging software. Jaeger is a tracing tool developed by Uber for distributed applications and is currently used to manage over 1,200 individual microservices. Jaeger can be used to track problems across different services, each of which may have multiple instances operating at any given time.
Envoy supplies and manages external connectivity across different services and is an API-driven platform for multiple services to discover and communicate through each other. Envoy was originally developed at Lyft and heavily contributed to by both Google and IBM. It is also yet another service mesh to be adopted by the CNCF, which looks like it just can’t get enough service meshes.
The CNCF adds new big-name members
On Oct. 23, the CNCF announced that a total of 30 new members joined the Foundation including Azure, AWS, and Oracle. With literally the who’s who of enterprise technology all on board together working on open source projects, it’s no wonder licenses for traditional server-based software is dipping.
Prometheus emerges as the monitoring favorite
Containers have basically dragged the enterprise into the next level of software development, and a lot supporting teams have had to play catch up, container security and monitoring teams in particular. It’s probably been emphasized enough how different monitoring and securing microservices are compared to traditional applications and how a much more “active” approach is needed.
Prometheus (another CNCF project) has emerged as the leading monitoring tool. In a recent report on a survey of 470 container users, Prometheus was by far the most cited tool for monitoring Kubernetes clusters. In fact, over 63 percent said they monitor their clusters using Prometheus, though almost 80 percent said they use at least one other tool as well.
Twistlock and Calico step up networking and security
Networking has changed, too, and we have policy-based network security like Project Calico that creates little firewalls around each workload. These “microfirewalls” are probably the best chance your containers have against attackers and they protect each workload individually so there’s no breaching dam effect.
Calico also integrates with CoreOS’s open source container networking project called Flannel, and what you get as a result is a double-edged sword called Canal with both modern day networking and security. Additionally, Twistlock has just announced a bunch of new updates for version 2.2, which build on its shift to the Go programming language and enhances security for your microservice infrastructure.
Among the new features in the Twistlock 2.2 update is a Cloud Native Network Firewall (CNNF), which is basically a real-time model of all your containers communicating with each other in deployment. The model includes insight into which network ports and cache services are in use and which external services are connected.
The whale in the Docker ecosystem
Let’s not do a Docker ecosystem recap and forget about Docker, although talking about the Moby Project as a year highlight leaves a lot to be desired. Apart from a change in leadership and the addition of Steve Singh as the dynamic new CEO of Docker, it does look like Docker is distancing itself from its open source project almost out of bitterness, perhaps because its child won’t listen to it anymore.
The good news, however, is that in spite of Kubernetes’ success, every tool in the Docker ecosystem is expendable, and with Docker’s latest update you can choose your orchestration engine at setup. Nevertheless, Singh is always extremely confident and calm during all interviews always makes a point to remind everyone that the surface hasn’t even been scratched yet with regards to the cloud’s potential.
What he probably means is that once the cloud has fully matured and is an integral part of every company’s infrastructure, the demand for orchestration will be way more than Kubernetes, Swarm, and Mesos combined can handle. That’s also probably why all the big cloud giants are investing so massively in new data centers to support this Docker ecosystem that is the hottest ecosystem on the planet right now.
It’s also not just about the size of this ecosystem, but the speed at which it is expanding as well. That the speed continues to grow and develop to include more and more aspects of the workflow is mind-boggling and easily the quickest adoption anyone alive has witnessed. In the end, it’s still almost tragic that most of the action in the container space today is outside of Docker. But we can’t count out Docker just yet. After all, it is still called the Docker ecosystem.
Photo credit: Flickr / Glen.
