Google open source tools: How they are enabling continuous delivery

Continuous delivery along with DevOps culture has pretty much changed the game with regards to software deployment. In an industry where releases were few and far in between, releasing updates can now be compared to getting coffee. Companies like Netflix and Google go through tens of thousands of deployments each day to a growing number of services, many of which have over a billion users each. The search engine giant has always been the good guy, and if there was a fair play award for the enterprise, Google would probably get it. Unlike a lot of its competitors, Google seems to have figured out that giving back to the ecosystem that you live off is what actually builds sustainability, and Google open source tools are leading the way.

With much power comes much responsibility, and Google has used that power well. As far as containers and the technology enabling them go, Google had been using Linux containers way before Docker came into the picture, and to say they’ve done a lot to make the technology available to everyone would be an understatement. Kubernetes was launched under Google leadership and is in essence a product of over 15 years of running production workloads at Google. In addition, the hottest new monitoring tool, Prometheus, also has some of its roots at Google and recently was added to the Cloud Native Computing Foundation.

Continuous delivery

Deployment of code that is spread across multiple platforms and teams can often be chaotic. The need for continuous integration and continuous delivery (CI/CD) arises from a certain level of uncertainty with microservices deployment. Continuous delivery is the process of breaking up releases into smaller and more frequent updates, as opposed to monolithic releases that have everyone losing sleep. The theory here is that smaller releases are a lot less riskier and are easier to make changes to, whether it’s to add new code or rollback to a previous version

Google open source tools
Flickr / Emran Kassim

To be successful with CD, an important thing to remember is you need to have a lot of short-lived development and testing environments to handle all the frequent code changes, and Kubernetes is perfect for this. The core principles that CD is based on is staging and preparing for every possible eventuality, automation and doing everything to avoid outages. For example, Netflix’s open source tool Chaos Monkey actively disrupts its own services to make sure that everyone is always prepared for any eventuality.

Kubernetes to Spinnaker

Kubernetes has some serious automated deployment and scaling functionality and gives you the ability to roll out new code and resize your datacenter without much hassle. An added advantage here is that there is hardly any downtime during deployment, as Kubernetes updates a fraction of pods at any given time. While performance may not be optimal, the important thing is the user experience is not disrupted.

While Kubernetes is great at what it does, deploying to multiple environments does still require some expertise. Although Kubernetes has great deployment APIs, this isn’t always enough to give you a fully automated deployment. Spinnaker fills this gap by allowing you to orchestrate multiple steps that define your deployment pipelines and customize each step separately. Netflix has extensive experience in this field since it uses a microservices architecture to power streaming video with frequent code deployment across all platforms.

Spinnaker, which is an open source, multicloud, continuous delivery platform, is especially useful for releasing updates with speed and precision. It has been the latest object of Google’s attention. Originally designed as a replacement for Netflix Asgard, Google took interest in the project around 2014 when it was about a year into development and production use at Netflix. Spinnaker was released on GitHub in November 2015 in collaboration with Google and has been making waves in the enterprise ever since.

Spinnaker was designed with cross compatibility and microservices architecture in mind. Other major partners include Microsoft and Pivotal. As of today, Spinnaker can deploy to and manage clusters simultaneously across both AWS and Google Cloud Platform with full feature compatibility across both cloud providers.

How it works

Spinnaker is currently being used in production at a number of major companies including Netflix, Waze, Target, and Cloudera. Its increasing popularity is due to the fact that you can install it locally, on-premises, or in the cloud, running either on a virtual machine or Kubernetes. It also helps streamline the deployment process by decoupling delivery pipelines from your cloud provider. It does this by breaking down pipelines into stages that can be triggered by events from Jenkins, Travis CI, Docker registries, Git, on a cron-like schedule, or even other pipelines. These stages can consist of pretty much anything you want like integration, system tests, or spinning a server group up or down, or manual approvals.

Spinnaker also integrates seamlessly with existing continuous integration workflows and currently has built in support for Google Compute Engine, Google Container Engine, Google App Engine, AWS EC2, Microsoft Azure, Kubernetes, and OpenStack.

Making the multicloud a reality

google open source toolsWhether you’re releasing to multiple clouds or just paranoid about vendor lock-in, Spinnaker helps deploy your application on a need basis. With its current architecture, the Netflix platform is built on AWS, and the obvious implication here is that Netflix has been working its way out of this relationship with AWS. This could be a rumor based on the common knowledge that Amazon has entered the video service market themselves with Amazon Prime Video, and has a history of treating rival products on its online shopping portal with disdain. Netflix maintains that the main motive behind open sourcing this project is for recruitment purposes and to get in touch with the top talent in the industry. It’s something that a lot of companies have admitted to as there’s no better way to test developers than have them work on your project for free.

As more and more products and services evolve with software, the cloud is quickly becoming a new-age marketplace for every bit and bob that goes into making a fully functional stack. Interoperability here is key, and though DevOps teams may not necessarily want to change from one cloud to another, they sleep better at night knowing they can if they wanted to. Additionally, a multicloud environment is also extremely useful for maintaining costs and boosting availability by being closer to the user — and Spinnaker makes this possible.

Spinnaker 1.0

With the recent release of Spinnaker 1.0, a new CLI tool called Halyard has been introduced to help admins install, configure and upgrade a production-ready instance of Spinnaker as opposed to the previous release where each microservices that made up Spinnaker had to be managed separately. By running your deployment code through a predictable software deployment cycle that is highly customizable, Spinnaker takes away the uncertainty that previously plagued a lot of microservices deployments. As of now, Spinnaker is servicing over 97 percent of the internal deployments at Netflix, and with its close compatibility with Kubernetes, there’s no surprise that Google was all over it before anyone else.

As the mass migration to continuous delivery is underway, we need all the help we can get from trailblazers like Google. With Kubernetes, and now Spinnaker, Google open source tools are playing their part by making significant contributions to the open source community and DevOps teams everywhere.

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top