Habitat is Chef’s solution for application packaging and delivery: automation that travels with the application. This is the first in a multi-part series on the concepts behind Habitat.
Application Automation is a big topic and relies upon multiple services from packaging, to service discovery, to runtime supervision, and deployment topologies. As a starting point, it is important to understand what has changed with application packaging and delivery and how Habitat helps automate apps and in doing so improve the velocity of app delivery.
Why is Application Packaging Important Now?
For many years, there has been a shift to decouple applications from the infrastructure they run on. Businesses have shifted over time from physical environments, to virtual environments, to – most recently – containerized environments with the growth of Docker, CoreOS, Kubernetes and others. This shift highlights the traditional separation of concerns between Apps and OS, and allows us to question if those concerns are still valid in an automated world where the compute runtimes are treated as immutable artifacts. Or in other words, how can we decouple the application from the underlying operating system on which it runs.
Specifically, decoupling the App from the OS enables particular advantages:
- We can define an app ‘package’ that completely contains (or has references to) all the dependencies for a given app, and confidently, repeatedly deploy that app at scale with consistent results.
- We can avoid the hell of managing multiple apps (and competing dependencies) on one OS.
- We can eliminate larger parts of the OS when the application deployment and management is automated
A simple and useful illustration of how containers focus on application packaging to achieve decoupling can be found in the Kubernetes documentation:
Similarly, Habitat solves this problem by creating reusable packages for libraries that need to be consumed by applications. These libraries can be mixed and matched depending on the needs of the given application. Of course, the operating system vendors have been shipping reusable packages of libraries for some time; however, the current method is broken from a couple of perspectives.
- OS provided libraries have a release cadence based on the vendor’s release cycle. If you require a newer version of a library, you’ll need to compile and package the library yourself.
- OS provided libraries often have tightly coupled dependencies on other parts of the operating system.
- OS provided packages do not have a declarative interface for configuring the package at deploy or run time.
These problems have been around for sometime and are starting to be addressed by the OS vendors through things such as Ubuntu SNAP packages or Software Collections in the Red Hat world. These solutions are steps in the right direction, but by committing to one of these technologies, you’re still declaring an explicit dependency to a specific operating system.
How Habitat Approaches Application Packaging
We’ve practiced this philosophy of decoupling applications from operating systems at Chef for a number of years. It is the underlying idea behind Chef’s Omnibus Application Packaging technology. Omnibus, and now Habitat, can free you from having to move at the velocity of the operating system developers. Given that some operating system vendors prefer stability over bringing in newer libraries faster, coupling to the OS becomes foolhardy.
The above diagram also illustrates another point: when the application is responsible for its dependencies, the underlying operating system can be minimized greatly. You see this in the design of newer OSes such as CoreOS, but time and time again, we are still using distributions built for the old way in running our applications inside containers.
Habitat addresses this in a few ways:
- Habitat packages work to package only the concerns of the given application or libraries which the package requires.
- Habitat packages provide an area isolated from the OS to 1) store their provided files and libraries, and 2) to run the service.
- Habitat plans provide isolation of the build dependencies and runtime dependencies. This ensures that requirements for building your application or libraries are not packaged with the code required to run your application.
New Tech, Old Approaches
As we move towards a more application centric view of the underlying libraries needed to run that application, what is required is a solution to easily help you package the application artifacts and the dependencies they require. If you take a look at the current container ecosystem around the official container images, you find that virtually all tightly couple the application runtime to the operating system by doing what we’ve all done for years, running apt-get install.
FROM debian:jessie # add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added RUN groupadd -r www-data && useradd -r --create-home -g www-data www-data
ENV HTTPD_PREFIX /usr/local/apache2 ENV PATH $HTTPD_PREFIX/bin:$PATH RUN mkdir -p "$HTTPD_PREFIX" \ && chown www-data:www-data "$HTTPD_PREFIX" WORKDIR $HTTPD_PREFIX
# install httpd runtime dependencies # https://httpd.apache.org/docs/2.4/install.html#requirements RUN apt-get update \ && apt-get install -y --no-install-recommends \ libapr1 \ libaprutil1 \ libaprutil1-ldap \ libapr1-dev \ libaprutil1-dev \ libpcre++0 \ libssl1.0.0 \ && rm -r /var/lib/apt/lists/*
What we need is a completely new way of thinking about how we build, package, and run our applications given the abilities containers give us. No longer do we need a bloated system filled with libraries intended for systems that users interact with. Rather we need a system that provides the bare bones for what an application needs to operate.
Another way to think about the problem is that Habitat provides a complete build system for your containers. Once you’ve created a package of your application, you can use the functionality provided in hab pkg export to easily export Habitat packages to a variety of different runtime environments (Docker, ACI, or Mesos). This export will provide the basic skeleton OS required to run your application, your declared dependencies. Also embedded in the container is the Habitat supervisor to run your application and provide the level of automation required to configure your application.
See this in Action
The problems Habitat is solving are varied and spread across many different concerns in your application and container lifecycle. Plans and Packages are a core component of beginning to effectively build, deploy, and manage your application. In our next blog post, we’ll give you a walk through of how to package a Node.js application with Habitat, and compare this to how you would package the same application with something such as a Dockerfile.
Get started with Habitat
This introductory tutorial for Habitat shows you how to setup your environment, create a plan, build an artifact, configure it, and re-configure it during start-up.
Continue to part two of this series.