Using Docker in Heroku: Understanding the Tradeoffs

The following post is technical in nature and assumes familiarity with Docker and containers.

Summary

Heroku recently announced their container runtime, which enables deployment of docker images to Heroku. The overall experience is on par in terms of features and ease with the traditional approach, which uses the slug compiler. It enables greater flexibility and control of your software system, but you assume the responsibility to maintain your system and keep it secure. This post discusses the tradeoffs and reasons to favor one approach over the other.

Background

At RoleModel, Heroku is our default choice of platform. We’ve found that it consistently saves us significant time in setting up, deploying, and managing the infrastructure that our apps run on. In our mission to deliver value to our customers via software, we’ve found it to be a net win in most cases: the cost of the platform is less than the cost of our time managing infrastructure with a cheaper provider like AWS.
We are also increasingly taking advantage of Docker in our projects, as it affords good isolation of development environments without sacrificing performance. Similarly, Docker Compose enables us to specify the entire software system context of one of our apps, making it easy for a developer to spin up on a new project and switch between projects, even those with different versions of tools such as Ruby, PostgreSQL, or Node.js.


When Heroku announced their container runtime in October, 2017, we wanted to take a closer look to see if this offering might be an excellent fusion between Docker and Heroku that would deliver value to our customers faster.

Heroku’s container runtime

The standard deployment to Heroku involves a snapshot of the code in a repository at a given commit. There are two approaches. First, you can use git to push a git commit to a git remote that Heroku owns. Second, you can use Heroku pipelines and Heroku review apps, which are associated 1-to-1 with GitHub pull requests and are aware of new and updated pull requests through GitHub integrations. In either case, Heroku learns of a commit and invokes the slug compiler to create a “slug” for the code at that commit. A slug is analogous to an image in Docker: it contains a static snapshot of everything that will be needed to run the app. The slug compiler identifies which language and framework is being used and invokes the appropriate buildpack to fetch language libraries and package everything together.

Heroku’s new container runtime offers an alternative: you tell Heroku which Dockerfile in your repo to use for build instructions. This approach will be familiar to Docker users, because Dockerfiles are the same mechanism Docker itself uses to build images. Heroku can then build and deploy your docker image instead of a slug.


Note that we are using the container runtime in conjunction with the Heroku build manifest (i.e. heroku.yml). The build manifest, which is currently in developer preview, is what enables Heroku to build and deploy the docker image itself. Without it, many features such as pipelines, review apps, and the release phase are not available with Docker-based deployments.

Comparison

We were interested to know how the container runtime compares with the more traditional approach of the slug compiler and language specific buildpacks.

Similarities

In many ways, the two approaches are identical. In both cases,you can see that “developer experience” or “DX” is a strong value to Heroku. Here is a list of things I tried that behave the same way, modulo minor differences like whether to specify a configuration setting in Procfile or heroku.yml:

  • The Heroku CLI, including: heroku open to open the deployed app in a browser; heroku logs, which are as helpful as ever; and heroku run bash to run a shell in a production environment isolated from the “dyno formation” of running servers, for things like database migrations or access to a Rails console.
  • Heroku addons, including Heroku Postgres, HoneyBadger, and New Relic, all of which we specifically tested.
  • The Heroku dashboard, including configuration and environment variables.
  • Pipelines, for a sequence of related apps from staging to production.
  • Review apps, which are automatically provisioned and track 1-to-1 with GitHub pull requests.
  • The Release phase, which enables things like database migrations to run whenever a new release is created for your app. (The postdeploy steps in app.json work too, although this is not recommended because the postdeploy steps will not be re-run in a review app when new commits are added to the corresponding pull request.)
  • GitHub integration, so that review apps in a pipeline can be automatically provisioned and the production code can be continuously deployed from a branch.

Differences

When comparing the container runtime and the slug compiler, the similarities far outweigh the differences. Nevertheless, it’s worth mentioning the differences too:

  • Assuming you want Heroku to assemble your code into something runnable, instead of the traditional Procfile, you use the new build manifest at heroku.yml, which is structured a bit differently.

Note: The build manifest documentation says that you can specify commands to run before or after compilation with pre and post keys under the build key. However, as of March 29, 2018, these commands are silently ignored when using a build -> docker key to specify docker-based builds. This behavior is not presently documented, so I wanted to mention it here.

  • It’s possible with the container runtime to deploy code without creating a git commit by saying heroku container:push.
  • One feature that the container runtime doesn’t currently have, even when combined with the build manifest, is pipeline promotions (we don’t often use this feature, so it is not considered a great loss). I suspect Heroku is working to make them available to container-based deployments too, however, so this may change in time.

Analysis of the container runtime

Advantages

The container runtime has two advantages over the traditional slug compiler and language buildpacks. First, if you’re already using docker in a development environment, you can achieve better parity between the development and production environments. Second, you can have better control over how your software is built.

However, the quality of the standard Heroku platform has eroded some of these advantages. Parity between development and production environments is valuable because it can avoid problems that only surface when deploying code to production. But such issues almost never happen with Heroku because the platform is designed and engineered well to avoid them. Greater control over the build process is not something we’ve felt a need for when using Heroku. The official buildpacks work sufficiently well that we have never needed to debug or customize them. If you have felt this pain and wrestled with Heroku to get your app deployed then these advantages might be compelling.

Disadvantages

The primary disadvantage of a container runtime is the classic tradeoff between increased control and increased responsibility. When you use the traditional slug compiler and default Heroku stacks, Heroku is responsible for noticing relevant security updates, upgrading their stacks, and migrating your app to the upgraded stack. When you use the container runtime, Heroku cannot do these things for you. The responsibility falls to you as the application maintainer.

For short-lived apps or apps where security is not important, this is not a problem. But at RoleModel many of our apps don’t meet these criteria, and we actively benefit from Heroku’s vigilance.

Recommendation

As we at RoleModel have learned from the Dreyfus Model of Skills Acquisition, the best course of action depends on the context and the goals. So, while the container runtime might not be a great fit for many of our projects, perhaps your situation is different. Consider using the container runtime when:

  1. you have experienced problems or anticipate problems related to the disparity between your local development environments and the production Heroku environment;
  2. you believe that there is enough value in available Docker images to forgo the conveniences of Heroku’s stacks and buildpacks; or
  3. you spend time or anticipate spending time wrestling with the constraints of Heroku, for example in one of these ways: (a) you use an unusual language that has no officially supported buildpack, like Elixir (although note that there are many third-party buildpacks, including one for Elixir); (b) you need Ubuntu packages beyond what’s already installed on the official stacks (although note that the heroku.yml build manifest and other approaches can also solve this problem); (c) you use a combination of languages in your app (and you don’t want to use multiple buildpacks); or (d) you need to use a non-Ubuntu flavor of Linux for some reason.

However, if you choose the container runtime, remember that you must assume the associated maintenance responsibility. For this reason, if you don’t have a compelling use case for the container runtime, we recommend the traditional stack as a default.

Have you had any experience with using Docker on Heroku? We’d love to hear about it. Please leave a comment below.