12 Factors of 12 Factor Apps
Podcast: Play in new window | Download (53.7MB) | Embed
Subscribe: Apple Podcasts | Spotify | Email | RSS | More
Back in the day, applications often went years between deployments. When deployments happened, it was an ugly, slow, and painful process that took a tremendous amount of time. Application management and diagnostic tools were limited and these limits were often imposed by the software itself, as most management functions were built into the app. It was even more annoying if you needed to spin up multiple copies of an application, move it to a different server, or change configuration information, as this information was scattered all over the place in a variety of formats, the application was probably dependent on locally installed tools and libraries, and the application wasn’t stateless. It was a nightmare (and still is for some of Will’s old employers).
With the advent of cloud computing, we stopped treating servers like pets and started treating them like cattle. While this made it easier to quickly spin up infrastructure as we needed it, it also meant that certain old patterns of application development had to go away. In particular, we had to rework our applications so that their configuration information was stored in a predictable location, that application state was stored in a “saner” way, and so that processing could be interrupted at any time (because you never know when an admin is going to push a new configuration that will wipe out your runtime environment). It was a bit of adjustment, and many companies are still struggling with this to some degree.
As things have gotten more complex, we’ve all had the urge to simplify our work. Not only does doing so make it easier to comprehend, it hopefully makes it easier to tell when we are doing something wrong. However, best practices always take a while to spread through the community and often are not platform agnostic at first. Thankfully, now we have some better general application guidelines that make it a lot cleaner and easier to develop cloud-native, modern applications that play nicely with modern devops tools, especially tools built around orchestration, deployment, logging, and storage.
12 factor applications are a set of best practices that make it much easier and less risky to deploy and manage applications on modern cloud infrastructure. In addition to this lowered risk, they also make it easier to deploy applications continuously, by limiting the variance between development and deployed versions of the application. Finally, they also offer industry standard ways of dealing with cross-cutting concerns, like logging, configuration, and process management with good tooling, rather than stuff you cobbled together in house.
Episode Breakdown
The Codebase
12 factor apps are always tracked in version control in a one-to-one correlation between the codebase and the app. Use shared libraries instead of multiple apps in the same codebase.
While there is only one app per codebase, there can be many deployments of the same app. Codebases are the same across all deployments, but different versions may be active in each deployment.
This makes it easier for anyone on the team to make changes and redeploy the application, without worrying that someone deployed directly from their machine.
Dependencies
Explicitly declare and isolate dependencies. Do not implicitly rely on things being installed on the system. The declaration of dependencies will be accomplished via a manifest and will use a dependency isolation tool to make sure that implicit dependencies don’t leak in from the surrounding system.
This also includes system tools, like curl and other commandline tools. If these are used, they need to be explicitly listed. This makes it easier to set up a new environment for new developers, or to determine if a vulnerability that just hit the news is a problem for your application.
Config
Configuration is strictly separated from code and is not checked into revision control systems. Note that this doesn’t include configuration that doesn’t vary between deploys (like routing information in a web app). 12 factor apps store configuration in environment variables to keep them out of the repository, make them easier to change, and keep them more OS agnostic.
In a 12 factor app, environment variables are never grouped, but instead of independently managed for each deploy. Note that many use cases for grouped configuration items are actually examples of backing services. This practice also makes it easier on SREs and devops to reason about how your application will behave.
Backing Services
A backing service is any service the application consumes over the network. While not part of the application itself, they are required for it to run. These backing services should be configured in such a way that they can easily be swapped out without changing code.
Resources can be attached and detached from deployments at will. This means that your application should not cache resource information, in order to achieve loose coupling. Allowing backing services to easily changed without touching code makes it easier for SREs and devops people to fix things without bugging you and can also facilitate failover in the event of a crash.
Build, release, run
These phases of producing usable code are strictly separated. The build phase makes the code executable, the release step combines this output with config and the run stage actually runs the code. Every release has its own release id and releases cannot be changed once created.
Builds and releases are developer-initiated, while runs can also be machine-scheduled. Breaking these steps apart and making them repeatable makes it easier to iterate on their implementation, resulting in faster deployments over time.
Processes
Applications are run as a set of one or more processes. 12 factor application processes are stateless and share-nothing. Shared state is kept in a persistent store. Above all, nothing stored in memory or on disk is assumed to be available for future work.
Note that this includes things like sticky sessions and cached assets that are output by some asset packagers. The former should be stored in persistent data store with expirations, while the latter should be composed during the build process.
Building processes in this way makes it easier to move them between machines, scale them, or kill them off if they start having issues.
Port Binding
12 factor applications don’t require an outside webserver. Instead, they are completely self-contained and export things like HTTP by binding to a port and listening to requests.
If a webserver is required as a host, it must be explicitly declared in a dependency declaration. It will then run in user space (no admin account). Note that this dependency is not injected at run time – it happens as part of the build and release cycle.This approach also means that an app can be a backing service for another app.
This approach means that the configuration for the server you are using ships with the application, rather than being something separate that varies independently from the application.
Concurrency
In a 12 factor app, processes are first class citizens. Each type of work can be assigned a different process type. Because processes are stateless and share-nothing, it’s much easier to scale horizontally.
12 factor apps rely on their host operating system’s process manager, rather than attempting to manage these things themselves (or to spin up a daemon or windows service). Boutique process management systems tend to be difficult to code, and usually aren’t necessary. This approach allows development teams to focus on the value they are trying to provide, rather than esoteric process management constructs.
Disposability
12 factor app processes are disposable, meaning that they can be spun up or shut down on very short notice. This helps with scaling and speeds up deployment. When a process is shut down, it is sent a SIGTERM signal from the process manager, is expected to service any current requests and then shut down.
This means that any work being done needs to be fairly granular if possible, and restartable if not. This tends to require some queueing infrastructure. This approach allows for rapid shutdown and spinup of new deployments, which makes it easier to devops people to manage infrastructure without noticeable downtime.
Dev/Prod Parity
12 factor apps are constructed with continuous deployment in mind. This necessitates that dev and production are as similar as possible. This also has implications for backing services, making it a bad idea to use lightweight (or different) versions of backing services in development and production, because the differences will eventually cause problems.
All deploys of the same version of the app should use the same versions of backing services to avoid this problem. This also makes development much easier, because you don’t have to think about the differences between local and production when writing code or trying to troubleshoot a production issue.
Logs
12 factor apps don’t concern themselves with routing or storage of their output stream, nor with the management of logfiles. Instead they use stdout, unbuffered. During runtime in production, this means that dealing with the output is a configuration detail, rather than an application detail.
This allows for better tooling around log management and also maintains statelessness and disposability. There are lots of very robust and powerful tools for digging through logs. Doing things this way allows your operations team to use those tools, rather than whatever you thought was “good enough” when designing the system on your local environment.
Admin Processes
Occasionally, one-off processes crop up. These should be run against a particular release and in an identical environment to the typical app environment. This code should also ship with the application code.
These scripts can be invoked in the development environment from the shell, and in the production environment using ssh or other remote command execution mechanisms. This approach means that the content of any of these processes remains available for future troubleshooting, rather than having conversations like “well, Bob ran a script on prod, but we don’t know what he did”.
Tricks of the Trade
Simplification goes beyond just the your workload. Simplifying your life can lead to higher quality living and even a happier lifestyle. This doesn’t mean moving out of your McMansion into a Tiny Home, nor does it mean going through everything you own and throwing it out if it doesn’t “bring you joy.” You can start simply simplifying your life by choosing quality over quantity. In the old days it could be described as having a small collection of your favorite DVD’s instead of a massive collection of movies you never watch. As a musician, BJ likes his guitars and has several different ones. However, he doesn’t buy every cheap guitar he likes instead choosing to wait and save for the nicer ones that he wants. Simplifying your life is basically removing the excess so even cleaning out a closet or uncluttering a junk drawer will help. It doesn’t have to be a massive change to start to live simply, start one step at a time.