Making Passenger: How we’ve recently upgraded our development environments

On the surface, Passenger is an easy to use, customer-facing digital product. Look more deeply and you’ll find that the system comprises a number of different technologies and databases. It is the combination of these, working as one beneath Passenger’s app and web surface, that makes the end-product a joy to use.

We recently invested in Docker to help us build, manage and secure each Passenger application in a more efficient way. Read on to better understand the behind the scenes work we’re doing to make Passenger a stronger, more reliable solution for all that use it.

Why we moved to Docker

Passenger’s numerous development environments were previously built using Vagrant – a tool for building and managing virtual machines (VMs) environments in a single workflow – and Ansible – IT automation software that turns Vagrant’s VMs into configurable resources.

“VMs” are emulations of computer systems that provide the functionality of a physical computer. Each VM runs a Passenger system, providing separation between Passenger’s different development environments. This is great, with the exception of being extremely resource intensive.

To put this into context, Vagrant creates a new VM for each back-end service that comprises Passenger, which requires fixed CPU, disk and memory to be allocated. These “full-fat” VMs with fixed resource allocation can be a time sink. If we ever wanted to rebuild the whole Passenger stack (the tools and applications that make up Passenger’s back-end) the process could take up to an hour – not to mention the expensive hardware required to run the process.

When developing, full requests – depending on the complexity – can also take 5-20 seconds to compute. That can feel an eternity in development time.

When we increased the amount of people working on the Passenger stack, it quickly became apparent that this development environment was no longer the fit-for-purpose solution it once was. Technology and common practice have shifted towards containerised environments.

Containing the problem

Application containerisation is another method of “sandboxing” applications. It can deploy and run Passenger’s distributed applications within a stack without launching an entire VM for each app. As these “containers” use dynamic resource allocation, they do not require an up-front allocation of disk, CPU or memory. With no virtualisation required, containers are a more native approach – which means increased portability, scalability and security.

Given these benefits, we’ve implemented the best solution for implementing Passenger’s newly containerised environments available today: Docker. Docker can bring a container into a fully functioning development environment with far fewer lines of configuration than the Vagrant/Ansible solution – with the handy added benefit of keeping everything in a single file, which makes the whole application environment visible at a glance.

We’ve also adopted Docker Compose, a tool that enables us to further define how Passenger’s multiple application containers can come together to build an overall service.

What does this mean for the product?

Shifting our development environment from one solution to another was a big task, but it’s come with huge benefits for the Passenger product. For instance…

• We can now rebuild our entire development stack in ~8 minutes rather than 1 hour
• Our development machines have significantly more available CPU, memory and disk space:
  • We have gone from 8GB of memory used to < 1GB
  • …and from ~100GB of disk space to < 8GB
• Requests with full debugging have reduced from 20-2 seconds to 800-125 milliseconds

With these reductions, the Passenger engineering team can iterate faster, address bugs at speed and spend more time working on new features and ideas that improve the product.

Keep an eye on the blog for more updates on how we’re improving Passenger behind the curtain. If you’d like to know more, please get in touch with the Passenger team or sign up to our newsletter.