Posted on

An UberService to test them all

One of Elon’s tweets brought back an interesting case I handled a few years back where we squeezed about 15 services into 1.

The issue

Part of building a healthy codebase from day one, is making sure that you have good test coverage. In fact, you might become a victim of your own success, which is what happened to us: Early on, we built such a friendly and streamlined integration testing framework, that our engineers used it extensively, way more than we should have. We wrote tests at the scope of either unit tests or ITs. This had two effects on the system:

  1. Quality was excellent! – Practically anything deployed to production had full test coverage and very few bugs.
  2. Productivity was coming to a halt – Our friendly IT testing framework was causing everyone to write tests that took too long to run, required most services to be up and running, and caused your laptop to glow in the dark.

Keep in mind that we were an early-stage startup; we didn’t have a dedicated Devops team to handle Devex. At this point in time, developers were still responsible for doing Devops work.

To solve this issue, we quickly realized that we had to cut the habit of constantly adding new ITs, and move to component testing – tests that only required a single service to be up for the test to execute.

We needed to migrate to component tests instead of our IT testing framework. But in the meantime, dev machines where still wheezing from running all those services locally, and we couldn’t delay new services from coming online, just because we hadn’t yet made the move to component tests.

Let’s try something

We needed to buy some time. Was it possible to reduce the footprint required for running ITs on dev machines?

Each JVM-based service required a few hundred megabytes of memory. Multiply that by 15 and you see the issue. We quickly ran out of memory on our machines.

What if we could “squish” the services into a single process? This is the experiment I tried out.

Technical background

Let’s get a few technical details out of the way first:

Our services were based on Dropwizard, which is a group of technologies that work well together. As far as the web server, it’s an embedded Jetty server. Each service is an instance of class Service which inherits io.dropwizard.Application and has a main() function that calls a run() command.

The solution

What if we could bundle all of these run() commands into a single main() function? Would they run well together?

Turns out, that multiple Jetty servers can work well within a single JVM process! All we had to do was a bit of deduplication of ports and resources:

  • We had to make sure that each service listened to a different port
  • Resources, such as upgrade scripts that sat in the same subdirectory in all services had to move to a uniquely named subdirectory

The code ended up looking something like this:

class UberService {
  private val allServices by lazy {
    mapOf(
      "foo" to FooService(),
      "bar" to BarService(),
      ...
    )
  }

  fun startAllServices(servicesToStart: List<String>) {
    // Loop on the services that we want to start and launch each of them in a separate thread
  }
}

fun main(args: Array<String>) {
  // Get list of services that we want to start from args
  include = .... args .....
  UberServer().startAllServices(include)
}

Nothing too complex going on here! Since many of the services rarely needed to work in scale on a developer machine, this single process worked well with a memory footprint of much less than 1GB.

This idea ended up buying us about 3 years of breathing room. While these days we have other, better designed solutions for testing our code such as private Kubernetes namespaces for developers, you can still run UberService on your local machine with an input list of dozens of services and it will work nicely.

So, getting back to Elon’s tweet, maybe there is a way to tell your boss that you eliminated 80% of your microservices 🙂