API end to end testing with Docker
Testing is a pain in general. Some don’t see the point. Some see it but think of it as an extra step slowing them down. Sometimes tests are there but very long to run or unstable. In this article you’ll see how you can engineer tests for yourself with Docker.
We want fast, meaningful and reliable tests written and maintained with minimal effort. It means tests that are useful to you as a developer on a day-to-day basis. They should boost your productivity and improve the quality of your software. Having tests because everybody says “you should have tests” is no good if it slows you down.
Let’s see how to achieve this with not that much effort.
The example we are going to test
The example will cover a simple set of CRUD endpoints for users. It’s more than enough to grasp the concept and apply to the more complex business logic of your API.
We are going to use a pretty standard environment for the API:
- A Postgres database
- A Redis cluster
- Our API will use other external APIs to do its job
Your API might need a different environment. The principles applied in this article will remain the same. You’ll use different Docker base images to run whatever component you might need.
Why Docker? And in fact Docker Compose
This section is a lot of arguments in favour or using Docker for testing. You can skip it if you want to get to the technical part right away.
The painful alternatives
To test your API in a close to production environment you have two choices. You can mock the environment at code level or run the tests on a real server with the database etc. installed. Mocking everything at code level clutters the code and configuration of our API. It is also often not very representative of how the API will behave in production. Running the thing in a real server is infrastructure heavy. It is a lot of setup and maintenance, and it does not scale. Having a shared database, you can run only 1 test at a time to ensure test runs do not interfere with each other.
Docker Compose allows us to get the best of both worlds. It creates “containerized” versions of all the external parts we use. It is mocking but on the outside of our code. Our API thinks it is in a real physical environment. Docker compose will also create an isolated network for all the containers for a given test run. This allows you to run several of them in parallel on your local computer or a CI host.
You might wonder if it isn’t overkill to go end to end tests at all with Docker compose. What about just running unit tests instead?
For the last 10 years, large monolith applications are being split into smaller services (trending towards the buzzy “micro services”). A given API component relies on more external parts (infrastructure or other APIs). As services get smaller, integration with the infrastructure becomes a bigger part of the job. You should keep a small gap between your production and your development environments. Otherwise problems will arise when going for production deploy. By definition these problems appear at the worst possible moment. They will lead to rushed fixes, drops in quality, and frustration for the team. Nobody wants that.
You might wonder if end to end tests with Docker compose run longer than traditional unit tests. Not really. You’ll see in the example below that we can easily keep the tests under 1 minute. At great benefits: the tests reflect the application behaviour in the real world. This is more valuable than knowing if your class somewhere in the middle of the app works OK or not. Also, if you don’t have any tests right now, starting from end to end gives you great benefits for little effort. You’ll know all stacks of the application work together for the most common scenarios. That’s already something! From there you can always refine a strategy to unit tests critical parts of your application.
Our first test
Let’s start with the easiest part. Our API and the Postgres database. And let’s run a simple CRUD test. Once we have that framework in place, we can add more features both to our component and to the test.
Here is our minimal API with a GET/POST to create and list users:
Here are our tests written with chai. The tests create a new user and fetch it back. You can see that the tests are not coupled in any way with the code of our API. The
SERVER_URL variable specifies the endpoint to test. It can be a local or a remote environment.
Good. Now to test our API let’s define a Docker compose environment. A file called
docker-compose.yml will describe the containers Docker needs to run.
So what do we have here. There are 3 containers:
- db spins up a fresh instance of PostgreSQL. We use the public Postgres image from Docker Hub. We set the database username and password. We tell Docker to expose the port 5432 the database will listen to so other containers can connect
- myapp is the container that will run our API. The
buildcommand tells Docker to actually build the container image from our source. The rest is like the db container: environment variables and ports
- myapp-tests is the container that will execute our tests. It will use the same image that myapp because the code will already be there so there is not need to build it again. The command
node db/init.js && yarn testran on the container will initialize the database (create tables etc.) and run the tests. We use dockerize to wait for all the required servers to be up and running. The
depends_onoptions will ensure that containers start in a certain order. It does not ensure that the database inside the db container is actually ready to accept connections. Nor that our API server is already up.
The definition of the environment is like 20 lines of very easy to understand code. The only brainy part is the environment definition. User names, passwords and URLs must be consistent so containers can actually work together.
One thing to notice is that Docker compose will set the host of the containers it creates to the name of the container. So the database won’t be available under
db:5432. The same way our API will served under
myapp:8000. There is no localhost of any kind here. This means that your API must support environment variables when it comes to environment definition. No hardcoded stuff. But that has nothing to do with Docker or this article. A configurable application is point 3 of the 12 factor app manifesto, so you should be doing it already.
The very last thing we need to tell Docker is how to actually build the container myapp. We use a Dockerfile like below. The content is specific to your tech stack but the idea is to bundle your API into a runnable server.
The example below for our Node API installs Dockerize, installs the API dependencies and copies the code of the API inside the container (the server is written in raw JS so no need to compile it).
Typically from the line
WORKDIR ~/app and below you would run commands that would build your application.
And here is the command we use to run the tests:
This command will tell Docker compose to spin up the components defined in our
docker-compose.yml file. The
--build flag will trigger the build of the myapp container by executing the content of the
Dockerfile above. The
--abort-on-container-exit will tell Docker compose to shutdown the environment as soon as one container exits. That works well since the only component meant to exit is the test container
myapp-tests after the tests are executed. Cherry on the cake, the docker-compose command will exit with the same exit code as the container that triggered the exit. It means that we can check if the tests succeeded or not from the command line. This is very useful for automated builds in a CI environment.
Isn’t that the perfect test setup?
The full example is here on GitHub. You can clone the repository and run the docker compose command:
Of course you need Docker installed. Docker has the troublesome tendency of forcing you to sign up for an account just to download the thing. But you actually don’t have to. Go to the release notes (link for Windows and link for Mac) and download not the latest version but the one right before. This is a direct download link.
The very first run of the tests will be longer than usual. It is because Docker will have to download the base images for your containers and cache a few things. Next runs will be much faster.
Logs from the run will look as below. You can see that Docker is cool enough to put logs from all the components on the same timeline. This is very handy when looking for errors.
We can see that db is the container that initializes the longest. Makes sense. Once its done the tests start. The total runtime on my laptop is 16 seconds. Compared to the 880ms used to actually execute the tests it is a lot. In practice tests that run under 1 minute are gold as it is almost immediate feedback. The 15’ish seconds overhead are a buy in time that will be constant as you add more tests. You could add hundreds of tests and still keep execution time under 1 minute.
Voila! We have our test framework up and running. In a real world project the next steps would be to enhance functional coverage of your API with more tests. Let’s consider CRUD operations covered. It’s time to add more elements to our test environment.
Adding a Redis cluster
Let’s add another element to our API environment to understand what it takes. Spoiler alert: it’s not much.
Let us imagine that our API keeps user sessions in a Redis cluster. If you wonder why we would do that, imagine 100 instances of your API in production. Users hit one or another server based on round robin load balancing. Every request needs to be authenticated. It requires user profile data to check for privileges and other application specific business logic. One way to go is to make a round trip to the database to fetch the data every time you need it, but that is not very efficient. Using an in memory database cluster makes the data available across all servers for the cost of a local variable read.
This is how you enhance your Docker compose test environment with an additional service. Let’s add a Redis cluster from the official Docker image (I’ve only kept the new parts of the file):
You can see it’s not much. We added a new container called
redis. It uses the official minimal redis image called
redis:alpine. We added Redis host and port configuration to our API container. And we’ve made tests wait for it as well as the other containers before executing the tests.
Let’s modify our application to actually use the Redis cluster:
Let’s now change our tests to check that the Redis cluster is populated with the right data. That’s why the myapp-tests container also gets the Redis host and port configuration in
See how easy this was. You can build a complex environment for your tests like you assemble Lego bricks.
We can see another benefit of this kind of containerized full environment testing. The tests can actually look into the environments components. Our tests can not only check that our API returns the proper response codes and data. We can check that data in the Redis cluster have the proper values. We could also check the database content.
Adding API mocks
A common element for API components is to call other API components.
Let’s say our API needs to check for spammy user emails when creating a user. The check is done using a third party service:
Now we have a problem for testing anything. We can’t create any users if the API to detect spammy emails is not available . Modifying our API to bypass this step in test mode is a dangerous cluttering of the code.
Even if we could use the real third party service, we don’t want to do that. As a general rule our tests should not depend on external infrastructure. First of all, because you will probably run your tests a lot as part of your CI process. It’s not that cool to consume another production API for this purpose. Second of all the API might be temporarily down, failing your tests for the wrong reasons.
The right solution is to mock the external APIs in our tests.
No need for any fancy framework. We’ll build a generic mock in vanilla JS in ~20 lines of code. This will give us the opportunity to control what the API will return to our component. It allows to test error scenarios.
Now let’s enhance our tests.
The tests now checks that the external API has been hit with the proper data during the call to our API.
We can also add other tests checking how our API behaves based on the external API response codes:
How you handle errors from third party APIs in your application is of course up to you. But you get the point.
To run these tests we need to tell the container myapp what is the base URL of the third party service:
Conclusion and a few other thoughts
Hopefully this article gave you a taste of what Docker compose can do for you when it comes to API testing. The full example is here on GitHub.
Using Docker compose makes tests run fast in an environment close to production. It requires no adaptations to your component code. The only requirement is to support environment variables driven configuration.
The component logic in this example is very simple but the principles apply to any API. Your tests will just be longer or more complex. They also apply to any tech stack that can be put inside a container (that’s all of them). And once you are there you are one step away from deploying your containers to production if need be.
If you have no tests right now this is how I recommend you should start. End to end testing with Docker compose. It is so simple you could have a first test running in a few hours. Feel free to reach out to me if you have questions or need advice. I’d be happy to help.
I hope you enjoyed this article and will start testing your APIs with Docker Compose. Once you have the tests ready you can run them out of the box on our continuous integration platform Fire CI.
One last idea to succeed with automated testing
When it comes to maintaining large test suites, the most important feature is that tests are easy to read and understand. This is key to motivate your team to keep the tests up to date. Complex tests frameworks are unlikely to be properly used in the long run. Regardless of the stack for your API, you might want to consider using chai/mocha to write tests for it. It might seem unusual to have different stacks for runtime code and test code, but if it gets the job done … As you can see from the examples in this article, testing a REST API with chai/mocha is as simple as it gets. The learning curve is close to zero. So if you have no tests at all and have a REST API to test written in Java, Python, RoR, .NET or whatever other stack, you might consider giving chai/mocha a try.
If you wonder how to get start with continuous integration at all, I have written a broader guide about it. Here it is: How to get started with Continuous Integration.
Published on by