e2e vs unit e2e vs clean unit tests

All images in this post were created by Ratko Torma

Introduction

Recently I came across the concept of unit e2e testing.

You probably already know what e2e tests are, but what are unit e2e tests? They are almost the same as e2e but with a few slight differences that make them less accurate and reliable.

The e2e test takes into account all layers of the application and can run against any environment, including production. However, unit e2e tests start 1 - 2 layers closer to DB, and run in "build" dependent environments such as development, test, pre-production or just faked environments because they are invoked by unit framework/s.

They work on the same principle:

1. Send request
2. Assert response/result
   OR
3. Send a second request to get the data stored by the first request
4. Assert response/result

At first sight this looks like a good solution, but unfortunately, it is just better than nothing. If you already have e2e tests, then the unit e2e tests are unnecessary duplicates that require additional maintenance.

The happy/ideal path

Let's imagine that we have only single URL in the application.

We send a request and receive the response. If we repeat this in the same environment, using the same privileges in a single thread, then we can say we are testing the "happy" or "ideal" path.

It is like a bus transporting passengers from point A to point B on a wide private highway without traffic. After the bus arrives, a person on a bike uses the same highway, to transport a message from B back to A confirming the bus has arrived at its destination.

To ensure that all passengers have arrived at their destination, we send a second person on the bike to ask how many passengers arrived, and than we assert this information. (Note that these are steps 3 and 4 where the second request is sent to check if the data is okay.)

What happens if we invoke this same URL three times in the same environment, single thread, under the same user account?

We get nicely ordered buses heading from A to B one after the other in a single line. As they arrive, three men on bikes return to A in the same line in order to confirm that each bus has happily arrived.

Let's now see what happens if we invoke this same URL in a multi-threaded, multi-user environment with different privileges?

Even on the wide private highway without traffic, we start getting racing conditions where unexpected things start happening.

Until now we were just pretending that we live in a happy ideal world with a little bit of racing, but let's make things more realistic.

Let's imagine we have 50+ different URLs with the same and different query and post parameters, randomly accessed by 100 different users with same or different roles and permissions that can share some or most of the resources based on time and date.

Suddenly, the single happy path we tested, is replaced by many overlapping and shared paths, and crossroads where everyone is trying to get from one point to another as soon as possible.

The questions are:

• Can all requests (vehicles) came to correct destination carring expected data (passengers) and return with expected response?
• What happens in case of failure? Will it block all the traffic, or just single line?
• What happens if passengers jumps out of the window without noticing driver, will the response be OK?
• What happens if reserved hotel rooms are mistakenly occupied by other passengers?

The unexpected list can grow much further based on the quality and complexity of your application and the number of users.

In real life, testing just the happy paths can very easily put you into a situation where you have to fix bugs you can't reproduce. Not to mention deadlocks, and the worst bugs that can happen when the application is generally working okay, but under unknown conditions, stores the wrong data.

Imagine doing money transactions where only sometimes, in unknown conditions, the transaction ends with the wrong amount.

So why would you want to do unit e2e tests then? There can be a few reasons:

1. the real e2e tests are missing.
2. to cover the basics because there is no need for in-depth tests.
3. legacy or ill-written code that can't be easily tested by a clean unit test.
4. when you can't see the whole picture, and the unit e2e truly looks like the correct approach.
5. laziness.

Good old clean unit tests

To close as many gaps in the application as possible, you should cover as much of your code as possible with clean unit tests.

Compared to any type of e2e tests, clean unit tests are super fast, and not dependent on any specific environment.

Note that in the case of legacy, or ill-written code, a clean unit test can be challenging to write.

All parts of the application should be tested, including their integration, so you are as safe as as possible in any environment.

In the case of just testing that a vehicle can get from A to B in an ideal environment, you risk parts falling apart, leaving you in the middle of nowhere.

And this leads me to the old parable "For Want of a Nail".

For want of a nail the shoe was lost.
For want of a shoe the horse was lost.
For want of a horse the rider was lost.
For want of a rider the message was lost.
For want of a message the battle was lost.
For want of a battle the kingdom was lost.
And all for the want of a horseshoe nail.

Conclusion

When the difference between e2e, unit e2e, and clean unit tests are clearer, you can choose which one to use based on your needs.

My recommendation is to write clean unit tests to test the parts, and then e2e tests to test the whole system integration.

In case you are not familiar with unit testing, there are plenty of books you can learn from.
I personally recommend the book Clean Code: A Handbook of Agile Software Craftsmanship by Robert C Martin, where you can learn how to write unit tests as well as clean code.

And for the end of this article just a few points about unit tests.
Taken from wikipedia https://en.wikipedia.org/wiki/Unit_testing:

- "Ideally, each test case is independent from the others. Substitutes such as method stubs, mock objects,[5] fakes, and test harnesses can be used to assist testing a module in isolation. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended."

- "The goal of unit testing is to isolate each part of the program and show that the individual parts are correct.[1] A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits."

- "Unit testing allows the programmer to refactor code or upgrade system libraries at a later date, and make sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified. Unit tests detect changes which may break a design contract."

- "Unit testing may reduce uncertainty in the units themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts, integration testing becomes much easier"

Edited and corrected by Alex.