Test runner features

What would the world be like without a test runner? Let's say you don't know what a test runner is, and you want to code a unit test. Would that be possible? I think it would. For instance, say we have this small Cart class:

class Cart {
    constructor() {
        this._cart = [];
    }
    total() {
        return this._cart.reduce((acc, v) => acc + v.price, 0);
    };
    addToCart(item) {
        this._cart.push(item);
    };
}
module.exports = Cart;

If we want to test it, we could run some code like this:

const Cart = require('./cart.js');
const c = new Cart();
c.addToCart({ productId: 10, price: 5.5});
c.addToCart({ productId: 15, price: 6.5});
if(c.total() !== 12)
    console.error('Nooo!!!');
else
    console.log('Yes!!!!!');
 

A test is basically a piece of code testing our code. Will this work? Yes. Is this a unit test? Yes. Will this scale? Definitely not. This file will become massive and hard to maintain. Keeping track of what has failed would be an impossible task. We need a tool to help us scale and to help us keep our tests maintainable. We need a test runner.

Before exploring possible test runners, I would like to review what we would expect from a test runner. What are the features we would need in a test runner?

Easy to learn and run

We have a lot of things to learn. We need to learn Node and React; we even have to buy a book about Puppeteer. We want a test runner that is simple and easy to use.

Group tests by functionality

We want to have our tests separated by functionality, component, or workflow. Most test runners have a describe function that helps us to group tests.

Ignore tests if needed

We want to skip a test if it becomes noisy, but we don't want to remove it.

Run only one test

Being able to run only one test is extremely important while debugging. Imagine you have over 1,000 tests (yes, you are going to have over 1,000 tests). If you want to fix only one test, you wouldn't want to run all of them. You would like to run only the one you are working on.

Assertions

Assertions are essential. An assertion is an expression to check whether the program we are testing worked as expected. Do you remember my console.log and console.error to check whether the cart worked as expected? Well, Assertions are way better than that. What do we want to check with Assertions? This is a possible list:

• Whether a value is equal to a test value.
• Whether a value is null or not null.
• Whether a string or a list contains a value. We might have a huge block of text, and we only want to check whether it has some string in it, or an item in an array.
• Whether we expected something to fail, because sometimes, we would expect some piece of code to fail.

Tools to set up and clean up the environment

Before starting the tests, we need our application to be in a certain state. For instance, in the cart test, we would like to make sure that the customer has not already purchased the product before starting the test.

There are also technical setups that might need to be performed. In our case, we would need to have Puppeteer and a browser ready to be used before each test.

Another important concept is that tests should be independent and detached from each other. This means that the result of one test must not affect other tests. This is why, very often, it is required to clean up after each or all tests.

Reports

We want to see which tests passed and which tests failed. We would expect a test runner to at least show a good report in the terminal. It could be even better if we can get results in other formats, such as JSON, XML, or HTML.

There are many other features we could mention, but these are the most important features we need to know about before getting started.

Let's now see what the test runners available on the market that can cover the features we are requesting.