Mastering testing with Vue.js by testing a real application and setting up CI/CD

Published in

Clockwork

13 min readFeb 26, 2019

We are going to use a real application, hacker news clone, built by Evan, I simply forked it since I don’t want changes to it breaking this article, here is the forked version. At the time of writing this article, there are no tests in this project, and that’s good for us to learn how to write tests, we are also going to create a CI/CD (mostly CI) pipeline with CircleCI so tests will run automatically for any change in our git repository.
Here are the subjects we will cover:

Unit tests with Jest.
Snapshot tests with Jest.
Setting up and understanding coverage.
Why some components are harder to test and how to write testable code.
E2E tests with TestCafe.
Recording E2E tests with TestCafe Studio.
Multi-browser testing with TestCafe.
Setting CI/CD with CircleCI.

Why should we write tests?

A lot of developers don’t like writing test, but I do love it, tests make your code more resilient and error-prone, most unwanted changes or errors can be caught easily by tests before they cause damage in production, and after setting up a CI/CD environment, it’s much easier to validate pull requests and changes.

Setting up jest

If you use the vue-cli, you can either start a new project with the jest preset, or add jest using vue add @vue/unit-jest, but since the project doesn’t have a vue-cli, we are going to configure it from scratch.

Step 1: clone the repository:

# with ssh
git clone git@github.com:liron-navon/vue-hackernews-2.0.git# or with https
it clone https://github.com/liron-navon/vue-hackernews-2.0.git# install dependencies
npm install

Step 2: add our dependencies for unit tests:

npm install --save-dev jest vue-jest jest-serializer-vue jest-transform-stub @vue/test-utils

Step 3: add a script to package.json, assign an environment variable called NODE_ENV to equal “test” using the cross-env package, and call jest.

"scripts": {
  "test:unit": "cross-env NODE_ENV=test jest",
  ... the other scripts
}

Step 4: configure jest, create a file named jest.config.js and put this inside:

Step 5: we use Babel, so we need to set up a node environment when testing, it’s important for jest, change the content of .babelrc to this:

Step 6: Add our own directory for tests:
Add a directory called tests, inside it add another called unit, and inside, a directory called components. Up to this point — it would all be ready for you if you use the cli plugin.

Unit Tests with Jest

Unit tests are simple, they are meant to test the functionality of the smallest building stones, usually called “dumb components”, in our case those are the components under /components, though they are not really dumb and do access vuex and other things that make testing them harder, but we will go through it too 😁.

In components create a file named Spinner.spec.js, and write our first test using ‘@vue/test-utils’, let’s add this test:

And run our script using “npm run test:unit”. That’s it, you should see that the test passed:

PASS  tests/unit/components/spinner.spec.js
  Spinner
    ✓ has svg (16ms)Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.41s
Ran all test suites.

Now let’s add another test for a more interesting component, like ‘/components/Comment.vue’, it uses internally the store and filters that are registered globally, so we need to import and mock those 🤔.
Now the way to mock them is fairly simple, we can create our own local vue instance, and mount the component on it with the store and router.
Let’s make a new file to set up such a thing, we can put it at /tests/unit/test-utils/mountWithPlugins.js.

Now we can use this to write a test for the Comment component, it expects the store to have specific data that describes a comment, we can mock this easily and pass some props to the component, create a file named “Comment.spec.js” and copy this test there:

Running “npm run test:unit” again should result in 4 tests passing 🎉

PASS  tests/unit/components/spinner.spec.js
PASS  tests/unit/components/Comment.spec.jsTest Suites: 2 passed, 2 total
Tests:       4 passed, 4 total
Snapshots:   0 total
Time:        3.064s
Ran all test suites.

We can use this technique to test every component in the components directory, unfortunately, since Evan uses firebase with some added black magic — it makes unit testing the views very difficult 😓, we will have to mock a lot of other things to work with it, fortunately we can also write tests that do real interactions and we don’t have to mock anything for them! (E2E tests) although it is always good practice to have at least dev and production environment for the server and databases that we use so we won’t contaminate them with the tests data.

Snapshot testing with Jest

Snapshot tests are pretty simple, and we can do them with Jest too, Snapshot tests are testing that the structure of our component doesn’t change unexpectedly.
To make things simple, let’s create a directory called snapshots under the tests directory, and add a file named “ProgressBar.spec.js”, you can use this test:

And we can write another test for Item.vue, we can call it “Item.spec.js”, we need to pass to it a prop called Item, or it will fail to render, also it uses the filters “timeAgo” and “host” so we can reuse our function “mountWithPlugins”, notice that Item can include a “time” property, I will ignore it since it will make our snapshot look different every time, and we want it to be reproducible and predictable.

See? snapshot tests are very simple and intuitive, after running “npm run test:unit” again, jest will generate a directory next to our test named __snapshots__ where it will keep our past snapshots and check that they are the same, If you wish to change the component, you must explicitly delete that directory and generate new snapshots, which can be done by running:

npm run test:unit -- -u

here is the result for running the tests again (for the first run it will also log to tell you that it created a snapshot).

PASS  tests/unit/components/spinner.spec.js
PASS  tests/snapshots/components/ProgressBar.spec.js
PASS  tests/snapshots/components/Item.spec.js
PASS  tests/unit/components/Comment.spec.jsTest Suites: 4 passed, 4 total
Tests:       6 passed, 6 total
Snapshots:   2 passed, 2 total
Time:        3.636s
Ran all test suites.

Setting coverage

Coverage is a report that defines how much of your code is covered by the tests we have written and tells us what other tests we need to write, we will set coverage for jest only. To enable coverage for tests we can simply add to our jest.config.js a few more lines:

module.exports = {
    ....    // we should collect coverage
    collectCoverage: true,    // set a directory for coverage cache
    coverageDirectory: '<rootDir>/tests/__coverage__',    // set patterns to ignore for coverage 
    coveragePathIgnorePatterns: ['/node_modules/']
};

We are telling jest to collect a coverage report, put the coverage data and cache into a directory called __coverage__ under tests, and we can tell it to ignore some files or directories, by default jest will ignore ‘node_modules’, but I added it to show that option.

Now when we run our unit tests again with “npm run test:unit” we should see a table, it will take longer for the first time since jest will have to generate it’s cache files. Let’s take a look at the coverage information we get:

We can use this table to analyze the effect out tests made since we started on a project with no tests, it’s a nice improvement.
Jest uses Istanbul under the hood to generate this report, here is an explanation for those columns:

File: Tells us what file/directory we are looking at.
Stmts, short for Statement coverage: Has each statement in the program been executed?
Branch: Has each branch of the control structure such as if/else/elseif statements been executed?
Funcs is short for Functions: Has each function been called?
Lines: Has each executable line in the source file been executed?
Uncovered Line #s: Tells us what lines have not been covered, so we know to write tests for those too.

There are 3 colors here, green means the coverage is good, yellow means it should be improved, and red means there is little to no test coverage.

We can take a look at All files (the first line) where we have 22.56% of the lines covered, not great, but it’s better than nothing and we have just written our first tests 😎, you can improve that by writing more tests for every possible component, but this specific project has some files which are harder to test and we will cover why in the next section.

Why some components are harder to test and how to write testable code.

Not all of the components were written as equal, most of the harder components to test are in the “src/views” directory, they are harder to test since they are emitting mutations and actions to the Vuex store, which in turns run functions through firebase and does some crazy manual caching (at src/api/index.js).
In order to test those, we would need to have an easy way to mock the API, unfortunately, the way that the api.js file is written right now makes it difficult, and mocking firebase is a big hassle if we don’t know the exact data structures we expect to get from it (and that is why I always love to use a type system like typescript).
So let’s define some rules that will allow us to write more testable components, which will allow us to have better coverage and more robust code.

Separate dumb and smart components:
We can see that the authors almost did it, but not quite, the components in the components directory still have access to the $store plugin and that makes mocking data more complicated, a good practice is to keep those as stupid as they can, pass them the data they need as props and let them use those props as they please — that way we don’t have to worry about the high-level implementations when writing our tests.

Separate logic from the components and write testable functions:
The authors did a good job at separating the logic as much as possible from the components, we can look at /src/util/filters.js, let’s look at timeAgo, the (~~) operator is less common, but you can relate to it as Math.floor for now.

export function timeAgo (time) {
  const between = Date.now() / 1000 - Number(time)
  if (between < 3600) {
    return pluralize(~~(between / 60), ' minute')
  } else if (between < 86400) {
    return pluralize(~~(between / 3600), ' hour')
  } else {
    return pluralize(~~(between / 86400), ' day')
  }
}

We can see that we can accept a new date and that we get a Date.now(), variable and count from it, we can improve this function in a few ways: make it more clear (the ~~ operator is very confusing and will break over numbers larger than 2147483647 (32bit)), if we will just use the current date without a division it would break 🤷‍♀️, and we can pass an optional time variable, this would allow us to keep the function interface the same (accepting one variable), while adding functionality that will help us test it, we will ignore those “MAGIC NUMBERS” for now, but it’s a good idea to name and refactor them for clarity, here is a new function:

export function timeAgo (time, fromTime = null) {
    const between = (fromTime || Date.now()) / 1000 - Number(time)
    if (between < 3600) {
        return pluralize(Math.floor(between / 60) || 0, ' minute')
    } else if (between < 86400) {
        return pluralize(Math.floor(between / 3600) || 0, ' hour')
    } else {
        return pluralize(Math.floor(between / 86400) || 0, ' day')
    }
}

And with that it’s much easier to test it, here is a test for our timeAgo filter:

And now our coverage has increased for filters from 84.62 lines to 92.31 lines!

Mock everything!
In order for us to completely test the store, we need to mock, mocking is the process of passing fake (mocked) data instead of the real data, we would need to have a firebase mock, I will not cover this here since it’s cumbersome, but testing regular rest API’s is super easy, we will use fetch-utils, a small library I wrote myself, it will cover most of the simple cases for mocking rest API’s:

let’s say our component/function/store is using fetch like so:

fetch('/api/users')
  .then((response) => response.json())
  .then((json) => {
    this.users = json;
  });

we can mock this in one of our tests very easily like that:

import { Mocker } from 'fetch-utilities';

// setting up the mocker
const mocker = new Mocker(window.fetch);

// binds mocker.fetch to the window object
mocker.bindToWindow();

mocker.get('/api/users', [
  { name: 'john', id: 1, age: 22}
])

Now whenever our component/store/function calls fetch, the mocker will mock and return fake data, that way we don’t need to have the server running for our unit tests and if there are issues with the server, our tests will not fail, they will also be faster and safer.

E2E tests with TestCafe

End to end tests are more complicated and time-consuming then unit or snapshot tests, they are running real interactions with a live browser, or with a headless implementation.
We are going to use TestCafe for this, it is very simple to set up, simply install it:

npm install --save-dev testcafe

And add a script to our package.json:

"scripts": {
  "test:e2e": "testcafe chrome tests/e2e/**/*.e2e.js",
  ... the other scripts
}

Now we need to write our first test, create a new directory named e2e, and add a file called home.e2e.js, we are doing very short and simple tests currently since the web app doesn’t have too much functionality, we can check that the navigation works properly, each of the test cafe tests will give us a test context (t), and we can use it to make our test, we can also create ClientFunction, a function that can be run on the browser.

And with that we are done, In order for TestCafe to find our web app, we must run it in another terminal, we can do it with “npm run dev” and then running “npm run test:e2e” in another terminal will result in a chrome page opening and running the actual tests, we can change this to run in headless mode by changing the script in package.json, in headless mode we will not see the browser, and the tests will be faster.

"scripts": {
  "test:e2e": "testcafe 'chrome:headless' tests/e2e/**/*.e2e.js",
  ... the other scripts
}

The test should pass easily, you should be able to write more tests yourself now.

Recording E2E tests with TestCafe Studio

Writing E2E tests requires a certain amount of skills, but most QA Engineers don’t want to write code, or at least, want to write as little code as possible — this is where test recording comes in to play, TestCafe generated a tool for automating the process of writing the tests called TestCafe Stodio.
To use TCS (that’s what I’ll be calling it from now), we first need to download it, you can do so from here.
After downloading it, open the directory of your project and create a subdirectory in tests, called test-cafe-studio, this is where we will store our tests, then open TCS and select the directory, put the URL to our web app in place and click start recording.

A browser will open, just play around in it, click stuff like navigation, if we had any input you could have entered text and more.
Once you are happy, just exit the browser or click stop recording in TCS, you should come up with something like this:

From TCS, on the right menu, you can add browser actions, assertions, statements, etc… Once you're happy with the result, TCS will save them as .testcafe files, now you can use those test files to run the tests, we can change the command in package.json, we just need to add another glob pattern to match the .testcafe files.

"test:e2e": "testcafe 'chrome:headless' tests/e2e/**/*.e2e.js tests/test-cafe-studio/**/*.testcafe"

Now the QA and other team members can record tests, add assertions and they should work like magic! 🧙‍♂️

Multi-browser Testing

Thanks to TestCafe, this is a very short section, we simply need to change the script in package.json:

"test:e2e": "testcafe 'chrome:headless,firefox:headless' tests/e2e/**/*.e2e.js tests/test-cafe-studio/**/*.testcafe"

Notice that I can just give a comma-separated list of TestCafe supported browsers:

'safari,edge,chrome:headless,firefox:headless'

Notice that our machine must have the browsers for TestCafe to use them, so you might want multiple scripts that will run on multiple machines, some for OSX with safari and some for Windows with IE/Edge.
These tests will test the usability of the product, but if you want to also test the looks you would take screenshots and compare them across browsers.
For the next step let’s stay with ‘chrome:headless,firefox:headless’ since CircleCI gives us those with their Docker images.

CI/CD with CircleCI

CI/CD refers to Continuous Integration and Continuous Deployment,
they are usually used together to refer to a pipeline that runs tests and then deploys the code, you can read more on the exact definitions here, but I’m just going to say that CI is the part that runs our tests, and CD is the part that deploys the project to a server or wherever you want to use it, we will only focus on the CI part.

To create our CI environment we will use CircleCI, you need to go to their website, and log in with Bitbucket or GitHub, CircleCI will show you a list of your repositories, select the repository you wish to set automated tests for, and on the bottom click Follow, CircleCI will then listen to changes made to the repository and run the tests for each change.

Now the next step is to set up a configuration file for CircleCI to use, create a directory called .circleci, and inside it a file named config.yml.
Paste this definition to the config.yml, it defines what docker container CircleCI should use (I use node 11.8 since it’s the version I use locally, you might want a lower version for some projects), and what steps to make in order to test our project, look at each step’s name to understand what it does:

.circleci/config.yml

That’s it! you can now add the files, push them to the repository and see that the test pass, you should go to the jobs tab on CircleCI and see something like this: