The words “automated testing” elicit many kinds of emotion.
For some it evokes painful memories of blindly chasing the elusive “100% test coverage,” spending more time writing test code than application code. For others it inspires confidence, a reminder of how automated tests saved them from shipping detrimental bugs. For others still, “automated testing” inspires guilt, because you know you should make the investment, but haven’t quite found the time.
If you are in the latter group, don’t worry—we’re going to dive into automated testing, why you should care about it, and how you can get started.
First, let’s focus on testing. The truth is, everyone tests their software. From developers to customers, everyone plays a part in testing your application. At the most fundamental level, testing software is about providing some input(s), getting some output(s), and then judging whether you get the result you expect.
Every time a programmer runs their code, they’re testing whether it runs without crashing, looks right visually, and calls all the calculations/functions correctly. When a quality assurance (QA) engineer tests code they might be manually walking through a user story, such as registering a new account, and checking that the behavior is correct at every step.
Ultimately, whether they know it or not, your end users test your software too, every time they use it. And you can tell that users are testing your code because, when things go wrong, most people are not afraid to let you know.
So there’s no avoiding it, everyone everywhere is always testing software. But as developers and as a company, you get to choose how you want to test your applications.
What are computers good at? A quick search on the internet will reveal that computers are really good at running repetitive, boring tasks. What is a test if not a set of boring repetitive tasks? Like: open this url, click in the email field, type “mycoolemail@emailsRus.com”, click in the password field, type “password1234”, click the “login” button…
What are humans better at than computers? Making mistakes, of course! That includes making mistakes doing repetitive tasks. Humans may correctly complete the checklist of a formalized test once or twice, but with continued repetition, chances are that they’ll stumble along the way and ruin the test.
Computers are actually very bad at making mistakes, as long as they receive crystal clear instructions. So, if computers are good at repetitive tasks, and bad at making mistakes, they make the perfect candidates to be your primary testers!
An automated test uses code to tell a computer how to run different functionality, check the results, and let you know whether or not everything went as expected.
There are many different types of tests, all serving different purposes. One of the most advanced and powerful types of tests is an end-to-end (e2e) test, where you run your code in the same environment as your users, simulating their actions as closely as possible. In the world of web development this means running your site in an actual browser, while the computer clicks around and types things to see how everything behaves. More on this later.
At this point automated testing seems pretty attractive. So why isn’t everyone doing it?
There are several reasons a software engineer or team might not be using automated testing. For example, setting up the testing takes time, and sometimes the budget/scope of the project doesn’t allow for it. Testing can also take a lot of work to set up if you’ve never done it before: there are libraries to think about, environments to configure, and competing testing philosophies to consider. Additionally, if you’re dealing with databases or APIs, you typically have to mock them, or create a test database so you don’t accidentally erase all of your users’ data, or blow the roof off your server bill.
Here’s everyone’s least favorite answer: “It depends.”
Are you working on a side project just for yourself? Then probably no. Are you building financial software that is handling super sensitive information for paying customers? Then yes, you’d likely want to use automated testing.
There’s a broad spectrum of testing needs, and all sorts of situations in which automated testing may or may not be a good investment. Overall you and your team should take the time to really consider if it’s worth it for you.
One thing that can make your decision easier is realizing that testing is not an all-or-nothing proposition. You don’t have to have 100% test coverage, especially not at the beginning. Since ultimately, given enough time and enough customers, all of your code does get tested, ask yourself: “Would I rather have a computer test this first, or my customer?”.
So, it’s probably a good idea to have at least some automated tests in your application, which leads to the question…
There’s no perfect answer to this question, but there are several good options
Start with something easy
It will probably take longer to write your first test than the second one. For the first test, you have to acquire all the tools, configure the environment, read the documentation, and so on.
It can be really helpful to start with something simple to get that first win. Test your `addTwoNumbers` function first: you’ll actually feel great getting a test suite to run your code and tell you whether or not you remember elementary math.
The actual test can be something as simple as:
Start with the most important piece
What part of your application would cause the most stress if it failed? This could be your checkout flow, your authentication, or your automated email system. Having confidence that this mission-critical piece of code won’t break without you knowing first can be enough motivation to get you going.
Start with the thing that always breaks
Do you have a piece of code which just falls apart any time someone touches it? That one function, first written four years ago by the developer who’s long gone, which has been patched over more times than your git history can tell you? That can be an excellent place to start. You’ll feel great knowing that your test identified the issues with this problem code and that when you next update it, you’ll quickly know which changes might cause it to fail. As an added bonus, you might actually understand the code a little better by thinking about how to test its behavior.
Start with new features
Sometimes the easiest way to implement a test is to do it when first adding the feature. In the next standup meeting or the next time you get a request for a new feature, try to carve out some time in the schedule to add a few automated tests.
Automated tests serve a major purpose: they let developers know they made a mistake before they share that mistake with their users. So at the most fundamental level, tests should run before the code gets to your user. Here are some ways to run your tests strategically between the time you write code and when you ship it:
While you’re developing
Run test suites in “watch” mode, which means any time you make a change to your code, the appropriate tests run automatically and give you near-instant feedback. Some developers go so far as to write their test first, and then code the feature until the test starts passing. This is known as Test-Driven Development (TDD) or “red/green/refactor”.
Before you commit your code
Run all the tests that depend on the files you changed right before you commit your code. You can automate this step easily with git hooks and, if you’re using JS, you can leverage great tools like husky to set up your hooks.
In CI/CD
Running your test in CI (Continuous Integration) and/or CD (Continuous Delivery) typically means running it somewhere in the cloud, off of your computer, in a special environment that is very similar to your production environment. This can take a bit more work to set up, but it has the benefit of not interrupting your workflow while ensuring that erroneous code doesn’t slip into production.
If you want to read more about CI/CD, here is a great introductory article.
There are many excellent tools available for pretty much every language. If you are in the JavaScript or web development world, you can use the classic Mocha and more modern Jest testing framework. If you want to run your test in a browser as a typical user would, there are lots of great e2e testing suites available. Selenium is one of the oldest of these tools, and there are many newer options like Nightwatch or Cypress.
BugReplay is another great tool. BugReplay started as a tool for manual testing where QA teams could record videos of bugs with time-synced Javascript console and Network traffic logs. BugReplay now has some amazing integrations with several modern e2e test suites. Whenever an automated test fails, BugReplay automatically generates a bug report, and the developer or QA team can simply watch the video and view the JS console and network traffic logs to see what broke and how to fix it.
