Uncharted Waters

Nov 14 2018   3:04PM GMT

Modern Test Fundamentals – Quick Tests

Matt Heusser Matt Heusser Profile: Matt Heusser

Software Quality
Software testing

Neo from the Matrix: Quick Tests

There are plenty of approaches to software testing. The most common is probably what I like to call “wamby-pamby confirmatory tests.” These take the requirements, turn them sideways, and test one simple example for every type of test. So, for example, a good log-in in works, a bad username or password does not. Confirmatory test are simple, cheap, obvious … and they miss a lot.

One huge category that confirmatory tests miss is the platform. Printers, for example, have problems when the tray is removed mid-print. Laptops used to have problems when the lid was shut during work. Websites fail to render as the user resizes the screen. Add to that invalid, blank, or incorrect input — it is common that programmers fail to consider incorrect input, so the results, while unpredictable, are probably not “right.”

Quick tests, sometimes called “quick attacks”, are a set of tests that cover these sorts of problems. They easy to come up with, cheap to run, yield bugs fast, and do not require an understanding of the underlying platform.

Getting Started With Quick Tests

The basic idea behind quick attacks is to overwhelm the software with unexpected behavior. Ironically, people with a testing mindset know to look for these problems. For them, the problems are not unexpected at all. Testers dive into the software, sending in data that is not correct, bouncing around the user interface, going to places the software does not expect in an order that it does not expect. Testers force the wireless on the mobile phone to go out during an important transaction – which is exactly what will happen when the customers are in a bus and lose cell coverage. They simulate the loss of Facebook when it goes down. They send in words to a field that expects numbers and text that is too long (and has no spaces) to a field that expects a few words.

The best testers find their own quick attacks, building their own lists, based on the platform, the software, and the team. To get started you can always review some of my old scripts, including Ten Quick Attacks for Web-Based Software, responsive design quick attacks, and Security quick attacks.

Those are mine, and some are dated. The exact tests you choose will depend on the platform and the programmers. One place to mine for quick attacks is old bug reports and fixed issues. If a category of defect just keeps happening, then you’d better test for it. As you find those categories, find a way to feed the data into the whole team (“One third of all defects this sprint were from null input. Remember now – if the input is no, the submit button don’t go.”) Eliminating entire categories of defects will push testers to find new ways to add value. That’s a good thing.

Here’s a little video on some of the platform problems with old versions of Windows running memory intensive applications.

Dogs In the Vineyard

Eric Sink once wrote that you can’t solve problems, you can only trade problems you don’t want for problems that you do. Introducing quick tests  does add a few new problems.

Quick tests are so easy to learn, and generate defects so quickly, that it is easy to think of them as all of testing. If you think about it, Quick tests are much less about the actual software under test and much more about the platform and it’s common failure modes. It is, after all, possible to test a complex chemical-bonding application with quick tests with no knowledge of chemistry. Finding bad errors and exception handling and doesn’t really tell you if the software is calculating the right answer! Domain testing, which we’ll talk about next time, is a fancy way to analyze the business logic and come up with a subset of test ideas to run out of all the possible tests.

These sorts of tests are also hard to automate. They are doing something else in the background, to change the data underneath user number one, resizing the browser and seeing rendering problems. Pulling the paper out of the printer, and so on, are user actions that can be hard to simulate. As a result, test automators tend to forget about these tests entirely, which we’ll cover in a future post on test automation.

For now, enough to know how to perform quick tests, track and give feedback to eliminate them. There is one more tool for your toolbox.

Use it wisely.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: