One definition for exploratory testing is “simultaneous learning, test design and execution.” In Paul Carvalho’s concurrent session at STAREAST, he spoke specifically about the test design part of that equation. Though exploratory testing does not use a step-by-step script, it is not the “adhoc” testing that many people mistake it for either. Exploratory testing uses knowledge of the system under test to target areas that are most prone to bugs. Using that knowledge, as well as knowledge gained as testing progresses, the tester is able to zero in on potential problem areas to uncover defects.
Carvalho’s session covered different techniques that testers could use to help design their exploratory tests. For example, if you take a look at a computer, there are four basic areas you might want to focus your tests on: input, memory, processing and output. For each of these there are a variety of types of tests that can be performed. With inputs, for example, you can check for boundary conditions, valid and invalid inputs, internationalization and constraint attacks. For memory, you can check for memory attacks, cookies, cache and database CRUD (create, read, update, delete). In the area of processing, you can test for computation constraints, analysis and timing. For outputs, you can check output constraints, error logs and resource usage.
These are just some starting points and one of many ideas given by Carvalho. He also talks about the multi-dimensional quality of test design, pointing out five dimensions that must be considered when we test: features, configuration, quality attributes (ie. the ‘ilities’), the build, and the time.
As you start to use techniques and brainstorm, the ideas of how the system can be tested multiply, but Carvalho warns to stay focused, using a top-down strategy. Look at the whole product or solution– not just the software; consider input from the stakeholders and current risks to help prioritize the tests, making the best use of the resources that are available.
For more videos, tips and interviews with the experts, check out our SSQ STAREAST page.