I’ve had a couple of opportunities now to do some workshops with embedded software test guru, Jon Hagar. Most recently at last week’s SQuAD conference, we took a look at testing a hand-held 20-questions game.
Hagar had us look at several creative methods for determining what to test. We are all familiar with traditional black-box testing, where we test against user requirements or white-box testing, where we look at code, and test the various code paths. But some of the techniques Hagar described are what I’m beginning to think of as “out-of-the-box testing.”
He had us look at things such as risks and users. What were risky things that could happen to the user? For example, if the user is a small child, could there be a risk that a battery might be swallowed? This may be why changing the battery required a screwdriver. I’ve always thought this was very cumbersome and in the past, whenever a game required a screwdriver to change its battery, I would have been prone to thinking of it as “poorly designed.” However, recognizing the potential risk to a small child, helped me realize that the battery compartment may be purposely difficult to open for safety reasons.
Still, I couldn’t help but question if some of the areas we were exploring were outside the scope of what a tester should be looking at. I challenged Hagar, “Shouldn’t looking at areas of safety and usability be done by product owners or business analysts?” As a developer, I remember getting quite annoyed with testers who would report bugs on areas that were completely out of my control, wishing they would just stick with testing the code.
Hagar answered that it’s true that we don’t want testers to go down a rat hole into areas, or become obsessed with questions of usability, but that it’s good to at least ask the questions. Bring up areas of concern that perhaps were overlooked by others. If the answer is, “Don’t worry about it,” the tester has to be able to let it go, but that thorough testers will explore areas outside of typical boundaries.
To read more about embedded software quality and the workshops with Jon Hagar see: