Presenting correct information isn’t just a function of how you write your report at the end of a software project. Instead, it is the result of a complex process that starts with analyzing the needs of your stakeholders, moves on to gathering accurate and timely data from all your multiple project sources, and then results in presenting your findings in the correct format, at the right time and to the proper audience. Joel Montveslisky calls this “Testing Intelligence” and is presenting a talk on that topic at this year’s Conference for the Association for Software Testing (CAST), July 13-16th in Colorado Springs.
“Testing intelligence is a term to describe a slightly different perspective to software testing that places the focus on the needs of our project stakeholders instead of on the application under test. The main idea is to position the testing team as a service organization within the development group, whose purpose is to provide the timely and actionable testing-based visibility needed by the project stakeholders to make their tactical and strategic decisions.”
“In principle this is nothing new, but in practice many testing teams tend to get disconnected from the changing needs of the Organization during the project and end up working for the sake of their own “product coverage needs” or the old information needs of their Project Stakeholders.”
Montvelisky is one of the founders and Product Architect of PractiTest, a company providing an SaaS (Software as a Service) test and quality assurance (QA) management system. He is also a QA consultant specializing in testing processes and a QA Instructor for multiple Israeli Training Centers. A member of the Advisory Board of the Israeli Testing Certification Board (the Israeli chapter of the ISTQB); he publishes articles and a periodic QA blog and is an active speaker in local and international conferences.
According to Montvelisky, the process of gathering testing intelligence starts by correctly identifying your stakeholders, then working with them to understand their needs, and finally providing them with correct and timely information they need. He thinks there are a number of things that make this a hard process, or at least not a trivial process, for testing teams:
“We can start by the fact that many times we are not aware who are all our project stakeholders, we tend to miss some that are not physically close or that enter the project late in the process. Secondly, we testers are not really trained to work with customers […], so many times we don’t communicate correctly and we assume their needs without consulting with them about what information is important for their decisions, what format they need it, and when. And finally, we don’t take into account the dynamic nature of our projects. We don’t understand that people require specific information at certain times. Nor do we take into account that as the project evolves the information we need to provide changes.”
Montvelisky started developing his CAST talk when he was a QA Manager working for an enterprise software company. There he realized that many stakeholders were frustrated with the existing bureaucracy of the QA work. The stakeholders thought the QA team’s work was dictated by test-planning documents written months beforehand. Montvelisky noticed that those documents were not staying relevant to the current issues affecting the release.
“In this company we made a mind-shift and decided to set aside time during the project for ‘specific testing tasks’ that would be given to us by the Development Team in real-time. Soon enough the demand for these tasks increased and we realized that we were providing real value to the process by becoming the eyes and ears of the project. After I left this company and became a consultant I took this approach with me and created a process around it to help organizations make the mind-switch and start working more effectively with their stakeholders throughout the project.”
The chance to get feedback is one reason Montvelisky is excited to be presenting at CAST. “It’s not easy to receive hard criticism,but once you learn to take it in a positive light and use these comments to continue developing your work, it makes it one of the most fruitful encounters for people looking to improve and develop ideas in the field.”
I asked Montvelisky where he thought he might get some pushback on his approach:
“In the past, I’ve heard two main areas of criticism to my approach, both of them fair. First, people explain to me that all their professional lives they’ve worked based on what I describe as testing intelligence, and that this is nothing new to them. To these people I usually come asking for their best practices and asking for their inputs in order to improve my approach.”
“Second, people tell me that our job should limit itself to test, and ‘we should be proud’ of it instead of trying to find big names for what we do, leaving this to the marketing team. To these people, I try to explain that every team in the organization needs to contribute value to the process, and if they think that all their value comes from reporting bugs and coverage percentages then they can continue working like that.”
“Having said that, there is a lot more value that can be provided by the Testing Team, and we don’t need to change what we do in order to provide it we only need to make sure we stay connected with our stakeholders and help them throughout the project and not only at the end of it.”
Montvelisky is currently focusing on a couple of research topics. One of them is related to adding value by correctly utilizing the test management tools in the organization. The other is related to collaboration between testers from different organizations, and different cultures and countries, in order to improve their overall work.