Software Quality Insights

May 10, 2011  11:19 PM

Exploratory test design with Paul Carvalho

Yvette Francino Yvette Francino Profile: Yvette Francino

One definition for exploratory testing is “simultaneous learning, test design and execution.” In Paul Carvalho’s concurrent session at STAREAST, he spoke specifically about the test design part of that equation. Though exploratory testing does not use a step-by-step script, it is not the “adhoc” testing that many people mistake it for either. Exploratory testing uses knowledge of the system under test to target areas that are most prone to bugs. Using that knowledge, as well as knowledge gained as testing progresses, the tester is able to zero in on potential problem areas to uncover defects.

Carvalho’s session covered different techniques that testers could use to help design their exploratory tests. For example, if you take a look at a computer, there are four basic areas you might want to focus your tests on: input, memory, processing and output. For each of these there are a variety of types of tests that can be performed. With inputs, for example, you can check for boundary conditions, valid and invalid inputs, internationalization and constraint attacks. For memory, you can check for memory attacks, cookies, cache and database CRUD (create, read, update, delete). In the area of processing, you can test for computation constraints, analysis and timing. For outputs, you can check output constraints, error logs and resource usage.

These are just some starting points and one of many ideas given by Carvalho. He also talks about the multi-dimensional quality of test design, pointing out five dimensions that must be considered when we test: features, configuration, quality attributes (ie. the ‘ilities’), the build, and the time.

As you start to use techniques and brainstorm, the ideas of how the system can be tested multiply, but Carvalho warns to stay focused, using a top-down strategy. Look at the whole product or solution– not just the software; consider input from the stakeholders and current risks to help prioritize the tests, making the best use of the resources that are available.

For more videos, tips and interviews with the experts, check out our SSQ STAREAST page.

May 10, 2011  9:38 PM

Kristan Vingrys from Thoughtworks: An Agile build pipeline

Yvette Francino Yvette Francino Profile: Yvette Francino

Another presenter that I was able to speak with at last week’s STAREAST conference in Orlando was Kristan Vingrys from Thoughtworks. Vingrys is the Global Test Practice Lead for ThoughtWorks. In his session titled, “The Agile Build Pipeline: A Tester’s Lessons Learned,” he describes the experience of Insurance Australia Group as they launched a new complex service requiring early integration and strong testing capabilities.

The project called for continuous integration and a build pipeline that allowed for production deployment on the same day the code changes were made with a high degree of confidence that there would be no regression bugs.

I was reminded of my conversation with Thoughtworks Studio’s Jez Humble last summer about continuous delivery and the difference between continuous delivery and continuous deployment.

As both Vingrys and Humble point out, it’s the customer’s decision about whether or not to flip the switch to deploy to production, but having an automated build process that allows for quick delivery can certainly be a benefit to the business.

May 10, 2011  8:02 PM

Gojco Adzic: A changing culture is the most difficult problem testers face

Yvette Francino Yvette Francino Profile: Yvette Francino

I had the pleasure of sharing breakfast with Gojco (pronounced “Goyco”) Adzic at last week’s STAREAST conference. Adzic delivered one of the keynotes at the conference, title “Sleeping with the Enemy. He talked of the need to change the traditional culture, in which testers work independently from developers, to one in which there is trust and teamwork between business, development and test.

In fact, in the short video clip I took, Adzic listed culture change as the most difficult problem that testers face. Naomi Karten addressed the challenge of organizational change in her keynote as well, talking about the chaos we all face during transition and the need to give people time to adjust to that change.

One cultural change that Adzic and I discussed was around how we measure quality. We were chatting about Lisa Crispin’s upcoming presentation about defect tracking and I asked Adzic whether or not he felt it was possible for managers to get the metrics they needed to make decisions if a defect tracking tool wasn’t used.

Adzic answered rather emphatically that the number of defects is not a good measure of quality. He reiterated some of the lessons I’d heard from Lloyd Roden’s session about challenges testers face, in particular the misleading nature of metrics.

Adzic felt that rather than using defect numbers as a measure of quality, managers should be looking at things like customer satisfaction and product revenue. What good is a product with no bugs if no one wants to use it?

He has a good point, but can QA managers make decisions or recommendations about the readiness of a product to go live without gathering metrics on defects? It’s an interesting question with differing viewpoints. What do you think? Is this another area where our culture needs to change?

May 9, 2011  11:14 PM

Data warehouse automation: Vasudeva Naidu from Infosys at STAREAST

Yvette Francino Yvette Francino Profile: Yvette Francino

Many of the attendees at STAREAST are looking for ways to become more efficient with their test efforts. Infosys leader Vasudeva Naidu is responsible for helping to find accelerators for the industry, one of those being an automation tool for data warehousing.

In his STAREAST presentation, Maximizing Automation in Data Warehouse Testing: Infosys Clearware, Naidu explained how automation could be used to test data where volumes exceed 25 million records. He also explained the usage of utilities for testing various aspects of a data warehouse such as metadata analysis, business rules builder, pattern matcher and crosswalk rules.

Naidu, has helped companies define and implement testing efficiency improvement programs and has successfully deployed Test Centers of Excellence (TCoEs).

Naidu has contributed to SearchSoftwareQuality recently with these two tips:

Manual testing vs. automated testing: A decision point
Scripted testing vs. exploratory testing

Check out our SSQ STAREAST page for more videos, tips and interviews with industry experts.

May 9, 2011  10:27 PM

Janet Gregory explains why some people prefer story boards over tools

Yvette Francino Yvette Francino Profile: Yvette Francino

I’m a huge proponent of Agile methodologies, but I have to admit, one thing I never really understood is why some people prefer story boards over tools to track projects. It always seemed to me that it would be much better to have everything in a tool in which you could run reports and more easily share data.

I asked Janet Gregory about this during a session she led at STAREAST 2011: Seven Key Factors for Agile Testing Success. Gregory explains in this video clip how the visual and physical presence of a story board provides more visibility to the team and organization.

I’ve talked to others about this who agree that a story board often can really highlight status and areas of concern that are often not so easily identified in tools. However, most people agree that in certain circumstances such as distributed teams or large teams, the use of a tool is required. In some cases, smaller teams use the story board and then someone will have the task of updating a tool appropriately so that the data can be captured electronically.

A common message amongst industry experts at the STAREAST conference is there is no single right answer. Each team must assess what works best for them, given their project and team culture.

In STAREAST: Agile testing success with Janet Gregory, you’ll find out more about Gregory’s thoughts as she talks to Lisa Crispin. Gregory and Crispin are co-authors of, “Agile Testing – A Practical Guide for Testers and Agile Teams.”

Check out our SSQ STAREAST page for more videos, tips and interviews with industry experts.

May 9, 2011  9:31 PM

Naomi Karten: Chaos is normal with change

Yvette Francino Yvette Francino Profile: Yvette Francino

Author and speaker Naomi Karten delivered the Wednesday morning keynote at this year’s STAREAST conference, and talked about managing and embracing organizational change.

She opened her presentation by telling us that “most of the great wisdom in the world comes from greeting cards,” followed by one of those witty greeting-card aphorisms: “I know I can be a difficult person to get along with, but don’t worry… You can change.”

Karten stepped us through the difficult feelings of change — something we all experience both in our personal and professional lives. When we’re going through transition, we are in a state of “chaos,” but Karten reminds us that we’re not crazy! Those crazy feelings are normal. With every change, even good changes, we are giving up something familiar, and often that loss brings on feelings of anxiety or grief. We need to be aware that we’re going through a transition and be kind to ourselves, recognizing that acceptance of a change doesn’t happen overnight.

Karten’s presentation included many 140-character tweetable reminders for managers who are dealing with organizational change. Advice included:

  • Acknowledge success and progress when going through changes.
  • Recognize the power of listening and empathy as change management tools.
  • Don’t forget about the messy stuff as change is taking place. Change takes time.
Karten’s most recent books include Presentation Skills for Technical Professionals: Achieving Excellence and Changing How You Manage and Communicate Change: Focusing on the Human Side of Change.
For more videos, tips and interviews with industry experts, check out the STAREAST page on

May 9, 2011  8:23 PM

Agile leadership with Bob Galen

Yvette Francino Yvette Francino Profile: Yvette Francino

Bob Galen is a regular presenter at Agile and software quality conferences so it was no surprise to find him at STAREAST 2011. He led a half-day tutorial entitled, “A Test Leader’s Guide to Agile,” which explored some of the challenges leaders face when dealing with Agile adoption.

As difficult as Agile adoption may be on small projects, it’s especially difficult for large-scale projects. In Large-scale Agile: An interview with Bob Galen, Galen answers questions about challenges, methodologies and techniques that can be used that help large Agile teams succeed.

In both the video clip and the interview, Galen recommends enlisting the help of a coach if your team is new to Agile.

First, go out and find yourself a qualified coach — one that has experience in your application and business domain and matches your culture. I’d also look for pragmatists over purists. Those that allow for methods adjustments based on your contexts and not those that trivialize your real-world challenges.

I look forward to seeing more of Bob Galen when he’ll be presenting at two upcoming conferences being covered by SSQ: Agile Development Practices West in Las Vegas and Agile 2011 in Salt Lake City.

Check out our SSQ STAREAST page for more videos, tips and interviews with industry experts.

May 9, 2011  4:37 PM

Shmuel Gershon on fuzzing and more at STAREAST 2011

Yvette Francino Yvette Francino Profile: Yvette Francino

Earlier this month, SSQ contributor Matt Heusser interviewed Shmuel Gershon, previewing his presentation at STAREAST about software testing with fuzzing and fault modeling. The interview gives a good overview of how you can simulate attacks or create conditions that are not ideal that will uncover bugs in real-world situations rather than a typical pristine test environment.

Heusser had suggested titling the interview, “Schmoozing with Shmuel.” Although we ended up going with a different title, “schmoozing with Shmuel” does have a nice ring to it, and I felt lucky to have my own opportunity to schmooze with Shmuel at a one-on-one interview at STAREAST 2011. I met Shmuel last year at STAREAST 2010, and I’ve always been impressed by his warmth and attention, not just to the business of software testing, but to connecting with people. A leader at Intel, he’s quite active in various software test communities and a mentor and friend to many.

Gershon is an active blogger and recently released an open source test tool, Rapid Reporter, which helps keep track of notes while exploratory testing.

Check out our SSQ STAREAST page for more videos, tips and interviews with industry experts.

May 6, 2011  6:51 PM

Ideas for tester networking and training from Agile coach Selena Delesie

Melanie Luna Melanie Luna Profile: Melanie Luna

Agile coach and software tester Selena Delesie was a speaker at this year’s STAREAST conference. She offered ideas for testers to network with others and share their experiences. Attendees were treated to her engaging and interactive presentation. Delesie runs her own company, Delesie Solutions, and has been working in the testing field for about a decade.

May 6, 2011  6:25 PM

Tom Delmonte talks about Test Centers of Excellence at STAREAST

Yvette Francino Yvette Francino Profile: Yvette Francino

To be honest, though I’d heard about Test Centers of Excellence (TCoEs) before, I wasn’t exactly sure what they were until I talked to Tom Delmonte. Delmonte, from Progressive Insurance, is a quality advocate and leader in TCoEs as well as passionate about education for testers.

Read STAREAST: An interview about Test Centers of Excellence with Tom Delmonte to find out more about TCoEs and how you might implement one in your organization.

Check out our SSQ STAREAST page for more videos, tips and interviews with industry experts.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: