Exploratory testing’s “intelligent” creativity

I first became interested in exploratory testing while reading James Whittaker’s book on testing and testing “tours.” Reiterated by “buccaneer tester” James Bach, I learned that exploratory is more than just “unscripted” testing; it’s unscripted testing done intelligently. The tester’s experience and knowledge of the application and business allows for a deeper, more creative level of testing.

Recently, SearchSoftwareQuality.com contributor Chris McMahon penned two tips about his experiences with exploratory testing.

In The link between automated UI tests and exploratory testing, McMahon explored the relationship between automated user interface (UI) tests and exploratory testing (ET). In it, he gave this observation:

“It is a common mistake on agile teams to view UI test automation and ET as entirely separate activities. In fact, each activity should inform the other. In particular, an automated UI test that fails should always be a sign that some ET is required.”

In Exploratory vs. planned testing: Which yields better results?, McMahon describes a project in which he worked side-by-side a co-worker, testing the application using the two different approaches, using metrics to help determine which method resulted in the higher-quality outcome.  Take a look at his findings. You may be surprised by what he learned.

Automated test design: STARWEST 2010 preview

I recently got some pointers on automated test design and model-based testing — and some clarifications, too — from Antti Huima, CEO of Conformiq and software researcher, developer and educator. Huima will be giving a presentation at STARWEST this week on “Automated test design in Agile development environments.”

Huima explained the distinction between test design and test execution in model-based testing. When automating test execution, test cases are written often in the native language of an application and executed automatically with results logged. However, automated test design — also referred to as “model-based testing” – is the automation of the creation of the test plans and test scripts using a modeling language. One of the benefits of using a modeling language allows for test cases and test scripts to be generated for a variety of platforms.

I asked Huima what was used as input for the modeling code. Requirements? Huima answered: “What you put into the model is like an abstract version of the correct behavior of the system.”

“So who writes the modeling code?” I asked. “The developer? The architect?”

Huima pointed to the testing department, indicating that there is some advantage to have the tests designed independently from the programming team. ”However, it’s good to have some kind of programming background because the models are a kind of program,” he added. More and more companies look for testers with a programming background, not just because of modeling, he said, because it has become the trend for testers to be more involved in automation of execution as well as design.

Huima’s presentation is specifically about using ATD in agile development environments. “Is it used in traditional environments?” I asked.

“Sure, it can definitely be used in traditional development environments,” Huima answered, indicating that that was really the norm, and that it was often those agile groups that use test driven development (TDD) that had more questions about how to use ATD in conjunction with TDD. He elaborated, saying:

“They think they have to model the whole system before they can get any tests out of a model-based testing tool. But this isn’t actually the case. What we have found is that they can improve the test design productivity, especially in the context of interactive processes, by five-to-20 times. So it fits in with agility, and that’s what I’m going to talk about [in my presentation],” said Huima.

Huima speaks on September 29 at STARWEST 2010, which takes place this week in San Diego, Calif.

Agile practices: Continuous integration, automation, and TDD

As I was researching material needed to write, Distributed agile: Fostering development collaboration without collocation, I had the privilege of interviewing two of the authors of the IBM Press book, “A Practical Guide to Distributed Scrum.” I was able to catch up with Elizabeth Woodward at Agile 2010 and pick her brain about her experiences working in distributed agile environments.

More recently, I had the pleasure of doing a podcast interview with Steffan Surdek. In the interview with Surdek, we spoke about three practices which were described in the book to be particularly valuable to fostering collaboration during a Sprint: continuous integration, test automation and test-driven development (TDD). Surdek describes each of these and their importance in software development.

I realized, as Surdek described these practices, that although they are often associated with agile development, they could just as easily be used in traditional software development and certainly they could be used regardless of your team distribution. I managed teams who used JUnit and other programming techniques to automate unit tests and integration tests many years ago. Though our overarching methodology was more traditional with phases and gate reviews, we still used practices to ensure quality was being considered and tested for from the beginning.

Quality gurus continue to argue the pros and cons of various methodologies, but one thing seems to be clear: the earlier we can find defects and get them fixed, the better. Regardless of the methodology you use, practices such as continuous integration, automation and TDD will help you test for quality throughout the lifecycle.

Infrastructure integration testing with David Watt

Software applications are no longer ”just standalone systems, but part of a much more complex whole,” said David Watt at this month’s Software Quality Association of Denver (SQuAD) meeting. Watt introduced the concept of “infrastructure integration testing” (IIT) in his presentation, “Infrastructure Integration Testing: The Oft Ignored Test Type.” “Our traditional training is for standalone. We have to move beyond that mindset,” counsels Watt.

Watt describes IIT as more specialized testing that goes beyond traditional system integration testing. In his presentation he describes the many layers of complexity that are involved in today’s enterprise software. He also introduces the idea of an Enterprise Test Bed – a centrally managed environment for IIT to take place, consolidating existing test environments.

In this short video clip, Watt briefly describes more about IIT:

[kml_flashembed movie="http://www.youtube.com/v/i7k8XuD1mbg" width="425" height="350" wmode="transparent" /]

Has your organization adopted ALM practices?

I’ve been struggling with the phrase, “application lifecycle management” (ALM), for some time now. The term has been used to describe processes, tools and practices, but what exactly does it mean? Colleen Frye helps clarify with a 3-part series of stories that she’s written for SearchSoftwareQuality.com.

She starts by describing roles and responsibilities in an ALM-focused organization. As seen with agile methodologies, the focus is shifting towards increased communication and collaboration throughout the entire lifecycle. “While some of the lines between jobs are blurring, ALM is really about knocking down the walls between silos,” writes Frye. 

In Changing industry roles in ALM-focused organizations, Frye continues the exploration of ALM roles and how those may differ from traditional environments. The changing roles of project manager, business analyst, architects, QA and test are discussed as well as examining a new role, that of an ALM specialist.

In the last of the series, How ALM development tools impact team work, Frye talks about some of the characteristics of ALM tools. Tool integration is a common theme as well as more automation of reporting and project status. However, again, emphasis is primarily  on tools that allow for collaboration, communication and the breaking down of silos.

Freelance opportunities for QA and software test professionals

It’s no secret that there are a lot of people unemployed these days. However, if you have access to the Internet, there are plenty of opportunities to hone your technology skills, gain additional experience and even earn some cash while you’re at it.

One community I joined myself a year or so ago is uTest, an organization that facilitates crowdsource testing. When you join the uTest community, you fill out a questionnaire, indicating the skills you currently have as well as the hardware and software that you have available. From there, you are sent information about applications that are a good fit and are in need of testers. There is plenty of information about how to collaborate with the team that is testing the application and how to submit bugs, which, if valid, will earn you a little money.

However, beyond the money-making opportunities, there is that added benefit of being able to learn and grow testing and technology skills. There are discussions, webcasts, links and information about the abundance of free resources available for those in the QA and software test profession. It’s nice that you can join this community and benefit, and only work when you have availability.

Another site that freelancers might want to check out is Elance. The Elance home page lists categories of freelance opportunities for Programmers, Designers, Writers, Marketers, Admins and Consultants. Under Programmers, are several sub-categories, including QA. When I clicked the link I was brought to a page listing the first 25 of 5,654 results for QA freelance jobs! Job-seekers are able to submit a proposal to the clients who have listed and negotiate a deal.

Just for grins, I checked what was necessary to sign up. It appears there is a free subscription which will allow you to connect with up to 10 clients, take up to five skills tests that will help get you noticed with potential employers, give you a virtual workroom and manage the billing and payment system between you and the employer. There are upgraded plans that will give you additional benefits.

There are other freelance groups out there as well. Of course, to be a strong tester, most employers will want to see technology skills in the areas you are testing, so it’s best to make sure you are staying current in programming and Web development skills. Pick some technologies in which you want to excel and gain as much expertise as you can through self-study and hands-on application. Then target freelance opportunities in those areas and start building your portfolio. Before you know it, you’ll have more work than you can handle.

Mastering application performance testing

As a former performance test manager at Sun Microsystems, I am well aware of the complications of performance testing. The problems start early when trying to gather the requirements. “The application needs to go fast enough that people don’t complain about performance,” were the typical type of requirements I’d hear. In Oliver Erlewein’s recently published tip, Seven quick tips for better performance requirements, Erlewein gives useful advice on working with stakeholders and project members to identify quantifiable, testable requirements.

But the challenges don’t stop at requirements. Performance tests need to be executed throughout the development cycle, but teams struggle with how to test for performance on an incomplete product that is running on different hardware, in a different environment, with different loads and different users than they will be running with once they go live.

This week, SSQ published, Mastering application performance testing: A tutorial, filled with useful information and articles from performance test experts. Find out how to address the many challenges of performance testing as well as how to measure and monitor for success.

Author Lisa Crispin joins SearchSoftwareQuality as Agile expert

I’m happy to announce that Lisa Crispin, co-author with Janet Gregory of the book,  Agile Testing: A Practical Guide for Testers and Agile Teams, has joined SearchSoftwareQuality as an agile expert. This means that each month Crispin will be answering our readers’ questions about agile methodologies and practices. She started this month by answering the question: How do Agile teams handle changes to requirements?

Crispin is no stranger to the SSQ staff. I’d recorded a podcast with Crispin earlier this year where we discussed some of the differences between agile development and traditional development. I’ve also had the opportunity to shoot several short videos from the StarEast Conference and Colorado-area user group meetings discussing topics such as her favorite open-source tools or her opinion of methodology wars.  Most recently, Crispin and I both attended the Agile 2010 conference in Orlando, Fla., where Crispin was often found manning a station promoting diversity and women in agile testing.

Besides her role on our Ask the Experts staff, Crispin is lined up to help SSQ with upcoming specials which address challenges distributed agile teams face. You can be sure that whatever Crispin is up to, whether it’s tending to her donkeys or tweeting the latest agile news to her 1540 followers, she will be doing so with great energy and passion.

Innovation games provide a new way of thinking

One of the sessions I attended at Agile 2010 was a workshop entitled, “Life’s not a Beach, It’s a Game.” The session was facilitated by Luke Hohmann, CEO and Founder of The Innovation Games Company, and Cory Foy, senior consultant at Net Objectives.

In this video clip, Foy describes a little more about how innovation games are used in business to help us think outside of the box.

[kml_flashembed movie="http://www.youtube.com/v/Up2PMoI80GA" width="425" height="350" wmode="transparent" /] 

Read more about my experience playing Prune the Product Tree and My Worst Nightmare in Innovation games: Who says work can’t be fun?

The top things to ensure Agile success

The Agile 2010 conference provided plenty of opportunities for agile enthusiasts to learn and share both what worked well and what problems they encountered in the world of agile software development. In a panel session titled “Agile Trends and Future Directions,” industry experts were asked what the top things that could be done to ensure agile success. 

 Answers included:

• Make it part of your organization. Many are unprepared. Prepare for change.
• Make it an end-to-end process. Look at the whole, not just development.
• Continuous integration, continuous testing, continuous delivery.
• Make it part of the culture.
• Strong collaboration with business, end users and operations – build strong relationships. 
• Don’t impose it on people, but drive by self-motivation.

One of the panelists, Forrester Researcher analyst David West, met up with me for lunch later to discuss more about trends he was seeing in the agile ALM (Application Lifecycle Management) arena.

I also met with Elizabeth Woodward, who Elizabeth Woodward spoke with me further about the book she co-authored, A Practical Guide to Distrubed Scrum. Woodward and others stess the importance of communication and relationships amongst team members.