Last fall I listened to James Whittaker talk about his book, “Exploratory Testing,” at a virtual presentation hosted by uTest. I have since been able to read the book and follow-up with Whittaker to pick his brain a little more about his view of exploratory testing.
“What is the definition of exploratory testing?” I asked, in an attempt to get a definition I could use for TechTarget’s popular “WhatIs” column. Whittaker was quick to note that his view of exploratory testing may not be the same as others, but he took a stab at it: “Testing without a pre-defined test plan.”
Whittaker explained that the adjective “exploratory” is used in the same way it would be used to describe “exploratory surgery.” “There’s something wrong. You’re not sure what it is and so you explore. Eventually exploratory surgery will lead to other types of treatment.” Similarly, Whittaker explains, exploratory testing is used to explore problem areas so that then further more directed tests can be done. “Exploratory testing helps us document some of the primary testing paths through the software and discover problematic areas so we can explore further testing.”
Whittaker stressed that he felt exploratory testing was a means to an end and not the end of the testing, and said that this is where his opinion may differ from those of his colleagues.
“Using it as the actual end is very naive. It’s one part of the testing process. Don’t assume anything about the solution until you know more about the problem.”
Whittaker also noted that he felt there were some problems with exploratory testing.
“Exploratory testing doesn’t always apply. Some applications require automation to thoroughly test. Concepts like tours are very important. Exploratory surgeons have techniques. a certain way to enter depending on whether or you’re operating on the pancreas or the heart or a toe.” Similarly, there are different ways you test depending on the technology. The tours can help structure your testing efforts without scripting them.
“Isn’t exploratory testing simply unscripted testing?” I asked. Whittaker agreed but pointed out that, as he states in Chapter 5 of his book, there are “hybrid” exploratory testing techniques which can include a combination of scripted tests or scenarios and unscripted tests. These can complement one another and provide for something Whittaker refers to as “scenario-based exploration.” Traditional scenario testing takes the tester down a pre-defined path reflecting expected usage of the application. But scenario-based exploration more closely imitates users who often stray from the expected path.
Come check out six of the exploratory testing tours in this tip Six tours for exploratory testing the business district of your application.
Proceeding a Webcast session with Theresa Lanowitz earlier this week, Lanowitz and I entered the post Webcast debriefing room (basically a private phone conference line) where we had the opportunity to pick one another’s brain. Somehow, Toyota was brought up and we ended up finding some common ground in our interests, even those outside of software development and testing–the automotive industry. Both Lanowitz and I agreed, the auto industry is (and has been) suffering from many of the same ailments we are concerned with in software.
The first spark was Toyota and the quality assurance disaster they are currently sorting through. And then a documentary we had both seen “Who killed the electric car” also found its way onto the discussion palette. Both topics were ones we were very passionate about. For me especially, it was the car angle. How the American auto manufacturer GM had really dropped the ball–by killing a market they were the first to enter (the electric vehicle market).
GM was quick to recognize that the combustion engine was a motor we could not rely on infinitely. They knew that one day oil and gas would be either depleted or so expensive that it was no longer affordable to anyone in their consumer base. They decided to build the first electric car, the EV1, which was based and sold on the Saturn platform.
After being at the blunt end of scrutiny and ridicule from their investor’s (Mostly made up of Oil and Gas distributors) GM decided to pull the plug on the project and recall–all 77 EV1’s already sold and on the road. They based the recall on “potential software, and electronic complications”, which they had uncovered in post-sale testing. They arrived in trucks in front of every EV1 owner’s household and loaded up these forward thinking vehicles. Their final destination was one of two places, a handful of fortunate EV1’s made it into auto museums, the bulk of them (just under 75) were dismantled and buried in an Arizona desert.
Lanowitz was more or less baffled by GM’s plug pulling on their production line electric car, but the conversation quickly moved onward to Toyota. There is a real debate going on in the media over whether or not this Toyota fiasco is related to a mechanical issue or to their drive-by-wire computer controlled throttle. It’s a good question, and no one seems to have all the facts but it leads to a sensitive area of technology–with all these various software vendors building function into modern vehicle drive trains, braking systems, radios, communication systems, etc–could there be a problem of over-integrating systems?
Let me give you the benefit of my extensive background in the automotive space. Fifteen years ago, the level of engineering we see today was virtually non-existent. Sure, there were computers mounted in vehicles which controlled the basics of the engine by monitoring rpms, transmission temperature and fluid levels–but they weren’t in constant contact with all other aspects of the vehicle. They were very, very basic, if something broke they signaled a light to flick on the dashboard, and that is about it.
These days, every system is interacting with every other system. The brakes communicate with the radio, the radio is speaking to the door locking sensor, the door locking sensor is dependent on the engine speed–it is like being in a crowded room with numerous conversations going on, all about different topics and somehow everyone is directly involved. It will only get more complex.
As experts on SearchSoftwareQuality have been saying for years now, software is becoming increasingly complex–and automotive integration is mimicking that level of complexity. These days if there is a vehicle issue, you bring it to a mechanic, this mechanic (believe it or not) probably spends just as much time in front of a diagnostic computer as he or she does under the hood. There may come a day, in the not too distant in the future where a run of the mill mechanic will need a Master’s degree in computer engineering just to become certified.
You software testers dealing with multiple platforms and continuous integration are probably plagued with similar issues and nodding your heads in agreement. But in either industry, if huge strides are taken to keep quality high in these software based systems, then perhaps someday soon we can lay many of these modern quality and integration issues in the Arizona desert alongside GM’s first electric vehicle.
For most of you in the software development space, and for me in choosing online software journalism as a career, there is a tremendous amount of our time spent in front of computers and online. If you are like me, being on your computer and having Facebook open are synonmous with one another. Few could argue that Facebook and its legions of followers, friends and groups are replacing numerous aspects of what was once everyday life. The newspaper, mail, telephone, address books, valentine’s day cards and basic phone calls have all found themselves in social networking’s crosshairs.
That being the case, we would be foolish not to ride the Web 2.0 wave–so we have. SearchSoftwareQuality is now on Facebook. And while you may already be a daily reader of SearchSoftwareQuality.com, let me assure that there are many fantastic reasons to “friend” us.
Think of SearchSoftwareQuality’s Facebook profile as quick way to get daily updates on what is happening in your career, the software industry and cyberspace in general. We update our Facebook profile often, providing you with teasers and links to important software quality and testing stories (which are archived in the “Notes” portion of the page). We have a video library set up that will be updated LIVE from conferences and software venues–finally you’ll get up to the minute video coverage of the conference you weren’t able to expense through your company.
We are hoping that our presence on Facebook might also open up the communication flood gates that have remained fairly restricted on our other media channels. As our friend, we warmly welcome you to ask questions, submit software tips, “like” certain content and not hold back on telling us what you didn’t like. Communication with us on Facebook might be the ticket you need to have your company, application problem or solution story heard across our vast fanbase and readership.
To “friend” us type “SearchSoftware Quality” in the search bar on Facebook and prepare yourself for a wealth of incoming software information that matters to you.
TechTarget:Where Serious Technology Buyers Decide
Recently SearchSoftwareQuality published a Q&A with Navot Peled, CEO of Gizmox Ltd, entitled How new Web application platforms put dev/test pros’ careers at risk. It’s not often that I disagree with the experts, but as a former developer who’s way past my spring chicken days, I have to say I was a bit insulted at the implication that older developers would not be able to learn new technologies.
Certainly, emerging technologies can be a challenge to keep up with. I’d just attended a SQuAD meeting last week in which the speaker, Igor Gershovich, espoused the difficulties of automated testing of Rich Internet Applications. However, I absolutely don’t buy into the notion that an older person is at a disadvantage in being able to learn these new technologies. In fact, I would say often it’s just the opposite. Most of us have lived through many shifts in technology and are used to the constant changes that our industry throws at us. We don’t run from it. We thrive on it!
The most brilliant technologist is not well-versed in every technology. There are just too many programming languages, operating systems, databases, tools and systems to become an expert in all of them. It’s important to understand what is happening and what is changing in the world and to update our skills to meet the needs of the industry, but does that mean that if we don’t know AJAX we’re over the hill? Those of us that have been around software for years and years usually are very quick to pick up new skills. I can’t speak for the entire older set, but I can tell you that I absolutely love progress and when some new technology comes out, I want to be the first to jump on the new bandwagon and test it out. One of the great things about my job at SearchSoftwareQuality.com is that I get to be on the forefront of new developments, reading about the latest industry trends.
Not knowing a particular new technology is not what will hurt us in the job market. What will hurt us is if we stop wanting to learn. If we stay stuck in a world where we only know one way of coding and we refuse to be open to the wonderful changes that surround us, we are indeed going to limit our potential. Instead, we need to read, learn, grow and embrace change. If we do that, whether we are 20 or 90, we will be a valuable resource to any employer. There most likely will come a day when we will want to retire, but as long as we keep learning, there will never come a day when we are unemployable.
QA managers, take heart! Not all Java developers think that software quality processes are an unnecessary overhead. The proof? This week’s strong attendance and busy during and post-session Q&As at TheServerSide Java Symposium session, Software Quality: The Quest for the Holy Grail.
Defining the basic requirements for and viewing dependencies as an integrated part of a project are critical elements in project quality and ultimate success today, said speaker Jesper Pedersen, core developer for JBoss by Red Hat and project lead for JBoss JCA, a Java Connector container.
Because development platforms have more business-specific code and platforms are a larger piece of the pie, finding where issues are located is more difficult, Pedersen said. It’s become more important to do good integration testing. Also, dependencies must be managed well and as if they are part of the application.
I caught up with Pedersen after his session and asked him why managing dependencies is so tricky and why developers aren’t crazy about doing quality assurance processes. He answers those questions in this video.
On TheServerSide 2010 Java Symposium slideshare.net site, you can view Pedersen’s notes for this presentation.
Igor Gershovich, president and principal consultant of Connected Testing, Inc. spoke this month at the Software Quality Association of Denver (SQuAD) meeting. The topic is one the group had been clamoring for: Automation of Web 2.0 Rich Internet Applications (RIA).
Gershovich started by talking about Web 2.0, sometimes known as “social software,” and the technologies used to create these types of applications. Popular RIA Frameworks and toolkits incude AJAX, Adobe Flash/Flex, Google Web Toolkit and Silverlight. Gershovich said there were hundreds more, but focused his presentation on AJAX, one of the most popular RIA technologies, probably due to it’s pricetag: free!
Gerchovich described some of the technical challenges involved in automating GWT-based applications:
- They use custom or 3rd party Web controls
- They have no unique object properities
- Synchronization for AJAX
- Cascading Style Sheets (CSS)
- No common design framework between GWT applications
- Can’t view HTML using View->Source
Gerchovich went on to show the technical details about how these obstacles can be overcome, but the bottom line is advanced test automation expertise is required. Gerchovich’s examples used QTP, but he said the same techniques could be used with other automated tools such as Selenium. Coordination with the development team is required as well in order to gain insights into the objects and their properties.
Gerchovich’s presentation, as well as past SQuAD presentations are available for download.
As a recent job-seeker, I’m well aware that it’s a tough market out there. Employers can afford to be picky and many of them are looking for software quality professionals with years of agile experience. If you’ve always worked in a traditional software environment, are your skills obsolete? How can you get experience in an agile environment if you can’t get a job? And what are “agile” skills anyway? Aren’t the skills that testers gain from working in a traditional environment transferable to an agile environment? What exactly are these skills that employers are looking for?
These are some of the questions I asked Lisa Crispin in a recent podcast, as we were discussing the first chapter of “Agile Testing: A Practical Guide for Testers and Agile Teams” co-authored by Crispin and Janet Gregory.
I won’t repeat what you can listen to in the podcast, but Lisa and I stayed on the phone and chatted for another 30 minutes about the job market and ways that job seekers can better their skills. Crispin and I are both firm believers in professional development. There are so many opportunities, thanks to the Web, for job-seekers to continue to grow and learn. Not only is there an abundance of free tutorials, white-papers, and technical content available, there are open source tools galore!
Crispin said she was surprised by the number of people that she’s interviewed that were not engaged in professional development activities. She said that she was most interested in those candidates that showed active interest in learning and growing and that there are plenty of learning opportunities and ways to experience agile outside of a work environment.
If this is a trend amongst employers, then job-seekers are in luck. Again, there is no shortage of learning opportunities available on the Web, including on our own SearchSoftwareQuality.com. Stay tuned as we continue to update our agile learning guide with new content and additional podcasts from industry experts. Whether you’re a job-seeker or happily employed, the industry continues to change and grow. The key is to never stop learning!
Trends in software quality assurance’s “four Rs” — Risk, Return on investment (ROI), Regulatory compliance and Rich customer experience — were the main thrust of my recent conversation with Aparna Sharma of Infosys, whose career includes QA consulting for Fortune 500 companies.
Organizations today are doing more risk-based testing, figuring out ways to quantify risk and prioritize tests more effectively, said Sharma, Head of Client Services for Infosys’ Independent Validation Services (IVS). Many are introducing application lifecycle management (ALM) tools and integrated processes as early as possible and throughout the lifecycle to reduce overall risk for the project team.
Regarding ROI, Sharma recommends automation. Maximize ROI much earlier in the application lifecycle by using automation, environment optimization, and reducing dependencies, Sharma advised. She sees companies adopting Business Process Models (BPM) to directly generate test cases and business process flows. Though she admitted that some manual testing may be necessary, she said she advises automating as much as possible in all phases of the lifecycle.
Regulations and Compliance
Sharma noted that there were growing issues in regulations and compliance due to privacy issues. Two industries that have increased reliance on regulations and compliance are health care and financial services. Different entities within the organization will be responsible for regulation and compliance but many will require independent verification and validation services.
Offshoring application development and lifecycle management has been and still is a growing trend, but offshoring doesn’t shift the compliance burden. All companies need to make sure their software data is protected.
Rich customer experience
More users are participating in testing for usability, flexibility and accessibility early in the application lifecycle, Sharma told me. Some use tools that allow for “eye-capture” for usability testing. Accessibility software can also help test ease of use of software for people with disabilities. By creating early code, prototyping, or other various techniques, the graphical user interface is available for early usability testing and accessability testing.
SOA, ERP trends
Sharma noted a couple of other industry trends that she thought readers may be interested in:
- In the SOA space, the industry is moving away from UI to more end-to-end testing with distributed, multi-tier architectures. Organizations need a specific approach taking into account end-to-end testing using specific tools.
- Unique approaches to testing ERP are emerging. New technologies allow for automated pre-packaged test cases to test the various ERP implementations.
During a discussion about Scrum at the Boulder Agile User Group (BAUG) meeting recently, Paul Quarles of Oppenheimer Funds shared his experience as a Product Owner demoing to the business in a Scrum environment. I’ll share that story and my take on the moniker, Agilists, in this post.
There are no speakers or formal presentations at BAUG meetings. This is more of a casual sharing of experiences and dilemmas that people are experiencing as they use agile at their workplaces. What’s cool about this group is that people describe what’s happening in the real world — not necessarily what the books tell you or what the classes teach you, but what is really happening. As we all know, real life rarely works out exactly as the books describe. This may be why many people think that software development is not something you can learn from a text book or a class, but something you must experience.
Of the many topics discussed that evening, the use of Scrum came up most often. Everyone was very positive about their experiences with Scrum and were strong supporters of agile. I’ve been hearing the word “agilist” to describe agile supporters; but, being a bit of a wordsmith, I’ve decided the word “evagilist” might be a better way to describe these evangelists.
I asked the group if I might take back a “real world agile story” for Software Quality Insights, and that’s when Paul Quarles shared his Product Owner experiences.
In Scrum, there’s a Product Owner, Scrum Master, and a Team, comprised of developers and testers. Scrum uses short iterations ending in a deliverable piece of code that can be demoed. The books say that it’s the Team’s responsibility to demo the work results to the Product Owner. However, Paul — the Product Owner for his Scrum project — says that he is the person that demos the code for the business users. Listen to his accounting of why this works well for his group.
[kml_flashembed movie="http://www.youtube.com/v/DCboKQ6_gpA" width="425" height="350" wmode="transparent" /]
I plan to gather more of these Real World Agile stories each month. Maybe I could even make a Reagility TV Series!
VersionOne, an Agile-focused application lifecycle management product (ALM) vendor, has just released Ultimate Edition, a suite of software testing and management tools that includes current and new features. Customizable reporting capabilities and enhanced regression test functionality are new features in the new Ultimate Edition package.
Other VersionOne solutions include Team Edition, a free set of tools for a single team new to agile. It is a web-based tool based on spreadsheets with an agile workflow. The Enterprise Edition builds on the Team Edition, scaling for multiple teams, projects and locations. The Ultimate Edition adds customizable analytics platform and enhanced regression test management to the Enterprise Edition.
I spoke with VersionOne CEO Robert Holler shortly after the Ultimate Edition announcement. He said that VersionOne uses its own IdeaSpace software to collect input from users about what features they’re intested in. Requests for reporting enhancements have been repeatedly asked for, he said, and new features will allow users more choices in reporting.
“They’ll be able to use one of the many role-based best practice dashboards, a configurable analytics grid which is essentially a spreadsheet on steroids or use our wizard-driven user interface allowing roll-your-own reports.”
In talking about the enhancements for regression test management, Holler explained that VersionOne’s Ultimate Edition allows for the creation of a repository of regression tests that can be chosen from the acceptance test buckets and run automatically with builds as scheduled. I asked if the regression test enhancements were a result of user requests as well, but Holler said that was more of a strategic decision for VersionOne.
As for the bundling of the Ideas Management module, I couldn’t really see the advantage for this piece of the announcement. The Ideas Management module is still available as a separate add-on and doesn’t add any improved functionality when packaged with the Ultimate Edition. Holler said that they were trying to move away from the modular approach and provide more of a full-function solution with the bundling.
When asked how he felt about the recent announcement of CollabNet’s acquisition of Danube’s ScrumWorks, Holler felt that it was a positive sign of the industry’s recognition of the agile trend. “There is consolidation going on and incredible interest and momentum around agile,” he said.
VersionOne uses agile methodologies in internal software development. Also, iterative releases and agile concepts are also used in VersionOne’s marketing and services organizations. “Fundamentally it’s a good way to do business,” Holler said.