Software Quality Insights

March 31, 2010  5:12 PM

Software tool and quality certifications: Valuable or useless?

Yvette Francino Yvette Francino Profile: Yvette Francino

There are a bunch of software quality certifications and more popping up all the time. Some of the classics are those offered by ASQ (American Society for Quality), IIST (International Institute for Software Testing), ISTQB/ASTQB (International / American Software Testing Quality Board) and QAI (Quality Assurance Institute. ) Then there are certifications that are being offered for learning particular tools. These certifications are put out by organizations such as HP, IBM and Microsoft, to name a few.  There are also Scrum Master and Scrum Product Owner certifications for the agile crowd.  With so many different certifications to choose from, which are the most valuable for QA professionals? Or are any of them valuable? It seems as though many people feel they are useless.

I asked agile practitioner Mark Levison in a recent podcast what he thought about certifications. Mark felt it was best to stay away from certifications that encouraged a “throw-it-over-the-wall” mentality or those certifications that recommended expensive tools.  He also didn’t feel certification was necessary for agile testers or developers and thought the same skills could be picked up from reading.

I, myself, have recently gotten a Scrum Master certification so I’m quite interested in this controversy. Personally, I think any kind of industry training or professional development is beneficial. I learned a lot from the class and think that people that are new to Scrum would benefit from attending.  However, I agree that the “certification” label does not mean much.  The test is very straight-forward and if you fail, you can simply take it again. Basically, this particular certification just proves I took a two-day class. It doesn’t prove whether or not I’d be able to apply the skills I learned. I don’t put listings for all of the university classes I’ve taken on my resume and those are much more comprehensive than the Scrum class. Wny should a two-day class carry enough weight on a resume to make-or-break whether you will make it past HR’s first filter?

This lack of substantial proof of skill seems to be the basic issue for those that argue against certification.  Does that make certifications useless? A recent press release by ASQTB showed that 89% of software testers surveyed felt they were more valuable to their organization after ISQTB certification. That makes sense to me. If you take a class, one would hope you would gain some knowledge and become more valuable to your organization. I don’t think anyone would argue with that. And if I were taking a survey asking if taking a class made me more valuable to an organization, I would absolutely answer ‘yes.’ This doesn’t really quantify how much more valuable to an organization one is after getting certified.  However, a “certification” loses some of the ability to impress, if it’s attained simply from passing a test.

But what else can be done? There are a couple of networks that are available to help those wanting to gain additional agile skills.  One of these is called the Agile Skills Project. Using a wiki, this group has been using agile methodologies to define an Agile Skills Inventory, identifying skills necessary to work in an agile environment as well as offering suggestions about how to get those skills. Another group that offers a network, particularly for those interested in distributed agile, is the network I facilitate, Beyond Certification. The network, powered by Ning, allows those interested in this topic to discuss and share idea for gaining agile skills.

In general, I think taking classes and then taking exams to test your knowledge is one way to learn.  Getting certified in a particular skill proves that you took the intiative to take a class or study a body of knowledge and pass a test. Though I don’t this passing a test should be a primary factor in gaining employment, it does show that you have enough interest in a topic to want to learn. After you’ve gotten certified, work towards getting experience.  If you’re not able to do this from a workplace, then check into local user group meetings. Check the search engines and find a software test and QA community to join. Volunteer to join a Scrum Team for a non-profit or as a hobby.  There are always opportunities to learn.

So, no, I don’t think quality certifications are useless. It’s a first step. Just don’t stop there. Apply the knowledge and keep on learning.

March 29, 2010  5:12 PM

James Whittaker on exploratory testing and scenario-based exploration

Yvette Francino Yvette Francino Profile: Yvette Francino

Last fall I listened to James Whittaker talk about his book, “Exploratory Testing,” at a virtual presentation hosted by uTest. I have since been able to read the book and follow-up with Whittaker to pick his brain a little more about his view of exploratory testing.

“What is the definition of exploratory testing?” I asked, in an attempt to get a definition I could use for TechTarget’s popular “WhatIs” column. Whittaker was quick to note that his view of exploratory testing may not be the same as others, but he took a stab at it: “Testing without a pre-defined test plan.”

Whittaker explained that the adjective “exploratory” is used in the same way it would be used to describe “exploratory surgery.” “There’s something wrong. You’re not sure what it is and so you explore. Eventually exploratory surgery will lead to other types of treatment.” Similarly, Whittaker explains, exploratory testing is used to explore problem areas so that then further more directed tests can be done. “Exploratory testing helps us document some of the primary testing paths through the software and discover problematic areas so we can explore further testing.”

Whittaker stressed that he felt exploratory testing was a means to an end and not the end of the testing, and said that this is where his opinion may differ from those of his colleagues.

“Using it as the actual end is very naive. It’s one part of the testing process. Don’t assume anything about the solution until you know more about the problem.”

Whittaker also noted that he felt there were some problems with exploratory testing.

“Exploratory testing doesn’t always apply. Some applications require automation to thoroughly test. Concepts like tours are very important.  Exploratory surgeons have techniques. a certain way to enter depending on whether or you’re operating on the pancreas or the heart or a toe.”  Similarly, there are different ways you test depending on the technology. The tours can help structure your testing efforts without scripting them.

“Isn’t exploratory testing simply unscripted testing?” I asked. Whittaker agreed but pointed out that, as he states in Chapter 5 of his book, there are “hybrid” exploratory testing techniques which can include a combination of scripted tests or scenarios and unscripted tests. These can complement one another and provide for something Whittaker refers to as “scenario-based exploration.” Traditional scenario testing takes the tester down a pre-defined path reflecting expected usage of the application. But scenario-based exploration more closely imitates users  who often stray from the expected path.

Come check out six of the exploratory testing tours in this tip Six tours for exploratory testing the business district of your application.

March 29, 2010  5:12 PM

Software development QA issues run parallel to automobile manufacturing mistakes

Yvette Francino Daniel Mondello Profile: Daniel Mondello

Proceeding a Webcast session with Theresa Lanowitz earlier this week, Lanowitz and I entered the post Webcast debriefing room (basically a private phone conference line) where we had the opportunity to pick one another’s brain. Somehow, Toyota was brought up and we ended up finding some common ground in our interests, even those outside of software development and testing–the automotive industry. Both Lanowitz and I agreed, the auto industry is (and has been) suffering from many of the same ailments we are concerned with in software.

The first spark was Toyota and the quality assurance disaster they are currently sorting through. And then a documentary we had both seen “Who killed the electric car” also found its way onto the discussion palette. Both topics were ones we were very passionate about. For me especially, it was the car angle. How the American auto manufacturer GM had really dropped the ball–by killing a market they were the first to enter (the electric vehicle market).

GM was quick to recognize that the combustion engine was a motor we could not rely on infinitely. They knew that one day oil and gas would be either depleted or so expensive that it was no longer affordable to anyone in their consumer base. They decided to build the first electric car, the EV1, which was based and sold on the Saturn platform.

After being at the blunt end of scrutiny and ridicule from their investor’s (Mostly made up of Oil and Gas distributors) GM decided to pull the plug on the project and recall–all 77 EV1’s already sold and on the road. They based the recall on “potential software, and electronic complications”, which they had uncovered in post-sale testing. They arrived in trucks in front of every EV1 owner’s household and loaded up these forward thinking vehicles. Their final destination was one of two places, a handful of fortunate EV1’s made it into auto museums, the bulk of them (just under 75) were dismantled and buried in an Arizona desert.

Lanowitz was more or less baffled by GM’s plug pulling on their production line electric car, but the conversation quickly moved onward to Toyota. There is a real debate going on in the media over whether or not this Toyota fiasco is related to a mechanical issue or to their drive-by-wire computer controlled throttle. It’s a good question, and no one seems to have all the facts but it leads to a sensitive area of technology–with all these various software vendors building function into modern vehicle drive trains, braking systems, radios, communication systems, etc–could there be a problem of over-integrating systems?

Let me give you the benefit of my extensive background in the automotive space. Fifteen years ago, the level of engineering we see today was virtually non-existent. Sure, there were computers mounted in vehicles which controlled the basics of the engine by monitoring rpms, transmission temperature and fluid levels–but they weren’t in constant contact with all other aspects of the vehicle. They were very, very basic, if something broke they signaled a light to flick on the dashboard, and that is about it.

These days, every system is interacting with every other system. The brakes communicate with the radio, the radio is speaking to the door locking sensor, the door locking sensor is dependent on the engine speed–it is like being in a crowded room with numerous conversations going on, all about different topics and somehow everyone is directly involved. It will only get more complex.

As experts on SearchSoftwareQuality have been saying for years now, software is becoming increasingly complex–and automotive integration is mimicking that level of complexity. These days if there is a vehicle issue, you bring it to a mechanic, this mechanic (believe it or not) probably spends just as much time in front of a diagnostic computer as he or she does under the hood. There may come a day, in the not too distant in the future where a run of the mill mechanic will need a Master’s degree in computer engineering just to become certified.

You software testers dealing with multiple platforms and continuous integration are probably plagued with similar issues and nodding your heads in agreement. But in either industry, if huge strides are taken to keep quality high in these software based systems, then perhaps someday soon we can lay many of these modern quality and integration issues in the Arizona desert alongside GM’s first electric vehicle.


March 22, 2010  2:51 PM

Follow SearchSoftwareQuality on Facebook and Twitter

Yvette Francino Daniel Mondello Profile: Daniel Mondello

For most of you in the software development space, and for me in choosing online software journalism as a career, there is a tremendous amount of our time spent in front of computers and online. If you are like me, being on your computer and having Facebook open are synonmous with one another. Few could argue that Facebook and its legions of followers, friends and groups are replacing numerous aspects of what was once everyday life. The newspaper, mail, telephone, address books, valentine’s day cards and basic phone calls have all found themselves in social networking’s crosshairs.

That being the case, we would be foolish not to ride the Web 2.0 wave–so we have. SearchSoftwareQuality is now on Facebook. And while you may already be a daily reader of, let me assure that there are many fantastic reasons to “friend” us.

Think of SearchSoftwareQuality’s Facebook profile as quick way to get daily updates on what is happening in your career, the software industry and cyberspace in general. We update our Facebook profile often, providing you with teasers and links to important software quality and testing stories (which are archived in the “Notes” portion of the page). We have a video library set up that will be updated LIVE from conferences and software venues–finally you’ll get up to the minute video coverage of the conference you weren’t able to expense through your company.

We are hoping that our presence on Facebook might also open up the communication flood gates that have remained fairly restricted on our other media channels. As our friend, we warmly welcome you to ask questions, submit software tips, “like” certain content and not hold back on telling us what you didn’t like. Communication with us on Facebook might be the ticket you need to have your company, application problem or solution story heard across our vast fanbase and readership.

To “friend” us type “SearchSoftware Quality” in the search bar on Facebook and prepare yourself for a wealth of incoming software information that matters to you.

If Twitter is your thing, we’re there, too! Our content will automatically be tweeted, so that you’ll be able to see the latest information, as soon as it’s available. Follow us at @softwaretesttt

TechTarget:Where Serious Technology Buyers Decide

March 19, 2010  9:55 PM

Too old to learn new technologies? Never!

Yvette Francino Yvette Francino Profile: Yvette Francino

Recently SearchSoftwareQuality published a Q&A with Navot Peled, CEO of Gizmox Ltd, entitled How new Web application platforms put dev/test pros’ careers at risk.  It’s not often that I disagree with the experts, but as a former developer who’s way past my spring chicken days, I have to say I was a bit insulted at the implication that older developers would not be able to learn new technologies.

Certainly, emerging technologies can be a challenge to keep up with.  I’d just attended a SQuAD meeting last week in which the speaker, Igor Gershovich, espoused the difficulties of automated testing of Rich Internet Applications.  However, I absolutely don’t buy into the notion that an older person is at a disadvantage in being able to learn these new technologies. In fact, I would say often it’s just the opposite. Most of us have lived through many shifts in technology and are used to the constant changes that our industry throws at us. We don’t run from it. We thrive on it!

The most brilliant technologist is not well-versed in every technology. There are just too many programming languages, operating systems, databases, tools and systems to become an expert in all of them. It’s important to understand what is happening and what is changing in the world and to update our skills to meet the needs of the industry, but does that mean that if we don’t know AJAX we’re over the hill? Those of us that have been around software for years and years usually are very quick to pick up new skills.  I can’t speak for the entire older set, but I can tell you that I absolutely love progress and when some new technology comes out, I want to be the first to jump on the new bandwagon and test it out. One of the great things about my job at is that I get to be on the forefront of new developments, reading about the latest industry trends.

Not knowing a particular new technology is not what will hurt us in the job market.  What will hurt us is if we stop wanting to learn. If we stay stuck in a world where we only know one way of coding and we refuse to be open to the wonderful changes that surround us, we are indeed going to limit our potential. Instead, we need to read, learn, grow and embrace change.  If we do that, whether we are 20 or 90, we will be a valuable resource to any employer.  There most likely will come a day when we will want to retire,  but as long as we keep learning, there will never come a day when we are unemployable.

March 18, 2010  9:41 PM

TSS Java Symposium 2010: Dependencies, complexity make software QA tricky

Jan Stafford Jan Stafford Profile: Jan Stafford

QA managers, take heart! Not all Java developers think that software quality processes are an unnecessary overhead. The proof? This week’s strong attendance and busy during and post-session Q&As at TheServerSide Java Symposium session, Software Quality: The Quest for the Holy Grail.

Defining the basic requirements for and viewing dependencies as an integrated part of a project are critical elements in project quality and ultimate success today, said speaker Jesper Pedersen, core developer for JBoss by Red Hat and project lead for JBoss JCA, a Java Connector container.

Because development platforms have more business-specific code and platforms are a larger piece of the pie, finding where issues are located is more difficult, Pedersen said. It’s become more important to do good integration testing. Also, dependencies must be managed well and as if they are part of the application.

I caught up with Pedersen after his session and asked him why managing dependencies is so tricky and why developers aren’t crazy about doing quality assurance processes. He answers those questions in this video.

On TheServerSide 2010 Java Symposium site, you can view Pedersen’s notes for this presentation.

March 17, 2010  6:51 PM

Automated testing of Rich Internet Applications (RIA)

Yvette Francino Yvette Francino Profile: Yvette Francino

Igor Gershovich, president and principal consultant of Connected Testing, Inc. spoke this month at the Software Quality Association of Denver (SQuAD) meeting. The topic is one the group had been clamoring for: Automation of Web 2.0 Rich Internet Applications (RIA).

Gershovich started by talking about Web 2.0, sometimes known as “social software,” and the technologies used to create these types of applications. Popular RIA Frameworks and toolkits incude AJAX, Adobe Flash/Flex, Google Web Toolkit and Silverlight. Gershovich said there were hundreds more, but focused his presentation on AJAX, one of the most popular RIA technologies, probably due to it’s pricetag: free!

AJAX is a framework that combines Asynchronous JavaScript and XML and uses techniques that combine and exploit long-standing Web technologies. Examples of Web technologies AJAX uses are XHTML and CSS for structure and presentation and the Document Object Model (DOM) for displaying and manipulating objects. Gershovich went over some of the pros and cons of using AJAX and then explained how Google Web Toolkit (GWT) could be used to write AJAX front-end code in Java, which can then be compiled into optimized, standalone JavaScript files.

Can automation tools, such as HPs Quick Test Pro (QTP) or Selenium be used to create automated scripts for AJAX? Gershovich says he is often asked this question. He says the real question is: Does QTP [or Selenium] work with custom objects from various JavaScript toolkits? The answer is yes, but it’s not easy!

Gerchovich described some of the technical challenges involved in automating GWT-based applications:

  • They use custom or 3rd party Web controls
  • They have no unique object properities
  • Synchronization for AJAX
  • Cascading Style Sheets (CSS)
  • No common design framework between GWT applications
  • Can’t view HTML using View->Source


Gerchovich went on to show the technical details about how these obstacles can be overcome, but the bottom line is advanced test automation expertise is required. Gerchovich’s examples used QTP, but he said the same techniques could be used with other automated tools such as Selenium. Coordination with the development team is required as well in order to gain insights into the objects and their properties.

Gerchovich’s presentation, as well as past SQuAD presentations are available for download.

March 15, 2010  6:15 PM

How do job-seekers gain Agile skills?

Yvette Francino Yvette Francino Profile: Yvette Francino

As a recent job-seeker, I’m well aware that it’s a tough market out there.  Employers can afford to be picky and many of them are looking for software quality professionals with years of agile experience.  If you’ve always worked in a traditional software environment, are your skills obsolete? How can you get experience in an agile environment if you can’t get a job? And what are “agile” skills anyway? Aren’t the skills that testers gain from working in a traditional environment transferable to an agile environment? What exactly are these skills that employers are looking for?

These are some of the questions I asked Lisa Crispin in a recent podcast, as we were discussing the first chapter of “Agile Testing: A Practical Guide for Testers and Agile Teams” co-authored by Crispin and Janet Gregory.

I won’t repeat what you can listen to in the podcast, but Lisa and I stayed on the phone and chatted for another 30 minutes about the job market and ways that job seekers can better their skills. Crispin and I are both firm believers in professional development. There are so many opportunities, thanks to the Web, for job-seekers to continue to grow and learn. Not only is there an abundance of free tutorials, white-papers, and technical content available, there are open source tools galore!

Crispin said she was surprised by the number of people that she’s interviewed that were not engaged in professional development activities. She said that she was most interested in those candidates that showed active interest in learning and growing and that there are plenty of learning opportunities and ways to experience agile outside of a work environment. 

If this is a trend amongst employers, then job-seekers are in luck. Again, there is no shortage of learning opportunities available on the Web, including on our own Stay tuned as we continue to update our agile learning guide with new content and additional podcasts from industry experts. Whether you’re a job-seeker or happily employed, the industry continues to change and grow. The key is to never stop learning!

March 8, 2010  6:50 PM

Sharma opines on software QA trends: Risk, ROI, compliance

Yvette Francino Yvette Francino Profile: Yvette Francino

Trends in software quality assurance’s “four Rs” — Risk, Return on investment (ROI), Regulatory compliance and Rich customer experience — were the main thrust of my recent conversation with Aparna Sharma of Infosys, whose career includes QA consulting for Fortune 500 companies.


Organizations today are doing more risk-based testing, figuring out ways to quantify risk and prioritize tests more effectively, said Sharma, Head of Client Services for Infosys’ Independent Validation Services (IVS). Many are introducing application lifecycle management (ALM) tools and integrated processes as early as possible and throughout the lifecycle to reduce overall risk for the project team.


Regarding ROI, Sharma recommends automation. Maximize ROI much earlier in the application lifecycle by using automation, environment optimization, and reducing dependencies, Sharma advised. She sees companies adopting Business Process Models (BPM) to directly generate test cases and business process flows.  Though she admitted that some manual testing may be necessary, she said she advises automating as much as possible in all phases of the lifecycle.

Regulations and Compliance

Sharma noted that there were growing issues in regulations and compliance due to privacy issues. Two industries that have increased reliance on regulations and compliance are health care and financial services. Different entities within the organization will be responsible for regulation and compliance but many will require independent verification and validation services.

Offshoring application development and lifecycle management has been and still is a growing trend, but offshoring doesn’t shift the compliance burden. All companies need to make sure their software data is protected.

Rich customer experience

More users are participating in testing for usability, flexibility and accessibility early in the application lifecycle, Sharma told me. Some use tools that allow for “eye-capture” for usability testing. Accessibility software can also help test ease of use of software for people with disabilities. By creating early code, prototyping, or other various techniques, the graphical user interface is available for early usability testing and accessability testing.

SOA, ERP trends 

Sharma noted a couple of other industry trends that she thought readers may be interested in:

  • In the SOA space, the industry is moving away from UI to more end-to-end testing with distributed, multi-tier architectures.  Organizations need a specific approach taking into account end-to-end testing using specific tools.
  •  Unique approaches to testing ERP are emerging. New technologies allow for automated pre-packaged test cases to test the various ERP implementations.

March 3, 2010  8:12 PM

Real-world Agile: The advantage of Product Owner demos

Yvette Francino Yvette Francino Profile: Yvette Francino

During a discussion about Scrum at the Boulder Agile User Group (BAUG) meeting recently, Paul Quarles of Oppenheimer Funds shared his experience as a  Product Owner demoing to the business in a Scrum environment. I’ll share that story and my take on the moniker, Agilists, in this post.

There are no speakers or formal presentations at BAUG meetings. This is more of a casual sharing of experiences and dilemmas that people are experiencing as they use agile at their workplaces. What’s cool about this group is that people describe what’s happening in the real world — not necessarily what the books tell you or what the classes teach you, but what is really happening. As we all know, real life rarely works out exactly as the books describe. This may be why many people think that software development is not something you can learn from a text book or a class, but something you must experience.

Of the many topics discussed that evening, the use of Scrum came up most often. Everyone was very positive about their experiences with Scrum and were strong supporters of agile. I’ve been hearing the word “agilist” to describe agile supporters; but, being a bit of a wordsmith, I’ve decided the word “evagilist” might be a better way to describe these evangelists.

I asked the group if I might take back a “real world agile story” for Software Quality Insights, and that’s when Paul Quarles shared his Product Owner experiences.

In Scrum, there’s a Product Owner, Scrum Master, and a Team, comprised of developers and testers.   Scrum uses short iterations ending in a deliverable piece of code that can be demoed.  The books say that it’s the Team’s responsibility to demo the work results to the Product Owner. However, Paul — the Product Owner for his Scrum project — says that he is the person that demos the code for the business users.  Listen to his accounting of why this works well for his group.

[kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

I plan to gather more of these Real World Agile stories each month. Maybe I could even make a Reagility TV Series!

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: