Software Quality Insights

A blog

November 14, 2011  4:16 PM

What do you think of Agile?

Posted by: Yvette Francino
agile, survey, Theresa Lanowitz

“We hear a lot of talk about Agile in the media. We want to hear from you:
-Is Agile hype or helpful?
-Do you love Agile or hate it?
-Or is your team proceeding with business as usual?”

So starts the Agile survey being administered by voke, inc. Here’s your chance to offer up your point of view on Agile.

The survey is split into the following nine sections:

1. Demographics
2. Organizational background
3. Software methodologies
4. Use of Agile (if applicable)
5. Use of Sprints (if applicable)
6. Use of Scrum (if applicable)
7. Platforms, tools, and organizational roles
8. Quality
9. Impressions of the Agile Manifesto

The survey will run through November 30 and will take about 15 minutes to complete. After taking it, you’ll have the opportunity to receive a free copy of the published report.

Founder of voke, inc Theresa Lanowitz has provided insights to SSQ on a variety of topics. Check out this Webcast about APM tools titled, Application Performance: The Intersection of Business and Technology.

November 11, 2011  2:40 PM

How do you measure software quality?

Posted by: Yvette Francino
defect tracking, defects, Software Quality

Though software test experts do agree on a lot, the question of how to measure software quality is a subject of great debate. In his opening keynote at STPCon 2011, Rex Black, former president of the ISQTB, talked about the importance of deriving metrics from defects, encouraging the audience to track defects throughout the development lifecycle. Meanwhile, James Whittaker, another industry expert, told testers in his STARWEST keynote to stop wasting time with bug reports (especially logging bugs that will never be fixed), and instead focus on the “only artifact that matters: the code.”

We explore more about how software quality is measured with a three-part interview with co-authors Capers Jones and Olivier Bonsignour, who have come out with a new book, The Economics of Software Quality. The book includes a list of 121 software quality attributes which are ranked according to how valuable or detrimental they are to software quality.

It appears Jones and Bonsignour would side with Black as they have “use of formal defect tracking” ranked as 10.00, indicating the highest degree of value for quality. This surprised me since I know many respected leaders, incuding Whittaker, who I’m guessing would disagree. I asked why formal defect tracking was so important to quality. Check out the Q&A to find out how they answered.

Quality metrics: The economics of software quality – Part one


Quality metrics: Software quality attributes and their rankings – Part two


Quality metrics: Changes in the way we measure quality – Part three

November 9, 2011  9:00 PM

Recruiters’ advice to testers: Get technical!

Posted by: Yvette Francino
James Whittaker, job search, Role of tester, Scott Barber, SQuAD, Starwest

The days of being able to get a job as a “manual tester” are no more. According to a panel of QA recruiters at a recent Denver SQuAD meeting, employers are looking for testers who have highly technical skills and specialize in a particular discipline such as security, performance or automation implementation. The panel, which included Bev Berry of Modis, Elias Cobb of Gunther Douglas and Samantha Schreiner of ProtoTest, unanimously agreed that the role of the tester is changing and that employers are expecting candidates who have at least some scripting experience or experience with some of the more popular automation tools, now including the open source tool, Selenium.

The advent of Agile development, of course, plays a part in creating the demand for more technical skills as testers are expected to work side-by-side with the development team, and those testers need to be able to understand code.

This message seemed to confirm much of what James Whittaker had to say at his somewhat controversial keynote at StarWest, “All That Testing is Getting in the Way of Quality.” Eric Jacobson did a great job of summarizing the keynote in his blog, and notes a common theme: “Programmers are getting better at testing, and testers are not getting better at programming.”

Jacobson’s blog also points out suggestions from Whittaker on tester survival that confirm what the recruiters were saying: “Get a specialty and become an expert in some niche of testing (e.g., Security, Internationalization, Accessibility, Privacy), or learn how to code.”

Scott Barber weighed in on the discussion with his own blog post, in which he said: “The state of the testing practice is not evolving nearly as quickly as development, business or products containing or depending on software.”

Should testers be concerned? Certainly Whittaker thinks so. He said to his audience of testers, “You are under threat more than you’ve ever been under threat before.” His keynote underscored the idea that testers must stop justifying the need for traditional testing and get busy learning how to contribute to the code, which he said was “the only artifact that anyone cares about.”

Barber answers the question of whether a tester should be concerned this way: “Only if they are afraid of change, have stagnated in their own professional development, and/or believe what they are doing today ‘is right and will continue to be right’ for at least as long as they will be testers.”

The recruiters, too, advise the testers to be prepared to grow and change. Taking an active part in professional development will make a difference. With all the open source tools available, all testers have the opportunity to learn new technologies, and doing so shows an interest and aptitude.

What if you don’t have the interest or aptitude? The recruiters say that there are growing needs for business analysts (or Product Owners for those who practice Scrum). Technical writing is also a possible career path for testers who are not interested in coding.

But let’s face it. Whether you are a programmer, a tester, a business analyst or a technical writer, if you work with software, things change, and they change at a fast pace. Don’t fight it. Embrace it. Be part of it.

Related articles on SSQ:
Is automated testing replacing the software tester?
Are coding or testing skills more important in the corporate world?
Software development: Benefits of pairing programmers with non-programmers

November 9, 2011  5:38 PM

Gorilla Logic announces automated testing tool for Android

Posted by: Melanie Webb
Android, mobility testing, open source tools

Recently Gorilla Logic announced the release of their newest automated testing tool, FoneMonkey for Android. I spoke with Stu Stern, Gorilla Logic CEO, about this new tool, and he explained the FoneMonkey tool suite in general, which is available free and open source. Now Android developers and testers will have access to the same automated testing features that iOS developers have already been using, which includes record and playback functional testing.

One of the features that differentiates FoneMonkey testing tools from others according to Stu is the high-level action recording. He described how this works, saying:

We record an actual action. So rather than saying, ‘Oh you dragged your finger on the screen,’ we are recording ‘oh, you scrolled the table to row 3.’ Obviously, that is much more readable; you can create a command from scratch for that recording to try to specify a slight command manually… This high-level action recording, because we are native, and we live so-to-speak right inside the application that is being tested, we’re able to interpret a slight gesture and understand that you are scrolling a table.

He also mentioned how these tools aid in the increasing practice of collaboration between testers and developers in Agile development. FoneMonkey tools have an interactive point-and-click console that records the app in a straightforward manner, requiring no prior programming knowledge, which enables testers to become a more active part of the team in many cases. The tests also generate native code, and they are increasingly generating JavaScript, appealing to a broader audience.

With this testing tool, developers and QA testers can create tests that perform user operation sequences and confirm the results. It is also compatible with continuous integration environments.

For other recent SSQ articles related to mobile application testing, see:

Smart phones: Implementing a test automation architecture

Mobile application testing: Five differences testers must take into account

Mobile applications: Testing and monitoring using SaaS solutions

October 27, 2011  8:45 PM

Software Test Professionals Conference (STPCon) coverage

Posted by: Yvette Francino

You know how when you go on a cruise you feel stuffed from all that delicious food that’s served around the clock? Well, that’s sort of how my brain feels at the end of a conference: happy and full!

I’ve been able to meet with so many test leaders here at STPCon and listen to some interesting presentations. As usual, I’ve been running around with my little camera, shooting videos of speakers. This time I wasn’t the only one with a camera. Stanton Champion of uTest was at the event, too, and I must admit, I think his videos outnumber mine. In fact, he even shot a video of me! It was kind of fun being on the other side of the camera!

Check out this short video clip to see Stanton’s view of the conference:
[kml_flashembed movie="" width="425" height="350" wmode="transparent" /]
And for more, take a look at our STPCon page for conference videos, exclusive interviews, and these tips from well-known experts Peter Walen, Matt Heusser, Scott Barber and Catherine Powell.

  • STPCon: Stepping up to leadership in software testing by Peter Walen
  • Software testing from the ground up by Matt Heusser
  • Performance testing in the Agile enterprise by Scott Barber
  • Software test professionals: Five tester personality types by Catherine Powell

  • October 27, 2011  3:11 AM

    Catherine Powell on testers’ personality types

    Posted by: Yvette Francino
    Catherine Powell, STPcon, tester personality

    Catherine Powell of Abakus is another SSQ contributor who I had the pleasure of meeting in person here at STPCon in Dallas. Catherine led an Open Jam session, an interactive learning event in which we played games and learned about the different personality types a tester might have. Catherine explains more about the personality types in Software test professionals: Five tester personality types and about the Open Jam session in the following short video clip:
    [kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

    October 26, 2011  1:26 AM

    STPCon: The tester’s role on an Agile team

    Posted by: Yvette Francino
    Agile adoption, Agile test, Robert Walsh, STPcon

    We hear a lot about teams transitioning to Agile, but what does this mean for the tester? Robert Walsh of Excalibur Solutions presented, “Adapting Conventional Testing Strategies for an Agile Environment” this morning at STPCon.

    [kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

    Rob started with a list of myths or misconceptions:

    • Agile doesn’t test.
    • Agile doesn’t need testers.
    • There’s no place in Agile for manual testing.
    • Agile doesn’t write documentation.
    • Agile doesn’t plan.
    • Agile has a public release at the end of each iteration.

    Agile methodologies do encourage planning, documentation and testing; however, unlike traditional methodologies, with Agile we want to do enough but not so much that we will have to do it again.

    If we are going to write that documentation, let’s do it once, and make it valuable, so we don’t need to do it again. Agile believes in ‘just enough’ but no more. If we don’t see a need for it, don’t do it.

    Traditional methodologies are notorious for requiring detailed documents throughout the lifecycle. Most of us know the pain of writing these documents and wondering if they’re ever even read. Walsh said that he heard of someone who stuck a brownie recipe in the middle of a document just to check if anyone was actually reading it. He was met with understanding nods from the audience.

    Though there were many other examples that Walsh gave of working in a more Agile way, the anecdote that I enjoyed most was one that I’ve often used as well.

    Walsh told the story of the family that cut the end off of roast for generations without really knowing why. It’s just that it was the way it had always been done. Upon investigation, it was learned that the practice started because Great Grandma’s roast pan was too small to fit a full roast.

    “Once upon a time there was a reason. If it’s not valuable, don’t do it.”

    We need to use our critical thinking skills and question why we are doing what we do. Is there a better way? We should never do things simply because it’s the way it’s always been done. And, in fact, Agile tells us to always be looking for ways to do things better.

    October 25, 2011  10:24 PM

    uTest and Mozilla partner to create a test case management tool

    Posted by: Yvette Francino
    announcement, SOASTA, STPcon, uTest

    Crowdsource test organization uTest has partnered with Mozilla to create a new test management system, CaseConductor™ . This system will allow testers to collaborate on test case creation, management and execution, regardless of where they are physically located.

    CaseConductor beta is now available for download in a repository, where developers and QA managers are encouraged to start leveraging the tool, view the source code, provide feedback on usability and ultimately help advance the development of this TCM system.

    I caught up with Chief Marketing Officer Matt Johnston, who delivered the lunchtime keynote at this year’s STPCon, “In-the-wild testing: Your survival may depend on it.” He has this to say about the annoucement:

    [kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

    But that’s not all. Another announcement made here at STPCon was the expanded partnership with SOASTA to include uTest’s “JumpStart” service for the SOASTA CloudTest Platform. Now customers using SOASTA CloudTest Lite can conduct both performance and automated functional testing using uTests crowdsourced service.

    October 25, 2011  9:37 PM

    Data testing on data warehouse projects

    Posted by: Yvette Francino
    Data Warehousing, DW/BI, Karen Johnson, STPcon

    When Karen Johnson told a barista that she was working on a data warehouse, the barista said, “Oh, where is it located?”

    While most of us that work with software realize that a data warehouse is sort of a mega-database, not a physical building, we still might feel a little lost when tasked with testing data on a data warehouse project.

    “A BI [Business Intelligence] project is like a foreign language. You can’t really see it. If you’re in management and you’re trying to staff for it, you’re not sure how to hire for it,” said Johnson, talking about the difficulty of trying to explain the skills needed for testing data on BI/DW systems.

    I attended Johnson’s presentation this morning at STPCon titled, “Data Testing on Business Intelligence and Data Warehouse Projects.” She gives a quick overview in the short clip below.

    [kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

    Johnson described a lot of the basics, including the term ETL: Extract, Transform, Load. She explained that this really was all about sucking data out of systems, transforming it to a common format, and then loading it into a database which will allow for some sophisticated analytics. One simple example she gave was with date formats. If you are working with multiple systems that are using different date formats, you would first extract the appropriate data, use an algorithm to transform the data into a consistent format, and then load it into a table in that common format so that reporting could be done.

    So what should a tester look for when testing data? Here are some bug opportunities:

    • data type mismatches
    • rounding
    • truncation
    • boundary conditions
    • data accuracy
    • is the data current
    • data refreshes
    • data security

    What about when you’re working in an Agile environment? Check out the interview I had with Ken Collier about his book, Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing.

    October 25, 2011  2:26 AM

    STPCon: Meet Lanette Creamer

    Posted by: Yvette Francino
    Lanette Creamer, STPcon

    Going to industry conferences is wonderful, not just because of the additional learning and professional development that’s available, but because of the connections we make. We go beyond recognizing a name; we have the opportunity to really get to know some of the people who we’ve heard speak or whose blogs or books we’ve read.

    I had the opportunity to really get to know one of the conference speakers, Agile test consultant Lanette Creamer. I’d done two pre-conference interviews with Lanette about her sessions, one about pairing programmers with non-programmers, and one about Agile games.

    [kml_flashembed movie="" width="425" height="350" wmode="transparent" /]

    But in order to personally get to know someone, I am always interested in finding out how they started in their careers and what led them to the place they are now. I asked Lanette about her background and what series of events led her to the place she is today. She answered:

    In the early 1990’s I was involved in a local bulletin board system and made a group of friends. They helped me put together a second-hand PC from parts at the computer swap meet for a little more than $200, all I could afford at the time, and the sysop (owner of the local BBS) put it together, and taught me enough DOS to get online to chat with my friends and play TeleArena.

    Back in those days, it was pretty unusual for females to be interested in technology. In a few years, when I graduated with my AA from the community college, I went to Western Washington University, majoring in Graphic Design, but my favorite part of design was in the use of the computer programs, especially desktop publishing and photo editing. I didn’t enjoy the competitive nature of graphic design, and frankly, I learned that while I wasn’t a terrible designer, I wasn’t that great either. Not awful, just not talented enough to make the cut at that time, in 1995. I wasn’t sure what to do.

    Honestly, I knew that I didn’t want to be a Graphic Designer at that point. I just wanted to be on the computer all day, every day, and I knew that. To this day my design background gives me an advantage in testing UI and usability aspects of software. I can identify font inconsistencies, and why something is attractive or not, in more description than before I had experience with graphic design. While I didn’t end up in the field of my major, I credit my love of the arts for getting me over the first barrier to move into a technology career. The creative part of testing remains my favorite.

    After college, I was working at a charity bingo parlor, and I was known as the “computer whiz” who made our manual spreadsheet into a self-calculating excel workbook. I was the person who knew that if you had just a “flashing c,” it would tell the panicked manager, “Type “win” and press enter.” Of course, in college I’d already fallen in love with the Mac OS, but something like that was far out of my bingo budget. It took me six months to save up my money to buy my used car in cash, and I’d made myself ill with food poisoning twice my last year of college trying to eat cheaply off of leftovers for too long. I make it sound awful, but besides the stress of not enough money, being poor wasn’t much of a problem. Those were some of the happiest years in my life to date. I had friends and family all around me, and I was in love and newly married. Best of all, I had good health. I felt good most days, and those I didn’t, I believed were temporary.

    I started in testing working on Adobe InDesign in 1999. I worked on several Adobe products, and ended up leading workflow testing across the Creative Suite products at Adobe. In 2010, I worked my first consulting job for Sogeti, on a project for a large Seattle based coffee company. It was quite a shift to go from a company that makes software to a company that makes something else, but needs the software to have a good user experience to efficiently deliver coffee. When the project for the coffee purveyor was completed, I had a great opportunity to work as a consultant and Agile testing coach for a great company working in the medical field. That is when I started Spark Quality LLC, my own consulting company! I’m currently consulting for a great company in California called Silicon Publishing, where I’m working with some amazing developers to create custom solutions for clients in the publishing, printing, and design industries.

    In consulting, I’m finding that meeting with a client to get a clear picture of their needs is a wonderful advantage if it can be arranged. The more we understand about our users, the easier it is to delight them and target our tests to reflect their top priorities in addition to risk. Getting feedback as a result of your testing from the client is what I love about working with a small company. As much as I loved working on the Adobe Creative Suites, there are some corporate reasons why it is more difficult legally to share demos with customers and have a free and open conversation when you are representing a publically held company. In the context of a small business, we can demo real code for the client just as soon as it is ready with fewer privacy concerns.

    Forgot Password

    No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

    Your password has been sent to: