Last month, I blogged about the different advice you can get from experts about how to measure software quality. I was chatting about the range of opinions on defect tracking with performance test guru Scott Barber, and he asked if I’d like to speak at the upcoming STP Online Summit: Delivering Value with Testing Metrics, Dec 13-15. Heck, yeah! I’m usually in the audience or facilitating one of these shindigs, so it’s a real honor to be included amongst such an expert team of presenters.
In preparation for my presentation, I wrote two articles outlining the arguments both for and against defect tracking:
- Quality metrics: Defect tracking throughout the software lifecycle
- Software quality: When defect tracking is not necessary
I also asked Paul McMahon, author of Integrating CMMI and Agile Development for his take on defect tracking, which he wrote about in his tip:
My research on the topic unveiled so many informative tips that we put together a metrics guide for our readers: Quality metrics: A guide to measuring software quality.
Take a look and let us know what you think!
A number of announcements have come out from crowdsource test group uTest in the past couple of years. Today the organization is celebrating as they complete a $17 million Series D round of funding. The company has achieved year-over-year growth of 250% over the past three years and is on track to become the first crowdsourcing company to file an IPO.
What makes uTest so successful? Well, take a look at CEO Doron Reuveni’s blog post for the stats on the history of their tremendous growth. As we heard from VP of marketing and community Matt Johnston in his keynote about “into the wild” testing at STPCon, SoLoMo (Social, local and mobile) has huge growth opportunities, and uTest is well-positioned to capitalize on those markets.
My opinion? It’s uTest’s great sense of community and their amazing and unique use of social media that contributes so greatly to their success.
Our SSQ theme for December is the use of social media in ALM and in this industry, uTest is a leader in using social media to build community amongst their 45,000+ group of testers and their customers, and they also lead the way in gathering data that will help enhance their products and services.
But what really makes the difference to me, personally, is the genuine sense of kindness I’ve felt from the leadership in this company, and that, of course, filters down to the employees who emulate that sense of community and generosity.
I first learned about uTest in October 2009. Having recently been laid off from Sun after the Oracle acquisition, I was just an unemployed blogger who had an interest in social media and QA. I was intrigued by the idea of crowdsource test and blogged about it. The blog post earned me a personal email from uTest’s community manager Peter Shih and an introduction to Matt Johnston, who agreed to be interviewed for my first podcast!
In January, 2010, I got this job as a Site Editor for SSQ at TechTarget, and now I have the privilege of talking to VIPs, including uTest CEO Doron Reuveni, regularly, but I have a very special place in my heart for those who talked to me when I was simply an unemployed blogger. Community managers and VPs are busy people. I understand that they need to spend their time judiciously and can’t talk to everyone. So when they take the time to talk to me — not because of my title or what I can do for them — simply because I ask… that means something!
But the real reason I love this organization is that they did something that touched my heart and will remain with me for the rest of my life. Earlier this year, my son went missing. When I put out the word on Twitter, my friends at uTest used their huge network to help spread the word of his disappearance. We found my son shortly after and I am eternally grateful to all those who helped me in that search. It was particularly heart-warming for me to get care, support and help from people who didn’t know me personally.
Though we all talk about the value of “face-to-face” relationships, there is something very special and touching when someone who has never met you reaches out to help you. And when your professional life mixes with your personal life, well, that’s what makes us human. Could that be the key that bonds us together and fosters a loyalty, perhaps even stronger than a face-to-face relationship?
Businesses are using social media in many ways. In order to be effective, they cannot simply market their products and promote themselves. They have to give back to the community in some way. uTest is successful for a lot of reasons. But what I believe is the secret ingredient behind the success is the genuine goodness that’s at the heart of their culture.
Congratulations uTest. You’re tops in my book.
“We think [performance testing] needs to be part of that continuous process,” said Ajit Sancheti , co-founder of Mu Dynamics, Inc. Today Mu Dynamics announced the integration of their Blitz cloud-based load test tool with Atlassian’s Bamboo, a Continuous Integration (CI) server.
The trend of integrating ALM tools seems to be growing, as vendors are exposing their APIs or putting in appropriate hooks to allow for data to flow seamlessly between tools. This seems to be especially true in the area of DevOps as demand grows for a faster release process using automated tests and continuous integration.
“When you look at how cloud and mobile applications are built, people are building the app, making changes 20 times a day,” says Sancheti. He believes that the simplicity in load testing with Blitz as well as the integrations with other tools has aided in its popularity.
The key for anything in the cloud is simplicity. If you start to impose complexity up front, people don’t adopt the solution. Even though we’ve stayed very, very simple, we don’t do the complex transactions, we see more and more people use it, because 80-90% of performance testing does not require the complexity.
Performance testing in the cloud continues to grow in popularity and does offer several benefits, including lowered capital and operational costs, and support for distributed development teams. Now, by integrating your load test into your CI solution, you can be sure with each build that any issues with performance are caught early, before your code hits production.
One of the many perks of my job as Site Editor for SSQ is that I often get the latest books on topics related to software quality or Agile practices. These are usually technical books — not the type you typically sit down and read cover to cover. However, Individuals and Interactions – An Agile Guide, by Ken Howard and Barry Rogers, is the exception.
I’ve always been interested in the people side of software development, and that’s what this book is all about. In fact, many of the principles described in the book are strong leadership practices that would apply for any team regardless of which software methodology was being used. Howard and Rogers explore the importance of strong leadership, communication, collaboration and teamwork using techniques such as self-directed teams and DISC analysis to understand communication styles.
So which would be more Agile? A military team with strong leadership and discipline or a Scrum team with a Scrum Master who imposes embarrassing penalties on team members? Find out what Howard and Rogers say in our two-part interview:
Last week, I got an interesting email from LinkedIn which read:
“On Veterans Day, 11/11/11, LinkedIn and the White House will join forces to kick off the first ever Veterans Hackday. We are looking for hackers to put together projects that can improve any aspect of a veteran’s life.”
Cool! I started a discussion on LinkedIn asking if anyone wanted to be part of a distributed team. Before I knew it, there were all kinds of responses from others who wanted to participate. After generating all this interest, I read the rules and realized that all development work had to take place between 11/11 and 11/14! Could an application be built by a team with varying skill sets in such a short time? Not only did we only have four days, but two of those days were work days. We all had jobs and other responsibilities that took priority.
Well, the short time frame did create a challenge, but in the end, thanks to a talented group who was willing to volunteer a good chunk of their weekend, everyone found his or her place. Rick got the ball rolling by creating and administrating an environment. Anand, who acted as development lead, created a Forum and Job Board for Veterans using Drupal. Ravi helped to build out the site and looked into a feature that would highlight hero’s whereabouts, and Dimas quickly set to work on a logo for our new site. Suann acted as product owner and used her social media connections to solicit feedback from Vets. Manasa stepped in to help test and to hunt down answers for the team. Doug provided DBA and leadership skills and took responsibility for documenting our team submission. I acted in the role of “ScrumMaster,” organizing conference calls and team tasks. Partha and others who wanted to help but had limited availability provided feedback and acted as “cheerleaders,” supporting the team’s efforts.
Though I was uncertain whether or not we’d really be able to pull anything off, in the end the team did it! It was fun for me to be part of this self-directed team and get a feel for how it works to be constrained to a challenging “time-box.” But even more fun for me was working with a distributed team in which none of us had ever met. I’ve read and written a lot about distributed versus co-located teams. It’s a common argument that a team needs face-time to build trust. Having this experience proved to me that a team can be successful despite the many challenges of working remotely. In fact, I have additional respect for these team members because their commitment was strong enough that they were willing to work as a team without the benefit of knowing one another at all. Even without meeting in person, there is a bond when a team has a common purpose and works together to achieve something.
Thanks to this competition, there’s a whole gallery of submissions of innovative applications for veterans. It’s amazing what can be accomplished in four short days! This is one contest where we’re all winners. Congratulations to all the participants who were able to deliver this special thanks to the veterans who serve our country.
“We hear a lot of talk about Agile in the media. We want to hear from you:
-Is Agile hype or helpful?
-Do you love Agile or hate it?
-Or is your team proceeding with business as usual?”
The survey is split into the following nine sections:
2. Organizational background
3. Software methodologies
4. Use of Agile (if applicable)
5. Use of Sprints (if applicable)
6. Use of Scrum (if applicable)
7. Platforms, tools, and organizational roles
9. Impressions of the Agile Manifesto
The survey will run through November 30 and will take about 15 minutes to complete. After taking it, you’ll have the opportunity to receive a free copy of the published report.
Founder of voke, inc Theresa Lanowitz has provided insights to SSQ on a variety of topics. Check out this Webcast about APM tools titled, Application Performance: The Intersection of Business and Technology.
Though software test experts do agree on a lot, the question of how to measure software quality is a subject of great debate. In his opening keynote at STPCon 2011, Rex Black, former president of the ISQTB, talked about the importance of deriving metrics from defects, encouraging the audience to track defects throughout the development lifecycle. Meanwhile, James Whittaker, another industry expert, told testers in his STARWEST keynote to stop wasting time with bug reports (especially logging bugs that will never be fixed), and instead focus on the “only artifact that matters: the code.”
We explore more about how software quality is measured with a three-part interview with co-authors Capers Jones and Olivier Bonsignour, who have come out with a new book, The Economics of Software Quality. The book includes a list of 121 software quality attributes which are ranked according to how valuable or detrimental they are to software quality.
It appears Jones and Bonsignour would side with Black as they have “use of formal defect tracking” ranked as 10.00, indicating the highest degree of value for quality. This surprised me since I know many respected leaders, incuding Whittaker, who I’m guessing would disagree. I asked why formal defect tracking was so important to quality. Check out the Q&A to find out how they answered.
The days of being able to get a job as a “manual tester” are no more. According to a panel of QA recruiters at a recent Denver SQuAD meeting, employers are looking for testers who have highly technical skills and specialize in a particular discipline such as security, performance or automation implementation. The panel, which included Bev Berry of Modis, Elias Cobb of Gunther Douglas and Samantha Schreiner of ProtoTest, unanimously agreed that the role of the tester is changing and that employers are expecting candidates who have at least some scripting experience or experience with some of the more popular automation tools, now including the open source tool, Selenium.
The advent of Agile development, of course, plays a part in creating the demand for more technical skills as testers are expected to work side-by-side with the development team, and those testers need to be able to understand code.
This message seemed to confirm much of what James Whittaker had to say at his somewhat controversial keynote at StarWest, “All That Testing is Getting in the Way of Quality.” Eric Jacobson did a great job of summarizing the keynote in his blog, and notes a common theme: “Programmers are getting better at testing, and testers are not getting better at programming.”
Jacobson’s blog also points out suggestions from Whittaker on tester survival that confirm what the recruiters were saying: “Get a specialty and become an expert in some niche of testing (e.g., Security, Internationalization, Accessibility, Privacy), or learn how to code.”
Scott Barber weighed in on the discussion with his own blog post, in which he said: “The state of the testing practice is not evolving nearly as quickly as development, business or products containing or depending on software.”
Should testers be concerned? Certainly Whittaker thinks so. He said to his audience of testers, “You are under threat more than you’ve ever been under threat before.” His keynote underscored the idea that testers must stop justifying the need for traditional testing and get busy learning how to contribute to the code, which he said was “the only artifact that anyone cares about.”
Barber answers the question of whether a tester should be concerned this way: “Only if they are afraid of change, have stagnated in their own professional development, and/or believe what they are doing today ‘is right and will continue to be right’ for at least as long as they will be testers.”
The recruiters, too, advise the testers to be prepared to grow and change. Taking an active part in professional development will make a difference. With all the open source tools available, all testers have the opportunity to learn new technologies, and doing so shows an interest and aptitude.
What if you don’t have the interest or aptitude? The recruiters say that there are growing needs for business analysts (or Product Owners for those who practice Scrum). Technical writing is also a possible career path for testers who are not interested in coding.
But let’s face it. Whether you are a programmer, a tester, a business analyst or a technical writer, if you work with software, things change, and they change at a fast pace. Don’t fight it. Embrace it. Be part of it.
Related articles on SSQ:
Is automated testing replacing the software tester?
Are coding or testing skills more important in the corporate world?
Software development: Benefits of pairing programmers with non-programmers
Recently Gorilla Logic announced the release of their newest automated testing tool, FoneMonkey for Android. I spoke with Stu Stern, Gorilla Logic CEO, about this new tool, and he explained the FoneMonkey tool suite in general, which is available free and open source. Now Android developers and testers will have access to the same automated testing features that iOS developers have already been using, which includes record and playback functional testing.
One of the features that differentiates FoneMonkey testing tools from others according to Stu is the high-level action recording. He described how this works, saying:
We record an actual action. So rather than saying, ‘Oh you dragged your finger on the screen,’ we are recording ‘oh, you scrolled the table to row 3.’ Obviously, that is much more readable; you can create a command from scratch for that recording to try to specify a slight command manually… This high-level action recording, because we are native, and we live so-to-speak right inside the application that is being tested, we’re able to interpret a slight gesture and understand that you are scrolling a table.
With this testing tool, developers and QA testers can create tests that perform user operation sequences and confirm the results. It is also compatible with continuous integration environments.
For other recent SSQ articles related to mobile application testing, see:
You know how when you go on a cruise you feel stuffed from all that delicious food that’s served around the clock? Well, that’s sort of how my brain feels at the end of a conference: happy and full!
I’ve been able to meet with so many test leaders here at STPCon and listen to some interesting presentations. As usual, I’ve been running around with my little camera, shooting videos of speakers. This time I wasn’t the only one with a camera. Stanton Champion of uTest was at the event, too, and I must admit, I think his videos outnumber mine. In fact, he even shot a video of me! It was kind of fun being on the other side of the camera!
Check out this short video clip to see Stanton’s view of the conference:
[kml_flashembed movie="http://www.youtube.com/v/oOLj33hDBXg" width="425" height="350" wmode="transparent" /]
And for more, take a look at our STPCon page for conference videos, exclusive interviews, and these tips from well-known experts Peter Walen, Matt Heusser, Scott Barber and Catherine Powell.