Software Quality Insights

October 19, 2012  7:34 PM

Electric Cloud, Wind River team up for faster testing and deployment of Android apps

Melanie Luna Melanie Luna Profile: Melanie Luna

As the Android marketplace continues to grow at a rapid pace, Android developers face the problem of maintaining software quality while also meeting the time constraints for deployment. Electric Cloud, a DevOps optimization company, recently announced a partnership with mobile testing experts Wind River. This collaboration aims to ease the burden on Android developers by offering a solution that dramatically increases the speed of testing and deployment while upholding high quality standards.

In a recent conversation, Electric Cloud CEO Mike Maciag described how the company’s solutions accelerate the optimization process, and provide a framework to integrate disparate tools, as well as the reporting infrastructure needed to produce a cohesive view of the application.

Vice President of Product Marketing at Wind River, Ido Sarig, explained how Wind River has “reproduced reams and reams of log files that are very difficult to manually parse through. We’ve unified all those inside test management, providing you a single-click, single-point method to run tens of thousands of test cases and then consolidate all the test results and artifacts using a standard format in a single repository.”

Wind River has integrated its testing solutions with the Electric Cloud solution set, creating a seamless flow between development and testing processes. Maciag discussed the advantages of the partnership, explaining how Electric Cloud offers solutions that manage the complexity of the development lifecycle, while Wind River provides a homogeneous output of test data.

“The combination of the automation to deal with the complex matrix and the homogeneity brought by the Wind River test management suite takes a very complex environment, slows it down, makes it very simple and allows the process to move quickly,” Maciag said.

For related stories from, see:

Cloud-based tools, Agile and DevOps accelerate application deployment

Software quality and testing: Does the DevOps movement shortchange QA?

DevOps: Project management and leadership that align IT and business

October 18, 2012  1:58 PM

Requirements: Are we there yet?

jdjlent Profile: jdjlent

There is a tendency to want to plow through the requirements phase of an application development project, as in the sooner requirements are set, the sooner the coding, and then testing, can get underway. But the requirements process doesn’t proceed in a linear fashion, and nor should it, says business analyst Laura Brandenburg, founder of Clear Spring Business Analysis. “It’s a funny task. You go through the cycle quite a few times. You get closer and closer, but you’re not really done until you have a baseline spec.”

What makes the process nonlinear is that you are not actually “gathering” requirements—you are eliciting information that helps define an app’s features and functionality, says Brandenburg. Although the term “requirements gathering” still gets a lot of play, it isn’t really accurate, because “gathering” implies that requirements are sitting around fully formed just waiting for someone to pick them up. Elicitation, on the other hand, is a complex process of getting information from stakeholders, interviews, surveys and other sources, then analyzing and validating the information, says Brandenburg.

A good business analyst asks questions of stakeholders, listens actively, clarifies what she hears and works to get alignment among group members. When Brandenburg leads a group, she says things like: “I heard what you said. We have an idea on what we want. Now let’s see if we tear this apart.” This back and forth takes place before you actually write down a single requirement, she says. “When you are ready to write down the requirements, you are working from a shared understanding.”

Her requirements projects typically include business stakeholders and a project manager. As ideas begin to crystallize, Brandenburg brings a developer into the mix as well. Developers are most effective when they are willing to listen and understand the real problem we are trying to solve,” says Brandenburg. “You want to involve them in the creative part of the process. They are engaged, they offer options, helping me and the business understand that more things are possible.”

October 12, 2012  2:53 PM

Developer to tester: Cannot reproduce the error…

jdjlent Profile: jdjlent

How do test professionals deal with developers who insist there are no defects in their code? I talked about this with a test professional I met at the STARWEST Conference in Anaheim last week. This situation has come up so often for her that she has figured out some good ways to work around it. Here is her advice for other test professionals struggling to get things right with devs who give them a hard time.

Don’t get mad. It’s important to work effectively with developers otherwise you are hurting yourself. Devs don’t like it when you find a bug and they sometimes take it personally. Don’t retaliate by taking it personally, too.

Don’t use the word “defect.” One developer told me he hates that word because it sounds so harsh. So we just decided to call them something else – we came up with word “bunnies.”

Document the problem. Years ago I worked with a developer who always came back to me with “cannot reproduce” no matter what I found. When this happens, stay calm and document the problem in whatever tool you’re using in great detail. It’s was hard for her to refute the evidence.

• Get to know the devs you work with on a personal level. If you make an effort, you can always find some way to get along. We are all human beings. We are all on the same team.

October 5, 2012  6:15 PM

Fourteen ways it takes courage to be a good tester

Brein Matturro Profile: Brein Matturro

by Jennifer Lent

Courage may be an unlikely topic to come up at a testing conference. But in a Lightning Strikes the Keynotes session at STARWEST, Bob Galen, of RGalen Consulting, talked about the ways in which courage should guide the work of test professionals.

  1. The courage to try new things. You might fail, but go for it anyway.
  2. The courage to slow down. Experiment with a different approach to a test project. It will slow you down in the short run but it might be more efficient for the long haul.
  3. The courage to listen to your customer. Be willing to make changes.
  4. The courage to engage in reality during retrospectives. Tell the truth – not want people want to hear.
  5. The courage to challenge silo thinking. Encourage “swarming” around tasks instead of going it alone.
  6. The courage to apply craftsmanship to everything you do.
  7. The courage to push back on leadership and negotiate. Under what conditions would you say no to a project?
  8. The courage to give yourself a break. Take some slack time to think.
  9. The courage to be totally transparent.
  10. The courage to change. Don’t be stuck in your ways.
  11. The courage to build quality into your work.
  12. The courage to trust your team, trust your leaders, trust yourself.
  13.  The courage to allow others to shine. Give them the spotlight. It’s not about you.
  14. The courage to deliver courage in a graceful way so that you are effective. Courage needs grace.

October 5, 2012  6:10 PM

When software testing becomes unmanageable

Brein Matturro Profile: Brein Matturro

by Jennifer Lent

At a breakfast at the STARWEST 2012 conference I met tester Laurie Lantgen, who works at a financial services firm in Sioux Falls, S.D. I asked her about the most challenging aspects of her job, and here’s what she told me:  

“The most difficult situation I have dealt with in my 17 years as a tester was a project so huge, it was virtually unmanageable for one person. In addition to testing the code, I had to work with about a dozen business users the app had been developed for. My project manager wanted me to engage them to do some of the testing with me. In theory, usability testing is important, but this project was nowhere near ready for that. The code still had defects, and new requirements were being written and rewritten all the time. It was tough to get through — I worked a lot of hours. I reached the point where I stopped conducting sessions with the users and went back to working on my own. I told my manager that this is what I had to do. I mean, I’m the one who finds all the weird stuff.”

Lantgen finished the project successfully. But her story brings home some of the points Johanna Rothman made in her keynote address yesterday Becoming a Kick-*** Test Manager. Test managers shouldn’t put their people in unmanageable situations and they shouldn’t turn a blind eye when a project isn’t going well. Laurie Lantgen was savvy enough to find her own way out. But lots of people aren’t capable of doing that.

For more on Johanna Rothman’s keynote, see: Advice for test managers: How to develop, coach and lead your team

October 5, 2012  6:03 PM

Hackers at work: Security testing at STARWEST

Brein Matturro Profile: Brein Matturro

by Jennifer Lent

Frank Kim opened his STARWEST Conference session Security Testing: Think Like an Attacker by asking attendees how many of them were familiar with the concept of a cross-site scripting error. Virtually every hand in the room of 60 to 70 test professionals shot up. But when he asked how many had actually come across this security vulnerability during the testing process, only a handful said they had. 

Kim, who is a curriculum lead for computer security training organization the SANS Institute, wasn’t surprised by the response. “Testers think about functionality. They focus on what an application should do,” he said. But hackers concentrate on getting an app to behave in ways that weren’t intended. These days, testers need to assume the hacker mindset as well, said Kim, founder of software security consultancy ThinkSec. “Make one big assumption,” he told attendees, “everyone using your website is evil.”

In the session, Kim demonstrated how a hacker uses cross-site scripting, SQL injections and cross-site request forgeries to steal user names and passwords, and even get an online bank customer to unknowingly transfer money of her account into the hacker’s. It was powerful to watch the demonstration, to see the actual code the hacker was sending to the application, and the data he was pulling out. One attendee asked whether virus protection software prevents these kinds of errors. Kim said no.

Kim noted the widespread availability of application security tools – open source and commercial – designed to scan code and flag these vulnerabilities. He likes the tools well enough, but he said when introducing people to the concept, it’s more effective to give a live demonstration of how a hacker works. “It easy for software and test professionals to dismiss the results from the tools.”

For me, this was one of most interesting sessions at the conference. And it raises a bunch of questions. Will application security testing become a key process for test organizations? Will it move into the mainstream? How do we make that happen? I am interested to see how this all shakes out.

October 4, 2012  6:59 PM

STARWEST 2012: What keeps you from being an awesome test manager?

Melanie Luna Melanie Luna Profile: Melanie Luna

By Jennifer Lent

In her keynote address “Becoming a Kick-*** Test Manager” at STARWEST 2012, Johanna Rothman of the Rothman Consulting Group asked the audience: “What prevents you from being an awesome test manager?” She offered on-the-spot advice on how to deal with each situation.

Problem: Meetings. Laughter ensued and lots of hands went up when Rothman asked the audience who else had problems with meetings. Rothman’s solution to the dreaded meeting problem? “If the meeting has no agenda, you don’t have to go.” But you can’t just not show up, she said. “That wouldn’t be awesome.” Give 24-hour notice if you plan not to attend. And if you decide to go, make sure the meeting has action items associated with it before you accept the invitation. “If there aren’t any action items, what is the point of the meeting in the first place?” she wondered.

Problem: Resources are in short supply. When you don’t have enough testers to staff current projects, you have to prioritize. The key thing here is to staff only active projects. The worst thing you can do assign a tester to more than one project. Testers who multi-task don’t get anything done. The best way to convey this test project info to interested parties is to depict it graphically – essentially you’re showing them a picture of project portfolio. Here’s what we are doing now. Here are the projects that haven’t been staffed yet.

Problem: They keep telling me to hurry up and wait. Hurry-up-and-wait syndrome occurs when the business has a hard time understanding what exactly you are releasing, said Rothman. “Even when they decide what the priority is, they still have trouble deciding.” The only way to fix this is to release more often. You can do one-week iterations, or take the Kanban approach and release really small stories every one to two days, she said. “This is challenging, but just keep doing it; keep pumping out really small chunks of work.”

For more STARWEST conference coverage, see:

Agile leadership: Start supporting, stop ‘Dilbert-izing’

Mobile device testing: Placing testers in the field, not just the lab

Bid old test rules goodbye: New mind-set needed to test mobile apps

October 4, 2012  2:20 PM

STARWEST keynote address: Emotions matter in mobile testing

Melanie Luna Melanie Luna Profile: Melanie Luna

by Jennifer Lent

Of the many things that test professionals must take into account, the emotions of the person using the app isn’t usually one of them.  

But for mobile apps, emotions matter, test consultant Jonathan Kohl said in his keynote address “Tapping into Testing Mobile Applications,” at the STARWEST conference in Anaheim. “The emotions associated with the app going wrong are visceral. People get angry if the app doesn’t do what they need it to do.”

Kohl advised conference attendees to think about emotions as they test mobile apps. “Is it offering the right type of experience? What happens to the app when the user moves around and the connection gets weaker, when the device switches between networks, or – worst of all – encounters a dead spot?”

When mobile app users get angry, they can delete the offending app in about three seconds. “They press down the button and it’s gone.” And if they’re really, really mad, they will rant about it on Facebook.  And when your app becomes a “rantable offense,” that’s devastating, Kohl said.

For more on this topic, see Bid old test rules goodbye: New mind-set needed to test mobile apps.

October 2, 2012  5:03 PM

Serena survey reveals current state of Agile

Melanie Luna Melanie Luna Profile: Melanie Luna

At the recent Agile2012 conference, Serena Software conducted a survey amongst attendees to take the pulse of Agile development within the Agile community. Serena polled approximately 100 IT professionals, asking a few brief questions about how they were using Agile within their organizations, and how well they thought it was working.

One of the key takeaways highlighted by Serena’s Senior Product Marketing Manager, Miguel Tam, is that while overall development is doing well with Agile, “the problem seems to be more that once you extend the principles of Agile outside of the core development team, there are some potential pitfalls that companies need to be cautious about.”

Respondents cited communicating with customers more effectively as the biggest challenge; over 50% indicated that understanding and prioritizing customer demand needed the most improvement, according to the survey.

Another survey finding revealed that customers only have visibility into releases 45% of the time. “This one to me was pretty surprising because the whole concept of Agile is to really deliver what the customers are looking for, and also to eliminate the whole black box of IT of not knowing what’s going on,” said Tam. This lack of visibility indicates a need for Agile development teams to share information more readily, he said.

Serena offers three main recommendations in response to the survey results. Tam explained the first recommendation, saying, “Not everybody is Agile, and not everyone interprets Agile the same way. You need to think that there are different ways to communicate and orchestrate your processes, the way you talk to each other, common terminology so that you have Agile teams and non-Agile teams able to work in sync very closely.”

Other recommendations describe the importance of focusing on the entire lifecycle and remembering the mainframe. Details about the survey results and recommendations are included in the infographic.

To view the “There’s More to Agile than Development” infographic, click here.

For related stories, see:

Serena announces updates to Orchestrated IT tools, supporting IT agility

Agile2012 preview: Defining and delivering value with structured conversations

Agile2012 preview: Emerging strategies for Agile enterprises

September 20, 2012  2:45 PM

Vulnerability management tool ThreadFix facilitates development and security efforts

Melanie Luna Melanie Luna Profile: Melanie Luna

Application developers and security analysts can communicate and collaborate more easily using Denim Group’s new open source vulnerability management tool ThreadFix.

In a recent announcement, Denim Group explained that “ThreadFix imports the data from automated dynamic and static scanners as well as manual testing reports into a centralized platform.” This provides a single view into all application security vulnerabilities—information which is exported into a bug tracker tool that application developers are familiar with using.

Ultimately, ThreadFix decreased the time needed to repair software defects and uses a “virtual patch” in the form of a Web application firewall, to protect corporate assets while defects are being fixed.

“Denim Group’s ThreadFix is taking an innovative approach to application vulnerability management,” said principal analyst Eric Ogren of The Ogren Group. “ThreadFix’s normalization of data from multiple scanning sources brings much needed de-duplication to vulnerability reports, while the virtual patching of discovered application vulnerabilities significantly helps security teams protect corporate data from external threats. Organizations should look to technologies such as ThreadFix to accelerate the closing of dangerous security holes in applications.”

Dan Cornell, chief technology officer at Denim Group, added that ThreadFix is a “useful component of DevOps toolchain,” and that it enables teams much versatility when tools are able to communicate with each other.

In regards to cloud environments, he explained, “If an organization is using cloud-based testing providers– such as Veracode, WhiteHat or Qualys– they can use ThreadFix to pull data from those cloud providers’ APIs and merge it with results from other non-cloud-based security testing activities.”

Furthermore, Cornell said, “If an organization has both in-house-hosted applications as well as cloud-based providers where they need to do security testing for compliance purposes, they can use ThreadFix to store the results of the testing of cloud-based systems alongside the testing they perform for custom-developed applications.”

To read more about the recent release of ThreadFix, see ThreadFix: Open source defect management tool speeds security vulnerability fixes.

To learn more or to download ThreadFix, visit the Denim Group resource page.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: