How do test professionals deal with developers who insist there are no defects in their code? I talked about this with a test professional I met at the STARWEST Conference in Anaheim last week. This situation has come up so often for her that she has figured out some good ways to work around it. Here is her advice for other test professionals struggling to get things right with devs who give them a hard time.
• Don’t get mad. It’s important to work effectively with developers otherwise you are hurting yourself. Devs don’t like it when you find a bug and they sometimes take it personally. Don’t retaliate by taking it personally, too.
• Don’t use the word “defect.” One developer told me he hates that word because it sounds so harsh. So we just decided to call them something else – we came up with word “bunnies.”
• Document the problem. Years ago I worked with a developer who always came back to me with “cannot reproduce” no matter what I found. When this happens, stay calm and document the problem in whatever tool you’re using in great detail. It’s was hard for her to refute the evidence.
• Get to know the devs you work with on a personal level. If you make an effort, you can always find some way to get along. We are all human beings. We are all on the same team.
by Jennifer Lent
Courage may be an unlikely topic to come up at a testing conference. But in a Lightning Strikes the Keynotes session at STARWEST, Bob Galen, of RGalen Consulting, talked about the ways in which courage should guide the work of test professionals.
- The courage to try new things. You might fail, but go for it anyway.
- The courage to slow down. Experiment with a different approach to a test project. It will slow you down in the short run but it might be more efficient for the long haul.
- The courage to listen to your customer. Be willing to make changes.
- The courage to engage in reality during retrospectives. Tell the truth – not want people want to hear.
- The courage to challenge silo thinking. Encourage “swarming” around tasks instead of going it alone.
- The courage to apply craftsmanship to everything you do.
- The courage to push back on leadership and negotiate. Under what conditions would you say no to a project?
- The courage to give yourself a break. Take some slack time to think.
- The courage to be totally transparent.
- The courage to change. Don’t be stuck in your ways.
- The courage to build quality into your work.
- The courage to trust your team, trust your leaders, trust yourself.
- The courage to allow others to shine. Give them the spotlight. It’s not about you.
- The courage to deliver courage in a graceful way so that you are effective. Courage needs grace.
by Jennifer Lent
At a breakfast at the STARWEST 2012 conference I met tester Laurie Lantgen, who works at a financial services firm in Sioux Falls, S.D. I asked her about the most challenging aspects of her job, and here’s what she told me:
“The most difficult situation I have dealt with in my 17 years as a tester was a project so huge, it was virtually unmanageable for one person. In addition to testing the code, I had to work with about a dozen business users the app had been developed for. My project manager wanted me to engage them to do some of the testing with me. In theory, usability testing is important, but this project was nowhere near ready for that. The code still had defects, and new requirements were being written and rewritten all the time. It was tough to get through — I worked a lot of hours. I reached the point where I stopped conducting sessions with the users and went back to working on my own. I told my manager that this is what I had to do. I mean, I’m the one who finds all the weird stuff.”
Lantgen finished the project successfully. But her story brings home some of the points Johanna Rothman made in her keynote address yesterday Becoming a Kick-*** Test Manager. Test managers shouldn’t put their people in unmanageable situations and they shouldn’t turn a blind eye when a project isn’t going well. Laurie Lantgen was savvy enough to find her own way out. But lots of people aren’t capable of doing that.
For more on Johanna Rothman’s keynote, see: Advice for test managers: How to develop, coach and lead your team
by Jennifer Lent
Frank Kim opened his STARWEST Conference session Security Testing: Think Like an Attacker by asking attendees how many of them were familiar with the concept of a cross-site scripting error. Virtually every hand in the room of 60 to 70 test professionals shot up. But when he asked how many had actually come across this security vulnerability during the testing process, only a handful said they had.
Kim, who is a curriculum lead for computer security training organization the SANS Institute, wasn’t surprised by the response. “Testers think about functionality. They focus on what an application should do,” he said. But hackers concentrate on getting an app to behave in ways that weren’t intended. These days, testers need to assume the hacker mindset as well, said Kim, founder of software security consultancy ThinkSec. “Make one big assumption,” he told attendees, “everyone using your website is evil.”
In the session, Kim demonstrated how a hacker uses cross-site scripting, SQL injections and cross-site request forgeries to steal user names and passwords, and even get an online bank customer to unknowingly transfer money of her account into the hacker’s. It was powerful to watch the demonstration, to see the actual code the hacker was sending to the application, and the data he was pulling out. One attendee asked whether virus protection software prevents these kinds of errors. Kim said no.
Kim noted the widespread availability of application security tools – open source and commercial – designed to scan code and flag these vulnerabilities. He likes the tools well enough, but he said when introducing people to the concept, it’s more effective to give a live demonstration of how a hacker works. “It easy for software and test professionals to dismiss the results from the tools.”
For me, this was one of most interesting sessions at the conference. And it raises a bunch of questions. Will application security testing become a key process for test organizations? Will it move into the mainstream? How do we make that happen? I am interested to see how this all shakes out.
By Jennifer Lent
In her keynote address “Becoming a Kick-*** Test Manager” at STARWEST 2012, Johanna Rothman of the Rothman Consulting Group asked the audience: “What prevents you from being an awesome test manager?” She offered on-the-spot advice on how to deal with each situation.
Problem: Meetings. Laughter ensued and lots of hands went up when Rothman asked the audience who else had problems with meetings. Rothman’s solution to the dreaded meeting problem? “If the meeting has no agenda, you don’t have to go.” But you can’t just not show up, she said. “That wouldn’t be awesome.” Give 24-hour notice if you plan not to attend. And if you decide to go, make sure the meeting has action items associated with it before you accept the invitation. “If there aren’t any action items, what is the point of the meeting in the first place?” she wondered.
Problem: Resources are in short supply. When you don’t have enough testers to staff current projects, you have to prioritize. The key thing here is to staff only active projects. The worst thing you can do assign a tester to more than one project. Testers who multi-task don’t get anything done. The best way to convey this test project info to interested parties is to depict it graphically – essentially you’re showing them a picture of project portfolio. Here’s what we are doing now. Here are the projects that haven’t been staffed yet.
Problem: They keep telling me to hurry up and wait. Hurry-up-and-wait syndrome occurs when the business has a hard time understanding what exactly you are releasing, said Rothman. “Even when they decide what the priority is, they still have trouble deciding.” The only way to fix this is to release more often. You can do one-week iterations, or take the Kanban approach and release really small stories every one to two days, she said. “This is challenging, but just keep doing it; keep pumping out really small chunks of work.”
For more STARWEST conference coverage, see:
by Jennifer Lent
Of the many things that test professionals must take into account, the emotions of the person using the app isn’t usually one of them.
But for mobile apps, emotions matter, test consultant Jonathan Kohl said in his keynote address “Tapping into Testing Mobile Applications,” at the STARWEST conference in Anaheim. “The emotions associated with the app going wrong are visceral. People get angry if the app doesn’t do what they need it to do.”
Kohl advised conference attendees to think about emotions as they test mobile apps. “Is it offering the right type of experience? What happens to the app when the user moves around and the connection gets weaker, when the device switches between networks, or – worst of all – encounters a dead spot?”
When mobile app users get angry, they can delete the offending app in about three seconds. “They press down the button and it’s gone.” And if they’re really, really mad, they will rant about it on Facebook. And when your app becomes a “rantable offense,” that’s devastating, Kohl said.
For more on this topic, see Bid old test rules goodbye: New mind-set needed to test mobile apps.
At the recent Agile2012 conference, Serena Software conducted a survey amongst attendees to take the pulse of Agile development within the Agile community. Serena polled approximately 100 IT professionals, asking a few brief questions about how they were using Agile within their organizations, and how well they thought it was working.
One of the key takeaways highlighted by Serena’s Senior Product Marketing Manager, Miguel Tam, is that while overall development is doing well with Agile, “the problem seems to be more that once you extend the principles of Agile outside of the core development team, there are some potential pitfalls that companies need to be cautious about.”
Respondents cited communicating with customers more effectively as the biggest challenge; over 50% indicated that understanding and prioritizing customer demand needed the most improvement, according to the survey.
Another survey finding revealed that customers only have visibility into releases 45% of the time. “This one to me was pretty surprising because the whole concept of Agile is to really deliver what the customers are looking for, and also to eliminate the whole black box of IT of not knowing what’s going on,” said Tam. This lack of visibility indicates a need for Agile development teams to share information more readily, he said.
Serena offers three main recommendations in response to the survey results. Tam explained the first recommendation, saying, “Not everybody is Agile, and not everyone interprets Agile the same way. You need to think that there are different ways to communicate and orchestrate your processes, the way you talk to each other, common terminology so that you have Agile teams and non-Agile teams able to work in sync very closely.”
Other recommendations describe the importance of focusing on the entire lifecycle and remembering the mainframe. Details about the survey results and recommendations are included in the infographic.
To view the “There’s More to Agile than Development” infographic, click here.
For related stories, see:
Application developers and security analysts can communicate and collaborate more easily using Denim Group’s new open source vulnerability management tool ThreadFix.
In a recent announcement, Denim Group explained that “ThreadFix imports the data from automated dynamic and static scanners as well as manual testing reports into a centralized platform.” This provides a single view into all application security vulnerabilities—information which is exported into a bug tracker tool that application developers are familiar with using.
Ultimately, ThreadFix decreased the time needed to repair software defects and uses a “virtual patch” in the form of a Web application firewall, to protect corporate assets while defects are being fixed.
“Denim Group’s ThreadFix is taking an innovative approach to application vulnerability management,” said principal analyst Eric Ogren of The Ogren Group. “ThreadFix’s normalization of data from multiple scanning sources brings much needed de-duplication to vulnerability reports, while the virtual patching of discovered application vulnerabilities significantly helps security teams protect corporate data from external threats. Organizations should look to technologies such as ThreadFix to accelerate the closing of dangerous security holes in applications.”
Dan Cornell, chief technology officer at Denim Group, added that ThreadFix is a “useful component of DevOps toolchain,” and that it enables teams much versatility when tools are able to communicate with each other.
In regards to cloud environments, he explained, “If an organization is using cloud-based testing providers– such as Veracode, WhiteHat or Qualys– they can use ThreadFix to pull data from those cloud providers’ APIs and merge it with results from other non-cloud-based security testing activities.”
Furthermore, Cornell said, “If an organization has both in-house-hosted applications as well as cloud-based providers where they need to do security testing for compliance purposes, they can use ThreadFix to store the results of the testing of cloud-based systems alongside the testing they perform for custom-developed applications.”
To read more about the recent release of ThreadFix, see ThreadFix: Open source defect management tool speeds security vulnerability fixes.
To learn more or to download ThreadFix, visit the Denim Group resource page.
voke, inc.’s recent survey of Agile use, discussed in two reports: “Market Snapshot Report: Agile Realities” and “Strategic Brief on the Cost of Rework for Agile and Non-Agile Projects,” caused a bit of controversy in the Agile world.
Ellen Gottesdiener, a seasoned Agilist and consultant at EBG Consulting, took issue with some of voke’s conclusions, though she affirmed that she did not have access to the specific survey questions or answers. She shared her thoughts in a recent interview.
One of the survey’s findings, referenced in TechTarget’s Executive Editor Jan Stafford’s first article in a two-part series, Software defects increase cost of Agile projects, is that “In effect, Agile embraces the fundamental realignment of business ownership of requirements to developer ownership of requirements through frequent changes in source code,” according to voke co-founder Lisa Dronzek.
Gottesdiener disagrees. “That is not what Agile is about. It’s in fact the opposite. It’s not true that you’re realigning business ownership of requirements. There’s a huge burden and responsibility of ownership of the requirements on the business.” She emphasized that the business responsibility when it comes to requirements is actually extremely valuable.
“I’ve found that there’s a tremendous amount of discipline in Agile and responsibility that the business has about what needs to be built and when it needs to be built.”
Another assertion, cited in Stafford’s article about documentation, is a misconception according to Gottesdiener: “Agile values documentation less while advocating self-organizing teams and a constant pace of development,” said Dronzek. “This creates a huge problem for ongoing maintenance when individuals or teams move on to a new project and are unavailable to help support a prior project that has little or no documentation.”
Gottesdiener explained that while there is a myth that Agilists don’t care about documentation, the truth is “just like all other practices, documentation has to be calibrated for the situation at hand.”
“If the team is going to develop and not maintain their own product—which, right there is a questionable practice because you learn a lot by maintaining your own product—then, there needs to be an understanding by the business that it is likely to be more costly in the long run to maintain a product without useful and usable documentation. Therefore, part of each delivery of the product should include maintenance-related documentation.”
In regards to evaluating QA practices, the article mentions: “Another key factor in cost of quality is how early QA is started in a development project. In the voke survey, most enterprises (71%) reported not starting QA testing at a project’s beginning, during requirements definition.”
“Then they’re not implementing Agile appropriately,” Gottesdiener responded. Thinking about how to validate that the team is building the right product and verifying that it is built correctly is everyone’s responsibility, she explained.
One point that Gottesdiener does agree with is “The importance of laser focus on requirements is what comes out of this survey,” said co-founder Theresa Lanowitz. “Requirements are at the center of every successful or unsuccessful software project, regardless of the style of development.”
The focus on requirements is essential, though challenging, explained Gottesdiener. “I think that Agile practices, when you apply them correctly, help you do a much better job with requirements and business analysis.”
In the second article of the series, The Agile method remains confusing for software professionals, Lanowitz and Dronzek call into question the standards of Agile as described in the Agile Manifesto.
“Building software is complex. The best organizations will have some standard practices that they know are good practices,” said Gottesdiener. She explained that the assumption that there can be a standard set of practices which apply across all organizations implementing Agile is mistaken. The practices must be adapted to suit the specific needs within the context of what each organization is creating. “There is no such thing as best practices. There are good practices in context.”
Business and IT are becoming more closely linked and collaborative. Serena announced Monday that they have updated their Orchestrated IT tools, which facilitate IT agility and enable enterprises to better utilize mobile, social and cloud technologies.
In an interview with David Hurwitz, senior vice president of worldwide marketing at Serena, he discussed the nature of DevOps in today’s organizations: “We find that in mid-sized to larger enterprises that are not pure dot-coms, nobody really has DevOps on their business cards. It’s more of a mindset that needs to get supported through live artifacts and by automating the whole pre-release and release-to-production flow.”
He added that “release management is where the rubber meets the road in DevOps,” highlighting the new Serena Release Manager product, which adds development-driven release management to Serena’s existing operations-driven release management support tools.
Other updates include Serena Service Manager, designed to support social IT collaboration; Serena Mobile Request Center, which takes advantage of mobile device capabilities; and Serena Mobile Dashboard, which enables users to perform IT analytics on the iPad.
Hurwitz commented that the phrase “online Agile business” doesn’t just mean Agile as in “Agile development,” but agile in a more general sense. Collaboration between business and IT is more important than ever, as “IT has finally reached the promised land where they are intrinsic to what the business needs to do,” he said.
Major players such as IBM Rational and CA offer competitive products, but “Serena is the only one to approach IT management from an orchestrated point of view. Human workflow automation and integration with existing systems are what differentiate orchestrated IT from other systems,” Hurwitz explained.
In conjunction with the release of these updated solutions, Serena is holding the “IT Agility Deserves a Beer” Haiku Twitter Contest. The winner will receive beer for a year. For details, click here.