by Jennifer Lent
Frank Kim opened his STARWEST Conference session Security Testing: Think Like an Attacker by asking attendees how many of them were familiar with the concept of a cross-site scripting error. Virtually every hand in the room of 60 to 70 test professionals shot up. But when he asked how many had actually come across this security vulnerability during the testing process, only a handful said they had.
Kim, who is a curriculum lead for computer security training organization the SANS Institute, wasn’t surprised by the response. “Testers think about functionality. They focus on what an application should do,” he said. But hackers concentrate on getting an app to behave in ways that weren’t intended. These days, testers need to assume the hacker mindset as well, said Kim, founder of software security consultancy ThinkSec. “Make one big assumption,” he told attendees, “everyone using your website is evil.”
In the session, Kim demonstrated how a hacker uses cross-site scripting, SQL injections and cross-site request forgeries to steal user names and passwords, and even get an online bank customer to unknowingly transfer money of her account into the hacker’s. It was powerful to watch the demonstration, to see the actual code the hacker was sending to the application, and the data he was pulling out. One attendee asked whether virus protection software prevents these kinds of errors. Kim said no.
Kim noted the widespread availability of application security tools – open source and commercial – designed to scan code and flag these vulnerabilities. He likes the tools well enough, but he said when introducing people to the concept, it’s more effective to give a live demonstration of how a hacker works. “It easy for software and test professionals to dismiss the results from the tools.”
For me, this was one of most interesting sessions at the conference. And it raises a bunch of questions. Will application security testing become a key process for test organizations? Will it move into the mainstream? How do we make that happen? I am interested to see how this all shakes out.
By Jennifer Lent
In her keynote address “Becoming a Kick-*** Test Manager” at STARWEST 2012, Johanna Rothman of the Rothman Consulting Group asked the audience: “What prevents you from being an awesome test manager?” She offered on-the-spot advice on how to deal with each situation.
Problem: Meetings. Laughter ensued and lots of hands went up when Rothman asked the audience who else had problems with meetings. Rothman’s solution to the dreaded meeting problem? “If the meeting has no agenda, you don’t have to go.” But you can’t just not show up, she said. “That wouldn’t be awesome.” Give 24-hour notice if you plan not to attend. And if you decide to go, make sure the meeting has action items associated with it before you accept the invitation. “If there aren’t any action items, what is the point of the meeting in the first place?” she wondered.
Problem: Resources are in short supply. When you don’t have enough testers to staff current projects, you have to prioritize. The key thing here is to staff only active projects. The worst thing you can do assign a tester to more than one project. Testers who multi-task don’t get anything done. The best way to convey this test project info to interested parties is to depict it graphically – essentially you’re showing them a picture of project portfolio. Here’s what we are doing now. Here are the projects that haven’t been staffed yet.
Problem: They keep telling me to hurry up and wait. Hurry-up-and-wait syndrome occurs when the business has a hard time understanding what exactly you are releasing, said Rothman. “Even when they decide what the priority is, they still have trouble deciding.” The only way to fix this is to release more often. You can do one-week iterations, or take the Kanban approach and release really small stories every one to two days, she said. “This is challenging, but just keep doing it; keep pumping out really small chunks of work.”
For more STARWEST conference coverage, see:
by Jennifer Lent
Of the many things that test professionals must take into account, the emotions of the person using the app isn’t usually one of them.
But for mobile apps, emotions matter, test consultant Jonathan Kohl said in his keynote address “Tapping into Testing Mobile Applications,” at the STARWEST conference in Anaheim. “The emotions associated with the app going wrong are visceral. People get angry if the app doesn’t do what they need it to do.”
Kohl advised conference attendees to think about emotions as they test mobile apps. “Is it offering the right type of experience? What happens to the app when the user moves around and the connection gets weaker, when the device switches between networks, or – worst of all – encounters a dead spot?”
When mobile app users get angry, they can delete the offending app in about three seconds. “They press down the button and it’s gone.” And if they’re really, really mad, they will rant about it on Facebook. And when your app becomes a “rantable offense,” that’s devastating, Kohl said.
For more on this topic, see Bid old test rules goodbye: New mind-set needed to test mobile apps.
At the recent Agile2012 conference, Serena Software conducted a survey amongst attendees to take the pulse of Agile development within the Agile community. Serena polled approximately 100 IT professionals, asking a few brief questions about how they were using Agile within their organizations, and how well they thought it was working.
One of the key takeaways highlighted by Serena’s Senior Product Marketing Manager, Miguel Tam, is that while overall development is doing well with Agile, “the problem seems to be more that once you extend the principles of Agile outside of the core development team, there are some potential pitfalls that companies need to be cautious about.”
Respondents cited communicating with customers more effectively as the biggest challenge; over 50% indicated that understanding and prioritizing customer demand needed the most improvement, according to the survey.
Another survey finding revealed that customers only have visibility into releases 45% of the time. “This one to me was pretty surprising because the whole concept of Agile is to really deliver what the customers are looking for, and also to eliminate the whole black box of IT of not knowing what’s going on,” said Tam. This lack of visibility indicates a need for Agile development teams to share information more readily, he said.
Serena offers three main recommendations in response to the survey results. Tam explained the first recommendation, saying, “Not everybody is Agile, and not everyone interprets Agile the same way. You need to think that there are different ways to communicate and orchestrate your processes, the way you talk to each other, common terminology so that you have Agile teams and non-Agile teams able to work in sync very closely.”
Other recommendations describe the importance of focusing on the entire lifecycle and remembering the mainframe. Details about the survey results and recommendations are included in the infographic.
To view the “There’s More to Agile than Development” infographic, click here.
For related stories, see:
Application developers and security analysts can communicate and collaborate more easily using Denim Group’s new open source vulnerability management tool ThreadFix.
In a recent announcement, Denim Group explained that “ThreadFix imports the data from automated dynamic and static scanners as well as manual testing reports into a centralized platform.” This provides a single view into all application security vulnerabilities—information which is exported into a bug tracker tool that application developers are familiar with using.
Ultimately, ThreadFix decreased the time needed to repair software defects and uses a “virtual patch” in the form of a Web application firewall, to protect corporate assets while defects are being fixed.
“Denim Group’s ThreadFix is taking an innovative approach to application vulnerability management,” said principal analyst Eric Ogren of The Ogren Group. “ThreadFix’s normalization of data from multiple scanning sources brings much needed de-duplication to vulnerability reports, while the virtual patching of discovered application vulnerabilities significantly helps security teams protect corporate data from external threats. Organizations should look to technologies such as ThreadFix to accelerate the closing of dangerous security holes in applications.”
Dan Cornell, chief technology officer at Denim Group, added that ThreadFix is a “useful component of DevOps toolchain,” and that it enables teams much versatility when tools are able to communicate with each other.
In regards to cloud environments, he explained, “If an organization is using cloud-based testing providers– such as Veracode, WhiteHat or Qualys– they can use ThreadFix to pull data from those cloud providers’ APIs and merge it with results from other non-cloud-based security testing activities.”
Furthermore, Cornell said, “If an organization has both in-house-hosted applications as well as cloud-based providers where they need to do security testing for compliance purposes, they can use ThreadFix to store the results of the testing of cloud-based systems alongside the testing they perform for custom-developed applications.”
To read more about the recent release of ThreadFix, see ThreadFix: Open source defect management tool speeds security vulnerability fixes.
To learn more or to download ThreadFix, visit the Denim Group resource page.
voke, inc.’s recent survey of Agile use, discussed in two reports: “Market Snapshot Report: Agile Realities” and “Strategic Brief on the Cost of Rework for Agile and Non-Agile Projects,” caused a bit of controversy in the Agile world.
Ellen Gottesdiener, a seasoned Agilist and consultant at EBG Consulting, took issue with some of voke’s conclusions, though she affirmed that she did not have access to the specific survey questions or answers. She shared her thoughts in a recent interview.
One of the survey’s findings, referenced in TechTarget’s Executive Editor Jan Stafford’s first article in a two-part series, Software defects increase cost of Agile projects, is that “In effect, Agile embraces the fundamental realignment of business ownership of requirements to developer ownership of requirements through frequent changes in source code,” according to voke co-founder Lisa Dronzek.
Gottesdiener disagrees. “That is not what Agile is about. It’s in fact the opposite. It’s not true that you’re realigning business ownership of requirements. There’s a huge burden and responsibility of ownership of the requirements on the business.” She emphasized that the business responsibility when it comes to requirements is actually extremely valuable.
“I’ve found that there’s a tremendous amount of discipline in Agile and responsibility that the business has about what needs to be built and when it needs to be built.”
Another assertion, cited in Stafford’s article about documentation, is a misconception according to Gottesdiener: “Agile values documentation less while advocating self-organizing teams and a constant pace of development,” said Dronzek. “This creates a huge problem for ongoing maintenance when individuals or teams move on to a new project and are unavailable to help support a prior project that has little or no documentation.”
Gottesdiener explained that while there is a myth that Agilists don’t care about documentation, the truth is “just like all other practices, documentation has to be calibrated for the situation at hand.”
“If the team is going to develop and not maintain their own product—which, right there is a questionable practice because you learn a lot by maintaining your own product—then, there needs to be an understanding by the business that it is likely to be more costly in the long run to maintain a product without useful and usable documentation. Therefore, part of each delivery of the product should include maintenance-related documentation.”
In regards to evaluating QA practices, the article mentions: “Another key factor in cost of quality is how early QA is started in a development project. In the voke survey, most enterprises (71%) reported not starting QA testing at a project’s beginning, during requirements definition.”
“Then they’re not implementing Agile appropriately,” Gottesdiener responded. Thinking about how to validate that the team is building the right product and verifying that it is built correctly is everyone’s responsibility, she explained.
One point that Gottesdiener does agree with is “The importance of laser focus on requirements is what comes out of this survey,” said co-founder Theresa Lanowitz. “Requirements are at the center of every successful or unsuccessful software project, regardless of the style of development.”
The focus on requirements is essential, though challenging, explained Gottesdiener. “I think that Agile practices, when you apply them correctly, help you do a much better job with requirements and business analysis.”
In the second article of the series, The Agile method remains confusing for software professionals, Lanowitz and Dronzek call into question the standards of Agile as described in the Agile Manifesto.
“Building software is complex. The best organizations will have some standard practices that they know are good practices,” said Gottesdiener. She explained that the assumption that there can be a standard set of practices which apply across all organizations implementing Agile is mistaken. The practices must be adapted to suit the specific needs within the context of what each organization is creating. “There is no such thing as best practices. There are good practices in context.”
Business and IT are becoming more closely linked and collaborative. Serena announced Monday that they have updated their Orchestrated IT tools, which facilitate IT agility and enable enterprises to better utilize mobile, social and cloud technologies.
In an interview with David Hurwitz, senior vice president of worldwide marketing at Serena, he discussed the nature of DevOps in today’s organizations: “We find that in mid-sized to larger enterprises that are not pure dot-coms, nobody really has DevOps on their business cards. It’s more of a mindset that needs to get supported through live artifacts and by automating the whole pre-release and release-to-production flow.”
He added that “release management is where the rubber meets the road in DevOps,” highlighting the new Serena Release Manager product, which adds development-driven release management to Serena’s existing operations-driven release management support tools.
Other updates include Serena Service Manager, designed to support social IT collaboration; Serena Mobile Request Center, which takes advantage of mobile device capabilities; and Serena Mobile Dashboard, which enables users to perform IT analytics on the iPad.
Hurwitz commented that the phrase “online Agile business” doesn’t just mean Agile as in “Agile development,” but agile in a more general sense. Collaboration between business and IT is more important than ever, as “IT has finally reached the promised land where they are intrinsic to what the business needs to do,” he said.
Major players such as IBM Rational and CA offer competitive products, but “Serena is the only one to approach IT management from an orchestrated point of view. Human workflow automation and integration with existing systems are what differentiate orchestrated IT from other systems,” Hurwitz explained.
In conjunction with the release of these updated solutions, Serena is holding the “IT Agility Deserves a Beer” Haiku Twitter Contest. The winner will receive beer for a year. For details, click here.
HP has embraced the concept of DevOps. “The idea of creating true agility” is to ensure that applications function exactly the way they were designed to, explained Matt Morgan, VP of hybrid IT and cloud, product marketing, HP software, in a recent interview.
Earlier this summer, HP announced the release of enhanced versions of HP Application Lifecycle Management (ALM) and HP Performance Center (PC). These offerings facilitate continuous delivery of applications, which enable visibility, collaboration and quality in a DevOps environment.
Morgan highlighted a continued emphasis on quality: “Quality has always been a core principle. It’s a core principle whether you’re building a mainframe app, a client server app, a Web app or you’re maintaining apps that are 20 years old. What we’re finding is that while the practice of building software has changed, and is changing, the need to ensure quality has become more important.”
The role of the project manager in DevOps is different than in traditional models. PMs can digitize the key performance indicators, determining success early by looking at a digital dashboard. HP’s approach focuses on providing this type of visibility through an executive scorecard that enables the project manager to track information and disseminate information amongst team members.
“HP believes that performance is a key tenet to quality,” Morgan said. Basically all consumers have high expectations for performance. He explained that HP seeks to facilitate the collaboration between the two sides—development and operations—by creating an application performance management solution that operates bi-directionally, enabling information to travel back and forth from pre-production to resolve defects.
As far as the relationship between agility and quality, Morgan asserted, “There is a lot of concern that QA professionals have that agility means less quality. And it is the absolute opposite. People are betting their business in a more key way on business software than ever before. And that software cannot fail.”
“The idea of quality being in at any point marginalized needs to be flipped on its head. Actually, there has been a spike in the importance of quality. I think this is a call to all quality assurance professionals to elevate their conversations, both internally and externally, about how key quality becomes when competition is just a click away,” Morgan concluded.
For recent stories on DevOps, see:
SearchSoftwareQuality.com has offered a wide variety of coverage on Agile in the enterprise this month, coinciding with the Agile2012 conference last week in Dallas, Texas. Here is a recap of conference coverage:
Mike Cottmeyer, enterprise Agile coach at LeadingAgile, LLC, gave a preview of what to expect from both his Wednesday talk, “Patterns for Agile Adoption and Transformation,” and his Thursday talk, “Understanding Agile Program and Portfolio Management.” Each offers strategies for managers to adopt Agile and change the structure of project management within the organization.
Structured conversations enable team members to communicate more effectively and meet delivery expectations, according to Ellen Gottesdiener, principal consultant and Mary Gorman, VP of quality and delivery at EBG Consulting, who co-presented “The Product Partnership: Using Structured Conversations to Deliver Value.”
Consultants Ellen Gottesdiener and Mary Gorman provide insight into logical strategies that aid in team communication in Agile organizations that include structured conversations in this article, which explores these ideas more in-depth.
How can Lean-Agile principles help guide teams in successfully adopting Agile in the enterprise? Alan Shalloway, founder and CEO of NetObjectives, presented “Scaling Agile with Multiple Teams: Using Lean to Drive Business Value.” His presentation highlighted effective principles as well as case studies that demonstrate successful implementations.
Agile practitioners who seek to improve their skills have a new training option available to them. At Agile2012, Emergn announced its launch of Value Flow Quality (VFQ) Education, a work-based learning program that enables project managers, developers, testers and other team members to complete self-driven training, immediately applying skills while working on their current projects.
Shunra recently announced a new mobile app performance testing tool, vCat for Mobile, which expands their existing vCat technology to mobile devices. Dave Berg, senior director of product management and Shunra COO Bill Varga, offered details on how this tool works and the benefits it provides.
In developing vCat for Mobile, Shunra added two new improvements, including an enhanced API that allows the ability to remotely control the product through Web services. “You can control it through a phone, or another application, which makes the total cost of ownership go down and simplifies different use cases,” said Berg.
They also added the capability to virtualize more than one network on one piece of software, supporting up to 10 simultaneous networks.
Analysis has shown that vCat for Mobile improves mobile app performance by over 40%. Berg explained that Shunra tested the product by cloning very popular websites and implementing vCat recommendations to see how performance improved. The controlled environment of vCat technology enables companies to experiment with various changes before implementing them across their applications.
Varga explained the value of pre-deployment testing, warning that “just a 250 millisecond delay is enough to lose customers to a competitive offering and damage company brand. With vCat for Mobile, organizations now have a powerful tool to measure, evaluate and optimize their applications to ensure a positive customer experience.”
He discussed how Shunra uses discovery tools called NetworkCatcher that provide primary and secondary tool options for specific use conditions. NetworkCatcher was designed specifically for application performance engineering, Berg added.
For more on application performance engineering, see the white paper “Mobile Application Performance Engineering: A lifecycle approach to achieving confidence in application performance.”
Ultimately, vCat for Mobile offers companies a strong ROI and reduction in total cost of ownership. “Pre-production helps avoid significant remediation expenses for all of our customers,” concluded Varga.