quality assurance

CAST 2009: The challenges of regulation, an interview with Jean Ann Harrison

Veteran software testing and quality assurance pro Jean Ann Harrison will be presenting a software testing case study based on her experiences at a medical device study at this year’s Conference for the Association for Software Testing (CAST), slated for July 13-16th in Colorado Springs.

In her session — titled “A Balancing Act: Satisfying Regulators, End Users, Business Leaders and Development” — Harrison plans to provide guidance to testers who have to deal with conflicting priorities between developers, project managers, customers/patients, and regulators.

“Priorities clash and inevitably software testers are in the middle of a battlefield between developers trying to get their work done and delivered while project managers are trying to make a deadline, customers/patients want to make sure the product works as expected and the regulators demand the proper documentation delivered in a sequential timeframe.” Harrison went on to share some of the questions she hopes to answer in the talk. “How do testers balance all these shareholder’s priorities? How can testers decide which direction to take as the project matures? Which shareholders’ take precedence over another?”

With 10 years of experience in software quality assurance and testing and three years testing embedded software on portable devices, Jean Ann Harrison has gained broad experience in various software testing processes and has worked in varied contexts including large multi-million dollar corporations, venture capital, and start-up companies. Harrison is currently works for CardioNet, Inc. where her primary role is testing software embedded in medical devices that provide diagnostic data for physicians to determine their patient’s heart condition.

“I developed the talk through my own learning process as a software tester working in a regulated environment for the first time. What to do, what not to do, what one can expect, and how to handle the demands of a regulated company helped formulate my subject. And most companies producing software usually have some sort of description of what is wanted, needed, or expected when the project is completed. Most companies usually have some method of traceability of software requirements, software design, and product information. In a regulated environment, the role of documentation is the centerpiece of any project.”

When asked to expand a bit on the challenges of regulation, Harrison continued:

“First, a single source location for documentation must be identified, implemented and then monitored. Then documentation is distinct in a regulated environment by the level of detail provided, the sequence of submittal of documentation, identifying appropriate reviewers to approve the documentation, and a historical record is maintained for traceability purposes. This process is extremely formalized and dates of submittals are critical to the project. Non-regulated environments tend to be more relaxed and even the most formal processes have allowable slips. In regulated environments, slips are not acceptable, and contingency plans must be implemented to explain deviations. If regulated environments do not meet regulators demands, the certifications are rescinded.”

One of the things I found most interesting about my interview with Jean Ann Harrison was her biggest influence for the talk, which came not from the field of testing, but instead from political science. Harrison majored in Political Science 25 years ago. She’s found that the analytical thinking skills her professors emphasized play a large part in her success.

“In the four years and loads of courses, exercises were given to force students to practice analytical thinking. Software testers are constantly required to analyze how to do something, how to improve, what is the data telling you, analyze different perspectives, create. Over the years, my analytical skills have evolved but certainly were given a solid foundation because two professors teaching the subject of Political Science felt the skill was critical to the coursework. One exercise that was given to me in a course called Research Methods, I use today to train and mentor software testers. It is simplistic in nature but very difficult to implement. The exercise requires them to generate a new hypothesis that they personally have not read about, been trained in, or been given any sort of research material on. Then they are required to describe and prove the hypothesis using empirical means.”

When asked why she chose CAST as the venue for her talk, Harrison shared that this year’s theme for the conference, “Serving our stakeholders,” is directly relevant to some of the lessons her current company is learning, as it’s experiencing growth. “Each department is learning who the customers are,” Harrison says, “How we can better be of service, and what can we learn from our mistakes.”

For more on the upcoming show, check out the CAST conference website.

Software security threats: Changing QA a must

Software quality assurance (QA) and software security teams have long been separate islands within development organizations. That division is giving data pirates carte blanche to compromise software, cyber-security industry veteran Barmak Meftah told me recently.

“Today, we are witnessing the greatest increase in cybercrime,” said Meftah, Fortify Software’s technology senior vice president. “Yet most organizations continue to bolt security on after the fact. Organizations need to build in security first, not layer it on later.”

Hackers have always exploited security vulnerabilities missed during development, but security experts worry the scope of attacks will increase. For instance, they think hackers will increase SQL injection attacks on Flash, JavaScript errors. Another unchecked threat is cross-site scripting, which is still rampant in the Web 2.0 world. “It’s literally on every site I view,” Web security consultant Kevin Beaver told me in another conversation.

Mehta and I talked about how this code-red software environment has pushed software security assurance to the forefront in 2009. He listed these reasons:

• Heightened executive pressure for software security: As
  software’s critical role in running business operations becomes apparent, non-technical executives have become increasingly interested in understanding broader approaches that manage risk, not technology.
• The need for a holistic platform for software security: Traditionally, companies have only been able to buy point products to secure applications during development, QA or in production. That doesn’t work now that data predators, who are continually becoming more sophisticated, use multiple methods to leverage software vulnerabilities to attack organizations. Reacting to this pressure, companies have been forced to adopt multi-prong approaches that include pen testing, static analysis, web application firewalls and so on.
• A business-driven focus on more security from development to production: Eliminating risk leads to preventative, not reactive, security testing. There’s a move from SDLC only to security teams are attempting to bring about significant cultural and process changes to elevate security awareness and governance across an organization.

Today, Mehta said, QA teams must pick up responsibility for security and work with development to make fixes. This is a marked change from the recent past, when QA and security managers have rarely crossed paths. “They have been different organizations with different objectives,” Meftah said. QA has focused on common use cases to uncover problems that typical users may encounter. Security has focused on abuse cases, hoping to uncover a corner case that could allow an adversary to penetrate and exploit a system.

Now, Mehta told me, QA and security teams have to cross the chasm to work on these issues and more together. QA involvement is key way to shift from a reactive security approach — such as patching and firewalls — towards a preventative approach that builds security in from the beginning.
This conversation left me wondering how many companies are pairing or have paired their QA and security teams and to what extent. Or, perhaps some QA and security managers believe they should continue as separate entities. I’d like to hear from you on this subject, either in comments below, or via email at  editor at searchsoftwarequality.com.