software development

Using Agile, scaling back helps software projects in recession

“You are about to embark on a dynamic quality endeavor. It is time to throttle down on your already oversized and over-budget test and development teams.” That was the opening salvo of Michael Mah’s (QSM Associates) SQE Agile Development Practices Conference session, Rightsizing your project in a down economy.

Project teams are just too large, Mah said. Sure, the poor economy has prompted downsizing in this industry; but that could be a blessing in disguise. After all, there have been many years of impractical spending, mismatched teams and dysfunctional workplace goals in software projects.

For too long, Mah said, executives and managers have emphasized speed over quality, judging success by whether teams turn out code lines on time, rather than on delivering in a reasonable time frame quality code that’s free of obvious functional flaws. He’s seen promotions being given to project leaders who met deadlines without ensuring quality. The results? We are paying the ultimate price in lost jobs and market share, he said.

“Now that I have pointed out where the trouble began, let me now explain where the trouble continues,” said Mah. Today, he sees many companies using the recession as an excuse for adopting seriously questionable practices, like trying to cut steps to recovery by cutting down on employees. In the short term, this does free up some cash; but once a company has rebounded or is recovering, new team members will have to be hired. Of course, it takes precious time and money to train new employees and pay for newbie mistakes.

On the other end of the spectrum, there are the overly-cocky executives who have spent more wisely over the years, they see this time where other companies are cutting back and pinching pennies as time to throw money and software at problems. This rarely works, and usually sends these companies into the downward spiral.

In a time of recession, throwing enormous manpower and software at problems will rarely resolve issues. It is more appropriate and intelligent to scale back, says Mah.

To scale back, project leaders should build strong relationships with a small core group of commited employees. Research your service and tool providers more thoroughly, gather references and check them.

“How many of you have some fear about loosing your job?” Mah asked the audience, and nearly all of them held up their hands. “And how does that make you feel? Bad, right?” Company and project leaders are responsible for alleviating fear, not creating it. Fearful teams don’t work productively. Often fear is responded upon with aggression.

The most effective way in managing concerned workers in a recession is by showing “concerned optimism,” Mah said. Explain the situation, but encourage the team. Leaders’ optimism and relationship helps team members feel confident in their abilities and have more belief that they are capable of working on innovative projects. A motivated and innovation-welcoming team is critical to success and growth. Optimism makes for more tolerable working environments and more creative problem solving.

“I remember a good many times I would work through 70+ overtime work weeks; partly because work needed to be done, but also because there was always a chance that the company president would see me there,” said Mah. “On the occasions when he did, he would always compliment me with a thumbs up and a ‘good job.’ That was all the praise I needed.”

More coverage from Agile Development Practices:
Early adopters, Agile philanthropy key points in ADP keynote

Agile philanthropy, one man’s ambition for a brighter future

Scanning source code for security flaws: Three best practices

Here’s some quick advice on scanning source code for security flaws. Maty Siman, CTO of Checkmarx, shares his top three best practices for source code vulnerability inspection.

1. Scan early and scan often. “The beauty of not having a compiler-based approach is that code can be scanned any time, anywhere,” Siman said.
2. Use code analysis as a risk benchmark. Be sure your security-optimized code analysis practices and tools eliminate false positives, allowing auditors and CISOs to get a strong handle of enterprise risk.
3. Use code analysis to introduce a culture of security to development.

Remember, said Siman, “the best defense is a strong offense.”

Getting a jump start on software test collaboration

Having tested software during many projects, I’ve seen that the most effective testers are the ones who start early — and I don’t mean the ones who start testing early. I’ll explain what I mean with this step-by-step tour of a pattern I’ve noticed multiple times in software testing projects.

1. A project starts.

2. All the testers work very hard to understand the problem they are trying to solve.

3. Eventually, some small amount of “working” — a very subjective term — software gets delivered to a development environment for the developers to do their unit testing and debugging.

4. One group of testers — often those doing exploratory testing or those working in an agile project context — ask the development team if they can get involved and help test that early code. They don’t care if they can’t log “defects,” because they just want to see the product and provide feedback when and if the developers think it would be helpful.

5. Another group of testers (often those doing scripted testing or those working in more “traditional” corporate testing environments) say they want to wait until unit testing is complete before they start their testing.

6. The group that starts early develops an early collaborative relationship with the developers who let them test their code.

7. Eventually, some small amount of formally “working” software gets delivered to a test environment for all the testers to begin their first test cycle.

8. At this point, both groups of testers start their first test cycle, and both find and log issues.

9. The issues found by the first group — those who worked more closely with the developers — tend to get resolved first, not based on priority or severity, but based on the personal relationship of the tester to the developer.

10. If a defect from that first group — those who worked more closely with the developers — can’t be reproduced, the developer comes over to work with the exploratory tester to isolate the problem.

11. If a defect from the second group — those who waited to start testing — can’t be reproduced, the developer sends it back as “can not reproduce” or the ever famous response, “It works on my machine.”

Before someone points out that “it’s not always practical to start testing early,” and that “there are a lot of good reasons to wait,” I get it. I’m not advocating that in all circumstances you should start testing early. There are plenty of reasons where that might not be practical. However, I’ve never worked in anywhere I would say that’s been the case. And I’ve seen this pattern a lot. It’s not universal, but it’s very common.

The testers who get a jump-start on collaborating with the development often have a closer relationship, and are thus viewed as an asset. It is these testers who are often pressed to help isolate a defects, even if it wasn’t logged by them. They are also the testers who get invited to design review meetings, because their opinions are highly valued.

Does that mean you have to get involved early to be an asset or to have those relationships? Absolutely not. But, I suspect that if you can, you’ll have a better chance of collaborating with people on your project team before pressures are high, stress is up and your “feedback” is viewed as a call for another late night debugging session.

Data visualization by example: Real-world uses

Data visualization technologies have advanced at warp speed. While Edward Tufte might be able to break data visualization down into its basic elements, for those of us who don’t live the topic, it can be hard to keep up and, more importantly, put these new technologies to work. Just to show it can be done and is being done, I’m briefly sharing some real-world use cases in this post.

Simply put, it’s all about the visuals, or every picture telling a story. My love of graphics is one reason why I’m a regular reader of the Wall Street Journal. I like WSJ’s front-page news crib sheet and financial news; but I’m really into their charts and graphics. I find that I’m constantly cutting out charts that I see and saving them as examples of a creative way to display complicated information. (Yes, I’m the one guy still reading the paper version.) This story by Steve Myers outlines how media outlets are trying to leverage visualization to better tell their stories.

As testers, data visualization helps us not only in communicating our findings, such as when performance testing, it can also help us recognize problems. For example, if you have a bunch of data — database transactions, source code, network traffic, log files, etc. — and you’re looking for trends or possible issues, figuring out different ways to graph that information can help you understand what you’re looking at.

Because it can be difficult to talk about hard and fast rules for when to visualize your data, I thought I might instead offer some examples I’ve seen of creative uses of visualization:

• I’ve seen a development team create an animated view of their source code commits that you could run a visual animation of commits over a repository, project, or branch for a period of time that you set. Why’s that a big deal? Because you could quickly see where all the development activity took place, who was working on it, and when. As a tester, this can give me insight into where to focus, because it tells me where there’s churn in the code.
• I’ve seen an architect take runtime data, like log files, and use them to build graphs of customer transactions. You could then scroll through all the different images quickly and look for the outliers. Each time you stopped on an image that didn’t look roughly like the others, odds are you found some sort of interesting edge condition or error that tool place. As a tester, this can help you not only recognize when you — or an automated test — might have found an issue; but it can also help you identify and document test scenarios.
• I’ve seen a test manager chart automated tests that ran off of production data to see what kind of coverage they were getting from their random samplings. Because the tests ran thousands of randomly selected scenarios, it was difficult to understand what was and wasn’t getting covered. Simple pie charts across the various variables involved became a simple dashboard allowing them to better focus their sampling algorithm and manual test coverage.

As you think about what you might have on your project that you might want to look at, take some time to look at examples. If you don’t like looking to the media for examples, check out some of the examples on tools like verifiable.com or many eyes.

How app visualization enables picture-perfect requirements

Readers of Mike Kelly’s post on software security uses for data visualization –- the ability to visualize patterns of data –- asked us for info on application visualization, too. Application visualization enables software developers, testers and users to see and interact with a fully functional prototype or simulation of a proposed application before coding. We’ll be covering both visualization topics more in this blog and on SearchSoftwareQuality.com.

In this post, I share excerpts from my interview with application visualization expert, Maurice Martin, president, COO and founder of iRise, a maker – of course – of application visualization products.

Developers and business analysts seeking to get software requirements right will find visualization far more effective than text documentation and diagrams, according to Martin. Planning to visualize applications in high fidelity before coding can ensure that the right requirements are captured early in the process.

Imagine giving stakeholders a fully-functional, working prototype early in the process; development teams can secure the right feedback early in the process, before any expensive coding happens.

“Instead of a stakeholder review meeting where everyone is flipping through giant binders of text requirements and struggling to understand what’s been written, now everyone can ‘test drive’ the application live,” Martin said. “The magic happens when changes are made to the visualization right in the stakeholder review meeting, cutting days, weeks or even months off the requirements cycle time.”

Application visualization is a paradigm-changing technology and approach for software development, in Martin’s opinion. Business analysts are no longer just reviewing text requirements, they’re “now rapidly iterating functional prototypes directly in front of the business stakeholders,” he told me. The result is quicker capture of more accurate requirements for designing and developing code, building test scripts and writing documentation or training content.

Until business analysts, developers and users can see, interact with and fully experience the application, they can really only guess at the requirements, Martin said and then continued, saying:

“Add to this the fact that the tools and processes used by most BAs are antiquated at best, and it’s no wonder project overruns, delays and missing features are all-to-common outcomes. The sad truth is this: the state-of-the-art in requirements definition for many organizations is still a giant text document, maybe annotated with a few static screen shot mock-ups or use case diagrams. Let’s face it, the cycle times to create these documents are long and the business stakeholders don’t understand –or read — these giant binders. This would be analogous to designing a new car, airplane, building or semiconductor today with a drafting board. It simply doesn’t happen in other industries; why are we still designing software this way?”

Visualizations can provide guides for what to build, eliminating confusion and enabling developers to focus on what’s important; architecture, design, coding and delivery, Martin said. He’s found that visualization can cut down on change orders and eliminate about 70% of downstream rework. In short, visualizations forces the business to articulate what they want in a language everyone can understand.

Ways to drive adoption of test-driven development

At a recent Indianapolis Workshop on Software Testing, I joined a group of software testing pros in examining the practice of test-driven development (TDD). We brainstormed ahout how to increase adoption by helping teams new to the practice. While there are general principles for introducing change into an organization that come into play, we came up with the following practices that would make the TDD practice sticky in most organizations.

One topic that came up often was making sure the team understands why implementing test-driven development is needed. Managers need explain the benefits in terms of how teams will benefit. A key to success here is finding out what the team’s objections and concerns are before you begin implementation. Without that information, objections can’t be addressed in advance, and they’ll show up later in ways that will slow down the project.

Once people understand the goal, the next step will likely be to pilot the practice. Some ideas from the group included starting with the strongest advocates in the group, targeting the programmers with most influence first, or pulling in external resources that already know how to do it and can share their experience with the team. By starting with a smaller team and growing from there, you’re (hopefully) building a success story. You want to create a sub-culture where people want it to succeed.

To help make the tests a more visible and tangible part of the development practice, make sure they are included in code reviews and that when people pair they are still working test-first.  You want to create a culture where programmers don’t just challenge each other on code and design, but also on tests. You can also make unit testing tasks a visible part of the development process; include them in stories, on your kanban board, or as part of task-out.

In addition to making sure people understand the goals and getting the practice integrated into your development practices, there are specific tools and metrics that can help. Here are some good practices:

• Get people looking at static analysis metrics — complexity, magic numbers, etc. — for the code before and after test-driven development.
• Get people looking at code coverage for unit tests and notice how it changes as adoption picks up.
• When you start, it can be helpful to have an endpoint in mind and trace how test-driven development can assist you in getting there.
• Try to find and measure other metrics to support the end goal; like time to fix with test-driven development versus without.

Some of the tools that came up during the workshop that can help included Heckle, Flog, and Blame. There are tons of tools out there to help teams manage unit tests and provide code and testing metrics. Figure out what works for you, based on your programming language, IDE of choice, and how your team wants to work. As your team starts down the road of implementing test-driven development, recognize that they’ll need support and reinforcement. Finally, understand that the change likely won’t happen overnight.

These are just a few of the many good ideas offered in the March meeting of theIndianapolis Workshop on Software Testing. Besides yours turly, participants of the workshop included Chris Achard, Patrick Bailey, Anthony Bye, Jason Gladish, Matthew Gordon, Tim Harvey, Frank Jaloma, Courtney Jones, Baher Malek, Joel Meador, Elijah Miller, Steve Pollak, Russell Scheerer, Elizabeth M. Shaw, Dustin Sparks, Miles Z. Sterrett, Jon Strayer, Nick Voll, and Jeff White.

Software testing careers: What’s your hedgehog concept?

I’ve worked with a lot of people new to testing. Helping people find an area within testing they like is something I enjoy doing. Testing is fun. It’s challenging. I think it’s got something to offer everyone who’s willing to learn, is comfortable dealing with conflict and doesn’t mind hard work.

So what does that have to do with hedgehogs?

I’m a big Jim Collins fan. I know I’m on the bandwagon, but if you’ve not read Good to Great or Built to Last, please give them a try. There is some great material in there that I think applies to all levels of contributors. For example, I’d like to take a second to look at what Collins calls the hedgehog concept. If you’re not familiar with the hedgehog concept, please take a quick look at it.

It’s ok, I’ll wait.

So let’s apply the hedgehog concept to software testing. You have three circles:

1. What you can be the best in the world at
2. What drives your economic engine
3. What you are deeply passionate about

Best in the world
For companies, we have fairly well defined measures of success. For individual contributors, it becomes a bit more difficult. The idea here isn’t that you would literally become the best in the world, it’s that you have the ability to be one of the best, you know what it would look like to be one of the best, and that you recognize the depth and breadth of what it would likely take to be the best within that aspect of software testing.

This question reminds me of a conference talk Rob Sabourin gave a couple years ago on the topic of what baseball taught him about metrics. Think of the back of a baseball card and the stats that you would find. Do you know what stats are important for measuring testers? What stats do performance testers care about compared to security testers?

I think what’s important isn’t that you’re actually the best. There’s no real way you’d ever know. Instead, what’s important is that you’re thinking about what being the best would look like. You’re asking yourself how you’d measure it. And as far as your hedgehog concept is concerned, you believe you can be the best in some aspect of your testing.

Figuring out your economics
Economics in this case has a double meaning. It’s not just about how much money you make. However, that’s certainly part of it. You can’t work as a tester if you can’t pay the bills. But it’s more general than that. As a project team member, you play into the “economics” of the project team as well. How do you drive the project economic engine?

Think about the statement “profit per X.” If you substitute the word “value” for “profit,” what X’s might a tester care about?

• Is it value per test?
• Is it value per test idea?
• Is it value per requirement covered?
• Is it value per line of code covered?
• Is it value per defect identified?
• Is it value per document?
• Is it value per project?
• Or is it something else? Or some combination?

How do you affect the project’s bottom line? What value to you provide to the project? Do you know how you affect the project’s economic engine and do you know how that in turn drives your personal economic engine? Is the connection clear and do you actively try to manage it?

Finding your passion
This is typically the easiest circle to figure out. What types of activities or issues keep you at work late, not because you have to, but because you want to? Where’s your passion? Is it performance testing, security testing, usability testing, exploratory testing, test management, testing automation, or some other aspect of testing? Think about when you’re most satisfied in your work. You likely won’t like every aspect of testing, but which aspects give you energy?

Thinking about your hedgehog concept can be a valuable activity in trying to figure out what you want to do with your career. It’s also useful in recognizing what value you provide to the project team so you can actively manage how much value you provide. Just as Collins says in Good to Great, you likely won’t be able to answer any of these questions right away. They take thought, questioning, and self-discovery. If you candevelop a hedgehog concept (and follow it), I suspect you’ll have a clearer understanding of what sucess looks like for you and you’ll be more fullfilled as you work to follow it.

Application performance testing issues: Cloud, virtual labs, scale-up

Application performance testing used to be a standalone process, but the emergence of dynamic, complex mission-critical applications, virtualization and cloud computing calls for putting it into a larger practice, Mark Kremer, CEO of Precise Software Solutions of Redwood Shores, Calif., told me recently. In our discussion, he offered some advice about how to handle new challenges facing those who must ensure top application performance.

I asked Kremer what complications porting apps to the cloud add to application performance testing and management. He replied that the dynamic nature of development in the cloud means that application performance must be monitored constantly.

“In physical environments, application performance management assumes quasi-static resource configurations; the computing power, network bandwidth, memory pools, and system overhead are invariable over time or at least until the next configuration upgrade,” Kremer said. “Under these assumptions, time measurements are consistent as they were measured under the same terms. Once an application is run on a cloud, its configuration may change from one invocation to another, or even within the same run, as processes may be transparently moved around the cloud. This phenomenon of ever-changing resources makes time measurements inconsistent as they have been taken under a different condition. Correcting, or normalizing time measurements to a standard scale is conditional to self referencing performance monitoring, and is a daunting challenge to model and implement.”

(For more info on software testing and cloud computing, check out my interview with Eugene Ciurana, director of systems infrastructure at LeapFrog Enterprises, a large U.S. educational toy company.)

The dynamic nature of virtualized environments also requires changes in how application performance is monitored and testing, Kremer said. The development/testing team should keep an internal application clock — app time, if you will — that is invariant to the underlying hardware. He explained:

“For example, a transaction will spend the same time measured by the application clock in a Java method regardless of the power of CPUs used in each invocation,” he said. “As application performance management evolves to include this concept, developers building applications for virtual or more commonly mixed mode — virtual and physical — can get around the semantics of time in virtual environments.”

Talking about application performance in general, Kremer stressed that testing can’t just take place in a lab, because it’s so hard to replicate real production environments there. Even if the production environment can be created in a lab, often performance still changes when apps are placed on a real, dynamic production line.

“This dynamic manner of problem resolution analyzes the data that causes performance-loss by tracking spikes in user behavior, patterns in data accumulations, and changes to the systems configurations,” Kremer said. “Application performance testing relies more on static test models which makes it tough to replicate real-world production environments.”

I asked Kremer how scale-up changes what must be tested to ensure stellar application performance. In response, he said that when applications scale up, performance testing must change from being input oriented – focusing on test patterns, synthetic transactions, etc. — to being throughput oriented, where the focus is on transaction monitoring, performance base lining and so on.

“As systems scale up, their performance testing paradigm shifts from predefined synthetic tests to monitoring and self-reference,” Kremer added. “For optimal results, IT needs to identify the top, say 20, transactions of the system, constantly monitor their performance, their component’s performance, and the time allocations of various tiers in the system. Then it must self reference these measurements hour-to-hour, day-to-day, season-to-season…to detect performance degradation, offending transaction components or performance hot-spots.”

That’s all from my interview with Mark Kremer. SearchSoftwareQuality.com news writer Colleen Frye is covering application performance topics, so watch for more articles in the news section. Here’s a sampling: CareGroup solves application performance issues with APM tool and Don’t let poor website performance ruin e-commerce sales.

New book: Step-by-step Eclipse plug-ins, plus Java testing tool

I found a handy tool for automating testing of Java graphical user interfaces (GUIs) while reading
Eclipse Plug-ins, third edition (Addison-Wesley) by Eric Clayberg and Dan Rubel. This month, I also got a chance to ask them some questions about the newly-minted third edition of this book, which gives step-by-step directions for plug-in development and descriptions of specific plug-ins. Here are excerpts from our Q&A and some information about the book’s contents and the Java testing tool.

First off, both Clayberg and Rubel are co-founders of Instantiations Inc., maker of GUI-building software and automated testing and code quality tools. They’ve been working with Eclipse since 1999 and developed CodePro on it.

Usage of Eclipse, now in its eight year of existence, is on the rise, the authors told me. They see potential for greater growth with OSGi and Equinox.

Taking developers beyond the basics to a point where they can create high-quality commercial Eclipse plug-ins is the goal of the book, said Clayberg. “In the world of Eclipse plug-ins, very few people take the time to really go the extra mile, and most plug-ins fall into the open source, amateur category.”

Describing and offering use cases of the Eclipse Command Framework (ECF) is one way the book helps developers get up to commercial speed. ECP replaces the older Action framework. “Throughout the book, use of the older Action framework has been replaced with new content describing how to accomplish the same thing with the new command framework,” said Rubel. In particular, the book covers use of commands with views and editors in ECF.

Here are some other ways the book provides updates and detailed information about some beyond-the-basics development steps:

All of the screen shots, text and code examples throughout the book have been updated to use the latest Eclipse 3.4 API and Java 5 syntax. New capabilities in Eclipse 3.4 are detailed, including a new overview of using Mylyn and a discussion of new preferences and PDE and SWT tools available in Eclipse 3.4.

In Chapter 20, you’ll find a step-by-step guide to using GEF, the Graphical Editing Framework from Eclipse.org. This toolkit is designed for building dynamic interactive graphical user interface elements. The authors walk through the process of building a GEF-based view for graphically presenting the relationships between the favorites items and their underlying resources. Then, taking a bigger step, they show how to build a GEF-based editor with the ability to add, move, resize, and delete the graphical elements representing those favorites items.

Clayberg and Rubel practice what they preach. They’ve recently released WindowBuilder Pro v7.0, an Eclipse plug-in tool for Java GUI developers, and continually update their CodePro AnalytiX software that adds enhancements to Eclipse and any Eclipse-based IDE.

Another plug-in the authors have made is WindowTester Pro, and it’s the Java GUI testing plug-in I mentioned reading about earlier in this post. As I said, it enables automated testing of Java GUIs that use SWT, JFace, Swing or RCP, eliminating the need to create and maintain test code through various phases – recording, test generation, code coverage, etc. – of GUI application interactions. Among other functions, WindowTester Pro facilitates integration of test case execution into a continuous build system, so your application is tested each every time it’s built.

Next year, the fourth edition of Eclipse Plug-Inswill deliver information on how to fix long-standing issues and jettison old, deprecated APIs.

How test-driven development increases overall usability

Test-driven development (TDD) can be the path to not having to reinvent the wheel with every new test. In the test-driven development I’ve done, I’ve found that my tests force me to write a more manageable interface over time. Early in a programming a class, I find that I can get away with something simple that might not be intuitive. But the more tests I add (so I can add more and more functionality), the more refactoring I have to do to get the previous tests to pass. This continually forces me to think about the interfaces I have and the best way to test them. That gets me thinking about simplifying my previous crufty code. In the end, I have a more usable interface for me, which I suspect also makes it a more usable interface for others.

The folks over at UX Booth recently posted an article on “How Test-Driven Development Increases Overall Usability.” As with all of their articles, it’s a well-researched and well-written look at the topic. In the article, they contrast testing at the user interface with testing at the application interfaces. It’s not an in-depth technical article, but it’s an interesting look at the topic. I think they accurately express one of the core ideas of test-driven development—to “make the application more usable to everyone involved.”

Another great example of TDD increasing usability can be found in Dale Emery’s analysis of Brian Button’s article “TDD Defeats Programmer’s Block—Film at 11.” Dale points out Brian’s general pattern of naming his tests using stimulus, result, and context. The naming scheme makes the tests more readable and allows you to readily interpret what the code/tests are trying to accomplish. It’s another, more detailed example of the UX Booth idea of usability at the application interface. Dale further expands on this pattern in a follow-up post on the anatomy of responsibility.