The entire panel was quick to point out that you’re never alone, and that you always have a team to work with. Brian Ball, a software engineer with Software Engineering Professionals, said that it’s important for you to realize you’re part of a larger team — whether they feel like it or not. He said you need to let the rest of the team know that you’re concerns are really their concerns.

Robby Slaughter, founder of Slaughter Development, a workflow and productivity consulting company, pointed out the following:

“What you really need to do in those situations is identify who are the stakeholders around you who care about the results of your work and the results of what’s happening, and find a way to leverage their expertise, their ability, their commitment and their passion towards what you do. Now that’s a problem that is magnified when you have technical expertise in some area of what you do, and the people around you — the stakeholders — don’t.”

Slaughter also pointed out that one of the most important things you can do is to help people give you information in a way that enables you to do your work better.

“You need to realize that you might have this expertise or this area of technical facility [and they do not]. You need to go back to the stakeholders and identify ways to communicate better. Maybe they need a form to fill out. Maybe they need a tracking system, your ticket tracking for the request. Or maybe you need to find a way to give them information in a more organized fashion, so they can find it more easily so they are not constantly calling you.”

Chris Wingate, principal software engineer for Liberty Mutual, summed up his thoughts on the question the following way:

“To me is seems to come down to two things…and those are trust and education. So you’re the only tester in your organization at the moment, and so it’s very important to build the trust in your competence with the rest of the team — with the developers and the managers. That way when you say, ‘Yes I can do that.’ or ‘No I can’t do that for these technical reasons.’ you don’t have to spend a lot of time explaining all of that and rehashing all of those things that you continually do.

“The other thing is educating the rest of the team about what your capabilities are, what timelines are, and they go together… you have to educate your customer in terminology. Educating your customer is going to help improve the communication and help make you more efficient.”

You can find the complete audio for the panel members’ advise for one-person test teams on the IWST website. There you can also find more information about the speakers along with additional audio from the meeting broken out by topic.

1. Scan early and scan often. “The beauty of not having a compiler-based approach is that code can be scanned any time, anywhere,” Siman said.
2. Use code analysis as a risk benchmark. Be sure your security-optimized code analysis practices and tools eliminate false positives, allowing auditors and CISOs to get a strong handle of enterprise risk.
3. Use code analysis to introduce a culture of security to development.

Remember, said Siman, “the best defense is a strong offense.”

Virtual Compiler will be used by development teams “test uncompiled and unlinked code, their independent modules or any other application subsets in a desktop deployment that reinforces good security awareness and practices as the code is written,” said Siman.

Auditors and chief information security officers (CISOs) can benefit from being able to test code earlier in the software development lifecycle (SDLC), said Siman. They can also use Virtual Compiler to inspect legacy code.

Using source code analysis tools is a must for building secure software, according to SearchSoftwareQuality.com security expert Kevin Beaver. In his tip, The role of QA pros in software security, he wrote that source code vulnerability checkers “are essential for rooting out software vulnerabilities that would otherwise be next to impossible to find.” He names Checkmarx, QAInspect and Klocwork tools as good options.

Virtual Compiler solves some problems that static code analysis tools haven’t addressed previously, Siman said in our interview. Most major static code analyzers have only scanned post compilation and required buildable code. As a result, static code analysis required a complete, buildable project to run against. So, scans usually have to take place near the end of development, and repairs required going back and fixing code whose problems probably manifested themselves and possibly grew during development.

“Checkmarx Virtual Compiler eliminates the buildable code requirement by removing the dependency on compilation and linking for software testing,” said Siman. “It transforms code, whether freshly written or old legacy applications, into a form that contains structure and application flow properties. Testing is not dependent on having all modules complete, duplicating the development environment or creating a final build-test harness. Instead, scanning can take place early, and often, as the code is developed. Once scanning is complete, all code and flow properties are stored in a data base that can be interrogated for vulnerabilities.”

Checkmarx Virtual Compiler is part of a suite of products that can be purchased for onsite use or as a service. Prices for onsite usage start at $15,000.

Similar to lightning talks, SpeedGeeking is a series of seven-minute talks, in rapid fire, one after the other, with one big “BUT”.  Instead of having the whole audience of 100 some-odd people, sitting through the talks in order, we split the room into seven different tables and had the speakers walk around. They chose not to use powerpoint but instead using old-fashioned flip charts and markers.

Justin Hunter, David Gilbert, James Bach, Michael Bolton, Jon Bach, and Scott Barber all lined up, waited for the bell, and … pandemonium ensured. I was stuck between Jon ad James Bach - yes - “inside the Bachs.”

My talk was on productivity on test teams, how teams measure the time they spend testing, as compared to support activities like documentation, status, meetings. Some were surprised to find that as little as 10-30% of time is spent actually testing. Thus, to improve throughput, many test teams don’t need to adopt a fancy methodology or purchase a tool. Instead, they could simply spend more time testing. James Bach covered his “Huh? Really? So?” approach to testing certain claims, while Michael Bolton talked about Testing Vs. Checking. David Gilbert talked about testing as a social, humanistic activity. Aside from some issues with the acoustics, I was really happy with how SpeedGeeking turned out.

Ironically after being stuck “Betwen the Bachs” at SpeedGeeking, the next session I attended was a keynote called “Testing outside the Bachs”, where Jon and James explained exploratory testing

In my “Testing Web 2.0 Applications” talk, I discussed the user-centric web including, ironicaly, blogging, where the buisness model is to have the user’s themselves develop the value, with the tools acting as a enabler.  I covered techniques such as testing with a dirty environment, segmenting the tests, and testing race conditions.  I’ve also placed the slides for the talk online here.

Then I went out to dinner with a few attendees, where I asked for a blueberry beer and got regular beer with physical blueberries in it.  (I was expecting a blueberry flavored beer.) I guess I didn’t specific enough with the requirements, eh?

U.S. Air Force Colonel Mike Mullane — now retired after also serving as a space-shuttle astronaut — gave the opening keynote. Mike’s talk was on fundamentals of teamwork. He began by taking us through what happens on the day of a shuttle launch: Suiting up, strapping in, countdown and liftoff.

After explaining the tremendous pressure and risk of a shuttle mission, Mike asked us one question: If you were on a project like that, what kind of team do you want to be working with?

The best, right?

Mike talked about these three fundamentals of teamwork:

• Normalization of deviance;
• Responsibility; and
• Courageous self-leadership

I’ll tell you about one of them, “normalization of deviance.” Yes, Mike has a way with words.

To start with, Colonel Mullane reviewed the tragedy of the Challenger Accident, when O-ring failure causes the destruction of the shuttle. O-rings are basically rubber rings; they create a seal or barrier. You’ve probably seen them on your faucet. In the case of your faucet, they keep the water from leaking, but in the case of the Space Shuttle, they prevent the heat of the engines from leaking out from where the engines connect to the rest of the shuttle, instead forcing the heat down the hole in the bottom.

In the case of the Challenger accident, the o-ring failed, the heat leaked out, the section of the booster melted, this changed the pressure situation and the rockets, essentially, vaporized. This directly resulted in the death of the entire crew, including four astronauts Mullane had trained with directly.

How did this happen? Mullane used a fancy term: normalization of deviance. It basically means taking shortcuts, often under intense schedule or budget pressure. The first time you do this, you probably have no negative consequences. This leads to what Mike called “false feedback,” or that the absence of something bad happening means it was safe to do so.

Eventually the shortcut becomes the norm.

Now the original safeguard was put in place for some reason. Ignore it long enough, and eventually, you’ll have a “predictable surprise.” In almost all cases, the predictable surprise is not good for the team. Colonel Mullane refers to the Challenger accident not as an accident, but as a predictable surprise.

Then he pulls out documents. It turns out that in 14 of the 24 missions before Challenger, the contractor who recovered the solid fuel boosters from the water inspected the boosters and found critical or urgent problems in the o-rings. Colonel Mullane read multiple lettters from the contractor to NASA urging immediate action. One of those letters predicted the failure of the shuttle to the mission number.

The problem? Mission One was a success. So was Mission Two. For that matter, NASA had promised congress 26 missions a year, massively decreasing the cost per mission.

It’s really hard to ground a shuttle fleet when you promised the American people to put up a shuttle every two weeks.

Instead of grounding the fleet, NASA tested a damaged o-ring under pressure, and it worked, so NASA granted a waiver.

This subtle change meant that something intolerable had become expected. To avoid normalization of deviance, Mike recommended that we:

• Recognize we are vulnerable;
• Yes, plan the work and work the plan, but not blindly; and
• Maintain situational awareness.

By the time Mike finished, I found myself tearing up a little.  And the talk only got better.

We software testers may not be flying to the moon, but I submit there are still a few lessons we could stand to learn from the astronauts.

In the morning, Scott Barber presented a half-day workshop on quick and easy performance wins. In the presentation, he split the hard engineering performance testing work — say, simulating and cranking up load –from the “easy stuff.” To do the easy stuff, Scott had four concrete suggestions:

1) Know the mission

Instead of stating with a document — “100% of web pages shall load and display within five seconds” — Scott proposes a model of conversation to collaboratively explore and figure out what the goal is – before testing begins. He calls this “accept / converse / understand.”

2) Am I annoyed?

Scott suggests (and I agree) that it’s actually pretty hard to make decisions on 4.8 seconds verses 6.5 seconds. So instead of working for hard numbers, you can simply have testers write down their annoyance on some scale, perhaps 1-5, at various key points as the test suite runs. (Ideally, do it under simulated load; but if performance stinks for one user, that will tell you a lot in and of yourself.)

3) Watch the trends

Scott suggested that developers can write timers, wrap unit tests in those timers, and simply observe trends to see if the software is getting faster or slower. Likewise, the team can do the same thing to document acceptance and human-run testing. Several people in the room observed performance with a stop watch.

That’s just the tip of the iceberg. Most of Scott’s talk wasn’t on specific technique, but instead on dynamics of projects and general systems thinking skills.

Then, in the afternoon, we had a panel on innovations in performance testing, then a quick mini peer conference where we shared our experiences with perf testing.

In my book, the peer conference idea beats PowerPoint-driven lectures by a mile. Still, I have to say, Scott’s talk was pretty good.

Heusser focused on his development work at SocialText in one session. He told me he had been preparing for months on his presentation on developing software that helps remote workers stay in touch with co-workers and up to date on their businesses, work agendas and projects. Also, he had the challenge of describing how SocialText uses the agile development model.

In the session, he described SocialText’s model step by step, showing such things as how the client’s work request is handed down linearly to the appropriate team chapters. He explained what happened in client-SocialText developer kick-off meetings, noting that all team members attend online using their networking and communication tools. The kickoff is an opportunity for client questions can be asked and requirements clarified, he said. After that initial meeting, the business model becomes a staircase of iteration constructing events. But the whole team has the availability to follow the project, even at phases they are not directly involved with.

SocialText’s programming and products are based on a wiki, making them easy to modify and fairly easy to write the first place. “Anyone dedicated enough, could write something similar,” said Heusser, but SocialText got a head start in this area.

In SocialText’s agile model, Iterations are normally completed bi-weekly. Once approved, the next functional addition is begun on the approved code. This process recycles through all of the testing. Testing is done throughout the entire development process, so that when “crunch time” come, the team doesn’t have to run the full train of tests. Most are already completed, and the few that remain can be held off until a more appropriate time can selected.

Heusser admitted that their process probably wouldn’t work every software development company. “One of the rarest things about working SocialText, is that we use our software to create new software,” he said. “This helps us in the usability testing. If a function doesn’t work properly, we notice it very quickly, as we use every feature numerous times in the development of every product.”

Another rare thing about SocialText is its hiring of many remote developers. Generally, quality of applicant aced location everytime. “In the 21st century, it is ridiculous to rely on 18th century hiring models. Those who worked in mines and on railways had to be geographically placed in the vicinity of their work, of course. That just isn’t and shouldn’t have to be the case these days. We have the technology, why not use it?”

Poole gave various examples and analogies of how agile development methodology works, noting that group involvement was a key of agile success. Agile developers benefit from receiving more frequent feedback from customers and having short-term, reachable deadlines, Poole said.

“Agile versus waterfall or any other methodology can be compared to touchdowns versus yardage,” he said. “Where is the credit given? Clearly you can’t win the game based on ridiculous yardage without scoring.”

This analogy points to the major downfall of waterfall, a process in which the team scrambles in all different directions and gains some ground, but often fails to complete the project on time. Agile, he said, is different; you set reachable goals for short phases, or iterations, of development.

This process of “continuous integration” involves completing one set of functional code, getting feedback and then adding new coding to it and retesting. Doing projects in phases assures a higher quality because of multiple tests and retests carried out during the SDLC (Software Development Life Cycle).

“With agile, there is an absence of confusion,” Poole said. “Do you guys remember the childhood game ‘telephone’?” As nearly every hand in the room went skyward, Poole smiled: “How did that game go?” The audience chuckled, and appeared follow his point. If one person tells a group a direction or a goal and the group collaborates there is less chance for project digression. The project is more likely to be what the customer is looking for which is the equivalent of success.

Agile doesn’t only benefit the industry or a company alone, Poole concluded. It also can benefit the individual. He listed these benefits for the agile team member

1. Less cancelled and or shelved work
2. Less pressure or stress over project duration, especially during crunch time as testing is done throughout SDLC
3. More individual ownership and feeling of involvement in a project
4. Increased project success

Poole frequently repeated the phrase, “the simplest thing that could possibly work.” Agile is the simplification of complex tasks, he said, satisfying the customer today and tomorrow, allowing for full software development team collaboration and good returns on investments.

<
Boston SPIN presenter, Damon Poole (on right) speaking with SearchSoftwareQuality.com contributing writer Matt Heusser

As I write this entry, I’m in the air above Boston, just about to land for the Software Test and Performance Conference. The trip is actually a twofer; I’ll be at the conference tomorrow, but tonight I’ll be out at the Boston SPIN (Software Process Improvement Network), a local user group.

… time passes …

After touchdown at Logan International Airport (Boston), there is no airport shuttle to the conference. Believe it or not, one of the editors at SearchSoftwareQuality.com works and lives in town. Not only is he going to the conference, but he also offered to pick me up at the airport.

And, true to his word, there he is. Also, true to his word, his modified black T-Bird is the loudest car in the parking lot.  I suppose I can’t complain about a favor from a friend.  And the FireBird has some serious horse power.

After I check in at the hotel, it’s time for The SPIN meeting on “Agile Software Development: Does it make a difference?”

Nearly as importantly, there will be pizza.

The Boston SPIN

Yes, there was pizza. So much pizza, in fact, that I missed a little meeting in the corner: a roundtable discussion of Reliability Metrics. Instead, I sit through an informal discussion with a few people who choose to sit at our table; deep enough that Dan Mondello does his own blog post on it.

I do get to see the main event, which was an introduction to Agile Software Development by Damon Poole. Damon is also the author of Do it Yourself Agile, which is nice - and free - e-book on the concept.

The slides from Damon’s talk are online. You can view them here.

A few of the ideas in the talk caught my ear enough to want to share here, in my words:

* We tend to like to think of our work as “done” when we have written code, or run tests, or completed requirements documents. The only “done” that matters is running tested features.

* Development with Legacy Software is just hard to do in general. Yet if you take it a piece at a time and slowly begin to apply agile techniques, over time, the software will become more malleable.

* There is no business value in a complex architecture. Instead of designing for extensibility, just build what the customer asks for and design a system that can be easily changed in the future. (Matt Heusser adds: In my entire career, every time we designed the software to be extensible in a specific direction, that darn customer asked to have it extended in a direction we hadn’t anticipated, and we had to start over at square one.)

* Agile development enables the team to change priorities every iteration.  In this age where a new product like the iPod can radically change the business landscape in a matter of months, it might be better to not be locked into a requirements document with a multi-year release plan.  Just a thought.

The big reveal

At the end, Damon didn’t see a business case for ‘Agile’ development. Instead, he layed out 14 different practices – such as continuous integration, daily standups, refactoring, user stories, and one piece flow. According to Damon, each of these practices can stand on their own, with perhaps a 5% performance improvement.  Because the practices build on each other, doing several of them have a compound-interest effect.

I don’t think Damon meant us to take this literally or scientifically, but merely metaphorically, as a thought experiment. If every Agile practice stands on it’s own, and the lot of them, taken together, provide even more benefit – then why not try an Agile approach?

At the meeting, I joined a group gathered around a circular table to finish off my pizza slice and drink my apple juice. The group was already deep in conversation over the topic of our assembly: What exactly is agile software development, and where and how can it be practiced?

For Tru Hong, a QA engineer and analyst, “agile describes the process of telling small stories, small stories that would help in the telling of larger stories.” “Stories” are used commonly in requirements-based in agile, and storytelling is defining long-term goals and then setting shorter goals in which to accomplish the final vision.

Everyone seated at the table had a reasonable working knowledge of agile and its derivatives. While all agreed that agile is their preferred methodology, they also admitted that it’s difficult to define and sometimes to implement.

Two testers from Pembroke, Mass., discussed their experience with agile, which seemed to be similar to that or others at the table. Their company, which will remain anonymous in this post, was recently bought by a rival vendor that uses agile. As a result, the company had begun a restructuring process which included a move to agile. Outside a general knowledge of the term “agile,” his company’s management had only a vague estimation of what the buzzword referenced. So, management didn’t really understand the scope of the move, and it was left to the development team to implement.

Keith Willett, software product director for American Science and Engineering, Inc. (AS&E) located in neighboring Billerica, Mass., spoke highly of his company’s success with Scrum. His team was an early adopter of the methodology, and he’s Scrum-certified. well. Willett’s background, much like the others at the table, was in traditional waterfall approaches. He called waterfall the “the most medieval of software development methodologies,” and others agreed.

Complete adoption of agile is AS&E’s goal, Willett said. His team is making everything agile, everything from approaching a software project to tool usage to the way in which revenue is gathered and distributed amongst the project.

Matt Heusser — a test pro, agile advocate and frequent SearchSoftwareQuality.com contributor — joined the discussion and shared his work experiences with agile. Company-enforced agile had led to his development teams turning out iterations bi-weekly. Sometimes achieving that goal meant “a modest amount of overtime,” Heusser said. Though the deadlines are challenging, he said, he’s found the work completed in agile is more rewarding, because it leaves you with a sense of ownership over a project.