STPcon

STPCon wrap-up: SpeedGeeking, testing Web 2.0 applications

It’s been an amazing week here in Boston while attending the Software Test&Performance Conference.  Friday morning, we ran SpeedGeeking; in the afternoon, I gave my testing Web 2.0 talk, then it was time for dinner and some amazing conversation, and, eventually, sleep.

Similar to lightning talks, SpeedGeeking is a series of seven-minute talks, in rapid fire, one after the other, with one big “BUT”.  Instead of having the whole audience of 100 some-odd people, sitting through the talks in order, we split the room into seven different tables and had the speakers walk around. They chose not to use powerpoint but instead using old-fashioned flip charts and markers.

Justin Hunter, David Gilbert, James Bach, Michael Bolton, Jon Bach, and Scott Barber all lined up, waited for the bell, and … pandemonium ensured. I was stuck between Jon ad James Bach - yes - “inside the Bachs.”

My talk was on productivity on test teams, how teams measure the time they spend testing, as compared to support activities like documentation, status, meetings. Some were surprised to find that as little as 10-30% of time is spent actually testing. Thus, to improve throughput, many test teams don’t need to adopt a fancy methodology or purchase a tool. Instead, they could simply spend more time testing. James Bach covered his “Huh? Really? So?” approach to testing certain claims, while Michael Bolton talked about Testing Vs. Checking. David Gilbert talked about testing as a social, humanistic activity. Aside from some issues with the acoustics, I was really happy with how SpeedGeeking turned out.

Ironically after being stuck “Betwen the Bachs” at SpeedGeeking, the next session I attended was a keynote called “Testing outside the Bachs”, where Jon and James explained exploratory testing

In my “Testing Web 2.0 Applications” talk, I discussed the user-centric web including, ironicaly, blogging, where the buisness model is to have the user’s themselves develop the value, with the tools acting as a enabler.  I covered techniques such as testing with a dirty environment, segmenting the tests, and testing race conditions.  I’ve also placed the slides for the talk online here.

Then I went out to dinner with a few attendees, where I asked for a blueberry beer and got regular beer with physical blueberries in it.  (I was expecting a blueberry flavored beer.) I guess I didn’t specific enough with the requirements, eh?

Drilling deep into performance testing at STPCon

After yesterday’s Boston SPIN meeting, I got a full night’s sleep, woke up, checked email, and it was time for the full-day workshop on performance testing.

In the morning, Scott Barber presented a half-day workshop on quick and easy performance wins. In the presentation, he split the hard engineering performance testing work — say, simulating and cranking up load –from the “easy stuff.” To do the easy stuff, Scott had four concrete suggestions:

1) Know the mission

Instead of stating with a document — “100% of web pages shall load and display within five seconds” — Scott proposes a model of conversation to collaboratively explore and figure out what the goal is – before testing begins. He calls this “accept / converse / understand.”

2) Am I annoyed?

Scott suggests (and I agree) that it’s actually pretty hard to make decisions on 4.8 seconds verses 6.5 seconds. So instead of working for hard numbers, you can simply have testers write down their annoyance on some scale, perhaps 1-5, at various key points as the test suite runs. (Ideally, do it under simulated load; but if performance stinks for one user, that will tell you a lot in and of yourself.)

3) Watch the trends

Scott suggested that developers can write timers, wrap unit tests in those timers, and simply observe trends to see if the software is getting faster or slower. Likewise, the team can do the same thing to document acceptance and human-run testing. Several people in the room observed performance with a stop watch.

That’s just the tip of the iceberg. Most of Scott’s talk wasn’t on specific technique, but instead on dynamics of projects and general systems thinking skills.

Then, in the afternoon, we had a panel on innovations in performance testing, then a quick mini peer conference where we shared our experiences with perf testing.

In my book, the peer conference idea beats PowerPoint-driven lectures by a mile. Still, I have to say, Scott’s talk was pretty good.

STPCon: How SocialText uses Agile, wikis and remote developers

At the Software Test and Performance Conference (STPCon) this week, I had the pleasure of spending time with Matt Heusser, a software tester for a social networking software development company, SocialText, he is also a SearchSoftwareQuality.com contributor and spoke in a few STPCon sessions this year.

Heusser focused on his development work at SocialText in one session. He told me he had been preparing for months on his presentation on developing software that helps remote workers stay in touch with co-workers and up to date on their businesses, work agendas and projects. Also, he had the challenge of describing how SocialText uses the agile development model.

In the session, he described SocialText’s model step by step, showing such things as how the client’s work request is handed down linearly to the appropriate team chapters. He explained what happened in client-SocialText developer kick-off meetings, noting that all team members attend online using their networking and communication tools. The kickoff is an opportunity for client questions can be asked and requirements clarified, he said. After that initial meeting, the business model becomes a staircase of iteration constructing events. But the whole team has the availability to follow the project, even at phases they are not directly involved with.

SocialText’s programming and products are based on a wiki, making them easy to modify and fairly easy to write the first place. “Anyone dedicated enough, could write something similar,” said Heusser, but SocialText got a head start in this area.

In SocialText’s agile model, Iterations are normally completed bi-weekly. Once approved, the next functional addition is begun on the approved code. This process recycles through all of the testing. Testing is done throughout the entire development process, so that when “crunch time” come, the team doesn’t have to run the full train of tests. Most are already completed, and the few that remain can be held off until a more appropriate time can selected.

Heusser admitted that their process probably wouldn’t work every software development company. “One of the rarest things about working SocialText, is that we use our software to create new software,” he said. “This helps us in the usability testing. If a function doesn’t work properly, we notice it very quickly, as we use every feature numerous times in the development of every product.”

Another rare thing about SocialText is its hiring of many remote developers. Generally, quality of applicant aced location everytime. “In the 21st century, it is ridiculous to rely on 18th century hiring models. Those who worked in mines and on railways had to be geographically placed in the vicinity of their work, of course. That just isn’t and shouldn’t have to be the case these days. We have the technology, why not use it?”