testing

What software testers should know about databases

By day, Matt Heusser is a software tester for Socialtext. By night, he is a mild-mannered information systems instructor at Calvin College.

In neither case do I, Matt Heusser, wear a cape, mask or colorful outfit: but I have been learning a thing or two about teaching database programming and what a tester might need to know about it.

Oh, I imagine a number of you are yawning and saying to yourselves, “Ho hum.  I work at a big financial services company and slice and dice SQL all day.  What’s the big deal?”

This post is not for that guy. It’s for for the one who read that and thought to himself, “Er, um, what’s SQL?”

I would like to suggest that if you are a tester and don’t work directly with databases, it might be helpful to learn a little, just enough to be a little more effective as a tester.  After all, from Google to Twitter to my cell phone to my iPod, databases are everywhere, and being able to get “under the hood” for testing can make us that much more effective.

Here are a few techniques that might be possible to do with ’just a little bit’ — maybe a couple hours – of database training:

- To test reports, write your own, and compare them

- To test correctness of mass adds and updates of your software

- To do before and after checks on a large batch operation

- To get “under the hood” to test stored procedures and other business logic encases in the database

- To test import/export from contact management software

- To test the integrity of the database after an upgrade, or a restore from backup

- To create ad-hoc reports and queries for any particular need

While direct coding might be a bit much, most databases support Open Database Connectivity (ODBC) and connect right into Microsoft Access.  From Access you can drag and drop tables and design queries visually.  My class, mostly college sophmores, were off to the races in an hour or two.

O’Reilly even makes a book in its award winning “Missing Manual” series for Microsoft Access. You know, the books you wish came in the box but don’t? It’s one of those. And by the way, SQL, or Structured Query Language, is a programming language for databases that can often be avoided by using Access.  Just don’t use it to write queries against a large production database!

A couple of years ago I wrote an article on professional growth for software testers.  Learning about databases is one way to go.  If you woud like to hear more about databases for testers, or something else, just leave a comment.  I would enjoy hearing from you.

STPCon wrap-up: SpeedGeeking, testing Web 2.0 applications

It’s been an amazing week here in Boston while attending the Software Test&Performance Conference.  Friday morning, we ran SpeedGeeking; in the afternoon, I gave my testing Web 2.0 talk, then it was time for dinner and some amazing conversation, and, eventually, sleep.

Similar to lightning talks, SpeedGeeking is a series of seven-minute talks, in rapid fire, one after the other, with one big “BUT”.  Instead of having the whole audience of 100 some-odd people, sitting through the talks in order, we split the room into seven different tables and had the speakers walk around. They chose not to use powerpoint but instead using old-fashioned flip charts and markers.

Justin Hunter, David Gilbert, James Bach, Michael Bolton, Jon Bach, and Scott Barber all lined up, waited for the bell, and … pandemonium ensured. I was stuck between Jon ad James Bach - yes - “inside the Bachs.”

My talk was on productivity on test teams, how teams measure the time they spend testing, as compared to support activities like documentation, status, meetings. Some were surprised to find that as little as 10-30% of time is spent actually testing. Thus, to improve throughput, many test teams don’t need to adopt a fancy methodology or purchase a tool. Instead, they could simply spend more time testing. James Bach covered his “Huh? Really? So?” approach to testing certain claims, while Michael Bolton talked about Testing Vs. Checking. David Gilbert talked about testing as a social, humanistic activity. Aside from some issues with the acoustics, I was really happy with how SpeedGeeking turned out.

Ironically after being stuck “Betwen the Bachs” at SpeedGeeking, the next session I attended was a keynote called “Testing outside the Bachs”, where Jon and James explained exploratory testing

In my “Testing Web 2.0 Applications” talk, I discussed the user-centric web including, ironicaly, blogging, where the buisness model is to have the user’s themselves develop the value, with the tools acting as a enabler.  I covered techniques such as testing with a dirty environment, segmenting the tests, and testing race conditions.  I’ve also placed the slides for the talk online here.

Then I went out to dinner with a few attendees, where I asked for a blueberry beer and got regular beer with physical blueberries in it.  (I was expecting a blueberry flavored beer.) I guess I didn’t specific enough with the requirements, eh?

Moving away from NAT for testing

For test and development systems, one practice over the years to allow developers to build test systems or applications has been to use network address translation or NAT. NAT basically puts some device in front of other systems. Development teams can use NAT a number of ways. These include running a virtual machine behind a host’s network, a network appliance or firewall rules.

NAT is bad for testing for a number of reasons. The primary reason is because the test system is behind a (presumed) protected device, there is no pressure to put security as a priority in the test process. Practice points such as weak password, application defaults, unnecessary network configurations and other items leave the test system at risk for propagating poor configuration and practice forward in the lifecycle.

Instead of using NAT, many organizations are using dedicated networks for test and development purposes. There can be firewall rules in and out of the network, yet within the network the test systems are fully present. These dedicated networks can also be configured to be fully isolated or connected upward for important things such as Windows Updates for Microsoft systems.

NAT is a limited in real practice for development cycles. What may not be known is what developers are doing individually with local virtual machines on desktops that may be using NAT.

The governing principle is to treat all levels of test and development with the same network rules that you may subject them to in a live environment.

CAST 2009: How to teach yourself testing, an interview with James Bach

At this year’s Conference for the Association for Software Testing (CAST), taking place July 13-16th in Colorado Springs, James Bach will be presenting a tutorial on self-education for software testers. The tutorial, titled “Teach YOURSELF Software Testing,” is about teaching yourself testing, instead of waiting for some testing guru to tell you all the answers. Bach often boasts that he invented testing for himself, and he believes you can too. In his tutorial, Bach plans to share his personal system of testing self-education. Based on his upcoming book “Secrets of a Buccaneer-Scholar,” it’s a system of analyzing experiences and questioning conventional wisdom.

James Bach is a high school dropout who taught himself programming and testing. He’s been a tester, test manager, and consultant since 1987. A founding member of the Context-Driven School of testing, he has taught his class, Rapid Software Testing, around the world. He is co-author of “Lessons Learned in Software Testing,” and author of “Secrets of a Buccaneer-Scholar,” a book about technical self-education which is being published in September.

I first heard Bach talk at a conference in 2000 where he outlined a method of approaching testing problems that I found both engaging and effective. A few years later, I met him at a workshop and (after trying to hack his laptop) was lucky enough to get along well enough with him that he invited me to study software testing with him. I’ve had the pleasure of studying under him and have first hand experience working through his syllabus of software testing concepts; developing my own understanding of how to identify, articulate, and test my own heuristics; and developing methods of how to assess my progress. All of those topics are covered in this tutorial.

I opened the interview with Bach by asking him why he thinks self-education, as opposed to more traditional methods like classes or certification, is so important for someone’s career:

“Technical self-education is traditional, going back hundreds and thousands of years. Electricity, for instance, was discovered and developed by individuals working on their own, outside of any institution. So was chemistry and physics, for the most part. We are in the process of developing the testing craft, and that requires people who innovate.

Testing classes and certifications are pretty bad, for the most part. Of course, I try to teach a good one, but there’s not a whole lot I can do in three days. What I try to do is start a fire in the minds of the testers to pursue their education further without me telling them the answers.

Self-education is available to each of us, all the time. We don’t need a budget. We don’t need anyone’s permission. Institutional education, on the other hand, is expensive and limiting.”

Bach first presented a tutorial along these lines at CAST 2007. He often talks fondly about pulling the material together for the first time as a “bold boast.” A bold boast is a self-education technique.

“A bold boast is a trick I use to get myself going on a project. It’s basically a promise to accomplish some feat, such as to write an article or teach a class. I make the boast that I can teach something, and then my mind gets serious about solving the problems that need to be solved. So, the first time I did this tutorial was when I was writing the Buccaneer book and wanted to develop new material for it, more quickly. I told the CAST organizers ‘I’ll teach a class on self-education for testers,’ not knowing what I was going to do, at first.”

If you follow Bach’s work at all, a lot of what he puts forth in the tutorial mirrors the way he talks about and teaches software testing. So I asked him how the tutorial builds on, or extends, some of his past work.

“I have lots of odd ideas about testing. They run counter to the traditional ‘Factory School’ ideas that you find in most testing textbooks. I teach and demonstrate these ideas, in all my classes, but in this tutorial I get to share how I came up with them in the first place.

I’m nervous when I teach this tutorial, because I expect more from the students than in my normal classes. People who only want quick answers about how to test will be disappointed, because my goal is to show them how to create their own answers. In fact, I show them how they already know a lot of the things they don’t think they know!

In the tutorial, I also talk about how the testing craft is going through a great struggle. The various testing schools are fighting with each other for dominance. The certificationists, of course, being the most visible and aggressive of those. I stand up for the Context-Driven School - against the certificationists and Factory folks - and do my best to recruit testers to our cause. I’m up front about that.”

Bach’s ideas about self-education have, in the past, faced some criticisms. I asked him to anticipate a couple of the more likely criticisms and asked him how he addresses them.

“The most likely criticisms, I think, are these two:

In the tutorial I attack other schools of testing thought instead of trying to find common ground. My response to that is - that’s right. I think those other schools are harming the craft. I don’t see common ground. But I’m glad that our field is not regulated. I’d like to see the other schools go down in flames, but not through any mechanism other than the efficient operation of a well-informed market of ideas.

The second criticism is that it’s all well that the great James Bach can make it up as he goes along, but what about people who aren’t famous? My response to this criticism is that I was developing my own methodology before anyone outside of my team at Apple Computer knew my name. I became well known because some folks found out about what I was doing and thought it was interesting. ANYONE can be a testing methodologist. The market will decide, in the long run, whether it is interested in your methods.”

Bach’s ideas on self education have been influenced by the works of Jerry Weinberg (also speaking at CAST this year), Herbert Simon, Daniel Kahneman, and Nicholas Taleb. “I would recommend ‘The Invention of Air’ as a great book that shows the development of one man - Joseph Priestly - and the development of his field of electricity and chemistry through vigorous and collegial self-education.”

“CAST is the conference that attracts the core contributing thinkers in the Context-Driven School. These are the people who, like me, are engaged in the creation of a vibrant testing craft that has roots in many disciplines and in the history of science. No other testing conference is like that. At CAST, I don’t have to apologize for using Joseph Priestly as an example of a good tester.”

For more on the upcoming show, check out the CAST conference website. For more on James Bach’s work, you can check out his website, or either of his books “Lessons Learned in Software Testing,” and “Secrets of a Buccaneer-Scholar.” Bach also runs two very popular blogs, one on testing and one on self-education.

Virtual networks: Fully representative and protected

Many IT environments of all sizes are enjoying virtualization in some capacity to save on costs, provision systems quicker and consolidate equipment. While the time to market availability for virtual systems is much quicker, there are built-in safeguards that can be rolled into any test, development or software quality effort.

All virtualization platforms offer some form of virtualized network functionality. I’ll talk about VMware’s ESX and vCenter Server, as it is the most popular in the space. A virtual switch is where virtual machines are connected to the network, and on the host side that virtual switch has a configuration as well. For test purposes, a fully isolated network can be provisioned for use by virtual machines on the host. Take the following example, an organization’s internal IT staff provisions a special virtual local area network (VLAN) to the host system, which is then configured as a port group in the virtual switch. At that point a virtual machine can be configured to access that VLAN for a private, testing only environment. The figure below shows an example of this configuration where four virtual machines are assigned to a private VLAN:

The private VLAN is not connected to the rest of the company network, yet the four virtual machines can communicate with each other. I like using the isolated virtual networks as they can be populated with other systems to test communications between virtual machines. Further, built-in virtual machine functionality of cloning and conversion allows copies of live systems to be put into the test environment quite easily. This example is not specific enough for many to take and use in their internal testing, but should be a springboard for conversation to see how network virtualization technology can be applied to your software testing and quality procedures.