Author Archive

Converting subtasks in JIRA with a Watir script

Whenever I do large data uploads into JIRA, I often find that I have a number of tickets that require a parent/child relationship. That is, some of my tickets get imported as tasks, and I want them to be subtasks. To solve this problem, I wrote a simple Watir script that will subtask out all my tickets for me after upload. There’s nothing that special about it, but it can be quite a time saver.

The only assumption I have in my script is that you’ll have a hyphen in your subtasks and no hyphens in your parent tasks. When I import data like this, I often following the pattern “<parent summary> - <child summary>” for my Summary fields when I import. For example, a parent ticket might be “Write a book on testing” and a couple of subtasks for that ticket might be “Write a book on testing - Chapter 1″ and “Write a book on testing - Chapter 2.”

The first step is to setup a query of only the tickets you want to task and subtask. Often, this is as simple as creating a query in the Issue Navigator where you select the project, Issue Type, and whatever other compoents/values uniquely identify that subset of data. I’ll then sort by Summary, so my parent tickets will always be before my child tickets — again because of my naming convention.

Below, the link to that filter is where you start your browser in the second line of code. In the third line, I simply click on the first result returned. Then the code loops through all the tickets and establishes the parent/child relationships as needed.

Status reports can help provide software testing focus

Recently, I found myself struggling on a program to help plan and manage the testing effort for a couple of different concurrent projects. For both projects, testing is a critical path, and resources are tight. For both projects, there’s a mix of planning and execution activities happening within the next couple of weeks. I was just overwhelmed with everything that needed to be done and balancing that against my need to come up to speed because I was new.

When I start to feel like I’m slipping, I ask for help. I went to one of the project managers and asked for some feedback. After talking for about 30 minutes, wherein several good ideas were offered, she finally said: “I think it would also be good for you to submit a status report each week.”

You know the status report. It’s a project classic; percent complete by task, what you accomplished this week, what you hope to accomplish next week and risks to your part of the project.

While this would normally send of my this-is-busy-work-that-doesn’t-add-value alarm, for some reason I felt like she was right. It would help. A big part of what I had found myself struggling with was keeping track of what needed to be done, when it needed to be done, who was working on it and how I would communicate that in a structured way to the rest of the project managers.

As I started to pull together the report, I noticed that it started to change the way I was thinking about the project. It helped me clarify what needed to be done first. It helped me develop my story of where the testing was, why it might be behind, or why I didn’t think we were ready to start on certain activity. Even though I’m not a big fan of coming up with vacuous percent complete numbers, I recognize that in doing so it forced me to assess our status and develop a more meaningful story for where we were.

When you start to feel overwhelmed with a testing effort your managing, try the following:

• Ask someone you respect on the team if you can talk to them about it. Sometimes just explaining what you’re struggling with can provide you with insights that will help.
• Take that person’s feedback seriously, even if it’s not an approach to the problem you’d normally take.
• Develop a clear method for communicating your status and issues to the rest of the team. It doesn’t have to be a formal status report.
• Until you feel more comfortable with your status, continue to ask for help and continue to refine the story of your testing.

Capturing feedback during sprint reviews

After sitting in a number of different sprint review meetings, with a number of teams, I’ve noticed that not all teams have a formal way of capturing feedback obtained in sprint review. Feedback can be any of the following:

• A defect noticed by anyone in the room during the demonstration of the software developed during the sprint
• Enhancement request ideas by anyone in the room for the product backlog based on what’s being shown during the review
• Feedback from the product owner on something that needs to be changed for the story to be accepted the story isn’t done until this change is made

That feedback should be captured and managed. In theory:

• Defects get tracked and managed: Different teams handle defects requests in different ways. For some, they become backlog items, for others they story isn’t done until they are fixed, and for others they get tracked as part of a release.
• Enhancement requests are captured and managed: Enhancement requests are almost always handled as backlog items. They get reviewed and prioritized along with all the other stories.
• Acceptance feedback is addressed before the sprint “officially” ends. It might be tracked as a new “task” for the story before it’s completed.

In practice, no one tracks any of these. Nothing gets written down, and two hours after sprint review all the feedback is either completely forgotten or is just hearsay.

Don’t let this happen to you.

One way to address this is to have someone in the sprint review meeting enter stories, defects, and tasks as they come up. Have them do this using the same tools you always use note cards, JIRA, whatever… At the end of the sprint review, have that person review everything they captured to ensure they captured it all. Regardless of what processes you use to actually address these issues, you now have the ability to because you’ve captured this information.

Rapid prototyping for faster collaboration

This weekend, I was building houses for developers. As we were framing the first house, I suggested that the location of the door might be improved if it were shifted from where the original plans stated it should be. After a very short debate, both the product owner and the developer agreed, and the updates for the plans began.

As the new designs started to emerge, we began to understand that by moving the door we’d have a 3 to 5 inch strip of wall between the edge of the wall and the door. This created a problem. Since the entire front of the house was originally whiteboard, it no longer made sense financially or functionally to have a three to five inch strip of whiteboard: you’d never write on that strip and you’d end up with a lot of scrap whiteboard if you were to cut for that space.

Let me lay out the scene in a different way and you tell me if this all seems familiar:

The actors:

• A product owner, who has a clear internal vision of what he wants, and strong opinions.
• A developer, who knows what is and is not possible, and the trade-offs involved in what’s being asked for.
• A tester or potential user, whose opinion is occasionally asked for, but whose feedback is more often than not, given knowing it’s going to cause friction.

The requirements:

• On the front of the house we require a door, a light, and a space for whiteboard.
• We have a standard that says anywhere there’s a seam between building materials we need to put up trim to make it look nice.

What’s open for debate:

• The size and placement of the door, the light, and the whiteboard, all with the goal of balancing functionality and aesthetics.

When you think about the problem in these terms, it’s a classic software development problem. And after thirty minutes of discussing what our options were to solve this problem, I decided to leverage a classic software development solution to this problem – I suggested that the product owner and the developer create a mock-up that they could interact with.

Using various building supplies laying around our workspace, we did the following:

• We laid one of the framed out walls on the flat on the floor.
• On top of that, we laid out two sheets of siding and a sheet of whiteboard.
• On top of that, we laid out our door, trim, and a cordless drill which represented our light fixture.

We had a full-size very rough model of what we were talking about. At this point, instead of going back and forth trying to discuss the trade-offs in design by saying things like “If you move the door over three inches you’ll…<insert bad thing here>” the product owner and developer could just move stuff, and the trade-offs were immediately apparent. No detailed explanations where required – everyone was on the same page.

When most people think of wire framing, paper prototypes, or other forms of interface mock-ups, I think they envision expensive tools and long lead times so trendy designers can get it “just right.” And sometimes that’s true. But there’s value having the ability to pull together something physical – something you can interact with – in a quick and dirty way.

Once we made our final decision, the developer went back and updated the formal plans which provided a wire — frame picture of the final house with precise measurements. You still need those final design specs. But you also need tools that allow for faster collaboration. You need the ability for the product owner and the developer to push and pull on the same objects at the same time.

What do you use to do that type of rapid prototyping on your projects?

Testing estimates in Scrum, taking it one story at a time

I really enjoy like doing small projects where I get to put hands on a keyboard and execute tests. Ideally for me when I’m doing this, I’m either doing exploratory testing or performance testing. There’s a challenge that comes with doing this however. For these small projects to work well for both me and my clients, I need to be really good at estimating my time. If someone asks if I can test their application for certain risks, I need to be fairly certain of how much testing I’ll need to do to shed light on those risks and how much it’s going to cost them. Often when I provide these types of estimates, I do the following:

• provide the usual disclaimers about what I can and can’t do as a tester (there’s no such thing as complete coverage, just because I test an area of the application doesn’t mean I’ve found all the issues, this is what I mean when I talk about test coverage, etc..)
• provide a scope summary of what I think they’re asking me to do
• provide a range for how long I think it will take (for example, 5 to 10 hours)
• provide a due date for when I commit to having it done
• provide a rate for what it will cost (either hourly, fixed bid, or other - but I’d never price my work by the defect, sorry uTest)

If I’m working with a team I’ve worked with in the past, all of this fits in a couple of a paragraphs in an email. That’s because they won’t need the disclaimers. If I’m working with someone new, it might be a one or two page statement of work.

Here’s why I think this is important…

I see a lot of testers working on sprint teams struggle with estimating their testing for stories. In my mind, it’s no different then the process I just outlined above.  Assuming you don’t know in advance which stories will be delivered with other stories, then each story is it’s own little project waiting to happen. Based on this risks involved with that story, you outline your testing scope, provide an estimate (low/high) of what it will take to do your first pass of testing, and provide an idea of costs. (Costs might not be dollars, they might be trade offs.)

However, it’s my experience that many times testers can get overwhelmed by the scope of a sprint. They feel they need to “test everything” - even if they know that’s not possible - so they don’t bother to estimate the work. Or they forget to draw a distinction between testing the individual story and the overall release the story is potentially going to production with. This can cause confusion around scope and timing.

In my mind, part of the value of breaking the work up into stories is that you get little chunks of (hopefully) independently testable code. This allows me to more easily esitmate my testing work, because I can focus on just those features and risks involved with the individual story. Testing for the overall release, would be something I might estiamte seperately. With that testing, instead of looking at issues with the individual stories, I’d be looking at issues around integration and other overall quality criteria like performance, consistency, security, etc….

Why you don’t need to buy a testing tool, except when you do

I consult with a lot of teams that think they need expensive tools. Some teams do need the features offered by expensive test tools. But most teams I’ve worked with don’t, they just think they do. Likely they could have gotten by with a workflow tool like JIRA or Bugzilla and some simple scripting languages and/or open source testing tools. I don’t understand the fascination with expensive tools. And I say that as a former technical writer for IBM Rational, whose tools I — mostly — really like!

I’ve used most major vendors’ tools; and while they’re nice, I don’t typically have the problems they propose to solve. A lot of times, I’m only solving problems because the tool says I should. An example is detailed traceability or distributed test execution. I like all the LoadRunner extensions, and I use them when I need them; but I also like JMeter, SQL, and Ruby. I like distributing thousands of GUI-level automated tests across a test lab using Rational Functional Tester, but I rarely have projects that have thousands of GUI-level automated tests.

I’m working on a project now where we don’t have a performance test tool, and someone on the team continues to mention that we need to purchase LoadRunner. I’m almost positive that we can do the testing using straight SQL and Perl along with some run-of-the-mill monitoring tools; but for whatever reason LoadRunner keeps coming up. This isn’t unique to this project. In my experience this happens a lot.

Here are some keys for figuring out if you really need that tool and its associated price tag:

• The tool provides a solution that has been proven to work for your non-trivial problem. An example is LoadRunner’s SAP extension; you likely don’t want to solve that problem with Ruby scripts. Buy or rent the expensive tool. Trust me, it’s likely cheaper. Another example might be iTKO LISA. If you have a couple of SOAP web services to test, you want SoapUI. If you have an enterprise of web services to test, with a variety of architectures and protocols, you might need iTKO LISA.
• They integrate with your existing suite of tools in a way that’s relevant to your context. For example, I’ve worked in regulated environments where traceability was truly required; purchasing tools that facilitated that was relevant to our project. I’ve also worked on projects where traceability wasn’t required, but we used certain tools because it was possible. We didn’t do it mind you, but hey, it was possible. Sheesh!
• You can’t find a cheaper alternative and don’t want to build your own. Some teams don’t want to be toolsmiths and they just can’t find a cheaper alternative. There’s nothing wrong with that. Just make sure you really looked around for cheaper alternatives before you select the big expensive tool. Don’t let it be your default choice just because they have a large marketing budget. Also, don’t read that to mean you should always be cheap. I personally like JIRA more than I like Bugzilla. That’s because I think they have a good price-to-value ratio. For me, they are often worth the price. But not always - sometimes I use Bugzilla.

Adam Goucher, Mike Kelly look inside “Beautiful Testing”

A little over a month ago O’Reilly released the book Beautiful Testing: Leading Professionals Reveal How They Improve Software. The book offers 23 essays on software testing, with 27 different authors contributing. Those contributors include SearchSoftwareQuality’s own experts Karen N. Johnson and Matt Heusser, as well as some other testing noteworthys like Linda Wilkinson, Scott Barber, Rex Black, Alan Page and Lisa Crispin.

I recently sat down to talk with the book’s co-author/co-editor Adam Goucher to talk about the process of putting the book together. If you don’t know Adam Goucher, he has been testing software for the last 10 years. He has worked at both large corporations and start-ups, and is currently running the QA/Test department for Zerofootprint Software and he maintains a popular blog.

The book, Beautiful Testing, appeals to a wide audience. It has stories of start-ups, regulated projects, and open source. When asked how the focus became so broad, Goucher relayed some of the project history:

“We originally set out to write a book which showed how open source projects tested themselves with the idea that readers could get a copy of the application and directly apply the techniques they discussed. O’Reilly, rightfully, turned that down saying that it was too niche. We retooled the proposal to include people and projects that were not from the open source world and that is what got accepted.

One thing that was important in both proposals was the need for diversity. We wanted to get people who wouldn’t just say the same thing or parrot something from prominent Tester X. There were a few people we couldn’t arm twist into participating that would have upped that significantly, but I think we still have a wide range of views expressed.”

Pulling together a number of different authors and asking for their opinions on software testing has also created an interesting tone for the book - one that Goucher is quite happy with:

“Most [contributors] are active bloggers and not really what could be classified as being in research of academia. The blog medium is usually more conversational. Also, Beautiful Testing is at its heart a book of personal opinions and ideas about what is beauty and beautiful about testing.”

So how hard is it to get 27 people to write a 350-page book? You know, in case you’re thinking of starting a similar project? I asked Goucher:

“I have a mailbox just for Beautiful Testing emails. It currently sits at over 1500. Granted, that was over the period of a year, but writing something like this results in a flurry of emails around deadlines. It could have gone a lot worse than it did, but we were very open about deadlines and the ‘why’ behind them. I think that helped people focus and made them real. (Unlike some dates we encounter while testing.)

From an editing perspective, make sure that you can handle both the creative side of things and the administrative stuff. Tim Riley [Director of QA at Mozilla and co-editor] deserves a lot of credit for keeping track of contract statuses, feedback reviews and the other organizational things that a project like this demands.

As a tester, or someone who just cares about their career, contributing to something like this is a fantastic way to learn about the book creation process and quite frankly, it looks good on a resume. Start writing a blog and increase your visibility in the community of writers and you too could be asked to participate in something like this.”

One of the chapters in the book, “Beautiful testing is efficient testing,” was written by Goucher. I asked where his particular story came from:

“When I was a non-management tester, I firmly believed that the business wanted to know about the threats to customer value. When I started to become a team lead and later manager I learned that what they really want is to know are the threats to the release date. This means I need to get them information quickly and geared towards to the threats that could have the greatest impact to the schedule.

In my chapter, I outline three of the techniques I use for that.

a) SLIME - SLIME is a mnemonic which stands for Security, Languages (i18n, l10n), requIrements (for thing that is causing the testing event), Measurement (performance, stress, load, scale) and Existing (regressions)

b) Scripting - Scripting is a fantastic tool, not just for creating automation for the front-end. The sound bite I’ve been using recently is ‘automates checking and facilitates testing’. Unfortunately for the book, Michael Bolton’s Testing vs. Checking meme started after the manuscript was completed.

c) Mindmaps - Mindmaps are brilliant for concise, visual test documentation. And it is fast. None of this spending 2 weeks writing test documentation that no one will ever read, just get your ideas out quickly and start testing. It’s brainstorming origins are imply that it will change as we learn more.”

And if you’re looking for another reason to buy the book, or to make the book a gift for testers on your team, here’s another great reason to make Beautiful Testing the book of choice. None of the authors or contributors are receiving a single penny in revenue from the sale of Beautiful Testing. All profits from the book are being donated directly to Nothing But Nets to buy mosquito nets to help combat the spread of Malaria in Africa.

You can learn more about Adam Goucher on his website: adam.goucher.ca

Excluding certain elements while recording JMeter tests

In a recent article on JMeter, I related some tips for recording JMeter tests using the proxy recorder. Sometimes when you’re recording tests, you want to exclude certain transactions like images, script code, or page types. To do that, you can enter specific patterns to have the proxy include or exclude:

Figure 1: Including or excluding patterns in the JMeter HTTP Proxy Server.

In figure 1 above, you see that I’ve explicitly included various image files, and excluded JavaScript files. For any particular type of file you’d like to list, just use that same regular expression:

If both fields are left blank, all transactions are recorded. If at least one field has a regular expression, than only requests matching (or not matching) the regular expression will be included. Any regular expression provided is matched against the full URL JMeter samples.

Using filters like this can greatly reduce the number of transactions you record. That, in turn, can reduce the amount of time it takes you to sort through the results of what you recorded. If a typical recording session gives you 200 transactions, you need to look at most of those to figure out which ones to include in your actual test scenario. Using filters, you can reduce the amount of work involved by recording only the various types of transactions you think you’ll need.

Building an online community for software testers, an interview with Rosie Sherry

There are a lot of online places for software testers to collaborate, but one of the more recent and popular entries to the world of online communities is the Software Testing Club. The Software Testing Club (STC) isn’t even three years old, and already it boasts over 4,000 members and it’s growing every day. On STC you can participate in forums, join one of the almost 100 groups, post a video, post a job, start a blog, or you can get involved through STC’s LinkedIn Group, FaceBook page, or Twitter feed.

Given the site’s popularity, I thought it would be interesting to hear a bit about how STC came about. With this in mind, I took some time to talk with Rosie Sherry, the founder of STC.

“I started it because as someone into software testing and after looking around I didn’t feel like there was anywhere online that I felt comfortable hanging out. To be honest, it was a bit of an experiment. I didn’t really think it would take off like it has. [...] I have had to rethink it all. I hope it’s going somewhere bigger than it is now. We’re trialing out some new ideas and concepts at the moment. This includes the Software Testing Magazine and The Exchange. Increasingly, other people are getting involved too, which is great.”

Rosie Sherry is a bit of a social media and online community guru. Her current company Schux, does web development and online community building. Her other projects include Flash Mob Testing (a solution for crowd-sourced web testing), the STC Exchange (think of it as a StackOverflow for Software Testing), and Lewes Werks (a social collaborative workspace in Lewes, East Sussex).

If you’re new to STC, you’ll want to start with the forums. “The forum is by far the greatest attraction of the STC,” says Sherry. “There are always lively discussions and debates going on. The Testers Feeds are also hugely popular. I’ve spent the past few years collecting tester blogs, last time I counted there were over 120.” Sherry is also very focused on making STC and it’s content independent and unbiased toward any particular type of testing. “We also try to be fresh and fun, something much needed in the world of software testing. I think this is what makes it so popular.”

A few months ago, Sherry blogged about a difficult decision she made to move STC towards a paid membership model. I asked her about her decision to move in that direction.

“I’ve spent a good two and a half years working on STC. When something unplanned happens, it makes us rethink our positions. The current main forum is now still open for free, but am working on other things - which include membership - to make it sustainable as a business. I love the STC. It makes my heart tingle. But to keep it alive and stop it going stale, time, effort and money need to be invested.”

What are some of those “other things” Rosie is working on? Well, two of them are Flash Mob Testing and the STC Exchange.

“Flash Mob Testing is about using the talented crowd at STC to do testing in the real world. We’ve trialled out a few projects and are currently working towards figuring out the best way to proceed with this. Crowd sourced testing is most suitable for smaller companies who don’t do any testing or lack the resources to find someone who can actually test properly.

I think there is a market for it. uTest seem to be doing alright, though it is still early days. Flash Mob Testing will keep the ethos from STC. What this means has not been defined exactly, but we are trying to figure out how we can educate testers in the process whilst also providing a service that can guarantee great results. This probably means creating real relationships with the testers rather than making it an anonymous thing.”

While Flash Mob Testing focused on putting the communities knowledge to work in a very hands on sense, the STC Exchange takes a slightly different approach. The idea of the exchange is to be able to ask specific questions rather than to raise discussion. Discussion would instead happen in the STC forums or groups. “We’ve just launched the STC Exchange,” said Sherry. “I hope that will link in nicely with our ethos and help testers build up their reputation whilst also being helpful.”

Hot topics on STC right now include “Best tool for UI/Click based automated testing for Websites” and “Developers as Testers?” Or, checkout some of the community-developed content, like “What Do Testers Hate about Testing.” If you’d like to know more about Rosie Sherry, check out her website to read her blog, find her twitter feed, or to see what else she’s working on.

How testers can handle switching to Agile’s short iterations

Software testers today are under a lot of deadline pressure due to Agile development’s shorter iterations. I recently hosted a panel on time management for a local software testing group, and attendees asked for advice on how to handle the frustrations associated with moving to short development iterations. While the theme of “making sure you have team buy-in” underlay the entire discussion, there were some particularly insightful nuggets related to making sure people understand why the work is structured the way it is in short iterations.

Robby Slaughter, founder of Slaughter Development, a workflow and productivity consulting company, set up the problem this way: Most software professionals in highly-tactical roles — whether they are authors, developers or marketers — are used to the idea that their work is a representation of professional capacity. So, the pro has this self-concept, and then their deadlines are changed to weeks from the traditional span of three-to-six months. Instead of working on one larger task, the task is broken up into 20 smaller pieces.

“All of a sudden that way in which you represent your own professional capacity, your expertise, is now violated. Now it’s being divided up.”

Slaughter’s main point is that the work is no longer yours. It’s the team’s. The individual parts that you work on or someone else works on are not as important as the overall result. Many times people working in iterations are dealing with that frustration. They no longer have the same level of ownership of the piece that they’re working on.

Brian Ball, software engineer with Software Engineering Professionals, had a slightly different take on how you might illustrate the same point.

“There are principles to why you do things in Scrum,” said Ball. “So if they’re uncomfortable with why we’re only doing a short section of work here, why we’re breaking this up and I’m not doing the entirety of the feature, [or] why I’m only dong this little unimportant subset; then when you start throwing stuff out of the backlog because the customer changed their mind…celebrate that. Say this is work we didn’t do and we no longer have to do. Just throw it out…So they see a concrete result of why the principle was there in the first place. They get the payoff of, ‘Oh yeah, we delayed it and we didn’t have to do it.’ [That's] because it wasn’t important — really important — at the time. It just left us with this uncomfortable and unfinished feeling at the time. “

If buy-in might be part of your problem, the panel offered a question checklist for helping understand what might be blocking buy-in.

“How do you set it up to provide it to them?” asked Francine Carter, founder of Action Coaching & Training. “How do you roll it out? Because the rollout’s the important piece to get people to begin to think about how they’re going to approach it. And if you’re thinking I can’t, then you’re not going to. What about it don’t you feel good about?”

Brian Ball recommends looking at your planning meetings and making sure all the work is being accounted for in the estimates — something that takes practice and measurement. Make sure everyone understands why the work was broken up the way it was, he said. In other words, paint the big picture.

Cory Mausolf, a career coach with Authentic Business and Career Coaching, pointed out that it can be important to make sure people are separating their feelings about the work itself from how the work is being done. Are they uncomfortable about the work? Or are they uncomfortable, he asked, “because that’s not how you did it before?”

You can find the complete audio for the panel members response to this question on the IWST website. There you can also find more information about the speakers along with additional audio from the meeting broken out by topic.