Uncharted Waters

September 18, 2014  12:51 PM

Twenty Years, Seven Companies, Nine Different Styles of Testing

Michael Larsen Michael Larsen Profile: Michael Larsen
Software testing

Hello to everyone who reads “Uncharted Waters”. I am excited to be one of the writers to contribute to this space, and looking forward to collaborating with Matt and Justin on future posts. For those who do not know me, I’ve been a software tester for twenty years (a lot more about that below) and I have had a wide and varied career, touching many different industries and disciplines. Through them all, the primary theme has been to test software and devices, and to encourage other testers to learn more, do more and dare more. It’s my hope that I can continue that tradition here and in future posts.

Cisco ASM Router

In addition to testing Cisco routers, in my earliest days, I helped build them. This is an ASM router, on display at CERN (image courtesy Wikipedia).

I officially became a “software tester” in 1994, when I was working with Cisco Systems. Actually, the title I was given was “Development Test Engineer”. I’ve had a variety of job titles since then, such as Application Engineer, QA Engineer, QA Tester, Lead QA Engineer, and other variations that HR departments deemed necessary. I was given the choice of choosing my own title (within reason, of course) when I went to Sidereel. I asked them “Can my title be “Software Tester?” After a quizzical look, they said “Sure, if that’s what you want.” Only once in my career have I had my actual work discipline in my official title. By any other name, do we not still test software the same?

Actually, no.

The past twenty years have taught me that experiences with testing are all unique. I have worked for seven different companies. Each has had its own take on what testing is and what made testing “good”.

At Cisco, planning your work and working your plan was a huge part of the job. In the 90s, ISO 9000 was all the rage. I would create test plans, trying to think of every possible area, only to have it turned back with “sorry, we need more detail”. I learned the art of “turning the requirements sideways”. Every listed requirement was reworded into a test case. Did it help me test better? It made for a massive checklist, but I’m not sure it did much to improve the quality of the code.

Testing Virtualization Software

Wow, what a blessing. The systems were streamlined. It didn’t matter what the real machine hardware was, the virtual machine abstracted away everything. One machine to be used everywhere. We did not have voluminous test plans. Instead, we did lots and lots of bug reporting, followed by numerous bug fixes and retesting. Each setup and teardown was clean and seamless, but we worked with a lot of images for all the possible software combinations.

Heat pump chamber for environmental testing.

Some of the most interesting and challenging testing I ever did was based around environmental extremes.

Synaptics was all about touch. Everything had some aspect of how the body carried electricity. Additionally, I learned a lot about environments, including extreme heat, extreme cold, extreme wet and dry, extreme turbulence and various other conditions, not to mention getting pretty handy with an ESD gun. It was heady and scientific. I often struggled to understand what I was doing. When I did understand it, though, what a ride!

Time For Video Games
“Testing games; that has to be the most fun job ever!” Truth is, do not mistake playing games with testing games. For example, take a movable character and rub it over every surface to see if you fall through any cracks. Repeat for every scene. It’s not fun, though that was just one aspect of testing games. The most memorable experience was the language barrier. Every test plan, bug report and test report had to be reviewed for consistent language use. A QA Liaison would take all of our test artifacts and translate them into Japanese. The next day, we’d receive replies from Japan (translated back into English) that we’d review in our daily meetings. I learned a lot about the precision of words. It takes discipline to write unambiguously (or as close to that as possible).

Immigration Law Case Management
Case management for foreign nationals requires strong analytical skills and attention to detail. My testing had to pass legal audits with the tiniest details intact. My best testing tool was an abundant use of personas, making up entire families and back stories, to see if I could follow them all the way through the necessary workflows. It required learning many corner cases and scenarios, but it proved worth it in the long run.

Testing a Video Content Site
The world of television required a total rethinking of who my customers were at any given time. The end users, the studios that generate content, aggregation services that gather links, the advertisers, etc. are all different customers, each with a different expectation. The ability to focus and defocus rapidly proved extremely valuable.

Enter Socialtext
Today, I work for a company that creates collaboration tools. Our system focuses on one piece flow. There are no bugs while a story is in process. We talk about issues, rework, missed test cases and under-scoped features. We deal with bugs when they hit our staging server (which is our production environment). Testers get involved early in the game. To minimize rework and story churn, we aim to be at every story design and kickoff so we can articulate issues with requirements or implementation ideas.

Is There a Right Way to Test?
Anyone looking to say “there is a right way to test” will feel silly if they trace my path. There are many right ways to test. They are as unique as each organization. I am certain of one thing. Each company helped build skills that informed every job that followed. I’ll learn a lot more in the next twenty years, though I can’t say how many companies or ways of testing that will include.

September 17, 2014  9:46 AM

Ageism in the tech world

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

Next time you are at the office, take a look at the average age of the folks around you. The people doing the tech work; programmers, testers, ops and admin people. You’ll probably see some folks in their 20s, a few more in their 30s. From there the numbers quickly drop off.

My data sample is admittedly biased, and definitely unscientific. I’ve worked at mostly early stage startups for the past few years and those have a tendency to attract younger people for a variety of reasons.

Probably partly because startup benefits usually aren’t very competitive, and partly because of the crazy hours most tech startups expect from people.

The question of why people leave tech so early is interesting.

I want to know where people are going when they leave hands on tech jobs, and why.

Continued »

September 15, 2014  8:42 AM

Ballmer’s Exit, Legacy, and the Future of Microsoft

Matt Heusser Matt Heusser Profile: Matt Heusser

Steve Ballmer in 2010If you’ve been on the internet in the past few weeks, then you’ve probably heard that Steve Ballmer resigned from the board of Microsoft in August, just seven months after he resigned as CEO. The commentary on the internet is about what you’d expect: Not just Top Ten Reasons Steve Ballmer Failed as Microsoft CEO, but also similar-themed articles with the exact same title “Why Steve Ballmer Failed” in The Atlantic, The New Yorker, and on BusinessWeek.

Ironically, all the folks who said Steve didn’t innovate picked the same title for their article.

Yet let’s look at reality here: From 2000 to 2014, Ballmer took Microsoft from $25 billion in sales to $70 billion. That computes out to 7.5% growth in sales per year — starting when dotComs were over-inflated and continuing through the real estate crisis of 2008. Growing at 7.5% in one year may not sound impressive, and it might not be hard if you have annual sales of a few million, but try having to add a few billion in new sales each year and doing thirteen times in a row.

How it is possible to call Steve Ballmer a failure as a CEO? In what way?

Let’s talk about it. Continued »

September 9, 2014  12:44 PM

Starting freelance: Budgets and personal finance

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

In the first part of my Starting freelance series, I talked about a few tactics for developing negotiation skills. Hopefully you were able to use that to get a little more money from your tech freelance gigs, or at least get the rate you were shooting for.

In the second part I wanted to talk about a few ways to I am currently preparing myself financially for the ups and downs of the independent life. I’m not independent yet, but preparing has been an interesting experience.

I’m hoping that preparing and developing these ideas now will pay off in the long run.

A lot of this could apply to any type of freelance / independent worker, but this will be directly related to going independent in the tech world.

Continued »

September 4, 2014  10:44 AM

More Economics of Hiring

Matt Heusser Matt Heusser Profile: Matt Heusser

A job interviewAfter three years as an independent  (and fifteen more with a day job), I’ve noticed a few trends. Among them: I’d say that a minority of the people actively looking are demonstrably capable of the work. It’s not that there is a ‘shortage’; it is that many good people are not actively looking.

Imagine this scenario: Hiring managers want above average people, HR pegs salaries to market averages, while 3rd parties, who claim to offer a service are actually arbitraging the market, trying to buy labor cheap and sell expensive. Temp agencies, contract houses, companies that do to temp-to-hire and outsourcers are all trying to profit from the billable workers. At the same time, the companies are under intense pressure to get good people who can “hit the ground running” and are available “right now.” Say the steps above are the recipe for a cake. When you try to bake it, what do you expect will happen?

Continued »

September 2, 2014  2:55 PM

The iCloud leak and consumer advocacy

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

Over the weekend there was a very public iCloud breach that led to personal materials from several celebrity women being leaked and then published onto public sites. The bug appears to have been weakness in Apple’s Find My Phone feature. The security problem allowed access to a persons private iCloud with a brute force attack using a library called iBrute. Apple may have fixed this issue; if that is true the leak should be done.


Continued »

August 29, 2014  9:34 AM

Software Developers: A Dime a Dozen?

Matt Heusser Matt Heusser Profile: Matt Heusser

Consulting PictureA few months ago I was talking to an executive from a consulting company I respect. We were talking about doing business together. The executive pointed out that his company was known for across-the-board development, from concept to production, but they also wants to start more specialized practices, like security, big data, cloud operations, and project rescue.

Later in the summer, I read the same ideas as part of IBM’s new strategy in Cringley’s book, The Decline and Fall of IBM. It made me wonder – why does everyone want to get out of writing code?

It could be that IBM is taking a page from its own play book, exiting markets that are no longer profitable, just like it did with mainframes, PC’s, and laptops.

What does that mean for the rest of us?

Continued »

August 25, 2014  9:56 AM

ISO 29119: What you should know about software testing standards

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

ISO 29119 is a 5 part standard for software testing process and practice that was published in 2013. Standards are commonly used in other professions such as law and medicine to protect the consumers of these services. Medicine and law have existed for quite a while now and have had time to mature to the point where they have mostly accepted bodies of knowledge. Software is a very young profession, and software testing younger still and very much in flux.

stop 29119

Continued »

August 20, 2014  12:10 PM

CAST 2014 Keynote: The state of healthcare.gov

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

Ben Simo, former president of the Association for Software Testing did a keynote this past week at CAST2014 on his experiences with Healthcare.gov. The keynote was about an hour of Ben describing very real problems he experienced while trying to seek an insurance plan for his grand daughter. To be honest, what he experienced was horrifying. The problems Ben experienced ranged from not being able to create an account to significant security issues. You can find descriptions that experience here and here on Bens personal blog.

It is important to note that Ben was not ‘hacking’ the site in any regard. He had an authentic healthcare need, and actively sought to communicate the issues he found to the proper people. He was able to isolate and describe these problems because of the years he spent developing developing skills as a software tester.

Here is an interview with Ben about some of his work.

One interesting question came up during the talk:

Was the massive initial struggle with healthcare.gov caused by bad software testing?

Screenshot 2014-08-20 08.24.28

I’m not so sure.

A lot of the pressing issues with the website have been resolved now and people have been successfully using it to sign up for insurance. There are some clear lessons we can take away from this that are representative of how most software projects work.

Continued »

August 19, 2014  9:39 AM

Of IBM’s Decline

Matt Heusser Matt Heusser Profile: Matt Heusser

Book: The Decline and Fall of IBMAnyone can write a “hatchet job” article attacking a tech company — for awhile there, the daily Yahoo bad news was a running joke. Robert X. Cringely’s newest ebook,  “The Decline and Fall of IBM” is something entirely different.

Cringely didn’t do it for the money; “Decline and Fall” is an ebook priced at $3.99. At the time he published it, he had a different, hardcover book contract with a publisher requiring their work be the next book Cringley would publish. With a heavy heart, Cringely returned the advance, worth hundreds of thousands of dollars, and put the IBM book into the world.

This book is a labor of love.

“Decline and Fall” is the work of a real journalist, investigating what had happened to a company he had once admired.  A story of a love lost, the plot is all too familiar, full of outsourcing, layoffs, and incompetence.  Sadly, it is familiar for a reason. The root of problem is a poison that is all too common in American business, and may be infecting your company as well. Continued »

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: