Uncharted Waters

Jun 20 2019   9:39AM GMT

Removing bias from Katas

Matt Heusser Matt Heusser Profile: Matt Heusser


Katas with less biasHow do you an unbiased interview to find out if they can program?

No, not do they know a library, like selenium, or POI in Java to import Excel, or Cucumber, or Junit. I mean, can they actually take a problem and implement some code to solve that problem?

To do that, I used Katas.

It turns out that has problems.

Bias In Code Kata

The fundamental concept in most code katas is to take a familiar problem and ask the programmer to implement it in code. The problem is that word “familiar.” What is familiar to me will also be familiar to people like me. People who are then “like me” will had an advantage; people who are not will have to learn the concept while coding it. This creates a bias towards people like me.

Let me give you some examples.

  • The bowling kata will be easier to those with a knowledge of the game of bowling
  • The roman numeral kata will be easier to those who learned roman numerals in grammar school
  • The Greed or “Farkle” kata will be easier to those who played six-sided “dice games” as children or adults
  • Palindrome or Anagram katas will be easier to those exposed to a certain western tradition of education
  • Fizzbuzz requires a knowledge of the modulus, or “remainder”, or “%” operators.

These are all real problems I have actually encountered recently when using katas for interviews. I can’t tell if the candidate can’t program, or just grew up in a religious context where dice games were completely forbidden.

I take fizz buzz the hardest, because I like Fizzbuzz. As a classic programmer with a math degree, I have this natural bias. If you don’t know the % operator, you must be clueless. Yet there are plenty of self-taught programmers who don’t know the mod operator. I’ve written hundreds of thousands of lines of code and can’t think of a time I have actually used it on real production or even test code. Not once.

Overcoming Bias in Katas

I brought this issue to twitter, and got a few ideas. Paul Boose suggested letting people pick from a handful of katas, to select the concepts they were most comfortable with. Jamie Knight suggested something he called the Clifford Kata:

The Clifford Kata Has Less Bias

Clifford is fun because you can brute force it, with “if” statements looking at the values, or you can transform the middle number into a symbol, then interpret the statement as code. I tried both the first and second way. I haven’t done it test driven yet.

Eventually, after googling around, I found the weather kata, the first part of this. The only “tough” thing it requires, I think is a knowledge of file i/o and string manipulation. The candidate did not do well particularly well, but I don’t have a lot of comparison points yet, and it could be because File I/O isn’t a regular part of the tester/developer role anymore, at least for most companies. Next time I give the test, we’ll try to make it more specific to the job at hand (in this case, create some API tests given this API structure) or, perhaps, tell the candidate in advance we’ll be asking them to create a program that. requires a little simple File I/O and string manipulation.

Of course I made my own simple implementation and put it up on github. It might be fun to do test-driven as well.

Ironically, Clint Hoagland at Techsmith found a bug in my code. There was a variable that was mis-named in an assignment in Ruby. A typo. As a result, the variable was never assigned, so the output of (name) was zero. Even after running it by hand and looking at the result quickly, I did not notice the result was incorrect.

A Kata with Less Bias

It turns out we still need someone doing customer-facing testing, and it likely should not be the person that wrote the program.

Whoda thunk?

1  Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • efficacy
    The problem with such Katas can be more subtle than just not knowing. While I have been bowling a few times it has always been at a bowling alley where you play in groups of six or so and most players are *very* amateur. So when asked if I know bowling, the answer would be yes, but I see the job of bowling software as primarily telling the players who throws next (which seems to be complicated) and drawing an animation if someone accidentally gets a strike. I have seen several "bowling game" software solutions and none of them are remotely like this!
    Similarly, I was once asked to do a "write out numbers in words" task, but was criticised by the (American) assessor for adding "and" after the hundreds. In British English we would say, for example "One hundred and five", but it seems that's not how it is done in the USA!
    The Catch-22 of all this, of course is that for such tests to be fair they need to have excruciatingly detailed requirements, but we (usually) don't want to hire someone who can only work if there are excruciatingly detailed requirements.
    20 pointsBadges:

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: