A few years ago, in graduate school, we studied market-basket analysis, where retail companies looked at the receipts to try to find patterns. At the time we called it Data Mining, but when you looked at the number of receipts we were talking about, thousands per day at hundreds of stores for a year, averaging a two dozen items or more … we might call this big data.
The claim was that the stores in the analysis, and found that late at night, customers tended to purchase beer and diapers together. The theory was that the beer was an impulse purchase made by a husband on a diaper run. The company began to put some beer next to the diapers, and boom sales went up. The success of this program led to more data warehouses, data mining, and, to some extent, inspired big data.
Except it isn’t true, or at least, not the way the story implies. The Register tracked down the story; it originated in 1992, at Osco Drug stores, that do not currently place beer near diapers. Most of the details of the story turned out to be false — for example, in graduate school I was told it was Meijer, a local mega store. At the time, the professor in my information systems policy class said there were “tons of” additional examples, but was unable to produce additional information.
The new generation of Data Mining is big data; data so large, and sometimes unstructured, that it needs to be processed in something less like a relational database and more like google. As one Microsoft employee at a conference put it to me last summer, the reason Facebook, Chrome, and other tools are free is to get your data. By monitoring media consumption, the advertisers will be able to serve up exactly what the customers wants, when they want it, delivered in such a way that the customer isn’t really even aware they are being sold.
Color me skeptical.
I doubt those tools can actually do what they promise to do. When Edward Snowden, the American Expatriate, is afraid of government surveillance, America tends to listen – but we seem to be ignoring the same risks when it comes to corporate data gathering.
I have heard a lot about phantom managers, or sock puppet managers, for a long time now. It was just recently that I actually saw one.
The team I was working with was working on a three week release cycle. About a third of that was spend on tasks that had absolutely nothing to do with the release — mainly updating documentation. The value of that was unclear to the people doing the work, but for some reason it had to be done.
Things got weird when we started asking questions.
Hiring a unicorn, a person with the exact skill set you need across the board, is the hardest thing every company wants but will never get.
The irony that these people are called unicorns seems to have been lost along the way.
Let’s take a look at the problem and what you might do about it.
How honest is your resume, really?
I am a member of an old school yahoo group themed around software testing. It is mostly inactive now but every once in a while a new topic will come through and reignite the group. Last week the topic was when it is appropriate to lie on a resume. The answer is obvious: never. It is never appropriate to lie about experience on a resume.
There are systems, I mean that literally and figuratively, that encourage dishonesty in job hunters. There is also a way out of the mess if you’re willing to do some work.
The history of late 19th to middle 20th-century business is one of consolidation; companies bought other companies to expand. Paul Graham called the new emergent companies Duplo companies because they dominate an industry and act as building blocks for the economy. A hundred years before the outsourcing fad, National Biscuit, American Steel, Standard Oil, and AT&T did everything themselves.
The extreme end of this was Henry Ford’s Rouge River Plant, which was designed to literally take iron ore in at one end and send assembled automobiles out of the other. Built between 1917 and 1928, the factory employed over 100,000 workers at its peak, and it seemed like the future.
Today, a great deal of design and manufacturing happens in a long supply chain, whose products the car companies ultimately assemble and sell. The reason car companies operate this way is that they think it works better. Each company in the supply chain focuses on what they know best. The have to; if the suppliers can’t compete, they’ll either be replaced or go out of business.
Now let’s talk about technology projects and specializing roles. Continued »
It was a short trip; in Tuesday, out Thursday. See a client, work out of their office a bit, go to a user group meetup, head home. You’d think, by now, I would have this down pat.
In some ways I did. All my tech and clothes in one backpack, fast-on off shoes, a lightweight coat with all my things so nothing would be in my pants pockets for security. The flight was out of Kalamazoo Michigan at 7:00AM, an easy half-hour drive for me with great parking and fast security. My things were by the door to go out, the alarm set for 5:00AM. Assuming 15 minutes to dress, I could be at the airport by 6:00 with plenty of time to spare; my mind even work me up at 4:55 to turn off the alarm without waking anyone up.
Then things fell apart. Continued »
It seems like I’m starting a software testing meetup in Nashville.
There is a Slack group for software developers in the area that has channels, one of which is dedicated to testers. A few of the regulars decided we should get together to have a cup or two of coffee together, and hang out and talk shop. That was probably a year ago. We had a second meet in January of this year, and then another one just happened. And, now people are thinking about March.
There was no grand plan for getting a new event going, it just ‘sort of happened’. I think I understand the how and why, if you are interested in meetups or want to build one yourself, this is for you.
Automation is here for good in software development, and it will have big affects on how we work. But not in the way you might think.
In 1779 a man names Ned Ludd got tired with mechanization and industrialization reducing the number of jobs available for skilled craftspeople. Ned led a group of people in what he called ‘machine breaking’, sabotaging the machines in hopes that it would stall progress from industrialization. They broke a few machines, but couldn’t keep up with the pace that they were being made. These people eventually became known as the Luddites.
Automation and using machines isn’t going anywhere for the software industry either. If anything, it is on a rapid upswing right now. Let’s take a look at the unintended side effects of this trend.
Agile has never been anything more than a set of guiding principles. That is both a blessing and a curse. Teams that are trying out something new have very little guidance on where to go without an experienced person to give guidance. Others that have been ‘doing agile’ for some time see deviating from the popular frameworks and practices (TDD, BDD, KanBan, Scrum, automate all the things) as being less agile.
If you missed part 1, where I introduced Andy and the GROWS method, you can find that here.
As an employee, I never had enough time to do everything. When you think about it, that sort of made sense; management wanted to wring every drop of value out of me. If I was working on four things at the same time and could meet my deadlines, why not assign Matt a fifth?
That was a long time ago, back when projects worked in big batches, and you would “juggle” projects, with one that needed serious attention to code while another needed less in test and a third even less in requirements. I understood the thinking, and figured once I started my own business, then things would be different.
Then I started running my own business, and things got worse.
Let’s talk about why that is, and what to do about it.