When I started my IT Career I heard this joke about developers…,

…A team of IT experts/developers are sitting together in a room bragging to each other about their skills.  Then the phone rings.  The head developer picks up the phone and talks for a few minutes.  Then hangs up the phone and says to the group, “That was the boss; he has a new software project for us.  You guys start coding; I’ll run upstairs to see what he wants.”

At least I thought it was a joke.  Too often working with software development teams, IT experts and even for managers this is the exact scenario I run into over and over again.  There’s this idea that we need to start getting something done before we know where we are at or where we are going.  What happens a year later we hear the lament,

“If I knew now what I knew then, I would have done it completely differently, but now it’s too late.”

Long before there was a PMP certification, Agile or Scrum we were developing strategies for avoiding these types of “cart before the horse” problems.  I’ve found that the first successful step is a strategic Assessment.  I outline this in my planning process on my site.

I like to think of a project like a walk through a dark crowded forest.  (I probably read the Hobbit at too young and age and it has affected how I think about these things.) If you had a map of the forest you would need to identify two things before you find a path.  You must understand “Where you are at” and “Where you want to go”.  The strategic assessment identifies where you are at today.

I’ve noticed that lots of developers prefer to start over from scratch.  Especially when working on a website.  I know that building a Network for the first time is similar for the infrastructure expert.  I think this is because when there is nothing, we know where we are at today.  We are at ground zero.  We have to build something from scratch.  As the business grows and the network grows, networks become convoluted.  So without a good change control process, most IT groups have no idea where they are starting from.   When looking at a map, it’s obvious that in order to create a path from point A to point B, you must know where point A is.  Yet IT experts seldom know where Point A is.  So when trying to get to Point B, they miss steps.  These steps become problems that haunt the IT department for years.  Unfortunately if the IT department has no idea where they are at today, they have to admit this to management.  Admitting this to anyone means admitting to fallibility in the minds of some IT experts.  Instead these experts start “coding” like our experts in the joke.  This gives the impression that they know what they are doing.

Just as important as knowing where you are going on a project is knowing where you are starting from.  “Winging it” means that you really don’t know where you are starting from.  For the client this is going to be wasteful of the time and resources while the IT team tries to figure out where they are going.  The strategic assessment becomes the tool for identifying where you are at today.  Consultants like me, work with IT departments who have never done a strategic assessment.  It often puts us at odds with IT departments.  I would recommend to any IT expert, to develop an IT assessment process.  This way when the IT consultant is brought in, it’s only a verification that you do know where you are at

Back in the ‘60’s my father was an engineer working for a NASA contractor.  His specialty was acoustics.  At first this may seem like a strange thing for NASA to worry about.  Acoustics is the study of sound.  There is no sound in outer space, why would anyone need an acoustics expert?  We usually think about acoustics and speakers.  Was my father working on a speaker system for the Apollo rocket systems?  Well kind of, but not in the way you might first imagine.

While normally we don’t think about the effect of sound itself when we engineer a solution.  If you stand near a vibrating speaker at a concert though, you can feel the sound waves beating on your skin.  The job of my father’s team was to take each stage of the rocket, put it into a room and then shake the rocket section apart with these same types of sound waves.  To do this father designed a speaker that could produce the same level of sound vibrations the rocket would create while taking off.  Then the team would focus that energy on a rocket section and try to shake it apart, again just with the sound waves.  Unlike particles that go in a straight line, sound waves are able to curve.  Potentially the sound ways could curve back battering the skin of the rocket ship again and again.  So the final goal of the team would be to develop a solution that could negate this acoustic effect on the rocket frame.

I tell this story, to explain a common hurdle for any engineering team.  Throughout my father’s career, he complained about the accounting department or as he called them “The bean counters.”  When I walk onto projects I often hear this same complaint.  From technicians, “There just isn’t enough money for our projects!”  One of the things I do as a Seattle IT Consultant is to find that money for IT projects. 

When an IT expert says, “I’ve been trying to get money for that for years.” I know that there is an opportunity.  If IT can’t get money for their projects, it’s usually because of the same problem my father was running into.  The IT department is not integrated into the leadership culture of their organization.  Instead of being part of the solution, these engineers chose to see the “Healthy Conflict” designed into the business leadership model as something that needed to be broken down and conquered. 

I spend a lot of time teaching management how to work with IT teams.  When I hear this lament from technicians I know it’s time to teach IT teams to work with management.  Often IT leaders find themselves in conflict with management.  My suggestion to you is the same advice that NASA did to engage engineers.  Instead of conflict, NASA leaders decided that if you can’t beat them, Join them.  In NASA’s case this meant bringing the engineers into the NASA management teams.  When working with IT Departments, rather than teaching them about technology, I teach them how to become part of the leadership culture within the organization.  Instead of fighting for money, management starts pouring money into the IT department.

In my conversations with clients this week something came out.  That something was one of those unmentionable subjects that many business owners are uncomfortable talking about.  There are three forms of political power in any business.  I’ll call them authoritative, charismatic and technical.  Technical power can be held by the accounting department or manufacturing or any department.  Every owner I meet agrees that the technical department is a poor wielder of their political power.  Often that political power gets in the way of doing good business.  I’m sorry to say that often the modern network architect is guilty, along with the IT department, IT Support teams and even IT Consultants of misusing that power.  Leaving many business owners feeling like IT Hostages 

70% of IT failure is actually caused by human error

Nobody becomes good at Technology without making errors.  The best developers, project managers, support technicians and technology experts in general make lots and lots of errors throughout their career.  So it’s no real surprise that a lot of the software, server and other types of technical errors happen.  Yet how often do we blame a problem on the hardware or the software instead of the technical team where the blame really belongs?  I am always surprised at how a worried technical expert will try to cover up why a technical failure.  Often there was nothing that the expert could have done.  It’s almost like technical experts have no understanding of how business really works. 

We really aren’t fooling our business counterparts when we try to cover up the failure.  As you read this example ask yourself if the business owner is really fooled.  

• Business owner:  Why is this failing?
• Technologist:  I think it’s a hardware failure…
• Business owner:  If we replace the hardware will that fix it?
• Technologist: I think it will
• Business owner:  If the problem is the hardware, then I want it replaced.  Will that work?
• Technologist: I think so…
• Business owner: Yes or no… will it fix it?
• Technologist: Yes it will 

Now move forward 6 months.  The hardware is replaced and the failure is still occurring.  The business owner comes back.

• Business owner: So why after replacing the hardware is it not working?
• Technologist:  It’s working much better…
• Business owner:  I agree, but the systems are still failing, I wanted it fixed?
• Technologist:  I think the problem is that there is more demand on the system.
• Business owner:  …and…???
• Technologist:  The new growth is taxing the system.
• Business owner:  Why didn’t you account for that?
• Technologist:  You should have mentioned that growth would be an issue.
• Business owner:  If the business hadn’t grown, would these failures be re-occurring?
• Technologist:  You asked me to fix a problem that was occurring 6 months ago.  Now I believe we have a new problem.  If we had factored in business growth 6 months ago, we may not be having this problem now.

This cycle can go on and on with the technologist trying slip away from accountability.  In the first scenario, the technologist had no idea what was wrong.  Nor did the technologist know how to handle the problem.  So the technologist made up something.  The owner knows that what he/she is being told doesn’t make sense.  The technologist keeps digging a deeper and deeper hole.  Often what ends up happening is that finally the technology falls over.  Then either someone like me is called in to fix the problem or the business fails. 

The technology architect needs to build business technology systems with accountability.  The modern network is now built with systems that measure, track and report the problems on the network.  How often though do the reports for these systems make it too the business owner in a language the owner can understand?  It’s obvious when the network fails that something is wrong.  It’s important for the modern network architect to build in the check and balance systems into the network and share responsibility for managing those systems with the owner and management team.

Planning

If you’ve worked in the IT world for any length of time you’ve implemented technology.  If you’ve implemented technology you probably know what it’s like when things go wrong.  Two or three hours on a weekend can turn into 48 hours of sheer terror.  To avoid the terror, planning is the key.  Many IT technicians think they are pretty good because they have successfully followed a set of directions or a wizard.   The reason they are so successful is because of the planning that went into creating that set of directions or that wizard.  The modern network architect needs to not only know how to follow directions, but also how to write the directions.  When the directions fail, the team is in for a long sleepless weekend.

Imagine that it’s Saturday morning you’ve started working on the upgrade since 6:00.  You’ve run into a few problem and you think you’ve just about got it.  A call interrupts your concentration on the problem.  It’s your significant other asking if everything is ok?  You had predicted about 2 hours to complete your work and it’s been three.  Not a big problem but this is definitely taking longer than expected.  You predict another 30 minutes.  Four hours later another call interrupts your concentration.  Again your significant other asking how things are going…??  This is how the rest of the weekend goes.

In the beginning of my IT career this was the way many of my weekends went.  Luckily technology was not complicated back then and I learned a lot over those weekends.  As the technology and infrastructure became tougher and more complicated whole teams of IT professionals would be having the same calls with the significant family members waiting at home.  Most of the time, our loved ones began recognizing the cycle and stopped believing we’d ever come home.

A successful weekend project is about planning.  Then testing the plan and fixing the plan until we get it just right.  In the early days we could work after hours or on the weekends.  Now with international teams there are times of the day when it’s impossible and other times of the day when it’s almost impossible.  Over the years though I noticed that my projects were getting less and less crazy on the weekends.  I wish it was because I could say that I was a better technician, or incredibly experienced.  The reality is that I was just becoming more organized.  One of my technicians summed it up by saying, “James I like working on your projects, because the instructions are so clear, nobody can make a mistake.”

With experience comes wisdom.  Get hit over the head enough times and you remember to duck the next time.  Here are some of the pitfalls I’ve found that plague weekend technology work.

Poor technical directions – So if you have a team coming in for the weekend, how much do they know about the project you are working on?  Most project managers have been thinking about the project for weeks, months and even years.  For the technical team this could be the only time they’ll be working on the project. 

Complete Documentation – Writing bullet proof instructions for the team members is an art form, but it means they make fewer mistakes.  The team will just run through the instructions, and then they are done.  Documentation for many projects is non-existent.  So technicians begin making mistakes.  Time consuming mistakes that you, the project lead, will be required to troubleshoot and fix.

No Testing of process – How often have you tested your own documentation?  Every time I walk through a process I learn where I made a mistake in documentation or in understanding the real problem.  Every large implementation has at least one test environment to simulate the problem.  Every project lead should be testing the weekend process in this environment.

Forgetting about the details – I was working on a project in Los Angeles California building out a small server room.  Suddenly we were stopped because we had not included patch cords for the servers.  I am a Seattle IT Consultant not an LA IT Consultant.  I know all the vendors in Seattle and the fastest routes to find them.  It seems silly in retrospect, but we hadn’t even planned in the costs associated with the patch cords.  It just seemed like a detail that would take care of itself.

If you’ve ever dealt with LA traffic it can take an hour just to drive to a store that carries just what you need.  So purchasing the cabling, the driving etc. took ½ the day waiting on congested freeways.  Imagine our frustration after our first trip when we realized some of the cables were too short.  Which meant another two hour round trip drive for more patch cables.  A week later we had taken at least one trip a day for small details we had ignored.  To this day I associate patch cables with traffic congestion and a symbol of unplanned expenses.

The first time I started using Microsoft Project, I finally began learning about how to track the details of a project.  An application like project allows tracking of each step between each milestone of every project.  Then by analyzing the steps you begin to see and list dependencies and then prioritize each step.  Finally dates and time can be associated with each step and the calendar can be accurately predicted.

Over time I began to realize that my portions of the projects were no taking any time at all.  Not because they weren’t complicated.  Rather because my projects were well planned with contingencies and roll back planning.  People wanted to work on my projects because they knew we would be waiting for the other projects teams rather than feeling the stress knowing that their team was holding up the project.

I’m curious to hear what other aspects of planning may have helped your projects.