Uncharted Waters

Dec 17 2019   11:36AM GMT

Agile History: Revisited

Matt Heusser Matt Heusser Profile: Matt Heusser

Tags:
Agile

The growth rate of the software field in the USA means that more than half the people have less than ten years of experience. Meanwhile, Scrum has been established as the default way of doing software for about ten years. Combine these facts and you get one stark reality: The majority of people in software today have spent their entire careers in a world where Scrum was “how we do it” and waterfall was “old and busted.”  It seems that on a daily basis I am told that “Agile” is bad and “sucking the life” out of team members. We have forgotten our Agile History.

This is a pattern. I’ve seen it before.

And, with a little Agile History Lesson, we can stop it from happening again.

In the olden days

History of AgileI mean the actual olden days, before interactive terminal screens. In the 1970’s, the typical line manager had not seen a computer at any time during their schooling. The number of CS Degrees in the general population was essentially zero. There was no code completion, no rapid prototypes, most programmers were self taught.

To traditional management, what the programmers were doing might as well have been magical incantations.

While I wasn’t in computing at the time, Jerry Weinberg was, and wrote about his experiences in his Quality Software Management, volume I. Weinberg describes various patterns. His pattern one, variable, is defined as “do whatever you feel like at the moment.”

Imagine all those managers, so used to a sense of control, asking a programmer how the work is going, and hearing back “it’s going.” Asked when the work will be done, the programmer shrugs, and explains that the “bing-boing isn’t connecting to the flurb-flob” but they may be able to use an “RSS Twinkle.”

Well, it sounds like that, at least to the manager.

Consider the brick and mortar companies, where Information Technology (IT) was viewed as overhead and not competitive advantage. In that case,  IT was in a place to tell the customer what they would get, and to change their minds. In Weinberg’s example “… Company C’s Engineers schedule dates to meet announcement pressures, then forget about the schedule and work any way they want.” Weinberg called this an “engineering-driven” culture. Companies that developed software tended to do a little better, as software drove the bottom line, and the leaders tended to be more technical.

When it came to military and government spending, the schedule carnage was often incredibly bad. The software was late, and the government would continue to pay for the delays. This did not work for everyone. Something must be done about those technocrats. Can’t we find someone that can translate for us?

Enter the CMM

The folks at the US Department of Defense (DoD) were sick of paying too much for projects that were too late and did not work, so they asked for proposals. The University of Maryland Proposed Goal Question Metric, that first asks where you want to go, then what could be measured to help you get there. That was a bit meta for DoD, they wanted something more concrete, so they went with Carnegie Mellon’s proposal, The Capability Maturity Model, or CMM.

The CMM proposed five maturity levels – Ad-Hoc, Repatable, Managed, Measured, and Optimized. Each level had a set of practices required to advance. Weinberg’s work on patterns and culture was heavily influenced by the CMM.

Those CMM practices required a lot of things. Project Plans, Job Descriptions, Requirements Documents, Peer Reviews of every work product, coding standard, documented test plans. In my own personal analysis, “doing” the CMM meant spending a lot more money on documents and project-shepherd type roles. On the plus side, projects were more likely to meet their new, inflated budgets and schedules. It was poison for innovation, but might be fine for a contract order to deliver to defined specifications.

One side effect of the CMM was the people actually doing the work were separated from the customers. We tended to call people embracing the CMM the “plan driven” types – whose goal was to limit or prevent change on projects. The alternative, the push-the-pendulum the other way types, were pushing what we called “lightweight methods.”

Agile History

Brian Marick, a co-author of the Agile Manifesto, once told me that the whole point was to point out that the people actually doing the software development had been studying it for a real long time now, and we had figured it out. At least, figured it out well enough that we didn’t need the handlers and the people people in the middle. If you just let us, you know, talk to the customer, collaborate on what we were building, and allow that to change throw incremental refinement, the end result could be something more fit for use than what we envisioned at the beginning of the project. By working the most important thing first, we might actually finish early, finding the next project more valuable than the bottom few items on this project.

Looking at the wording of the Agile Manifesto, with the emphasis on working software over comprehensive documentation, you can almost see a trick. It is almost as if the manifesto was written so the big, giant, expensive consulting firms of the day could not claim to be agile. Their very marketing materials stressed the things the Manifesto said were less important. For example, requiring that every change trace back to a requirement that is funded up-front, or creating a change control board to prevent changes. The Agile Manifesto would portray emergent opportunities for new ideas as competitive advantage!

The pendulum swung back.

… time passes …

The Agile Alliance is created to support the manifesto, then the Agile Conference becomes the premier event. The Agile Conferences merges with XP Universe, a technical conference.

… more time pasess …

Today the Agile Conference has one or two technical tracks, and a dozen more about metics, leadership, the enterprise, the team, and workplace culture. There is even a track on how to coach when you cannot understand or do the work the technical team is doing.

The people-people have taken over again.

It’s Happening Now

Save Our ScrumExtreme Programming (XP) challenged organizations to go where they did not want to go. Eliminate the Project manager as a role, eliminate the “QA Department”, the business analysts and requirements people. There is not even a role in XP for development manager. What are we going to do with all those people?

And so the world adopted Scrum, turned project managers and analysts into product owners and Scrum Masters. Teams adopted Stories, Standups and Sprints. Markus Gaertner and I wrote a book about it.

The people-people are firmly in the building. That is okay. From what I’ve seen, this generation is a lot more open to studying what is actually happening in a group before prescribing solutions. Besides, it is almost inevitable that the people who pay attention to social groups take over any social group they participate in, at the expense of the folks who focus on things.

Things and People

Twenty years ago I was firmly in the “things” camp. I remember walking into a meeting, firmly convinced that there was no way the project could get done in less than three months. Half an hour later, I walked out committed to six weeks. What I thought was about estimation turned out to be a negotiation, and I was negotiated into the impossible.

Eventually I realized I needed to learn the people-people skills in order to even be in the room. (Here’s one example; I’ll list more on my next post.)

So there you have a brief overview of where Agile came from, the problems it solves, the very real possibility that history will repeat itself … and what to do about it.

Epilogue

In the blog post above I make a set of sweeping generalizations. Generalizations are false by definition; there have to be cases where they are untrue. Otherwise, they are not generalizations.

Two things.

First, you actually read it.

Getting people to read is a bit like getting a child to take medicine. You need need to entertain, at least a little, to get it to go down.

Second. I submit the details are accurate enough for a blog post. Maybe, even, to get some important conversations to happen that we, as a community, have been avoiding.

Who knows?

Stranger things have happened.

 

 

2  Comments on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • lcrispinagile
    A nice perspective, looking back to the early days of IT. We can definitely learn from history!

    When I first read Kent Beck's _XP Explained_, I noticed it was all about people, hiring good people and enabling them to do their best work. About the same time, Alistair Cockburn published his paper with his study conclusion that the best predictor of project success was hiring good people and getting out of their way. (I think that study also showed the most effective communication is two people and a whiteboard).

    I've heard that 80% of what we call "bugs" are really communication problems, so maybe it IS wise to focus on people?

    With XP and later the Manifesto, we focused on the values and principles, and of course the practices were also important, though after a couple years I eased up on the "We must do all these practices!' idea. Scrum is a nice way to help a team transition to agile. But it does not promote any values, principles or practices, just a project management framework. I've worked on a high-performing Scrum team, and I also worked on a great waterfall team back in the day. Because it's really about getting good people and letting them do their best work.

    Logan Miles wrote a post recently  where he pointed out that teams who work to adopt continuous delivery and DevOps culture end up also transitioning to agile. But teams who have the goal "transition to agile" often get stuck in the ceremonies and process. Focusing on *doing* seems more powerful than trying to fit into a framework.
    10 pointsBadges:
    report
  • Matt Heusser
    thank you lisa! It may have inspired something ...
    5,050 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: