Alias: Reflective Improvement Frameworks
Alias: The Rise of Reflective Improvement Frameworks
We have finally matured. What we now produce are Reflective Improvement Frameworks, not methodologies (though we often still say that out of habit). Crystal, Scrum, Kanban are examples. Each contains a few rules, not enough to run a project, but enough for people to improve their situation better than merely inspecting and adapting. Old-style Big-M Methodologies are gone. And that’s good.
At the Agile Roots conference in 2010 I gave a talk entitled The New Methodology Isnt a Methodology (see also the video of the talk: Video of Alistairs Agile Roots 2010 keynote - the new methodology isnt a methodology.
The point of that talk was that we, by habit, call things “methodology” in habit from what was produced back in the 1980s and early 1990s. We have matured quite a bit as an industry since then, and the things we produced starting around 1995 stopped being anything that fit the old and real definition of the word.
Scrum, my Crystal family, and David Anderson’s capital-K Kanban (please, no matter how tempting and confusing it may be, do NOT confuse with the little-k kanban, a marker system for tracking system state): these are much too loose of constructs for the term Methodology. They contain no process to speak of, and are mostly Reflective Improvement frameworks, or something with another name, or something for which we do not yet have a name. But they are not methodologies in any recognizable sense.
Let’s go after this in 2 steps:
- they are not processes/methodologies;
- and we still need processes/methodologies.
1. They are not processes/methodologies.
A process or methodology describes roles and steps and connections. Post-CSM Scrum sort of does that: Product Owner, ScrumMaster, Sprint planning meeting, etc. But that’s really (a) still too loose, (b) a bit of decoration on Real Scrum. Real Scrum (imagine a TM there) simply says:
- Assemble a cross-functional group of sufficiently skilled people
- Have them demo or deliver often but otherwise stop micromanaging them
- Have them inspect & adapt daily and monthly.
- Allocate one person with free time to remove roadblocks & watch the quality of the team
- Have the business side speak through a single person.
That’s not a process. That’s a <something>. It’s enough words so that the team can deliver and get better.
Crystal is less and more than Scrum. It says,
- Every project needs it’s own, locally derived process/methodology
- We have learned some rules/theory/laws of how people work well together; pay attention to those.
- Here are 7 properties of successful teams, it’s a good idea to pay attention to them: e.g., delivery often, reflect often, pay attention to the quality of the community and of the communications, get feedback at every level in every direction.
That’s less than Scrum because it doesn’t call for the process elements of Scrum. It is more than Scrum because it actually lists the properties the “laws” or theory of team design, which no other similar framework does.
Capital-K Kanban is somewhere between Scrum and Crystal. It does not have any fixed process elements, but it adds the idea of cross-organizational queue observation, the idea of placing hard limits on work in progress (WIP), and discussing what’s happening whenever the WIP limits get violated or stuck. In other words, there is a state-based trigger to reflection, rather than a time-based trigger.
All three of these are really great – and they aren’t methodologies or processes. We don’t have a phrase for them. For now, I’ll suggest
- Reflective Improvement Frameworks.
2. We still need processes and methodologies.
Reflective Improvement Frameworks are wonderful, but they don’t tell a team how to work and a team needs that. In What is a process good for? (discussion: Re: What is a process good for?), I discovered that a process serves
- To work with reduced communication
- As a checklist so people don’t forget something
Those are both important. A methodology (different from just the process portion but also containing the process portion) serves to define the roles and assignments and points of interaction of people across assignments and departments.
As you can see, specific, local processes and methodologies are still needed, will always be needed. (I describe how to capture these efficiently in Describing methodologies more simply (discussion: Re: Describing methodologies more simply)).
What I am projecting in this little blog entry is that “Process” or “Methodology” as we have seen it over the last 30 years – a thick manual that lists roles, assignments, techniques to be used, points of interaction – written as a book of rules, intended to be used across many organizations, many projects, many countries, many cultures, is over.
Some people will still expect those, and we can expect those people to ask for it/them. But we can’t satisfy those people, because we, the software industry, have realized in large numbers that such Processes and Methodologies are severly sub-optimal, and are increasingly refusing to produce them. Woe to anyone who produces such a thing, woe to those that buy them and expect them to work.
(And, just for fun, I am Almost signed up to prepare a SHU-level (discussion: Re: Shu Ha Ri) document for a Very Large client, so that they can move n,000 developers in n00 companies in N countries to a gong-driven transition to agile! Won’t that be a test?!!! I really hope I get to do that, because that will stretch me to my limit. I don’t really think the contract will mature, or they’ll let me, or they’ll do what I write :),but it’s a superb thought experiment. (postscript a while later: the contract never matured; whew! close call)
Your thoughts?
(This little article was translated into Japanese through an interesting crowd-sourced translation effort initiated & orchestrated by Kenji Hiranabe using the Astah mind-mapping tool. Even though I couldn’t read the Japanese, I liked what I thought I detected – an unusual and interesting use of mind maps for crowd sourced wordsmithing. Congrats and thanks, Kenji.)
Great post Alistair. I agree with you that when trying to frame these topics in to a methodology you can get blank looks when you can’t define roles and process definitely. I would argue these terms are a compilation of actives which promote cultural shifts intended to improve or promote something which ultimately increases business value (in terms of time to market or value based). Again this definition is loose but I believe if you try to narrow it down you do exactly what these terms are trying to move away from and that’s framing too narrowly.
-by Patrick Phillips on 5/25/2011 at 3:09 PM
I love the idea of using the phrase, “Reflective Improvement Frameworks”. This idea is mind-expanding, and shows how critical terminology can be when we communicate. Additionally, is places emphasis on reflective improvement. As you have stated, it is “The Secret Sauce” to an effective team.
A big takeaway I have had from this article, is that it seems as rules decrease, discipline must increase. This places more emphasis on the individual, and the value of having the right people in place. Whereas, the industrial age required more emphasis on process.
Small side note; there is a typo in one of the bullet points under header #1: “Have them insspect ”
Thank you again,
Matt
-by Matt Hansen on 5/25/2011 at 6:14 PM
1. Best of luck getting the megaClient.
2. Is this fairly close to what Jacobson suggested with his practice cards?
Our approach has been to set a framework with (UP-based) milestones, then let the teams decide which practices to implement to attack the risks required to achieve those milestones. It isn’t as if they don’t have a process, it’s more that the practices within that process are selectable and then adaptable to meet the needs of their purpose.
This is slightly different than offering the team full freedom: “Here’s a supermarket, go feed yourselves!”
It’s more like a meal plan a la carte: “For appetizer, I’d like escargot; no soup today (feeling a bit bloated); the super deluxe bubba burger as entree, but hold the mayo; peas and carrots as a side dish, use sea salt and lightly, please; and for dessert, the tira misu.”
Process ‘courses’ are established and served in an order; available items are listed (you don’t roll your own), but you can choose practice seasonings and skip a course if it isn’t necessary for your goal. (Where ‘you’ = ‘team’)
Seems close to Crystal. Or am I really not getting what you mean?
-by Skip Pletcher on 5/26/2011 at 11:05 AM
- Hi, Skip. I think, No, not quite… the difference is that Scrum, Crystal and Kanban all say what you really /ought/ to do, not as process steps, but as something specific to pay attention to. Ivar’s cards don’t. (RUP did say what to do, it didn’t provide much in the way of a reflective improvement framework.) Merely having cards doesn’t give much guidance. Scrum says, “Make sure you have a proper cross functional team, single voice of customer, then ship every month and reflect every day and every month.” That is a lot tighter set of rules than merely holding a bunch of practice cards in your hands. Kanban says, “Stare at the WIPs and the queues, then reflect periodically and when things get out of balance.” Crystal has a lot more guidance, though fewer rules.
- The point is, I think, that each of these provides some guidance in the stare-and-reflect business and some rules that are supposed to help you tell if things are working or not. Ivar’s cards miss all of that. Is this making sense? (thanks, Alistair)
-by Alistair on 5/26/2011 at 1:02 PM
very interesting perspective.
The pogot about reflective improvement is in line with my experience and my they that software (and other knowledge work industries) requires a fundamental shift in thinking about work dynamics.
As said above, The industrial paradigm focused on process and external management of the process and the workers. Now we need the workers to manage themselves (for quick adaptation) and especially the process, so as to fine tune it constantly.
The reflective paradigm also highlights the importance of building an environment that is resilient to failure (rather than robust).
I think you are on to something important and would encourage you to read Esko Kilpi’s blogs. He is approaching the same conclusion, but for managers not IT projects.
-by Vasco Duarte on 5/26/2011 at 2:17 PM
- _Hi, Vasco! Excellent lead, thank you! I just read http://eskokilpi.blogging.fi/2009/12/04/interactive-value-creation and agree – he and I are singing the same song with almost the same words. And so is Luke Hohmann, who studied under the socioligist Karl Weick. It is sad to me that the agile community has not yet embraced sociology results as a way to jump rather than creep forward. thanks again, Alistair
-by Alistair on 5/26/2011 at 3:15 PM
I like the term “meta method”. Each of the approaches described are consistent methods for designing and maintaining context-specific methods.
-by Russell Healy on 5/26/2011 at 4:45 PM
I think it is a really good concept. What I think is missing is the ‘belief’. It is hard to reflectively improve unless there is a common consensus about which beliefs are good versus bad.
To put this into context –
Kanban example beliefs:
1) Limiting WIP results in faster delivery
2) Pulling work results in a sustainable pace and a standardised throughput of work to enable more reliable metric analysis.
These beliefs are based upon a value proposition of a principle where the principles in the above belief examples are ‘Limit WIP’ and ‘Pull’.
The belief is proven by scientific methods or thrown out there as an un-yet proven idea.
It is through these beliefs that we self reflect. We are always trying to associate the way in which we work against these beliefs.
What is interesting is the guise that these beliefs go under – Some relate to practises, princples and techniques, some to the organisational culture and a number are inherant with our self Core Beliefs (using the Neuropower term here).
TLDR: Reflection Improvement is done through beliefs. Some of these frameworks have defined beliefs.
-by Renee Troughton on 5/28/2011 at 3:00 AM
- Hi, Renee – I like your framing this as beliefs, for that’s indeed what they are. I hope I recall during the next Advanced Agile class to to call them that and see if we can list some of them out, and then discuss them. Thanks, Alistair
-by Alistair on 5/28/2011 at 11:55 AM
In former days people came up with new ideas of improving processes/ methodologies. Since you can hardly sell a single idea they wrapped a p/m around it and sold it as THE new approach.
These days it’s very much the opposite (like, we had a single great idea and it saved the entire project). What isn’t better, just different.
So it’s certainly a good thing to remind everybody that the p/m are still around.
-by |= on 5/28/2011 at 2:43 PM
Alistair,
Thanks for the link to my mind map !
We, members of an adhoc project, enjoyed translating this interesting blog entry into Japanese.
Here in Japan, we have natural understanding of Reflective Improvement(Kaizen), much better than that of Agile movement itself.
-Kenji
-by Kenji Hiranabe on 5/29/2011 at 8:34 AM
Nice! By the way, do you have any on line example of the kind of project-specific process description that do did in Agile Software Development? (Heading was Describing Methodologies More Simply IIRC). If so, it would make a nice supplement to the second section, above, since it would give a concrete example of the kind of methodologies that we still need.
PS really liked your “Crystal is both more and less than Scrum” description too.
-by John Rusk on 5/30/2011 at 1:56 AM
Done. See Describing methodologies more simply (discussion: Re: Describing methodologies more simply) and Example of a more simply written methodology (discussion: Re: Example of a more simply written methodology)
-by Alistair on 5/30/2011 at 11:27 AM
While reading your post I was struck by how similar your “Reflective Improvement Frameworks” are to Jack Mezirow’s Transformative Learning Theory. The idea that we learn through the challange of long held prinicipals leading to reflection, is very much a part of how we are improving our craft. If there is a process or method it may just center around the steps we take to analyze, challange, reflect and offer new truths or discipline.
Mezirow saw this cycle as non-ending. With each new challange we adapt to a dynamic ever changing world. The keys are the ability to examine ones own frame of reference, becoming open to alternative view points and a process of reflection when encountering elements that don’t fit well with current processes. From this we explore new potential solutions analyzing each and ultimately selecting the most appropriate.
-by S.Withrow on 11/3/2011 at 9:11 PM
I was not familiar with that – thanks for the lead, Alistair.
Alistair,
Reflective Improvement Framework (RIF) is a start. It certainly has a catchy acronym. However, I don’t like it. Framework implies “incomplete” and to be completed (perhaps in context) by a 3rd party – a distribution channel partner or end user.
I do not consider the Kanban Method incomplete. I consider it contains everything it needs in order to do its job – both at an abstract and a concrete implementation level. Concrete practices such as operations review can be replaced by an equivalent practice but without such a “system of systems” level feedback loop the method will not work as fully intended.
So I prefer Reflective Improvement Method (or Capability)
My observation is that Kanban deliberately contains a known catalyst of reflective improvement – the pull system and its WIP limits. While I may have discovered this accidentally it is now included as a deliberate design element. Reflective improvement is not “forced” as in the A3 method but happens naturally through the inclusion of the visualization (kanban board) and kanban system. Perhaps this puts Kanban in a subset of Reflective Improvement Methods where reflection and improvement are naturally catalyzed rather than stimulated by a element non-native to the performing of the work-in-hand?
Regards,
David
-by David J Anderson on 9/12/2013 at 11:15 AM
In response to Renne Troughton…
The idea that Kanban is based on a beliefe system is utter nonsense. Such a statement is in denial of the large body of empirical evidence and some considerable body of knowledge showing mathematical (or other scientific model based) proof of many of the elements in the method.
To say that Kanban has an underlying philosophy would be accurate. To say it has a value system would be accurate. To say there are a set of principles that guided the creation of it and its use in the field would be accurate. To suggest it is based on a belief system is nonsense!
-by David J Anderson on 9/12/2013 at 11:18 AM
Twitter today, David Anderson tweeted:
”@totheralistair I referenced your “end of methodology” blog in my #sepgna key note today but argue we no longer need (or want) methodologies eg #leanstartup & #kanban coupled to known good practices w/ fast feeback loops – echo Jeet Kune Do for software eng instead we want role & responsibility free defns of tech practices that include fast feedback loops inside evo frameworks”
This is cool. I find I am going back and revisiting why organizations have methodologies, and what might work or not work with David’s idea. It certainly works for many contexts and would be super it stretched to apply more widely.
I suspect social practices and social norms are also needed. I look forward to seeing how it evolves.
What’s also cool about this idea is that not only is it disruptive, but it is to organizational methods what XP was to programming methods: too simple to work, but working ridiculously well where it does.
-by Alistair on 10/2/2013 at 2:50 AM