Agile at Boeing in 1990s – the 777 Program

777The year 1995 recorded two seemingly very unrelated events: the entry of the first Boeing 777 airplane into commercial service and the introduction of Scrum to the world. As the twists and turns through history go, Boeing was Agile before its time.

I love the Boeing 777. I have flown more than a few miles over the years, and for the majority of them it has been the 777 that carried me and millions of other passengers safely across oceans and continents. For most of us the flying experience is judged by the quality of the food and in-flight entertainment options, and whether the flight is on-time or not. We don’t pay much attention to what airplane we’re in and just want to get to where we need to go, but somehow I have grown fond of the 777. It’s like an old reliable friend now, so when I see the 777 on my itinerary I don’t mind the flying part so much.

But this blog isn’t about someone’s creepy love story with an airplane – although that could be interesting enough. It’s about Product Development.

How do you develop an airplane like the 777? It turns out the story of the 777 development program is even more interesting than the plane itself. The 777 program included elements that were both Iterative and even Agile. I enjoyed learning about the program from various sources and found it hard to reduce it all into a single blog post. There is a lot of information out there if you are interested, including a really good 5-part PBS documentary “21st Century Jet – the Building of the 777”. See the bottom of this blog post for some good places to start.

The 777 Development Program

In the late 80’s Boeing was trying to decide on how to fill the product line gap between the 767 and 747. One option was to evolve the 767 design but in the end it was determined that a completely new aircraft was needed. It’s a big investment – an estimated $5BN for the 777. Even though Boeing was in a strong financial position in 1990, a $5BN expenditure would sink any company if it wasn’t successful.

It’s tempting to avoid any kind of risk when you’re in that position but Boeing broke many barriers in this next-generation aircraft design, both in engineering and manufacturing. The 777 was Boeing’s first “fly-by-wire” (computer-assisted control) aircraft, had the first fully computerized cabin, and was also the first airplane ever designed using 3D CAD systems. Seems impossible to understand how it could be done any other way today, but before the 777 airplanes were designed with 2D drawings and it wouldn’t be until the first prototype was built that form and fit could be tested.

The old tried and true approach had its share of familiar problems: in Boeing’s previous development effort, on the 767, an estimated 13,000 individual design changes (large, small and tiny) were made to the door assemblies at various stages in the development process. The 777 would be even worse if something didn’t change.

10,000 engineers were involved in the 777 program spread across the world in the US, Europe, Japan and Australia. Talk about coordination nightmares and opportunity for error. Large-scale project, indeed.

The 777 program was incredibly successful, as evidenced by the timelines and the results. Conceptual design started in January 1990. Manufacturing of the first prototype started in early 1993. The 777 had its first flight on June 12, 1994 and less than a year later (May 1995) the first plane went into service with United Airlines. Boeing estimates that the number of changes and errors was reduced by 80% compared to previous design projects.

Another first for Boeing: their very first airplane delivery was accepted by United Airlines on the first walkthrough, with only a handful of smaller defects to be noted. That’s beyond remarkable for any complex system, and difficult to comprehend given the safety concerns and reliability requirements that an airplane has to satisfy. A defect-free delivery of the very first airplane on-time as promised speaks volumes about any complex development program. I don’t understand airplanes but I know complex systems don’t come together very easily.

If that wasn’t enough… the very first 777 prototype plane built (W001) was good enough to be updated and eventually put in service by Cathay Pacific in 2000. Not bad for a prototype!

Alan Mulally

mulally

The engineering story of the 777 is also the story of Alan Mulally. The management team was initially led by Phil Condit as the executive in charge of the 777 program and Alan Mulally as the director of engineering. Alan Mulally would later replace Phil Condit as the person overall in charge, and it is Mulally’s management style that formed the framework of the 777 program and the “Working Together” model.

A hint of Agile

The first hint of Lean/Agile mindset can be seen in one of Mulally’s weekly program review meetings. A partial glimpse of what looks like program/meeting rules is barely visible, but you can make out at least some of the principles:

  • Plan for zero overtime [Agile: sustainable pace]
  • Weekly DBT reviews [Agile: weekly scrums]
  • Panic Early [Agile: fail fast]
  • Quality and/is Schedule [Agile: Quality is a critical ingredient]
  • Make decisions faster [Agile: act and learn fast]

meeting-rules

Working Together

The labels “Agile” and “Scrum” were not yet born when Boeing kicked off development of the 777 in January of 1990. However the management team knew that something had to change. Boeing’s environment had become bureaucratic and department-focused. Specialists in various departments would design their own parts and then it was up to the manufacturing team (the system integrators) to figure out how to make it all come together. It was a “throw-it-over-the-wall” environment where communication and disconnect was a constant problem.

This time, Boeing would work closely with their customers to design the airplane, and would also tear down the walls between departments by organizing their own workforce across discipline boundaries. People that were normally separated by organizations or development phases would now be engaged together and at the same time, talking and collaborating in real-time.

“Working Together” essentially boils down to “cross-functional” teams but it meant more than that. It meant working closely -really closely- with the customer airlines and suppliers. It signaled a radical departure from the bureaucratic project organization of the past and set completely new expectations going forward. Today we would call such a thing an “Agile Transformation”.

“Working Together” foreshadowed Agile, and the model is -unfortunately- still lightyears ahead of the majority of product development teams on the planet.

Although the 777 program was planned and executed in an overall phase-gate process with key milestones, integration points and deadlines, we can see many elements and attitudes which are definitely Lean/Agile-inspired, if not outright Agile. Looking at the 777 program through the lens of the Agile Manifesto we can recognize many familiar concepts.

Individuals and Interactions over Processes and Tools

The Working Together model pulled people from many disciplines together in what we today call cross-functional teams. This was a radical departure from the way things used to work, where design engineers would design their piece in isolation, then throw their designs over the wall to the manufacturing team – waterfall-style. This was the mode of operation in Boeing in 1990, and chances are  this is also the mode of operation in your company today if you work on a project of any significant size.

To address the problem of communication, Condit and Mulally organized the program in cross-functional Design-Build Teams (DBT). There were almost 250 DBT’s on the program. DBTs were formed around functional areas of the airplane, for example there was one DBT for the engine, one for the cargo door, one for the passenger door, one for the leading wing edge, one for the trailing wing edge, one for the flaps, one for the rudder etc.

Each DBT was staffed with the right people that could carry a particular design from concept through manufacturing and maintenance. Teams had design, manufacturing, tooling, finance, materials, maintenance, subcontractor representatives and even customer representatives to make sure that every aspect of operation, manufacturing and maintenance was covered.

This gives us a hint as to how we can organize Agile at large scale. You can’t possibly coordinate or co-locate 10,000 engineers, but you can make sure that the individual DBT’s are co-located and then coordinate the DBTs. DBTs were organized in a cascading hierarchy according to how functions of the airplane could be decomposed. For example, there were ten DBTs responsible for the wing’s Trailing edge, including a DBT each for Inboard Flap, Outboard Flap, Aileron, Spoilers and so on.

One potential problem of decomposing the project into DBTs is that individual teams lose track of the whole design. Weekly DBT reviews helped counteract that, but Boeing realized that the teams always need to be bonded together by a higher-level goal. When you are invested and care about something, you look out and make sure your work, and the work of the person next to you, is at the highest level.

To achieve that alignment Mulally did something incredible which I still have a hard time imagining. He pulled together all 10,000 engineers working on the 777 program for all-hands meetings once a quarter. By mid-1993 the 9th (!) all-team meeting convened in Seattle. That’s 9 of those in 30 months…

All-teamWe can only speculate how much it cost Boeing to bring everyone on the team together once per quarter, but Boeing understood the real economics of product development: total team alignment allows you to reduce errors and disconnects and avoids prolonged delays. The cost of bringing everyone together for 2 days (10,000 x 2 days of salary plus travel) is much smaller than the cost of a one-month delay to the program (10,000 x 30 days of salary plus downstream fallout).

When making resource vs. time decisions, you always need to consider the burn-rate and cost of delay for the project. The burn-rate of a 10,000-person program is much higher than the cost of these all-team meetings.

Working Software (or planes) over Comprehensive Documentation

It’s all about creating fast feedback loops in order to discover problems early while they are still correctable. Agile is a Fail-Fast model.

Quick iterations and virtual integration

CADIt is of course not practical -yet- to build and test an airplane incrementally in Sprints, but in the early 1990s CAD software was just being introduced. The 777 was the first airplane to be designed almost entirely using 3D CAD software instead of 2D drawings. This allowed Boeing to simulate form and fit quickly instead of building the traditional physical mock-ups which took both time and resources. Now for the first time 3D CAD models could be fit together and verified almost in real-time, before the first prototype was built.

Prototyping

The 777 program built 9 working airplane prototypes, compared to the usual 6 for a traditional airplane development program. Considering that the list price of a 777 would be in the 100M range, the extra 3 prototypes must have put a big dent into the development budget. However if you value fast feedback loops, then the cost of not building the extra 3 airplane prototypes would be even greater.

I don’t know for sure of course, but assuming that 9 prototypes each cost at least $100M, the  prototype expenditure for the 777 program was in excess of 20% of the total budget.

Flight-deckNot all subsystems needed a full-scale prototype. To test the maneuverability and visibility of the new flight-deck layout, a flight-deck prototype was built and mounted it on a wheeled frame which allowed them to taxi around the airport to get a feel for the handling, controls and visibility of the new flight-deck design.

Mock-ups and test-beds

If you can test early, then do so – especially for the risky new components.

A prototype of Pratt & Whitney’s new engine was fitted to an old 747 and given a test flight. It’s a costly experiment which was debated internally, but in this case it was justified: a surge (engine back-fire) was experienced on that first flight, and was discovered early enough that Pratt & Whitney could address the problem without delay to the 777 program.

The new Fly-by-Wire system was similarly tested on a 757 airplane first to make sure that everything would work smoothly on the 777.

These added prototypes and test beds resulted in positive improvements for the customers too. The accelerated testing schedule made possible by the additional prototypes made a difference in getting the necessary FAA certifications in record-time, allowing for a much faster customer deployment of the plane.

Customer Collaboration over Contract Negotiation

Boeing needed a lead customer for the 777 and after fierce competition, United Airlines selected Boeing and the 777 for a $22BN deal that effectively launched the 777 program from concept into reality.

United Airlines and Boeing did write a formal contract, but the essential agreement upon which United Airlines would award Boeing their business was the famous “Condit-Guyette” memo. This hand-written note, drawn up in the early morning hours by United Airlines executive James Guyette punctuated several days of competitive negotiations between Airbus, McDonnell-Douglas and Boeing.

Signed by both Boeing and United Airlines executives, it stated that Boeing and United would work together to design a new service-ready airplane.

Thanks due to the PBS documentary flashing the original memo on the screen for a few seconds we can reverse-engineer what it actually said:


The Condit-Guyette memo, transcribed

B777 Objectives

United + Boeing + Pratt-Whitney

In order to launch on-time a truly great airplane we have a responsibility to work together to design, produce and introduce an airplane that exceeds the expectations of flight crews, cabin crews and maintenance and support teams and ultimately our passengers and shippers.

From day one:

– Best dispatch reliability in the industry

– Greatest customer appeal in the industry

– User friendly and everything works

October 15, 1990

Signed by United Airlines, Pratt-Whitney and Boeing executives


Seems quite reasonable from a customer perspective yet unrealistic from a product development standpoint. What a great way to kick off a project, and it doesn’t get any clearer than that: customer collaboration over everything else.

Responding to Change over Following a Plan

Although the design of the 777 followed a master production plan, we can see a few ways where Boeing and its suppliers expected and responded to change within the boundaries of that plan.

As many as 8 airline customers had full-time representatives sitting alongside Boeing in Seattle, with British Airways peaking at 75 people integrated with Boeing on the 777 program. 1,500 design features were reviewed with the airlines, and changes were made to 300 of them as a result.

As another example, for the first time Pratt & Whitney (the jet engine manufacturer) held several open design reviews where they invited airplane mechanics from customer airlines to critique and provide feedback about the serviceability of their new engine. This helped reduce human error in maintenance, and helped build confidence with the customer airlines that they would have a reliable and serviceable engine on the 777.

There are many other stories of requirement change, such as the rudder team in Australia which had to endure changing requirements twice after rudder manufacturing had started. Those are the big and visible ones. But how would we deal with all the small day-to-day changes?

On a day-to-day basis the DBT’s would deal with new and emerging information, and had put in place fast feedback loops between manufacturing and design engineers. Change requests which normally took weeks to process were handled in a single day with the DBT approach. When you can collaborate quickly, it is also much easier to implement (and undo) changes.

Postscript

The 777 program is a technical and commercial success, and has garnered numerous innovation awards. Unfortunately Boeing did not push the “Working Together” model across the company. The amount of change, training and continuous attention experienced in the 777 program was quite high, so it was decided to leave it optional for each development program to decide. Of the next 3 development programs, only one implemented the “Working Together” model. Judging from the 787 impressions (numerous delays and early equality issues, fleet groundings), it seems a lot of the 777 lessons have been forgotten or ignored.

Alan Mulally himself later joined Ford in 2006 as President and CEO. There he continued the corporate turnaround with similar management methods. It just goes to show: principles are transferable but practices are not.

References and Resources

You can find out quite a bit about the 777 development program if you just do a bit of searching. Here are some of the links/resources which i came across:

Advertisements

AntifrAgile

“Wind extinguishes a candle and energizes fire. Likewise with randomness, uncertainty, chaos: you want to use them, not hide from them. You want to be the fire and wish for the wind.” – Nicholas Nassim Taleb

Agile and Antifragile… one word you know, and another perhaps you don’t.

Taleb BooksAntifragile. The word is so new it doesn’t appear in any dictionary… yet. It was introduced to the world by one of my all-time favorite authors, Nassim Nicholas Taleb, in his 2012 book “Antifragile – Things that gain from disorder”.

One might imagine that the list of “things that gain from disorder” could be written on a single post-it note with room to spare. Short in length and narrow in scope indeed, but the more you dig the more applications you see. There are numerous parallels to product development here, waiting to be picked.

When someone offers you a way to benefit from disorder instead of being harmed by it, wouldn’t you be curious? Sign me up! Should be enough material for a few blog posts. Not sure how many – all I know is that this is part 1.

Disorder Defined

Things that gain from disorder… intriguing. Better clarify what is meant by disorder then. Disorder can be many things, so Taleb provides a rough summary of the “extended disorder family”:

(i) uncertainty, (ii) variability, (iii) imperfect, incomplete knowledge, (iv) chance, (v) chaos, (vi) volatility, (vii) disorder, (viii) entropy, (ix) time, (x) the unknown, (xi) randomness, (xii) turmoil, (xiii) stressor, (xiv) error, (xv) dispersion of outcomes, (xvi) unknowledge.

As a shorthand, I’ll refer to the family as either disorder or randomness, depending on context or the whim of the moment.

To a traditional Project Manager, these family members are all harbingers of bad news, like an unwelcome visit from the Mafia. To the Developer, they are facts of life. The “project management” activity aims to wrestle the disorder family to the ground and squeeze out as much predictability, control and stability as possible.

That, as Taleb likes to say, is a “sucker game”.

It’s a sucker game because the problem is unsolvable. We’ll never eliminate or control randomness. If we could, randomness wouldn’t be coming at us at inconvenient times from unexpected directions. The winning game is not to tame randomness (we can’t), but to manage our exposure to the effects of disorder by assuming change and volatility is ever-present. Here you can already see the first signs of parallels to Agile: we embrace change and new emerging information. We adapt to a changing environment. Variability and uncertainty are not to be resisted, but used.

We need a way, then, to help us get a handle on this disorder thing. You can’t exploit what you can’t see. And so we introduce the Triad.

Fragile – Robust – Antifragile

Everything in the world can be classified into The Triad: Fragile, Robust or Antifragile according to how they respond to disorder.

Something that breaks when subjected to disorder is Fragile. For example, a porcelain teacup which is dropped onto the tile floor has nothing to gain and probably will break.

The Robust neither breaks nor gains from disorder. Robust or resilient things retain their shape after being exposed to shocks. That is good, as they don’t get any worse – but they don’t improve either.

We typically think of the opposite of Fragile as robust or resilient, but as Taleb is the first to articulate, the opposite of breaking under stress is not to stay the same but to get better. What kinds of things could possibly benefit from randomness and disorder? Well, a lot of things, and Taleb gave us the word Antifragile to give the concept a name. Things that are antifragile respond positively to disorder and randomness. They grow and get stronger.

OK, we are engineers with logical minds trained in linear thought. This can be hard to wrap your mind around, so let’s look at some examples.

The easiest approach is to look at the opposite end of the spectrum first. Fragility is easily detected. Anything that looks like a line of dominoes that have to fall in place perfectly is fragile because any kind of variation or instability (one domino out of order is enough) will upset the plan.

For example, if you are traveling by air and your itinerary depends on making 3 different connections with no allowance for delay, then your plan is fragile to delays. The more such dependencies you have, the more fragile your trip will be.

Think about all the fun project Gantt-charts with carefully laid-out dependencies you have been exposed to in your lifetime. None of them (none!) executed according to plan. The bigger your project is, the more moving parts you have. The more moving parts, the more fragile you become because you have more opportunity for things to wrong. Size makes you fragile (this will be a separate blog post for sure). You can hide fragility behind buffers, but it doesn’t reduce the fragility of your reality. It just lets you pretend it doesn’t exist and delivers a bigger surprise when things go really wrong. This we mistake for confidence.

Robustness in R&D typically come in the form of buffers or backup-plans. Both are expensive because in both cases we trade stability for schedule time or resources. Since most R&D projects are people-intensive (easily 80% of project costs come from salaries), this is the most expensive way to purchase perceived predictability.

Mother Nature is Antifragile. Here is  system that depends on random events. Evolution can not do its work in a perfectly stable system where there is no variation. If there is no genetic variation, then there is also no way to improve the species. Genetic variation (mutation – say sharper teeth for a carnivore) is necessary for the improvement of species. Evolution is fueled by randomness. It’s a volume business, for sure, and most variations don’t result in meaningful improvements. But once in a while a new ability surfaces which ends up dominating. Here we see at least two members of the disorder family at play: variability and time.

AntifrAgile

There is a lot more in common between Agile and Antifragile than contracting the two words into AntifrAgile – although the word combination is cute and rather serendipitous. And maybe prophetic.

Yes, Agile methods embrace change as a fact of life but the dive into AntifrAgile goes much deeper than that. It boils down to a different way of looking at risk – and therefore also acting differently on a daily basis.  We stop trying to predict or eliminate random events, and instead put ourselves in a position where surprise is not an unwelcome guest but rather the wind that energizes the fire.

As we shall see later, like Mother Nature we can exploit disorder in product development rather than being harmed by it. Taleb outlines several strategies for how to do that in other parts of life, and I’ll try to relate them to the Lean/Agile mindset. As we draw the parallels, you might get a better appreciation into how randomness is also Agile’s friend. And – perhaps in the process we’ll tread into uncharted Agile waters. Who knows.

Summary

Ok that is almost enough for this first post on Agile and Antifragile. Maybe you can start to see the trajectory of how Agile and Antifragile intersect. They are overlapping in so many places, sharing much of the same DNA.

So what’s the big deal? Understanding how Antifragility can work to your advantage will unlock more options and maybe even push our application of Agile methods forward in new directions. I am a big believer in adopting and adapting ideas from other areas, and if we can overcome our domain dependence then we have one more source of inspiration to draw on which will help us move away from the Fragile world of Gantt-charts and into the more dynamic world of Agile.

And hey, it doesn’t hurt to get independent confirmation from other parts of life about the sensibility of Agile, and to see that Agile’s way of dealing with uncertainty has merit outside of Software Development.

A Tale of Two Systems

Ok, a cheesy headline. But it fits.

I recently relocated to Berlin and part of the fun is to get set up with new… everything. A few trips to Ikea, hardware stores, grocery stores, the local Bürgeramt (government bureaucracy is alive and well) etc… and then the real fun: communications. We need new mobile phones and internet access.

I like the convenience and simplicity of the one-stop-shop model so we visited the local telecom provider in our area. No problem, they can help with all of that. It didn’t take long to find ourselves citizens of two worlds.

First of all, I don’t need or want a land-line. It’s 2014 and we just use mobile phones and Skype. Neither did we really need or want TV since we watch online on-demand on our own schedule. But… you can’t get Internet without TV or TV without a land phone line anywhere in the world these days. So now we occupy space in the public phone system for a number that will never ring, and we have lots of German TV channels we will never watch. But it’s nice to know they’re there, I suppose.

But that’s a digression. Step one is to get mobile phones, and that was easy. Pick a plan, pick a phone and less than 30 minutes later (most of which was spent filling in the application forms) we had mobile phones in hand, activated and ready to go. Great!

Not so easy for Cable TV and Internet. The store clerk patiently explained that we will receive a set-top media receiver via mail courier in two weeks. Then, two days after that, the actual service would be activated and away we go. The DSL modem for internet connection is yet another piece of equipment which, incidentally, the TV set-top box depends on. We picked up the DSL modem in-store.

One company, two very different levels of service. Mentally I form a picture of a company that has one logo but two distinct business units and logistical setups, probably as a result of a merger along the way.

The mobile unit is new, formed around internet-time service expectations and delivery capability. Same thing for internet connectivity. It’s new, not saddled with legacy procedures and expectations.

The cable TV unit is stuck with old infrastructure, not just for physical connection but also in terms of logistics. I imagine someone has to take the order, re-type it into another system, print it on a dot-matrix line printer, send it to Nuremberg or wherever where someone puts their stamp on it, makes two copies which get inserted into thick dusty binders never again to be seen, sends another paper work order via internal mail to the service activation center where the operator finally flicks the electronic switch.

Well maybe not exactly like that, but perhaps not too far off. It’s 2014 and there is a 2-week waiting period to get a simple service turned up, where it appears that most of the time is spent simply waiting in line.

One company, two service models. I guess the merger wasn’t as thorough as the company logo might suggest.

Predictably, as anyone familiar with complex systems will know, the longer lead-time model with more links in the logistics chain is more prone to problems and surprises.

Somewhere along the way our set-top box got lost in the shuffle, and nobody was able to find it or tell us when it would arrive. “Maybe tomorrow” we heard a few times, the clerk’s voice unable to really convey the confidence we were hoping for. Can’t order a new one because the previous one is already “in the system and on its way”. The fun part here is that as we were tracing the steps, we discover that the service provider actually sent the set-top box to the courier service the same day we ordered the service. So they were in effect told to hold the package in the delivery warehouse for 2 weeks before even shipping. It was lost for almost a week before it arrived. We finally had internet working 3 weeks after the order was placed.

I check the calendar. Yes, it’s 2014. But at least we’re in Berlin and living life in a way that’s not possible anywhere else in the world. Some things are inconvenient, yes, but easily purged from active memory by blogging about it. Kind of feels like we passed the first test. Now we move on and become Berliners – whatever that is.

Domain Dependence

Occasionally I come across a golden nugget that puts things in perspective. Usually it’s a simple and beautiful idea which elicits the “duh yes of course” gut response when it’s first explained but then continues to reverberate and come back into focus again and again. Ideas such as this one articulated by Nicholas Nassim Taleb.

“Some people can understand an idea in one domain, say, medicine, and fail to recognize it in another, say, socioeconomic life. Or they get it in the classroom, but not in the more complicated texture of the street. Humans somehow fail to recognize situations outside the contexts in which they usually learn about them.” – N. N. Taleb

He refers to this as the domain dependence of our mind. He puts forth a few examples, including that of the business man who pays the hotel porter to carry his luggage upstairs, then heads for the gym to lift weights – mimicking the natural action of lifting a suitcase. Simply put – we put blinders on and fail to recognize familiar patterns in unfamiliar situations.

It explains a lot of the frustration that comes about in process improvements and agile transformations. At least for those of us that like to think and speak in analogies, Taleb’s words ring true. We are simply too accustomed to thinking about things in our familiar ways to see the simple truth staring at us from the other side of the mirror. It gets worse as our expertise deepens. Expert knowledge is deep, and therefore becomes more and more domain-specific. The groove wears deeper and obscures our view of the world. It gets harder and harder to look outside our own domain for solutions to the problems we are working on.

But for innovation that’s exactly where we need to look – outside of our own domain.

We experience the pattern clearly when we discuss Lean/Agile concepts with someone that has a firm waterfall view of the world. Most people can understand and agree with the concepts, but many also have clear convictions that those same ideas simply don’t fit in their area.

There are a couple of mental shifts that have to be made in order to recognize foreign ideas and implement them locally. First you have to take a particular situation and understand the general idea and principle behind it. That takes some abstract thinking abilities which some people are better at than others. Nonetheless you need to extract the general principle at play. Then, that same general idea needs to be applied in a different way in the new domain. The application may look different but the principle behind it is the same.

For example the idea that an apple will fall from the tree to the earth is well-known and accepted in the domain of everyday things; The idea that the earth actually pulls the apple towards the planet’s center until it tears loose from the tree is harder to imagine.

At a different level, we have almost no trouble learning about the gravitational pull of planets. Once we understand the principle of gravity explained using planets as the domain we can start to see the commonality. The only difference between the moon and the apple here is scale – it’s just that you can’t see the similarity unless you shift your viewpoint significantly. Once we look at them both from outer space, it’s a bit easier to visualize the apple as a really tiny moon. Now the apple doesn’t fall down any more. It gets pulled inwards towards earth’s center.

Or for a more obvious example – since you are reading this blog – consider the difficulty of seeing the translation of Lean Manufacturing methods to Product Development. The Software crowd are starting to get it, but the Hardware/FPGA folks? “That won’t work in our special domain!” (BTW to me that’s not a problem at all – it’s an opportunity to be the first HW/FPGA developers to outpace the competition). Check out www.agilesoc.com and @nosnhojn as an example of pioneering work here.

Domain dependence, then, is one hurdle that we repeatedly have to cross. That is especially true for Lean/Agile transformation because the Lean principles (which are fundamental to Agile methodology) are so counter-intuitive to “common” knowledge. Lean/Agile principles are really sensitive to domain-dependence because of that.

But there is a bit of good news: if you can shift the viewpoint and get to the principle behind the practice, then this can be your detour around domain dependence. Talking about general principles without being weighed down by “common knowledge” will open the door. Once you have developed an understanding of how general principles work (like gravity) then it is a bit easier to move forward since rejecting a fundamental truth just because it looks different is, in a word, irrational. Of course calling someone irrational would not be a rational thing to do in most situations, so find another way to make the point. Patience helps; most of the time you just have to wait for the gears to turn and the point will make itself.

So, domain dependence is a real obstacle and it’s usually not recognized as a factor when we gauge resistance to new ideas. Remembering that simply shifting the viewpoint and focusing on the general principles at play is one way to reduce the impact of domain-dependence. Plotting that course is a tough job which falls to you – the change agent.

Curiosity didn’t kill the cat…

..perhaps something else did. I don’t know – I’m a dog person.

I am starting to see a common thread, kind of a tapestry of supporting ingredients that either enables or blocks change. It’s not just with Lean/Agile, but any change effort. Many times the implementation of new ideas don’t get off the ground because they never had chance to take root in the first place. Why?

(can you mix conflicting aviation and farming metaphors? I think I just did)

I think there are at least three fundamental ingredients we overlook when we prepare for change: Imagination, Curiosity and Introspection. These are not Values, but Traits. Values can be taught and even enforced; Traits are cultivated.

(yes, the farming metaphor works better)

Imagination

If we want to effect change, we first have to be able to imagine a world where (1) what we currently do is not good enough any more, and (2) there is a better way.

If you can’t leap that far, then you really don’t have anywhere to go to. It’s that simple. Many folks I have met and worked with over the years have done well and succeeded in their careers doing things a certain way so why change now? It’s hard to imagine that what has been successful in the past isn’t the best way to do business anymore (I’m working the best way I know!).  But change is constant -whether you accept that premise or not- and it is a harsh reality that what made us successful yesterday may be a liability tomorrow.

We have to imagine the new behaviors we want. What do small batches look like in your environment? What would it mean to rely on self-directed work teams? What implications are there from facing risk head-on (as opposed to the traditional avoidance maneuvers)? How would Planning change with a Lean/Agile mindset?

Purpose and motivation come from the question “Why?”. Tools, practices and processes answer the question “How?”. Imagination springs from the “What if?” domain.

Lean ideas are so far away from current popular wisdom, even counter-intuitive, so making the paradigm shift requires a good dose of imagination. In some people it’s innate, in others it needs to be cultivated.

Imagination plays a huge role in overcoming one of our biggest challenges: our difficulty in translating good ideas from one domain to the next. As Nicholas Nassim Taleb remarks, “Humans somehow fail to recognize situations outside the contexts in which they usually learn about them.” Taleb labels this phenomenon domain-dependence.

That is, we easily recognize good ideas in one domain (say Manufacturing) but fail to apply the same ideas in another domain (say Software development). Even years after Mary and Tom Poppendieck provided compelling insights on how to map the principles behind Lean Manufacturing to Software Development we have trouble accepting that the same laws of nature which influence manufacturing apply in knowledge work.

To see something in one place and imagine how it could apply somewhere else is the essence of innovation. You can’t do it without imagination and unfortunately we don’t pay much attention to that particular trait. Chances are you didn’t ask your most recent hire anything to gauge their imagination. You probably focused on skills and experience, right? If you have Imagination and Curiosity -the subject of the next section- you will acquire the right skills and experience. Imagination and Curiosity are the more important success factors.

More on domain-dependence in a later blog post perhaps – this is one of our biggest problems w.r.t. process innovation and progress.

Curiosity

Very well, we want to improve and achieve better results but without a certain level of curiosity your options are limited to variations on familiar themes. You can streamline this or optimize that without stepping too far way from the playbook or without asking too many fundamental questions. Curiosity doesn’t do well within your comfort zone – it needs to break out. New solutions develop when we are curious about how well we actually perform today, and curious about how things work in other places. Curiosity about how things work. Curiosity about why we do things the way we do. Curiosity about why (the ultimate “5-Why”-type-of-why) we are feeling the pain we do in our projects.

The best people I have worked with have all been very curious. Never satisfied that they know everything, but always having this internal unsettling feeling that there is something else that needs to be discovered or questioned, always wondering what’s there at the next level or behind that next door. For Curious People, the questions are more important and exciting than the answers, and every answer triggers ten new questions.

If you are curious, you don’t need management to tell you to use the 5-Why method. You already ask more questions than can be answered. Keep going, and use your Imagination to come up with new ways of working based on the answers you find.

Introspection

This is perhaps the most difficult one. Introspection is the precursor to Reflection. Reflection fuels Humility. Humility is a necessary condition for un-learning, especially as we acquire more experience, success and seniority.

Yes, un-learning. Lean/Agile is a paradigm shift. To make a paradigm shift we usually have to un-learn something we know to be obvious and true to make room for the new paradigm. For example, we have to un-learn the “obvious” application of mass-production scale economy (and thereby the division of labor) to knowledge work and the usefulness of Pert-charts.

Once you are armed with curiosity and imagination, the easy part is to look at the system as a whole and identify where the waste is in the value stream etc. It moves the fault-assignment away from the individual and onto the system, which I totally agree with. But it could also (incorrectly) be a way to move responsibility away from oneself and onto the system.

It can be quite difficult to look inward for an honest assessment. But inwards we must look, because if you are leading change, then your own convictions and sense for what is the right thing to do must also evolve along the way. That is after all the Agile way. Inspecting and adapting at the leadership level requires introspection. If you don’t live along those principles, your team won’t either.

Introspection then enables us to stay open enough to consider new knowledge, to be flexible enough to adapt to change, and to be humble enough to continuously re-orient your own world view. Remember, finding the “one way” to “do Agile” (ouch! you don’t “do Agile”) and then pushing it through is just another way of commanding & controlling.

Bottom Line

So… the trinity of Imagination, Curiosity and Introspection. These values can’t be commanded from the ivory tower, but they can be cultivated at all levels. It’s an indirect approach. You reap what you sow… sometime later. I am fortunate to have been surrounded by people through my career which display these traits. As I realize now the importance of them, it only underscores how subtle these influences are. Or maybe it shows that I am a slow learner. In any case, hopefully this post triggered some thoughts on your end.

It’s a long-term and indirect investment with no clearly visible cause-effect relationship, but if you cultivate these traits then you are moving int he right direction – no matter where you end up.

Why not SAFe?

So let me get this out right away: I’m a SAFe proponent. But hang on – it doesn’t make me a “pure-Agile” non-supporter.

I have my PSM certification from scrum.org and my SPC certification from Scaled Agile Academy, so I feel comfortable enough to comment on the discussion. The discussion is not as intense as it was a few weeks ago, but I sense that the residual effect is for some to bypass SAFe as even an option to consider.

It’s too bad that what would otherwise be a healthy discussion surrounding SAFe and “pure Agile” is causing division. Peer review and feedback is the best way to ruthlessly drive things forward and evolve, but if we allow discussion to degrade into both (or one) sides digging their heels in and simply trying to convince the rest of the world that their approach is best instead of understanding each other and move forward, the value of discourse quickly dissipates.

On this front, Dean and SAFe scores the point: every time I heard Dean speak about SAFe, he has been very clear that there is a lot to be gained from being pragmatic and tolerant of other’s views. Rather than taking a polarizing “my-way-or-the-highway” stance, the creators of SAFe already has an advantage here that will work in their favor in the long run.

So what is it about SAFe that makes it worth the investment, that other Agile approaches haven’t covered well enough yet? In my view…

SAFe addresses the people not on the Agile team.

This insight by itself should be enough to pique your interest. Most Lean/Agile material I’ve seen focuses on how to implement Agile methodologies, but very little attention is paid to the stakeholders outside the team. Yes I know, there are exceptions out there, but generally I feel that we don’t pay enough attention to the folks outside the immediate development team.

SAFe addresses these stakeholders as first-class citizens, not as impediments to progress that have to be “turned around”. That’s a smart thing. Those “impediments” often sign your paycheck, and if they are not comfortable with what’s happening, your Agile Adventure will be short-lived or at least very limited in reach. Let’s invite them to contribute in a format they can relate to and be effective in.

SAFe has the right ingredients and building blocks

  • Lean principles as the leadership backdrop
  • Scrum as the proven team-level project management framework
  • XP-inspired coding practices
  • Don Reinertsen’s Principles of Product Development Flow

You may have a different set of favorites, but when I saw the above 4 ingredients in the same recipe, I definitely paid attention. Ever since I read and re-read Don’s book I’ve been on the lookout for applications of this really wonderful and clarifying set of ideas.

SAFe is for large programs.

The “S” in SAFe stands for “Scaled” – a point which I feel is lost in some of the SAFe critics. I have spent all my working life in large organizations in both software development and project/program management roles, and I can confidently say that if you want to manage anything at a scale beyond a handful of people you will need some sort of framework to keep everything aligned. Simply relying on a shared set of values and a common goal isn’t enough. Multiple teams simply won’t self-organize in a single direction. It doesn’t matter whether you are doing Agile or not – large groups of people working together is going to require alignment and guidance to enable some sort of orchestrated delivery.  As Reinertsen points out: “there is more to be achieved from overall alignment than local optimization”.

Statistically we will have some success-stories where overall alignment happens in an organic manner, but don’t count on it for your project. You N. N. Taleb fans out there know what I mean.

SAFe pulls it all together and communicates a holistic view

Again, selling Lean/Agile to folks outside the Agile teams is where you either achieve an Agile enterprise transformation, or you don’t. You don’t have to “sell” Agile to the teams themselves, they are usually the ones that get onboard first. If you want to scale Agile to the enterprise-level, you have to sell the idea to those that have the most influence.

The value of having a single-page graphic which acknowledges the various roles and the main flow of events can’t be overstated. Here, for the first time, we have a map of the world in which everyone in the existing organization can see a place for themselves in an Agile setup, post-transformation. That immediately defuses a lot of the potential friction and apprehension associated with introducing Agile to the C-level office and the middle management layer. You know, the ones who decide what you can’t and can’t do in the enterprise. Kind of important to get those folks on your side. Showing the full context of an Agile Release Train is definitely much easier than selling the idea of “Agile self-organizing” teams.

SAFe may not be for you

…and that’s ok.

If you’re starting out and introducing Agile to a small organization, you don’t need or want SAFe. If you’re working with a small set of teams, it’s probably not for you. That’s ok – SAFe starts to make sense at scale. Keep that in mind as you grow.

SAFe is not a one-size-fits-all solution. Neither would you want one. Personally I just need a model that fits my situation and environment. Most of us work in one company, and in one environment. Chances are slim that we will need a large-scale Agile solution one day and a small-team solution the next. If SAFe works for what I need to do, then that is fine. If not, then move on and find something else. But please don’t make it your mission to convince others (who you don’t even know) that some approach or other is not an option for them. SAFe is indeed an option, and so is “pure Agile” and any other variant such as DAD, but only you can decide which one is right for you.

Use SAFe (or any other model) responsibly

A lot of the SAFe criticism I’ve seen seems to assume that our teams don’t think for themselves, but rather assumes SAFe would be adopted blindly and without real-world context.

Isn’t that anti-Agile? Thinking people can understand when to apply the mechanisms offered by SAFe and when to do something else. Not all aspects of SAFe will work for you, but it would be a shame to throw out 100% of the model just because there is 10% that doesn’t work for you. Use the parts that are helpful to you, discard the rest (for now). Whatever defined approach you choose, you’ll need to tune it to your particular situation anyway. I think Dean and the SAFe crew would agree.

…which brings me to the point of this blog post

Although a polarizing discussion is entertaining and can provoke some great insights, at some point we reach diminishing returns, and instead of improving we alienate. It would be too bad if this argument causes some folks to bypass the good work done by the SAFe team. SAFe can provide a huge head-start for some companies, especially those who need to visualize a pathway out of the waterfall.

If you are rejecting SAFe on the basis of comments made by some Agile rock-stars, pause for a moment and put it all in context. What alternatives are they suggesting? In most companies, it’s not enough to simply rely on self-organization around a shared goal and scrum-of-scrums. At some point at scale you will eventually need some sort of framework and holistic planning involving the rest of the enterprise, whether it’s called SAFe or not.

If you are embracing SAFe because it looks less threatening to your existing organization and an easy fix, you should probably take a step back and ask yourself if you really understand what Lean/Agile is about. Your organization will likely have to change its ways, and it can be a difficult transformation.

Ultimately, I don’t see that SAFe competes for territory with pure Agile/Scrum but rather complements our current practices to enable larger-scale Agile projects. There is a big world out there that largely still hasn’t transitioned to Agile, and the sandbox is big enough for everyone to play. Let’s move Agile adoption forward in any way we can, even if it doesn’t fit a purist’s view of what Scrum should be.

If SAFe brings Lean/Scrum/Agile within reach for companies that would otherwise not even entertain the idea, how can that be a bad thing?

Conway’s Law meets Lean/Agile

“Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” – Melvin Conway, 1968

You’ve probably heard of Conway’s Law in the context of system architecture and design. The architectural blueprint of a system tends to mirror the layout of the group that built it. Conway noted in his paper that the concept has much broader scope than just software systems, as it can be applied to any type of “system”. It’s a sociological observation, not a technical one.

If you overlay your company’s organization chart on top of your development process diagram you can see that Conway’s Law applies in a different kind of “system”. We can state a corollary of Conway’s Law as

“Organizations which design processes are constrained to produce a process flow which is a copy of the communication structures between departments”

Image

Try it. Map out your own organization and see how well it matches the phases of your development process. You should get a close enough match between departments and process phases to convince yourself that it’s not a coincidence. Functional departments are formed around technical domain expertise (e.g. Software, Hardware, Test), and the development process phases align with those boundaries. It doesn’t matter which came first – the process or the organization. Whoever framed up the process probably didn’t set out to create this match, it’s just a natural consequence of Conway’s Law.

So if the development process and the organizational structure are so closely matched, then what happens when we introduce Lean/Agile which has a fundamentally different process flow?

You already know the answer – friction and discomfort. Departments are typically preoccupied with their own internal and local efficiency. Lean/Agile is concerned with the overall end-to-end global process efficiency. The two rarely align. Departmental sub-optimizations are in direct conflict with the Lean/Agile flow.

So… Agile means a new organization structure?

The “straightforward” remedy to fix this conflict is to re-draw the organizational lines to line up with cross-functional teams rather than departmental domain expertise. Unless you are ready to push the reset button on many things that currently aren’t broken, this is not a great option. In an entrenched organization it will be a very upsetting and difficult thing to do. If you have the power, guts and the ability to actually do this, it might be the way to go. But for the rest of us (the 99% majority) we will have to be a bit more pragmatic about it and try to preserve the things that actually work, not drive good people away.

Good people should not be threatened by Lean/Agile

Some level of change is necessary when you move to Lean/Agile, and the new system is threatening to the old system. Sure. But remember if Lean/Agile is threatening to the organizational structure, there is no reason it should be threatening to the organizational managers. These are good people who have brought the company to where it is today, and if they can transition into a Lean mindset along with the company then that’s a big win: we can retain a lot of valuable experience and knowledge which simply can’t be found in a new hire.

Most of the people I have worked with over the years have the ability to re-tool and engage with the Lean/Agile model. Those who can’t or won’t tend to have a stressful time and it is not fair to ask them to work in a way that creates discomfort to them. That would violate Lean’s pillar of “respect for the individual”. Find other meaningful positions for them if at all possible, but for the good of everybody involved: don’t put them in an authority position (or even support position) on an Agile project. Here they will do more damage than good, and stress themselves out in the process.

Department heads are excellent domain experts

I find that in an R&D organization the department head typically has a lot of knowledge and experience in their technical domain. These folks are great resources! Thinking about departments as “centers of excellence” who provide skilled staff to the cross-functional teams instead of compartmentalized workshops is perhaps a way to reconcile the need for functional organization with cross-functional Agile teams. In that model there is still room for a “software organization”, a “test organization”, a “verification organization” and so on.

Image

The last thing we want to do is to sacrifice the continuity of technical domain knowledge simply in order to form cross-functional project teams. Someone needs to move the individual disciplines forward, identify and close skills gaps, provide coaching, keep up with technological advances, ensure the right tools are in place and so on. In such a setup there is a meaningful place for the departmental manager. Department members may be assigned to cross-functional Agile teams (even for the long term) but they always have a home base and an identity based on their technical domain skill set, and they have somewhere to go for help and advice in their discipline.

It can be a win-win to preserve the organizational structure and have the departments contribute their value-add to the Lean/Agile process. Most people will do good work if you let them. It’s better to join forces with department heads in established organizations. They are great allies, and as you jointly start to deliver Lean/Agile projects the sun shines on everyone involved.

At some point the atmosphere may become more relaxed and the option of drawing organizational boundaries closer to the cross-functional team structure may become more attractive. That’s a much better position to re-organize than at the start of a Lean/Agile transformation – but it requires patience.