Deepening Our Understanding Of Good Agile: General Issues

My last article “What Good Agile Looks Like” generated many excellent questions and suggestions. for which my sincere thanks. In this new article, I offer possible answers and invite further conversation on the following five general issues.

·       Theoretical Underpinnings Of Agile

·       Key Elements Of The Agile Synthesis

·       Deciding Between Competing Principles

·       Conversion Rather Than Choice

·       What To Call The New Paradigm

A separate article will deal with the specific issues about the content of the principles, processes and practices outlined in the earlier article.

Can Theory Clarify The Meaning Of Good Agile?

One reader agreed that we need a holistic, integrated perspective on management if we really want to advance management as a discipline; but also asked: how do we solve disagreements at the level of principles? Shouldn’t principles be grounded in sound theory—theory that explains causal relationships, not just a matter of taste and opinion?

A good way to think about this question is to recognize that a new management paradigm has analogies to a revolutionary new theory in science. We can learn much from Thomas Kuhn’s famous book, The Structure Of Scientific Revolutions (1962).

Kuhn distanced himself from the idea that a new theory in science was about the discovery of objective truth. Instead, he viewed each new scientific revolution or synthesis as “less problematic” and “more fruitful” than the previous synthesis, with fewer anomalies and greater predictive power and maybe greater simplicity and clarity. For example, Copernicus’s heliocentric theory of the galaxy had no greater predictive power than the previous earth-centric theory. But it won support because it was simpler and seemed more plausible. As it turned out, Copernicus’s theory involved the idea of rotating spheres which was dead wrong, but the heliocentric part turned out to be right. The theory won broad support, despite its flaws.

It is in this sense that we should not be expecting to discover a theory of management that explains the objective truth about management or that prescribes the perfect organizational structure. We should be content if we can find a synthesis that has fewer anomalies and greater predictive power than the previous synthesis.

That is so a fortiori for management compared to physical science, because human society is constantly changing, unlike the physical universe. So there is even less likelihood of attaining even temporary truth about the human universe.

It is in this sense that top-down bureaucracy used to be a better fit with the human world of the 20th century than any previous synthesis. It led to vast material benefits the human race in the 20^th century, even though those benefits were unevenly shared.

But as the century wore on and as the human world kept changing, the anomalies in management kept growing and growing. Many of them related to the faster pace of change, the growing complexity of the marketplace, the shifts in technology, and the growing importance of the customer. Organizations that had been successful with economies of seope and scale encountered increasing difficulty in adjusting  to these changes.

Within traditional management, efforts to resolve the anomalies became ever more frantic. As in science, the adherents of the prevailing paradigm “knew” that there must be a solution within the current paradigm, if only they could find it. So they kept trying harder.

Firms tightened management control. They downsized. They reorganized. They de-layered. They empowered their staff. They re-engineered processes. They expanded sales and marketing campaigns. They acquired new companies. They shed businesses that weren’t doing well. These fixes sometimes led to short-term gains, but they didn’t solve the underlying problem.

Roger Martin’s new book, When More Is Not Better (HBRP, September 2020), gives a clear explanation of the underlying problem and the need for greater adaptivity. "The complexity aspect of a complex adaptive system,” writes Martin, ”means that the system in question is largely inscrutable, with causal relationships among elements in the system that are ambiguous and nonlinear. Even more challenging, those relationships aren’t stable. The actors in the system are continuously driving adaptation of the system. By the time we-decide what to do, it is quite possible, if not likely, that the system has changed in a way that renders our decision obsolete by the time it is acted upon. And by the time we have figured that out, the system will have changed again. Because of that adaptability, our design principle must be to balance the desire for perfection with the drive for improvement."

Eventually management rebels started emerging with different syntheses, including the Agile Manifesto of 2001. As in science, the new syntheses were at first half-baked and full of anomalies and contradictions. Eventually, by 2020, many anomalies had been resolved to such an extent that the synthesis was enjoying increasing numbers of adherents. A key factor in gaining support was that leading firms implementing the principles of the new paradigm were making unprecedented amounts of money. But the paradigm was also seen by increasing numbers of practitioners as a better and more human fit with the world of 2020 than the previous synthesis. It was fundamentally about human beings creating more value for other human beings.

As with Kuhn’s view of science, the Agile synthesis of management doesn’t claim to be “objective truth” or “the final solution” as to how to organize. It merely claims to be a better fit with the world of 2020 and a better synthesis than its predecessor, with fewer anomalies and greater predictive power and fewer ethical and social issues.

A Key Element Of The New Synthesis

A key element of the new synthesis is dealing with fact that we live in an increasingly VUCA world—volatile, uncertain, complex and ambiguous. The Internet dealt a powerful blow to the concept of the firm as the center of the commercial  universe and validated Peter Drucker’s insight of 1954 that “there is only one valid purpose of a corporation: to create a customer.” The result is a Copernican revolution in management—the customer is now the center of the commercial universe. The customer has choices, and instant reliability information about those choices and an ability to communicate with other customers. The bureaucratic model of the firm is radically out of sync with this reality.

Just as Copernicus’s theory had a vast social impact by freeing the human mind from the intellectual stranglehold that religion and the Divine Right of Kings, so the revolution in management is breaking the stranglehold of top-down bureaucracy. As Thomas Kuhn wrote in The Copernican Revolution: (1957): “To describe the innovation initiated by Copernicus as the simple interchange of the position of the earth and sun is to make a molehill out of a promontory in the development of human thought. If Copernicus' proposal had had no consequences outside astronomy, it would have been neither so long delayed nor so strenuously resisted.”

Similarly, the ongoing revolution in management is great deal more than the simple interchange of the position of the corporation and the customer. It is leading to vast changes in society as a whole. Once great corporations are barely relevant. Upstart firms are suddenly richer than governments. Everything is different.

How To Decide Between Different Paradigms?

In thinking about the basis and process for deciding between different paradigms, Kuhn’s writing is also helpful. Kuhn said that competing paradigms were "incommensurable" and that, even in science, there wasn’t really any objective basis for choosing between two competing scientific theories. There was usually no way to conduct a simple experiment to show that one theory was right and the other wrong, at which point all scientists would abruptly drop the old theory and espouse the new. Instead, there was generally evidence both for supporting and questioning the competing theories. Scientists had to weigh up different kinds of evidence and then decide to put their careers behind one theory or the other. This didn’t happen overnight.

Kuhn noted that we tend to think of revolution in political terms : the American Revolution, the French Revolution, the Russian Revolution. Everything is overthrown; a new world order begins. Kuhn notes that revolutions in intellectual matters happen much more slowly. “When an individual or group first produces a synthesis able to attract most of the next generation’s practitioners,” Kuhn writes, “the older schools gradually disappear. In part their disappearance is caused by their members ’ conversion to the new paradigm. But there are always some men who cling to one or another of the older views, and they are simply read out of the profession, which thereafter ignores their work. The new paradigm implies a new and more rigid definition of the field. Those unwilling or unable to accommodate their work to it must proceed in isolation or attach themselves to some other group.”

In Kuhn’s view, the issue presents itself  more like a conversion than a choice. He noted that scientists had never really “chosen” the original set of theoretical assumptions of the old paradigm in the first place. They had started studying at a university and then working in a laboratory where everyone had the same basic assumptions, habits and attitudes. Young  scientists had little choice but to accept those assumptions and attitudes if they wanted to go on working there. Working in that way may have gone on for years or even decades. So when someone came along and said that those assumptions, habits and attitudes had to be set aside and that there were the new assumptions and the new ways of doing things, not all scientists suddenly leapt to embrace the new theory.

For one thing, once scientists opted to pursue the new synthesis, the world started to look and feel different. The meaning of once-familiar words changed. Old beliefs and habits and attitudes had to be set aside. New practices had to be learned. This, said Kuhn, amounted to "living in a different world," which of course made skeptics even more upset and frustrated.

To make matters worse, the new theory was seldom just an increment to what is already known. Its assimilation often required the demolition and reconstruction of prior theory and the re-evaluation of prior knowledge, and decisions needed to be made about what things to pay attention to in future and what things, formerly thought important, to ignore. It is not until the re-evaluation of old knowledge has been complete that the new paradigm is fully accepted.

Conversion rather than choice

These considerations led Kuhn to posit that after a scientific revolution, the decision to go from operating in the old mode to operating in the new mode isn’t properly described as a choice. It is more like a conversion.

This in turn can lead on to the impression  that the supporters of the new paradigm must be some kind of cult. This issue can be aggravated by the strenuousness with which the supporters of the new paradigm urge the case for its acceptance and denounced the old way of looking at things.

All of these phenomena are observable in the ongoing transition from the 20^th Century management to the Agile paradigm of managing. Managing in the Agile mode is in some ways like being in a new world compared to 20th Century management. Familiar words like “manager” have very different meanings. Decades-old ways of doing things are suddenly no longer appropriate. New ways have to be learned. Attitudes and behaviors have to change. To old hands, Agile can come to be seen as a strange new cult.

What To Call The New Paradigm

What to call this new synthesis or paradigm is also an issue. It has roots that go back to Peter Drucker in 1954. It gained adherents in the world of Agile but now is something very different from what was envisaged either by Peter Drucker in 1954 or the drafters of the Agile Manifesto of 2001.

In my 2010 book, I called it “radical management.” My 2018 book called it “Agile.”

Each firm managing in the new way is rightly proud of its own approach. Google may talk of Project Aristotle while just as Amazon talks of “two pizza teams” and its 14 principles. Johnson & Johnson talks of “the J&J credo.” Given the large amount of “Agile in name only” or “fake Agile” around, there are understandable hesitations in many firms to call what they are doing “Agile.”

Roger Martin in his forthcoming book, When More Is Not Better (HBRP, September 2020). doesn’t use the term “Agile” and talks about something that sounds like the new paradigm which he calls “adaptivity”.

While there is much to be said for each firm customizing its approach to management and to call what they are doing as something uniquely as “their own”, the diversity of labels can cause critics to wonder whether these firms have anything in common at all.

Yet we, as outsiders, are entitled to note the similarities among these various home-grown core principles, despite the diversity of processes and practices. What we is doing are observing the underlying commonality among apparently disparate phenomena, in the same way that Newton pointed out that gravity works equally on feathers and stones, even though the way feathers and stones fall to the ground looks very different as a result of the greater air friction on feathers.

So Google and Facebook may look very different from Amazon and Microsoft,, but beneath their superficial differences in terms of processes and practices, we can discern common underlying principles that are driving their growth and wealth, and undermining firms that are pursuing a different paradigm, such as GE and IBM.

And read also:

What Good Agile Looks Like

Why Agile Is Eating The World