The Logic of Failure, a decades-overdue book review

I'm interested in the psychology of flawed reasoning. Two broad categories of unsound reasoning--logical fallacies and Groupthink--are familiar to most educated people.

However there's a third type of mental minefield. Black Box Problems are unavoidable, and they are quite challenging. Bureaucratic approaches to solving BBPs have several inherent pitfalls.

For example, men are not born with the software that's needed for understanding where women are coming from, in an emotional sense. In that respect, it's helpful for a young male to have an older sister, who can clue him in. Self-help books can also be helpful. But even with the best of intentions, most men--including myself--have many episodes of "Open mouth, insert foot." We muddle our way through relationships, and through the other gray areas of life.

It's best if we can learn from our mistakes as soon as possible. Here's a famous Einstein quote:

The definition of insanity is doing he same thing over and over again and expecting a different result.

Unfortunately, the skills needed to solve Black Box Problems are not taught in school. Dietrich Dörner's book, The Logic of Failure, gives an in-depth treatment of Black Box Problems.

Two caveats. Dörner's simulations involved occupational settings. He did not intend to write about fulfilling relationships. That was my extension of his ideas.

Second, "Black Box Problem" is my expression, not Dörner's. Black Box Problems are situations in which you must make a decision, but don't have as much information as you would like. Some of the essential information is locked away inside a 'black box'. You can't open the box, but you can poke it in various ways, and take note if it rattles, squeaks, creaks, or whatever.

Trial and error are necessary parts of this problem-solving process, as is the ability to learn from experience. Good judgment, levelheadedness, practical intelligence, and horse sense are all expressions that describe the ability to solve Black Box Problems. People skills can also be important. Dörner's experiments are a window into the human psyche as it grapples with perplexing situations in the real world.

These are the propaganda techniques that dishonest people use to manipulate us into believing things that are not true; and into acting against our own best interests, and against our own core values. Here are four of the classics.

1. Argumentum ad hominem is an attack on a person or on a class of people, in order to discredit their ideas. Of course, the inconvenient reality is that cute and cuddly people are capable of having bad ideas, and that uncool nerdy people can have good ideas. Yes, some sources are better than others for the facts. However the source of an idea is not an adequate gauge of the quality of the analysis of the facts, once these facts are agreed upon.

Anthropogenic Global Warming (AGW) alarmists frequently use this rhetorical device. Informed skeptics are labeled as Deniers, suggesting that they are on a par with Holocaust Deniers. I think that this particular ad hominem is worthy of a Godwin Award.

2. Argumentum ad populum is about the bandwagon effect. It's an appeal to the comfort of conforming to a real majority, or to an Astroturfed majority. Sometimes the majority gets it right, and sometimes they don't. Truth is not democratic.

3. Argumentum ad veracundiam is an appeal to authority. It's a variation on the themes of "Trust me, I'm an expert" and "Dada knows best." Many otherwise educated people still think in this way; they merely switch to a different dada in their university years.

The big picture on appeals to authority: Sometimes experts get it right, and sometimes they don't. For example, if a person is an expert on the folklore of astrology, that fact does not make astrology valid--even if that expert happens to believe in astrology. Moreover some authority figures are honest, and others have hidden agendas.

Notwithstanding my high school English Lit teacher, trotting out the opinions of mavens does not qualify as critical thinking. What if two of the mavens have opposite opinions? Watching them duke it out can be entertaining, but they can't both be 100% correct.

4. A Straw Man Argument is a rhetorical technique in which one puts words into the mouth of his opponent, and then argues disingenuously against that deliberate misrepresentation.

If you're skilled at recognizing logical fallacies, you're less likely to be buy into fairy tales. However Groupthink is more insidious.

Groupthink is a subtle form of flawed reasoning, in which group members are unable to distinguish between the veracity of a proposition and a manufactured consensus. At first blush, Groupthink looks like argumentum ad populum. But there is a distinction.

Because individual members of the group are afraid to speak their minds--and often aren't even consciously aware of that fear--each contributes to the mediocre and sometimes disastrous conclusions and policy decisions that the group makes. There's a larger component of self-deception in Groupthink than in classical logical fallacies. In a Groupthink milieu, group intelligence is seldom greater than that of the least intelligent member of that group.

The concept of Groupthink has been with us since 1952, when William H. Whyte, Jr. first articulated it. Most educated people are at least dimly aware of the concept. Yet the same educated people, as individuals, are capable of making monumentally stupid decisions when they find themselves in positions of power. Why is that?

Dietrich Dörner’s book, The Logic of Failure, addresses this larger issue. Dörner's psychological experiments utilized computer simulations of Black Box Problems (BBPs) that could easily arise in the workplace.

In one of the simulations, a would-be city manager tackles a scarce resource allocation problem that his counterpart in the real world is likely to encounter. Sim City is a computer game that's similar to the one that Dörner used in this particular BBP.

A second Black Box Problem involves a malfunctioning commercial refrigerator. There are other simulations as well.

The volunteer in each experiment is given an overview of his particular Black Box Problem. Then he matches wits with a computer simulation of the BBP.

On the first pass, the volunteer makes an input, and takes note of the corresponding output. He then attempts to grasp the underlying reality.

Then in the light of his tentative mental picture, the volunteer makes a second educated guess, and sees if the second output partially corroborates his intuition, or if it takes him in a new direction. Given a reasonable amount of time, the volunteer may or may not learn from experience what he needs to know.

Here are three of the many nonproductive approaches to solving Black Box problems. The first is magical thinking. The volunteer jumps to the conclusion that certain numerical inputs to the computer simulation have magical properties.

Scenario 2: Sometimes a volunteer will zero in on a sub-problem, and ignore the larger picture.

These first two types of bureaucratic errors occur because most people are uncomfortable with uncertainties. In the micromanagement case, a person emphasizes the aspects with which he is most familiar, in order to remain within his comfort zone--even though his neglected areas of responsibility are going to hell in a handbasket.

Scenario 3: The volunteer may assume that the system modeled by the computer is linear. However the system could spiral out of control while he is dithering. A meltdown in one of the first-generation nuclear reactors is a good example of non-linearity.

One can find many examples of these flawed thought processes in government bureaucracies. Interestingly, business people outscored university students in Dörner’s simulations.

The first part of The Logic of Failure goes into great detail about various approaches to solving and failing to solve Black Box Problems. Dörner's experiments used computer simulations of BBPs that could easily arise in the real world. A few of the volunteers did get it right. However most did not. Dörner thoroughly analyzed the failed approaches, and sorted them into categories.

The middle part of Dörmer's book is more of a prescription than a description. He tells us how to approach Black Box Problems. With each of Dörner's points, I found myself nodding in agreement. However, at the end of the middle chapters, I realized that there was no way that I could remember all of that stuff when I really needed it.

The last part of the book is more concrete. Dörner concludes that people could improve their Black Box Problem-solving skills by practicing on the kinds of computer simulations that he presented to his volunteers. The skills acquired in struggling with one type of Black Box Problem should be transferable to another type of BBP. SimCity, anyone?

I have a health problem that falls between the cracks. I've spent thousands of dollars on conventional medical approaches. In the long run, each medication was worse than the original problem. I finally realized that the underlying health issue was a Black Box Problem, and that I was mostly on my own.

Since that epiphany, I've tried to understand the problem, applying my scientific background to articles and books that I've read.

I also listen very closely to what my body is telling me. I've had a few limited successes. My hay fever is more manageable, without medications. My rheumatoid arthritis is in remission. I'm a bit leaner than I was in the late 1980s. And my sleep is a little better.

On the other hand, my neurological functioning has not improved. My health project is a work in progress. And Dörner's book was a partial validation for my approach.

Come to think of it, there's an article in Scientific American, "The Expert Mind," which complements Dörner's work. Here's a LINK to a preview of this piece. However the rest of the article is pay-walled.

Copyright by Larry Fields 2012