All 5 books, Edward Tufte paperback $180
All 5 clothbound books, autographed by ET $280
Visual Display of Quantitative Information
Envisioning Information
Visual Explanations
Beautiful Evidence
Seeing With Fresh Eyes
catalog + shopping cart
Edward Tufte e-books
Immediate download to any computer:
Visual and Statistical Thinking $5
The Cognitive Style of Powerpoint $5
Seeing Around + Feynman Diagrams $5
Data Analysis for Politics and Policy $9
catalog + shopping cart
New ET Book
Seeing with Fresh Eyes:
Meaning, Space, Data, Truth
catalog + shopping cart
Analyzing/Presenting Data/Information
All 5 books + 4-hour ET online video course, keyed to the 5 books.
Columbia Accident Investigation Board: The Boeing PowerPoint Slide





This thread has been replaced by PowerPoint Does Rocket Science, which extends and updates ideas about technical reporting at NASA and elsewhere.

ET, September 6, 2006









See the CAIB report on the Boeing PowerPoint slide at page 191 of the 10MB file of the full report,

or at page 15 of the 0.5 MB file for chapter 7:

http://www.nasa.gov/columbia/home/CAIB_Vol1.html

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Here is page 191 of the CAIB report.

The marked paragraph is astonishing, as members of the CAIB clearly had enough of PP from NASA. Serious analysis requires serious tools.

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Dr. Tufte,

I attended the seminar you conducted in Rye, NY on August 20, 2003, and when you briefed the afore-mentioned Boeing PowerPoint slide, you indicated that it might be in the Columbia Accident Investigation Board (CAIB) report. I downloaded the report today when it came out (FYI, www.caib.us is the permanent site) and I was very glad to see that your example made the final cut -- and even more glad that the CAIB expressed its displeasure with some of the PowerPoint briefings it received from NASA.

As I mentioned to you during one of the autograph sections in Rye, I have worked as a consultant to several similar government commissions, and over the years have learned a few things about how information is presented to them. While these observations do not apply to every commission all the time, I think they illustrate some interesting (and after reading "The Cognitive Style of PowerPoint," disturbing) trends in government these days:

+ A PowerPoint presentation makes for an actual "deliverable" in the world of government. Thus, the meeting or briefing to present the PowerPoint slides becomes a definable event that "answers the mail," and can fulfill an organizational obligation to produce something, anything. The presenting organization can then move on to other business in good conscience.

+ PowerPoint presentations can also be ambiguous, and therefore avoid any implied or actual commitments to action or to the facts. Concurrently, PowerPoint is also useful when an enterprise does not want to share much useful information with a government commission. It's hard to pin people down on this when their reply is, "What do you want, I gave the briefing?"

+ PowerPoint slides themselves can become the "minutes" of an event, with a corresponding loss of detail and clarity of who said what and when. Without such proper documentation, oftentimes what is agreed to is fuzzy in the memory of attendees -- and thus is difficult to resolve. In certain organizational cultures this is a distinct bureaucratic advantage since the slides can be designed to not leave much of a trail.

+ It's self-evident, but PowerPoint also becomes a crutch to avoid doing the real work that needs to be done -- detailed written communications that address the complexities of issues. And, most of the issues that government is dealing with these days are all incredibly complicated and multi-domain in scope. Even when the complex analyses are done -- they are then summarized and presented in PowerPoint style to their detriment. This is a key point in your monograph, but it needs to be emphasized more frequently.

+ The CAIB hinted that it did not welcome how NASA designed or presented PowerPoint presentations. They should put that in 96 point type on their website. Senior government and military officials don't seem to like PowerPoint presentations much -- yet they do not say this often enough to their subordinates and constituency organizations. If presenters' promotions, paychecks and careers were truly dependent on better presentations, they would do better. NOTE: This does not apply to coherent and interesting imagery, but rather to the bullet-point approach to information and related ills.

+ Without the accompanying text, PowerPoint slides make poor archival resources, and thus deprive future students, historians, officials and citizens of valuable knowledge that may have been created -- often at considerable cost. I have worked with several government commissions on their archiving, and can personally attest to this. In some cases, the presenters had not originally submitted the accompanying text because they didn't think the commission would be interested in it! Imagine students in the 24th Century reviewing PowerPoint slides compared to our reviewing the works of Newton and Galileo.

+ Finally, PowerPoint makes it way too easy to use gee-whiz animation and audio to overpower, obscure or otherwise skew the presentation of facts. While these presentations are often entertaining, the audience tends to remember the glitz and not the content. In one presentation I attended several years ago, I have forgotten all the content and the key messages of the speaker, but I do remember the musical score was "Flight of the Valkyries" and that the audio failed the first two times that the speaker powered up the presentation [INSERT JOKE HERE].

PowerPoint did not create all these problems any more than Outlook created get-rich-quick scams on the Internet. PowerPoint did, however, facilitate the packaging and distribution of a mindset that diminishes clear communication and dilutes the dissemination of knowledge.

I feel that the best solution to the problem is for the recipients of information, whether they be senior officials or seminar audiences or individual citizes to simply demand better from the presenters. And it has to be done frequently and vigorously until the prevailing mindsets behind the use (and misuse) of PowerPoint are overcome for the better. Everywhere in life we tolerate mediocrity, our reward is more mediocrity -- and visual communications are no exception.

The demand for better has to be matched with real action to change the cultures that produce failure. It is stunning to read the CAIB report and look at how little NASA changed since the Challenger disaster in 1986. Real change takes vision, resoures, political will and bureaucratic savvy to change what is wrong. The bookshelves of Washington, D.C. groan under the weight of government commissions whose recommendations and advice have gone unheeded by those who can make a difference -- and improving presentations is only a small but necessary part of the solution.

I close by repeating what I said to you at the end of the seminar in Rye -- that I am drenched in shame for all the presentations I have given over the years that upon reflection were poorly designed, poorly executed or reflective of intellectual sloppiness and laziness (although many of them looked really cool). As you may have guessed, I have also been a party at various times to each of the PowerPoint sins listed above -- and can attest to the ease in which they can be committed under organizational pressures and within certain bureaucratic cultures.

There is a clear choice to be made however, and if we so choose, communicating clearly and truthfully, regardless of the risks or consequences, is a better way to live by any standard or metric.

-- Paul Byron Pattak (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

To add to Paul Byron Pattak's thoughtful remarks, we ought to keep in mind that this now-famous Boeing slide epitomizes sloppy thinking and sloppy writing above all -- and to disastrous effect.

PowerPoint may be conducive to poor habits of mind, but such habits of mind would surely be present in a Microsoft Word document with a portrait orientation, using nicely kerned 12 point Times Roman.

Unclear antecendents -- such as "Tests show that it is possible at sufficient mass and velocity" -- would be unclear antecedents no matter what the software package used to communicate them. Failures in logic -- such as concluding that things are OK based on test data even though the foam ramp that plunged into the wing was 1920 cubic inches while test foam pieces were a mere 3 cubic inches -- would be failures in logic nonetheless.

As a communications consultant to business and industry, I have to say that the ubiquity of PowerPoint is disheartening. But even with its limitations, I have seen it bent to the will of a precise and logical train of thought. To me, what is more disheartening are the laspes in logic, and the inability to express ideas with precision and clarity.

So too in the case of Columbia.

-- Karl Keller (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

On page 191 of the report, the Columbia Accident Investigation Board is not only talking about my description of the Boeing slide; the CAIB also makes an important generalization about their experience with the NASA presentations made to the Board:

"At many points during its investigation, the Board was surprised to receive similar presentation slides from NASA officials in place of technical reports. The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA."

The Board is making distinctions among "methods of technical communication." Some methods are inherently better than others, independent of the user.

PP is inherently defective. PP is not serious. We need serious methods of communciation for serious problems.

Saying that it is a problem with the user rather than the tool blames the victims of PP (the audience, the content, and the user). My essay "The Cognitive Style of PowerPoint" reports evidence from thousands of PP presentations (from all sorts of fields) and then compares PP with other methods of technical communication. This evidence points to inherent defect, unless one advances the entertaining hypothesis that nearly all PP users are stupid and that nearly all users of other methods are not. PP inherent defect is a much more likely explanation. That explanation also has a direct practical prescription--abandon PP--rather than asking millions of PP users to learn tricks in an vain attempt to undo inherent problems in a slideware computer program.

Like the CAIB, Richard Feynman expressed annoyance at the bullet outline method of evidence presentation in his discussion of the Challenger investigation:

"Then we learned about 'bullets'--little black circles in front of phrases that were supposed to summarize things. There was one after another of these little goddam bullets in our briefing books and on slides." [Richard P. Feynman, "What do you care what other people think?" (New York, 1988), pp. 126-127.]

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Is there any independent, scientific evaluation of the learning outcomes of groups exposed to different methods of explanation?

For example, what is the objective, credible evidence of the evaluation of the effectivenes of explanation using Powerpoint versus other methods?

-- Mark Reilly


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

There is an interesting idea here: should PP be tested by the methods used in evidence-based medicine for treatments and in clinical trials for new drugs? That is, should the safety and efficacy of PP have to be demonstrated before it is forcibly administered millions of times a day to captive audiences? How does it compare with other methods? What are the test criteria? Right now, we are engaged in a massive uncontrolled non-randomized test of PP on millions of distinctly unwilling patients.

Oddly enough, PP is already deep into Phase I Clinical Trials--testing for the Maximum Dose to Toxicity! But normally for a new drug even to reach MDT trials, there must be considerable prior evidence, including test-tube experiments, tests on mice and other animal models,and pharmaceutical reasoning. PP has in effect flunked these prior tests and, if it were a drug, would never even have reached MDT trials. It is amusing, however, to contemplate laboratory mice being forced to watch PP presentations hour after hour. And what would the mice control group watch?

The experience of the Columbia Accident Investigation Board represents a fairly clean test of PP in action for serious problems--and CAIB reached a vivid conclusion about PP.

More generally, when information technology (IT) has been assessed for outcomes, the evidence has pointed to significant exaggerations of the alleged benefits of IT. Indeed, the economic benefits of IT technologies have been hard to find, at least until perhaps quite recently, in a good many microeconomic studies--as Robert Solow (Nobel prize economist) said "Computers show up everywhere except in productivity statistics." More on this point in our thread at https://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0000D8&topic_id=1&topic=

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

E.T., In a previous comment in this thread, you describe the users of PP as "victims". Certainly the managers and executives who establish the expectations and settle for PP presentations must be accountable for that choice. Peter Drucker, in "Management Challenges for the 21st Century" has this to say about the information responsibilities of managers and executives: To produce the information executives need for their work, they have to begin with two questions: "What information do I owe to the people with whom I work and on whom do I depend? In what form? And in what time frame?" Secondly, "What information do I need myself? From whom? In what form? And in what time frame?" Drucker continues: ".... by asking: 'To whom do I owe information, so that they can do their work?' communications are being focused on the common task and the common work. They become effective." If the managers in NASA worked from these questions, they would not accept superficial PP presentations.

Powerpoint is certainly not suitable for serious analysis, but the people who choose to accept it and use it cannot be characterized as victims.

-- Herb Burton (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

From the CAIB report, page 201:

"The Mission Management Team Chair's position in the hierarchy governed what information she would or would not receive. Information was lost as it traveled up the hierarchy. A demoralized Debris Assessment Team did not include a slide about the need for better imagery in their presentation to the Mission Evaluation Room. Their presentation included the Crater analysis, which they reported as incomplete and uncertain. However, the Mission Evaluation Room manager perceived the Boeing analysis as rigorous and quantitative. The choice of headings, arrangement of information, and size of bullets on the key chart served to highlight what management already believed. The uncertainties and assumptions that signaled danger dropped out of the information chain when the Mission Evaluation Room manager condensed the Debris Assessment Team's formal presentation to an informal verbal brief at the Mission Management Team meeting."

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Is it solely the fault of PowerPoint, or is it the belief that management wants "just the facts" in a summary presentation? In other words, is PP the cause or the symptom of the problem?

I have worked for many managers who ask for you to just tell them the outcome, without going into the detail of how that outcome was arrived at. PP, with its simplistic bullet-based format, makes it easy to give management sound bites. However, PP should not be used alone when delivering any form of report or presentation. It saddens and frustrates me when I arrive at a presentation (be it a sales pitch/technical briefing/conference/whatever) and receive a handout which merely contains printouts of PP slides. Presenters should provide solid material in their handouts, then use PP as an aid in presenting their material, rather than using PP as the material itself.

In the Columbia case, I think the key question to ask about the PP presentations should be: Is management to blame for demanding dot-point summaries, or is the technical staff at fault for "dumbing down" their information too far?

-- Amos Bannister


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

While I certainly agree that many communications sins are committed by PowerPoint users, I think the problem lies less in the limitations of PP than in the inability of Boeing's engineers (and so many others) to present concise information precisely. It also reflects the unwillingness of so many people to say "I don't know".

Their slide should have had the following bullet points: • Impact tests were conducted using small pieces of foam. • The foam that struck Columbia was 10-20 times larger. • We don't know how much damage this could cause.

Next slide: We recommend: • Use any and all means to determine actual damage. For example: • Visual examination by spy satellites • Space walk

I've spent many years as an advertising writer, so know how hard it is to communicate things as economically and effectively.

Summarising information and presenting it concisely is not (necessarily) "dumbing down". Doing it effectively is VERY demanding. A long-winded written piece (assuming the long wind includes plenty of raw data) might be hard work to digest, but the astute reader will sift the data from the chaff. When someone - especially someone who's not a professional communicator - does a sifting to produce a hurried PP (and they always are hurried), there's a good chance they'll create something ambiguous or misleading. Or simply something that relies on presumptions about the audience's knowledge ("Well everyone knew the tests were done with...")

I use slide shows (currently created with Apple's Keynote, previously with Acrobat from QuarkXPress documents) for presentations and lectures. However I have 25 years of professional writing and graphics experience. Just as often (for small presentations) I'll print graphics and mount them on foamcore board.

To summarise: • Bullet points are not inherently evil. • Muddy thinking is. • Condensed muddy thinking is even eviller.

[Sorry, couldn't resist]

PS: If you think PowerPoint can create rubbish, put QuarkXPress or InDesign into inexperienced hands and see what you get!

-- David Glover (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Mr. Glover has a good point. There is a serious need to summarize detail & visual aids (KeyNote, etc.) can help. [Not do, but help.] The CEO of a biz or the CO of 1 MarDiv cannot get bogged down in detail; that is why the task was delegated. The Boeing slide, to my utterly non-scientific mind, demonstrates that the tool (PP) was used inappropriately: to conceal ignorance & sloppy thinking.

PP is just a tool; it can present data & concepts well if allowed to do so. If you give me a block of marble, a hammer, & some chisels I will make dust & a smaller rock. The same items in Michaelangelo's hands gives us The Pieta. Newton used calculus, how? In my hands?

A tool is only as good as its user.

-- John Schedler (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

This makes one good point: responsibility for poor presentations rests ultimately with the presenter--and the presenter who chooses to use PowerPoint. But it is more complicated than that.

PP has a distinctive, definite, well-enforced, and widely-practiced cognitive style that is contrary to serious thinking. PP actively and routinely facilitates the making of lightweight presentations. My essay, "The Cognitive Style of PowerPoint," reports evidence based on several thousand PP slides, 5 case studies, and many quantitative comparisons between PowerPoint and other methods of communicating information. The comparative data are particularly telling: some methods of presentation are better than others. And PowerPoint is rarely a good method.

The Columbia Accident Investigation Board, in their analysis "Engineering by Viewgraph," also makes distinctions among methods of presentation: "At many points during its investigation, the Board was surprised to receive similar presentation slides [similar to the Boeing slide with all its problems] from NASA offcials in place of technical reports. The Board views the endemic use of PowerPoint briefing slides instead of technical papers as an illustration of the problematic methods of technical communication at NASA."

The tool metaphor does not provide any intellectual leverage. Some tools are in fact better than others; some poor performances are in fact the fault of the tool. Saying that the problem is with the user rather than the tool blames the victims of PP (audience, content, presenter). Nearly all the evidence of the essay suggests that there is inherent defect in PowerPoint, unless one advances the entertaining alternative hypothesis that nearly all PP users are lightweights and nearly all users of other methods are not. This is not the case; PP has inherent defect.

Thus the policy recommendation in my essay: Imagine a widely used and expensive prescription drug that claimed to make us beautiful but didn't. Instead the drug had frequent, serious side effects: making us stupid, degrading the quality and credibility of our communication, turning us into bores, wasting our colleagues' time. These side effects, and the resulting unsatisfactory cost/benefit ratio, would rightly lead to a worldwide product recall.

The argument "it is not the tool, it is the stupid user" has been spread around the internet by Microsoft pr types lately with regard to my essay and especially with regard to worm/virus/DOS attacks on Windows and Explorer. MS in effect says "If only users would keep up to date with the patches, and the patches of the patches, then users would have not have this problem. It is not the fault of a sufficiently well-patched-patched Windows operating system."

Doesn't anyone take responsibility for anything these days?

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

<<Thus the policy recommendation in my essay: Imagine a widely used and expensive prescription drug that claimed to make us beautiful but didn't. Instead the drug had frequent, serious side effects: making us stupid, degrading the quality and credibility of our communication, turning us into bores, wasting our colleagues' time. These side effects, and the resulting unsatisfactory cost/benefit ratio, would rightly lead to a worldwide product recall.>>

There is another serious side effect: PP presentations are contagious, moving through the air (sometimes at the speed of light), from presenter to presenter. Among PP presenters, one of the things they do is share their presentations with colleagues who regurgitate them after slight modifications.

-- Gene Prescott (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

"Among PP presenters, one of the things they do is share their presentations with colleagues who regurgitate them after slight modifications."

Indeed, I believe this is a good point among many already touched on that make it all to easy to create fatal mistakes in Powerpoint.

A presentation may have had a copy and paste of a slide or two or thirty onto a new background. Sometimes a perfectly good slide will become almost useless in a new format and 'template'. Fonts become distorted, graphs lose vector information, and the slide data meant for one subject is shoe-horned into being evidence for something else entirely. The amount of Frankensteined Powerpoints floating about is frightening.

-- Jeffrey Berg (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Mr Prescott's comment about sharing presentations and Mr Berg's comment about fatal content are important.

1. 'Sharing' presentations

I don't think that I'm the sole exception to Mr Prescott's generalisation that people who use PowerPoint share presentations. Any presentation in whatever format is personal. I would no more 'share' a presentation than I would a toothbrush.

However, if we can measure objectively the extent of this sharing of PowerPoint AND make comparions with other methods of presentation, then we can also make solid progress with the debate. So, is this level of sharing 5% or 55%?

2. Content with 'fatal' mistakes

In respect of Mr Berg's important point about 'fatal' mistakes in Powerpoint, how does the frequency of fatal content in PowerPoint compare with the fatal content in flip charts and acetates and printed lecture notes and laboratory records used for teaching or health care decisions? Is it higher or lower?

We need to measure what we are talking about, otherwise the argument just falls back time and again to impression and anecdotes. Interesting and important but biased by individual experience.

-- Mark Reilly (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

This is somewhat off-topic, as it has nothing to do with PowerPoint and the display of information, but the New York Times has shed more light on what went wrong:

http://www.nytimes.com/2003/09/26/national/nationalspecial/26ENGI.html?ex=1379908800&en=10772541a545b410&ei=5007&partner=USERLAND

[link updated February 2005]

-- Pierre Scalise (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Mark Reilly posted:

<<1. 'Sharing' presentations

I don't think that I'm the sole exception to Mr Prescott's generalisation that people who use PowerPoint share presentations. Any presentation in whatever format is personal. I would no more 'share' a presentation than I would a toothbrush.>>

Mr. Reilly then inquired regarding whether the % of sharing was significant. I don't have any sources of actual statistics, merely what I have encountered, personally, from participation in presentations for organizations and as an attendee of some presentations.

Regarding the former, persons with responsibilities regarding a common topic of a national organization to be presented in various states routinely share PP presentations, sometimes collectively agreeing on who will do the primary work. I've also noted officers of a national organization using a common set of PP slides over time.

Regarding the latter, sometimes CDs are distributed to attendees that include the PP presentation.

I don't think any of those folks would share their toothbrushes.

-- Gene Prescott (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Isn't "sharing" what was once called "plagiarism"?

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

pla•gia•rize

v., -rized, -riz•ing, -riz•es.

v.tr. To use and pass off (the ideas or writings of another) as one's own.
To appropriate for use as one's own passages or ideas from (another).
v.intr.
To put forth as original to oneself the ideas or words of another.

pla'gia•riz'er n.

In a literal sense it would seem so.

Some "professional" presenters "share" via "barter" ... that is, they both receive and furnish PP presentations from/to colleagues. So the subsequent uses are with the permission of the creator. I don't recall attribution being given in all cases, but I can recall it in some cases.

In a more general sense, most of what all of us know originated with somebody other than ourselves. So whether it is PP slides or just knowledge we all may be plagiarizing.

-- Gene Prescott (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

From William Langewiesche, "Columbia's Last Flight," The Atlantic Monthly (November 2003), pp. 82-84.

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

The Los Angeles Times (latimes.com/columbia)completed last week (12/21-26) a six chapter series, "The Butterfly and the Bullet," of Columbia events. This series provided a differnt perspective with details of workers' and individuals' activities. J. D. McCubbin

-- J. D. Mccubbin (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

I'm a newcomer, having attended the course just this week in Arlington, but I don't see what I found addressed anywhere else here.

In publishing Dr. Tufte's (ET) criticism of NASA's sloppy typography, CAIB introduced more careless errors. The lower half of the page appears to be a visual quote from ET's report: it is set off with a different page background, and captioned "The analysis by Dr. Edward Tufte..."

As in Tufte, the left side of this is a screen shot of the PowerPoint slide, and the right side is his commentary. Arrows point from commentary to slide, as in ET's essay. One would think it was the image Tufte has online.

This excerpt is not an image, as it appears, but was re-typed, and in the process, changed and made less clear and accurate, murking up the entire point somewhat.

a) The reference to "1920cu in" is mis-typed as "1920cu. in", making it no longer dissimilar to "3cu. in" (the phrase is correctly presented as "1920cu in" at two other points on the same page, and in all of ET's materials).

b) The word "ambiguous" is misspelled as "ambiquous" in the CAIB report.

c) Neutral quotation marks replace ET's left- and right double quotes -- no scanner would make that change.

d) ET's isolated question mark comment is removed, but

e) the arrow from that question mark to the non-sentence was not removed. This leaves the CAIB report with an arrow leading from the sentence about the pronoun "it" to a sentence that does not contain that pronoun.

f) The typographic orphans do not appear, and the sentence preceding them is in parentheses. This is consistent with Tufte's online version, but not with the published article, which CAIB cites in two footnotes on this page.

g) There is no citation for this exhibit, nor is the excerpt described properly as an "artistic impression" of Dr. Tufte's research. While retyping, however, the linefeeds from ET's originals was preserved manually, reinforcing the impression of it being an accurate image.

h) The shapes of Boeing's bullets is changed: from bullet-dash-diamond-bullet to bullet-dash-bullet-bullet. This (and the circling of the "it" arrow) reveals that the left-side screen shot is no more of a visual quote than the commentary.

The skill and care that went into producing this are obviously inadequate, the ethics are murky at best.

Marty Mangold

CAIB report: http://www.caib.us/news/report/pdf/vol1/full/caib_report_volume1.pdf

Tufte online version: https://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0000Rs&topic_id=1&topic=Ask%20E%2eT%2e

-- Martin Mangold (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

This is a very perceptive contribution. I averted my eyes to the disturbing reworking of my analysis. Direct quotes would have been better than a rephrasing.

The New York Times also badly reworked my analysis, but who am I to complain about being in the Sunday News of Week in Review? The Times does better than other newspapers, but still doesn't get it right (everything in the paper is exactly right, except for those things that you know something about). It makes one despair of the possibilities for accurate and precise communication.

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Admiral Gehman lectured on the behavior of large organizations recently at the Naval Academy. He said a couple of things I'd been musing about and now I definitely want to learn more. I'm hoping you can provide some additional insight as your own project manager and as someone who's involved with the CAIB.

1) Admiral Gehman said program managers have four tradespaces: time, money, quantity, and performance margin. You want more jets on the same schedule? You need more money or you will sacrifice performance margin: QA checks will be skipped; tolerances will slide. I think there was some assumption that either processes were fixed or already optimized. He said if you understood how to operate within those four parameters, you had all the knowledge contained in a master's degree of business administration. Do you have any suggestions as to where he may have gotten those four variables, or any further reading on this?

2) Admiral Gehman said the CAIB developed a template of the best way for a large organization to operate (and organize?), then compared NASA to the template and measured the difference. I didn't see such a template in the CAIB report. How can I get that template?

-- Niels Olson (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

I have no question, but here is a comment: I'm researching this item for a class here at Johnson Space Center. Having been immersed in the NASA bureacracy for a while, I don't excuse the misrepresentations in the Boeing slide. However, I perceive a pattern that is all too familiar - probably taken to the extreme under tight time pressure in the case of this particular slide.

I have seen (and been the victim of) presentations that were made in a few hours because the boss wanted "to know what's going on by close of business". Usually such presentations show sloppiness like this because they were "thrown together" in such a hurry. No excuses, but this is problematic in the organizations that I have been in.

I think that there is a spiraling, or dumbing, down that is going on. The underlying problem may be started when someone is able to put together a PP presentation in short order and impress the boss. The boss then expects all employees to be so 'productive'. The employees respond with more lightweight presentations in shorter and shorter times. This can be contagious and show up in other media such as Word or page formatting programs. The basic premise is to oversimplify and sometimes make nonsensical statements in order to shorten the presentation - no matter how the presentation is made.

So I may be a bit more pessimistic because I believe that PP dumbing down is spreading like a virus to all sorts of technical presentation and may even be fogging the engineers thoughts because they are thinking of how to present something before they get the content down.

Thus an even larger effort than just PP phluff removal is needed to stop this virus before it destroys engineering capability in America or for that matter the world.

We need to re-emphasize the job of the engineer. That job is not to "throw together" presentations but to determine facts as best as possible and then spend some quality time trying to communicate those facts to superiors who use them to make important decisions. Let's all step back and take a deep breath. Slowing down may help.

-- Joe Shreve (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

Yes it is much more than PPPhluff. Indeed the Phluff is a diverson. PP comes with a distinctive cognitive style. The Columbia Accident Investigation Board made much the same points as our Kindly Contributor above.

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

This material represents a shift in my interests, from a producer-oriented view of evidence presentations (strategies of information display) to a consumer-oriented view (how to evaluate an evidence presentation). This an obvious idea, but no less important for being so.

What happened in the cases of the Challenger and the Columbia accidents were not only failed presentations but, importantly, the failure to detect failed presentations. Two NASA contractors (Morton Thiokol and Boeing) produced presentations that allowed the Challenger to be launched under adverse conditions and the injured Columbia to continue its flight innocently without further analysis of the threat that led to its destruction.

This provokes all kinds of interesting issues: How do high-level decision-makers detect problems in presentations? What should one look for in deciding whether a presentation is credible? The analysis of the Boeing-Columbia slide and the Columbia Accident Investigation Board work are about how not to be fooled by a presentation.

-- Edward Tufte


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

PowerPoint reinforces a tendency to simplify the content of decisions the further you go up the organisation. A hundred bullet points on twenty slides at junior executive level becomes five on one when it reaches the board.. This might just be acceptable if the discussion stays within the same profession or technical discipline: a senior engineer will understand the principles of a material stress test carried out by junior staff without needing to have the whole experiment documented in a presentation.

However, at some point, responsibility for decisions passes from the professional and technical experts to general management. General management decisions are across disciplines and therefore the content of each professional decision needs to be examined. What frequently and regretably happens however is that one summary slide from each of, say, six disciplines is summarised into one slide with a bullet point from each. Taking a decision on this basis is the executive equivalent of painting by numbers.

At general management level, the most important contribution that the professionals can offer is an assessment of the limitations of their own analysis. A CFO for example will have a profit statement prepared by his department to the nearest cent, but has a duty at main-board level to express it as a probability plus-or-minus to the nearest ten million dollars. This is quite the reverse of the processes by which decisions are arrived at within professions. General management discussions therefore should never be a summary of professional decisions but a detailed revisiting of the content of professional discussions in the broader context of cost-benefit or political considerations.

Senior general managers require a greater level of detail for their decision-making than is required by their subordinate heads of professional departments.

-- Martin Ternouth (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

I appreciate the points Mr. Ternouth makes about decision-making processes. It is a critical point in that the boil-down that often results in PP phluff need not have been borne out of laziness, but a legitimate need to simplify so that authority can be exercised in the absence of deep, deep knowledge.

Perhaps more of a 'shareholder' model of decision-making would mitigate these effects. That is, a node's share in the authority of a decision would reflect that node's access to/share of the rationale. Perhaps in those situations where it is impossible for general management to process the complexity necessary to avoid a particular disaster, the decision should drift out of their authority by a commensurate amount.

But, of course, who would do that triage?

LeMel

-- LeMel (email)


Response to Report of Columbia Accident Investigation Board: The Boeing PowerPoint Slide

The underlying problem may be started when someone is able to put together a PP presentation in short order and impress the boss.

This seems to imply that putting together PowerPoint presentations is quick and easy. As someone who has created literally hundreds of PP (they are de rigeur in corporate life), I maintain that putting together a PP takes far too long. PP users are forced to fit the message to the medium, and that often takes a lot of time for non-trival information.

A related point is that since I am not the "big boss", I have mostly failed in the past to get business audiences to read a document in order to understand an important position or point. If I send out a document ahead of time, most in the meeting will not have read it. There is typically a revolt if I suggest we all take 5 minutes of meeting time to read a document, with angry demands to "net it out". These are the same people who will happily sit through an hour or more of PP. Typically, I then give them the bottom line (on one slide) and we spend the next hour verbally covering the ground of the written document. However, I avoid referring them to the relevant section of the document as we do this, as that is almost always counter-productive.

Lastly, in the case of Columbia, it also seems that team knew what message they were expected to deliver in their PP -- I suspect this is why they hedged with the word "significant". I base this on chapter 9 of Jim Surowiecki's "The Wisdom of Crowds" (which is, incidentally, a fascinating book). Linda Ham, leader of the NASA mission management team, told everyone "I really don't think there's much we can do so it is not really a factor" -- effectively working backwards from the conclusion that nothing could be done (which was false, it seems), to the premise that the foam collision couldn't be a problem.

Regards,

Mathew

-- Mathew Lodge (email)


Very interesting stuff. I have powerpoint issues myself. I have written two MScs, been published and have A level English language (applause?), but can't get a powerpoint "right". Powerpoint is the tool of the prevaricator. There is no quality standard, no requirements or gates to pass. It is too long, too short, not enough detail, too much detail, pitched too high-brow, pitched too low-brow. It can take longer to write the powerpoint than to design and develop the concept! I have another issue, too. The mindset of the reader. It appears that the challenge is to write about a complex subject (I too work in aerospace), but make it understandable in seconds, without voice-over. I think five-ten minutes and about five slides should be enough. Five-ten minutes is a significant amount of time, but not a lot in the scheme of an eight-hour day. It seems to be unnacceptable. Only seconds are allowed. If it can't be looked at and immediately understood it is not good enough. I think this is lazy and am surprised by how professionally acceptable this view seems to be and how professionally unacceptable my view is.

-- Paul Burton (email)


Be mindful of the working ecology of powerpoint. You put people in darkened space, with oneway interaction and diminish their sensitivity to those with whom they are to share the information. Optical rhetoric is important but needs to be created with the knowledge of how to fulfill the intent of the message. When one takes this perspective new possibilities arise as the knowedge base is sound.

-- Roger Daventry (email)




Threads relevant to PowerPoint:
Don't get your hopes up.
Compares tables, slopegraphs, barcharts for showing cancer survival rates.
A look at a rich and complex question: What are the the causes of presentations?

An intriguing but under-explored topic.
Account of the role of PP in the shuttle Columbia accident, followed by many good alternative methods and examples for technical presentations.
Mainly recent examples of leaked PP slides in the Iraq war.


Privacy Policy