All 4 books by Edward Tufte now in
paperback editions, $100 for all 4
Visual Display of Quantitative Information
Envisioning Information
Visual Explanations
Beautiful Evidence
Paper/printing = original clothbound books.
Only available through ET's Graphics Press:
catalog + shopping cart
All 4 clothbound books, autographed by the author $180
catalog + shopping cart
Edward Tufte e-books
Immediate download to any computer:
Visual and Statistical Thinking $5
The Cognitive Style of Powerpoint $5
Seeing Around + Feynman Diagrams $5
Data Analysis for Politics and Policy $9
catalog + shopping cart
Edward Tufte one-day course,
Presenting Data and Information
Music Animation Machine

I enjoyed your seminar today in Austin, and wanted to hear your thoughts on the similarities of the Music Animation Machine and player piano rolls (or equally similar, music box cylinders). I was struck while watching the video how much it reminded me of watching the player piano at Shakey's Pizza when I was a child and noticing that I could "see" the music as it was being played. I get the same effect watching a music box play, when the "guts" are visible.

-- Jeff Koke (email)

Response to Whitney Music Box

Bach meets Escher meets Basinski meets James Clerk Maxwell?

-- Niels Olson (email)

The Whitney Music Box is by the same guy who used sunset photos as data points in a graph (discussed here some months ago).

Namely, me.

Nice to know some folks I greatly admire are enjoying these little experiments.

-- Jim Bumgardner (email)

Some time ago, Neil Mussett asked in this thread, "Do you have any comments on musical notation itself? It seems to be an area rarely questioned (I get the impression that even the typography is duplicated from book to book)."

I'm just getting into a major project now on music visualization techniques, so I thought I'd share some thoughts on this here. The notational system for western music provides an elegant and efficient system for providing directions for reproducing a musical work. Of course, this system isn't really a "system," but an ever-changing set of conventional practices that evolves over time to keep up with the changing needs of its users. Some features have come and gone either because the musical needs changed or because they didn't work well or were impractical, but the basic elements of our notational system are now over a millennium old. The following brief history oversimplifies things a great deal.

The origins of Western music notation lie in the medieval monasteries. For a long time the music for plainchant was passed down through oral tradition. The ninth century saw the development of "neumes," which were basically squiggles that represented the shape of the melody and served a memory aid. Over the next couple of centuries, these neumes were aligned to a vertical (but undrawn) grid (making them essentially low-res Sparklines), aligned to a single reference line, and by the 11th century, aligned to a four-line staff, with letter-clefs used to identify the name of one of the lines. The five-line staff we are accustomed to now appeared by the Renaissance, as the range of musical parts expanded beyond that of the basic chant repertoire. What we think of as modern rhythmic notation wasn't fully developed until the 17th century.

The notational system reads in the same way as the prose of European languages--left-to-right, top-to-bottom--and is thus as easy to scan as text. The five-line staff for denoting pitch is probably perceptually optimal for the kind of rapid processing needed to read music "in time." I suspect people would be noticeably slower decoding the location of notes on a six-line staff, for instance. Because they are visually distinct from the staff, however, notes on up to three ledger lines can be read with virtually the same fluency as on the staff. The diatonic basis of the notational system (notes on the staff represent diatonic pitches unless made chromatic with an accidental) maps perfectly onto the physical layout of the piano and other keyboards (white vs. black notes) and woodwind instruments (whose "default" fingering layout also follows the diatonic scale).

The rhythmic notation based on shapes (open and closed noteheads, stams, flags, and beams) and not on horizontal spacing acknowledges the fact that we are not good as measuring precise differences in spatial distances and the hierarchical relationship among the durations is easily learned. Though it is conventional to give more horizontal space to notes of longer duration, and doing so does in fact help in mental processing of the notation, this is not done proportionally. The use of beams to gather shorter notes into beat-length groupings also draws on our innate cognitive abilities. Notation to indicate articulation and phrasing (e.g., slurs, staccato, tenuto) are effective pictograms that depict their function. Conventions for indicating fingerings, pedaling (for piano music), dynamics, tempo, etc., are easily distinguished from one another and rarely ambiguous.

Musical experimentation in the 20th-century has led to many new notational features (microtonal tuning, un-measured rhythms, constrained improvisation, new performance techniques, etc.). It should be noted, incidentally, that there have been numerous proposals over times for new, usually simplified, systems for music notation, but these invariably fail for one reason or another. Part of my future research will be to try to determine reasons, beyond simply entrenched tradition, why this might be.

I know relatively little about the art of music engraving and typesetting, incidentally, except to note that it is very much an art. Certain publishers have house styles, I believe there have been some "classic" approaches, which would account for some of the consistency that Neil observed. And for modern computer-set music, there is a very small number of programs in use, which also encourages a degree of uniformity.

-- Eric Isaacson (email)

Look at visualisation which very close to pictures:

I also very much enjoyed your presentation (December 11, 2002, San Francisco) of the Chopin and Scarlatti animations.

Although I'm a webmaster by trade, I was trained in classical music and have my masters' in performance from NEC in Boston and have played the double bass since childhood. I have always beleived very strongly that there is a certain visual beauty to the architecture of music. (Had that notion since I saw Disney's "Fantasia" as a kid...)

What was shown on the screen today really is not that different from the representations of traditional music notation. In fact, I think that such an animation would be extremely cool and just as effective by adding an additional dimension of the actual notation to the movement - so the colors would remain, but then you'd have the staff, key signatures, notes and accents and ligatures to go along with it. I love to observe such patterns and movements when reading a piece like the Chopin you showed us today or, even more intricately, when reading through an orchestral score - especially when the orchestration itself is part of the beauty, such as Bach Brandenburg #3, Tchaikovsky Symphony #4 1st and 3rd mvts., Strauss' Alpine Symphony, and Mahler Symphony #2 1st mvt. just to name a few. Great stuff!

-- Joe Lewis, December 12, 2002.

Under the FAQ's he has a link to downloading the software. It is free

-- Glenn Nevill, December 04, 2001.

There are now a couple of QuickTime short demos of various pieces off of this site.

-- Pat Boule, July 11, 2005.

a more tweakable method is to use a wav to midi converter, such as amazing midi, and then import into whatever software eg. cubase.

on another note:

i would appreciate any constructive critisim on a similar project: thank you

-- jim ryan, September 25, 2005.

If you're running Windows, you can view a scrolling bar-graph score while playing MIDI files with this freeware MIDI file player.

-- Stephen Malinowski, June 13, 2006.

-- Pavel (email)

No Music Animation Machine software for Mac OSX? You're killing me!

-- Michael Kentoff (email)

Dear Mr.Tufte:

I was fortunate to have been able to attend one of your seminars(The boston seminars in Fall 2008). It was quite a learning experience. I was particularly impressed by the Musical Animation Machine in your presentation.

Anyway I came across this wonderful piece of free software that seems based on the visualization principles of the Musical Animation Machine and takes it even a step further. I wanted to share the link with you and this forum.



-- Mathew (email)

When I watched the animation machine at your recent course in Atlanta, GA, I thought of how sometimes dance is a similar image of musical data. I was also reminded of a juggling performance by Chris Bliss, in which his juggling balls allow direct visualization of the music. Here is one of many possible links to his juggling:

-- Anna Edlund (email)

Re: My previous post about rotating the animation to increase its explanatory power. I did not realize that you also used a Scarlatti example. For Scarlatti, depending on whether you were explaining a performance on a single manual or double manual harpisichord, you would need more than one vertical scroll like the sparklines for multiple exchange rates. I would stack them vertically. You would also need to indicate active stops and manual couplings. I apologize for the multiple posts.

-- Deepak Pradhan (email)

Bobby McFerrin demonstrating the power of the pentatonic scale.

World Science Festival 2009: Bobby McFerrin Demonstrates the Power of the Pentatonic Scale from World Science Festival on Vimeo.

-- Niels Olson (email)

Recent works

There are many recent pieces by Stephen Malinowski at

-- Edward Tufte

hearing typography:


-- B Wasserman (email)

Music Animation Machine meets Guitar Hero

All of a sudden, my kids are rabidly learning piano. First, Mozart:

Here's a more detailed look at some of the main features

It does help to have a an actual USB keyboard, though Synthesia provides a QWERTY mapping if you want to get starting playing around. More on the Synthesia homepage.

-- Niels Olson (email)


the link in the blog post below reminded me of the MAM. I use the MAM to illustrate to my intro. histology students how with practice at the microscope they will begin to use their "eyes and brains" at the same time colors-1862/

all the best,

B Wasserman

-- BWasserman (email)

Robotics scientists Jeff Lieberman an Dan Paluska created a kinetic sculpture that performed a full size marimba by propelling small rubber balls at the bars arching several feet into the air. The effect was similar to everything described here, you could see the music before it happened - flying through the air in the gallery. Because of the arches, it almost felt like time slowed down for an instance before the chords hit as you imagined what was to come. There was also a percussion battery and glass harmonium. Of added interest was that the music was created by combining a melody from an online "audience" member and a purely musical form (with no melody, only variations) created by a composer. It was very satisfying composing for something when you have little knowledge of the end result!

Uploaded gallery performance: m/watch?list=UU9UDiX_vAD7ru5KL2V2IEXw&v=3H4ATMYugpc&feature=player_detailpage

-- Brad Kemp (email)

Visualisation which very close to pictures: Colour are Sounds website

-- Pavel (email)

In the 70's, Rainer Wehinger created a visual listening score to accompany Gyorgy Ligeti's Artikulation.

-- Matt R (email)

Crossing domains analytically

Cleve Moler (author of first MATLAB, a founder of MathWorks, currently Chief Mathematician at MathWorks) profiled the music visualizations of Tim Davis (professor at the University of Florida, a consultant to MathWorks, sparse matrix expert) a couple days ago, here:

ET: What are the most important principles for crossing domains analytically, such as what Moler describes as Davis' "use of sparse matrices to translate music into visual art"? The section titled "Music Visualizations" explains the algorithm Davis uses to "translate." In what ways can such an algorithm be biased to produce certain results? Under the section "Blue Monday" Moler comments, "Tim's rules do not specify 'build a mesh here.' Instead, mesh-like structures arise because of the repetitive bass notes that appear in electronic music." Do the mesh-like structures arise because of the music, or because of the algorithm? How should such algorithms justify the visualizations?


-- Joel (email)

Threads relevant to music:

Privacy Policy