I enjoyed your seminar today in Austin, and wanted to hear your thoughts on the similarities of the Music Animation Machine and player piano rolls (or equally similar, music box cylinders). I was struck while watching the video how much it reminded me of watching the player piano at Shakey's Pizza when I was a child and noticing that I could "see" the music as it was being played. I get the same effect watching a music box play, when the "guts" are visible.
The wonderful Music Animation Machine by Stephen Malinowski (a composer and computer programmer) is a 44-minute videotape displaying the note-by-note structure and timing of pieces by Scarlatti, Byrd, Bach (3), Beethoven, Chopin (2), Kasper, Webern, and Malinowski (2). In my one-day course, people see parts of a Scarlatti sonata for harpsichord and Chopin's Berceuse, opus 57, a lullaby. The Chopin is powerful and elegant; I have seen it several hundred times and often still see something new or different (just as I see new things in each viewing of Robert Altman's movie, Nashville).
The Music Animation Machine is a joy, deepening our understanding of music and intensifying our listening. It makes the eyes as well as the ears listen. The information in MAM (duration, overlap, pitch) is already available to those who can read music notation as they listen to music; MAM makes it available to the rest of us.
Recently a second video (38 minutes) became available, with pieces by Satie, Bach, Chopin (2), Liszt, Tchaikovsky (2), and Rimsky-Korsakov.
The animation of Stravinsky's Rite of Spring is truly impressive:
Inexpensive videotapes can be directly ordered from Stephen Malinowski, on-line or by mail from Music Animation Machine, P.O. Box 13622, Berkeley, CA 94712.
Under the FAQ's at musanim.com you can download the software for free. There is also a good discussion of various methods for visually characterizing music.
Some time ago, Neil Mussett asked in this thread, "Do you have any comments on musical notation itself? It seems to be an area rarely questioned (I get the impression that even the typography is duplicated from book to book)."
I'm just getting into a major project now on music visualization techniques, so I thought I'd share some thoughts on this here. The notational system for western music provides an elegant and efficient system for providing directions for reproducing a musical work. Of course, this system isn't really a "system," but an ever-changing set of conventional practices that evolves over time to keep up with the changing needs of its users. Some features have come and gone either because the musical needs changed or because they didn't work well or were impractical, but the basic elements of our notational system are now over a millennium old. The following brief history oversimplifies things a great deal.
The origins of Western music notation lie in the medieval monasteries. For a long time the music for plainchant was passed down through oral tradition. The ninth century saw the development of "neumes," which were basically squiggles that represented the shape of the melody and served a memory aid. Over the next couple of centuries, these neumes were aligned to a vertical (but undrawn) grid (making them essentially low-res Sparklines), aligned to a single reference line, and by the 11th century, aligned to a four-line staff, with letter-clefs used to identify the name of one of the lines. The five-line staff we are accustomed to now appeared by the Renaissance, as the range of musical parts expanded beyond that of the basic chant repertoire. What we think of as modern rhythmic notation wasn't fully developed until the 17th century.
The notational system reads in the same way as the prose of European languages--left-to-right, top-to-bottom--and is thus as easy to scan as text. The five-line staff for denoting pitch is probably perceptually optimal for the kind of rapid processing needed to read music "in time." I suspect people would be noticeably slower decoding the location of notes on a six-line staff, for instance. Because they are visually distinct from the staff, however, notes on up to three ledger lines can be read with virtually the same fluency as on the staff. The diatonic basis of the notational system (notes on the staff represent diatonic pitches unless made chromatic with an accidental) maps perfectly onto the physical layout of the piano and other keyboards (white vs. black notes) and woodwind instruments (whose "default" fingering layout also follows the diatonic scale).
The rhythmic notation based on shapes (open and closed noteheads, stams, flags, and beams) and not on horizontal spacing acknowledges the fact that we are not good as measuring precise differences in spatial distances and the hierarchical relationship among the durations is easily learned. Though it is conventional to give more horizontal space to notes of longer duration, and doing so does in fact help in mental processing of the notation, this is not done proportionally. The use of beams to gather shorter notes into beat-length groupings also draws on our innate cognitive abilities. Notation to indicate articulation and phrasing (e.g., slurs, staccato, tenuto) are effective pictograms that depict their function. Conventions for indicating fingerings, pedaling (for piano music), dynamics, tempo, etc., are easily distinguished from one another and rarely ambiguous.
Musical experimentation in the 20th-century has led to many new notational features (microtonal tuning, un-measured rhythms, constrained improvisation, new performance techniques, etc.). It should be noted, incidentally, that there have been numerous proposals over times for new, usually simplified, systems for music notation, but these invariably fail for one reason or another. Part of my future research will be to try to determine reasons, beyond simply entrenched tradition, why this might be.
I know relatively little about the art of music engraving and typesetting, incidentally, except to note that it is very much an art. Certain publishers have house styles, I believe there have been some "classic" approaches, which would account for some of the consistency that Neil observed. And for modern computer-set music, there is a very small number of programs in use, which also encourages a degree of uniformity.
Look at visualisation which very close to pictures:
I also very much enjoyed your presentation (December 11, 2002, San Francisco) of the Chopin and Scarlatti animations.
Although I'm a webmaster by trade, I was trained in classical music and have my masters' in performance from NEC in Boston and have played the double bass since childhood. I have always beleived very strongly that there is a certain visual beauty to the architecture of music. (Had that notion since I saw Disney's "Fantasia" as a kid...)
What was shown on the screen today really is not that different from the representations of traditional music notation. In fact, I think that such an animation would be extremely cool and just as effective by adding an additional dimension of the actual notation to the movement - so the colors would remain, but then you'd have the staff, key signatures, notes and accents and ligatures to go along with it. I love to observe such patterns and movements when reading a piece like the Chopin you showed us today or, even more intricately, when reading through an orchestral score - especially when the orchestration itself is part of the beauty, such as Bach Brandenburg #3, Tchaikovsky Symphony #4 1st and 3rd mvts., Strauss' Alpine Symphony, and Mahler Symphony #2 1st mvt. just to name a few. Great stuff!
-- Joe Lewis, December 12, 2002.
Under the FAQ's he has a link to downloading the software. It is free
-- Glenn Nevill, December 04, 2001.
There are now a couple of QuickTime short demos of various pieces off of this site.
-- Pat Boule, July 11, 2005.
a more tweakable method is to use a wav to midi converter, such as amazing midi, and then import into whatever software eg. cubase.
on another note:
i would appreciate any constructive critisim on a similar project: http://smallbrainrecords.com/text/animatedscore.html thank you
-- jim ryan, September 25, 2005.
If you're running Windows, you can view a scrolling bar-graph score while playing MIDI files with this freeware MIDI file player.
I was fortunate to have been able to attend one of your seminars(The boston seminars in
Fall 2008). It was quite a learning experience. I was particularly impressed by the
Musical Animation Machine in your presentation.
Anyway I came across this wonderful piece of free software that seems based on the
visualization principles of the Musical Animation Machine and takes it even a step
further. I wanted to share the link with you and this forum.
When I watched the animation machine at your recent course in Atlanta, GA, I thought of how sometimes
dance is a similar image of musical data. I was also reminded of a juggling performance by
Chris Bliss, in which his juggling balls allow direct visualization of the music. Here is one of many possible links to his
Re: My previous post about rotating the animation to increase its explanatory power. I did not realize that you also used a Scarlatti example. For Scarlatti, depending on whether you were explaining a performance on a single manual or double manual harpisichord, you would need more than one vertical scroll like the sparklines for multiple exchange rates. I would stack them vertically. You would also need to indicate active stops and manual couplings. I apologize for the multiple posts.
the link in the blog post below reminded me of the MAM. I use the MAM to illustrate to my
intro. histology students how with practice at the microscope they will begin to use their
"eyes and brains" at the same time
Robotics scientists Jeff Lieberman an Dan Paluska created a kinetic sculpture that
performed a full size marimba by propelling small rubber balls at the bars arching several
feet into the air. The effect was similar to everything described here, you could see the
music before it happened - flying through the air in the gallery. Because of the arches, it
almost felt like time slowed down for an instance before the chords hit as you imagined what
was to come. There was also a percussion battery and glass harmonium. Of added interest
was that the music was created by combining a melody from an online "audience" member and a
purely musical form (with no melody, only variations) created by a composer. It was very
satisfying composing for something when you have little knowledge of the end result!
Cleve Moler (author of first MATLAB, a founder of MathWorks, currently Chief Mathematician
at MathWorks) profiled the music visualizations of Tim Davis (professor at the University
of Florida, a consultant to MathWorks, sparse matrix expert) a couple days ago, here:
ET: What are the most important principles for crossing domains analytically, such as what
Moler describes as Davis' "use of sparse matrices to translate music into visual art"? The
section titled "Music Visualizations" explains the algorithm Davis uses to "translate." In
what ways can such an algorithm be biased to produce certain results? Under the section
"Blue Monday" Moler comments, "Tim's rules do not specify 'build a mesh here.' Instead,
mesh-like structures arise because of the repetitive bass notes that appear in electronic
music." Do the mesh-like structures arise because of the music, or because of the
algorithm? How should such algorithms justify the visualizations?