Data-Driven DJ

Lee and Jackson
The Rhythm and Sounds of Two Painters and Their Work

This song analyzes the body of work of two famous abstract expressionist painters, Lee Krasner and Jackson Pollock, who were married for 11 years. The song starts in 1929 with Krasner's early work, and spans 46 years of paintings which were created before and during the artists' marriage and after Pollock's death in 1956. The song's rhythm, notes, and instruments are controlled by the rhythm and color of the paintings by each artist over time.

The Song

Listen to the song by using the player above, or check out the song on Soundcloud if you prefer no visuals or would like to comment on a specific part of the song. Read further down to learn more about how the song was constructed.

A Brief Primer on Lee Krasner & Jackson Pollock

There are entire books written and movies made about Krasner and Pollock, so this introduction will barely scratch the surface of these artists' lives and careers. Lee Krasner (1908-1984) and Jackson Pollock (1912-1956) were highly influential American painters and major figures in post-World War II abstract expressionism, an art movement that challenged the accepted conventions of Western painting in both technique and subject matter.

Krasner and Pollock met in New York City in 1942, coinciding with the rise of the Abstract Expressionists, also known as the New York School of painters, poets, musicians, and performers. They married in 1945, and throughout their relationship, both had an immense effect on each other's artistic styles and careers. Krasner's extensive knowledge of modern art helped Pollock create works that were more organized and cosmopolitan as well as introduced him to many artists, collectors, and critics who appreciated abstract art. Pollock helped Krasner become less restrained, allowing her to become more spontaneous and gestural through her work. However, Pollock's increasing fame led Krasner to struggle with the public's reception of her identity, both as a woman and as the wife of Pollock. In addition, their relationship became strained as Pollock struggled with alcoholism and partook in an extramarital affair. In 1956, Pollock died in a single-car crash while driving under the influence of alcohol.

The intense emotion Krasner felt during this time caused her art to develop into a more liberated form of her self-expression and pushed the boundaries of conventional, developed concepts of art. Krasner suffered from insomnia in the years after Pollock's death and suffered an aneurysm in 1962 where she fell and broke the wrist she paints with. Throughout the 1960s and 1970s, Krasner's work was significantly influenced by postmodern art, and critics began reassessing Krasner's role in the New York School as a painter and critic. Krasner died in 1984 at the age 75.

Composition & Style

The song composition is entirely algorithmic and is composed of the following building blocks:

1. 36 images of paintings by Lee Krasner and Jackson Pollock from 1929 to 1975. These were obtained from various sources including The Museum of Modern Art, The Metropolitan Museum of Art, the Solomon R. Guggenheim Museum, and the National Gallery of Art. This selection of paintings is by no means comprehensive, but representative of the artists' bodies of work as defined by the established art instituations that collected them. I discuss later how I analyzed each painting to produce the song.
2. Synesthesia and the Circle of Fifths: Synesthesia is a perceptual condition where the stimulation of one sense triggers an automatic, involuntary experience in another sense such as hearing colors and seeing sounds. Alexander Scriabin, an early 20th century Russian composer and pianist, was considered to be influenced by synesthesia. He invented a musical instrument called the Clavier à lumières which mapped musical notes to color, where the colors are ordered by the visible spectrum and the notes are ordered by the circle of fifths, a visual representation of the relationships among the 12 tones of the chromatic scale. This color system is one way I map the paintings to musical notes.
3. The New York School: an informal group of American poets, painters, dancers, and musicians active in the 1950s and 1960s in New York City. Prominent composers in this group included John Cage, Morton Feldman, Earle Brown and Christian Wolff. Many consider this group of composers to be the contemporaries of the post–World War II abstract expressionist painters in New York which includes Krasner and Pollock. My song almost exclusively samples from performances of these composers' music. I go into more detail on this below.

How The Data Was Produced

Translating a painting to music can be a very complicated task and can become a rabbit hole very quickly. I decided to keep it as simple as possible and limit the composition to four primary rules:

1. Krasner's paintings control the song's vocals (e.g. ) and violas (e.g. ).
2. Pollock's paintings control the song's percussion (e.g. ) and cellos (e.g. ).
3. Paintings' colors controls the notes of the viola and cello instruments based on the Clavier à lumières (see above). Click on each color below to hear a sample musical note:

   Deep red (F)	Red (C)	Orange (G)	Yellow (D)	Green (A)	Sky blue (E)
   Blue (B)	Bright blue (F#)	Violet (C#)	Lilac (G#)	Flesh (D#)	Rose (A#)

4. The average size of distinct color spaces within the paintings control the quantity and speed of the vocals, percussion, and string instruments. If there are only large spaces of color, there will be fewer and slower instruments. If there are many small spaces of color, there will be more and faster instruments. Here are some examples:

Sounds Used

As mentioned above, I almost exclusively sample from music by the New York School of composers of the 1950s and 1960s who paralleled the abstract expressionist painters of that era.

• 34 percussion samples (e.g. ) from John Cage's Imaginary Landscape No. 2 (March) , Amores , and Living Room Music . Cage was an American composer and one of the leading figures of the post-war avant-garde. He is widely considered as one of the most innovative and influential American composers of the 20th century.
• 4 vocal samples (e.g. ) from Three Voices for Joan La Barbara by Morton Feldman. Feldman was a major figure in 20th-century music and can be considered the abstract expressionist of music. Felman befriended a number of abstract expressionist painters including Jackson Pollock, Mark Rothko, and Franz Kline. Feldman dedicated scores to them, and, he later insisted, they were the ones who most influenced his music.
• 14 vocal samples (e.g. ) from track 15 and 19 of Joan La Barbara's 73 Poems , an homage to e.e. cummings' book of the same name. La Barbara is an American vocalist and composer whose works often involve multiple layers of her own voice, creating a kind of sonic canvas on which she throws splashes of vocal colors. She has performed and recorded works by John Cage and Morton Feldman.
• 6 start-up sound samples from Windows (e.g. ) and Macintosh (e.g. ) computers. John Cage instructs performers to use any household objects or architectural elements as instruments in his composition Living Room Music. In a performance by Third Coast Percussion of Living Room Music in 2012, musicians utilized a personal computer as one of the instruments .
• 72 cello and viola samples (e.g. , ) were taken from the amazing Philharmonia sound sample library, thousands of free, downloadable sound samples specially recorded by Philharmonia Orchestra players. For both instruments, there are 36 samples comprised of 3 different durations (short, medium, long) of each of the 12 notes in the chromatic scale.

Tools & Process

This song was algorithmically generated in that I wrote a computer program that took data and music samples as input and generated the song as output. I did not manually compose any part of this song.

For those interested in replicating, adapting, or extending my process, all of the code and sound files is open source and can be found here. It also contains a comprehensive README to guide you through the setup and configuration. The following is a brief outline of my process:

1. Based on the project's objective, I decided upon a stylistic and compositional approach.
2. I identified and downloaded 36 paintings that were representative of Krasner and Pollock's bodies of work.
3. I compiled a list of major events in Krasner and Pollock's lives.
4. I created a mapping from colors to notes based on Alexander Scriabin's Clavier à lumières
5. Using Processing, a programming language with a visual focus, I analyzed each painting based on:
   1. Dominant colors in the painting, normalized to the 12 colors of the Clavier à lumières
   2. Average area of distinct color spaces
   3. Average brightness of color spaces
6. I extracted individual instrument and vocal samples from relevant music sources.
7. Using Python, a widely used programming language, I:
   1. Assigned instruments to each painting based on the colors in the painting and the average size of color spaces in the painting.
   2. Assigned instruments to each of the major events in Krasner and Pollock's lives.
   3. Generated a sequence of sounds based on the instrument assignments of the previous 2 steps.
8. The sequence of sounds from the previous step was fed into ChucK, a programming language for real-time sound synthesis and music creation. I used ChucK because it is really good at generating strongly-timed sequences. The output would then be an audio file that I could listen to.
9. I then repeated the previous steps numerous times, tweaking the sounds and the algorithms until I was satisfied with the result
10. I used Processing to generate the visualization using the data above.

If you happen to use my code and create something new, please shoot me an email at hello@brianfoo.com. I'd love to see and share your work!

Questions & Feedback

I'd love to hear from you. I'm sure I've also made some erroneous statements somewhere, so please correct me. You can use the widget below or email me at hello@brianfoo.com.