Algorithms Making Music
I’m someone who enjoys listening to music a lot. It’s become an important part of my daily life: I listen to music almost every day while working on my thesis or while doing things around the house. Being that I’m rather curious by nature, I’ve become interested in understanding music, so to speak. For slightly over a year now, I’ve taken a special interest in learning about music. I decided I’d like to combine my love of programming with my interest in music.
I love science, programming and other technical things. I enjoy building things and solving technical puzzles. The question that started to spike my curiosity with respect to music is the following one:
What is it that makes music sound “good”? What makes sound be music?
It’s a superficially simple question with a complicated answer, perhaps more than one answer, really. The most obvious issue with this question is that not everyone has the same musical tastes. Some people prefer musical genres that seem very different, from hip hop to dubstep to classical to jazz. Another issue is that it’s perhaps difficult to even define what music is. Where does something stop being noise and start becoming music? Your grandparents might say that Aphex Twin and dubstep aren’t music.
Perhaps my question partly answers itself. The main appeal I find in music is that some of it is very enjoyable to me. This is probably true for everyone. Music has the power to trigger a strong emotional response within us, that’s where the appeal lies, that’s why movies have soundtracks. Music comes in many varieties, but it’s also generally quite structured. To most people, music has rythm, it has repetition, it has tonal color and texture. As a technically-minded person, this is what interests me most: how do you structure sounds to build music out of them?
I took some piano lessons for a year or two when I was 6-7, but unfortunately, I barely remember anything besides the most basic things. In the interest of improving my musical knowledge, I bought several books about music, synthesizers and sound synthesis. Over the course of a year, by reading a little bit every day on the bus and the subway, and whenever I ended up in a waiting room, I was able to read the following books:
Music Theory for Computer Musicians
Composition for Computer Musicians
The Audio Programming Book
Computer Sound Synthesis for the Electronic Music
Welsh’s Synthesizer Cookbook
Power Tools for Synthesizer Programming
BasicSynth
Learning about synthesizers and sound synthesis was interesting as it gave me some understanding of how to create sounds and give them different textures. I ended up acquiring a microKORG synthesizer and installing synthesis software so I could play with this first-hand. Music theory is more directly related to the question of how to build music. People have been studying what makes music music for centuries. Music theory books don’t provide a magic recipe for songmaking, but they do provide some hints as to what kinds of pitches and musical structures tend to work well together.
I thought it would be interesting to attempt building software that can compose music on its own. If it was possible to infuse enough musical knowledge in a piece of software, it ought to be possible to get something that sounds musical out of it. There are obviously issues when it comes to making software be “creative”, but I still found the concept was interesting. What if you could at least use software to assist you in musical creation? Can’t think of a melody for your song? Generate random melodies and sample them. Use them as a starting point and modify them until you get something you like. This is computer-assisted musical composition.
A year ago, Dimitry Zolotaryov and I created a web application called EvoTune for a Google Hackathon event. The idea was simple: create software that can generate random drum loops using samples and a pattern sequencer. The software has elementary concepts about the structure of drum loops, but it relies on users voting for the best sounding patterns, which are then mated in a genetic algorithm sort of way. We found that EvoTune worked quite well in practice and produced usable drum loops.
I decided to embark on something a little more ambitious and try to create melodies using pitched instruments instead of drum samples. For the past few weeks, I’ve been building code for a virtual synthesizer and sequencer. I also wrote some code to generate chords and musical scales. I have a simple virtual analog synthesizer going that can be used to synthesize piano-like sounds. Below are some samples of my early experiments with procedural melody generation:
Sample 1 (C major)
Sample 2 (C major)
Sample 3 (A minor)
Sample 4 (G minor)
I’m thinking of possibly creating a GUI for my software (which is actually written in JavaScript) and package it as a web application almost as-is. I was thinking some musicians might find it interesting to be able to generate random melodic phrases based on chord progressions.
Whether I do that or not, I’ll probably try to generate more electronic techno/dance sounding melodies using this :)
Pointers Gone Wild 
I was once thinking of doing a project like this as well. Instead of trying to structurally create melodies I was going to do an entirely different approach. I’d just have the computer randomly create a sequence of notes, varying the pitch, duration, and spacing. I’d listen to the sample and rate what I hear, on a scale from total crap to awesome. From here I’d attempt to start filtering and judging the results automatically (what filters would be the question).
If it works then ultimately the computer will be able to start producing melodies that I like without ever having programmed any hard-coded progressions, chords, timings, etc.
I’ve thought of that too ;)
I believe the filter should be trained based on your ratings (e.g.: genetic programming, you evolve a short snippet of code).
You can then use this filter to “evolve” a song by mutating it randomly (e.g.: genetic algorithm). The hope is that the more songs you rate and the more time elapses, the better your filter gets, which also makes your generated songs get better as well.
I have another idea for a filter I could train based on example material instead (songs I like). Might try to implement that.
One issue I’ve always encountered while looking at this is the timbre of the instrument. Quite often certain melodic combinations only sound good if the correct instrument, or instrument “settings” are chosen.
Now obviously you can program the logical settings in the algorithm, but then the problem becomes one of synthesis. Sure, its easy to say you have a C chord on an electric guitar with thick nickel strings, played lightly palm muted, with only bridge pickups, but where would you get a synthesizer to produce that sound (I worked on one once, but had to give up the project).
Because I’m a big fan of electronic music, I’m not particularly interested in trying to re-create real instruments or songs. I’m mostly concerned with the music sounding coherent, consonant, and hopefully to my liking.
Have you ever heard of Markov chains? You can use them to produce melodies that sound “good” because they have similar statistics to human-composed melodies (random google result: http://peabody.sapp.org/class/dmp2/lab/markov1/). I saw a capstone project at my alma mater that did this, and it sounded pretty good.
A similar application is to generate random text that has the same statistics as English. Given a sophisticated enough Markov chain, what comes out is pronouncable, albeit nonsense.
Yes, I’ve heard of Markov chain and n-gram models of language.
I’m actually considering trying to generate a Markov model with hidden states for my next music project. I read about this method in a book recently and thought it sounded very promising.
You may also want to check out these two guides and consider learning a technique called absolute pitch if you haven’t already.
http://www.miles.be/