Teaching Computers about Music
I recently took an interest in music. More specifically, algorithmic composition, that is, designing computer programs that can compose music. I made some attempts at implementing such programs, based on the idea that it should be possible to come up with a set of rules for generating music that sounds pleasant. What’s pleasant to one person might not be pleasant for everyone, but I thought I could easily conjure rules for generating music that was pleasant to me.
Unfortunately, the more I looked into music theory, the more problematic this became. There are some known, seemingly loose and imprecise, “rules” associated with classical music, but it seems to me these “rules” are in no way sufficient to derive an algorithm. Furthermore, I find that music theory largely fails to account for modern music. Some people might tell you that pop and dance music are trivial once you know about classical music. I would argue that this point of view is arrogant and uninformed. Modern electronic music has a much stronger focus on rhythm and unpitched sounds. It has integrated musical elements from non-european cultures and features instruments and composition techniques that didn’t exist before the 1960s.
A few days ago, I started thinking about ways to flip the algorithmic composition problem on its head. I’m unable, at this point, to write out elaborate rules to compose musical pieces by hand. It seems like an overly complex, tiresome process. I do, however, have my own musical tastes. If you show me a snippet of music, I can tell you whether it sounds musical or dischordant to me. My idea then, is to have an algorithm that composes random musical patterns, and to have a human judge tell the algorithm what sounds best.
I was able to prototype a simple version of this idea in about 6 hours of programming time. The program I implemented essentially “evolves” a simple musical grammar based on the user’s tastes and renders the sounds using a synthesized bass patch. My girlfriend and I both tried teaching the program about our tastes.
Here are some samples from a single run I did last night. The training took about 15 minutes:
Here is one sample from a run supervised by my girlfriend:
The first 3 examples sound similar because they are based on the same grammar derived during one run. The algorithm is capable of generating rather different sounds, however, based on the choices made during a given run and your own personal preferences.
So far, I find the results very encouraging. I think there is clearly potential in the idea that someone can teach an algorithm how to compose based on their own tastes, even if they themselves do not have formal knowledge of music theory. I plan to explore this concept further. I will be trying to generate more complex music that integrates a drum kit and variable-length notes.
Pointers Gone Wild 
Wow, you’re awesome. Yes, music theory is QUITE broad, but if you’re looking for more musical melodic generators…. I would absolutely try to include LONGER notes on the generated rhythms and melodies… half notes, dotted quarter notes, dotted half notes, etc… they help things sound more musical, than these choppy eighth notes (or 16ths, depending on the tempo you used). Vocal phrases in particular need a good combination of short and long, to sound melodic and musical.
Major and minor scales, short and long notes… and phrases that usually only last for 2 measures max (8 beats)… that’s usually the best way to come up with a good melodic motif or phrase.
I’m interested in seeing where you take this. Glad I found your blog.
Here are some examples of songs I challenge your program to generate similar melodies to:
Azure Ray “Sleep”
Angus and Julia Stone “Take You Away”
Band of Horses “Ode To Lrc”
Lightning Seeds “Pure”
Keep us posted!
Thanks for your feedback. I will try to generate more varied patterns in my next attempt. Notes of various lengths, chords, drum and snare. I want to remove alot of the restrictions and have it sound more like a complete piece.
I’m pretty optimistic that I can have an algorithm to develop some “rules of musicality” that work even better than what I have now. The only problem is I can’t actually tell the algorithm what to compose. Based on user input, it will develop its own “taste” for music that aims to resemble the user’s. You can tell it what sounds better to you among what it composed so far.
More original music isn’t necessarily such a huge downside in my opinion though ;)
Oh, and something you may be interested in checking out— a program called Rapid Composer. http://www.musicdevelopments.com/
I actually am one of the beta-testers and “suggesters” of this program… he really covers a lot of great ground in the program, if you’re looking to learn more about music theory and understanding how it works with phrases and chords that fit together and stuff.
The demo is fully functional except saving and exporting…. FYI- it crashes sometimes, but only if you push it really hard. He’s planning a total redesign of the UI for version 2.0 as it can be a bit confusing. Try it out… it’s fun to just mess around with, especially as you learn more music theory. :)
-Chris
This is a fascinating post, thanks for writing it up. I was wondering, would you be willing to share the code you’re developing at all? Applying preference-based fitness selection to algorithmic music generation is something I think about every once in a while, but the honest truth is I’m more of a musician than a coder, so I’m pretty lazy about actually implementing this kind of thing. I’d be curious to see how you’re doing it, though.
Also, perhaps I flatter myself too much with this assumption, but if you’re interested, maybe I could provide some insight or thoughts on, e.g., rules for melodic generation. I studied music composition in college, and I’ve been making electronic music for some ten years now.
(Ah, I see from another blog post you’re reading Virtual Music, I was going to suggest that. Though, Virtual Music is pretty layman-oriented; I think Cope has published at least one more technically-oriented book or article on his algorithmic forays: that might be worth looking into as well.)
Sure, I’d be curious to at least hear suggestions you might have for melody generation rules. I’m more of a coder than a musician ;)
I might make the code available. If I can make something that sounds reasonably good, I might just put it online for people to try (I’m doing all of this in JavaScript with the Web Audio API), and I’m not going to obfuscate the source.
Hmm, well now that I’ve suggested it, I’ll have to do some thinking to actually have something worth saying. :)
One thing that comes to mind is the use of motifs–i.e., small melodic and rhythmic ideas that recur in the course of a piece. I’m not sure if your code specifically provides for that already: it does sound like there’s a bit of that in the sample melodies you posted, but I’m guessing that was a result of the selection process.
Perhaps you could have something of a two-step process, one where you first generate (and perhaps have the user select) motifs, then one where you generate melodies based on those motifs. Here’s a really simple example: take the rhythmic “motif” of repeating a note four times in a beat. (I.e., on each 16th note.) If we assign that to a series of pitches like D, C, A, A, and repeat that we’ll get:
http://www.mediafire.com/?5jsgflw21jeec27
Not terribly exciting, but I think it has a slightly more ordered sound than generating completely random pitches. Another simple example might be a syncopated rhythmic pattern ranging over two beats like this: http://www.mediafire.com/?s8ikyp8s8d5nv24
If we apply that to the same pitch sequence (D,C,A,A) as before, we get:
http://www.mediafire.com/?oe5c0yxamq5srtp
I don’t know what genre of electronic music you like, but I’ve definitely heard that kind of figure in the accompaniment of many techno songs.
Those were just rhythmic examples, of course. A very simple tonal example might be two notes of the same pitch followed by two notes 7 semitones (a perfect 5th) above that. For example, C, C, G, G (http://www.mediafire.com/?966bdc4lcj7qjav). If we transpose that to follow the same old pitch sequence, we’ll get: http://www.mediafire.com/?t204bfvfwsp6l5g
We don’t have to stick to a boring rhythm, though, we can map that onto another rhythm. In fact, we can map it onto the earlier syncopated rhythm I used in that previous eaxmple, like so: http://www.mediafire.com/?34uhvmaa08o4t5u
(Note that, for a lot of music, transposition of a motif should stay within a particular scale, rather than being a precise “chromatic” transposition.)
Obviously many melodies contain segments that don’t relate to any core motif at all. So I don’t think you’d want to generate music ONLY from that kind of pattern. Some element of random variation/deviation could, and probably should, be introduced (and could be selected for, perhaps). Melodies might also include several different motifs that alternate or overlap with each other.
Please, pardon me if I’m simplistically explaining things you already know or, contrarily, if I’m assuming a little too much familiarity with musical terms. I’m not sure what your level of music theory knowledge is. :)
Ack, that second link is wrong, it should be:
http://www.mediafire.com/?vxt370uax1aoegr
It says “comment is awaiting moderation” for one of mine still. Is that the problem?
Maxime, what rules are you assuming in advance?
I’m guessing that you’re assuming a scale, regular beat, fixed timbre, and notes with pitch value unchanging for the duration of the note.
It would be impressive if a trainable system could “learn” those rules from scratch.
And if a trainable system couldn’t learn those rules, is there any reason to suppose that it could learn whatever other rules are necessary to have a complete descriptive theory of music? Those other rules are likely to be harder to discover, given that, as people, we haven’t yet discovered what they are (compared to the rules we know which we have discovered).
Notes are either 16ths or 8ths selected among two octaves. Note choices are biased towards C minor, but all notes are allowed among those two octaves. Silences are allowed as well. What I’m generating is basically a MIDI stream of fixed-velocity notes for a synthesized bass based on a grammar that is learned.
It might be possible to go lower-level. Even to generate programs that generate raw audio waveforms. There was someone, some time ago, who did something like this by generating random one-liners of C code and piping the output into a soundcard (/dev/audio):
The issue is that the more flexibility there is, the more degrees of freedom, the more there is to learn. If my program can directly trigger notes, it only has to “know” something about music theory to make something musical. If the program has to generate a raw audio waveform, it has to implement its own sound synthesis and mixing mechanism inside of itself in order to make music. This is raising the bar quite a bit.
The C one-liners demonstrate that this isn’t impossible, but then those programs aren’t really the result of “training”. I believe it’s basically the result of a brute-force search of (a part of) the space of all expressions. If I ever go this route, I’d like to do it with something better than brute-force.
Rattling nice pattern and good subject matter, hardly anything else we need D. fdekfgbabecf