Procedural Music Generator

Melody Thinker

I thought I would make a little blog post explaining my procedural music generator! Maybe get some web traffic! I don't know!

This project has been the result of more than two years of iteration, so I will be going over some of its history as well as the music theory behind it.

Keep in mind, Melody Thinker does not use neural networks or machine learning. Sorry Google, IBM, Wolfram. This is all loops, objects, and hard coding. Nothing you couldn't learn in high school.

HarpGen - March 2017

I originally called this program HarpSeed, because I am not good with names. But the idea was you would type in some random text (the seed) and then a song would pop out. I used the Java library Jfugue, which lets you create music very quickly, without the hassle of setting up MIDI. However, I did eventually need to make my own classes and helper functions.

Music Theory, except not really because I did not know it.

Here you can see a list of notes from the video. On the left column is the bassline, and on the right side is the melody, as JFugue represents it.

Here's the gist of the program. First, a key is chosen. In the example picture, D major is used. Then, each measure, a diatonic chord of D major is played. So D, Em, F#m, G, A, Bm. I avoid the diminished chord. In the video, I actually only play the bass note, because in most cases it sounded better that way. Then a melody uses random notes (D, E, F#, G, A, B, C#). Important to note that there are actually two melodies going on, with a lower accompaniment just not being written down. Eventually this became three melodies, and then down to one in the Unity version.

Let's talk about rhythm. B5h, for example, is the note B, on the 5th octave and the length of a half note. So "[B5h, A6h]" is two half notes, forming one measure. In this version, and actually all versions after it, I would always split the melodies into measures.

As you can see, sometimes two half notes are chosen (h, h); or one half and two quarters (h, q, q) etc. These "rhythm presets" were hard coded, and then randomly chosen between, with equal probability.

Repetition Legitimizes.

Here is the instructions at the bottom of the video. As you can see, the melody is actually copying previous measures. So a "(-2)" symbol means that the melody measure was copied from two measures back. Take a look at how many times "[B5h, A6h]" was reused. Cool, huh?

Repetition legitimizes, gives the song shape. So by reusing previous measures, you get a sense of structure in the song that you wouldn't otherwise. It really improves the melody from "random series of notes" to "discount final fantasy soundtrack".

Repetition and Drums

I briefly tried adding drums, as you can see in the soundcloud link. Skip to about 2:05 for the best one. Drums definitely improve the experience but it can feel like cheating sometimes, because the location of the kick and snare are hard coded. It's just not what I chose to focus on, but I could come back to it later.

Funny thing, this sound snippet is from December 2016, so actually before my first video of the program!

Timeskip three months! May 2017

I'm skipping to the end of the Java version of Harpgen, but I've included three videos that take place in between. I added a note display, which took way too long to make, and naturally was done with g.fillRect();

New Features!

Let's go through this big mess of a user interface one by one.

Copy Measures %: Basically the probability that a measure of melody will copy a measure before it.
Up by one %: The probability that the next note will be "up by one" in the scale. This combined well with another option, which you can see on the right side: "Enforce Chord Matching". So basically, the first note in the melody would be a chord tone, and the one after would be up by one, and would be a non-chord tone, or passing tone.
Band Type: Just a basic change of rhythm instrument.
Four Previous: A drop-down for the copy-method most favored. There was also Two Previous, One Previous, and the default. Each give very different feelings! For example One Previous might be good for rock and pop.
Eighth Notes: Remember how I mentioned "Rhythm Presets"? Well these are like Rhythm Preset Groups. So one for eighth notes, one for quarters and halves, one for wholes and halves, etc.
Minor: Just the key. I eventually realized this drop-down was kind of meaningless, because being minor or major depends on actually starting on a minor or major chord, which hadn't occurred to me. Same applies for chords. So if you're in C major, but always start and end on an F major chord, then the mode being enforced is F Lydian.

Chord Maps: A quick musical detour before switching to Unity.

So I've been skipping the discussion on chords but chords are so, so essential. For example, here is a procedural music generator which doesn't even have melody, it just arpeggiates between chords with different rhythms.

I might argue that a melody is just normal chords (with 1, 3, and 5) alternating with sustained chords (sus2, sus4, sometimes maj7 or m7). I'll use some of this idea in the Unity version, you'll see.

Here's the website for chord maps, but I've included the reference to the right. You start at C, then go to any of the other chords, and follow the arrows down to get a chord progression. Cool, right? It really gets you to thinking that you can't just put in the chords at random.

Towards the end of my use of Java and Jfugue, I started using chord maps. It was so easy to implement, just a number to represent diatonic chords from 1 to 6 (the vii diminished chord is rarely needed) and then a bunch of IF statements based on the previous chord. I even implemented the use of the IV-iv-I progression (The Beatles use it often), and it totally worked! I'm definitely excited to add more modal interchange to my program in the future, as there are six chords in C minor that I could be making use of.

Sample track with IV-iv-I resolution. Uses VST instruments instead of default MIDI, so the overall quality is much better.

Finally, Unity Engine!

Eventually I just got fed up with Java. Here's my reasons.

Java isn't really meant for neat little graphic music apps.
I tried for a long time to find a way to put my music generator online. But, again, it's not what Java was built for. I considered Javascript for a long time, as well as the many confusing Java frameworks, but I kept reaching dead ends.
By using Unity I could potentially deploy my app anywhere. Android, iOS, desktop, even online.
I major in game design at UCSC, so I was already using Unity on a regular basis.
Midi Player Tool Kit on the Unity Asset Store. What a godsend.

Triple-Sets: a Ridiculous Hypothesis on how to make melodies.

Let's take a look at the chorus to Take Me Home, Country Roads by John Denver. Let's also transpose to C major for simplicity, and split it into its four chords. Try humming along as well.

C major. "Country roads". Melody is C, D, E. Chord tone, passing tone, chord tone.
G major. "Take me home". Melody is E, C, D. Passing tone, passing tone, chord tone.
A minor. "To the place". Melody is E, D, C. Chord tone, passing tone, chord tone.
F major. "That I belong". Melody (skipping first note) is E, G, A. Passing tone, passing tone, chord tone.

Now let's make some basic conclusions about how best to use chord tones and passing tones. These can easily be translated into melody rules.

End on a chord tone. Especially true for this song, because the last note of each chord is also the longest.
Have a passing tone in the middle.
G major is somehow "special"? Partially because it is the dominant chord of C, and partially because the melody on G major, "E C D", does not seem to work with the other chords. Try humming "Take me home" on every chord, you will see what I mean. "Take me home" seems to ask a question, even though it ends on a chord tone like the others. Now that I think of it, it's kind of poetic that the lyrics "Take me home" happen on a chord that is far from the tonal center.

Based on these conclusions, which I had figured out myself just this summer, I began to make a list of triple-sets like so, which are hard-coded in:

Normal Sets: CDE, EFG, GAC
"Question" Sets: FED, ECB, GAB

The question sets play on G major, and also E minor because it has similar qualities. The normal sets play on the other chords. That's basically the secret sauce of my program. This system of triple-sets also explains why I haven't implemented the copying of measures yet, even though that was the biggest strength of the Java version.