JavaScript Audio Synthesis Part 3: Making Music

Oct 01 2013 Published by under JavaScript

Send to Kindle

First of all, I am not a musician by any stretch of the imagination. But that fact will become obvious all too soon. But if you’re going to make sound with code, you wind up either making sound effects or music. So let’s start with music. My goal here was to create a simple “tracker” application. You program in notes, it plays those notes back in progression to form a song. Of sorts.

I’m going to keep the song very simple. Mary Had a Little Lamb. It’s a song that you can play every note with one beat, so you don’t have to mess around with different length notes. Here’s a simple version of the song transcribed into notes and rests:

g f e f
g g g -
f f f -
g b b -
g f e f
g g g g
f f g f
e – - -

A quick search on the ‘net gives you the frequency values for the notes b, e, f and g, the only ones you’ll need for this song. Code that into an object:

scale = {
	g: 392,
	f: 349.23,
	e: 329.63,
	b: 493.88
}

And then you can just code the song as a string.

song = "gfefgg-fff-gbb-gfefggggffgfe---";

Now you can create an AudioContext and an oscillator and set an interval that runs at a certain speed. In the interval callback, get the next note, find its frequency and set the oscillator’s frequency to that value. Like this:

window.onload = function() {

	var audio = new window.webkitAudioContext(),
		osc = audio.createOscillator(),
		position = 0,
		scale = {
			g: 392,
			f: 349.23,
			e: 329.63,
			b: 493.88
		},
		song = "gfefgg-fff-gbb-gfefggggffgfe---";

		osc.connect(audio.destination);
		osc.start(0);

	setInterval(play, 1000 / 4);

	function play() {
		var note = song.charAt(position),
			freq = scale[note];
		position += 1;
		if(position >= song.length) {
			position = 0;
		}
		if(freq) {
			osc.frequency.value = freq;
		}
	}
};

Now this actually works and you should be able to recognize the melody somewhat. But it leaves a lot to be desired. The biggest thing is that there is no separation between notes. You have a single oscillator running and you’re just changing its frequency. This ends up creating a sort of slide between notes rather than distinct notes. And when there’s a rest, well, there is no rest. It just keeps playing the last note.

There are various ways to try to handle this. One would be to call stop() on the oscillator, then change its frequency, then call start() again. But, when you read the documentation, it turns out that start and stop are one time operations on an oscillator. Once you call stop, it’s done. That particular oscillator cannot be restarted. So what to do?

The suggested answer is actually to create a new oscillator for each note. Initially, this sounds like a horrible idea. Create and destroy a new object for every single note in the song??? Well, it turns out that it’s not so bad. There are some frameworks that create a sort of object pool of notes in the background and reuse them. But the downside to that is that every note you create and start continues playing even if you can’t hear it. It’s your choice, and I suppose you could do all sorts of profiling to see which is more performant. But for Mary Had a Little Lamb, I think you’ll be safe to create a new oscillator each time.

To do this, make a new function called createOscillator. This will create an oscillator, specify its frequency and start it. After a given time, it will stop and disconnect that oscillator. You can then get rid of the main osc variable in the code and call the createOscillator function when you want to play a note.

window.onload = function() {

	var audio = new window.webkitAudioContext(),
		position = 0,
		scale = {
			g: 392,
			f: 349.23,
			e: 329.63,
			b: 493.88
		},
		song = "gfefgg-fff-gbb-gfefggggffgfe---";

	setInterval(play, 1000 / 4);

	function createOscillator(freq) {
		var osc = audio.createOscillator();

		osc.frequency.value = freq;
		osc.type = "square";
		osc.connect(audio.destination);
		osc.start(0);
		
		setTimeout(function() {
			osc.stop(0);
			osc.disconnect(audio.destination);
		}, 1000 / 4)
	}


	function play() {
		var note = song.charAt(position),
			freq = scale[note];
		position += 1;
		if(position >= song.length) {
			position = 0;
		}
		if(freq) {
	 		createOscillator(freq);
		}
	}
};

This sounds better already. Each note is distinct, and when there is a rest, no note plays. But you can do even better.

The notes are just flat tones at this point. You can improve this by giving them a quick and dirty sound envelope. In real life, a sound doesn’t usually just start at full volume and cut out to silence when its time is up. The volume usually ramps up a bit first (the attack) and fades out quickly or slowly (the decay). Actually, sound envelopes can be much more complex than this (http://en.wikipedia.org/wiki/ADSR_envelope#ADSR_envelope) but a simple attack and decay to 0 will do fine for now and give the notes a much better sound.

As a sound envelope controls the volume of a sound, you’ll need to create a gain node. Do this inside the createOscillator function:

	function createOscillator(freq) {
		var attack = 10,
			decay = 250,
			gain = audio.createGain(),
			osc = audio.createOscillator();
...

The attack and decay values there are in milliseconds. This means that the sound will ramp up from 0 to full volume in 0.01 seconds and then back down to 0 in .25 seconds.

Next, the gain node will need to go between the oscillator’s output and the destination in order to control the volume of that oscillator.

		gain.connect(audio.destination);
		gain.gain.setValueAtTime(0, audio.currentTime);
		gain.gain.linearRampToValueAtTime(1, audio.currentTime + attack / 1000);
		gain.gain.linearRampToValueAtTime(0, audio.currentTime + decay / 1000);
...

First you set the gain to 0, which in essence sets the volume to 0. You do this with the setValueAtTime method, passing in the current time from the AudioContext object. In other words, set the volume to 0, NOW.

Then you use linearRampToValueAtTime to set the attack and decay. This says go from whatever value you are currently at and interpolate to this new value so that you arrive there at the specified time. Note that the values you pass in here are in seconds, so you’ll need to divide by 1000 when using millisecond values.

Finally, you connect the oscillator to the gain, set the frequency and start it. And of course, when you clean up, you’ll need to disconnect everything as well.

		osc.frequency.value = freq;
		osc.type = "square";
		osc.connect(gain);
		osc.start(0);
		
		setTimeout(function() {
			osc.stop(0);
			osc.disconnect(gain);
			gain.disconnect(audio.destination);
		}, decay)
}

The final code is below:

window.onload = function() {

	var audio = new window.webkitAudioContext(),
		position = 0,
		scale = {
			g: 392,
			f: 349.23,
			e: 329.63,
			b: 493.88
		},
		song = "gfefgg-fff-gbb-gfefggggffgfe---";

	setInterval(play, 1000 / 4);

	function createOscillator(freq) {
		var attack = 10,
			decay = 250,
			gain = audio.createGain(),
			osc = audio.createOscillator();

		gain.connect(audio.destination);
		gain.gain.setValueAtTime(0, audio.currentTime);
		gain.gain.linearRampToValueAtTime(1, audio.currentTime + attack / 1000);
		gain.gain.linearRampToValueAtTime(0, audio.currentTime + decay / 1000);

		osc.frequency.value = freq;
		osc.type = "square";
		osc.connect(gain);
		osc.start(0);
		
		setTimeout(function() {
			osc.stop(0);
			osc.disconnect(gain);
			gain.disconnect(audio.destination);
		}, decay)
	}

	function play() {
		var note = song.charAt(position),
			freq = scale[note];
		position += 1;
		if(position >= song.length) {
			position = 0;
		}
		if(freq) {
	 		createOscillator(freq);
		}
	}
};

Now the notes have a more bell-like sound. It’s not awesome, but it’s a whole lot better. Mess around with trying to create different envelopes to see how that changes the sound. Code in your own song strings too. Don’t forget to add any additional note frequencies that you might use. You might even want to do something with allowing more than one-beat notes, though that starts to get a bit more complex.

Anyway, this is pretty rough and dirty code. A whole lot that could be improved upon here, but it’s good for demonstration purposes and hopefully gives you a better understanding of the underlying fundamentals than a more complex structure would. Have fun with it.

Send to Kindle

3 responses so far. Comments will be closed after post is one year old.

  • Ben Renow-Clarke says:

    Hi Keith – In the original example, you can create false pauses by adding a note with an oscillation value of 1 then inserting that between notes. This gets around having to create and destroy notes every time. (NB: it’s also inelegant and sounds like nursery rhymes at a rave.

    scale = {
    g: 392,
    f: 349.23,
    e: 329.63,
    b: 493.88,
    o: 1
    },
    song = “gofoeofogogo-ofofofo-ogobobo-ogofoeofogogogogofofogofoeo-o-o-o”;

    • keith says:

      Ben, that’s almost exactly what I was initially doing. I was using 0 not 1. :)

      I also discovered some other things since writing this.

      1. Doing osc.frequency.value = x seems to create a “slide” from the current frequency to the new one you set. It seems to go as fast as it can, but it still slides.

      2. If, instead, you say osc.frequency.setValueAtTime(x, audio.currentTime) you get a perfectly crisp transition.

      Doing #2 along with a decent gain envelope lets you do it all with a single oscillator and still sound pretty good. Prepping another post.

  • Ben Renow-Clarke says:

    Is there a way to play notes through separate channels to create chords? It comes out as a complete mess when I try. (A very interesting sounding mess, but a mess nonetheless.)

    All this messing about made me dig out my old BBC BASIC manual. The envelope command there was crazy…

    The statement ENVELOPE is followed by 14 numbers and the following labels will be used for the 14 parameters.

    ENVELOPE N, T, PI1, PI2, PI3, PN1, PN2, PN3, AA, AD, AS, AR, ALA, ALD

    A brief description of each parameter follows

    Parameter / Range / Function

    N / 1 to 4 / Envelope number

    T bits 0-6 / 1 to 127 / length of each step in hundredths of a second

    T bit 7 / 0 or 1 / bit 7 is 0 = auto repeat pitch envelope, 1 = don’t repeat pitch envelope

    PI1 / -128 to 127 / change of pitch per step in section 1

    PI2 / -128 to 127 / change at pitch per step in section 2

    PI3 / -128 to 127 / change of pitch per step in section 3

    PN1 / 0 to 255 / Number of steps in section l

    PN2 / 0 to 255 / Number of steps in section 2

    PN3 / 0 to 255 / Number of steps in section 3

    AA / -127 to 127 / Change of amplitude per step during attack phase

    AD / -127 to 127 / Change of amplitude per step during decay phase

    AS / -127 to 0 / Change of amplitude per step during sustain phase

    AR / -127 to 0 / Change of amplitude per step during release phase

    ALA / 0 to 126 / Target level at end of attack phase

    ALD / 0 to 126 / Target level at end of decay phase

Leave a Reply