We Make Awesome Sh

Audio Formats for Gapless Web Playback

A few weeks ago we launched a web app for Madeon. We decided to turn Madeon’s whole debut album, Adventure, into a live mashup concept that allowed fans to mix their way through the album on any device, or even by plugging in their Novation Launchpads and controlling the site and sounds via an external piece of DJ equipment.

The machine handled all the looping, timing and synchronization of playback so that the user didn’t need to keep any form of rhythm or timing, and it would always sound great. To keep things simple we’ll call the end result a "looper".

There were quite a few unknowns we needed to deal with and I’ve decided it best to note down these challenges and how we overcame them. I’m going to put out a series of posts about the three main challenges, audio formats, loop syncing, and Web MIDI. This first post is about the audio formats to use…

So to start with, when we first made the looper we wanted to make sure it had progressive enhancement. We knew that the new(ish) WebAudio API would help with the playback, but we wanted to make sure that browsers that did not support it still operated. So we began by just using the HTML5 <audio> element.

Everything supports mp3, right?

Naïvely, I presumed that since all of the latest browsers (Safari, Firefox, Chrome, IE) supported mp3, the obvious choice would be to use mp3.

One thing to state early on is that our main test browsers were Chrome and Safari.

So to test looping, we had a simple <audio> element with the loop attribute.

See the Pen QbjErL by Syd Lawrence (@sydlawrence) on CodePen.

One thing that was relatively clear straight away was that there was quite an obvious gap between looping in both Safari and Chrome as you can see with this demo.

So we decided to try out different audio encodings.

Safari Chrome Firefox IE
mp3 y y y y
ogg n y y n
m4a y y n y
wav y y y y
Browser audio format support using latest stable browsers at time of writing.

Straight away we switched to wav, knowing that both browsers supported this. wav performance was far better than mp3, however, as we all know, wav files are huge in size, so we wanted to avoid any excess file size, but we now had a decent fallback.

While testing on safari we tried all 4 audio encodings, and rather entertainingly in our tests m4a performed much better than mp3, and weirdly even better than aac. Now why is that weird you ask? Well m4a and aac are identical. The only difference is the file extension… To test this, we even just renamed the test file and there was a noticeable difference. aac however, still performed better than mp3, in fact, mp3 was possibly the worst file format to use for gapless looping on any browser.

Supporting Multiple Audio Formats

So we now understand that for Safari m4a is the best extension to use… However the bad news with this was that Chrome and others do not support m4a… So we had to work on the best file format for Chrome. For Chrome it was a clear winner, ogg worked the best. But ogg isn’t supported on Safari, but it is supported by Firefox!

So now we knew for sure we had to use multiple file formats depending on the browser that the user was using.

function canPlayAudio(ext) {  
  var a = document.createElement('audio');
  return ( !! (a.canPlayType && a.canPlayType('audio/' + ext + ';').replace(/no/, '')));
};

function getAudioExtension() {  
  var extension = 'wav';
  if (canPlayAudio('ogg')) {
    extension = 'ogg';
  }
  else if (canPlayAudio('mp4')) {
    extension = 'm4a';
  }
  else if (canPlayAudio('aac')) {
    extension = 'aac';
  }
  else if (canPlayAudio('mp3')) {
    extension = 'mp3';
  }
  return extension;
};

We now had the playback sorted, but what about the encoding… Encoding multiple file times is a pain, especially when the files might change…

Automating Audio Conversion

We use grunt here for most of our projects, and Hugo (Madeon) was providing us with wav files of the original samples. So we quickly got to work on a simple grunt task to convert the original wav files into ogg, m4a, aac and mp3.

For this we used a mix of lame and ffmpeg. If you don't know about them already, lame is a really simple mp3 command line tool, and ffmpeg is my personal go to media conversion suite. Although I believe ffmpeg supports mp3, I prefer lame for mp3s.

Using a mix of lame and ffmpeg made it relatively easy:

shell: {  
  m4a: {
    command: 'for f in audio/*.wav; do echo "Converting $f to m4a" && g=`basename $f .wav`&& ' +
      'ffmpeg -i audio/$g.wav -c:a libfdk_aac -b:a 128k app/assets/audio/$g.m4a;' +
    ' done'
  },
  aac: {
    command: 'for f in audio/*.wav; do echo "Converting $f to aac" && g=`basename $f .wav`&& ' +
      'ffmpeg -i audio/$g.wav -c:a libfdk_aac -b:a 128k app/assets/audio/$g.aac;' +
    ' done'
  },
  wav: {
    command: 'for f in audio/*.wav; do echo "Copying $f" && g=`basename $f .wav`&& ' +
      'cp audio/$g.wav app/assets/audio/$g.wav ;' +
    ' done'
  },
  mp3: {
    command: 'for f in audio/*.wav; do echo "Converting $f to mp3" && g=`basename $f .wav`&& ' +
      'lame -b 128 -h —nogap audio/$g.wav audio/$g.wav && mv audio/$g.mp3 app/assets/audio/; ' +
    ' done'
  },
  ogg: {
    command: 'for f in audio/*.wav; do echo "Converting $f to ogg" && g=`basename $f .wav`&& ' +
      'ffmpeg -i audio/$g.wav -c:a libvorbis -q:a 4 app/assets/audio/$g.ogg ;' +
    ' done'
  },
}

So there we have it, we now had the correct file formats to use for the correct browser to make sure there was gapless looping, when using the <audio> element.

Look out for our next posts on the synced playback, and how we used the new experimental WebMIDI API.

Recent Articles

comments powered by Disqus