by on / 8 comments

guitar-test

The quick answer: about 20 milliseconds.

Given that we are lucky enough to have two co-founders – Ryan (shown above) and Fred – that have also trained as professional musicians, we wanted to test for ourselves the amount of delay that could be tolerated by two performers that were trying to ‘groove’ together when separated by Internet-induced delays. This is an important number, because it is easy to imagine how amazing it could be if you could create a virtual world which enabled multiple band members to perform live together when each was not physically on the same stage or in the same studio.

Testing 1,2,3
Attempts and experiments at multi-person live performances have already happened in Second Life and other virtual worlds, but the large and variable delays have generally forced the performers to adopt a serialized strategy where, for example, a bass player might lay down a basic groove which is then streamed to the next musician who adds her own line and then streams that to the next one, etc.

Although this does result in a live performance, the earlier players in the stream are ‘blind’ to the later ones, so they are not truly playing together and responding with subtlety to each other. But the demanding example of musicians playing together and continuously adjusting millisecond-to-millisecond as they watch/hear each other is what we wanted to explore… how much latency could we add before we ‘broke’ their ability to play together, or at least made it discernibly less fun?

To test this, we set up our two experts as performers, separated visually and by enough of a distance that they could not ‘feel’ the floor moving with the music, and wearing high quality headphones connected by a device which allowed us to impose an exact amount of latency (single millisecond resolution) between them. An experimenter would then dial up various amounts of delay, allow them to play for a while, and then record their verbal assessments of how good or bad the experience was for them. By changing roles and trying various instruments, we were able to learn that much below about 10 milliseconds of one-way delay, they typically could not discern the impairment, but that by about 20 milliseconds they would uniformly report the experience as highly undesirable.

Looking Ahead
This outcome is a challenging one to the dream of having multi-performer live Internet jam sessions: to get one way latency down to 10 milliseconds would require very high packet rates – say 250 packets-per-second or more, along with the performers being separated by only modest real-world distances. At the speed of light, 5 milliseconds of delay corresponds to a physical distance of about 1,500 kilometers. Also, Internet routers today still impose a delay that can easily be greater than this, for example the one-way delay from San Francisco to our EC2 machines in San Jose is about 7 milliseconds. Ultimately we will get there, but for now we can expect that our virtual live performers may still need to be in the same room together.  But, of course, this will still make for some pretty amazing potential for live performers to interact with their audience while performing in a virtual world – more on this to come.

Brian on May 3, 2013

Excellent research – I’ve always wondered if it might be possible…

Imagine trans-Atlantic song writing sessions between you and your favourite artist… mind-blowing!

Reply

draxtor™ (@draxtor) on May 4, 2013

Now THAT is the stuff that needs to be done and you guys are doing it :) I am warming up my rusty chops as I type to get ready for the day! Meantime = good ol’ fashioned 20something Rockers from Spain make SL vibrate with awesome sound = http://www.youtube.com/watch?v=2Uus8j4kKw0

Reply

Kake Broek on May 21, 2013

not even half a millisecond so I play acoustic

Reply

Kaoru Sato on January 24, 2014

This is pretty interesting – my bandmate in Deathline has just moved to Finland (we were both based in the UK) so we’re starting to look at ways by which we can successfully continue to collaborate on songwriting and rehearsing.

Latency is obviously an insurmountable barrier to this, though a pure MIDI setup would presumably counter that as MIDI packets are tiny compared to sending streaming audio? We play stringed instruments which are traditionally not great as MIDI control surfaces, but there are software solutions like JAM Origin MIDI guitar.

I’ll follow your experiments with interest.

Reply

Sue Martin on April 7, 2014

We experimented with this on a regular basis at Molaskey’s Pub.

Reply

Add your thoughts

Author

Philip Rosedale
Lifelong entrepreneur and technology innovator. CTO at RealNetworks, founder and CEO Linden Lab, creator of virtual world Second Life. Cofounder High Fidelity, Inc.