- I attended a mixed-reality concert featuring a ghostly performance by the late Ryuichi Sakamoto.
- It had some technical hiccups that broke full immersion.
- But it shows the huge opportunity Apple has with AR.
Ryuichi Sakamoto was a pioneering Japanese composer. He died in March at the age of 71.
Three months later, I saw him deliver a 50-minute piano concert at The Shed arts center in New York City. Or at least, I saw Sakamoto’s digital avatar perform in a darkened room with a Magic Leap 2 headset clamped to my face.
The show, called KAGAMI, was created by a production studio called Tin Drum, and is marketed as a “mixed reality” concert. You could call it augmented reality, too.
Some technical issues aside, the experience showed me the potential that AR has to transform the entertainment industry. And it reveals why Apple and Meta are investing so heavily to assert themselves in the space. Microsoft and Google seemed to have stepped back from this technology recently, but they probably won’t give up.
Before the show started, we sat in a circular auditorium. An announcement gently warned people that recording anything with a phone through the headset’s lens wouldn’t work: “We won’t freak out if you try, but it’s probably not worth it.”
Because I wear glasses, I was given a headset with prescription lenses. I snapped it onto my face and wore its accompanying computing unit like a necklace.
As the lights dimmed, the audience became ghostly silhouettes, and Sakamoto’s avatar appeared center stage. He played “Before Long” in a cloud of virtual mist. At first, people watched demurely from their seats along the perimeter, but by the second song, a few started walking around Sakamoto, emboldening the rest of us to follow.
At one point, as Sakamoto went through his repertoire, digital snow drifted in from a window high above. During another song, a fake tree grew and extended its branches beneath the piano. Later, a galaxy reeled beneath our feet. I was transported.
Then, unfortunately, a fan in the Magic Leap computer around my neck growled to life, interrupting my digital reverie. Other distractions followed. The outline of a fellow audience member suddenly appeared inches away, a reminder of the physical world’s existence, and to please not bump into anyone.
The AR effects had a low-res graininess, and were limited to a small square directly in my line of sight. I couldn’t see anything in my periphery unless I turned my head and looked directly at it. And viewed up close, Sakamoto’s avatar slipped into the “uncanny valley,” that discomfiting space robots and digital avatars occupy when they’re not quite real enough to pass as human.
KAGAMI still showed a lot of promise for the future of AR entertainment. Thanks to the headset, participants don’t have the encumbrances that often undermine full immersion. In 2022, for instance, Elton John brought an AR element to his Farewell Yellow Brick Road Tour, but audience members had to hold up their phones and view the concert through an app.
If Apple’s Vision Pro succeeds, these experiences can be piped directly to someone’s living room, downloadable through an app like the Elton John activation, but offering fuller immersion like KAGAMI. If that happens, it could change the entertainment business.
As Disney CEO Bob Iger said, the promise of an AR device like the Vision Pro is that you could feel like you’re watching a sports match from the sidelines, rather than back in the bleachers where traditional TV often places you.
This could open up new pricing structures as fans pay a premium to be on stage as a pop star performs. Artists might sell tickets to performances that happened long ago, or license their likeness to make new ones without having to lift a finger. Scripted entertainment could have new interactive elements that combine video games and movies even more than they are today.
The tech company that builds the best AR hardware and software has a shot at controlling the next generation of entertainment. All the current tech giants know this because their existing digital platforms and operating systems control the business of online content. Apple got this new race started properly in June. I can’t wait to see who wins.
Read the full article here