Health Lab In The Wild

At the Worldwide Developer Conference (WWDC), Apple revealed its entry into the XR device market with the Apple Vision Pro. The tech giant made a big splash, using its typical fanfare, gorgeous product design, and introduction of new feature breakthroughs to reveal its mixed reality headset.

Undeniably, the features that Apple promises for their revolutionary “Spatial Computer” are extraordinary achievements. These include hands-free computing, advanced entertainment and gaming experiences, responsive and anticipatory performance tracking, fitness applications, and a more customizable and environmentally immersive experience in both image and sound.

But to what end?

Echoing Steve Jobs and Apple’s rallying cry from years ago, let’s pause to

‘Think Different’.

Looking through a future-facing lens at Apple’s latest innovation, we might catch a glimpse of their gradually emerging position in the Health & Wellness sector. Beyond the sleek design and the obvious appeal to XR enthusiasts and spatial spatial computing futurists, Apple has the opportunity to have its own ‘Lab in the Wild’ to study and revolutionize health and wellness technology.

Could there be potential long-term strategies at play for Apple to capture their share of the estimated $11T Global Health marketplace?

Body Real Estate and Biometrics

The concept of the human body as real estate often meets with blank stares. Yet, this perspective can offer fresh insights into the unfolding role of Apple products in health and wellness markets. So, what’s the advantage for brands and products that stake claim to the most valuable locations for accessing to our human OS? Let’s start with Apple’s global deployment of the AirPods as an example.

According to the retail tracking group NDP, by the end of Q3 2017, Apple had already claimed an impressive 85% of the market share of the 900,000 TWS (True Wireless Stereo) earbuds sold that year. By Q4 2022, with 79.5 million TWS headsets shipped industry-wide, Apple held 87% of the market for units in the $200 – $299 price range, the only segment with continued year-on-year growth.

Apart from being an undeniable product success story for Apple, this was a highly strategic ‘real estate’ move. In the digital health and wellness future, the human ear is one of the most valuable pieces of real estate on the body. Sophisticated, sensor-rich earbuds (or ear computers) can enable the capture of more biometric data for personal health management than any location on the body, all while providing a gateway personalized, responsive content. With Apple’s fiercely loyal consumer base, their dominant position (AirPods) in this gateway to the human mind is likely to endure, buying them development time as they blueprint the next game-changing ear computer design.

Meanwhile, the wrist (Apple Watch) has also offered Apple a testing ground for new sensor technologies, user engagement, and integration into their broader hardware and software ecosystem.

Beyond the ears and wrist, the next premium piece of body real estate—one that could give Apple a key market advantage and pave the way to building a trillion-dollar health tech company—is the face.

The right device positioned on the face affords valuable access to the eyes, voice, skin, ears, and body movement—in other words, to high-quality real-time biometric data sources essential for building personalized, closed-loop ecosystems for monitoring and responding to health biomarkers.

Welcome, Apple Vision Pro.

The Lab in the Wild

The introduction of Apple Vision Pro sparks a question: what makes this release different from that of the AirPods Pro and Apple Watch?

Market experts are questioning the scalability. Many fans criticize the $3,500 proposed price tag. As popular as Apple is, the Apple Vision Pro is unlikely to gain the same level of market share that they did with the AirPods, especially against popular XR/AR headsets and devices available for less than ⅓ of the price. However, these concerns assume that Apple is trying to compete in the current XR product market.

The AirPods were scalable because Apple restrained from building them out as next-level ear computers. They first captured the market at scale by focusing on the most in-demand consumer features like Apple’s signature design and craftsmanship, high-quality audio, seamless connectivity, and noise cancellation. Then Apple added Spatial Audio and the Dolby Atmos partnership to the Airpods Pro, making “spatial” a household word and laying the foundation for their emerging immersive roadmap.

That foundation was essential for the development of the Apple Vision Pro and gave Apple Music a differentiator in the overcrowded and devalued world of streaming audio content. Now, with that prime real estate and a near-billion Apple fans eager to be part of the community, Apple has bought themselves time to build the next generation of Ear Computers (aka Hearables).

For the Vision Pro, however, Apple needs only to capture enough of a user base to test their next generation of technology – both for spatial computing and for an Apple Health future. Both of these developments require a mass-consumer study of biometric data capture and personalized testing of responsive technology and content. The Apple Vision Pro is Apple’s Lab in the Wild, and even with only one million users, that would be an extraordinary lab.

For the sake of this article, we will focus on Apple Vision Pro’s potential contribution to a more viable, lucrative and near-field market opportunity than that represented by low-cost spatial computing at scale: Health & Wellness.

History of AR & VR for Health

There have been a number of market and research spikes around AR/VR over the past five decades. Through each of them, countless forward-thinking and well-intentioned ventures and researchers have explored ways to use these tools to improve human health.

Dr. Daniel Kraft, Founder of NextMed.Health and Founding Chair for Neuroscience & Medicine at Singularity University, states, “The state of the art for Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) continues to accelerate. These technologies are playing an increasing role in transforming healthcare delivery and augmenting the skills of healthcare providers.”

Common consumer wellness applications for AR and VR have included: fitness, meditation, and wellness-based training games. More medical and research-based practices have included simulation of complex medical procedures for testing and training, PTSD/Trauma treatment, advanced Telehealth, and remote and augmented operating rooms.

Boston-based health tech company XRHealth has a patented technology for the collection of biometric data, including blood pressure, heart rate, breathing rate, perspiration, and inhalation and exhalation volume, showing how a patient’s body and cognition function during therapy, as well as the patient’s psychological or physiological state. This technology will be used to combine Apple Watch with Apple Vision Pro, extending the sensor capabilities and future features for a biometric monitoring and responsive closed-loop system integrating other Apple products and services.

Since 2016, Los Angeles based AppliedVR has delivered research-validated immersive therapeutics and digital health solutions for chronic pain to thousands of patients. Other immersive audio solution startups, like Spatial, are shifting their focus to health and wellness.

The current spotlight on, investment into, and public awareness of XR and immersive technology was supercharged by Meta. However, with the Vision Pro, Apple has taken the field to the next level, tapping into the accelerating advancement of multiple exponential technologies, from AI and machine learning to nano-tech, big data, sensors, imaging, advanced computer power and microprocessors.

Apple’s bold move into the game may also give the field a boost of hope and restore consumer confidence due to its credibility over Meta when it comes to privacy. Unlike competitors Meta, Amazon, and Google, Apple’s business model is not reliant on selling user data to third parties.

Sensors & the Apple Advantage

From the Lab-in-the-Wild perspective, it is less about what you see from inside the Apple Vision Pro as it is about what the Vision Pro sees in you. Sensors dominate the Vision Pro’s spatial computing abilities. This leads us to the next level of inquiry: Based on all that Apple’s personalized lab can see in you, what can this new supercomputer on your head – and you – learn about your state of health and wellbeing?

The Apple Vision Pro sees inside you via a sophisticated array of audio and visual sensors, all integrated through the new proprietary R1 chip and VisionOS. The sensors include 14 cameras. Four of these are IR cameras inside the device which can track your eyes and perform 3D scans of your iris, providing both health biomarker data and predictive behavior and response, within a closed-loop personalized system.

The IR cameras, capable of working in the dark, also provide eye tracking and a 3D scan of your Iris. The LIDAR Sensor simultaneously tracks your environment, your position in it, and your hand and eye movements. The Vision Pro’s pupil tracking capabilities can predict responses and behavior in real time.

Other sensors that can be helpful for health monitoring are the built-in accelerometers and gyroscopes. For a more complete breakdown of the sensors, see this article by Sarang Sheth of Yanko Design.

Meanwhile Apple continues connecting the health and wellness dots across their ecosystem in other ways. At the WWDC this month, Tim Cook announced the addition of new Apple Health features (currently integrated into the Apple Watch) to the new iPad. In November 2022, they hired AppliedVR’s Chief Product Officer Aaron Robin as their new Health, Wellness and Fitness Lead.

Patents

Apple’s multiple patent filings for the Vision Pro cover the ability to provide responsive content and experiences based on environmental conditions and biomarkers captured by the unit’s sensors. Other parts of their patent address the use of machine learning and signals from the body and brain to predict how focused or relaxed you are, highly useful indicators for monitoring stress, performance, and wellbeing.

Sound & the Amplified Future

One ongoing issue for the past iterations of the VR/AR product and the companies that have developed them has been the lack of sufficient attention to and investment into creating the optimal audio experiences to accompany immersive visual environments. This has held relatively true despite ongoing pleas and evidence from the audio experts, neuroscientists and acousticians, and the consensus that sound can be responsible for more that half of the user experience in virtual environments.

With the advent of the Vision Pro and their commitment to hi-res spatial audio, Apple may be about to change that – hopefully setting a new standard for the industry at large. The Vision Pro has an undisclosed number of microphones to more accurately capture voice and environmental audio data. While more comprehensive spatial audio capture will advance the development of personalized spatial audio, voice is a data goldmine for capturing biomarkers.

In addition to providing a better, and healthier, immersive experience, sound sensors combining advanced voice analytics with environmental conditions have proven to be a robust way to measure mental and emotional health, as well as more diseases and physiological conditions. To provide audio fidelity that could rival the AirPods Max, and integrate environmentally and biometrically responsive content and experiences, the Vision Pro features dual audio drivers built into each side of the headset’s band.

One thing is certain: audio will continue to play a major role in the successful development of advanced digital technologies providing health and wellness solutions to individual users. This couldn’t be more true than in VR & AR experiences, and it is good to see Apple pushing the industry forward in this area with the introduction of the Vision Pro.

Conclusion

Returning to our Lab in the Wild theory and looking through a future-facing health lens, what Apple will learn from capturing real-time biometric data and personalized responses and preferences via the Vision Pro could be the biggest game-changer for Apple and the industry at large. How they use this information will help shape the Amplified Future and potentially redefine how we experience and develop the next generation of content and technology for health and human potential.

The Apple Vision Pro isn’t just a sophisticated piece of tech gear. It’s a comprehensive health monitoring and predictive behavior tool, a futuristic step towards personalized medicine, and an exciting platform for the development of groundbreaking health and wellness solutions. For Apple, and for the broader tech industry, the Vision Pro represents a significant leap forward and a glimpse of the Amplified Future that awaits us.