apple announces its first serious augmented reality product

A few days ago Apple announced the AirPods Pro.  It is largely being described as an AirPod with a noise cancelling feature and a higher price.  It turns out to be much more than that and a couple of features give a few hints: transparency mode and how it does noise cancellation.  I haven't played with them yet. My fun money budget is at zero dollars for the rest of the year.¹  In the meantime there are trustworthy reviews from Om Malik and John Gruber. 

 

Transparency mode is what it sounds like - ambient sounds around you are mixed in with those meant to go to the earbuds.  You get a bad version with poorly sealed headphones/earbuds.  Doing it propertly is tricky and  initial reports indicate Apple has pulled it off.  While most people will use the feature to be aware of their surroundings, it can easily become an augmented reality - specifically an acoustic augmented reality.

 

A variety of visual augmented reality devices and apps have appeared in the past decade.  All have issued and none have cracked the mass market.  iPhones and iPads happy to be visual AR ready.  They mix a live image from the front pointing camera with a computed overlay. The result can be cute and even useful, but mostly it's cumbersome and something you try approximately once.  Google, of course, has had a couple incarnations of their glasses.  There are a number of issues and they appear to have found a few niches in business.  At this point they're not exactly consumer clueful devices.

 

Augmented audio is less intrusive and a natural domain.  Your AirPods Pro could be sending you turn by turn directions as you walk or bike through a city.  Hey Siri is their ready to go interface.  It would be a natural for reading recipes or a thousand other things.  Anything where you want contextual information that is an augmentation of the acoustic soundscape around you.  Then you don't have to switch modes completely and take out a device and/or give up the acoustic soundscape.  It would be dandy for seniors -  hey Siri, where did I leave my glasses?²

 

Speaking of senior citizens AirPod Pros have almost all the features needed to make a very high quality hearing aid.  There is a test that uses feedback to find the best fitting silicone earpiece adapter. It woudn't be a big deal to do it over a range of frequencies for each ear and compute the corrections to sound in transparency mode. It would become an active computational hearing aid.   They could probably even correct for tympanic movement (suggestion by Bryan William Jones).  At the very least these could be the equivalent of non-prescription reading glasses and that could be free assuming you own these.  Apple could work on what it would take to do proper testing and get them certified.  

 

AirPods Pro augmented hearing could mean serious trouble for the conventional hearing aid industry. If you've ever had to deal with hearing aids - your own or for someone you help take care of - you find they cost two to five thousand dollars and they don't work very well. These hold the promise of being able to tune to new acoustic spaces in real time.. something regular hearing aids never have pulled off.  

 

Everyone has been expecting Apple's AR glasses.  They may well be coming, but in the meantime acoustic augmented reality can happen now and Apple could easily extend it into owning a new health market while they're doing it.  There are some other computational tricks to enhance sound for folks with normal hearing, but for now I think these are the two big areas.

 

And finally - to answer some emails from frustrated hearing aid users - they did not announce a hearing aid.  It's just the pieces are there for them.  It wouldn't be much of a stretch and there is frustration over price and quality among hearing aid users. 

___________

¹ When we got married we decided to pool our income except for individual fun money accounts. They aren't big, but the agreement is the other spouse can't complain about what they're used for. One of the best things we've ever done.  Most of my funds support time tithing activities.

² Apple has the tech that allows good location (like 10 cm)  using ultawide band beacons that could be very inexpensive and taped onto a pair of glasses for example.  If you've ever had to help a friend or relative with dementia, this is a killer app.

 

Comments

[Bill V]

I wonder if perhaps Apple Watch's guidance when using Maps would also fall into this category? The combination of taps + small audio cues (distinct ones for left/right/continue/arrived) as a form of semi-AR (as opposed to something more intrusive like CarPlay saying "In one mile, turn right" through the car speakers).