Biometrics at home – how smart will the smart home get?

Rapid advancements in chip designs along with algorithms for biometric voice and facial recognition are expanding the boundaries of what the Smart Home can do. Super high-resolution video surveillance with the ability to detect if someone has fallen and can’t get up; gesture controls for games and entertainment systems and more accurate speech recognition that does not require the use of ‘wake’ words. These features could be built into low-cost consumer electronics devices, many of which will be battery powered and untethered.

With these new capabilities for personalized interactions between human and machine come questions about balancing privacy with convenience and other benefits. New devices from companies like Amazon and Facebook can do a better job of achieving that balance by leveraging edge computing concepts.

Amazon’s “Halo” effect

The use of biometric data in the Smart Home came to the fore last week with Amazon’s introduction of a new device and health service dubbed Halo.

The Halo Band is a wearable device that at first glance looks like other devices on the market like the Fitbit Sense or Apple Watch. It does not have a display for information, and instead contains multiple sensors including an accelerometer, a temperature sensor, a heart rate monitor, two microphones, (and a button to turn the microphones on or off) among other functions. The Halo Band sends data to the AI-powered Halo service. What does it do? Some functions are similar to other devices, but there are some significant differences in its use of biometric identifiers – voice in particular.

– Activity: Rather than just counting steps, Halo counts the intensity and duration of movement towards points. Sitting too long during the day? Halo will deduct points for your sloth-like lifestyle.

– Sleep: Amazon Halo uses motion, heart rate, and temperature to do continuous measurement that goes beyond time asleep and time awake and offers hypnograms showing time spent in each phase of sleep and skin temperature while sleeping.

– Weight: Using new computer vision and machine learning algorithms, Halo can provide body fat percentage as accurate as methods used by doctors.

– Voice tone: If you remember your mother talking about using that tone of voice, you have an idea of how Halo’s Tone feature works. It is more scientific than your mother, “using machine learning to analyze energy and positivity in a customer’s voice,” offering customers an indication of stress levels on well-being.

(Amazon Halo Band and app. Source: Amazon)

Speculating on Halo’s role in an Amazon Smart Home

To be clear, Halo is currently being offered on a limited basis. But Halo’s biometric data gathering capabilities could play an interesting role in a Smart Home with other voice biometrics and facial recognition technologies. Think of how Alexa smart speakers and an Amazon Prime account could be used for grocery shopping based on dietary needs or preferences suggested by Halo’s health tracking. How about having the groceries automatically shipped from your local Whole Foods (which is owned by Amazon) by a drone or autonomous vehicle? Or, if you prefer shopping in a store, the company is experimenting with cashierless stores that update your bill as you add items into the cart through the use of biometrics or device-based digital ID – no need to scan and checkout.

If Halo detects sub-optimal levels of stress, Alexa could suggest a “Calm” or Meditation playlist on Amazon Music (the company has partnered with Headspace and other providers for the Halo service). Workout time means firing up Amazon Fire TV for an interactive fitness session based on your Halo-suggested fitness regimen.

Biometric data from your Smart Home could also eventually be leveraged by Haven, a healthcare venture created two years ago by Amazon, Berkshire Hathaway, and JPMorgan Chase & Co. With 840,000 employees, Amazon has a significant vested interest in the cost and efficacy of healthcare programs. To that end, the company started a pilot in 2019 for employees and their families in the Seattle region. The virtual health service benefit, called Amazon Care, is supported by Oasis Medical Group. Amazon also recently teamed up with Crossover Health to launch health centers near its fulfillment centers and operations facilities. Again, Halo data could be used to suggest you need to get to a clinic – where a doctor is already viewing your temperature readings for the last week and suggests a flu test.

The Smart Home market is growing fast, and edge computing will help

With companies like Amazon continuing to pour money into market development, the Smart Home market is expected to grow rapidly in the coming years. Revenue in the Smart Home market is projected to reach $2.52 billion in 2020 according to a forecast from Statista. Revenue is expected to show an annual growth rate (CAGR 2020-2025) of 15.8 percent, resulting in a projected market volume of $5.2 billion by 2025. Over 56 percent of households will have at least one smart internet connected device by 2025, and the most common uses for devices will be for security, control and connectivity, followed by entertainment, according to Statista.

Security and data privacy are going to be two key challenges to enabling rapid growth – especially where devices like Halo and their use of biometric data are concerned. That is why developments in edge computing will be essential.

Edge computing in the context of biometric information simply means processing data as close to the consumer as possible. For Smart Home products, the human-machine interface (HMI) is seen as key, and a significant amount of R&D is going in to using voice to identify home users and personalizing experience based on that. Vision and motion detection systems are also being deployed more frequently. These interactions generate significant amounts of data to be processed by AI and ML algorithms.

The less distance that personal information travels, the better, because there are fewer opportunities for the data to be stolen, misused or intercepted. Device performance can potentially benefit as well from not having to rely on data transmission to a centrally located cloud, as is often the case with today’s smart speaker systems, for instance.

The challenge, though, is that leveraging AI on edge devices instead of the cloud is not straightforward, because cost and complexity need to be balanced in cost-sensitive consumer applications.

Synaptics, which provides integrated chip solutions, recently suggested that Smart Home products have some unique requirements to support HMI. Though there are some variation in requirements based on whether the product is a TV or smart speaker or camera, they will be leveraging edge AI to support voice and video processing. Their conclusion: edge AI will be effective in home when the underlying chips and memory systems provide “multimodal” support for voice, vision and video display. In other words, processors need to have integrated support for all modes of HMI in order to hit combined price and performance requirements.

What lies ahead? So far in 2020, there have been a large number of significant advancements in chip technology that suggest that edge devices will continue to see massive leaps in performance. For instance, AI chip designer Kneron announced a new generation of edge AI chip that can hold a dictionary worth of words on the device itself for speech recognition – bypassing the need to leverage cloud services – but also having the ability to be used in augmented reality devices that could last months on a battery charge.

These developments and more suggest that device manufacturers and service providers do not have to sacrifice consumer privacy or device performance to realize fascinating new services built around voice and vision data.

Article Topics

AI  |  AI chips  |  Amazon  |  biometric data  |  biometrics  |  biometrics at the edge  |  consumer electronics  |  data collection  |  edge computing  |  Kneron  |  privacy  |  smart homes  |  Synaptics  |  wearables