Kayden Bowen is a big-smiling 12-year-old boy who loves music and fast cars. He can’t walk or talk, is legally blind and can only use the little finger on his left hand.
Kayden has cerebral palsy, and his father, James Bowen, has been experimenting with new technology to create tools that will give his son more independence. In one such experiment, James set up an Amazon Echo Show, a smart assistant device that uses AI to enhance voice control. Kayden already has a voice device that lets him press a button with his mobile finger to speak preselected words and phrases to the Echo.
James set it up so his son could ask things like, “Echo, tell me a joke” and “Echo, tell me the news.” He also programmed it so Kayden could call his cell phone, his grandparents and a few other people.
Twelve-year-old Kayden Bowen, who has cerebral palsy and is non-verbal, communicates with an Amazon Echo using a voice device. His father, James, says he’s “proud of Kayden’s journey and grateful for the technology that has made it possible.” Photo courtesy of the Bowen family.
But Kayden did more than that. One evening, while James and his wife were out and home with the babysitter, Kayden used his voice device to activate his Echo, instructing it to call his grandfather and asking him to get in the car and go on a trip, which his grandfather happily obliged. James, who shared the story in a Facebook post, said he was proud of Kayden’s efforts and grateful for the technology that made it possible.
“This was the first time he’d been able to interact with something on his own since he was a toddler playing with rattlesnakes,” James said. “This Echo device does a lot more than just order groceries or look up recipes.”
Necessity is the mother of invention
From electric scooters to hearing aids, technology has long opened doors for people with disabilities, and in the coming years, AI will begin to accelerate these efforts with new capabilities and expanded access. With more than 1 billion people with disabilities worldwide, there is a lot of work to do and a huge market to address.
“They’re our customers, they’re our friends, they’re everyone,” says Jenny Rae Fleury, Microsoft’s chief accessibility officer, who is deaf herself.
Whether related to vision, hearing, mental health, learning, cognition or movement, disabilities can be permanent, temporary or situational. By designing new products with different levels of ability in mind (a concept known as inclusive design), great progress has been made in ensuring technology works for everyone.
Making designs widely accessible also benefits people without disabilities: one of the first typewriters was born from its creator’s desire to help a blind friend (or lover) write more legibly; Alexander Graham Bell’s mother was deaf, and the invention of the telephone came from his work with the deaf community.
More recently, audiobooks were created as a way for the blind to enjoy literature. Video captions were invented to make content compliant with the Americans with Disabilities Act (ADA), but are now regularly relied upon by anyone who watches video who cannot hear the audio.
Developing new tools can help integrate demographics that are often left out of daily living activities and job opportunities. According to the Bureau of Labor Statistics, Americans with disabilities have twice the unemployment rate. Worldwide, only one in 10 people who can use assistive products have access to them.
Artificial intelligence hasn’t been around long enough to address every opportunity, but Microsoft wants to accelerate innovation with $25 million in funding in a program it announced earlier this year called “AI for Accessibility.” The goals include developing more AI in-house and offering grants to people who want to develop tools for the disability community. When it comes to AI, Ray Fleury says technologies he thought were five to 10 years away are now one to three years away.
“Honestly, this is very exciting for people with disabilities,” she says.
Microsoft’s Seeing AI app helps the visually impaired by translating text to speech. Photo credit: Microsoft
Of course, businesses have something to gain from this, too. Companies looking to recruit from a limited pool of AI talent can use philanthropic projects to convince potential employees that their work contributes to the world in a meaningful way. Developing products that serve previously ignored disability communities also expands the potential user base. And in many cases, building new features into products that non-disabled groups already use can make them better and more profitable.
It’s not just big companies that are innovating in this space: Over 100,000 deaf people use Ava, an app that allows them to participate in group conversations in English or French (with limited use in Spanish, Italian, German and Russian). Everyone who wants to join a conversation opens Ava on their phone and speaks normally while the app listens. Ava converts what is spoken into text in near real time and displays each speaker’s words in a different color for those who need to read it to follow the conversation.
Voiceitt is an app for people with speech disorders, such as those with temporary speech impairments after a stroke or brain injury, or those with longer-term conditions such as cerebral palsy, Parkinson’s disease or Down’s syndrome. Using machine learning, Voiceitt picks up on a speaker’s unique speech patterns, recognises pronunciation errors and normalises speech before creating audio or text output.
Products designed for the general public are also being used to improve accessibility. Smart assistants like Amazon’s Alexa and Apple’s Siri have been of the greatest help to the blind, helping them get online more easily. Users are also discovering new ways to open up voice-based assistants to the deaf and hard of hearing. One project used a webcam to enable an Echo to understand and respond to sign language.
“Accessibility is designed into everything these days,” says Ray-Fleury. “We need to make it part of the air we breathe.”
Driving innovation
Paulo Pinheiro, a 34-year-old robotics doctor from Brazil, was waiting for his flight at the airport when he saw someone his age in a wheelchair who didn’t seem to be very mobile.
“But she had a great smile,” he says.
That insight led Pinheiro to start a startup called HOOBOX Robotics. The company’s first product, Wheelie 7, lets wheelchair users use nine different facial expressions to initiate actions like moving forward, turning, or stopping. While the computer vision behind the technology is cutting edge, the kit itself takes just seven minutes to add to a wheelchair (hence the name), making it easy for even non-technical family members to set up. Currently, 55 people in the U.S. use Wheelie 7, most of whom are quadriplegics, people with amyotrophic lateral sclerosis, or elderly people.
“Five years ago this wouldn’t have been possible,” Pinheiro said.
Pinheiro’s team is still testing the Wheelie 7, which retails for $3,600, but is already working on a feature that would let users activate and use smart assistants using facial expressions.
“It’s not just a way to get around, it’s also a way to connect with the world,” Pinheiro says.
HOOBOX Wheelie uses machine learning to recognize facial expressions and control the user’s wheelchair. Photo credit: hoobox2017, Flickr
An inclusive future
AI of any type needs diverse data sets to ensure algorithms don’t learn biases or produce results that discriminate against certain groups. The issue is usually raised in the context of racism and sexism, but according to Henry Claypool, a technology consultant at the American Association of People with Disabilities (AAPD), people with disabilities are also at risk.
“We need a more progressive plan to ensure that people with disabilities are reflected in the data,” Claypool said.
Inclusion of all types of disabilities will be difficult; data profiles of people with disabilities can be easily spotted, and privacy is a concern, especially for conditions that are likely to carry stigma, like mental illness. But it’s possible to collect the data we need, with appropriate efforts to anonymize identifying information. Finding ways to do this properly is essential, Claypool adds. “If we’re not reflected in the data from the start, we’re denying people with disabilities access to the ubiquitous technologies that are becoming an essential part of modern society.”
“This has the potential to be a great equalizer,” Claypool said, but added that “if we’re not careful, these data innovations won’t benefit everyone.”
James Bowen’s family has felt the effects of a lack of accessibility planning: an update to his Echo required confirmation before making a call, making it harder for Caden to use. Still, James sees new technology as a way to expand his son’s world, and is already researching and trying new tools.
“What we’re looking at is the future and how we’re going to motivate him,” James said.
Receive emails about upcoming NOVA programs, related content, and feature reports on current events from a scientific perspective.