A collaborative team of USC researchers has been awarded a National Science Foundation (NSF) Convergence Accelerator grant to develop new technologies for people with disabilities. From left: Steven Aguilar, Suk Lei Liu, Maja Matarić, Gisele Ragusa and Jesse Thomason. Photo/USC Rossier/USC Viterbi.
A collaborative team of University of Southern California (USC) researchers, led by Maja Matarić, Chan Soon-Shiong Professor and Distinguished Professor of Computer Science, Neuroscience and Pediatrics, has been awarded a $698,161 National Science Foundation (NSF) Convergence Accelerator grant to develop new technologies for people with disabilities.
The team, which includes experts in computer science, education, kinesiology and occupational therapy, aims to develop a personalized prototype interface enhanced by artificial intelligence to help people with disabilities learn and practice programming skills.
“Our team’s goal is to break down barriers and make coding accessible to everyone, including individuals with diverse abilities,” Matarić said. “Here, the focus is on connecting deeply with the user community and doing a lot of co-design to develop prototypes, all in less than a year. This is an incredible opportunity, a very demanding one, and a great example of USC’s ability to bring together multidisciplinary teams focused on addressing a societal need.”
USC is one of 16 interdisciplinary Phase 1 teams selected by NSF for its 2022 cohort, “Track H: Expanding Opportunities for People with Disabilities.” Co-principal investigators are Gisele Ragusa, professor of engineering education, Stephen Aguilar, assistant professor of education, Sook-Lei Liew, associate professor of occupational science and occupational therapy, and Jesse Thomason, assistant professor of computer science.
They will be joined by USC senior faculty including Heather Culbertson, assistant professor of computer science, and Mark Redekopp, professor of electrical and computer engineering and of the practice of computer science, as well as partners from the University of Colorado Boulder, University of Washington, Microsoft, Regional Center Institute, Rancho Los Amigos Rehabilitation Center, New York University, Tufts University, Northwestern University, Children’s Hospital of Los Angeles and Los Angeles-area community colleges.
“This research work means a lot to me,” Ragusa said. “As a longtime practitioner and researcher of access and inclusion for people with diverse abilities, I believe the work we undertake with our community partners has great potential to transform the lives and employment opportunities of people with disabilities.”
Adaptation to physical abilities
One billion people, or 15% of the world’s population, experience a disability. But disability poses significant barriers to entry into the workforce, the researchers note in their recommendations. In fact, roughly 80% of people with disabilities will be excluded from the workforce in 2021.
Meanwhile, programming jobs continue to explode, but many are difficult to access for people with disabilities. For example, a person with a disability may have difficulty writing code using a keyboard and mouse, but may be able to dictate code using voice recognition or select commands using a touchscreen.
“We still ask people with physical disabilities to learn to code using outdated technology that doesn’t adapt to their physical abilities. Our grant tackles this problem head on,” said Aguilar, an expert in educational technology and educational psychology.
“We work with people with physical disabilities to design platforms that adapt to their physical capabilities. This work will increase the variety of peripherals they can use to learn and allow them to learn programming using commercially available programming courses.”
As part of Phase 1, the team will gather data-driven insights for technology designers, develop machine learning techniques to customize interfaces, and use large-scale language models to assist with programming. Working together, the teams will develop and evaluate prototypes of interfaces (operated using voice, eye tracking, pedals, etc.) that enable people with disabilities to learn, practice, and use programming skills.
Thomason, an expert in natural language processing and robotics, will explore how voice recognition, eye tracking, gesture recognition, pressure pedals and other inputs can be used to reduce the number of keystrokes needed to program efficiently without a keyboard or mouse.
“Programming involves transcribing a lot of text like ‘open brace,’ which is tedious for a speech recognition system,” Thomason says, “and also combining additional modalities like eye tracking, gesture recognition, and pedal depressing with speech actions like ‘brace’ and ‘open brace,’ which is lifting the left shoulder.”
The project will also create a pathway for people with disabilities to receive programming training and enter the workforce, significantly impacting available job opportunities.
“As an occupational therapist, I hope to develop new tools with and for people with disabilities to help them become more comfortable interacting with computers and ultimately achieve their employment goals,” Liu said.
“I look forward to applying the results of my neuroscience research over the past decade as I work to build brain-computer interfaces and virtual reality tools that will enable stroke patients to participate in meaningful activities. I am pleased to be working with such an incredible interdisciplinary team, focusing on the everyday occupation that many people with disabilities say they most want: work.”
The Convergence Accelerator program and the awarded grants are part of NSF’s Office of Emerging Technologies, Innovation, and Partnerships. After the first year, teams will participate in a formal NSF presentation and proposal for Phase 2. Selected teams will advance to Phase 2 and receive up to $5 million in additional funding.
Published on February 9, 2023
Last updated: May 16, 2024