As they imagine typing, implants translate brain signals into keystrokes on a standard digital keyboard.
Image Credit
BrainGate, Leigh Hochberg, Daniel Rubin and Justin Jude via YouTube
Share
It’s hard to picture a keyboard layout other than the one we know best. From laptops to smartphones, it’s an integral part of our digital lives.
Scientists at Massachusetts General Hospital have now restored the ability to communicate by keyboard to two people with paralysis—using their thoughts alone.
Both people already had brain implants that could record their minds’ electrical chatter. The new system translated brain signals in real time as each person imagined finger movements. The system then accurately predicted the character they were trying to type.
The system learned to translate brain activity to physical intent after just 30 sentences. Typing speeds reached 22 words per minute with few errors, nearly matching speeds of able-bodied smartphone users.
“To our knowledge, this system provides the fastest… [brain implant] communication method reported to date based on decoding from hand motor cortex,” wrote the team.
The participants are part of the BrainGate2 clinical trial, a pioneering effort to restore communication and movement by decoding neural signals in people who have lost the use of all four limbs and the torso. One of the participants previously used the implants to translate his inner thoughts into text, but with mixed success.
Controlling a digital keyboard is far more intuitive and familiar, which makes it easier to grasp. Once a person learns to use the system, they don’t have to look at the keyboard, giving their eyes a break as they type with their minds. It also allows users full control of when, or when not, to share their thoughts, preventing accidental leakage of private musings onto a screen or broadcasted with AI-generated speech.
All Hands on Deck
Parts of the brain hum with electrical activity before we speak. Over the past decade, brain implants—microelectrodes that listen in and decode signals—have translated these seemingly chaotic buzzes into text or speech, allowing paralyzed people to regain the ability to communicate.
Methods vary. Some hardware takes the form of wafer-thin disks sitting on top of the brain and gathering signals from vast regions; other devices are inserted into the brain for more targeted recordings.
These systems are life changing. In a recent example, an implant translated the neural activity controlling a man with ALS’s vocal muscles. With just a second’s delay, the system generated coherent sentences with intonation, allowing him to sing with an artificial voice. Another device turned a paralyzed woman’s thoughts into speech with nearly no delay, so she could hold a conversation without frustrating halts. People have also benefited from a method that uses the neural signals behind handwriting for brain-to-text communication.
Brain implants aren’t purely experimental anymore: China recently approved a setup allowing people with paralysis to control a robotic hand. It’s the first such device available outside of clinical trials.
Perhaps the most widely used clinical solution is eye-tracking. Here, patients move their eyes to focus on individual letters, one at a time, on a custom digital keyboard. But the pace is agonizingly slow and prone to error. And prolonged screen time strains the eyes, making extended conversations difficult.
“Those systems take far too long for many users,” said study author Daniel Rubin in a press release, causing them to abandon the technology.
Tapping Away
For people who already know how to type, the standard keyboard layout—known as QWERTY—feels familiar and comfortable. Fingers stretch to hit letters in the upper row, tap directly down for ones in the middle, and curl into a loose claw to hit bottom letters and punctuation.
Be Part of the Future
Sign up to receive top stories about groundbreaking technologies and visionary thinkers from SingularityHub.
As fingers dance across the keyboard, parts of the motor cortex that control their motion spark with activity, precisely directing each placement. Mind-typing using a familiar keyboard, compared to a custom one, could feel more intuitive and relaxing.
Two people with tetraplegia gave the idea a shot. Participant T17 was diagnosed with ALS at 30, a disease that slowly destroys motor neurons, weakening muscles and eventually impairing breathing. Three years later, when he enrolled in the study, he’d lost control of his vocal muscles and relied on a ventilator. He could move only his eyes, but his mind was still sharp. The second participant, T18, was paralyzed by a spinal cord injury 18 months before enrollment. Both had multiple brain implants in different areas. These were connected to cables that shuttled recordings to a computer system for real-time processing.
The participants used a simplified QWERTY digital keyboard containing all 26 letters, a space key, and three types of punctuation—a question mark, comma, and period. To train the system, the volunteers imagined stretching, tapping, or curling their fingers to type text prompts, while implants captured and isolated neural signals for each finger. After training, a deep learning model predicted intended characters, and a language model continuously attempted to autocomplete the sentence.
After practicing just 30 sentences, both participants could copy on-screen text or type whatever they wanted. When asked “what was the best part of your job,” T18 cheekily replied “the best part of my job was the end [of] the day.” Meanwhile, T17, a fan of The Legend of Zelda video games, told the researchers “you should try oracle of ages and seasons…another is skyward sword…the music in those games is great.”
Their typing speeds broke records. T18 communicated at 110 characters or roughly 22 words per minute, which is 20 characters more than a previous state-of-the-art method based on handwriting, wrote the team. The rate is nearly on par with able-bodied smartphone users similar to his age. Typing errors were consistently low and neared perfection after practice.
T17, with incomplete locked-in syndrome due to ALS, typed 47 characters a minute at a higher error rate. He had full use of his vocabulary, unlike with previous systems that imposed word restrictions, and communicated much faster.
The performance differences could be due to where their implants are located. T18's microarrays are on both sides of the brain, with some covering an area that controls all four limbs. T17’s implants are on only the left half of his brain, with less coverage of finger motor areas.
The team is now tweaking the system for longer use tailored to individuals. As disease progresses, the link between brain signals and keyboard characters may drift and produce more errors. But updating the algorithm is easy. The system needs only a few sentences to learn, so users could start each day mind-typing some thoughts to keep things dialed in.
Updates to the digital keyboard, like adding numbers or the return and delete keys, are in the works. Temporarily disabling the language model could also let participants type strong gibberish passwords, internet slang (ikr, btw, lol), and other non-standard words without being autocorrected.
The brain implant “is a great example of how modern neuroscience and artificial intelligence technology can combine to create something capable of restoring communication and independence for people with paralysis,” said study author Justin Jude.
Dr. Shelly Xuelai Fan is a neuroscientist-turned-science-writer. She's fascinated with research about the brain, AI, longevity, biotech, and especially their intersection. As a digital nomad, she enjoys exploring new cultures, local foods, and the great outdoors.
Related Articles
Hackers Are Automating Cyberattacks With AI. Defenders Are Using It to Fight Back.
Thousands of Everyday Drone Pilots Are Making a Google Street View From Above
Scientists Send Secure Quantum Keys Over 62 Miles of Fiber—Without Trusted Devices
What we’re reading