Imagine handcuffs that can never be removed, or leg shackles that hold for life. Whatever strength of spirit comes of misfortune, it imposes difficulties on 10,000 people each year who suffer spinal cord injury. A combination of technologies now holds promise to restore mobility and at last break the shackles.
A robotic arm in an ASU laboratory whirrs and stretches out. It reaches not only for a point in space,
but also for knowledge.
What is learned will be used
to restore useful movement
to paralyzed limbs. While the robot
now moves according to information
gleaned from the study of living motion,
the computerized arm must reach further.
A team of engineers and scientists from
Arizona State University is working to control the robotic arm by interpreting signals from
the brain. If they are successful, the work could eventually lead to methods for replicating that control in the muscles of living subjects.
Such a system would allow spinal cord lesions to be bypassed by rerouting nerve impulses.
Spinal cord lesions disrupt two-way communication between the brain and the body’s muscular structures. The brain continues to send and receive messages from above the lesion—the break point. Muscles do the same below the point of interruption. But there is no middle, no point of connection.
Other researchers have worked to establish computer-controlled stimulation systems to the nerves and muscles below the break. The ASU
scientists are working to reconnect muscles from below the break
directly to the brain. If they are successful, resulting limb motion would be significantly more real-time and natural in appearance.
“Grasping systems that make use of remaining muscle function have existed for more than 20 years,” says James Sweeney, an ASU biomedical engineering professor. “These systems involve electrodes that are implanted under the skin and wired to functioning muscle, which is then retrained to perform new tasks.”
But such retraining is often painstakingly slow and the resulting movement somewhat awkward. The process also compromises the intended function of the host muscle, since its use is diverted to arm control.
“What we want to do now is to refine artificially controlled motor function by restoring linkages directly to the brain,” Sweeney explains.
Mapping the Motions
Brain mapping is, literally, just that: Mapping out which brain cells trigger which specific movements.
“This project actually started with work being done by neuroscientist Andy Schwartz,” explains Gary Yamaguchi, an ASU professor
of biomedical engineering. “Schwartz has painstakingly studied motor cortex cells one cell at a time, recording how cell firing rates relate to hand motion. As a result, the fidelity of his neuro-control signal is more reliable than any other in this field.”
The motor cortex is the part of the brain that seems to somehow control movement. As Schwartz and the ASU team figure out the exact relationships between cell firing patterns and resulting movement, they hope to eventually replicate fluid, natural motions in prostheses and in paralyzed limbs.
To date, Schwartz’s neuro-signal work is very precise. He can predict both an animal’s intent to move a hand, and the direction of intended motion a mere 100 milliseconds (one-tenth of a second) before that movement actually begins. The accuracy of his predictions is within a few degrees of motion. Predicting that movement is critical to creating the current computer simulations. The simulations attempt to replicate the natural movements as they occur.
There is one down side to Schwartz’s work. He can only “map” a maximum of three to four brain cells per day. Before he can reconstruct the natural movement trajectories, he typically maps out precisely the signals of about 450 brain cells.
Admirable work? Yes, but there are still thousands of relevant cells within the brain’s motor cortex whose function and relation to movement control remains unknown.
A technological breakthrough has enabled the ASU researchers to cut through the tedium of the data collection process. The scientists now use a new computer-based device for monitoring responses in the brain. Using this device, scientists can map hundreds of cells at once.
“With our neural recording system, we now have the technology to monitor 96 different electrodes simultaneously on separate channels. Each channel measures and sorts data about the firing rate of up to four cells at a time,” explains neuroscientist and bioengineer Daryl Kipke.
Kipke says that many motor cortex cells are tuned directionally. That means the firing rate of a cell varies depending upon the direction someone intends to move his or her hand. Movements in a cell’s preferred direction correspond to its highest firing rates, while movements in the opposite direction correspond to its lowest firing rates. In a population of many motor cortex cells, many preferred directions are represented.
For example, Kipke explains that research results indicate that if the hand moves up and to the right, then the cells whose preferred direction is in that direction will have the highest firing rates. Other cells will have lower firing rates that depend on how far the movement direction deviates from the cell’s preferred direction.
Researchers monitor the firing rates of many cells at the same time. They know their preferred directions. As a result, the scientists are able to predict where the hand will move before the actual movement.
Kipke concedes that the data compiled to date is not perfect enough to calculate the exact fingertip position. He also admits that mapping has been hampered by problems in maintaining solid connections between all 96 electrodes and their surrounding cells. Still, he says, what he and his ASU colleagues have learned is sufficient enough to at least
establish the hemisphere of motion.
The ASU scientists are now working to modify specific mathematical algorithms that Schwartz developed so that they work with simultaneous signals. The team must also study how factors such as fatigue and motivation levels of the subject might alter neuro-responses to stimulus.
“Once we can consistently measure firing rates from many cells at given times and under given conditions, we can create a map of the motor cortex that will allow us to estimate movement of the fingertip a fraction of a second prior to the actual movement,” Kipke says. “We’re really on the forefront of this area of research.”
Decoding the Signals
All this data can get pretty confusing. Cell firing patterns are extremely complex. Not only do researchers measure whether or not a cell is firing, they also measure how often it fires—the firing rate. They record this data for each cell they study. That adds up to a lot of data.
Team members can display the information by charting a series of spikes, which represent the firing of cells over a period of time. Even a simple arm motion can produce hundreds of spikes from each brain cell involved.
The relevant brain data used by the team members is in the form of firing rates. Schwartz uses a relatively simple algorithm that combines firing rates from each cell. These combinations can be calculated many times throughout a movement to give a series of small movement pieces. But most actions use a combination of movements. Suppose you were to trace a spiral shape in the air. You would have to change direction hundreds of times, moving a tiny bit in each direction.
For a human being, interpreting all the spikes associated with such a gesture would be nearly impossible. That is why the research team uses an algorithm developed by ASU electrical engineer Jennie Si. The algorithm “decodes” the data. The new algorithm has the potential to give a more accurate prediction using fewer cells than Schwartz’s original method.
The “Self Organizing Neural Network” algorithm is what Si calls
“a learning algorithm.” It actually learns from experience, adapting itself as it receives new information.
Si enters the raw data—the firing pattern—into her computer.
In return, she gets a two- or three-dimensional map of the motion
associated with the pattern. The map shows the direction of movement, as well as its speed and position. Best of all, the algorithm analyzes the data almost instantly, so there is no wait.
As the researchers gather data from more brain cells, and as Si’s
program acquires more information, the team will be able to use cell
firing patterns to predict movements with increasing accuracy.
“Still, there’s a tremendous difference between being able to predict movements and actually being able to replicate them,” Yamaguchi says.
All the Right Angles
For example, the predicted motion trajectories created by the scientists are actually endpoint trajectories for the limbs. There are an infinite number of ways to combine joint motions to
create the same endpoint motion. That is where Jiping He’s work comes into play.
He is a robotics expert. The ASU scientist works primarily with joint, rather than muscle movements. He studies the angles and angular velocities that are necessary to get the right endpoint motion while at the same time using joint positions and motions that appear to be natural.
“We’re facing a redundancy problem. There are an infinite number of possible ways joints can move to get a hand to move at the time, speed, and direction we want it to,” He says. “Our job involves quickly picking one combination that looks natural, rather than spastic, and which uses as little energy as possible.”
To demonstrate, look closely at your left index fingertip.
Now think about the many different ways your shoulder, elbow, wrist, and knuckle could rotate and move directionally to make that
fingertip go just where you want it to go.
Joints move. In turn, they also alter the movement possibilities for joints further along the limb. Joints also move with particular speeds and in response to applied forces.
A fingertip can be moved to arrive at a certain place by many
different combinations of shoulder, elbow, and wrist motions. It also can arrive at the same point in space tugging a heaving weight or barely touching a dewdrop.
For He, each joint is mathematically described by sets of variables. These variables account for position in three dimensions and movement at different speeds and strengths.
Ultimately, He must define the “best” way to get that fingertip to and from every possible set of x, y, z coordinates—at given strengths and speeds. The tools he uses are mathematical expressions called algorithms.
He began by examining a biological-based algorithm called the Berkinblit. The Berkinblit algorithm mathematically describes the movements frogs make when wiping irritants off their hind legs.
The algorithm is only a starting point. The Berkinblit deals primarily with two-dimensional joint movement. Human joint movements have proven to be much more three-dimensional, and more complex.
Yamaguchi and Si are developing an alternate mathematical method called the pseudoinverse algorithm. Their work uses a computer-simulated arm model to predict the optimum ways to actuate muscles and to create the desired fingertip motion. Yamaguchi’s hope is that the “neural” trajectory defined by the brain cell activity will be used
to control this simulation in the very near future.
“Jennie Si is a very bright and talented person,” Yamaguchi says. “The algorithm we developed was specifically for complex musculoskeletal movement simulations. The work solved a very difficult and longstanding problem in the field of biomechanics.”
Once refined, the pseudoinverse algorithm might allow the ASU researchers to skip right by the Berkinblit joint roadblock. They will be one step closer toward their real goal: providing fluid, natural motion from the shoulder to the fingertip.
“If we can solve the neuro to brain interface problem, we will have to implement Schwartz’s or Si’s algorithm in real time. We will take the output from the brain mapping program and instantly turn that into a desired endpoint velocity,” Yamaguchi explains. “Then we take the endpoint velocity and run the Berkinblit algorithm and compute the required joint angular velocity. Or we use the pseudoinverse algorithm to compute required muscle activation patterns. It all depends on whether we are trying to control a robotic arm or a paralyzed limb.”
Completing the Circuit
Mind boggling? Believe it. Then consider that
it takes massive computer time and intensive mathematics just to
imitate what the human brain and body is engineered do almost
instantaneously. Yet, the job still is not complete.
Think of neuro messages in terms of telephone transmissions. Your telephone can easily communicate with every other telephone—until an underground phone cable gets cut. While the phones on the “command” side of the break still work, the phones past the break do not.
The same is true with messages from the brain to other parts of the body. As long as the spinal cord is intact and functioning, messages move quickly back and forth with no interruption. But when the spinal cord
is damaged, the direct line of communication from brain to body is
damaged as well.
Return for a second to the telephone analogy. It is possible to bypass the break in a phone cable by using cellular phones, because their signals do not travel in straight lines. Cellular signals bounce from place to place, via a series of relay towers and satellites high in orbit above the Earth.
Knowing the concepts and calculations behind a communications link is one thing. Actually wiring it to work is another entirely.
That job falls to Sweeney, the neuromuscular stimulation expert. Unlike a cellular telephone network, Sweeney must actually devise methods for hard-wiring a communications network within a living person.
“With spinal cord lesions, the body no longer has intact circuitry to transmit electrical impulses below the break to activate muscles,” Sweeney explains. “My main job is to design and implant electrode
systems that use brief electrical pulses. Such stimuli actually make
muscles contract artificially. The muscles move when and how a person with an injured spinal cord needs them to move.”
The system is called functional neuromuscular stimulation (FNS).
Early FNS research findings showed that it could require as many as 80 different electrodes, each attached to numerous muscles, to move endpoints in a desired way. But maintaining electrode integrity and
connectivity proved challenging—although not nearly as challenging as trying to control so many electrodes simultaneously.
Sweeney believes that one solution might lie in controlling muscle groups instead of individual muscles. He has looked deeper within the nervous system to where major nerve bundles branch outwards from the spinal cord. Over time, he has developed a “neural cuff” system.
A neural cuff is a series of miniature electrodes and lead wires encased in a cylindrically shaped tube. By wrapping a neural cuff around each nerve bundle, Sweeney can selectively activate groups of nerves, rather than individual nerves. As a result, he has fewer electrodes
to implant and control.
“Eventually, we hope to design a system that includes intricate electrodes that attach directly to the motor cortex region of the brain at the upper end, nerve cuffs at the distal end, and a power stimulation
system that’s implanted inside the body,” Sweeney explains.
The ASU scientists know that much work remains to be done. Simply learning the intricate nuances of how the human brain commands and moves muscles and joints is a difficult task in itself. But actually
creating the sophisticated biological wiring necessary to generate fluid, natural movement—let alone getting muscles to move when and where they are needed instantaneously—is the massive challenge that must be confronted one step at a time.
—Lindsey Michaels and Diane Boudreau
