A new wearable device capable of recognizing hand gestures

Thumb up, closed fist, open palm, single finger movements. An international group of researchers has devised a bracelet capable of instantly recognizing hand gestures by “reading” the signals that travel from the brain through the forearm. The device could not only be used to control artificial prostheses, but also opens the way to the creation of new interfaces for the interaction between humans and machines: from computers without keyboards to cars without steering wheels. The research stems from a collaboration between scholars from the University of California at Berkeley (USA), the Polytechnic University of Zurich (Switzerland) and the University of Bologna: the results have been published in Nature Electronics.

“The device we made connects wearable biosensor technology with artificial intelligence algorithms: in this way the system is able to recognize up to 21 different hand signals”, explains Simone Benatti, researcher at the Department of Electrical Energy Engineering. and of the Information “Guglielmo Marconi” of the University of Bologna who participated in the study.

The first prototype of this type was developed at the Alma Mater in Bologna in 2015, in collaboration with the INAIL Prosthesis Center in Vigorso di Budrio, integrating a pattern recognition algorithm on a prosthetic system. Starting from these first foundations, the research team – led by engineers from the Berkeley Wireless Research Center (BWRC) – has now designed a new flexible cuff which, through a series of biosensors, is able to read muscle electrical signals in 64 different points. of the forearm. Once recognized, these signals are then processed by a low-consumption platform which, through artificial intelligence algorithms, is able to associate them with specific hand gestures.

When we want to move a hand, our brain sends a series of electrical signals that travel through the neurons in the neck and shoulders to the muscle fibers in the arm and hand. “The electrodes in the cuff are able to detect this electric field,” Benatti explains. “The device is unable to identify exactly which muscle fibers are excited, but the set of signals collected still allows us to map the gestures performed”.

To achieve this, the system must be trained to link the different sets of electrical signals in the arm to the corresponding hand gestures. Normally this training phase is very expensive from a computational point of view and requires the use of high performance systems. But the new device instead uses an innovative type of algorithms (known as Hyperdimensional Computing), which allows a very fast and efficient training phase.

“In our system, we have implemented a process where learning does not happen remotely, but on the device itself, extremely quickly,” explains Jan Rabaey, a professor at the University of California at Berkeley, one of the study coordinators. “In most cases, it is therefore sufficient to perform the hand movements only once for the system to start recognizing the gestures, and subsequently each repetition tends to improve the performance of the algorithm in a continuous learning process”.

Applications. In addition to speeding up times and increasing the efficiency of the system, this mechanism also makes it possible to ensure that the personal biological data of those who use the bracelet remain private. An element that offers many advantages also in view of the possible future applications of this technology. “An important field of application is certainly that of artificial prostheses,” explains Ali Moin, a scholar at the University of California at Berkeley and first author of the article. “But this technology also makes it possible to improve the interaction between man and machine, offering very intuitive new ways of communicating with computers and at the same time guaranteeing the privacy of users”.

The study was published in Nature Electronics with the title “A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition”. The research was coordinated by scholars from the University of California at Berkeley (USA). Simone Benatti and Luca Benini from the “Guglielmo Marconi” Department of Electrical Energy and Information Engineering participated for the University of Bologna.