Virtual and Augmented Reality

Human-machine interaction and cerebral waves: between Sci-Fi, dystopia and reality

27 March 2018 | Written by Cristina Pozzi

Human-machine interaction does not belong only to science fiction and can be applied to different sectors, including the medical one. The BrainControl platform, for example, helps ALS patients to interact with the outside world, as Pasquale Fedele, an IT engineer and Liquidweb Founder, told us.

 

Elon Musk, innovator, visionary and great entrepreneur, is among those who argue that reality could be a projection of our brain, a deception of the senses. So, we live in a huge videogame where our consciences meet and where physicality is nothing?
This theory is not a new one: it has always been the subject of discussion among thinkers such as the skeptics in the fifth century BC, in ancient Greece, David Hume and the empiricists during the 1700s, Edmund Husserl (between 1800 and 1900) and all who ask themselves questions about reality and our knowledge of the world.
This discussion today is widespread thanks to tools such as videogames, virtual reality and augmented reality, so immersive as to deceive our senses with ease.

In the sci-fi movie “Surrogates” (2009) Bruce Willis plays the role of a policeman who lives in a future where the danger of physicality is such that no one leaves home, preferring to live a virtual life through headset and helmets that, guided by brain waves, allow to move around the world through real avatar-robots. And it is from this film that begins a story with unexpected implications.

Il mondo dei replicanti Bruce willis

We need to go back a few years to find Pasquale Fedele, Liquidweb CEO and Founder, on a return flight from the United States. During the flight, to let the time pass, Pasquale decides to watch this movie. Perhaps a coincidence or maybe destiny, from some time Pasquale had started to develop the first prototypes of Brain Computer Interface and realizes that, precisely his work, gives him the opportunity to go in a direction totally opposite to the dystopian reality described in the film, but based on the same principle. He understands that it is possible to realize, through existing technology, devices that allow us to explore the world in our place and to interact, through the use of brain waves.
The computer engineer who has always studied with the aim of improving the quality of people’s life, decides to use this solution as an inspiration for a new project in the health sector: BrainControl, a platform Brain Computer Interface with CE medical certificate, for allow ALS patients to return to interact with the outside world.
We met and interviewed him to find out how an idea born from science fiction can create a new future starting from the same technologies.

How did you get the idea?

“The idea comes from a European research project. I was working on a benchmark on the state of the art of Brain-Computer Interface (BCI), a field of study on human-machine interaction through brainwaves. At that time the BCI already had many studies and prototypes, also in the field of assistive technologies, but nothing that was usable and robust enough to be able to leave the walls of research laboratories; this is how the challenge was born to find a solution for the use of such technologies to restore the possibility of communicating to those tetraplegic patients or those suffering from total paralysis who were still in possession of complete cognitive functions”.

What is BrainControl and how does it work?

“BrainControl is a framework based on Artificial Intelligence for human-machine interaction through biofeedback. A sort of “mental joystick” that allows you to overcome motor and communication disabilities”.

What is BrainControl and how does it work?

“BrainControl is a framework based on Artificial Intelligence for human-machine interaction through biofeedback. A sort of “mental joystick” that allows you to overcome motor and communication disabilities. The innovation is in the software that classifies biometric signals coming from one or more wearable or remote sensors, such as electroencephalographic sensors but also, when necessary, infrared, inertial, 2D and 3D cameras. All of this data is used to create a model for a custom recognition system on the user. This model of recognition is specific to each person. The machine learns over time and is able to adapt the model to the system’s modification, this is essential for people whose motor skills worsen over time, as for patients with ALS and other neurodegenerative diseases, trauma from accidents or cerebral ischemia “.

Joystick realtà virtuale


What was the moment when you realized that it could be done?

“The most important moment was about five years ago, when we were asked to try a prototype on a patient in an advanced state of ALS, in a condition that is called” locked-in”, meaning that you are not able to have any interaction with the outside. Generally one of the last movements that remains to patients with ALS is that of the pupils, so up to six years before, the patient had communicated by means of eye pointers, but when the pupils also stopped working, he totally lost the opportunity to communicate with the outside and with his loved ones. Thanks to our technology, the patient has returned to communicate and, first of all, has asked to open his eyelids: he wanted to see his nephew, who was born five years earlier. He had only heard about him, without being able to look him even once. Behind the eyelids, in fact, the eyes were still working and able to see: in that moment, when that man first saw his nephew, I realized that like him there were many other patients to help and I understood that I could not go back”.

Brain Control realtà aumentata

When Pasquale tells you this story personally, it’s hard not to get emotional in front of his sincere passion and his ability to make you feel in that room.

A beautiful reality in Italy: did you find it difficult to stay in our country? Which are the most advanced countries for research in this field?

“There are geographical contexts that are certainly much more favorable, as overall ecosystem, for the development of such innovations during the startup phase. On the research front we are not bad, we have never lacked ideas…other countries at the forefront of the sector are Austria, Germany, Holland and the United States “.

What are the new frontiers for the next 10 years? What are you working on?

“Future versions will implement advanced communication and entertainment features, home automation and robotics. We have already created working prototypes of wheelchair control and avatar (robotic telepresence device). Outside the medical sector, IoT and Robotics will be the main areas of use starting from automotive applications, industrial and home / office “.

You told us that today the patient learns to command the system, as if he were to learn a language: will these devices always require training? Will our brains have to adapt to these new tools for the future?

“Today the required commitment is comparable to that of learning to play a musical instrument or to practice a new sport, but the cardinal principle of the whole system is that it must be the machine that makes the greatest adjustment effort. We will not have to learn how to interact with the machine but vice versa “.

Human-machine interaction, the main focus of Pasquale and its Liquidweb, is today a sector in great excitement thanks to the advancement of technologies that enable applications that a few years ago we could only imagine through science fiction movies. In particular, one of the areas in which there are many investments at a global level and where it is possible to obtain economic returns even in the short term is gaming: a sector that, with its 91.5 billion dollars worth of value in 2016, is worth almost double the cinema and music put together (respectively 38 and 16 billion dollars in 2016). For example, we can think of the success of video games for smartphones such as Pokémon Go, Candy Crush, Clash Royal, or of the new consoles that are increasingly focusing on virtual reality. Here, the next frontier is precisely that of games that use the technology of the Brain Computer Interface to immerse players even more in virtual reality.

Speaking of the future, we have therefore asked Pasquale if he is thinking about applications for this sector, to exploit the opportunity that derives from it, but for him it would only be a distraction from the “precise choice to do something useful for people’s lives”.

Pasquale Fedeli intervista Brain Control

After all, any entrepreneur to succeed and pursue his goals must be moved by a vision. If it is strong enough to guide it through difficulties, to allow it to overcome obstacles and to extricate itself through the scarce resources available, then the motivation to continue becomes a magical tool that can go far beyond any science fiction movie.

Cristina Pozzi
Cristina Pozzi

Contributor

Co-founder and CEO of Impactscool, Cristina is a successful entrepreneur, a populizer, a social entrepreneur and an active citizen with the aim to impact society positively.

read more