When I was at the Polytechnic of Turin, I already had a strong passion for multimedia technologies, and since VR was still a dream at that time, my first passions were the two technologies that are its foundations, that is computer graphics and computer vision. My teacher of computer vision, that later on would have become my thesis supervisor, was professor Andrea Bottino, a young but already very talented professor.
Fast forward more than 10 years, Andrea and I are still in touch, and sometimes we collaborate on AR/VR projects (once I also did a lesson to his students!), or just meet and talk about immersive realities. He started working in the field many years before me, and he has seen all the transition from old-school VR to the modern devices kickstarted by Palmer Luckey, and so has a great knowledge on immersive realities and many other related technologies.
One of the projects on which I collaborated with him is Holo-BLSD, a training solution aimed at teaching everyone how to behave when someone has a stroke in front of us. I collaborated with Prof. Bottino and his team (among which I highlight the amazing Francesco Strada, Ph.D.) in the early stages of the project, that has been in these years carried on by them, that have transformed a POC into a complete product. Since I think that this is an amazing application that can be useful to many in the healthcare field, I decided to interview Andrea, and let him explain to you what it is about, and how his team has developed this experience. I’m sure you’ll find all of this interesting.
Hello Andrea, introduce yourself to my readers!
First of all, thank you very much for having me here. It is a real pleasure for me to be interviewed for your blog. So, coming back to your question, I am an associate professor of the Computer and Control Engineering department at the Politecnico di Torino, where I am heading the Computer Graphics and Vision research group. Our primary research areas are Computer Graphics and Mixed Reality and our works span different topics like MR for serious games, human-computer interaction, usability in MR, virtual heritage, and so on. The other main activities of our group are related to machine learning and computer vision. And when I started working as a researcher, my main interests were in computational geometry, which I think is a bit too far away from the interests of your readers…
I know that your team has created a tool called Holo-BLSD. Can you tell us a bit more about it?
Holo-BLSD is a self-learning tool in AR. BLSD stands for Basic Life Support and Defibrillation and defines how to manage the first response emergency in case of people affected by cardiac arrest, drowning, or suffocation. These procedures include cardiopulmonary resuscitation (CPR) and the use of an automated defibrillator (AED). They are easy to learn and do not require any specific medical knowledge.
The problem is that, despite the availability of public access AED in our cities, when there is no medical personnel in the proximity of the accident, only a minority of arrest victims receive bystander assistance because of their fear or inability to perform the emergency procedures. So, increasing the percentage of the population capable of delivering proficient CPR in an emergency can help save lives.
Currently, the gold standard for BLSD learning is instructor-led courses. However, they are time-consuming, and they are costly because you have to pay the experts and rent a room for the lectures, and learners have to travel to the course location and spend a whole day there. A self-instruction tool can solve many of these problems if it is indeed effective in achieving the same learning outcomes of a traditional course. If we can create such a tool, we can increase the number of trained people in a cost and time-effective manner. And this is what inspired us in developing Holo-BLSD.
What are its features? Which one do you think is the killer feature of this application?
Holo-BLSD reproduces the standard course settings since it has a virtual instructor and a low-cost CPR manikin, to teach learners CPR providing them with realistic haptic feedback. Then it enhances this educational setting by including a realistic emergency scenario through holographic interactive contents. In other words, we immerse our learners into an experiential learning experience where they can experience a variety of different environments and emergency procedures.
As another feature, we added the possibility of using Holo-BLSD as both a self-learning and a self-assessment tool. Besides teaching trainees step by step what they have to do during the emergency procedures and why they have to do it. Holo-BLSD includes a “rehearsal” and an “evaluation” session. During rehearsal, learners can practice the skills acquired in the learning session through a serious game, where they have to complete the procedure in the correct way and in a timely fashion to accumulate points. In the evaluation session, Holo-BLSD can automatically evaluate the trainees’ achievements using the same assessment form used during the classical instructor-led evaluations and provide the final decision to grant or not the BLSD certificate to the user. So, if I should identify one, I think this is the killer feature of the application.
Then, we also included many interesting features to simplify as much as possible the use of our application without requiring complicated setup and to make the tool as complete as possible. We also simulate dialogues with a bystander and a full telephone call with the emergency medical service. We even found a way for directly measuring, within the Hololens, the frequency of the cardiac massage, which is one of the most important parameters for an effective CPR.
What have been the greatest difficulties you encountered while developing it and how have you overcome it?
I think the main challenge we had to face was to find a common language between us (pure technicians) and the medical experts that were collaborating with us. The medical domain was quite new for us, and both teams were giving for granted many concepts they considered trivial. It took us several iterations before achieving a shared understanding of the real issues on the table.
How has it been working with HoloLens on this complicated project?
It has been fun. This was the first project we ever developed with the Hololens, so everything was brand new for us. The nice thing was that we started with a challenging task that allowed us to stress the device and understand its capabilities and its limits.
The first impression I had of the Hololens was extremely positive. The device was a leap forward to any other AR system I ever tried before. Then, of course, we started fighting with issues related to the field view and hand-tracking capabilities. I also found limitations in the anchoring system. This point is quite funny. I stressed everybody in my team because I wanted to be able to place anchors in the world space with maximal accuracy and directly within the Unity editor. I was not satisfied with the possibility of placing an anchor interactively on a surface. I wanted to be able to decide the exact anchor position and orientation. We finally came up with a solution just to find out that it was not so necessary in our project and that a manual anchor placement was easier and faster.
What have been the quantitative results of the tests you have performed with this application?
In our tests, we tried to answer two main research questions. Is Holo-BLSD an effective learning method? And is it a reliable self-evaluation tool? To answer these questions, we involved 58 first-year students of Health-care Nursing faculty, none of which had previous knowledge of BLSD. We divided volunteers into two groups. One was attending a traditional course and a standard examination with an instructor. The second group was following the AR course without any external human support and was evaluated in the same session by an instructor and the automatic assessment module of our application.
The results were extremely positive. There were no statistically significant differences between the assessment results of the standard and AR groups, which showed the effectiveness of Holo-BLSD as a learning tool. Then, for the AR group, there were no statistically significant differences between the automatic assessments of learners and that provided by the human experts, which means that Holo-BLSD is a reliable self-evaluation tool.
Besides that, we also collected several metrics that showed the users’ appreciation of the instrument and its overall usability. We also received some negative feedback about the field of view of the Hololens and gesture management since, for most of our users, the pinch gesture was a real problem. This latter result was a bit surprising for us. As computer scientists, we had no issues with this gesture. But apparently, most people have a lower level of nerdity…
Why have you used AR and not VR?
The primary rationale for choosing AR was that our BLSD training requires learners to move rapidly in a real environment where obstacles can be present. To this end, AR guarantees users much higher confidence compared to VR in using their actual walking for navigating the environment and interacting with the real objects that are part of the simulation.
Since our software structure is quite flexible, we did some experiments with a full VR version of Holo-BLSD. We have seen that users in VR were not at ease in moving around the room. What we could observe was that people were walking with unusual and unnatural patterns since they did not trust our virtual system. They were scared of stumbling into obstacles, especially when they were getting too close to the manikin. The results of our usability questionnaires confirmed these observations. In the end, our trainees were more focused on trying not getting hurt rather than in learning the procedures.
What are the future planned evolutions for the system?
Our experiments with traditional instructor-based training gave us a lot of new ideas. First, we noticed that learners were often interacting with the instructor to get help or clarification and that the instructors were providing suggestions to learners when they were in trouble with the things they had to do. So, we understood that this dynamic interaction is extremely relevant, and we are working toward mimicking the presence of an instructor. In detail, we are currently developing an embodied conversational agent that can provide a dialogue based question-and-answer mechanism to support the learners in overcoming their difficulties or intervene autonomously when needed.
Then, we also noticed that our system delivers educational content in a too “rigid” way. What I mean is that we should let the learners tailor training to their actual needs, for instance, by selecting particular actions in the procedure to experiment with or to adopt different ways to present the same content twice or more to avoid annoying repetitions.
The last feature we are planning to introduce is the possibility to handle team training, where you have different components of a medical team that play different roles and need to cooperate during the emergency procedure.
What kind of collaborations are you looking for?
We continuously receive positive feedback from any person that tries Holo-BLSD. Our medical partners are currently using the tool for training their students with very positive outcomes. So, one idea I had was to look for commercial partners interested in supporting the development of the tool. I am currently fighting against the bureaucracy of my university. In one year, nobody was able to tell me how to handle such a contract. Then, we are open to collaborations with other researchers interested in Mixed reality for medical education and training.
You work in the CGI research field since many years. What are the differences and the similarities in Virtual Reality before Oculus Kickstarter campaign and after?
Oculus revamped the whole idea of VR and changed its paradigms. What was once a niche technology accessible only to industry and research institutes has become a consumer option. And the number of applications for VR is expanding from gaming to business, medicine, manufacturing, and architecture. We have affordable and powerful hardware and software tools. The deploy time has reduced drastically.
At the end of the 90’s, I was working for Motek, a VR studio in Amsterdam. We were using a Silicon Graphic workstation that cost about one million dollars and was as big as a wardrobe but with less computing power than the smartphone I have in my pocket. Interaction devices were few and bulky, and HMDs were simply terrible. I remember one of the biggest headaches of my life at Siggraph after trying some of the latest products.
I started teaching VR at the university in 2005, and my students had very few chances even to see the VR devices I was talking about during lectures. Now everything has changed drastically, in a positive way. There are still a lot of issues that need to be addressed, but I think technology is making giant leaps, and we can expect exciting advances in a short time.
How do you envision the future of AR/VR?
VR is actually getting most of the attention, but I think the future of AR is brighter. As soon as we have low-cost and lightweight devices effectively supporting it, consumers and professionals will fully understand the benefits of an augmented view in their everyday life.
What are your plans for the future?
Since I have two kids, I stopped making elaborate plans for the future since I always have to change them.
Something more that you would like to add to this interview?
I would like to thank you again for having me here. And for the interesting and ispiring talk you made to my VR students.
I hoped you liked this interview, and if you are interested in Holo-BLSD, you can get in touch with prof. Bottino via e-mail or ask me for an introduction.
I’m also curious to know your impressions, so don’t forget to let me know your opinion here in the comments or on my social media channels!
(Header image by Politecnico di Torino)