Voice-controlled Robot Dog in Hospital
The FZI conducts research as a consortium partner on natural language human-robot interaction
Research focus: Applied Artificial Intelligence
A new research project at the University Hospital of Würzburg (UKW), in collaboration with Karlsruhe Institute of Technology (KIT), the FZI Research Center for Information Technology, and the Forschungszentrum Jülich (FZJ), is investigating how a robotic dog can support doctors and nurses in their daily work on the ward. At present, robo-dog HELIA is being trained and programmed for use in the dermatology clinic through verbal interaction. The project is part of the funding program “Natürlichsprachliche Integration von Robotik in Gesundheitseinrichtungen (Natural Language Integration of Robotics in Healthcare Facilities) (NLP.bot)” of the Federal Ministry of Research, Technology and Space (BMFTR) and is receiving a total of 1.78 million euros in funding. HELIA is still a little clumsy, stumbling loudly through the corridors and moving exclusively in response to remote control commands. But even during her first appearance on ward H21/22 of the dermatology clinic at the University Hospital of Würzburg (UKW), the red robot dog with silver legs won everyone’s hearts. Nurses, cleaning staff, doctors, and patients pulled out their cell phones or reached out their hands to pet HELIA or at least take a picture of her. HELIA is an acronym for the German equivalent of “Assistant robot in everyday clinical practice.” Over the next three years, the UKW will be working with Karlsruhe Institute of Technology (KIT), the FZI Research Center for Information Technology, and the Jülich Research Center (FZJ) to test exactly how the four-legged robot can and may be used in everyday clinical practice.
Recording visits, documenting findings, photographing wounds, and measuring vital signs
The goal is for HELIA to listen to and carry out verbal commands from authorized nursing and medical professionals in the future. It will accompany staff on rounds, relieving them of some of their workload – for example, recording the rounds and documenting findings. Ideally, it will transfer these to the hospital information system. Additionally, HELIA could collect its own data, for instance by photographing wounds and measuring vital signs. There are also plans for the walking robot to accompany patients to appointments, display what is said on a screen, and explain the technology to patients in their rooms.
Setting standards
“This, of course, requires that both staff and patients agree to HELIA accompanying them during the initial examination and throughout their stay in the hospital,” says Prof. Dr. Astrid Schmieder. The dermatologist and senior physician at the clinic and polyclinic for dermatology, venereology, and allergology is undoubtedly delighted with the four-legged robot. When Prof. Dr. Arne Rönnau, Scientific Director at the FZI and Professor for Machine Intelligence and Robotics at KIT, and Christoph Zimmermann, Head of the Medical Information Technology department at the FZI, approached her with the idea of a robot dog on the ward, the dedicated doctor and scientist was immediately enthusiastic about the new research project. She had no trouble getting the staff excited about HELIA. Everyone sees the potential of this future technology, which could be used in various healthcare facilities, in home care, and in emergency services, and is excited to set new standards here.
Effective, successful and reliable cooperation
Why Würzburg? ‘We worked together extremely well on the previous project, HybridVITA. We were effective, achieved a great deal, and could always rely on each other. We also enjoyed the project. For this reason, we wanted to research the implementation of a robot dog in Würzburg hospitals with Astrid Schmieder,” comments Christoph Zimmermann. The electrical and information engineers and his team at the FZI are investigating how specialist staff, robot dogs, and patients can communicate naturally with each other, and how the system can support the continuous recording of skin surfaces or wound observation. HybridVITA already focused on providing medical care for patients with chronic skin diseases using an app-based solution with contactless diagnostics.
HELIA is trained and programmed using speech
Why does the HELIA project rely on a dog-like walking robot instead of a humanoid machine? People often react sensitively when technology looks too human. This phenomenon is called the “uncanny valley”—the more human a robot appears, yet not completely real, the more uncomfortable many people feel. A four-legged robot remains clearly recognizable as technical support and avoids this effect. “We want to develop a system that employees intuitively perceive as helpful in everyday clinical practice,” says Arne Rönnau. His team at the Institute for Information Management in Engineering at KIT is researching how a walking robot can learn tasks simply from naturally spoken explanations, rather than having to be laboriously programmed using software. Hospital staff should be able to describe to the robot what it should do – and the Artificial Intelligence converts these instructions directly into functional programs. “It’s as if the robot had been given an explanation of a task and then learned it itself,” explains Rönnau. To this end, the researchers are developing the “Erklärt-Programmiert-Gemacht!” (Explained-Programmed-Made!) system, which combines speech processing, robotics, and person recognition.
An additional gripping arm for opening doors and clearing away food
This means that HELIA will first attend “dog training school” to learn who is authorized to give commands. Voice recognition software converts the employees’ sentences into corresponding codes so that the robot dog can perform the various tasks required on the ward. “We are excited to see how we can best help here,” says Arne Rönnau. In the next step, HELIA will be equipped with a gripper arm, for example, so that she can also open doors, restock supplies, clear away food, and photograph wounds.
The use of robots in hospitals naturally raises ethical, legal, and social issues – ELSA for short, which stands for Ethical, Legal, and Social Aspects. The FZJ is responsible for addressing these issues. Prof. Dr. Jan-Hendrik Heinrichs summarizes the challenges as follows: “It is essential that the HELIA system is used for the benefit of patients wherever possible and does not endanger them. In addition, HELIA should reduce the burden on medical professionals while maintaining a safe working environment. This allows resources to be used where they provide the greatest medical or nursing benefit without creating new inequalities in the use of healthcare resources.”
“Attention, robot dog for research purposes in use”
The robot dog HELIA was developed by the Swiss company ANYbotics and is actually called ANYmal. Currently, around 200 of these ANYmals are in use, but primarily in places that are dangerous for humans. As inspection robots, they monitor oil platforms, wind farms, and steel mills. Hospital wards are new territory. “Since no one expects to see a robot dog here, we have to put up a warning sign at the entrance,” smiles Astrid Schmieder.
Project participants HELIA – Robo-Dog
- Dermatology clinic, University Hospital of Würzburg (UKW)
Marco Stumpf, Jan-Hendrik Maiwald, Jörg Eberling, Marion Berthold, Tassilo Dege, Astrid Schmieder - Karlsruhe Institute of Technology (KIT)
Roberto Corlito, Louis Ensil, Hong Phuoc Nguyen Nguyen, Arne Rönnau - FZI Research Center for Information Technology
Dominik Beyer, Julia Konle, Christoph Zimmermann - Forschungszentrum Jülich GmbH (FZJ)
Dilara Diegelmann, Jan-Hendrik Heinrichs - Project management VDI/VDE Innovation + Technik GmbH (VDI/VDE-IT)
Sandra Beyer, Patrick Ehrenbrink
Funding
HELIA is participating in the funding program of the Federal Ministry of Research, Technology and Space (BMFTR) “Natural language integration of robotics in healthcare facilities (Natürlichsprachliche Integration von Robotik in Gesundheitseinrichtungen) (NLP.bot)” with €1.78 million. Eight projects were selected for funding from 54 proposals submitted. Project description: HELIA — Miteinander durch Innovation