Research Projects

HELIA
“Explained, programmed, done!” – Robots helping out in everyday hospital life („Erklärt, programmiert, gemacht!“ – Helfender Roboter im Klinikalltag, HELIA)
Start: 10/2025
End: 09/2028

High time pressure, complex care tasks, and increasing staff shortages characterize the everyday work of medical professionals in hospitals. This is exactly where HELIA comes in: an intelligent robot will support hospital staff in the future with initial medical history taking, admission examinations, and patient care. With the help of the newly developed “Explain-Program-Do!” system, the project combines speech processing, robotics, and medical care. The robot can be used flexibly with spoken instructions alone – no technical knowledge is required. This allows processes such as patient transport and other routine tasks to be digitized efficiently, noticeably reducing the workload on staff. In the future, the technology will open up new applications in hospitals, home care, and emergency services.
The HELIA robot system supports patient care in everyday clinical practice in a needs-based manner by being programmed in natural language by medical professionals themselves. The robot creates a semantic 3D scene graph of the environment (VLM) and uses a combination of current foundation models (LLMs) to automatically generate so-called template programs from the linguistic inputs (no-code approach). However, the LLMs must first be retrained to take into account the semantics of the environment, medical terminology, sensor data, and the robot's available capabilities [“Explained”]. The template programs are stored and can be easily reused by anyone in everyday life [“Programmed”]. Using natural language instructions and taking sensor data into account, the robot can then perform tasks independently in real time, even in hectic, dynamic environments. The HELIA system can then execute these tasks autonomously in a manner appropriate to the situation, effectively supporting and relieving the team in the clinic [“Done!”].
Role of the FZI
The FZI Research Center for Information Technology plays a central role in the HELIA project as a research and development partner for the intelligent core technologies of the robotic assistance system. Its main responsibility lies in the design, development, and integration of AI-based processes that enable intuitive, safe, and practical use of the robot in everyday clinical practice.

One of the FZI's main areas of focus is AI-supported image and sensor processing. Here, algorithms are being developed for the automatic recognition and segmentation of body regions and relevant skin areas. To this end, the FZI selects and integrates suitable camera and sensor systems, trains corresponding machine learning models, and ensures their reliable integration into the robotic platform.

In addition, the FZI is responsible for natural language processing in the project. It is developing a system for processing natural language inputs that recognizes and correctly interprets the information. In this context, large language models (LLMs) are being researched, trained, tested, and optimized for medical use. The goal is to translate voice inputs from specialist personnel into executable robot programs, thereby implementing a no-code programming approach. In addition, FZI is developing an interactive dialogue system that recognizes uncertainties and resolves them through targeted queries.

In the field of system integration and robotics, the FZI develops interfaces between robot systems, backends, and sensor technology, ensuring a modular, scalable, and secure architecture. It also supports data management, program storage, and interoperability with existing medical IT systems.

In addition to its technical tasks, the FZI is also involved in usability evaluation, user testing, ethical reflection, project communication, and knowledge transfer. It analyzes test results, derives optimization potential, and contributes its expertise on data protection, fairness, and acceptance of AI systems. In this way, the FZI makes a significant contribution to ensuring that HELIA can be used in a technically efficient, user-oriented, and responsible manner in the clinical environment.

Contact person
Staff
Division: Embedded Systems and Sensors Engineering
Headquarters Karlsruhe

Research focus
Applied Artificial Intelligence

In this research focus, the FZI concentrates on practical research into the key technology of Artificial Intelligence (AI). Innovative AI solutions are developed and transferred to application areas such as mobility, robotics, healthcare technology, logistics, production, and supply and disposal on behalf of our partners and customers.

Funding notice:
The HELIA project is funded by the Federal Ministry of Research, Technology and Space.

Go to Top