Research Projects
AuRorA
Autonomous Robots for Assistive Functions: Interactive Basic Skills
Start: 09/2017
End: 08/2020
The AuRorA project (Autonomous Robots for Assistive Functions: Interactive Basic Skills) was carried out by FZI Research Center for Information Technology in Karlsruhe as part of the German Federal Ministry of Education and Research (BMBF) funding program "Reusable, Interactive Behaviors for Proactive Robots in Smart Home Environments." The project aimed to develop autonomous service robots capable of proactively supporting people in everyday life — particularly seniors and individuals with mild cognitive impairments — without requiring prior technical training.
At the core of the system were adaptive, learning if-then rules based on so-called Behavior Trees. These enable the robot to react to events in a context-sensitive manner and continuously improve its decision-making through machine learning. A neural network selects the most probable response from a set of possible robot actions, and refines this selection over time based on feedback from successful or unsuccessful interactions, adapting progressively to each individual user's patterns and preferences.
A major focus was also placed on detecting non-verbal communication: the system analyzes the user's posture, facial expressions, gestures, and gaze direction in real time. Complementing this, a 3D environment perception module detects spatial changes in the surroundings — such as moved objects or newly initiated activities — and derives potential assistance needs from them. All of this information feeds into an adaptive dialogue system that enables the robot to act proactively without becoming intrusive or repetitive.
The developed technologies were validated in two real-world scenarios: a smart kitchen, where the robot assists with cooking tasks (e.g., handing over objects, reminding users of cooking steps, monitoring the stove), and a smart hotel, where the robot greets guests, guides them, provides information, and supports them with everyday needs. Person safety was ensured through GPU-based 3D safety fields and dynamic collision detection.
At the core of the system were adaptive, learning if-then rules based on so-called Behavior Trees. These enable the robot to react to events in a context-sensitive manner and continuously improve its decision-making through machine learning. A neural network selects the most probable response from a set of possible robot actions, and refines this selection over time based on feedback from successful or unsuccessful interactions, adapting progressively to each individual user's patterns and preferences.
A major focus was also placed on detecting non-verbal communication: the system analyzes the user's posture, facial expressions, gestures, and gaze direction in real time. Complementing this, a 3D environment perception module detects spatial changes in the surroundings — such as moved objects or newly initiated activities — and derives potential assistance needs from them. All of this information feeds into an adaptive dialogue system that enables the robot to act proactively without becoming intrusive or repetitive.
The developed technologies were validated in two real-world scenarios: a smart kitchen, where the robot assists with cooking tasks (e.g., handing over objects, reminding users of cooking steps, monitoring the stove), and a smart hotel, where the robot greets guests, guides them, provides information, and supports them with everyday needs. Person safety was ensured through GPU-based 3D safety fields and dynamic collision detection.
Role of the FZI
FZI Research Center for Information Technology served as consortium leader and the largest research partner within the AuRorA project. In this dual role, FZI was responsible both for the scientific and technical development of core system components and for coordinating all project partners — including organizing regular meetings, telephone conferences, and defining shared technical interfaces.
On the technical side, FZI focused on two key areas: first, the development of the adaptive, learning robot behavior based on Behavior Trees — including the learning fallback node, which uses neural networks to make context-sensitive decisions and continuously improves through interaction experience. Second, FZI led the detection of non-verbal communication, covering real-time analysis of the user's posture, facial expressions, gestures, and gaze direction.
Beyond these core contributions, FZI was involved in system specification, the development of ELSI criteria (ethics, legal compliance, and data protection), the implementation of robotic assistance functions, and the final system evaluation. FZI produced data flow diagrams for risk analysis and provided the technical foundation for the project's ethics and data protection framework. The overall integration of all subsystems into a coherent, functioning whole was likewise FZI's responsibility.
On the technical side, FZI focused on two key areas: first, the development of the adaptive, learning robot behavior based on Behavior Trees — including the learning fallback node, which uses neural networks to make context-sensitive decisions and continuously improves through interaction experience. Second, FZI led the detection of non-verbal communication, covering real-time analysis of the user's posture, facial expressions, gestures, and gaze direction.
Beyond these core contributions, FZI was involved in system specification, the development of ELSI criteria (ethics, legal compliance, and data protection), the implementation of robotic assistance functions, and the final system evaluation. FZI produced data flow diagrams for risk analysis and provided the technical foundation for the project's ethics and data protection framework. The overall integration of all subsystems into a coherent, functioning whole was likewise FZI's responsibility.
Contact person
Staff
Division: Embedded Systems and Sensors Engineering
Headquarters Karlsruhe
Research focus
Applied Artificial Intelligence
In this research focus, the FZI concentrates on practical research into the key technology of Artificial Intelligence (AI). Innovative AI solutions are developed and transferred to application areas such as mobility, robotics, healthcare technology, logistics, production, and supply and disposal on behalf of our partners and customers.
Funding notice:
The Project AuRorA was funded by the Federal Ministry of Research, Technology and Space.
Project partners: