SIQAS
Social Interaction Quantification using Acoustic Sensing
Start: 03/2024
End: 08/2025
Social isolation and loneliness pose growing challenges to mental health. A key challenge in clinical practice and research is accurately measuring the extent of social interactions. Previous approaches have mostly relied on the subjective perceptions of those affected, using questionnaires or diaries, which often leads to distorted results. The SIQAS (Social Interaction Quantification using Acoustic Sensing) project is developing a technological alternative for this purpose: an app-based solution that makes social interactions in everyday life measurable in an objective, privacy-compliant, and unbiased manner.
Through AI-supported analysis of audio signals, the system identifies interaction patterns, such as the intensity of conversation participation or the emotions experienced in various contexts. The goal is to create a reliable database for research, early detection, and the development of individualized therapeutic approaches. SIQAS combines insights from computer science with requirements from psychology and medicine. In doing so, the project provides a foundation for mobile assessment tools that can be used directly in users’ daily lives to reveal changes in interaction behavior.
The SIQAS project is dedicated to the technological challenge of translating complex human social behavior into measurable data. The focus is not on analyzing the content of conversations, but rather on extracting structural features of the interaction. At the heart of the project is the development of an AI system capable of analyzing social interactions and associated emotions in real time directly on mobile devices. SIQAS’s innovative approach utilizes state-of-the-art AI models. These architectures are specialized in recognizing patterns in audio data that allow inferences to be drawn about social activity.
Throughout the project, both real and synthetically generated training data are used to prepare the algorithms for a variety of scenarios. The AI analyzes specific parameters such as conversation intensity, the diversity of interaction partners, and the context-dependence of behavior. Particular attention is paid to data protection-compliant processing: The analysis takes place locally on the smartphone (“on-device”), so that no sensitive audio data needs to leave the protected environment.
By combining objective data collection with real-time processing, SIQAS aims to create a robust database. For health research, SIQAS marks the beginning of a new field in mobile diagnostics, where intelligent audio processing serves as an early warning system for mental health crises.
In the SIQAS project, the FZI Research Center for Information Technology serves as the technological pioneer and is primarily responsible for developing the core AI components. The FZI’s expertise is primarily applied to the field of applied artificial intelligence. A key focus of the work is the selection and preparation of training data. Since the quality of an AI depends largely on its data, the FZI develops strategies to prepare both real-world interaction data and synthetic datasets in a way that enables robust model development.
In addition, the FZI is responsible for the design and training of specific AI models. In this process, complex architectures such as transformers and neural networks are evaluated and tailored to the specific requirements of emotion and interaction recognition. A particular challenge that the FZI is addressing is the performance optimization of the algorithms. Since the analysis is to take place directly on mobile devices, the models must be highly efficient in order to conserve hardware resources (such as battery life and computing power) without sacrificing accuracy.
Finally, the FZI will handle the integration and evaluation of the developed models within a controlled test environment. This technical approach is intended to ensure that the theoretical innovations of AI research are translated into a practical, secure, and high-performance application that meets the stringent requirements.
Applied Artificial Intelligence
In this research focus, the FZI concentrates on practical research into the key technology of Artificial Intelligence (AI). Innovative AI solutions are developed and transferred to application areas such as mobility, robotics, healthcare technology, logistics, production, and supply and disposal on behalf of our partners and customers.
