An overview of the conceptual model of the Virtual Support Partner (VSP) system is provided in the Figure below. The different main elements of the VSP include:

  • Input Devices: This is the physical part of the platform. It accepts real time inputs from different modalities and sensors installed in the environment of the user. Cameras, 3D depth sensors (Kinect), microphones and touch screens collect all the necessary inputs for the emotional multimodal interaction of the user with the VSP.
  • Real time multimodal processing: This layer contains the necessary modules to recognize and interpret the input data, process it and indicate the meaning to the intelligent modules in order to generate a coherent emotional response to the user.
  • Reasoning, Decision Making: This is the “intelligent” layer of the platform, consisting of its reasoning and decision taking modules. These rely on inputs from the previous layer, to determine what the system should communicate to the user whether through the avatar or the different services.
  • Application Layer: This layer contains a set of ICT services to support the user and a Social network to ensure the exchange of information with other but also with formal and informal caregivers.
  • Output Generator: This contains modules to visualize the VSP to the environment through the physical parts of system (3D graphics rendering and speech synthesizer).
  • Knowledge base: In order to produce a coherent response the system includes an associated knowledge base that will serve as data management server where all data (e.g., information models, monitoring data, user profiles etc.) will be stored and disseminated to cooperating services.