Published September 1, 2018 | Version v1
Journal article Restricted

MixedEmotions: An Open-Source Toolbox for Multi-Modal Emotion Analysis

Authors/Creators

  • 1. National University of Ireland Galway
  • 2. University of Passau
  • 3. GSI Universidad Politécnica de Madrid
  • 4. Paradigma Digital
  • 5. Deutsche Welle
  • 6. Expert Systems
  • 7. Brno University of Technology
  • 8. Phonexia
  • 9. Siren Solutions

Description

Recently, there is an increasing tendency to embed functionalities for recognizing emotions from user-generated media content in automated systems such as call-centre operations, recommendations, and assistive technologies, providing richer and more informative user and content profiles. However, to date, adding these functionalities was a tedious, costly, and time-consuming effort, requiring identification and integration of diverse tools with diverse interfaces as required by the use case at hand. The MixedEmotions Toolbox leverages the need for such functionalities by providing tools for text, audio, video, and linked data processing within an easily integrable plug-and-play platform. These functionalities include: 1) for text processing: emotion and sentiment recognition; 2) for audio processing: emotion, age, and gender recognition; 3) for video processing: face detection and tracking, emotion recognition, facial landmark localization, head pose estimation, face alignment, and body pose estimation; and 4) for linked data: knowledge graph integration. Moreover, the MixedEmotions Toolbox is open-source and free. In this paper, we present this toolbox in the context of the existing landscape, and provide a range of detailed benchmarks on standard test-beds showing its state-of-the-art performance. Furthermore, three real-world use cases show its effectiveness, namely, emotion-driven smart TV, call center monitoring, and brand reputation analysis.

Files

Restricted

The record is publicly accessible, but files are restricted to users with access.

Additional details

Funding

European Commission
MixedEmotions - Social Semantic Emotion Analysis for Innovative Multilingual Big Data Analytics Markets 644632