Selected Publications
Body Talk: Crowdshaping Realistic 3D Avatars with Words
Abstract
Realistic, metrically accurate, 3D human avatars are useful for games, shopping, virtual reality, and health applications. Such avatars are not in wide use because solutions for creating them from high-end scanners, low-cost range cameras, and tailoring measurements all have limitations. Here we propose a simple solution and show that it is surprisingly accurate. We use crowdsourcing to generate attribute ratings of 3D body shapes corresponding to standard linguistic descriptions of 3D shape. We then learn a linear function relating these ratings to 3D human shape parameters. Given an image of a new body, we again turn to the crowd for ratings of the body shape. The collection of linguistic ratings of a photograph provides remarkably strong constraints on the metric 3D shape. We call the process crowdshaping and show that our Body Talk system produces shapes that are perceptually indistinguishable from bodies created from high-resolution scans and that the metric accuracy is sufficient for many tasks. This makes body “scanning” practical without a scanner, opening up new applications including database search, visualization, and extracting avatars from books.
Considering cross-cultural context in the automatic recognition of emotions
(doi 10.1007/s13042-013-0192-2)
Abstract
Automatic recognition of emotions remains an ongoing challenge and much effort is being invested towards developing a system to solve this problem. Although several systems have been proposed, there is still none that considers the cultural context for emotion recognition. It remains unclear whether emotions are universal or culturally specific. A study on how culture influences the recognition of emotions is presented. For this purpose, a multicultural corpus for cross-cultural emotion analysis is constructed. Subjects from three different cultures—American, Asian and European—are recruited. The corpus is segmented and annotated. To avoid language artifacts, the emotion recognition model considers facial expressions, head movements, body motions and dimensional emotions. Three training and testing paradigms are carried out to compare cultural effects: intra-cultural, cross-cultural and multicultural emotion recognition. Intra-cultural and multicultural emotion recognition paradigms raised the best recognition results; cross-cultural emotion recognition rates were lower. These results suggest that emotion expression varies by culture, representing a hint of emotion specificity.
Cultural dimension in emotion recognition for human machine interaction
Abstract
Human emotion recognition is a multidimensional task. In this paper we study the effect of the cultural dimension in emotion recognition models and their use in human computer interaction. We prepared two experiments to analyze the consequences of disregarding culture in emotion recognition models. The results show that failing to consider the user's culture while applying emotion recognition techniques in interaction scenarios decreases the system's performance, making the emotional input meaningless and detrimental to the system and interaction.
Towards Developing Robust Multimodal Databases for Emotion Analysis
Abstract
Understanding emotions can make the difference between succeeding and failing during communication. Several systems have been developed in the field of Affective Computing in order to understand emotions. Recently these systems focus into multimodal emotion recognition. The basis of each of these systems is emotion databases. Even though a lot of attention has been placed in capturing spontaneous emotion expressions, building an emotion database is a task with several challenges that are commonly neglected, namely: quality of the recordings, ground truth, multiple device recording, data labeling and context. In this paper we present a new spontaneous emotion database, with human-computer and human to human interactions. This database is composed by eight different synchronized signals, in four interaction tasks. Strategies on how to deal with emotion database construction challenges are explained in detail.
A non-invasive multi-sensor capturing system for human physiological and behavioral responses analysis
Abstract
We present a new noninvasive multi-sensor capturing system for recording video, sound and motion data. The characteristic of the system is its 1msec. order accuracy hardware level synchronization among all the sensors as well as automatic extraction of variety of ground truth from the data. The proposed system enables the analysis of the correlation between variety of psychophysiological model (modalities), such as facial expression, body temperature changes, gaze analysis etc... . Following benchmarks driven framework principles, the data captured by our system is used to establish benchmarks for evaluation of the algorithms involved in the automatic emotions recognition process.
Assessing Emotions in a Cross-Cultural Context
Abstract
Emotion recognition systems could support professionals in a wide range of areas. Several work in emotion recognition has been carried out in the last decades, yet few attention has been payed to cross-culture context emotion recognition. Multimodal emotional expressions from 36 subjects with different cultural backgrounds were collected. In the experiment, participants observed and assessed emotional images in a 5 point positive and negative emotional valence scale. This information was used as ground truth for the recorded information. The dataset was segmented for all the participants and partially labeled for 8 of them, for a total of 160 segments. Recognition of positive and negative emotions was obtained from the dataset suggesting agreement points in expression of emotion between cultures.
Benchmark Driven Framework for Development of Emotion Sensing Support Systems
Abstract
Emotion sensing support system to assist human decision making during interview scenario is a developing research field. This paper presents a new framework for the development of emotion sensing support systems that is a complete, easily extendible, flexible, and configurable environment with intensive benchmark capabilities. The design of the framework was inspired by behavior-driven development, agile software development technique. It provides: (1) effective collaboration platform between technological and psychological researches, and (2) intensive benchmarking capabilities to test the performance of the entire system as well as individual algorithms.