We propose a methodology for estimating human behaviors in psychotherapy sessions using multi-label and multi-task learning paradigms. We discuss the problem of behavioral coding in which data of human interactions are annotated with labels to describe…
Several lexica for sentiment analysis have been developed; while most of these come with word polarity annotations (e.g., positive/negative), attempts at building lexica for finer-grained emotion analysis (e.g., happiness, sadness) have recently attrac…
In social interactions between humans and Embodied Conversational Agents (ECAs) conversational interruptions may occur. ECAs should be prepared to detect, manage and react to such interruptions in order to keep the interaction smooth, natural and believable. In this paper, we examined nonverbal reactions exhibited by an interruptee during conversational interruptions and we propose a novel technique driven by an evolutionary algorithm to build a computational model for ECAs to manage user’s interruptions. We propose a taxonomy of conversational interruptions adapted from social psychology, an annotation schema for semi-automatic detection of user’s interruptions and a corpus-based observational analysis of human nonverbal reactions to interruptions. Then we present a methodology for building an ECA behavioral model including the design and realization of an interactive study driven by an evolutionary algorithm, where participants interactively built the most appropriate set of multimodal reactive behaviours for an ECA to display interpersonal attitudes (friendly/hostile) through nonverbal reactions to a conversational interruption.
Crowdsourcing is a popular approach to collect annotations for unlabeled data instances. It involves collecting a large number of annotations from several, often naive untrained annotators for each data instance which are then combined to estimate the …
Measurement of physiological reactions to persuasive messages can improve our understanding of psychological processes of persuasion, and potentially further enhance and personalize current persuasion interventions. However, little is known about the relationship between psychophysiology and persuasive processes. This study focused on four persuasion principles: scarcity, commitment, consensus, and authority, and people’s susceptibility to them. Physiological measures included the cardiovascular, respiratory, and electrodermal system, as well as facial motor systems. Psychological measures consisted of self-reported attitude towards oral care and susceptibility to persuasion (STPS). We performed a randomized within-subject experiment in which fifty-six participants viewed persuasive messages deploying the aforementioned persuasion principles to improve their oral care. Results indicated different physiological patterns during persuasion versus rest. We found no different physiological patterns in exposure to distinct persuasion principles, nor a clear correlation with susceptibility to individual persuasion principles. However, mixed model analysis illustrated that overall STPS scores help explain variance in reactivity of skin conductance level and skin conductance response, and reactivity in the zygomaticus major: lower susceptibility relates to higher reactivity. Summarizing, we have found no conclusive support for distinct psychophysiological patterns associated with different persuasion principles, although overall susceptibility seems to be reflected in physiology to some extent.
We propose a multi-task ensemble framework that jointly learns multiple related problems. The ensemble model aims to leverage the learned representations of three deep learning models (i.e., CNN, LSTM and GRU) and a hand-crafted feature representation …
This review investigates the effects of psychological stress on the human body measured through biosignals. When a potentially threatening stimulus is perceived, a cascade of physiological processes occurs mobilizing the body and nervous system to conf…
Emotion shapes all aspects of our interpersonal and intellectual experiences. Its automatic analysis has therefore many applications. In this paper, we propose an emotional tonal speech dataset, Mandarin Chinese Emotional Speech Dataset-Portrayed (MES-…
The problem of continuous emotion recognition has been the subject of several studies. The proposed affective computing approaches employ sequential machine learning algorithms for improving the classification stage, accounting for the time ambiguity o…
In recent years, emotion recognition has become a research focus in the area of artificial intelligence. Due to its irregular structure, EEG data can be analyzed by applying graphical based algorithms or models much more efficiently. In this work, a Gr…