Paper Blogs 04

Reference Paper

Towards Real-Time Affect Detection Based on Sample Entropy Analysis of Expressive Gesture

Donald Glowinski and Maurizio Mancini

(Digital Object Identifier 10.1007/978-3-642-24600-5_56)

Overview of the Paper

During human-human communication, body movements play a vital role of affective information in nonverbal communication. Different affect detection systems have been developed based on movement direction and kinematics and arm extensions. In this paper, the authors propose a real-time affect detection system based on Sample Entropy (SampEn) method. 

Most of the methods developed to analyze behavior dynamics is fail to handle two main properties of human movements, they are non-linearity and non-stationarity. In this paper the authors try to handle these two properties of human movements. Their model is based on Camurri et al.’s [2] framework of expressive gesture analysis. According to Camurri et al. gesture analysis is performed in three steps,

      1. Low-level physical measures

      2. Overall gesture features

      3. High-level information

In this paper the authors try to focus on first two part of the above framework.

Microsoft Kinect is used here to capture the RGB image here. Three parts are considered here to form the bounding triangle; they are head, left hand and right hand. To extract the dynamic features of this triangle, two indices are used, smoothness index (SmI) and symmetry Index (SyI).

Smoothness index is computed using the curvature and velocity of the movements of left and right part. After computing the left and right smoothness index, the overall smoothness index is calculated by averaging these two values. Symmetry index is calculated from the horizontal and vertical symmetry value of the bounding triangle. The authors derive a dynamic updating formula for SyI and SmI. From these SyI and SmI value, the SampEn value is calculated. If the hand movements are symmetric and smooth for several frames, then the SmapEn value will be zero. On the other hand, if the movements are not symmetric or not smooth, then value of SampEn will be greater than zero. 

Evaluation and Validity of the Paper

One sample output is available for download here. In the following figure, as the movements are smooth in between t₁ and t­₂, so the SampEn(SmI) is zero. But, as there are some abrupt movement is between t₂ and t₃, SampEn(SmI) increases. Similarly, as the hands symmetry change in between t₄ and t₅, SampEn(SyI) increases. And, as the symmetry is not changes in t₅ and t₆, SampEn(SyI) value becomes zero.

 

Improvement Scopes

In my opinion, the future work should include incorporating these features to detect the high-level information in human-human communication. Also, next extension should include other 3D analysis like forward and backward movements, distance from the camera etc.

 

Further Reading

One of the interesting articles, which are cited by this paper, is “Communicating Expressiveness and Affect in Multimodal Interactive Systems”, by A. Camurri, G. Volpe, G. De Poli, M. Leman [2] (Digital Object Identifier: 10.1109/MMUL.2005.2). In the cited article, the authors preset the frame work for expressive gesture analysis.

[1] D. Glowinski and M. Mancini, “Towards real-time affect detection based on sample entropy analysis of expressive gesture,” Affective Computing and Intelligent Interaction, pp. 527–537, 2011. 

[2] A. Camurri, G. Volpe, G. De Poli, and M. Leman, “Communicating expressiveness and affect in multimodal interactive systems,” MultiMedia, IEEE, vol. 12, no. 1, pp. 43 – 53, 2005.