Paper Blogs 09

Full body expressivity analysis in 3D Natural Interaction: a comparative study

George Caridakis and Kostas Karpouzis

Overview of the Paper

Non-verbal behavior and communication are very important research topic in the field of psychology and cognitive science. Measuring expressive characteristics of body motion, posture, gestures qualitatively are primarily focused on human-human communication. These methods also can be extended to human-computer interaction. In order to model expressivity properly, many researchers studied human movement closely and categorized them mainly in five binary categories. They are slow/fast, restricted/wide, weak/strong, small/big, unpleasant/pleasant. These expressivity dimensions are selected as the most complete approach to body expressivity modeling. This paper presents preliminary research work on defining and extracting full body expressivity features.

    In this paper, the authors consider five parameters for modeling behavioral expressivity. They are,

      1. Overall activation 

      2. Spatial extent 

      3. Temporal expressivity 

      4. Fluidity 

      5. Power

To formulate the full body expressivity feature, the authors define the body pose P as

P = [l, r, S, D, F, J]

Where, l and r are the 3D coordinates of left and right hand respectively,

                S is binary silhouette image,

                D is the depth image,

                F is the face information, and

                J is skeleton joint of left and right arm.

Using this information, three different methods for formulating expressivity are presented in this paper. They methods are named as silhouette method, limbs method and joints method. From these methods, the expressiveness parameters are modeled.

 

Evaluation and Validity of the Paper

In this paper, the authors only present the initial work. But, they try to formalize the framework of expressivity measure. Some initial phase result is shown here. Initial dataset was generated by capturing the video of four users by Microsoft Kinect. Sample images are shown in figure 1.

They S (silhouette) and D (depth) image result are shown in figure 2.

From the S and D image, the Joint (J) are calculated and shown in figure 3. 

 

Improvement Scopes

One extension of this paper will be to implement all the methods describes in this framework to see how it works. Then it needs to be validated against some other proposed approaches. Also it needs to be validated by experience movement analyst for considering it as a valid framework.

 

Further Reading

One of the interesting articles, which are cited by this paper, is “How to Distinguish Posed from Spontaneous Smiles using Geometric Features”, by Michel F. Valstar, Hatice Gunes, and Maja Pantic [2] (Digital Object Identifier: 10.1145/1322192.1322202). In this paper, the authors used geometric approach for automatic detection of posed and spontaneous expression. They fused head, shoulder and face modalities to distinguish between two smiles.

[1] G. Caridakis and K. Karpouzis, “Full body expressivity analysis in 3D natural interaction: a comparative study,” in Affective Interaction in Natural Environments workshop, ICMI 2011 International Conference on Multimodal Interaction. 14-18th November 2011, Alicante, Spain, 2011.

[2] M. Valstar, H. Gunes, and M. Pantic, “How to distinguish posed from spontaneous smiles using geometric features,” in Proceedings of the 9^th international conference on Multimodal interfaces. ACM, 2007, pp 38–45.