Movement Recognition

Movement recognition in the Blackbox

The Blackbox currently supports Laban Basic Effort recognition through the EffortDetect system. See also the discussion below on LMA recognition.

Movement recognition outside the Blackbox

In general, a variety of heuristic techniques are applicable to movement information derived from sensor data. The following techniques below represent some that have been reported in the literature:

• Frequency-domain analysis (Yang & Hsu, 2010)
  • Analysis of variance
  • Analysis of frequency peaks
  • Discrete wavelet transform(Sekine, Tamura, Togawa, & Fukui, 2000)
  • Signal magnitude area (Karantonis, Narayanan, M. Mathie, Lovell, & Celler, 2006)
• Statistical approaches (Yang & Hsu, 2010)
  • Decision trees (M. J. Mathie, Celler, Lovell, & Coster, 2004)
  • k-nearest neighbor
  • support vector machines
  • Naïve Bayes classifier
  • Gaussian mixture model
  • Hidden Markov models
  • Dynamic Conditional Random Field (Morency, Quattoni, & Darrell, 2007)
  • Boltzmann machines (Taylor & Hinton, 2009)

(Note: Dynamic conditional random fields and Boltzmann machines were suggested by AAAI reviewers for any future versions of EffortDetect.)

Sensor-based considerations in LMA recognition

Different types of sensors are able to measure different low-context properties of human movement, such as the position, velocity, and acceleration of specific body parts. By applying computational techniques to these measurements, we can infer higher-context properties, such as gait information (stride length, walking speed), postural changes (falling, standing up), and energy expenditure. What has been underexplored is how these can properties can be used to recognize higher-context qualities of human movement for creating a semantics of expressive motion. One such semantic framework is Laban Movement Analysis (LMA). On this page, we enumerate some existing sensor types and how these can be used towards movement analysis, and show they could be used towards the application of LMA. This table places the embodied experience of movement as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the qualia of moving when combined with conventional mechanisms for sensing movement.

Type of sensor	Example	Commercially available products	Fundamental movement phenomena sensed(movement primitive)	Mid-level movement feature (inferred from movement primitives)	Higher-level semantics based on LMA (inferred from mid-level features)
Acceleration	Gyroscopes	iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus	Rotational accleration	Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)	A prototype for recognizing LMA Effort using accelerometers is being developed by the Institute for Advanced Computing Applications and Technologies at the University of Illinois and the University of Illinois Dance Department, with the expertise of movement analyst Sarah Hook from the Dance Department, and in collaboration with Dr. Thecla Schiphorst (Subyen, Maranan, Schiphorst, Pasquier, & Bartram, 2011).
	Accelerometers	Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote	Linear acceleration
Touch	Pressure sensors	Tactex	Pressure		LMA recognition of has been applied to touch-based interfaces in interactive art (T. Schiphorst, Lovell, & Jaffe, 2002).
Position	Vision	Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems	Position of body segments	Postural information; anything that can be inferred from acceleration sensors	Recognition of some aspects of Space, Space, and Effort categories have been reported (J. Rett, J. Dias, & Ahuactzin, 2008; J. Rett, Santos, & J. Dias, 2008; Jorg Rett & Jorge Dias, 2007a, 2007b; Santos, Prado, & J. Dias, 2009; Santos et al., 2009; Swaminathan et al., 2009; L. Zhao, 2001; Liwei Zhao & Badler, 2005).
	Magnetic
	Infrared
Biometric	Eye-tracking		Gaze	Visual attention; intent	Muscular tension is related to Effort Weight. Attention and intent are key themes in Effort Space. We propose that arousal can be affined to the extent by which a mover uses �fighting� qualities over �indulging� qualities.
	GSR		Electrical conductance of the skin	Arousal
	Breath sensors		Rate of breathing; volume of inspiration/expiration	Arousal; energy expenditure
	EMG		Electrical activity produced by skeletal muscles	Muscular tension
	Heart rate sensors		Heart rate	Arousal; level of physical activity

References

Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.

Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.

Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on �Concept Learning for Embodied Agents.� Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf

Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436

Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564

Subyen, P., Maranan, D. S., Schiphorst, T., Pasquier, P., & Bartram, L. (2011). EMVIZ: The Poetics of Movement Quality Visualization. Proceedings of Computational Aesthetic 2011 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging. Presented at the Computational Aesthetics, Vancouver, Canada.

Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.

Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.

Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002