Difference between revisions of "Movement Recognition"

From BlackBox
Jump to navigation Jump to search
(Sensor-based considerations in LMA recognition)
Line 21: Line 21:
  
 
(Note: Dynamic conditional random fields and Boltzmann machines were suggested by AAAI reviewers for any future versions of [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect EffortDetect].)
 
(Note: Dynamic conditional random fields and Boltzmann machines were suggested by AAAI reviewers for any future versions of [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect EffortDetect].)
 
= Sensor-based considerations in LMA recognition =
 
 
Different types of sensors are able to measure different low-context properties of human movement, such as the position, velocity, and acceleration of specific body parts. By applying computational techniques to these measurements, we can infer higher-context properties, such as gait information (stride length, walking speed), postural changes (falling, standing up), and energy expenditure. What has been underexplored is how these can properties can be used to recognize higher-context qualities of human movement for creating a semantics of expressive motion. One such semantic framework is Laban Movement Analysis (LMA). On this page, we enumerate some existing sensor types and how these can be used towards movement analysis, and show they could be used towards the application of LMA. This table places the ''embodied experience of movement'' as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the ''qualia ''of moving when combined with conventional mechanisms for sensing movement.
 
 
{| border="1"
 
|
 
<center>'''Type of sensor'''</center>
 
|
 
<center>'''Example'''</center>
 
|
 
<center>'''Commercially available products'''</center>
 
|
 
<center>'''Fundamental<span> </span>movement phenomena sensed'''</center><center>'''(movement primitive)'''</center>
 
|
 
<center>'''Mid-level movement feature (inferred from movement primitives)'''</center>
 
|
 
<center>'''Higher-level semantics based on LMA (inferred from mid-level features)'''</center>
 
|-
 
| rowspan="2" |
 
Acceleration
 
|
 
Gyroscopes
 
|
 
iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus
 
|
 
Rotational accleration
 
| rowspan="2" |
 
Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)
 
| rowspan="2" |
 
A prototype for recognizing LMA Effort using accelerometers is being developed by the Institute for Advanced Computing Applications and Technologies at the University of Illinois and the University of Illinois Dance Department, with the expertise of movement analyst Sarah Hook from the Dance Department, and in collaboration with Dr. Thecla Schiphorst <span><span>(Subyen, Maranan, Schiphorst, Pasquier, & Bartram, 2011)</span></span>.
 
|-
 
|
 
Accelerometers
 
|
 
Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote
 
|
 
Linear acceleration
 
|-
 
|
 
Touch
 
|
 
Pressure sensors
 
|
 
Tactex
 
|
 
Pressure
 
|
 
 
|
 
LMA recognition of has been applied to touch-based interfaces in interactive art (T. Schiphorst, Lovell, & Jaffe, 2002).
 
|-
 
| rowspan="3" |
 
Position
 
|
 
Vision
 
|
 
Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems
 
| rowspan="3" |
 
Position of body segments
 
| rowspan="3" |
 
Postural information; anything that can be inferred from acceleration sensors
 
| rowspan="3" |
 
Recognition of some aspects of Space, Space, and Effort categories have been reported (J. Rett, J. Dias, & Ahuactzin, 2008; J. Rett, Santos, & J. Dias, 2008; Jorg Rett & Jorge Dias, 2007a, 2007b; Santos, Prado, & J. Dias, 2009; Santos et al., 2009; Swaminathan et al., 2009; L. Zhao, 2001; Liwei Zhao & Badler, 2005).<br />
 
|-
 
|
 
Magnetic
 
|
 
 
|-
 
|
 
Infrared
 
|
 
 
|-
 
| rowspan="5" |
 
Biometric
 
|
 
Eye-tracking
 
|
 
 
|
 
Gaze
 
|
 
Visual attention; intent
 
| rowspan="5" |
 
Muscular tension is related to Effort Weight. Attention and intent are key themes in Effort Space. We propose that arousal can be affined to the extent by which a mover uses �fighting� qualities over �indulging� qualities.
 
|-
 
|
 
GSR
 
|
 
 
|
 
Electrical conductance of the skin
 
|
 
Arousal
 
|-
 
|
 
Breath sensors
 
|
 
 
|
 
Rate of breathing; volume of inspiration/expiration
 
|
 
Arousal; energy expenditure
 
|-
 
|
 
EMG
 
|
 
 
|
 
Electrical activity produced by skeletal muscles
 
|
 
Muscular tension
 
|-
 
|
 
Heart rate sensors
 
|
 
 
|
 
Heart rate
 
|
 
Arousal; level of physical activity
 
|}
 
  
 
= References =
 
= References =

Revision as of 12:23, 23 December 2011

Movement recognition in the Blackbox

The Blackbox currently supports Laban Basic Effort recognition through the EffortDetect system. See also the discussion below on LMA recognition.

Movement recognition outside the Blackbox

In general, a variety of heuristic techniques are applicable to movement information derived from sensor data. The following techniques below represent some that have been reported in the literature:

  • Frequency-domain analysis (Yang & Hsu, 2010)
    • Analysis of variance
    • Analysis of frequency peaks
    • Discrete wavelet transform(Sekine, Tamura, Togawa, & Fukui, 2000)
    • Signal magnitude area (Karantonis, Narayanan, M. Mathie, Lovell, & Celler, 2006)
  • Statistical approaches (Yang & Hsu, 2010)
    • Decision trees (M. J. Mathie, Celler, Lovell, & Coster, 2004)
    • k-nearest neighbor
    • support vector machines
    • Naïve Bayes classifier
    • Gaussian mixture model
    • Hidden Markov models
    • Dynamic Conditional Random Field (Morency, Quattoni, & Darrell, 2007)
    • Boltzmann machines (Taylor & Hinton, 2009)

(Note: Dynamic conditional random fields and Boltzmann machines were suggested by AAAI reviewers for any future versions of EffortDetect.)

References

Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.

Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.

Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on �Concept Learning for Embodied Agents.� Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf

Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436

Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564

Subyen, P., Maranan, D. S., Schiphorst, T., Pasquier, P., & Bartram, L. (2011). EMVIZ: The Poetics of Movement Quality Visualization. Proceedings of Computational Aesthetic 2011 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging. Presented at the Computational Aesthetics, Vancouver, Canada.

Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.

Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.

Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002