Difference between revisions of "Movement Capture"

From BlackBox
Jump to navigation Jump to search
(HuMoS: An integrated approach to sharing movement data)
(Are you going to be tracking very small movements?)
 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
= <span id="Motion_capture_in_the_BlackBox" class="mw-headline">Motion capture in the BlackBox</span> =
+
= Introduction =  
 
+
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use.
 
 
 
== <span id="The_systems" class="mw-headline"> The systems </span> ==
 
  
 
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
 
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
Line 9: Line 6:
 
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]
 
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]
  
== <span id="What_type_of_sensor_is_best.3F" class="mw-headline"><span>What type of sensor is best?</span></span> ==
+
In addition, [http://wiki.iat.sfu.ca/BlackBox/index.php/HuMoS HuMoS] is a suggested integrated approach to sharing movement data and is a work in progress.
 +
 
 +
This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing [[Movement_Recognition| movement recognition]].
 +
 
 +
= What type of sensor is best? =
  
 
What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:
 
What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:
  
* Low-frequency movement is best captured using measurements of position (the Vicon or the Kinect will be your best)
+
* '''Low-frequency movement''' is best captured using measurements of position (the Vicon or the Kinect will be your best)
* Intermediate-frequency movement is best captured using measurements of velocity
+
* '''Intermediate-frequency''' movement is best captured using measurements of velocity
* High-frequency movement is best captured using measurements of acceleration (accelerometers will be appropriate) Generally, human movement tends to be slow enougho that intermediate
+
* '''High-frequency''' movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on [[Kinect#Errors_in_using_the_Kinect_for_motion_tracking| errors in using the Kinect for motion tracking]] for a discussion on when position-based sampling fails.
  
== Kinect versus Vicon ==
+
= Kinect versus Vicon =
  
The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal:
+
The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.
  
=== <span id="What_kind_of_movement_do_you_need_to_capture.3F" class="mw-headline">What kind of movement do you need to capture?</span> ===
+
== What kind of movement do you need to capture? ==
  
 
What kind of movement do you need to capture? Consider the following.
 
What kind of movement do you need to capture? Consider the following.
  
==== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ====
+
<div style = "background-color: #ddd; padding: 10px">
 +
=== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ===
  
 
If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.
 
If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.
  
==== Are you capturing movements of the '''entire body''' or only '''some body parts''' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) <span>'''and'''</span> small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ====
+
=== Are you capturing movements of the ''entire body'' or only ''some body parts'' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) ''and'' small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ===
  
In his seminal book on acting, <span>''An Acrobat of the Heart''</span>, theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as <span>''plastiques''</span> and <span>''corporeals''</span>. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon. <br /><br /><span>'''Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing) or will the body stay more or less in one spot (within a space of about 5 feet by 5 feet)? '''</span><br /><br /> If so, then you must use the Vicon. <br />
+
In his seminal book on acting, ''An Acrobat of the Heart'', theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as ''plastiques'' and ''corporeals''. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.  
  
==== Are you going to be tracking movement in the transverse plane of the body? ====
+
===Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing)? ===
  
These movements include
+
If so, then you must use the Vicon. Otherwise, if the body will within a space of about 5 feet by 5 feet, you might be able to stick with the Kinect.
  
* supination and pronation of the legs or arms?
+
=== Are you going to be tracking movement in the transverse plane of the body? ===
* left or right rotation of the head?
 
* twisting of the spine?
 
  
These movements <span>''cannot''</span> be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.
+
These movements include...
  
==== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ====
+
* supination and pronation of the legs or arms
 +
* left or right rotation of the head
 +
* twisting of the spine
  
These movements <span>''cannot''</span> be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine.<br /><br /> Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?<br /> The change in front <span>''cannot''</span> be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.<br />
+
These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.
  
==== Are you going to be tracking very fast movements? ====
+
=== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ===
  
The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].
+
These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine.<br /><br />
  
==== Are you going to be tracking very small movements? ====
+
===Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?===
 +
<br /> The change in front ''cannot'' be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.<br />
  
See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking] to learn about considerations you need to take into account if you intend to track small movements.<br />
+
=== Are you going to be tracking very fast movements? ===
  
=== <span id="How_much_time_do_you_have.3F" class="mw-headline">How much time do you have to plan and set up your motion capture session?</span> ===
+
The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].
  
If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.
+
=== Are you going to be tracking very small movements? ===
  
=== <span id="Do_you_need_to_process_the_movement_data_in_real-time.3F" class="mw-headline"><span>Do you need to process the movement data in real-time?</span></span> ===
+
See the section on [[Kinect#Spatial_resolution_errors| spatial resolution errors for the Kinect]] to learn about considerations you need to take into account if you intend to track small movements.<br />
  
The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.
+
</div>
  
= <span id="Movement_recognition_in_the_Blackbox" class="mw-headline">Movement recognition in the Blackbox </span> =
+
== How much time do you have to plan and set up your motion capture session? ==
The Blackbox currently supports Laban Basic Effort recognition through the [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect EffortDetect] system.
 
  
In general, a variety of heuristic techniques are applicable to movement information derived from sensor data. The following techniques below represent some that have been reported in the literature:
+
If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.
  
* Frequency-domain analysis (Yang & Hsu, 2010)
+
== Do you need to process the movement data in real-time?==
** Analysis of variance
 
** Analysis of frequency peaks
 
** Discrete wavelet transform(Sekine, Tamura, Togawa, & Fukui, 2000)
 
** Signal magnitude area (Karantonis, Narayanan, M. Mathie, Lovell, & Celler, 2006)
 
* Statistical approaches (Yang & Hsu, 2010)
 
** Decision trees (M. J. Mathie, Celler, Lovell, & Coster, 2004)
 
** k-nearest neighbor
 
** support vector machines
 
** Naïve Bayes classifier
 
** Gaussian mixture model
 
** Hidden Markov models
 
** Dynamic Conditional Random Field (Morency, Quattoni, & Darrell, 2007)
 
** Boltzmann machines (Taylor & Hinton, 2009)
 
  
(Note: Dynamic conditional random fields and Boltzmann machines were suggested by AAAI reviewers for any future versions of [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect EffortDetect].)
+
The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.
 
 
= References =
 
 
 
* Karantonis, D. M., Narayanan, M. R., Mathie, M., Lovell, N. H., & Celler, B. G. (2006). Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. Information Technology in Biomedicine, IEEE Transactions on, 10(1), 156–167.
 
* Mathie, M. J., Celler, B. G., Lovell, N. H., & Coster, A. C. F. (2004). Classification of basic daily movements using a triaxial accelerometer. Medical & Biological Engineering & Computing, 42(5), 679-687. doi:10.1007/BF02347551
 
* Morency, L. P., Quattoni, A., & Darrell, T. (2007). Latent-dynamic discriminative models for continuous gesture recognition. 2007 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1–8).
 
* Sekine, M., Tamura, T., Togawa, T., & Fukui, Y. (2000). Classification of waist-acceleration signals in a continuous walking record. Medical Engineering & Physics, 22(4), 285-291. doi:16/S1350-4533(00)00041-2
 
* Taylor, G. W., & Hinton, G. E. (2009). Factored conditional restricted boltzmann machines for modeling motion style. Proceedings of the 26th annual international conference on machine learning (pp. 1025–1032).
 
* Yang, C. C., & Hsu, Y. L. (2010). A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors, 10(8), 7772–7788.
 

Latest revision as of 13:39, 23 December 2011

Introduction

There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:

In addition, HuMoS is a suggested integrated approach to sharing movement data and is a work in progress.

This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing movement recognition.

What type of sensor is best?

What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on Sensor Selection. In general:

  • Low-frequency movement is best captured using measurements of position (the Vicon or the Kinect will be your best)
  • Intermediate-frequency movement is best captured using measurements of velocity
  • High-frequency movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on errors in using the Kinect for motion tracking for a discussion on when position-based sampling fails.

Kinect versus Vicon

The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.

What kind of movement do you need to capture?

What kind of movement do you need to capture? Consider the following.

Do you need to identify individual body parts or is it sufficient to treat the body as a blob?

If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.

Are you capturing movements of the entire body or only some body parts (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) and small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)?

In his seminal book on acting, An Acrobat of the Heart, theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as plastiques and corporeals. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.

Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing)?

If so, then you must use the Vicon. Otherwise, if the body will within a space of about 5 feet by 5 feet, you might be able to stick with the Kinect.

Are you going to be tracking movement in the transverse plane of the body?

These movements include...

  • supination and pronation of the legs or arms
  • left or right rotation of the head
  • twisting of the spine

These movements cannot be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)?

These movements cannot be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine.

Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?


The change in front cannot be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

Are you going to be tracking very fast movements?

The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on Errors and limits in using the Kinect for motion tracking.

Are you going to be tracking very small movements?

See the section on spatial resolution errors for the Kinect to learn about considerations you need to take into account if you intend to track small movements.

How much time do you have to plan and set up your motion capture session?

If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.

Do you need to process the movement data in real-time?

The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.