BlackBox - User contributions [en]

Movement Capture

2011-12-23T13:39:44Z

Diegom: /* Are you going to be tracking very small movements? */

= Introduction =
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:

* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect Kinect]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]

In addition, [http://wiki.iat.sfu.ca/BlackBox/index.php/HuMoS HuMoS] is a suggested integrated approach to sharing movement data and is a work in progress.

This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing [[Movement_Recognition| movement recognition]].

= What type of sensor is best? =

What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:

* '''Low-frequency movement''' is best captured using measurements of position (the Vicon or the Kinect will be your best)
* '''Intermediate-frequency''' movement is best captured using measurements of velocity
* '''High-frequency''' movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on [[Kinect#Errors_in_using_the_Kinect_for_motion_tracking| errors in using the Kinect for motion tracking]] for a discussion on when position-based sampling fails.

= Kinect versus Vicon =

The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.

== What kind of movement do you need to capture? ==

What kind of movement do you need to capture? Consider the following.

<div style = "background-color: #ddd; padding: 10px">
=== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ===

If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.

=== Are you capturing movements of the ''entire body'' or only ''some body parts'' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) ''and'' small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ===

In his seminal book on acting, ''An Acrobat of the Heart'', theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as ''plastiques'' and ''corporeals''. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.

===Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing)? ===

If so, then you must use the Vicon. Otherwise, if the body will within a space of about 5 feet by 5 feet, you might be able to stick with the Kinect.

=== Are you going to be tracking movement in the transverse plane of the body? ===

These movements include...

* supination and pronation of the legs or arms
* left or right rotation of the head
* twisting of the spine

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

=== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ===

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine. 

===Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?===
 The change in front ''cannot'' be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. 

=== Are you going to be tracking very fast movements? ===

The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].

=== Are you going to be tracking very small movements? ===

See the section on [[Kinect#Spatial_resolution_errors| spatial resolution errors for the Kinect]] to learn about considerations you need to take into account if you intend to track small movements. 

</div>

== How much time do you have to plan and set up your motion capture session? ==

If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.

== Do you need to process the movement data in real-time?==

The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.

Movement Capture

2011-12-23T13:38:44Z

Diegom: /* Are you going to be tracking very small movements? */

= Introduction =
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:

* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect Kinect]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]

In addition, [http://wiki.iat.sfu.ca/BlackBox/index.php/HuMoS HuMoS] is a suggested integrated approach to sharing movement data and is a work in progress.

This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing [[Movement_Recognition| movement recognition]].

= What type of sensor is best? =

What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:

* '''Low-frequency movement''' is best captured using measurements of position (the Vicon or the Kinect will be your best)
* '''Intermediate-frequency''' movement is best captured using measurements of velocity
* '''High-frequency''' movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on [[Kinect#Errors_in_using_the_Kinect_for_motion_tracking| errors in using the Kinect for motion tracking]] for a discussion on when position-based sampling fails.

= Kinect versus Vicon =

The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.

== What kind of movement do you need to capture? ==

What kind of movement do you need to capture? Consider the following.

<div style = "background-color: #ddd; padding: 10px">
=== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ===

If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.

=== Are you capturing movements of the ''entire body'' or only ''some body parts'' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) ''and'' small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ===

In his seminal book on acting, ''An Acrobat of the Heart'', theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as ''plastiques'' and ''corporeals''. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.

===Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing)? ===

If so, then you must use the Vicon. Otherwise, if the body will within a space of about 5 feet by 5 feet, you might be able to stick with the Kinect.

=== Are you going to be tracking movement in the transverse plane of the body? ===

These movements include...

* supination and pronation of the legs or arms
* left or right rotation of the head
* twisting of the spine

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

=== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ===

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine. 

===Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?===
 The change in front ''cannot'' be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. 

=== Are you going to be tracking very fast movements? ===

The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].

=== Are you going to be tracking very small movements? ===

See the section on [[Kinect#Spatial_resolution_errors spatial resolution errors for the Kinect]] to learn about considerations you need to take into account if you intend to track small movements. 

</div>

== How much time do you have to plan and set up your motion capture session? ==

If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.

== Do you need to process the movement data in real-time?==

The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.

Movement Capture

2011-12-23T13:08:12Z

Diegom: /* Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing) or will the body stay more or less in one spot (within a space of about 5 feet by 5 feet)?

= Introduction =
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:

* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect Kinect]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]

In addition, [http://wiki.iat.sfu.ca/BlackBox/index.php/HuMoS HuMoS] is a suggested integrated approach to sharing movement data and is a work in progress.

This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing [[Movement_Recognition| movement recognition]].

= What type of sensor is best? =

What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:

* '''Low-frequency movement''' is best captured using measurements of position (the Vicon or the Kinect will be your best)
* '''Intermediate-frequency''' movement is best captured using measurements of velocity
* '''High-frequency''' movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on [[Kinect#Errors_in_using_the_Kinect_for_motion_tracking| errors in using the Kinect for motion tracking]] for a discussion on when position-based sampling fails.

= Kinect versus Vicon =

The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.

== What kind of movement do you need to capture? ==

What kind of movement do you need to capture? Consider the following.

<div style = "background-color: #ddd; padding: 10px">
=== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ===

If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.

=== Are you capturing movements of the ''entire body'' or only ''some body parts'' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) ''and'' small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ===

In his seminal book on acting, ''An Acrobat of the Heart'', theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as ''plastiques'' and ''corporeals''. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.

===Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing)? ===

If so, then you must use the Vicon. Otherwise, if the body will within a space of about 5 feet by 5 feet, you might be able to stick with the Kinect.

=== Are you going to be tracking movement in the transverse plane of the body? ===

These movements include...

* supination and pronation of the legs or arms
* left or right rotation of the head
* twisting of the spine

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

=== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ===

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine. 

===Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?===
 The change in front ''cannot'' be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. 

=== Are you going to be tracking very fast movements? ===

The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].

=== Are you going to be tracking very small movements? ===

See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking] to learn about considerations you need to take into account if you intend to track small movements. 

</div>

== How much time do you have to plan and set up your motion capture session? ==

If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.

== Do you need to process the movement data in real-time?==

The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.

Sensor Selection

2011-12-23T13:01:01Z

Diegom: /* Sensor-based considerations in LMA recognition */

=Introduction and a cautionary tale: Why you need to choose your motion capture system carefully=
[[File:bus.jpg|200px|left]]In Vancouver, Canada, exit doors on buses are marked bus with a sign, "Touch here". Passengers interpret this instruction differently. Some firmly press the door with the palm of their hand; sometimes they leave their hand on the door until the door opens, while other times, they remove their hand the moment right after it make contact with the door. Other passengers slap or punch the door, particularly then the doors don't respond to their gesture right away. The more impatient they become, the stronger and more frequent their punches get. (This happens frequently!)
However, the doors do not respond to pressure at all. Rather, an ultrasound sensor positioned at the top of the door senses an obstruction in an ultrasonic beam. The most efficient way for a passenger to open the door is either to slowly move their hand towards the door, or to place their hand on the sign until the door opens. If their hand approaches the door too quickly or if it is withdrawn too soon, the system fails to detect the hand's presence and the door remains shut. 
The design of this system contravenes how humans have traditionally mobilized their bodies to open doors. Usually, the more direct force we apply to a door, the quicker it opens. This is a relationship that our bodies understand and that we apply to a wide variety of physical interactions with the world at large. The use of ultrasound sensors to interpret commuters' intention to exit a bus also illustrates how design decisions around the application of sensing technologies affect (for better or for worse) the way we move through an increasingly technology-intervened world. Other examples abound. We use our thumbs in a light, precise, staccato way to type out text on capacitive touch screens. We slow down when approaching secured doors so that infrared sensors can detect our presence and unlock the doors. When playing a dance game on the Microsoft Kinect, users need to orient their torsos to face the Kinect sensors directly to avoid body part occlusion, limiting the range of possible movements they can make.

= Different sensors, different uses =
Typologies of sensors used for measuring movement exist in the research literature (J. K. Aggarwal & Park, 2004; Garg, N. Aggarwal, & Sofat, 2009; Gavrila, 1999; Mitra & Acharya, 2007; Pavlovic, Sharma, & T. S. Huang, 1997; Rowe, 2008; Wachs, Kölsch, Helman Stern, & Edan, 2011; Wu & T. Huang, 1999; Yilmaz, Javed, & Shah, 2006). In a detailed review that summarizes many of the findings from earlier reviews, Berman and Stern (2011) propose a typology of sensors (shown as Figure 1) for gesture recognition systems. Their typology is organized around the properties of the sensor technologies and features three parent categories: sensor stimuli, context of use, and sensor platform. They also provide guidelines for selecting which data type to measure based on “movement frequency” (i.e., the rate at which salient aspects of the movement occur):
* Low-frequency movement: position measurements
* Intermediate-frequency movement: velocity measurement
* High-frequency movement: acceleration measurements
The authors place a large emphasis on the use of optical methods for motion capture, asserting that “in order to become universally accepted, gesture interface[s] must satisfy the ‘come as you are’ requirement”, i.e., that moving subjects should be “unencumbered” (to use Berman and Stern’s terminology) with markers, sensors, transmitters, and any other devices on their body. However, this only holds true for a subset of human activity. There are many areas of human activity that require, benefit from, or naturally incorporate some kind of encumbrance or extension of the body: conducting music with a baton, wearing a glove while boxing, cutting bamboo using a machete, and flipping a pancake are but a few examples. When we consider human interaction with existing digital technology, we note the importance of the capacitive sensors on touch-based tablets, the pressure sensors in a touch-responsive electronic piano, and the accelerometers in the Apple iPod that, when shaken, randomly selects the next track to play. In fact, movement is the primary way by which we interact with the world, and touch the fundamental relationship that initiates this interaction (Moore, 1988; Thecla Schiphorst, 2008). Thus, a wide variety of sensors must be brought to bear upon the measurement of salient aspects human movement.
 
 
[[File:BermanAndStearns2010_Typology.PNG|800px]] 
''from Berman and Stern (2011)''
 
 

On this page, we enumerate some existing sensor types and how these can be used towards movement analysis. Our review of sensor types places the embodied experience of movement as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the qualia of moving when combined with conventional mechanisms for sensing movement.
== Position-based sensors ==

A wide range of position-based sensors exist, but in essence, they all capture at a given moment the position of the body in space. The types of stimuli used by position-based sensors include electric, optical (including IR and near-IR stimuli), acoustic, and magnetic systems (Berman & H. Stern, 2011) They may require the subject to wear or hold specialized equipment, or they may leave the subject unencumbered, in which case the subject needs to be separated from the background through ''segmentation''.

Gyroscopes and magnenometers can also be used in concert to measure roll, pitch, and yaw with respect to the earth"s gravitational and magnetic fields as frames of reference.

== Acceleration-based sensors ==

Two types of acceleration sensors are commercially available.
 
''Linear accelerometers ''measure acceleration along a single spatial dimension. Three accelerometers can be oriented orthogonal to each other in order to measure acceleration in three dimensions. The Nintendo Wiimote's acceleration sensing system, for example, is [http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html built on] the [http://www.analog.com/en/mems-sensors/inertial-sensors/adxl330/products/product.html ADXL330], a 3-axis linear accelerometer (Analog Devices, 2007; Wisniowski, 2006). Texas Instruments produces EZ430-Chronos, a watch that [http://www.ti.com/lit/ug/slau292c/slau292c.pdf uses] the [http://www.vti.fi/en/products/accelerometers/consumer_electronics/cma3000_series/ VTI CMA3000], another 3-axis accelerometer (VTI Technologies, n.d.; Texas Instruments, 2010).
 
''Gyroscopes'' measure rotational acceleration. When the direction of gravity is known, gyroscopes can be used to measure ''pitch'' and ''roll''. Another measure of rotational movement, ''yaw'', can be derived if a device can sense its orientation with reference to the Earth"s magnetic fields. The iPhone 4 has a 3-axis linear accelerometer, a 2-axis gyroscope, and a magnetometer, providing six acceleration measurements as well as orientation information (Dilger, 2010). Earlier versions of the iPhone did not have a magnetometer and thus could not measure yaw (Sadun, 2007).
 
Yang and Hsu (2010) have summarized the uses of accelerometers in measuring physical activity. Accelerometers have been used to determine static postures (standing upright, lying down), postural transitions (standing, sitting, postural sway), and gait parameters (heel strike, gait cycle frequency, stride symmetry, regularity, step length, and gait smoothness). In combination with other sensors, accelerometers can also be used to infer falling (when combined with impact detection) and energy expenditure (particularly when combined with barometric sensors to determine changes in elevation). Higher-context knowledge can be generated through accelerometry, such as restfulness during sleep, which can be inferred from the number of postural transitions during the various sleep cycles (Yang & Hsu, 2010)<nowiki>; by tracking energy expenditure, we might also be able to infer fatigue.</nowiki>
 
Accelerometers are particularly good at detecting abrupt and frequent changes in velocity, compared to position-based sensors. While reconstructing positional information from acceleration is possible but prone to error (Giansanti, Macellari, Maccioni, & Cappozzo, 2003), we hypothesize that accelerometers are particularly adept for sensing data that are related to changes in movement quality.
 
Sensors that measure the amount of contact or pressure on a surface are a special case of acceleration- or velocity-based sensors. Pressure can be thought of as the outcome of impeded movement: when a finger attempts to push past a screen, the screen stops it from moving.

== Other sensor types ==
This is a work in progress. Other sensor types will be discussed here.
= Sensor-based considerations in LMA recognition =

Different types of sensors are able to measure different low-context properties of human movement, such as the position, velocity, and acceleration of specific body parts. By applying computational techniques to these measurements, we can infer higher-context properties, such as gait information (stride length, walking speed), postural changes (falling, standing up), and energy expenditure. What has been underexplored is how these can properties can be used to recognize higher-context qualities of human movement for creating a semantics of expressive motion. One such semantic framework is Laban Movement Analysis (LMA). In the table below, we enumerate some existing sensor types and how these can be used towards movement analysis, and show they could be used towards the application of LMA. This table places the ''embodied experience of movement'' as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the ''qualia ''of moving when combined with conventional mechanisms for sensing movement. The table headings also include the notion ''movement context'', taken from [http://wiki.iat.sfu.ca/BlackBox/index.php/File:Chapter6MovementAsMetaphor_-_Moore_-_1988_-_Beyond_Words_Movement_Observation_and_Analysis.pdf Chapter 6 of Moore and Yamamoto (1988)]. In this chapter, the authors ask, "If we take the metaphor of ''movement is a language'' seriously, would human movement be a universal language, a foreign language, or a private code?" They propose that movement could be all three, depending on context. I propose that a hierarchical semantics of movement could be closely linked to the Moore and Yamamoto's notion of movement context.

{| border="1"
|
<center>'''Type of sensor'''</center>
|
<center>'''Example'''</center>
|
<center>'''Commercially available products'''</center>
|
<center>'''Fundamental movement phenomena sensed (movement primitive)("low-context")'''</center>
|
<center>'''Mid-level movement feature (inferred from movement primitives)("mid-context")'''</center>
|
<center>'''Higher-level semantics based on LMA (inferred from mid-level features)("high-context")'''</center>
|-
| rowspan="2" |
Acceleration
|
Gyroscopes
|
iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus
|
Rotational accleration
| rowspan="2" |
Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)
| rowspan="2" |
A prototype for recognizing LMA Effort using accelerometers is being developed by the Institute for Advanced Computing Applications and Technologies at the University of Illinois and the University of Illinois Dance Department, with the expertise of movement analyst Sarah Hook from the Dance Department, and in collaboration with Dr. Thecla Schiphorst (Subyen, Maranan, Schiphorst, Pasquier, & Bartram, 2011).
|-
|
Accelerometers
|
Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote
|
Linear acceleration
|-
|
Touch
|
Pressure sensors
|
Tactex
|
Pressure
|

|
LMA recognition of has been applied to touch-based interfaces in interactive art (T. Schiphorst, Lovell, & Jaffe, 2002).
|-
| rowspan="3" |
Position
|
Vision
|
Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems
| rowspan="3" |
Position of body segments
| rowspan="3" |
Postural information; anything that can be inferred from acceleration sensors
| rowspan="3" |
Recognition of some aspects of Space, Space, and Effort categories have been reported (J. Rett, J. Dias, & Ahuactzin, 2008; J. Rett, Santos, & J. Dias, 2008; Jorg Rett & Jorge Dias, 2007a, 2007b; Santos, Prado, & J. Dias, 2009; Santos et al., 2009; Swaminathan et al., 2009; L. Zhao, 2001; Liwei Zhao & Badler, 2005). 
|-
|
Magnetic
|

|-
|
Infrared
|

|-
| rowspan="5" |
Biometric
|
Eye-tracking
|

|
Gaze
|
Visual attention; intent
| rowspan="5" |
Muscular tension is related to Effort Weight. Attention and intent are key themes in Effort Space. We propose that arousal can be affined to the extent by which a mover uses "fighting" qualities over "indulging" qualities.
|-
|
GSR
|

|
Electrical conductance of the skin
|
Arousal
|-
|
Breath sensors
|

|
Rate of breathing; volume of inspiration/expiration
|
Arousal; energy expenditure
|-
|
EMG
|

|
Electrical activity produced by skeletal muscles
|
Muscular tension
|-
|
Heart rate sensors
|

|
Heart rate
|
Arousal; level of physical activity
|}

= References =

* Aggarwal, J. K., & Park, S. (2004). Human motion: Modeling and recognition of actions and interactions.
* Analog Devices. (2007). ADXL330: Small, Low Power, 3-Axis "3g iMEMS" Accelerometer. Retrieved from http://www.analog.com/static/imported-files/data_sheets/ADXL330.pdf
* Berman, S., & Stern, H. (2011). Sensors for Gesture Recognition Systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, PP(99), 1-14. doi:10.1109/TSMCC.2011.2161077
* Dilger, D. E. (2010, June 16). Inside iPhone 4: Gyro spins Apple ahead in gaming [Page 2]. AppleInsider. Retrieved August 22, 2011, from http://www.appleinsider.com/articles/10/06/16/inside_iphone_4_gyro_spins_apple_ahead_in_gaming.html&page=2
* Garg, P., Aggarwal, N., & Sofat, S. (2009). Vision Based Hand Gesture Recognition. World Academy of Science, Engineering and Technology, 49, 972"977.
* Gavrila, D. M. (1999). The Visual Analysis of Human Movement: A Survey. Computer vision and image understanding, 73(1), 82"98.
* Giansanti, D., Macellari, V., Maccioni, G., & Cappozzo, A. (2003). Is it feasible to reconstruct body segment 3-D position and orientation using accelerometric data? IEEE Transactions on Biomedical Engineering, 50(4), 476-483. doi:10.1109/TBME.2003.809490
* Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311-324. doi:10.1109/TSMCC.2007.893280
* Moore, C.-L., & Yamamoto, K. (1988). Beyond Words: Movement Observation and Analysis. New York, N.Y., U.S.A: Gordon and Breach Science Publishers.
* Nakata, T., Mori, T., & Sato, T. (2002). Analysis of impression of robot bodily expression. Journal of Robotics and Mechatronics, 14(1). Retrieved from http://staff.aist.go.jp/toru-nakata/LabanEng.pdf
* Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997). Visual interpretation of hand gestures for human-computerinteraction: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677-695. doi:10.1109/34.598226
* Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.
* Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.
* Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on "Concept Learning for Embodied Agents." Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf
* Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436
* Rowe, D. (2008). Towards Robust Multiple-Target tracking in Unconstrained Human-Populated Environments. Department of Computer Science UAB and Computer Vision Center, Barcelona, Spain.
* Sadun, E. (2007, September 10). iPhone Coding: Using the Accelerometer. TUAW - The Unofficial Apple Weblog. Retrieved August 22, 2011, from http://www.tuaw.com/2007/09/10/iphone-coding-using-the-accelerometer/
* Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564
* Schiphorst, T., Lovell, R., & Jaffe, N. (2002). Using a gestural interface toolkit for tactile input to a dynamic virtual space. Conference on Human Factors in Computing Systems (pp. 754"755).
* Schiphorst, Thecla. (2008). Bridging embodied methodologies from somatics and performance to human computer interaction (Ph.D. dissertation). School of Computing, Communications and Electronics, Faculty of Technology, University of Plymouth, United Kingdom. Retrieved from http://www.sfu.ca/~tschipho/PhD/PhD_thesis.html
* Subyen, P., Maranan, D. S., Schiphorst, T., Pasquier, P., & Bartram, L. (2011). EMVIZ: The Poetics of Movement Quality Visualization. ''Proceedings of Computational Aesthetic 2011 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging''. Presented at the Computational Aesthetics, Vancouver, Canada.
* Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.
* VTI Technologies. (n.d.). CMA3000-D01 3-Axis Ultra Low Power Accelerometer with Digital SPI and I2C Interface. VTI Technologies. Retrieved from http://www.vti.fi/midcom-serveattachmentguid-1e05eb0417155de5eb011e0b896954a036ddb89db89/cma3000_d01_datasheet_8277800a.03.pdf
* Texas Instruments. (2010, December). eZ430-ChronosTM Development Tool User"s Guide. Texas Instruments. Retrieved from http://www.ti.com/lit/ug/slau292c/slau292c.pdf
* Wachs, J. P., K"lsch, M., Stern, Helman, & Edan, Y. (2011). Vision-based hand-gesture applications. Communications of the ACM, 54(2), 60. doi:10.1145/1897816.1897838
* Wisniowski, H. (2006, May 9). Analog Devices And Nintendo Collaboration Drives Video Game Innovation With iMEMS Motion Signal Processing Technology. Analog Devices, Inc. Retrieved August 22, 2011, from http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html
* Wu, Y., & Huang, T. (1999). Vision-based gesture recognition: A review. Gesture-based communication in human-computer interaction, 103"115.
* Yang, C. C., & Hsu, Y. L. (2010). A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors, 10(8), 7772"7788.
* Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking. ACM Computing Surveys, 38(4), 13-es. doi:10.1145/1177352.1177355
* Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.
* Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002

2011-12-23T12:28:08Z

Diegom: /* References */

Movement Recognition

2011-12-23T12:27:01Z

Diegom: changed references

Sensor Selection

2011-12-23T12:26:54Z

Diegom: added subyen, maranan, et al

=Introduction and a cautionary tale: Why you need to choose your motion capture system carefully=
[[File:bus.jpg|200px|left]]In Vancouver, Canada, exit doors on buses are marked bus with a sign, "Touch here". Passengers interpret this instruction differently. Some firmly press the door with the palm of their hand; sometimes they leave their hand on the door until the door opens, while other times, they remove their hand the moment right after it make contact with the door. Other passengers slap or punch the door, particularly then the doors don't respond to their gesture right away. The more impatient they become, the stronger and more frequent their punches get. (This happens frequently!)
However, the doors do not respond to pressure at all. Rather, an ultrasound sensor positioned at the top of the door senses an obstruction in an ultrasonic beam. The most efficient way for a passenger to open the door is either to slowly move their hand towards the door, or to place their hand on the sign until the door opens. If their hand approaches the door too quickly or if it is withdrawn too soon, the system fails to detect the hand's presence and the door remains shut. 
The design of this system contravenes how humans have traditionally mobilized their bodies to open doors. Usually, the more direct force we apply to a door, the quicker it opens. This is a relationship that our bodies understand and that we apply to a wide variety of physical interactions with the world at large. The use of ultrasound sensors to interpret commuters' intention to exit a bus also illustrates how design decisions around the application of sensing technologies affect (for better or for worse) the way we move through an increasingly technology-intervened world. Other examples abound. We use our thumbs in a light, precise, staccato way to type out text on capacitive touch screens. We slow down when approaching secured doors so that infrared sensors can detect our presence and unlock the doors. When playing a dance game on the Microsoft Kinect, users need to orient their torsos to face the Kinect sensors directly to avoid body part occlusion, limiting the range of possible movements they can make.

= Different sensors, different uses =
Typologies of sensors used for measuring movement exist in the research literature (J. K. Aggarwal & Park, 2004; Garg, N. Aggarwal, & Sofat, 2009; Gavrila, 1999; Mitra & Acharya, 2007; Pavlovic, Sharma, & T. S. Huang, 1997; Rowe, 2008; Wachs, Kölsch, Helman Stern, & Edan, 2011; Wu & T. Huang, 1999; Yilmaz, Javed, & Shah, 2006). In a detailed review that summarizes many of the findings from earlier reviews, Berman and Stern (2011) propose a typology of sensors (shown as Figure 1) for gesture recognition systems. Their typology is organized around the properties of the sensor technologies and features three parent categories: sensor stimuli, context of use, and sensor platform. They also provide guidelines for selecting which data type to measure based on “movement frequency” (i.e., the rate at which salient aspects of the movement occur):
* Low-frequency movement: position measurements
* Intermediate-frequency movement: velocity measurement
* High-frequency movement: acceleration measurements
The authors place a large emphasis on the use of optical methods for motion capture, asserting that “in order to become universally accepted, gesture interface[s] must satisfy the ‘come as you are’ requirement”, i.e., that moving subjects should be “unencumbered” (to use Berman and Stern’s terminology) with markers, sensors, transmitters, and any other devices on their body. However, this only holds true for a subset of human activity. There are many areas of human activity that require, benefit from, or naturally incorporate some kind of encumbrance or extension of the body: conducting music with a baton, wearing a glove while boxing, cutting bamboo using a machete, and flipping a pancake are but a few examples. When we consider human interaction with existing digital technology, we note the importance of the capacitive sensors on touch-based tablets, the pressure sensors in a touch-responsive electronic piano, and the accelerometers in the Apple iPod that, when shaken, randomly selects the next track to play. In fact, movement is the primary way by which we interact with the world, and touch the fundamental relationship that initiates this interaction (Moore, 1988; Thecla Schiphorst, 2008). Thus, a wide variety of sensors must be brought to bear upon the measurement of salient aspects human movement.
 
 
[[File:BermanAndStearns2010_Typology.PNG|800px]] 
''from Berman and Stern (2011)''
 
 

On this page, we enumerate some existing sensor types and how these can be used towards movement analysis. Our review of sensor types places the embodied experience of movement as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the qualia of moving when combined with conventional mechanisms for sensing movement.
== Position-based sensors ==

A wide range of position-based sensors exist, but in essence, they all capture at a given moment the position of the body in space. The types of stimuli used by position-based sensors include electric, optical (including IR and near-IR stimuli), acoustic, and magnetic systems (Berman & H. Stern, 2011) They may require the subject to wear or hold specialized equipment, or they may leave the subject unencumbered, in which case the subject needs to be separated from the background through ''segmentation''.

Gyroscopes and magnenometers can also be used in concert to measure roll, pitch, and yaw with respect to the earthï¿½s gravitational and magnetic fields as frames of reference.

== Acceleration-based sensors ==

Two types of acceleration sensors are commercially available.
 
''Linear accelerometers ''measure acceleration along a single spatial dimension. Three accelerometers can be oriented orthogonal to each other in order to measure acceleration in three dimensions. The Nintendo Wiimote's acceleration sensing system, for example, is [http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html built on] the [http://www.analog.com/en/mems-sensors/inertial-sensors/adxl330/products/product.html ADXL330], a 3-axis linear accelerometer (Analog Devices, 2007; Wisniowski, 2006). Texas Instruments produces EZ430-Chronos, a watch that [http://www.ti.com/lit/ug/slau292c/slau292c.pdf uses] the [http://www.vti.fi/en/products/accelerometers/consumer_electronics/cma3000_series/ VTI CMA3000], another 3-axis accelerometer (VTI Technologies, n.d.; Texas Instruments, 2010).
 
''Gyroscopes'' measure rotational acceleration. When the direction of gravity is known, gyroscopes can be used to measure ''pitch'' and ''roll''. Another measure of rotational movement, ''yaw'', can be derived if a device can sense its orientation with reference to the Earthï¿½s magnetic fields. The iPhone 4 has a 3-axis linear accelerometer, a 2-axis gyroscope, and a magnetometer, providing six acceleration measurements as well as orientation information (Dilger, 2010). Earlier versions of the iPhone did not have a magnetometer and thus could not measure yaw (Sadun, 2007).
 
Yang and Hsu (2010) have summarized the uses of accelerometers in measuring physical activity. Accelerometers have been used to determine static postures (standing upright, lying down), postural transitions (standing, sitting, postural sway), and gait parameters (heel strike, gait cycle frequency, stride symmetry, regularity, step length, and gait smoothness). In combination with other sensors, accelerometers can also be used to infer falling (when combined with impact detection) and energy expenditure (particularly when combined with barometric sensors to determine changes in elevation). Higher-context knowledge can be generated through accelerometry, such as restfulness during sleep, which can be inferred from the number of postural transitions during the various sleep cycles (Yang & Hsu, 2010)<nowiki>; by tracking energy expenditure, we might also be able to infer fatigue.</nowiki>
 
Accelerometers are particularly good at detecting abrupt and frequent changes in velocity, compared to position-based sensors. While reconstructing positional information from acceleration is possible but prone to error (Giansanti, Macellari, Maccioni, & Cappozzo, 2003), we hypothesize that accelerometers are particularly adept for sensing data that are related to changes in movement quality.
 
Sensors that measure the amount of contact or pressure on a surface are a special case of acceleration- or velocity-based sensors. Pressure can be thought of as the outcome of impeded movement: when a finger attempts to push past a screen, the screen stops it from moving.

== Other sensor types ==
This is a work in progress. Other sensor types will be discussed here.
= Sensor-based considerations in LMA recognition =

Different types of sensors are able to measure different low-context properties of human movement, such as the position, velocity, and acceleration of specific body parts. By applying computational techniques to these measurements, we can infer higher-context properties, such as gait information (stride length, walking speed), postural changes (falling, standing up), and energy expenditure. What has been underexplored is how these can properties can be used to recognize higher-context qualities of human movement for creating a semantics of expressive motion. One such semantic framework is Laban Movement Analysis (LMA). On this page, we enumerate some existing sensor types and how these can be used towards movement analysis, and show they could be used towards the application of LMA. This table places the ''embodied experience of movement'' as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the ''qualia ''of moving when combined with conventional mechanisms for sensing movement.

{| border="1"
|
<center>'''Type of sensor'''</center>
|
<center>'''Example'''</center>
|
<center>'''Commercially available products'''</center>
|
<center>'''Fundamental movement phenomena sensed'''</center><center>'''(movement primitive)'''</center>
|
<center>'''Mid-level movement feature (inferred from movement primitives)'''</center>
|
<center>'''Higher-level semantics based on LMA (inferred from mid-level features)'''</center>
|-
| rowspan="2" |
Acceleration
|
Gyroscopes
|
iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus
|
Rotational accleration
| rowspan="2" |
Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)
| rowspan="2" |
A prototype for recognizing LMA Effort using accelerometers is being developed by the Institute for Advanced Computing Applications and Technologies at the University of Illinois and the University of Illinois Dance Department, with the expertise of movement analyst Sarah Hook from the Dance Department, and in collaboration with Dr. Thecla Schiphorst (Subyen, Maranan, Schiphorst, Pasquier, & Bartram, 2011).
|-
|
Accelerometers
|
Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote
|
Linear acceleration
|-
|
Touch
|
Pressure sensors
|
Tactex
|
Pressure
|

|
LMA recognition of has been applied to touch-based interfaces in interactive art (T. Schiphorst, Lovell, & Jaffe, 2002).
|-
| rowspan="3" |
Position
|
Vision
|
Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems
| rowspan="3" |
Position of body segments
| rowspan="3" |
Postural information; anything that can be inferred from acceleration sensors
| rowspan="3" |
Recognition of some aspects of Space, Space, and Effort categories have been reported (J. Rett, J. Dias, & Ahuactzin, 2008; J. Rett, Santos, & J. Dias, 2008; Jorg Rett & Jorge Dias, 2007a, 2007b; Santos, Prado, & J. Dias, 2009; Santos et al., 2009; Swaminathan et al., 2009; L. Zhao, 2001; Liwei Zhao & Badler, 2005). 
|-
|
Magnetic
|

|-
|
Infrared
|

|-
| rowspan="5" |
Biometric
|
Eye-tracking
|

|
Gaze
|
Visual attention; intent
| rowspan="5" |
Muscular tension is related to Effort Weight. Attention and intent are key themes in Effort Space. We propose that arousal can be affined to the extent by which a mover uses ï¿½fightingï¿½ qualities over ï¿½indulgingï¿½ qualities.
|-
|
GSR
|

|
Electrical conductance of the skin
|
Arousal
|-
|
Breath sensors
|

|
Rate of breathing; volume of inspiration/expiration
|
Arousal; energy expenditure
|-
|
EMG
|

|
Electrical activity produced by skeletal muscles
|
Muscular tension
|-
|
Heart rate sensors
|

|
Heart rate
|
Arousal; level of physical activity
|}

= References =

* Aggarwal, J. K., & Park, S. (2004). Human motion: Modeling and recognition of actions and interactions.
* Analog Devices. (2007). ADXL330: Small, Low Power, 3-Axis ï¿½3g iMEMSï¿½ Accelerometer. Retrieved from http://www.analog.com/static/imported-files/data_sheets/ADXL330.pdf
* Berman, S., & Stern, H. (2011). Sensors for Gesture Recognition Systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, PP(99), 1-14. doi:10.1109/TSMCC.2011.2161077
* Dilger, D. E. (2010, June 16). Inside iPhone 4: Gyro spins Apple ahead in gaming [Page 2]. AppleInsider. Retrieved August 22, 2011, from http://www.appleinsider.com/articles/10/06/16/inside_iphone_4_gyro_spins_apple_ahead_in_gaming.html&page=2
* Garg, P., Aggarwal, N., & Sofat, S. (2009). Vision Based Hand Gesture Recognition. World Academy of Science, Engineering and Technology, 49, 972ï¿½977.
* Gavrila, D. M. (1999). The Visual Analysis of Human Movement: A Survey. Computer vision and image understanding, 73(1), 82ï¿½98.
* Giansanti, D., Macellari, V., Maccioni, G., & Cappozzo, A. (2003). Is it feasible to reconstruct body segment 3-D position and orientation using accelerometric data? IEEE Transactions on Biomedical Engineering, 50(4), 476-483. doi:10.1109/TBME.2003.809490
* Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311-324. doi:10.1109/TSMCC.2007.893280
* Moore, C.-L. (1988). Beyond Words: Movement Observation and Analysis. New York, N.Y., U.S.A: Gordon and Breach Science Publishers.
* Nakata, T., Mori, T., & Sato, T. (2002). Analysis of impression of robot bodily expression. Journal of Robotics and Mechatronics, 14(1). Retrieved from http://staff.aist.go.jp/toru-nakata/LabanEng.pdf
* Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997). Visual interpretation of hand gestures for human-computerinteraction: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677-695. doi:10.1109/34.598226
* Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.
* Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.
* Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on ï¿½Concept Learning for Embodied Agents.ï¿½ Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf
* Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436
* Rowe, D. (2008). Towards Robust Multiple-Target tracking in Unconstrained Human-Populated Environments. Department of Computer Science UAB and Computer Vision Center, Barcelona, Spain.
* Sadun, E. (2007, September 10). iPhone Coding: Using the Accelerometer. TUAW - The Unofficial Apple Weblog. Retrieved August 22, 2011, from http://www.tuaw.com/2007/09/10/iphone-coding-using-the-accelerometer/
* Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564
* Schiphorst, T., Lovell, R., & Jaffe, N. (2002). Using a gestural interface toolkit for tactile input to a dynamic virtual space. Conference on Human Factors in Computing Systems (pp. 754ï¿½755).
* Schiphorst, Thecla. (2008). Bridging embodied methodologies from somatics and performance to human computer interaction (Ph.D. dissertation). School of Computing, Communications and Electronics, Faculty of Technology, University of Plymouth, United Kingdom. Retrieved from http://www.sfu.ca/~tschipho/PhD/PhD_thesis.html
* Subyen, P., Maranan, D. S., Schiphorst, T., Pasquier, P., & Bartram, L. (2011). EMVIZ: The Poetics of Movement Quality Visualization. ''Proceedings of Computational Aesthetic 2011 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging''. Presented at the Computational Aesthetics, Vancouver, Canada.
* Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.
* VTI Technologies. (n.d.). CMA3000-D01 3-Axis Ultra Low Power Accelerometer with Digital SPI and I2C Interface. VTI Technologies. Retrieved from http://www.vti.fi/midcom-serveattachmentguid-1e05eb0417155de5eb011e0b896954a036ddb89db89/cma3000_d01_datasheet_8277800a.03.pdf
* Texas Instruments. (2010, December). eZ430-ChronosTM Development Tool Userï¿½s Guide. Texas Instruments. Retrieved from http://www.ti.com/lit/ug/slau292c/slau292c.pdf
* Wachs, J. P., Kï¿½lsch, M., Stern, Helman, & Edan, Y. (2011). Vision-based hand-gesture applications. Communications of the ACM, 54(2), 60. doi:10.1145/1897816.1897838
* Wisniowski, H. (2006, May 9). Analog Devices And Nintendo Collaboration Drives Video Game Innovation With iMEMS Motion Signal Processing Technology. Analog Devices, Inc. Retrieved August 22, 2011, from http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html
* Wu, Y., & Huang, T. (1999). Vision-based gesture recognition: A review. Gesture-based communication in human-computer interaction, 103ï¿½115.
* Yang, C. C., & Hsu, Y. L. (2010). A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors, 10(8), 7772ï¿½7788.
* Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking. ACM Computing Surveys, 38(4), 13-es. doi:10.1145/1177352.1177355
* Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.
* Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002

Movement Capture

2011-12-23T12:24:30Z

Diegom: removed references

= Introduction =
There are currently two major systems for movement capture and recognition in the Blackbox: The Vicon system and the Kinect. There are also a set of accelerometers that you can use:

* [http://wiki.iat.sfu.ca/BlackBox/index.php/Vicon Vicon]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect Kinect]
* [http://wiki.iat.sfu.ca/BlackBox/index.php/EffortDetect#Wearable_Accelerometers EffortDetect Wearable Accelerometers]

In addition, [http://wiki.iat.sfu.ca/BlackBox/index.php/HuMoS HuMoS] is a suggested integrated approach to sharing movement data and is a work in progress.

This page discusses movement capture systems in the Blackbox. A separate page is dedicated to discussing [[Movement_Recognition| movement recognition]].

= What type of sensor is best? =

What kinds of sensors best fit the motion phenomena you wish to capture. Motion capture systems use different kinds of sensors. To find out which type of sensor best suits your application, see the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/White_Paper:_Sensor_Selection Sensor Selection]. In general:

* '''Low-frequency movement''' is best captured using measurements of position (the Vicon or the Kinect will be your best)
* '''Intermediate-frequency''' movement is best captured using measurements of velocity
* '''High-frequency''' movement is best captured using measurements of acceleration (accelerometers would be appropriate). Generally, human movement tends to be slow enough that measurements of position are adequate. There are exceptions. See the section on [[Kinect#Errors_in_using_the_Kinect_for_motion_tracking| errors in using the Kinect for motion tracking]] for a discussion on when position-based sampling fails.

= Kinect versus Vicon =

The Vicon is a powerful motion capture tool and can be used to capture an enormous variety of movement. However, using the Vicon isn't always ideal. Consider the following questions when deciding whether to use the Vicon or the Kinect.

== What kind of movement do you need to capture? ==

What kind of movement do you need to capture? Consider the following.

<div style = "background-color: #ddd; padding: 10px">
=== Do you need to identify individual body parts or is it sufficient to treat the body as a blob? ===

If all you need to do is treat the body as a blob (for example, tracking the position of a body in space in order to project an image around that body), the Kinect is a more convenient choice than the Vicon. Recognizing human skeleton data using the Vicon data takes more work than processing Kinect data. While many of the Kinect APIs map the cloud of points to a human skeleton, for the Vicon, you will need to set up the skeleton yourself.

=== Are you capturing movements of the ''entire body'' or only ''some body parts'' (only finger gestures, or only full arm movements, or only the torso, or only leg bending and swinging)? Do you need to track both large movements (e.g., lunging, jumping, turning around) ''and'' small gestures (e.g., foot tapping, fingers drumming, head slightly tilting)? ===

In his seminal book on acting, ''An Acrobat of the Heart'', theater theorist and director Jerzy Grotowski distinguishes the two kinds of movements as ''plastiques'' and ''corporeals''. If you are tracking movement that happens on similar scales (either mostly plastiques or mostly corporeals) and for body parts that close to each other, then you can use the Kinect effectively by using the appropriate Kinect API.Otherwise, you are better off using the Vicon.

===Is the entire body going to be moving through a large volume of space (greater than about 5 feet by 5 feet) (e.g., walking, running, many forms of dancing) or will the body stay more or less in one spot (within a space of about 5 feet by 5 feet)? ===

If so, then you must use the Vicon.

=== Are you going to be tracking movement in the transverse plane of the body? ===

These movements include...

* supination and pronation of the legs or arms
* left or right rotation of the head
* twisting of the spine

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task.

=== Do you need to track movements of the separate parts of the torso (the cervical, lumbar, thoracic, and sacral areas)? ===

These movements ''cannot'' be tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. If you don't need to track precise movements of the parts of the torso, the Kinect is fine. 

===Will the mover be changing their frontal orientation (e.g., by turning around or spinning)?===
 The change in front ''cannot'' be automatically tracked by the Kinect APIs available. You will have to search for APIs that do this, or build it yourself using the cloud point data. The Vicon is a better choice for this task. 

=== Are you going to be tracking very fast movements? ===

The sampling rate of the Vicon is 200 Hz, which is sufficiently large to capture many nuances of human movement. The Kinect's, however, is (as most) 30 Hz. Is this fast enough? The answer is: in many instances, yes, but it really depends on what you want to do. See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking].

=== Are you going to be tracking very small movements? ===

See the section on [http://wiki.iat.sfu.ca/BlackBox/index.php/Kinect#Errors_in_using_the_Kinect_for_motion_tracking Errors and limits in using the Kinect for motion tracking] to learn about considerations you need to take into account if you intend to track small movements. 

</div>

== How much time do you have to plan and set up your motion capture session? ==

If you can prepare adequately for it, use the Vicon. But if you want to quickly capture movement data and don't have a lot of time on your hands, use the Kinect.

== Do you need to process the movement data in real-time?==

The current Vicon setup in the Blackbox prevents you from using it for real-time motion capture. A kludge was devised a long time ago by a former SIAT student, Rob Lovell, but the technical details on how to do this has been lost. Plans for upgrading the Vicon system are underway. In the meantime, the Kinect will allow you to stream real-time movement data very easily.

2011-12-23T11:42:29Z

Diegom: moved Movement Capture and Recognition to Movement Capture

#REDIRECT [[Movement Capture]]

2011-12-23T10:33:14Z

Diegom: added the list of heuristics

Sensor Selection

2011-12-23T10:13:29Z

Diegom: moved White Paper: Sensor Selection to Sensor Selection

=Introduction and a cautionary tale: Why you need to choose your motion capture system carefully=
[[File:bus.jpg|200px|left]]In Vancouver, Canada, exit doors on buses are marked bus with a sign, "Touch here". Passengers interpret this instruction differently. Some firmly press the door with the palm of their hand; sometimes they leave their hand on the door until the door opens, while other times, they remove their hand the moment right after it make contact with the door. Other passengers slap or punch the door, particularly then the doors don't respond to their gesture right away. The more impatient they become, the stronger and more frequent their punches get. (This happens frequently!)
However, the doors do not respond to pressure at all. Rather, an ultrasound sensor positioned at the top of the door senses an obstruction in an ultrasonic beam. The most efficient way for a passenger to open the door is either to slowly move their hand towards the door, or to place their hand on the sign until the door opens. If their hand approaches the door too quickly or if it is withdrawn too soon, the system fails to detect the hand's presence and the door remains shut. 
The design of this system contravenes how humans have traditionally mobilized their bodies to open doors. Usually, the more direct force we apply to a door, the quicker it opens. This is a relationship that our bodies understand and that we apply to a wide variety of physical interactions with the world at large. The use of ultrasound sensors to interpret commuters' intention to exit a bus also illustrates how design decisions around the application of sensing technologies affect (for better or for worse) the way we move through an increasingly technology-intervened world. Other examples abound. We use our thumbs in a light, precise, staccato way to type out text on capacitive touch screens. We slow down when approaching secured doors so that infrared sensors can detect our presence and unlock the doors. When playing a dance game on the Microsoft Kinect, users need to orient their torsos to face the Kinect sensors directly to avoid body part occlusion, limiting the range of possible movements they can make.

= Different sensors, different uses =
Typologies of sensors used for measuring movement exist in the research literature (J. K. Aggarwal & Park, 2004; Garg, N. Aggarwal, & Sofat, 2009; Gavrila, 1999; Mitra & Acharya, 2007; Pavlovic, Sharma, & T. S. Huang, 1997; Rowe, 2008; Wachs, Kölsch, Helman Stern, & Edan, 2011; Wu & T. Huang, 1999; Yilmaz, Javed, & Shah, 2006). In a detailed review that summarizes many of the findings from earlier reviews, Berman and Stern (2011) propose a typology of sensors (shown as Figure 1) for gesture recognition systems. Their typology is organized around the properties of the sensor technologies and features three parent categories: sensor stimuli, context of use, and sensor platform. They also provide guidelines for selecting which data type to measure based on “movement frequency” (i.e., the rate at which salient aspects of the movement occur):
* Low-frequency movement: position measurements
* Intermediate-frequency movement: velocity measurement
* High-frequency movement: acceleration measurements
The authors place a large emphasis on the use of optical methods for motion capture, asserting that “in order to become universally accepted, gesture interface[s] must satisfy the ‘come as you are’ requirement”, i.e., that moving subjects should be “unencumbered” (to use Berman and Stern’s terminology) with markers, sensors, transmitters, and any other devices on their body. However, this only holds true for a subset of human activity. There are many areas of human activity that require, benefit from, or naturally incorporate some kind of encumbrance or extension of the body: conducting music with a baton, wearing a glove while boxing, cutting bamboo using a machete, and flipping a pancake are but a few examples. When we consider human interaction with existing digital technology, we note the importance of the capacitive sensors on touch-based tablets, the pressure sensors in a touch-responsive electronic piano, and the accelerometers in the Apple iPod that, when shaken, randomly selects the next track to play. In fact, movement is the primary way by which we interact with the world, and touch the fundamental relationship that initiates this interaction (Moore, 1988; Thecla Schiphorst, 2008). Thus, a wide variety of sensors must be brought to bear upon the measurement of salient aspects human movement.
 
 
[[File:BermanAndStearns2010_Typology.PNG|800px]] 
''from Berman and Stern (2011)''
 
 

On this page, we enumerate some existing sensor types and how these can be used towards movement analysis. Our review of sensor types places the embodied experience of movement as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the qualia of moving when combined with conventional mechanisms for sensing movement.
== Position-based sensors ==

A wide range of position-based sensors exist, but in essence, they all capture at a given moment the position of the body in space. The types of stimuli used by position-based sensors include electric, optical (including IR and near-IR stimuli), acoustic, and magnetic systems (Berman & H. Stern, 2011) They may require the subject to wear or hold specialized equipment, or they may leave the subject unencumbered, in which case the subject needs to be separated from the background through ''segmentation''.

Gyroscopes and magnenometers can also be used in concert to measure roll, pitch, and yaw with respect to the earthï¿½s gravitational and magnetic fields as frames of reference.

== Acceleration-based sensors ==

Two types of acceleration sensors are commercially available.
 
''Linear accelerometers ''measure acceleration along a single spatial dimension. Three accelerometers can be oriented orthogonal to each other in order to measure acceleration in three dimensions. The Nintendo Wiimote's acceleration sensing system, for example, is [http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html built on] the [http://www.analog.com/en/mems-sensors/inertial-sensors/adxl330/products/product.html ADXL330], a 3-axis linear accelerometer (Analog Devices, 2007; Wisniowski, 2006). Texas Instruments produces EZ430-Chronos, a watch that [http://www.ti.com/lit/ug/slau292c/slau292c.pdf uses] the [http://www.vti.fi/en/products/accelerometers/consumer_electronics/cma3000_series/ VTI CMA3000], another 3-axis accelerometer (VTI Technologies, n.d.; Texas Instruments, 2010).
 
''Gyroscopes'' measure rotational acceleration. When the direction of gravity is known, gyroscopes can be used to measure ''pitch'' and ''roll''. Another measure of rotational movement, ''yaw'', can be derived if a device can sense its orientation with reference to the Earthï¿½s magnetic fields. The iPhone 4 has a 3-axis linear accelerometer, a 2-axis gyroscope, and a magnetometer, providing six acceleration measurements as well as orientation information (Dilger, 2010). Earlier versions of the iPhone did not have a magnetometer and thus could not measure yaw (Sadun, 2007).
 
Yang and Hsu (2010) have summarized the uses of accelerometers in measuring physical activity. Accelerometers have been used to determine static postures (standing upright, lying down), postural transitions (standing, sitting, postural sway), and gait parameters (heel strike, gait cycle frequency, stride symmetry, regularity, step length, and gait smoothness). In combination with other sensors, accelerometers can also be used to infer falling (when combined with impact detection) and energy expenditure (particularly when combined with barometric sensors to determine changes in elevation). Higher-context knowledge can be generated through accelerometry, such as restfulness during sleep, which can be inferred from the number of postural transitions during the various sleep cycles (Yang & Hsu, 2010)<nowiki>; by tracking energy expenditure, we might also be able to infer fatigue.</nowiki>
 
Accelerometers are particularly good at detecting abrupt and frequent changes in velocity, compared to position-based sensors. While reconstructing positional information from acceleration is possible but prone to error (Giansanti, Macellari, Maccioni, & Cappozzo, 2003), we hypothesize that accelerometers are particularly adept for sensing data that are related to changes in movement quality.
 
Sensors that measure the amount of contact or pressure on a surface are a special case of acceleration- or velocity-based sensors. Pressure can be thought of as the outcome of impeded movement: when a finger attempts to push past a screen, the screen stops it from moving.

== Other sensor types ==
This is a work in progress. Other sensor types will be discussed here.
== Summary (in table form) ==

{| w{| width="100%" border="1"
|
<center>'''Type of sensor''''</center>
|
<center>'''Example''''</center>
|
<center>'''Commercially available products''''</center>
|
<center>'''Fundamental movement phenomena sensed'''</center><center>'''(movement primitive)''''</center>
|
<center>'''Mid-level movement feature (inferred from movement primitives)'''</center>
|-
| rowspan="2" | Acceleration
| Gyroscopes
| iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus
| Rotational accleration
| rowspan="2" | Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)
|-
| Accelerometers
| Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote
| Linear acceleration
|-
| Touch
| Pressure sensors
| Tactex
| Pressure
| 
|-
| rowspan="3" | Position
| Vision
| Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems
| rowspan="3" | Position of body segments
| rowspan="3" | Postural information; anything that can be inferred from acceleration sensors
|-
| Magnetic
| 
|-
| Infrared
| 
|-
| rowspan="5" | Biometric
| Eye-tracking
| 
| Gaze
| Visual attention; intent
|-
| GSR
| 
| Electrical conductance of the skin
| Arousal
|-
| Breath sensors
| 
| Rate of breathing; volume of inspiration/expiration
| Arousal; energy expenditure
|-
| EMG
| 
| Electrical activity produced by skeletal muscles
| Muscular tension
|-
| Heart rate sensors
| 
| Heart rate
| Arousal; level of physical
|}

= References =

* Aggarwal, J. K., & Park, S. (2004). Human motion: Modeling and recognition of actions and interactions.
* Analog Devices. (2007). ADXL330: Small, Low Power, 3-Axis ï¿½3g iMEMSï¿½ Accelerometer. Retrieved from http://www.analog.com/static/imported-files/data_sheets/ADXL330.pdf
* Berman, S., & Stern, H. (2011). Sensors for Gesture Recognition Systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, PP(99), 1-14. doi:10.1109/TSMCC.2011.2161077
* Dilger, D. E. (2010, June 16). Inside iPhone 4: Gyro spins Apple ahead in gaming [Page 2]. AppleInsider. Retrieved August 22, 2011, from http://www.appleinsider.com/articles/10/06/16/inside_iphone_4_gyro_spins_apple_ahead_in_gaming.html&page=2
* Garg, P., Aggarwal, N., & Sofat, S. (2009). Vision Based Hand Gesture Recognition. World Academy of Science, Engineering and Technology, 49, 972ï¿½977.
* Gavrila, D. M. (1999). The Visual Analysis of Human Movement: A Survey. Computer vision and image understanding, 73(1), 82ï¿½98.
* Giansanti, D., Macellari, V., Maccioni, G., & Cappozzo, A. (2003). Is it feasible to reconstruct body segment 3-D position and orientation using accelerometric data? IEEE Transactions on Biomedical Engineering, 50(4), 476-483. doi:10.1109/TBME.2003.809490
* Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311-324. doi:10.1109/TSMCC.2007.893280
* Moore, C.-L. (1988). Beyond Words: Movement Observation and Analysis. New York, N.Y., U.S.A: Gordon and Breach Science Publishers.
* Nakata, T., Mori, T., & Sato, T. (2002). Analysis of impression of robot bodily expression. Journal of Robotics and Mechatronics, 14(1). Retrieved from http://staff.aist.go.jp/toru-nakata/LabanEng.pdf
* Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997). Visual interpretation of hand gestures for human-computerinteraction: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677-695. doi:10.1109/34.598226
* Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.
* Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.
* Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on ï¿½Concept Learning for Embodied Agents.ï¿½ Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf
* Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436
* Rowe, D. (2008). Towards Robust Multiple-Target tracking in Unconstrained Human-Populated Environments. Department of Computer Science UAB and Computer Vision Center, Barcelona, Spain.
* Sadun, E. (2007, September 10). iPhone Coding: Using the Accelerometer. TUAW - The Unofficial Apple Weblog. Retrieved August 22, 2011, from http://www.tuaw.com/2007/09/10/iphone-coding-using-the-accelerometer/
* Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564
* Schiphorst, T., Lovell, R., & Jaffe, N. (2002). Using a gestural interface toolkit for tactile input to a dynamic virtual space. Conference on Human Factors in Computing Systems (pp. 754ï¿½755).
* Schiphorst, Thecla. (2008). Bridging embodied methodologies from somatics and performance to human computer interaction (Ph.D. dissertation). School of Computing, Communications and Electronics, Faculty of Technology, University of Plymouth, United Kingdom. Retrieved from http://www.sfu.ca/~tschipho/PhD/PhD_thesis.html
* Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.
* VTI Technologies. (n.d.). CMA3000-D01 3-Axis Ultra Low Power Accelerometer with Digital SPI and I2C Interface. VTI Technologies. Retrieved from http://www.vti.fi/midcom-serveattachmentguid-1e05eb0417155de5eb011e0b896954a036ddb89db89/cma3000_d01_datasheet_8277800a.03.pdf
* Texas Instruments. (2010, December). eZ430-ChronosTM Development Tool Userï¿½s Guide. Texas Instruments. Retrieved from http://www.ti.com/lit/ug/slau292c/slau292c.pdf
* Wachs, J. P., Kï¿½lsch, M., Stern, Helman, & Edan, Y. (2011). Vision-based hand-gesture applications. Communications of the ACM, 54(2), 60. doi:10.1145/1897816.1897838
* Wisniowski, H. (2006, May 9). Analog Devices And Nintendo Collaboration Drives Video Game Innovation With iMEMS Motion Signal Processing Technology. Analog Devices, Inc. Retrieved August 22, 2011, from http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html
* Wu, Y., & Huang, T. (1999). Vision-based gesture recognition: A review. Gesture-based communication in human-computer interaction, 103ï¿½115.
* Yang, C. C., & Hsu, Y. L. (2010). A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors, 10(8), 7772ï¿½7788.
* Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking. ACM Computing Surveys, 38(4), 13-es. doi:10.1145/1177352.1177355
* Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.
* Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002

White Paper: Sensor Selection

2011-12-23T10:13:29Z

Diegom: moved White Paper: Sensor Selection to Sensor Selection

#REDIRECT [[Sensor Selection]]

Sensor Selection

2011-12-23T10:12:33Z

Diegom: /* Different sensors, different uses */

=Introduction and a cautionary tale: Why you need to choose your motion capture system carefully=
[[File:bus.jpg|200px|left]]In Vancouver, Canada, exit doors on buses are marked bus with a sign, "Touch here". Passengers interpret this instruction differently. Some firmly press the door with the palm of their hand; sometimes they leave their hand on the door until the door opens, while other times, they remove their hand the moment right after it make contact with the door. Other passengers slap or punch the door, particularly then the doors don't respond to their gesture right away. The more impatient they become, the stronger and more frequent their punches get. (This happens frequently!)
However, the doors do not respond to pressure at all. Rather, an ultrasound sensor positioned at the top of the door senses an obstruction in an ultrasonic beam. The most efficient way for a passenger to open the door is either to slowly move their hand towards the door, or to place their hand on the sign until the door opens. If their hand approaches the door too quickly or if it is withdrawn too soon, the system fails to detect the hand's presence and the door remains shut. 
The design of this system contravenes how humans have traditionally mobilized their bodies to open doors. Usually, the more direct force we apply to a door, the quicker it opens. This is a relationship that our bodies understand and that we apply to a wide variety of physical interactions with the world at large. The use of ultrasound sensors to interpret commuters' intention to exit a bus also illustrates how design decisions around the application of sensing technologies affect (for better or for worse) the way we move through an increasingly technology-intervened world. Other examples abound. We use our thumbs in a light, precise, staccato way to type out text on capacitive touch screens. We slow down when approaching secured doors so that infrared sensors can detect our presence and unlock the doors. When playing a dance game on the Microsoft Kinect, users need to orient their torsos to face the Kinect sensors directly to avoid body part occlusion, limiting the range of possible movements they can make.

= Different sensors, different uses =
Typologies of sensors used for measuring movement exist in the research literature (J. K. Aggarwal & Park, 2004; Garg, N. Aggarwal, & Sofat, 2009; Gavrila, 1999; Mitra & Acharya, 2007; Pavlovic, Sharma, & T. S. Huang, 1997; Rowe, 2008; Wachs, Kölsch, Helman Stern, & Edan, 2011; Wu & T. Huang, 1999; Yilmaz, Javed, & Shah, 2006). In a detailed review that summarizes many of the findings from earlier reviews, Berman and Stern (2011) propose a typology of sensors (shown as Figure 1) for gesture recognition systems. Their typology is organized around the properties of the sensor technologies and features three parent categories: sensor stimuli, context of use, and sensor platform. They also provide guidelines for selecting which data type to measure based on “movement frequency” (i.e., the rate at which salient aspects of the movement occur):
* Low-frequency movement: position measurements
* Intermediate-frequency movement: velocity measurement
* High-frequency movement: acceleration measurements
The authors place a large emphasis on the use of optical methods for motion capture, asserting that “in order to become universally accepted, gesture interface[s] must satisfy the ‘come as you are’ requirement”, i.e., that moving subjects should be “unencumbered” (to use Berman and Stern’s terminology) with markers, sensors, transmitters, and any other devices on their body. However, this only holds true for a subset of human activity. There are many areas of human activity that require, benefit from, or naturally incorporate some kind of encumbrance or extension of the body: conducting music with a baton, wearing a glove while boxing, cutting bamboo using a machete, and flipping a pancake are but a few examples. When we consider human interaction with existing digital technology, we note the importance of the capacitive sensors on touch-based tablets, the pressure sensors in a touch-responsive electronic piano, and the accelerometers in the Apple iPod that, when shaken, randomly selects the next track to play. In fact, movement is the primary way by which we interact with the world, and touch the fundamental relationship that initiates this interaction (Moore, 1988; Thecla Schiphorst, 2008). Thus, a wide variety of sensors must be brought to bear upon the measurement of salient aspects human movement.
 
 
[[File:BermanAndStearns2010_Typology.PNG|800px]] 
''from Berman and Stern (2011)''
 
 

On this page, we enumerate some existing sensor types and how these can be used towards movement analysis. Our review of sensor types places the embodied experience of movement as the central organizing concept. We thus include biometric sensors as part of this review, because we propose that they can be used to infer the qualia of moving when combined with conventional mechanisms for sensing movement.
== Position-based sensors ==

A wide range of position-based sensors exist, but in essence, they all capture at a given moment the position of the body in space. The types of stimuli used by position-based sensors include electric, optical (including IR and near-IR stimuli), acoustic, and magnetic systems (Berman & H. Stern, 2011) They may require the subject to wear or hold specialized equipment, or they may leave the subject unencumbered, in which case the subject needs to be separated from the background through ''segmentation''.

Gyroscopes and magnenometers can also be used in concert to measure roll, pitch, and yaw with respect to the earthï¿½s gravitational and magnetic fields as frames of reference.

== Acceleration-based sensors ==

Two types of acceleration sensors are commercially available.
 
''Linear accelerometers ''measure acceleration along a single spatial dimension. Three accelerometers can be oriented orthogonal to each other in order to measure acceleration in three dimensions. The Nintendo Wiimote's acceleration sensing system, for example, is [http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html built on] the [http://www.analog.com/en/mems-sensors/inertial-sensors/adxl330/products/product.html ADXL330], a 3-axis linear accelerometer (Analog Devices, 2007; Wisniowski, 2006). Texas Instruments produces EZ430-Chronos, a watch that [http://www.ti.com/lit/ug/slau292c/slau292c.pdf uses] the [http://www.vti.fi/en/products/accelerometers/consumer_electronics/cma3000_series/ VTI CMA3000], another 3-axis accelerometer (VTI Technologies, n.d.; Texas Instruments, 2010).
 
''Gyroscopes'' measure rotational acceleration. When the direction of gravity is known, gyroscopes can be used to measure ''pitch'' and ''roll''. Another measure of rotational movement, ''yaw'', can be derived if a device can sense its orientation with reference to the Earthï¿½s magnetic fields. The iPhone 4 has a 3-axis linear accelerometer, a 2-axis gyroscope, and a magnetometer, providing six acceleration measurements as well as orientation information (Dilger, 2010). Earlier versions of the iPhone did not have a magnetometer and thus could not measure yaw (Sadun, 2007).
 
Yang and Hsu (2010) have summarized the uses of accelerometers in measuring physical activity. Accelerometers have been used to determine static postures (standing upright, lying down), postural transitions (standing, sitting, postural sway), and gait parameters (heel strike, gait cycle frequency, stride symmetry, regularity, step length, and gait smoothness). In combination with other sensors, accelerometers can also be used to infer falling (when combined with impact detection) and energy expenditure (particularly when combined with barometric sensors to determine changes in elevation). Higher-context knowledge can be generated through accelerometry, such as restfulness during sleep, which can be inferred from the number of postural transitions during the various sleep cycles (Yang & Hsu, 2010)<nowiki>; by tracking energy expenditure, we might also be able to infer fatigue.</nowiki>
 
Accelerometers are particularly good at detecting abrupt and frequent changes in velocity, compared to position-based sensors. While reconstructing positional information from acceleration is possible but prone to error (Giansanti, Macellari, Maccioni, & Cappozzo, 2003), we hypothesize that accelerometers are particularly adept for sensing data that are related to changes in movement quality.
 
Sensors that measure the amount of contact or pressure on a surface are a special case of acceleration- or velocity-based sensors. Pressure can be thought of as the outcome of impeded movement: when a finger attempts to push past a screen, the screen stops it from moving.

== Other sensor types ==
This is a work in progress. Other sensor types will be discussed here.
== Summary (in table form) ==

{| w{| width="100%" border="1"
|
<center>'''Type of sensor''''</center>
|
<center>'''Example''''</center>
|
<center>'''Commercially available products''''</center>
|
<center>'''Fundamental movement phenomena sensed'''</center><center>'''(movement primitive)''''</center>
|
<center>'''Mid-level movement feature (inferred from movement primitives)'''</center>
|-
| rowspan="2" | Acceleration
| Gyroscopes
| iPhone, IDG500 dual-axis gyroscope, Wiimote MotionPlus
| Rotational accleration
| rowspan="2" | Postural transitions; gait information (Yang and Hsu, 2010); linearity, planarity, periodicity (based on the work by Mary Pietrowicz at UIUC)
|-
| Accelerometers
| Mobile phones (other than iPhone), Analog Devices triple axis ADXL335, Wiimote
| Linear acceleration
|-
| Touch
| Pressure sensors
| Tactex
| Pressure
| 
|-
| rowspan="3" | Position
| Vision
| Camera and webcams, icon, Kinect, Wiimote IR camera, IR-based motion capture systems
| rowspan="3" | Position of body segments
| rowspan="3" | Postural information; anything that can be inferred from acceleration sensors
|-
| Magnetic
| 
|-
| Infrared
| 
|-
| rowspan="5" | Biometric
| Eye-tracking
| 
| Gaze
| Visual attention; intent
|-
| GSR
| 
| Electrical conductance of the skin
| Arousal
|-
| Breath sensors
| 
| Rate of breathing; volume of inspiration/expiration
| Arousal; energy expenditure
|-
| EMG
| 
| Electrical activity produced by skeletal muscles
| Muscular tension
|-
| Heart rate sensors
| 
| Heart rate
| Arousal; level of physical
|}

= References =

* Aggarwal, J. K., & Park, S. (2004). Human motion: Modeling and recognition of actions and interactions.
* Analog Devices. (2007). ADXL330: Small, Low Power, 3-Axis ï¿½3g iMEMSï¿½ Accelerometer. Retrieved from http://www.analog.com/static/imported-files/data_sheets/ADXL330.pdf
* Berman, S., & Stern, H. (2011). Sensors for Gesture Recognition Systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, PP(99), 1-14. doi:10.1109/TSMCC.2011.2161077
* Dilger, D. E. (2010, June 16). Inside iPhone 4: Gyro spins Apple ahead in gaming [Page 2]. AppleInsider. Retrieved August 22, 2011, from http://www.appleinsider.com/articles/10/06/16/inside_iphone_4_gyro_spins_apple_ahead_in_gaming.html&page=2
* Garg, P., Aggarwal, N., & Sofat, S. (2009). Vision Based Hand Gesture Recognition. World Academy of Science, Engineering and Technology, 49, 972ï¿½977.
* Gavrila, D. M. (1999). The Visual Analysis of Human Movement: A Survey. Computer vision and image understanding, 73(1), 82ï¿½98.
* Giansanti, D., Macellari, V., Maccioni, G., & Cappozzo, A. (2003). Is it feasible to reconstruct body segment 3-D position and orientation using accelerometric data? IEEE Transactions on Biomedical Engineering, 50(4), 476-483. doi:10.1109/TBME.2003.809490
* Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(3), 311-324. doi:10.1109/TSMCC.2007.893280
* Moore, C.-L. (1988). Beyond Words: Movement Observation and Analysis. New York, N.Y., U.S.A: Gordon and Breach Science Publishers.
* Nakata, T., Mori, T., & Sato, T. (2002). Analysis of impression of robot bodily expression. Journal of Robotics and Mechatronics, 14(1). Retrieved from http://staff.aist.go.jp/toru-nakata/LabanEng.pdf
* Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997). Visual interpretation of hand gestures for human-computerinteraction: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677-695. doi:10.1109/34.598226
* Rett, J., Dias, J., & Ahuactzin, J. M. (2008). Laban Movement Analysis using a Bayesian model and perspective projections. In C. Rossi (Ed.), Brain, Vision and AI (pp. 183-210). Vienna: InTech Education and Publishing.
* Rett, J., Santos, L., & Dias, J. (2008). Laban Movement Analysis for multi-ocular systems. Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on (pp. 761-766). Presented at the Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on.
* Rett, Jorg, & Dias, Jorge. (2007a). Bayesian models for Laban Movement Analysis used in human machine interaction. Proceedings of ICRA 2007 Workshop on ï¿½Concept Learning for Embodied Agents.ï¿½ Retrieved from http://paloma.isr.uc.pt/pub/bscw.cgi/d56530/Paper@ICRA07-WS.pdf
* Rett, Jorg, & Dias, Jorge. (2007b). Human-robot interface with anticipatory characteristics based on Laban Movement Analysis and Bayesian models. 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 257-268). Presented at the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, Netherlands. doi:10.1109/ICORR.2007.4428436
* Rowe, D. (2008). Towards Robust Multiple-Target tracking in Unconstrained Human-Populated Environments. Department of Computer Science UAB and Computer Vision Center, Barcelona, Spain.
* Sadun, E. (2007, September 10). iPhone Coding: Using the Accelerometer. TUAW - The Unofficial Apple Weblog. Retrieved August 22, 2011, from http://www.tuaw.com/2007/09/10/iphone-coding-using-the-accelerometer/
* Santos, L., Prado, J. A., & Dias, J. (2009). Human Robot interaction studies on Laban human movement analysis and dynamic background segmentation (pp. 4984-4989). Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2009, St. Louis, MO. doi:10.1109/IROS.2009.5354564
* Schiphorst, T., Lovell, R., & Jaffe, N. (2002). Using a gestural interface toolkit for tactile input to a dynamic virtual space. Conference on Human Factors in Computing Systems (pp. 754ï¿½755).
* Schiphorst, Thecla. (2008). Bridging embodied methodologies from somatics and performance to human computer interaction (Ph.D. dissertation). School of Computing, Communications and Electronics, Faculty of Technology, University of Plymouth, United Kingdom. Retrieved from http://www.sfu.ca/~tschipho/PhD/PhD_thesis.html
* Swaminathan, D., Thornburg, H., Mumford, J., Rajko, S., James, J., Ingalls, T., Campana, E., et al. (2009). A dynamic Bayesian approach to computational Laban shape quality analysis. Advances in Human Computer Interaciton, 2009, 1-17.
* VTI Technologies. (n.d.). CMA3000-D01 3-Axis Ultra Low Power Accelerometer with Digital SPI and I2C Interface. VTI Technologies. Retrieved from http://www.vti.fi/midcom-serveattachmentguid-1e05eb0417155de5eb011e0b896954a036ddb89db89/cma3000_d01_datasheet_8277800a.03.pdf
* Texas Instruments. (2010, December). eZ430-ChronosTM Development Tool Userï¿½s Guide. Texas Instruments. Retrieved from http://www.ti.com/lit/ug/slau292c/slau292c.pdf
* Wachs, J. P., Kï¿½lsch, M., Stern, Helman, & Edan, Y. (2011). Vision-based hand-gesture applications. Communications of the ACM, 54(2), 60. doi:10.1145/1897816.1897838
* Wisniowski, H. (2006, May 9). Analog Devices And Nintendo Collaboration Drives Video Game Innovation With iMEMS Motion Signal Processing Technology. Analog Devices, Inc. Retrieved August 22, 2011, from http://www.analog.com/en/press-release/May_09_2006_ADI_Nintendo_Collaboration/press.html
* Wu, Y., & Huang, T. (1999). Vision-based gesture recognition: A review. Gesture-based communication in human-computer interaction, 103ï¿½115.
* Yang, C. C., & Hsu, Y. L. (2010). A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors, 10(8), 7772ï¿½7788.
* Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking. ACM Computing Surveys, 38(4), 13-es. doi:10.1145/1177352.1177355
* Zhao, L. (2001). Synthesis and acquisition of laban movement analysis qualitative parameters for communicative gestures (Ph.D. dissertation). University of Pennsylvania, Philadelphia, PA, USA.
* Zhao, Liwei, & Badler, N. (2005). Acquiring and validating motion qualities from live limb gestures. Graphical Models, 67(1), 1-16. doi:10.1016/j.gmod.2004.08.002

2011-12-16T19:43:53Z

Diegom: /* Temporal aliasing of periodic motion */

= '''Kinect libraries and APIs''' =

Several different libraries exist for the Kinect. Some of these have been installed in the Blackbox computers. 

* Microsoft SDK: Microsoft's official SDK for the Kinect is installed on the only Windows 7 machine in the Blackbox and is clearly labeled as such.
* OpenNI: The OpenNI libary

Other libraries (which have not been installed in the Blackbox machines yet) include the following:

* libfreenect
* iPi

One word of caution: some of these libraries cannot be installed on the same macine. For instance, the OpenNI library should not be installed along with the Microsoft Kinect SDK.

== Using Kinect motion data ==

You can export Kinect motion capture data into BVH format, a standard motion data format that can be imported into, say, Credo Interactive's DanceForms 2.0 choreography and animation software. The software is available on http://tech.integrate.biz/kinect_mocap.htm

= Errors in using the Kinect for motion tracking =

Three kinds of errors that can arise from using the Kinect for motion tracking: ''temporal aliasing'', ''spatial resolution errors'', and ''occlusion errors.''

== Temporal aliasing ==

The Nyquist theorem states that given a sampling rate ''f, ''any frequency above what is known as the Nyquist frequency (which is ''f''/2) will not be reconstructed properly. What does this mean in terms of motion capture? This means that movement that is ''very fast'' and ''nonlinear ''may suffer from aliasing. Two kinds of nonlinear motion are affected by temporal aliasing: 

* ''Periodic ''motion. Examples of period motion include: an arm being swung around in a wide circle; jumping up and down; a finger tracing a sinusoidal pattern in the air; a repeated dabbing motion.
* ''Non-periodic, non-monotonic''. Examples include: a single, martial-arts style punch in the air; starting from the time the hand extends from the chest and finishing with the arm near the chest again; tracing random, curvy patterns in the air with your nose

We consider each of these two cases in turn. 

=== Temporal aliasing of periodic motion ===

Consider the movement of a single point on the human body through 3D space, measured with reference to a 3D coordinate system that is fixed to the room. Let ''x''''g''(''t''), ''y''''g''(''t''), and ''z''''g''(''t'') represent the true (ground truth) values of the position of the limb'', ''as measured within that coordinate system, at time ''t''. We can plot the position of that point along each of three axes. For the purposes of illustration, imagine the position of the tip of a finger as it steadily traces a sine wave along a plane perpendicular to the ''z'' axis, moving from "left to right", that is, from one end of the ''x ''axis to the other. If we assume that the coordinate position (0, 0, 0) is located near the finger of the mover, the motion can be plotted along the three reference axes, and might look something like this: {|[[Image:x_g_t.PNG|x(t)]] | [[Image:y_g_t.PNG|y(t)]] | [[Image:z_g_t.PNG|z(t)]]|} The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? Since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (We assume here that the Kinect can sample equally well along all three spatial dimensions.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the ''Nyquist frequency''). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. [[Image:y_g_t_sampling.PNG|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|reconsturcted]] 

=== Temporal aliasing of aperiodic, nonmonotonic motion ===

However, the human body ''can ''do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) [[Image:punch1.png|punch]] [[Image:punch2.png|punch 2]] [[Image:punch3.png|punch 3]] This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is. In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of ''popping ''hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) 

== Spatial resolution errors ==

Khoshelham (2011) reports that random error of Kinect depth measurements increases quadratically with increasing distance from the sensor. The maximum random error is 4 cm. Khoshelham concludes that at a distance beyond the optimal distance of 1-3 meters, the quality of the data is degraded by noise and low spatial resolution. Keep this in mind when you plan your motion capture activities with the Kinect. For more about 

== Occlusion errors ==

If you are using the Microsoft SDK to perform skeleton tracking, know that the SDK can sometimes infer joint positions when the joint is occluded (Fernandez, 2011). You can query the SDK on the quality of the skeleton data. Refer to [http://channel9.msdn.com/Series/KinectSDKQuickstarts/Skeletal-Tracking-Fundamentals#time=1m24s this video] find out more about how to do this. The SDK allows you to apply filtering to smooth out "skeleton jitter" (Fernandez, 2011), but you will lose movement information through this smoothing. Whether the loss is significant depends on how you need to use the motion data.

= References =

Khoshelham, K. (2011). Accuracy analysis of kinect depth data. ''ISPRS Workshop Laser Scanning'' (Vol. 38, p. 1). Retrieved December 16, 2011, from http://www.isprs.org/proceedings/XXXVIII/5-W12/Papers/ls2011_submission_40.pdf Fernandez, D. (2011, June 16). Skeletal Tracking Fundamental. ''Kinect for Windows SDK Quickstarts''. Retrieved December 16, 2011, from http://channel9.msdn.com/Series/KinectSDKQuickstarts/Skeletal-Tracking-Fundamentals#time=1m24s

File:Z g t.PNG

2011-12-16T19:41:06Z

Diegom: