Kinect - Revision history

Diegom: /* Temporal aliasing of aperiodic, nonmonotonic motion */

2011-12-16T20:43:28Z

‎Temporal aliasing of aperiodic, nonmonotonic motion

Diegom: /* Temporal aliasing of periodic motion */

2011-12-16T20:37:04Z

‎Temporal aliasing of periodic motion

Diegom at 20:30, 16 December 2011

2011-12-16T20:30:44Z

Diegom: /* References */

2011-12-16T20:27:18Z

‎References

Diegom: /* Spatial resolution errors */

2011-12-16T20:26:52Z

‎Spatial resolution errors

Diegom at 19:50, 16 December 2011

2011-12-16T19:50:46Z

Diegom at 19:47, 16 December 2011

2011-12-16T19:47:57Z

Diegom: /* Temporal aliasing of aperiodic, nonmonotonic motion */

2011-12-16T19:46:18Z

‎Temporal aliasing of aperiodic, nonmonotonic motion

Diegom: /* Temporal aliasing of periodic motion */

2011-12-16T19:43:53Z

‎Temporal aliasing of periodic motion

Diegom: Added a lot of stuff

2011-12-16T19:39:11Z

Added a lot of stuff

@@ Line 40: / Line 40: @@
 === Temporal aliasing of aperiodic, nonmonotonic motion ===
-However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) <br /><br />[[Image:punch1.png|border|punch]][[Image:punch2.png|border|punch 2]][[Image:punch3.png|border|punch 3]]<br /><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
+However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most obvious example is a punch by a highly-skilled martial artist. (The motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) It may be difficult to capture the movements of this guy, for example: http://www.youtube.com/watch?v=qdSY-_qs_mg <br /><br />[[Image:punch1.png|border|punch]][[Image:punch2.png|border|punch 2]][[Image:punch3.png|border|punch 3]]<br /><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
 == Spatial resolution errors ==

@@ Line 36: / Line 36: @@
 <br /><br />
-The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? Since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (We assume here that the Kinect can sample equally well along all three spatial dimensions.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the <span>''Nyquist frequency''</span>). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. <br /><br /> The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. <br /><br />[[Image:y_g_t_sampling.PNG|border|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|border|reconsturcted]]<br />
+The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? For this particular example, since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (It should be noted at this point that the the Kinect's depth sampling performance falls quadratically with the distance from the lens. See the section below on spatial resolution errors.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the <span>''Nyquist frequency''</span>). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. <br /><br /> The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. <br /><br />[[Image:y_g_t_sampling.PNG|border|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|border|reconsturcted]]<br />
 === Temporal aliasing of aperiodic, nonmonotonic motion ===

@@ Line 33: / Line 33: @@
 Consider the movement of a single point on the human body through 3D space, measured with reference to a 3D coordinate system that is fixed to the room. Let <span>''x''</span><sub>''g''</sub>(<span>''t''</span>), <span>''y''</span><sub>''g''</sub>(<span>''t''</span>), and <span>''z''</span><sub>''g''</sub>(<span>''t''</span>) represent the true (ground truth) values of the position of the limb<span>'', ''</span>as measured within that coordinate system, at time <span>''t''</span>. We can plot the position of that point along each of three axes. For the purposes of illustration, imagine the position of the tip of a finger as it steadily traces a sine wave along a plane perpendicular to the <span>''z''</span> axis, moving from "left to right", that is, from one end of the <span>''x ''</span>axis to the other. If we assume that the coordinate position (0, 0, 0) is located near the finger of the mover, the motion can be plotted along the three reference axes, and might look something like this:
-<br /><br />[[Image:x_g_t.PNG|x(t)]][[Image:y_g_t.PNG|y(t)]][[Image:z_g_t.PNG|z(t)]]
+<br /><br />[[Image:x_g_t.PNG|border|x(t)]][[Image:y_g_t.PNG|border|y(t)]][[Image:z_g_t.PNG|border|z(t)]]
 <br /><br />
-The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? Since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (We assume here that the Kinect can sample equally well along all three spatial dimensions.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the <span>''Nyquist frequency''</span>). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. <br /><br /> The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. <br /><br />[[Image:y_g_t_sampling.PNG|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|reconsturcted]]<br />
+The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? Since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (We assume here that the Kinect can sample equally well along all three spatial dimensions.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the <span>''Nyquist frequency''</span>). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. <br /><br /> The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. <br /><br />[[Image:y_g_t_sampling.PNG|border|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|border|reconsturcted]]<br />
 === Temporal aliasing of aperiodic, nonmonotonic motion ===
-However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) <br /><br />[[Image:punch1.png|punch]][[Image:punch2.png|punch 2]][[Image:punch3.png|punch 3]]/><br /><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
+However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) <br /><br />[[Image:punch1.png|border|punch]][[Image:punch2.png|border|punch 2]][[Image:punch3.png|border|punch 3]]<br /><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
 == Spatial resolution errors ==

← Older revision		Revision as of 20:27, 16 December 2011
Line 53:		Line 53:

	Khoshelham, K. (2011). Accuracy analysis of kinect depth data. ''ISPRS Workshop Laser Scanning'' (Vol. 38, p. 1). Retrieved December 16, 2011, from http://www.isprs.org/proceedings/XXXVIII/5-W12/Papers/ls2011_submission_40.pdf<br /><br /> Fernandez, D. (2011, June 16). Skeletal Tracking Fundamental. ''Kinect for Windows SDK Quickstarts''. Retrieved December 16, 2011, from http://channel9.msdn.com/Series/KinectSDKQuickstarts/Skeletal-Tracking-Fundamentals#time=1m24s		Khoshelham, K. (2011). Accuracy analysis of kinect depth data. ''ISPRS Workshop Laser Scanning'' (Vol. 38, p. 1). Retrieved December 16, 2011, from http://www.isprs.org/proceedings/XXXVIII/5-W12/Papers/ls2011_submission_40.pdf<br /><br /> Fernandez, D. (2011, June 16). Skeletal Tracking Fundamental. ''Kinect for Windows SDK Quickstarts''. Retrieved December 16, 2011, from http://channel9.msdn.com/Series/KinectSDKQuickstarts/Skeletal-Tracking-Fundamentals#time=1m24s
		+	<br /><br />
		+	Smisek, J., Jancosek, M., & Pajdla, T. (2011). 3D with Kinect. Presented at the 1st IEEE Workshop on Consumer Depth Cameras for Computer Vision, Barcelona, Spain. Retrieved from ftp://cmp.felk.cvut.cz/pub/cvl/articles/pajdla/Smisek-CDC4CV-2011.pdf

@@ Line 44: / Line 44: @@
 == Spatial resolution errors ==
-Khoshelham (2011) reports that random error of Kinect depth measurements increases quadratically with increasing distance from the sensor. The maximum random error is 4 cm. Khoshelham concludes that at a distance beyond the optimal distance of 1-3 meters, the quality of the data is degraded by noise and low spatial resolution. Keep this in mind when you plan your motion capture activities with the Kinect. For more about <br />
+Khoshelham (2011) reports that random error of Kinect depth measurements increases quadratically with increasing distance from the sensor. The maximum random error is 4 cm. Khoshelham concludes that at a distance beyond the optimal distance of 1-3 meters, the quality of the data is degraded by noise and low spatial resolution. Keep this in mind when you plan your motion capture activities with the Kinect. For more experimental results on Kinect's spatial resolution, see also Smisek, Jancosek, & Pajdla (2011).
 == Occlusion errors ==

@@ Line 32: / Line 32: @@
 === Temporal aliasing of periodic motion ===
-Consider the movement of a single point on the human body through 3D space, measured with reference to a 3D coordinate system that is fixed to the room. Let <span>''x''</span><sub>''g''</sub>(<span>''t''</span>), <span>''y''</span><sub>''g''</sub>(<span>''t''</span>), and <span>''z''</span><sub>''g''</sub>(<span>''t''</span>) represent the true (ground truth) values of the position of the limb<span>'', ''</span>as measured within that coordinate system, at time <span>''t''</span>. We can plot the position of that point along each of three axes. For the purposes of illustration, imagine the position of the tip of a finger as it steadily traces a sine wave along a plane perpendicular to the <span>''z''</span> axis, moving from "left to right", that is, from one end of the <span>''x ''</span>axis to the other. If we assume that the coordinate position (0, 0, 0) is located near the finger of the mover, the motion can be plotted along the three reference axes, and might look something like this:<br /><br />{|[[Image:x_g_t.PNG|x(t)]]  | [[Image:y_g_t.PNG|y(t)]] | [[Image:z_g_t.PNG|z(t)]]|} The question is: at what point would this motion be erroneously reconstructed from Kinect sampling because of temporal aliasing? Since the x and z components of the motion will always be reconstructed correctly, we focus on the reconstruction of the y component. (We assume here that the Kinect can sample equally well along all three spatial dimensions.) Given the Kinect sampling rate is 30 Hz, the Nyquist theorem predicts the frequency of the finger moving must not exceed 15 Hz (called the <span>''Nyquist frequency''</span>). That's an awfully fast finger, and so we can safely use the Kinect to sample this motion. In fact, any other kind of periodic motion of the body (swinging an arm in a large circle, jumping up and down) will happen at a frequency far less than the Nyquist frequency. <br /><br /> The image below shows how a periodic motion at a frequency less than the 15Hz can be reconstructed from Kinect sampling data. <br /><br />[[Image:y_g_t_sampling.PNG|sampled regular]][[Image:y_g_t_sampling_reconstructed.PNG|reconsturcted]]<br />
+Consider the movement of a single point on the human body through 3D space, measured with reference to a 3D coordinate system that is fixed to the room. Let <span>''x''</span><sub>''g''</sub>(<span>''t''</span>), <span>''y''</span><sub>''g''</sub>(<span>''t''</span>), and <span>''z''</span><sub>''g''</sub>(<span>''t''</span>) represent the true (ground truth) values of the position of the limb<span>'', ''</span>as measured within that coordinate system, at time <span>''t''</span>. We can plot the position of that point along each of three axes. For the purposes of illustration, imagine the position of the tip of a finger as it steadily traces a sine wave along a plane perpendicular to the <span>''z''</span> axis, moving from "left to right", that is, from one end of the <span>''x ''</span>axis to the other. If we assume that the coordinate position (0, 0, 0) is located near the finger of the mover, the motion can be plotted along the three reference axes, and might look something like this:<br /><br />[[Image:x_g_t.PNG|x(t)]][[Image:y_g_t.PNG|y(t)]][[Image:z_g_t.PNG|z(t)]]<br /><br /
 === Temporal aliasing of aperiodic, nonmonotonic motion ===
-However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) <br />{|[[Image:punch1.png|punch]]|[[Image:punch2.png|punch 2]]|[[Image:punch3.png|punch 3]]|}/><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
+However, the human body <span>''can ''</span>do very fast, nomonotonic, aperiodic movement, which is susceptible to temporal aliasing. The most common example is a punch by a highly-skilled martial artist. (Th motion capture technician at Emily Carr, Rick Overington, has reported this to be true at their own facilities.) <br /><br /[[Image:punch1.png|punch]][[Image:punch2.png|punch 2]][[Image:punch3.png|punch 3]]/><br /> This phenomenon can also be understood in the context of the Nyquist theorem. Any movement gesture or phrase can be seen as a finite signal in 3 dimensions that is decomposable into a Fourier series. In the case of this example of a martial arts punch, one of the components of series is a high-amplitude signal with a frequency that greater than the Nyquist frequency. (An example of a movement that contains a low-amplitude signal with a high frequency might be very strong shivering.) This component will be aliased upon reconstructed from the sampled signal. And since this component is high amplitude and thus critical to our perception of the movement, an aliased reconstruction of the movement will be perceptually significantly different from the original gesture. The movement will be "smoothed out", appearing less jerky than it really is.<br /><br /> In fact, the usefulness of the Kinect for sampling needs to be closely paid attention to for any movement that contains very rapid changes in velocity (which is the first derivative of position as a function of time) or acceleration (which is the second derivative). For instance, the expressivity of the urban dance form of <span>''popping ''</span>hinges precisely on very rapid and sophisticated changes in acceleration. ([http://www.youtube.com/playlist?list=PLB52F45219B7297B9&feature=view_all This is a a playlist of popping videos from YouTube.]) <br />
 == Spatial resolution errors ==

@@ Line 1: / Line 1: @@
-'''TODO''':
+= '''Kinect libraries and APIs''' =
-* Where the Vicon equipment lives
+Several different libraries exist for the Kinect. Some of these have been installed in the Blackbox computers. <br />
-Kinect resources:
+* Microsoft SDK: Microsoft's official SDK for the Kinect is installed on the only Windows 7 machine in the Blackbox and is clearly labeled as such.
 You can export Kinect motion capture data into BVH format, a standard motion data format that can be imported into, say, Credo Interactive's DanceForms 2.0 choreography and animation software. The software is available on http://tech.integrate.biz/kinect_mocap.htm
-There are quality parameters that are associated with how reliable the depth data is (see 8:11 in http://www.brekel.com/?p=731)
+Khoshelham, K. (2011). Accuracy analysis of kinect depth data. ''ISPRS Workshop Laser Scanning'' (Vol. 38, p. 1). Retrieved December 16, 2011, from http://www.isprs.org/proceedings/XXXVIII/5-W12/Papers/ls2011_submission_40.pdf<br /><br /> Fernandez, D. (2011, June 16). Skeletal Tracking Fundamental. ''Kinect for Windows SDK Quickstarts''. Retrieved December 16, 2011, from http://channel9.msdn.com/Series/KinectSDKQuickstarts/Skeletal-Tracking-Fundamentals#time=1m24s