Xbox 360 Kinect

Xbox 360 Kinect

1. Depth detection

Depth detection is the core technology of Kinect. Developers can obtain the user’s depth information through Kinect and determine the user’s location. The depth detection of Kinect v1 uses the method of Light Coding: an infrared projector projects an infrared spectrum, and after irradiating a rough object or frosted glass, the spectrum is distorted and speckle is formed. Because speckle is highly random, each speckle in space has a different pattern. When such structured light is applied in the space, the entire space is marked. When the object is placed in this space, the infrared camera receives the feedback information, and can judge the position of the object according to the speckle pattern on the object to form a depth image. The depth detection of Kinect v2 uses the Time Of Light method: the infrared camera is used to project infrared rays to form reflected light, and the position of the object is judged according to the time of flight of the light to form a depth image.

2. Skeleton Tracking

In order to track the skeleton of the user, it is necessary to construct a skeleton diagram of the user and track the corresponding skeleton nodes. The first step is to identify the “human body” target from the depth image, that is, to track any “large” shaped object from the depth image, where clothing similar to the human body is also marked. Then scan the depth image pixels of these areas point by point to determine which part of the human body belongs to, including computer graphics techniques such as edge detection and noise threshold processing. Through this technique, the human body is finally distinguished from the background environment. After the system separates the human body from the depth image, the next step is to identify the various parts of the human body. Due to the difference in height, shortness, fatness and thinness of people, when users perform the same action, they will show different states. The machine cannot exhaustively list all human body postures, and can only abstract and understand common postures and behaviors through machine learning. Human intentions are classified by the eigenvalues of various parts of the human body. After obtaining different parts of the human body, the joint points can be identified. The joints of the human body are connected to the trunk, limbs, head, etc., and there will be many overlapping places, so it is necessary to analyze and machine learning from multiple angles such as the front and the side. According to each possible pixel, the joint point is determined to form the skeletal system.

3. Face recognition

Face recognition also needs the support of machine learning. The first step is to locate the existence of the face, and then to further analyze the input face image or video based on the facial features of the person, including the position, size of the face and the size of each facial organ. Location information, based on this information, the identity features contained in each face are further extracted and compared with known faces to identify each person’s identity.

4. Voice recognition

Speech recognition in Kinect includes many levels of technology, such as the simplest “voice command”, voice feature recognition, language recognition, word segmentation, tone and tone sexiness detection, etc. The audio data stream captured by the Kinect microphone array is processed through an audio enhancement effect algorithm to mask out ambient noise. Even in a large space, the recognition of voice commands can be performed even if the person is at a certain distance from the microphone. Kinect array technology includes effective noise cancellation and echo suppression algorithms, while beamforming technology is used to locate the sound source through the response time of each individual device, and to avoid the influence of environmental noise as much as possible.