00:00So my name is Karim and I get to talk about the brains. This is the really exciting part. This
00:04is where the full capability of Microsoft as a company all comes together. The engineers on our team get to
00:10work with top leading experts around the world in Microsoft research in fields such as computer vision, machine learning, speech
00:17recognition, facial recognition, digital signal processing, and we harness all that creative energy and innovation, combine it with the production
00:25engineering know-how on our team and combine it into the platform that you're about to see today.
00:30I'll point out the anatomical correctness. His hips are in the correct location. He has multiple spine joints. He has
00:36true shoulders and collarbones, multiple joints in his head and neck. So he can bend over, he can bend forward,
00:42he can move his head, shrug his shoulders.
00:45So, you know, we're connected. The first version was really about understanding gross human motion. We have true human understanding
00:51here and fidelity of expression as he moves around.
00:55Another key advancement I'd like to point out is the new joints we've added in the hands. We have joints
01:00that represent tip of hand and the thumb. So as Scott is opening and closing his hands, you can see
01:04it being represented on screen.
01:06The skeleton is a great representation of the various bones and how they're connected together, but we took it a
01:12step further and started with orientation.
01:15Now this is Block Man, as the team calls him, and what Block Man represents is the actual orientation of
01:20Scott's body as he's moving in 3D space.
01:23As he rotates his hands and his arms and his legs, you can see that Block Man is, you know,
01:28very accurately depicting what he's doing. So we're kind of getting beyond the joints.
01:31We have the actual rotations that his joints are experiencing. He can even draw small nose circles in the air
01:37and we can see his head rotating on Block Man in line with what it is that Scott's doing.
01:43Now, I'm a pretty tall guy. I'm 6'4 and my skeleton only weighs 8 pounds. If you really want
01:49to understand what's going on with a human, you need to understand and describe what's going on with the muscles,
01:54the forces, the torques.
01:55So that's our intro into Muscle Man. So Muscle Man is a real-time human-based physics model applied on
02:03top of our next-generation skeleton and our high-resolution depth map.
02:07What the colors represent is green equals no force and then the highest intensity red is the largest level of
02:13force.
02:14As you see as Scott kind of shifts his weight between one leg and the next, as he lifts a
02:19foot, you can see the red disc representing the greater force, you know, on the floor being exerted.
02:24You can see changes in his core and his legs. If I can get Scott to do a little bit
02:28of a squat for me. All right, go ahead and hold it.
02:31Like he's feeling it in his legs and you can see it on the screen. You know, normally I make
02:36him do that for a long time, but I'll go easy on him today.
02:39If we get Scott to jump up in the air as high as he can, you'll see that his body
02:43goes all green. You know, when you're in the air, obviously there's no force being felt.
02:47And when he hits the ground, you can see those red impact points. You know, beyond just understanding the forces
02:52in his muscles, we're able to estimate, you know, mass and velocity in the force of something like a punch.
02:57Give me a little jab. As you can see the nice little white circles that represent how much he's exerting.
03:02Now really put your back into it and shift your weight.
03:05So look at the size of that circle. So we're, you know, accurately calculating how much force he's exerting.
03:10You can see the weight shifting from his back leg to his front. You can see his core tightening up
03:14and you can see the representation on screen.
03:17So, you know, with these three things combined, you have a fantastic understanding of what's happening with the human body.
03:22But as any good trainer will tell you, this information is necessary, but it's not enough. You need one more
03:27thing. You need heart rate.
03:29So what we have here is a real time technology that is actually detecting the pulse in Scott's face.
03:36We're using our high definition color feed and our active IR feed to look for subtle variations in both color
03:44and intensity in the skin of his face.
03:46They're imperceptible to the human eye, but, you know, our Kinect sensor and the Xbox One hardware is able to
03:51pick it up.
03:52We're literally reading the pulse in his face. Pretty amazing stuff. You know, fantastic for exercise scenarios or for measuring
04:00exertion. Lots of possibilities.
04:02Please.
Commenti