00:05Hi there everyone, I'm Sean Tracy and for those that don't know me, I'm the Technical Director of Content here
00:11at Cloud Imperium Games and today I'm really lucky to be able to show you a feature that I'm personally
00:16really excited about and this is a collaboration between Faceware Technologies and Cloud Imperium Games and what I'm going to
00:22show you is how we've integrated facial tracking into Star Citizen.
00:26Here I am. You can see that there's a bunch of different dots on my face and these are landmark
00:30points that we use for facial tracking and the tracking is quite intelligent so as I turn my head and
00:35a face is lost we actually disable the facial tracking and if I turn this way this is handy for
00:40things like occlusion so if your hand gets in the way maybe a pen gets in the way we disable
00:45the facial tracking in those instances. Now you're not here to see my face let's go ahead and take a
00:50look at how this actually maps to a character.
00:53So you may recognize this gentleman this is Liam Cunningham who plays Captain White in Squadron 42.
01:00One of the really important things to notice or to note rather is that this is still early days with
01:06this technology.
01:07We're doing a lot to map and to improve the quality of the actual tracking as well as adding new
01:13phonemes in to the tracking so that the runtime logic reacts exactly as we expect it to.
01:18So this is never meant to be a replacement for facial animation but rather something for the players to be
01:24able to run and to drive an actor in real time or a character rather in real time.
01:30So to explain how this all kind of works is that we've got a bunch of outputs that happen from
01:35the live SDK technology and then through a layer called motion logic we map all of that to inputs of
01:41our runtime rig.
01:42And since every character within Star Citizen has its own unique runtime rig created dynamically they will always react appropriately
01:50to whatever the inputs are.
01:51So it won't exactly be my blinks it'll be Liam Cunningham's blinks.
02:01You can see even things like the ears moving when the scalp moves.
02:27So let's go ahead and show how actually different bits of rig logic might react a little bit differently giving
02:33a different character essentially.
02:37As Belosa he has really quite nice wrinkle maps that happen on the side of his face.
02:44So if I pull my face to the side.
02:50Where this feature really comes into its own is in the combination of it with our VoIP technology.
02:56We've talked about how this feature can work on a consumer level web camera.
03:00However, here's Peter Bush from Faceware to talk about a motion sensor they've developed specifically for this feature.
03:07Hi, my name is Peter Bush. I'm one of the co-founders of Faceware Technologies and Faceware Interactive.
03:13Faceware is a facial motion capture company.
03:15We create hardware and software products that can create animation simply from video.
03:20About a year ago Cloud Imperium asked us to do something we've never done before.
03:23Integrate our real-time technology into engine and allow game characters to interact and emote in real time.
03:29But by being live driven by the players themselves.
03:32This is quite an engineering challenge.
03:34But I knew my teams in Austin and Los Angeles could collaborate with the incredibly talented Cloud Imperium development team
03:40to make this happen.
03:42The new VoIP feature will work with nearly any camera.
03:45But we were asked what's going to create the best possible experience.
03:48So we set out to engineer the first gaming device designed specifically for facial tracking.
03:56Here it is. The first prototype facial motion sensor.
04:00This is the first one off the production line.
04:03Now there are many specs about this device that are going to give you the best possible experience for facial
04:07tracking.
04:08First, the image sensor runs at 60 frames per second.
04:11Now while that's important, is the way in which our technology works, from one video frame to the next, we
04:16basically can create an animated result.
04:18Meaning what we see in that first video frame, we're going to analyze the expression on your face.
04:22And by the next video frame, we have that animated result.
04:25So the faster that that frame rate is running, the faster you're going to be able to keep up and
04:29get that performance.
04:31And at 1 25th or 1 30th of a second, which is common in most webcams, it might not keep
04:36up and your lip sync might not read.
04:38But at 1 60th of a second, now the words that you're saying are going to be spoken by your
04:43character running in game.
04:45Now second, we looked at where most common you're going to be using this device.
04:48And more often than not, you're only going to be lit by the light coming from your monitor.
04:52So the sensor had to work in a low light environment.
04:55So we selected a 1 3rd inch 3.4 megapixel sensor that's capable of 720 or 1080 at 60 frames
05:03per second in a near dark environment.
05:06Third, most webcams are a wide angle lens, which makes your face a little bit smaller in frame and a
05:12lot less facial details and facial pixels for our technology to capture and track.
05:17So we selected a single user field of view.
05:20Now this allows our technology to basically get more pixels from your face from that frame and get the best
05:26possible facial details in the engine.
05:29Now fourth, we put on a high resolution glass lens that's going to resolve that data and give us the
05:35best possible and sharpest focus for facial tracking.
05:38These specs are going to allow you to have the best possible experience with the new VoIP feature.
05:43While this is only a prototype, we're going to go into mass production and we're going to bring this to
05:47you.
05:48We've been working on this technology for nearly seven years, but this is the first game that's wanted to integrate
05:53this into engine and allow character to character communication like you've never seen before.
05:59We're excited to bring this to Star Citizen and we can't wait to see what you do with it.
06:04Having integrated this technology, there's a lot of ideas and things we wanted to try with it.
06:08So one of the ideas that we came up with was that because we already know the position of the
06:14face itself, we actually know where the head is.
06:16So we can do something pretty exciting that I find very fun is head tracking.
06:23So as I move my head in real life, all the camera is moved just as if you were driving
06:30a track IR sort of simulation.
06:32But this is again all from a consumer level web camera.
06:35So this actually adds quite a bit.
06:37I find that this gives me a ton of situational awareness when I'm flying around.
06:40I can just move my head a little bit more.
06:42I can check on my power.
06:43I can check on my shields.
06:45I can even inspect things a little bit up close if I combine this with zoom.
06:49So we've got some towers over there.
06:56You can even do crazy stuff like rolls.
07:00Roll the other way.
07:02And it's kind of fun if we combine it with this.
07:04So here I'll try to keep my head on the horizon until it's just too much.
07:09And then we put it back over and maybe look up.
07:13Let's see if we can see anything from Levski there.
07:15Oh yeah.
07:16Cool.
07:17All right.
07:18And then we'll roll it back over.
07:21One of these.
07:21Whoop.
07:22That was the other way.
07:24And there we go.
07:27So as you can see, it is very natural.
07:31And you can just look around within it.
07:32And again, it's kind of coming for free because we are already tracking the facial position.
07:36So we can extrapolate from that what the head position is.
07:39So as you can see, the collaboration between Faceware and Cloud Imperium Games, even in this early stage of development,
07:47is already breaking new ground and doing things that haven't really been seen before.
07:51So we're really excited to develop these features and we can't wait to get them to you.
07:55So until then, this is Sean Tracy from Cloud Imperium Games Los Angeles saying goodbye.
08:14Eric's got a little song that he sometimes kind of comes and sings and it looks pretty awesome.
08:19So if you want to take it away.
08:22I once was a man outside.
08:30Perfect.
Commenti