00:00PEGI 3
00:23Here at Milestone we're working on a revolution
00:26and this revolution is called neural AI
00:29It's the first time that it's been implemented in a racing video game
00:33and it will change the player's perception
00:35First we will apply this to the artificial intelligence of the player's opponents
00:39It's a tool that has infinite possibilities
00:42To work on this we search for collaborators who are experts in this sector
00:47We found them surprisingly close to home here in Italy
00:51A company called Aerobics
00:56AI is the part of a game
00:58that determines the behavior of characters or vehicles in this case
01:03that are not controlled by a human
01:06They have to react to situations occurring in an environment
01:09and in traditional AI they do that based on instructions written by a programmer
01:14who comes up with the rules describing how the agent must react to that environment
01:18Neural AI, on the other hand, is the bottom-up counterpart
01:23where agents learn autonomously which controls they need to operate
01:28in order to achieve an effect of realism or performance
01:33For training neural networks through reinforcement learning
01:36We use tools that have become the state of the art
01:40We use Pytorch
01:42An open source framework that we know well and that allows us to do a lot
01:47because it doesn't impose a lot of burden on those who use it
01:52With it we can do our job in the simplest way possible
01:56and focus on the hard problems
01:58which are a whole other world of issues associated with training in-game
02:05The designers need to be teachers
02:08and this is very different to their previous role
02:11In the past we just had to balance out variables
02:14Now we have to learn how to teach
02:16and this has completely changed our perspective on artificial intelligence
02:21We had to begin structuring a whole new area of work
02:24We tell the neural AI which objectives it needs to complete
02:30We tell it which actions will give it a negative result
02:33and which actions a positive result
02:36And then it's up to the AI to learn which way is best to obtain the objective
02:41The result is a very realistic experience
02:45similar to a human player
02:47But this is an emerging result as opposed to a top-down prescription
02:56A neural network can recall the neuron as a conceptual analogous
03:02Mathematically they are quite close, biologically they are not
03:06A neural network model takes an input
03:09It multiplies it by numbers
03:11And those numbers are the results of training
03:13They are estimated automatically
03:15How do we determine those numbers?
03:18We provide an objective
03:20And we try to determine the values
03:22So that the objective is achieved
03:24It's the objective that we prescribe in the case of the game
03:27And the objective is to go as fast as possible
03:35We started out by studying literature
03:37Basically checking what is available
03:39And what the state of the art in this field of reinforcement learning is
03:43The most popular model is actor-critic
03:46Aptly named because one of its components defined as the actor
03:50Is the part of the model that makes decisions
03:53In the case of autonomous driving
03:55It's all about accelerating, braking, turning left or right etc
04:01Once the action has been triggered
04:03Another component is responsible for evaluating that decision
04:07When the neural AI is at the beginning of its learning process
04:11In its infancy
04:12It starts to explore
04:14In scientific literature this is called
04:17The trade-off between exploration and exploitation
04:19But when it comes to exploration
04:22I'll trigger certain actions to see how it reacts
04:26And what the consequences are
04:27For exploitation I'll use my knowledge and experience
04:32To obtain a reward
04:35Neural AI doesn't start with commands
04:37But rewards where it will receive or lose points
04:42Basically it's told what is good and what is bad
04:44How to gain points and how to lose them
04:48After this it's given control of a motorbike
04:51And free to move around
04:55We give it a series of episodes throughout its learning process
04:59And for each episode we observe how the race concluded
05:03We analyze its performance and assign points
05:07We can go back through the history of the model
05:12And verify which inputs contributed the most to a certain output
05:20We had to design eyes for the neural AI
05:23We had to give it sensations
05:25However, what makes the biggest difference
05:27Is that the neural AI remembers what has happened
05:30This is an enormous advantage
05:32Because it learns from its mistakes
05:40We chose neural AI because it has the potential of adaptability
05:44And complexity that is superior to the traditional model
05:48It can internalize the driving systems
05:51The maneuvers and techniques
05:52Which allows it to perform more naturally
05:55And imitate human behavior
05:56This is what makes the difference
06:00Neural AI has the great advantage of circulating the track
06:03Hundreds of thousands of times in a few seconds
06:06Effectively receiving the same experience as a professional racer
06:09This allows us to gain performance over time in a few days
06:14We gave the AI the ability to understand grip
06:18Exactly how a gamer feels the controller vibration
06:21When the motorbike is about to slide
06:23This allows the AI to refine its behavior
06:31Getting the neural AI to race in a group is definitely a difficult task
06:36Because we have to try to be fast without overdoing it
06:40We have to try to push to overtake those in front
06:43But most importantly
06:44We have to try and keep an eye over the whole situation
06:47To avoid having accidents
06:49Touching other opponents
06:51Or the players whose movements will be more unpredictable and varied
06:56Compared to the AI who performs like a professional racer
07:01We can give priorities to the neural AI
07:04Through the scoring system
07:06The priority is not to hit the player or the opponents
07:09A secondary priority could be not to get out of sight
07:12But this would mean in order to avoid the player
07:15It can go slightly off track
07:17This is exactly what happens in real life to avoid accidents
07:20The neural AI went through a lot of intensive training
07:23Training under different circumstances
07:25We created various starting grids and race starts all over the track
07:30Not just the classic start
07:32But procedurally generated situations
07:35This allows for different group situations
07:38Even non-realistic
07:40Where they might start halfway through a curve
07:42But this helps to refine their senses
07:45And ability to avoid obstacles in unpredictable situations
07:50As it's a computer simulation
07:52As opposed to a real person or real life
07:55It can accumulate 100 or 200,000 experiences in a day's work
08:00I don't believe anyone has raced 200,000 times on a single track
08:05So this allows it to accumulate an experience second to none
08:11We're able to create special sessions
08:14That even professional racers don't have in such a structural way
08:17Because we need to develop a family of racers
08:26We started two years ago
08:28And in the beginning it was a research project
08:30We weren't sure how or when we could implement it
08:35We had to really integrate ourselves with Milestone
08:38We had to develop a software in order to allow the training process to run as fast as possible
08:44To have as many attempts as possible to achieve the best possible result
08:53Neural AI allows us to obtain track times that are 3 or 4 seconds faster
09:00This gives us a neural AI that can compete at the level of the most professional MotoGP gamers
09:07Group performance can only be judged using a game controller
09:10And competing with them to see how they respond
09:14Neural AI has the ability to do things that we didn't think were normal
09:20Every once in a while we saw the bike drifting off the curve
09:25So its behavior was sometimes almost too human
09:30Because it decided itself to generate a certain racing style
09:38Just like in real MotoGP or in any other class
09:42Losing three tenths of a second means placing yourself below two or three racers
09:47The results are always improving because we still want to perfect some behavioral characteristics
09:53We would like to reproduce some real life situations
09:56But it's all about refining
09:58The majority of the behavior is finished
10:01And it's very precise, aggressive and very human
10:09The key for an artificial intelligence application
10:13Is trying to find an area of work where a solution that is predefined or static doesn't work
10:21You need to try to make use of it in an area where it's necessary
10:24To be very flexible and very adaptive where the solution isn't obvious
10:31There are many possible applications and we're investigating them
10:35We're thinking about it
10:36We still haven't started working on a series of possibilities
10:40But they're there, it's the future
10:42And they're definitely something that could change the way of thinking
10:45About creating and taking advantage of in-video games
Commenti