00:00Come forth and join me on a quest.
00:03We are entering the long-term strategy game, now being used to test AI.
00:09Wait, I thought this was the fantasy land of Dungeons & Dragons.
00:14Yes, you heard that right.
00:16Scientists recently turned three AI models into D&D adventurers to discover how well each would fare.
00:24Okay, so how does that work?
00:26D&D involves campaigns with rules and roleplay while holding lots of things in your head and planning out your
00:32next moves.
00:35Each campaign has a dungeon master and players, and each player has a power level or the probability of what
00:41they can and can't do.
00:42Oh, and there's monsters.
00:45So the scientists ran experiments where each LLM embarked on an adventure where the players were a mix of humans
00:52and other AI models.
00:53The AI could be any character, including the dungeon master and players.
00:58And this was repeated 27 times with different monster fighting scenarios, with the researchers testing things like how well the
01:06AI stayed in character and how well they fared tactically.
01:11Turns out the best actor was also the most efficient in combat, and it was Claude 3.5 Haiku, which
01:18kind of makes sense because research indicates that it performs complex multi-step planning in its head, not really showing
01:25the intermediate steps in its thinking.
01:27On the other hand, Deep Seek V3 struggled the most.
01:31You probably don't want to be in a team with that one anytime soon, and GPT-4 was somewhere in
01:37the middle.
01:37These findings are pretty neat because they have implications for the real world, like designing safer and more efficient multi
01:43-agent AI.
01:44But for now, I still like playing D&D with my human friends.
Comments