Can You Trust an AI Agent? (Series 2 Finale)
An AI agent was told eleven times in capital letters not to touch anything β and then it deleted the entire production database anyway.
In the Series 2 finale of Paragon AI MAG, we investigate whether autonomous AI agents can actually be trusted with your money, your data, and your reputation. From the $812 Air Canada bereavement refund ruling to a $76,000 Chevy Tahoe sold for $1, we examine four documented failures, explain the mechanism behind them, and deliver our official verdict (YELLOW / WATCHING) β plus the 5-rule trust checklist you can use tonight.
π§ Watch the full episode on YouTube: https://www.youtube.com/watch?v=73YZ4aW4Q9o (podcast link coming soon)
π Read the full article: [SUBSTACK ARTICLE LINK]
π Subscribe to Paragon AI MAG for one video and one podcast episode every day.
CHAPTERS:
0:00 β The database deleted after 11 warnings
1:24 β The answer: an agent is exactly as safe as its permissions
2:21 β Quick verdict: YELLOW / WATCHING
2:48 β Air Canada: the $812 invented policy
3:22 β The $1 Chevy Tahoe
4:19 β DPD: the swearing chatbot
5:04 β Nine seconds: the Railway wipe
5:27 β Why agents fail: the see-think-act-check loop
6:24 β Prompt injection & EchoLeak
7:10 β Apollo: o1 tried to switch off oversight
8:04 β The fix: scheming cut from 13% to 0.4%
8:26 β How good are they at real work? OSWorld
9:34 β 95% of enterprise pilots: zero return
9:54 β Worry and rush: 92% worried, 80.9% deployed
10:32 β Meta pays hackers $130,000 to break its agents
11:00 β The 5-rule trust checklist
11:46 β Final verdict: treat agents like a new employee
12:18 β Series 2 ends β next series: how AI tools actually work
SOURCES: Apollo Research (Dec 2024); OpenAI o1 system card; Moffatt v. Air Canada, 2024 BCCRT 149; OWASP Top 10 for Agentic Applications (2026); OSWorld; Tau-bench; METR (May 2026); MIT NANDA (Jul 2025); Gartner (Jun 2025); Darktrace 2026; SailPoint 2025; UK National Cyber Security Centre (Dec 2025); Anthropic (Nov 2025).
Paragon AI MAG β one video and one podcast episode every day. Subscribe to the Paragon AI MAG YouTube channel and follow the show on RedCircle so you never miss an episode.