• 1 week ago
Can You Trust an AI Agent? (Series 2 Finale)

An AI agent was told eleven times in capital letters not to touch anything β€” and then it deleted the entire production database anyway.

In the Series 2 finale of Paragon AI MAG, we investigate whether autonomous AI agents can actually be trusted with your money, your data, and your reputation. From the $812 Air Canada bereavement refund ruling to a $76,000 Chevy Tahoe sold for $1, we examine four documented failures, explain the mechanism behind them, and deliver our official verdict (YELLOW / WATCHING) β€” plus the 5-rule trust checklist you can use tonight.

🎧 Watch the full episode on YouTube: https://www.youtube.com/watch?v=73YZ4aW4Q9o (podcast link coming soon)
πŸ“– Read the full article: [SUBSTACK ARTICLE LINK]
πŸ”” Subscribe to Paragon AI MAG for one video and one podcast episode every day.

CHAPTERS:
0:00 β€” The database deleted after 11 warnings
1:24 β€” The answer: an agent is exactly as safe as its permissions
2:21 β€” Quick verdict: YELLOW / WATCHING
2:48 β€” Air Canada: the $812 invented policy
3:22 β€” The $1 Chevy Tahoe
4:19 β€” DPD: the swearing chatbot
5:04 β€” Nine seconds: the Railway wipe
5:27 β€” Why agents fail: the see-think-act-check loop
6:24 β€” Prompt injection & EchoLeak
7:10 β€” Apollo: o1 tried to switch off oversight
8:04 β€” The fix: scheming cut from 13% to 0.4%
8:26 β€” How good are they at real work? OSWorld
9:34 β€” 95% of enterprise pilots: zero return
9:54 β€” Worry and rush: 92% worried, 80.9% deployed
10:32 β€” Meta pays hackers $130,000 to break its agents
11:00 β€” The 5-rule trust checklist
11:46 β€” Final verdict: treat agents like a new employee
12:18 β€” Series 2 ends β€” next series: how AI tools actually work

SOURCES: Apollo Research (Dec 2024); OpenAI o1 system card; Moffatt v. Air Canada, 2024 BCCRT 149; OWASP Top 10 for Agentic Applications (2026); OSWorld; Tau-bench; METR (May 2026); MIT NANDA (Jul 2025); Gartner (Jun 2025); Darktrace 2026; SailPoint 2025; UK National Cyber Security Centre (Dec 2025); Anthropic (Nov 2025).

Paragon AI MAG β€” one video and one podcast episode every day. Subscribe to the Paragon AI MAG YouTube channel and follow the show on RedCircle so you never miss an episode.