- 1 day ago
Category
📺
TVTranscript
00:129-1-1, what is your emergency?
00:15I hit a bicycle that was in the road.
00:17Can you tell us they're injured at all?
00:18They are injured, they need help.
00:24A car crashes in the middle of the night.
00:31A woman is run down.
00:33What's her name?
00:38It's the first death of its kind in the world.
00:41Are you the driver?
00:45A pedestrian killed by a driverless car.
00:48The car was an auto-drive.
00:51I couldn't see it, and all of a sudden, it was just there.
00:54Although the vehicle was driving itself using artificial intelligence, there was a human operator behind the wheel.
01:02Over hit it?
01:03Yeah.
01:03Okay.
01:04Do you want me to be aware of that?
01:05I mean, I can't give legal advice.
01:08For the first time ever, the decision had to be made.
01:12Who was to blame for this crash?
01:14Was it the human or the AI?
01:21Artificial intelligence.
01:23A machine beyond the mind of man.
01:25For decades, scientists have dreamed of creating incredible machines that could talk like us, learn like us, think like us.
01:37But what we didn't imagine is the impact they would have on us.
01:43In this series, I'm exploring what happens when AI collides with human lives, unearthing stories far stranger than we could
01:52ever have imagined.
02:19I'm Professor Hannah Frey.
02:21I'm a mathematician, and I've spent my career examining the ways technology can transform our future.
02:28Here, in Phoenix, a multi-billion dollar industry has the potential to save millions of lives.
02:35That is weird.
02:39We're on the cusp of a self-driving revolution, and driverless taxis are now a reality in several countries around
02:47the world.
02:48All right, regressed it.
02:50You hail them on an app.
02:53Ooh, you have my initials on top.
02:55This is my first time in one of these.
02:58Worked with Google for years.
03:00Never been in a driverless car.
03:02Imagine.
03:04There it is.
03:06Where's it going to stop?
03:10Where's it going to stop?
03:12There?
03:14Ooh.
03:16Ooh, that was quite well done.
03:18Yeah, look.
03:19Big Jeff on the top.
03:24No one in the car.
03:26It's to the right.
03:28Go.
03:33It's quite nerve-wracking.
03:35We'll do all the driving, so please don't touch the steering wheel or pedals during your ride.
03:42It's green, but it's slowed down.
03:43Oh, it's turning right, that's why.
03:47Oh, yeah, I sort of don't trust it.
03:49I don't trust it.
03:51Ooh, we're changing lanes.
03:58It's not timid, is it?
04:01The technology needed for a car to drive itself is nothing short of miraculous.
04:07The car uses a combination of sensors to understand what's going on around it.
04:14First, cameras.
04:16Great at identifying road signs, traffic lights, pedestrians and other cars.
04:22But they struggle in bad weather.
04:26So, many driverless cars also have radar, often built into the front bumper, which sends out radio waves that bounce
04:35off objects and measure what comes back.
04:38It works well over long distances, but not so well up close.
04:44That's why some driverless cars also use LIDAR, the spinning cylinders you sometimes see on the roof and side of
04:52the car.
04:52It's like radar, but with lasers, and is able to build up a detailed 3D picture of nearby objects.
05:02Precise, but easily confused by reflective surfaces like windows and shiny buildings.
05:18The AI takes these three imperfect systems, pieces together what's actually out there, tries to predict what will happen next,
05:27and decides how the car should steer, brake and accelerate.
05:32I mean, I was writing about these things 10 years ago, but they were a research project.
05:38They were understanding the environment, advancing the engineering, kind of tweaking the software.
05:43It was only very, very recently that they'd become commercially available, that just anybody could hail one.
05:53And soon they're even coming to the UK, starting with the busy streets of London.
06:01Who's good at sticking to the speed limit, I'll tell you that?
06:07I think there is actually quite a lot to look forward to from this future with driverless cars.
06:13Like, humans make terrible drivers, and these things, they're not falling asleep at the wheel, they're not drink-driving, they're
06:20not getting road rage.
06:22Like, fine, maybe they're not going to be perfect, but there is a lot of scope for roads to be
06:28much safer than they currently are.
06:30And I think that is a goal that is worth pursuing.
06:35But the story of how we got to this point is marked by tragedy and loss.
06:44There was a pedestrian walking a bicycle.
06:47Once the pedestrian got into the lane of traffic, the vehicle struck the pedestrian.
06:51It was a self-driving vehicle.
06:54It was in the autonomous mode at the time.
06:58In Arizona in 2018, for the first time, someone was struck and fatally injured by a driverless car.
07:06Rafael Vasquez was the human backup driver in the self-driving Uber vehicle that hit and killed Elaine Herzberg.
07:15After years of avoiding the limelight, the woman at the center of the story, Rafael Vasquez, had agreed to meet
07:22me.
07:24Hey!
07:25Hey, how are you?
07:26Good, how are you doing?
07:26Do you want to come in?
07:27Yeah, thank you.
07:28You haven't spoken on camera about this before, have you?
07:30No.
07:31How are you feeling about it?
07:32I don't know.
07:33I have a whole plethora of emotions going through me.
07:35But my biggest issue going through this was not being able to rebut anything or even defend myself.
07:43It's like getting a chance to speak for yourself.
07:44Yeah.
07:45Because I've been dealing with it now for, what, seven years?
07:48So this is you?
07:50Yes, and that's to get you in and out of the buildings.
07:53Raffaella got a job at Uber in 2017, not long after the company had decided to develop their own self
08:00-driving cars.
08:01I like a lot of technology stuff.
08:04Self-driving vehicles were starting to emerge.
08:08I knew that Phoenix was becoming a hotbed for it.
08:11So I would see autonomous vehicles roaming around.
08:15I'm interested in that.
08:16So I went and applied.
08:19I passed everything instantly.
08:20So I was excited.
08:23Her job was to ride in the autonomous cars as Uber began testing their new technology on Arizona's public roads.
08:31There was a lot of engineers, coders, but everybody was super nice.
08:35And you have this chance to see the new technology from the inside?
08:38Not just see it.
08:38I'm testing it before it even comes out.
08:40I loved my job.
08:42I absolutely loved my job.
08:44When you were in the cars in those early days, how good did you think they were at driving?
08:50They were better than what I thought they were going to be.
08:52And it also depends.
08:53Uber vehicles, for the longest time, had problems with overreacting with things on the side of the road.
08:59And some builds were great, but then they do another update to try to fix something else,
09:03and it makes something else go haywire, and then you just don't know.
09:08So much has happened to you since that, right?
09:11Yeah.
09:19During its first year of testing in Phoenix, Uber assigned two operators to every self-driving car.
09:25All right.
09:26And we're engaged.
09:28One rode in the passenger seat to track the car's performance.
09:32On the laptop, I can monitor a lot of the prediction software, so I can see, like, where the car
09:37is going.
09:37Any unexpected behavior had to be logged on a computer.
09:42The second operator sat behind the wheel and kept their eyes on the road.
09:47They were expected to take control if the AI in the car malfunctioned.
10:01But a year into testing, Uber changed the setup.
10:05Now one operator had to watch the road and monitor the car's actions.
10:11OK.
10:12Come on then.
10:14This decision to switch to a single human in the car came just a few months before Raffaella's crash.
10:22Even riding in the car, I still get nervous.
10:25Oh, really?
10:25Yeah.
10:26That's why I'm hesitating, because I'm just trying to not have a panic attack.
10:29I understand.
10:31OK.
10:34Oh, I'm sorry.
10:35No, don't apologize, please.
10:37There is no pressure.
10:40Like...
10:41No, I want to do this.
10:41Are you sure?
10:42Yeah.
10:43OK, here we go.
10:46I don't want my driving to unnerve you.
10:49No, I'm just worried you're going to get pulled over.
10:52Because you're not driving like a typical person.
10:55Driving like a granny?
10:57Well, you're driving like an autonomous vehicle, actually.
10:59Oh, really?
10:59Right at the speed limit.
11:02Each operator was allocated a route to which the car would drive again and again.
11:07It could be monotonous work.
11:10Like, the most boring one I hated was this one through this little neighborhood.
11:13You come out, you go here, then this is all, like, 20 miles an hour down here, down here,
11:20and then boom, back.
11:22And that's all it is.
11:23Round and round.
11:24Three hours.
11:25With the car driving itself.
11:26Mm-hm.
11:29Humans are not good at paying attention when things get boring.
11:33And with only one operator monitoring a repetitive route, there was a danger of getting distracted.
11:41But when it wasn't quiet on the streets, there could be a lot for one person to do.
11:48God, there's lots of students out, isn't there?
11:50Yes, and this is what we're testing.
11:53Yeah.
11:53And you can't predict what somebody's going to do.
11:55So I would always take it out of autonomous mode during these areas.
12:00Right.
12:00That's why two people was important, but then all of a sudden they changed it.
12:06But there are several screens that you're continually having to look at.
12:10Yes.
12:10We were supposed to push buttons and enter codes any time something happened.
12:14I'm sort of trying to put something into the iPad while you're supposed to be monitoring the road.
12:19Yeah.
12:26On the night of March 18th, 2018, Raffaella was on her usual test route.
12:35The car was in autonomous mode and had been running for 19 minutes without incident.
12:48At 9.58 p.m., it turned right onto Mill Avenue.
12:57At the same time, a pedestrian began walking across Mill Avenue, pushing a bicycle by her side.
13:15Raffaella was looking down as the vehicle approached the woman.
13:20The car should have detected her.
13:24But it didn't.
13:33By the time Raffaella looked up and slammed on the brakes, it was too late.
13:38The vehicle struck the woman at 39 miles per hour.
13:43The pedestrian killed was Elaine Herzberg.
13:47She was often seen on her bike in the area.
13:52She never gave up.
13:54She always helped people.
13:56She was funny, funny, funny, funny.
13:58If you were her friend, you felt true love from her.
14:01She didn't deserve what happened to her.
14:13So this is the park?
14:14That's the park.
14:14Right.
14:15I did that venue, that theatre.
14:17Everybody propsed there, including the homeless people, because homeless people would come up here to the park.
14:22That's where she was going.
14:27This was the first time Raffaella had returned to the site of the crash.
14:37Do you see that sign on that post, that light?
14:39Yeah.
14:40Yeah.
14:40She was over there.
14:41Oh, gosh.
14:44And she's just screaming the whole time.
14:54The screaming, it was just terrible to hear it.
15:01And then, but then what was worse is when it stopped.
15:05And then the ambulance showed up.
15:08And then they said she's passed away.
15:10And then I lost it.
15:14Somebody died.
15:24Following the collision, the police launched a criminal investigation into the artificial intelligence car.
15:42Building a system that can drive a car involves much more than turning a wheel and pressing pedals.
15:48You also need to teach it to do things that humans do instinctively, like spotting pedestrians and recognising road signs.
15:58We do that without even thinking.
16:00But training a machine to do it has proved incredibly difficult.
16:05So back in the early days of AI, the only real form of intelligence that we knew about was human
16:11intelligence.
16:12And so people look to the human brain for some inspiration about how to build an electronic brain.
16:21And the thing about the human brain is that it is made up by these billions and billions of neurons
16:27that are connected together.
16:29And as you think, you are essentially sending these little electrical impulses, these little bursts that can be big or
16:36small, through this network in your brain.
16:40In the 1950s, people started trying to construct a much simpler computer version,
16:45where a network of artificial neurons would pass signals between each other.
16:51This concept became what we now call a neural network.
16:56But rather than all of your neurons being kind of intermeshed together, they appear in layers, like a sort of
17:04hierarchy.
17:08Decades later, people realised you could use these neural networks to recognise images like a stop sign.
17:16Here's a simplified version of how it works.
17:21Each neuron in the hierarchy has its own job.
17:24At the bottom, they're just looking at a single pixel each.
17:27And as you go further up the network, things get more sophisticated.
17:32Maybe there'll be a neuron up there that is checking to see if there's some red.
17:38Maybe there's another one up there that's checking to see if there's an octagonal shape that stands out from the
17:43background behind it.
17:45And all of this information and all of these signals get sent up.
17:48So you eventually get right to the very top to the big boss, who makes a decision based on all
17:56of the information that has flowed through the network to finally decide whether it thinks it's a stop sign, yes
18:02or no.
18:05The extraordinary thing about these networks is that if you only show it one picture, you'll just be guessing whether
18:12it's a stop sign or not.
18:13But show it thousands and tell it when it's right or wrong, and the AI learns to recognise the sign
18:21itself, adjusting its network every time it makes a mistake.
18:27In the neural networks you'll find in a car, they'll be classifying not just stop signs, but pedestrians, vehicles, lampposts,
18:36road markings.
18:37They are gigantic and gigantically complex.
18:41But the principle is still the same.
18:44This is a machine built through trial and error.
18:55But if errors happen in the real world, on our roads, the consequences can be fatal.
19:22In April 2019, a Tesla Model S failed to stop at a stop sign and ploughed through a T-junction.
19:31Who else is involved?
19:33For what I understand, he was driving the car.
19:36How are you, sir?
19:37I was driving, I dropped my phone and looked down, and I ran the stop sign and hit the guy's
19:41car.
19:41Sir, which car are you driving?
19:43This car, right here.
19:44The driver of the Tesla, 42-year-old George McGee, had been on his phone when it fell out of
19:51his hand.
19:52He bent down to pick it up, leaving Tesla's autopilot to drive the car.
19:57Take it back, take it back.
19:59But something went wrong.
20:02Boss?
20:03The Tesla hit 26-year-old Dylan Angulo's truck at 62 miles an hour.
20:10Sir, did you stop at the stop sign?
20:11No, I didn't, sir. I don't think. I honestly don't know.
20:13I looked down. I didn't know how close I was to the intersection.
20:16And I was driving on a cruise, going through it, and then I looked down, and to get the phone,
20:20I dropped, and I reached down, and I didn't see it.
20:24What do you do?
20:25I manage a private equity fund out of Boca.
20:28I'll explain the tussle.
20:30Yes, sir.
20:32Remarkably, Dylan survived the accident.
20:39He got ejected.
20:43But he wasn't alone that night.
20:49Wait a minute, though. There's ladies' flip-flops.
20:52Yep, but there was one of them.
20:54There's a pair of ladies' flip-flops.
20:56Please tell me this.
20:58Get back.
20:58Shit, I'm sorry.
21:11This picture was actually the day of the crash.
21:20We were going fishing, and we stopped to get bait at the bait store.
21:27At the time of the accident, Dylan had been with his new girlfriend, 22-year-old Nybel Benavidez.
21:34Is he her?
21:37No.
21:38The impact from the Tesla killed Nybel instantly.
21:43She's so gorgeous.
21:46Nybel always had this peace and happiness to her.
21:49Just being around her would just...
21:52It would rub off on you, you know?
21:55I was going to meet her mom the next day, you know, we were going to catch the fish,
21:59and then I was going to cook lunch for her mom the next day.
22:04Was that the first time you were going to meet her mom?
22:06Yeah, I was going to meet her the first time.
22:08And unfortunately, you know, the first time that I have to meet her mom,
22:13we're under these circumstances.
22:19Yeah.
22:22Oh, my gosh.
22:24I'm so sorry this happened to you.
22:31Right away, they started doing an investigation into the accident.
22:35And I finally get my hands on the police body cam video.
22:42And in that police body cam video, the driver, yeah, he's like,
22:45I was driving, I was on the phone, I had the car on autopilot cruise.
22:51I started to do research, and I had no idea this existed, you know?
22:58This thing called autopilot, where the cars can drive themselves, you know?
23:03And right then and there, I was like, this guy was relying on this car to drive itself.
23:14This is why this happened to us.
23:28Autopilot is Tesla's advanced driver assist function, highlighted in their slick promotional videos
23:35that sell a vision of technological sophistication, safety and convenience,
23:40showing that it can steer, brake and change lanes on real roads in the real world.
23:47Tesla car next year will probably be 90% capable of autopilot.
23:53Like, so 90% of your miles could be on auto.
23:55Other car companies will follow.
23:57Elon Musk has even tweeted that his cars can completely drive themselves.
24:03But they can't.
24:05The driver's manual says a human still has to be in full control of the car.
24:10This thing is psychotic.
24:12Oh my God.
24:14In our city, yeah, it's turning.
24:15I'm not involved in this, watch the road.
24:16What do you think, autopilot?
24:17Am I on the wrong side of the road?
24:18Whoa, Jesus, that was scary.
24:19Wait, it's a double yellow line.
24:21Am I on the right side of the road?
24:22No, you are not.
24:23Get the fuck over, Gus.
24:25Are you serious?
24:25Yes.
24:26Look, don't fucking trust this thing.
24:28And there's serious safety concerns over the autopilot feature.
24:35This Tesla crashed into a highway divider in California, killing the driver.
24:40Another slammed into a parked fire truck in Utah.
24:43Both had the autopilot feature on.
24:51This is the bend right before the intersection where the accident happened.
24:56And at night from right here, you could already see the red light blinking from right here.
25:03Oh yeah, I see it.
25:13This is where her body ended up laying, you know, we pulled over to look at the stars.
25:22It's crazy how far her body flew, you know.
25:27That's how fast the car was going straight through this intersection.
25:32Six months after the fatal collision, the man behind the wheel of the Tesla, George Magee, was charged with careless
25:39driving.
25:40Which he didn't contest.
25:43Which he didn't contest.
25:44But Dylan wanted Tesla to also be held accountable.
25:49These car manufacturers, they need to do a better job with designing these cars.
25:55We want justice.
25:56My Bell's family and I, we want justice.
26:00These cars were allowed on the road before they were ready to be on the road.
26:04They were not safe.
26:05And they were advertised as these cars that can drive themselves.
26:14Tesla offered Dylan and I Bell's family an undisclosed sum to draw a line under the case.
26:22But they refused.
26:24And a court date was set.
26:27Tesla would now face a jury trial over its autopilot system.
26:46First at five, the video showing the moments before an Uber self-driving car crashes.
26:51Backup driver, Raffaella Vasquez looks down for approximately four seconds.
26:56Back in 2018, the fallout from Raffaella's crash had left the whole self-drive industry
27:02hanging in the balance.
27:03If there's no human driver, who bears the responsibility when a car makes a misstep?
27:09Is there enough safety built in before we deploy these vehicles?
27:12Is there any suggestion at the moment that Uber has done something wrong?
27:17We don't know.
27:21Under intense media and political scrutiny,
27:24Tempe's police investigation left no stone unturned.
27:29So, the police have sent over all of their digital evidence.
27:35And I have not yet watched this.
27:38Let's have a look.
27:40The actual car in the police compound being analysed.
27:45Oh, my gosh, look at that.
27:47And then there's also one that says seize phones.
27:53They're knocking on someone's door.
27:54It's the police department.
27:55I want to speak.
27:56Blurred everyone's faces.
28:01Hey.
28:02Hey, Raffaella.
28:04We're coming up to do follow-up regarding the accident.
28:07Is we have a seizure warrant for your cell phone.
28:10They've got a warrant for her phone.
28:13Which one, which number did you want?
28:15Hey, Raffaella.
28:16How many phones do you have?
28:17What?
28:18How many phones do you have?
28:19I have my work phone and then my personal phone.
28:21But my work phone is the one I had to work.
28:24So, the war is saying we need both phones.
28:25Well, we need all the phones.
28:26We need how many phones?
28:28All your phones.
28:30Why did I take you to have phones?
28:36Thank you, Raffaella.
28:44Hi.
28:45Good morning.
28:46Casey Marsland was the lead detective on the case.
28:49He agreed to show me the footage he'd obtained from Uber.
28:53So, this is the footage with all three of the camera views.
28:57When I first hit play, we can see the driver looking down multiple times.
29:02Hmm.
29:05And there didn't seem to be a logical explanation as to why she was looking down.
29:10So, we're coming up to the crash now, are we?
29:12Yes.
29:14And there.
29:17Can you go back a few frames in this?
29:19I just want to see what was happening immediately before.
29:21Sure.
29:24So, looking down, looking down, looking down, looking down, looking down, looking down, looking up.
29:28Oh, oh, wow.
29:29Yes.
29:30What did she say she was looking at?
29:32Initially, there was a statement that it had to do with the iPad in the center of the vehicle.
29:40And does that broadly stack up?
29:42Were Uber requiring people to look at screens while they were driving?
29:46So, yes.
29:47One of the purposes of her job was to monitor the vehicle.
29:51And if there was anything abnormal, she would need to interact with that screen.
29:55I mean, the conflicting instructions, right?
29:58Yes.
29:58Yes.
29:59They're supposed to be looking at screens, but simultaneously monitoring the road.
30:02It definitely presents a bit of a challenge.
30:04Yeah.
30:05However, when we, of course, looked at what the screen was actually showing at the time,
30:09it didn't show that there was any sort of alerts or any sort of interaction with the screen.
30:14We looked into the phone.
30:15We saw the apps that were installed on her personal phone.
30:20We noticed Hulu was an active app on her phone.
30:23And when we wrote a search warrant to the company, they responded with a printout of the activity on
30:28the account.
30:28And that's what gave us probably the most important information at the time.
30:33And what did it say?
30:35So this was the information that was provided from Hulu.
30:38And it shows that the show, the voice was being streamed to her personal phone.
30:44So what was your take on seeing all of this?
30:47Essentially, this goes to the beginning of her shift.
30:49She started her shift by driving the vehicle out of the garage.
30:53And it was at that time before she even got onto the roadway that she had set up and started
30:57streaming the Hulu.
30:59And so I think that the conscious decision to provide yourself with a likely distraction
31:05before even getting onto the roadway is an absolutely reckless decision to be made.
31:22So I found a couple of clips around the time and lots of them are not on your side.
31:30Some of them were terrible, especially at first.
31:33We take a look at this.
31:33This is Hulu records.
31:35It turns out she was streaming that scene competition,
31:38The Voice, at the time when she should have been watching the road.
31:42Rafaela Vasquez was riding in autonomous mode at the time,
31:46but she was distracted and looking down for more than 30%
31:49of the nearly 22 minutes before the crash.
31:53Right there.
31:54They're right.
31:55I wasn't distracted.
31:57I was doing my job.
31:58I had other things to do other than operate the vehicle because we went to one person.
32:01So I had duties assigned.
32:03Like in the video, you see me look away.
32:05I am looking away.
32:06But if you time them, and even the police did, I never looked away for more than five seconds.
32:11Why? Because that's what we were trained.
32:12We were trained to look.
32:13Boom.
32:13I've had five seconds, then look back.
32:15Boom.
32:16Over here, five seconds, look back.
32:18Boom.
32:18Our cell phones, we have to Bluetooth them into the vehicle.
32:23It wasn't watching TV.
32:24I was listening to it, but the police said I was watching.
32:27That means that's a definitive statement.
32:29You just told the public that you have evidence that I was watching something.
32:33That's bullshit because you don't.
32:35You have evidence that I was Bluetooth streaming something.
32:38But hello, streaming Bluetooth.
32:40You can listen to stuff.
32:41People do it all the time with music.
32:45Everyone automatically assumed, because the police said it, that I was watching Hulu.
32:52And because of that, I was negligent, therefore I was unable to prevent the accident, therefore the charge.
32:58The investigation lasted for two and a half years before Raffaella was finally charged.
33:05Where were you when the indictment came through?
33:07I was at home, because my attorneys informed me.
33:10They just told me that I was going to be indicted for negligent homicide.
33:12I said, what?
33:13And they said, negligent homicide.
33:15Negligent homicide?
33:16I said, so murder? That's just to me how I interpret that.
33:23So now I'm like, I didn't know what to say or do.
33:28Now I'm in shock.
33:30If found guilty, Raffaella faced up to eight years in prison.
33:42With conflicting accounts from both sides, I decided to track down the official accident report.
33:52The report concluded that the crash was probably caused by Raffaella's failure to monitor the driving environment.
34:00But it was also critical of Uber, uncovering troubling flaws in the design of its AI programming.
34:10This table is particularly interesting because this is like seeing inside the brain of what the driverless car was thinking
34:17at the time.
34:19The car actually notices that there's something there 5.6 seconds before the collision, which is plenty of time to
34:28start braking or turning.
34:30And initially, it's radar that detects there's something in the distance.
34:34It predicts it's a vehicle, though.
34:36And then 0.4 seconds later, the LIDAR kicks in and picks up that there's something there.
34:44The LIDAR doesn't know what it is, though. It just classifies it as other, something unknown.
34:49Now, the LIDAR keeps changing its mind.
34:53So, a full second later, it changes its mind to site as a vehicle.
34:57Then 0.3 of a second later, it changes its mind again.
35:01And then it keeps switching.
35:04Vehicle, other, vehicle, other.
35:06The computer within this driverless car is not connecting the dots.
35:12It's not seeing this as one object whose classification keeps changing,
35:17whose path is slowly moving over time.
35:21Instead, it is seeing this as brand new objects every single time.
35:30In fact, it's only 1.2 seconds before impact
35:34that it finally decides to settle that it's a bicycle.
35:41Obviously way too late to actually do anything about it.
35:46Terrifying.
35:47You design a system that fails to track the path of an object until it's decided what it is.
35:53If you've got something that you're going to collide with,
35:56I don't care what it is. I care about that it's going to collide.
36:05That's insane.
36:08Although the system sensed the pedestrian nearly six seconds before the impact,
36:11the system never classified her as a pedestrian or predicted correctly her goal.
36:18The system design did not include a consideration for jaywalking pedestrians.
36:26It wasn't designed to recognise people unless they were on a crosswalk.
36:36I think this is pretty damning actually.
36:41What was this thing doing on the roads?
36:51The report made me want to know more about what was happening inside Uber
36:56around the time of Raffaella's crash in 2018.
37:01After some digging, I found Robbie Miller.
37:04Robbie.
37:05Hi.
37:05An operations manager who'd been at Uber at the same time as Raffaella.
37:10He had quit on safety grounds just a few days before the fatal collision.
37:16I'd been working in the self-driving space for four or five years at this point.
37:21I was running the self-driving truck fleet for Uber. Eventually there was a move to
37:27combine the operations of the testing for self-driving cars and self-driving trucks.
37:36How did you feel about that?
37:37I was not at all comfortable with it. The self-driving cars were having significant issues.
37:43There's a lot of really scary incidences that are occurring near misses, near collisions.
37:50We would have, you know, an incident where the car was driving on the sidewalk
37:56and in broad daylight. And you realize, like, they are headed on a path where someone is going to get
38:05seriously injured or worse. And so I gave notice.
38:11There's this overarching fear, I would say, at Uber that Waymo is about to release their self-driving
38:20cars. And it's, I think it's very scary for Uber's leadership to not have a response.
38:27So it's turned into a race?
38:28It is absolutely a race.
38:31Waymo, owned by Google's parent company, Alphabet, was Uber's arch rival. Just five months after
38:39Waymo launched its first public trial, Uber moved from two operators in the car to one.
38:48You need to show to your investors, hey, we're making this progress. An easy way to do that is just
38:54take
38:54someone out of the car. And this is something I mentioned. You need that second person in the
38:59vehicle. You're not ready to take that person out of the vehicle. Just a few days after I left,
39:05the crash occurred. And...
39:08When did you first hear about it?
39:11I was driving and I received a phone call from one of my former co-workers at Uber.
39:22And I sat in the parking lot and cried for 15 minutes.
39:31Wow.
39:33I am very passionate about the technology. I believe in the technology. I want the technology
39:40to succeed. But there's a way to do it.
39:53A few years after Raffaella's fatal crash, Uber sold off its self-drive division,
39:59leaving the path clear for Waymo.
40:05Here you go. Waymo's just crashed. AI technology not working.
40:13Why is this happening to me on a Monday? I'm in a Waymo car.
40:16Next is your rider support. This call may be recorded for quality assurance.
40:20This car is just going in circles.
40:22Hi there, Mike. This is a scab. I'm calling from Waymo support.
40:25Yeah, I got a flight to catch. Why is this thing going in a circle? I'm getting dizzy.
40:29I understand. I'm really, really...
40:30Waymo are now the dominant force in driverless taxis in the US,
40:34with 2,500 of them on the road. Experts have praised their safety record.
40:48But some see these cars as symbols of the powerful tech elite,
40:52and their disconnection from the lives of ordinary people.
40:56Hey. Hello.
40:58And there's one group who've decided to take action.
41:02Hello there. Nice to meet you. Nice to meet you.
41:04How are you all doing? Doing well, doing well.
41:06They call themselves the Safe Street Rebels, and carry out their operations incognito.
41:12What is it about the autonomous driving that you dislike, though?
41:18They are heavily promoting themselves as the future of public transit.
41:23And we just fundamentally don't think they're the future of public transit.
41:26They're just a taxi where you don't talk to someone.
41:29And when that car is not parked, it's still driving around waiting.
41:3350% of the miles they drive, there's nobody in the vehicle.
41:36In addition, they cannot be ticketed for any kind of moving violation in the city.
41:41We have videos of them driving 40 miles an hour on the wrong side of the road.
41:44And nobody can do anything about it. They're completely immune.
41:47What, they're above the kind of rules that would stand for an Uber driver or a Lyft driver?
41:51Yes. And if they can't do something about it, we can.
41:55To express their frustration, the group go around San Francisco disabling Waymo's.
42:04Let me understand the strategy, then. So, how easy are they to override?
42:08Pretty easy. It won't be pretty easy.
42:11We're not allowed to show you how they disable these cars, although it's not exactly high-tech.
42:17So, you're not putting them out of action permanently?
42:20No, no. And we're not causing any damage to them either.
42:21It's like, it's painter's tape, so it doesn't leave any residue when you remove it.
42:24All right, there's one. There's one coming.
42:26All right.
42:27There's one. There's one. Yeah, yeah.
42:28Check this one.
42:30Someone go, friend.
42:38Did you hear that? There was people on the sidewalk who were, like, cheering them on.
42:45The disabled car was now blocking the road.
42:53So, the one behind hasn't been taped, but is stuck.
42:57There's another one coming.
42:59Oh, my gosh.
43:02Three of them.
43:04And they're just stuck.
43:06And then now there's another car behind flashing,
43:09but it has a human driver, so they can go around the problem.
43:15Is it actually illegal, what you're doing?
43:17We don't think it's illegal. We can't find any law that'll break.
43:20It's not vandalism, so it's hard to point to any law we're breaking.
43:25Are there not other ways that you could do this?
43:27I mean, can you, I don't know, is there like a not more sort of democratic way?
43:31It would be nice if we could vote, if we had the opportunity to vote on them,
43:34but we have not been presented with that opportunity yet.
43:37Is this, does it feel a bit like this is something that has happened to you rather than with you?
43:41I mean, absolutely.
43:42Hey, yo, there's one coming out of the side.
43:44Other side, other side.
43:45Other side.
43:45It's coming, yeah.
43:49Yeah, you want to get them back?
43:56They are surprisingly easy to bully, aren't they?
44:01The other thing that I think was really surprising was just how many of them were going past, right?
44:08And maybe it's the time of day, but I mean, hardly any of them have people inside.
44:17Yeah, it's empty.
44:23They did say something that I hadn't considered before, which is about how they
44:27are continually circling when they don't have people inside them.
44:31And that I hadn't considered because, of course, there's a congestion aspect,
44:36but there's an environmental aspect too, right?
44:38I mean, they've got to be powered.
44:41Hmm.
44:42Interesting.
44:44Interesting.
45:08It happened fast.
45:10The hearing for Rafael Vasquez was scheduled as a settlement conference,
45:14but it quickly led to a plea deal.
45:17In July 2023, after five years of legal wrangling,
45:22Rafael accepted a plea deal to avoid going to jail.
45:25Do you even indicate that you wish to plead guilty to the crime of endangerment?
45:29The event is non-dangerous, non-repentant.
45:31Is that the crime that you wish to plead guilty to?
45:34Yes.
45:35She pled guilty to a reduced charge, endangerment,
45:39with the guarantee it would be lowered to the least serious category of offence
45:43following three years of probation.
45:50I wanted to meet Rafaela's lawyers, Al Morrison and Marcy Cratter.
45:56So good to see you.
45:59Why did you take this case on?
46:01Our job is to fight for the little guy, no offence, but she was the little guy.
46:07And the more we got into this case, the more we realised just how one-sided it was.
46:12Where do you think this came from then?
46:14I mean, is this the police or is this from Uber?
46:18Uber.
46:19Really?
46:20They did whatever they could to make it not their fault.
46:24And it's David and Goliath.
46:27We have this single person up against a multi-million dollar
46:32company with unlimited resources.
46:34The problem was that there were so many aspects of the way these cars were programmed.
46:39It didn't account for real-world situations.
46:41The most obvious one is the thing wasn't programmed to deal with jaywalkers.
46:46To save yourself in a campus, college campus, to not program vehicles to deal with jaywalking,
46:53I think is the height of negligence.
46:54I was scared to go to trial. If I lose, it's prison time, no fans or buts. And the media
47:03already destroyed me out there.
47:05That's always in the back of our minds as lawyers and certainly our clients' minds,
47:09that it's the exposure. What's going to happen to me if I go to trial and lose?
47:14But Marcy and I felt like we had a great case, but we also understood the risk was all hers.
47:19Me not going to trial has no reflection on them. These two saved my life.
47:24They absolutely saved my life.
47:32No criminal charges were ever brought against Uber.
47:45Raffaena got dealt a really rough hand and this didn't play out fairly.
47:53Maybe she was watching her phone. Maybe she wasn't. Like, I, honestly I don't know, but I also,
48:01I don't think that that's the point of this.
48:03The scandal here is that you have got this massive, corporate, multi-billion dollar project,
48:12which is not prioritising the safety of the people who are in the cars or, like,
48:19the members of the public who haven't even agreed to participate in the experiment.
48:23It is not enough to say that all of the responsibility lies with the person who was in the car.
48:32That's not enough. That's not enough for me.
48:47Back in Miami, Dylan and Nybel's family were taking Tesla to court over the fatal crash that had killed Nybel.
48:55Nice to meet you, Anna.
48:56I'm Anna.
48:56Adam, great to see you.
48:57Great to meet you.
48:59What's up, buddy?
49:03Dylan's lawyer, Adam Bumel, brought me up to speed on the case.
49:07This is a sort of situation where there's no precedent at all. Does that make it quite difficult
49:12to build a legal case when you're, I don't know, forging a new path?
49:17Extremely. I mean, we had to do it from the start. This is creating the law.
49:22Obviously, from day one, Tesla's position was this is 100% driver's fault.
49:28The thing is that it's difficult to untangle the responsibility in this case because,
49:32because you're not saying that the driver had no responsibility at all, right?
49:36Of course not. We 100% acknowledge that the driver had responsibility.
49:41And our position from day one has been this is a case of shared responsibility.
49:46Yes, the driver was at fault. He was distracted. He was disengaged from the driving task.
49:52But then you have to ask yourself, why was he disengaged from the driving task? And you
49:56realize that Tesla fostered this belief in him, this trust of the system that was unwarranted.
50:04He thought that the car would stop or swerve or do something before plowing into a parked vehicle.
50:12Do you have the data from inside the car? Do you know what the car was seeing?
50:16So we were able to finally get the data from inside the car, showing what the autopilot
50:22computer was detecting and processing. The computer understood that the car was speeding towards the
50:30end of the road. It appreciated the stop sign. It appreciated the blinking red light.
50:37It appreciated Dylan's parked SUV.
50:43So it identified a lot of things and it did nothing.
50:55The car knew what was going on in front of it and didn't do nothing to warn the driver to
51:02stop the
51:03car. So this isn't a case of misinformation inside the car system. The car has all of the information
51:10it needed in order to avoid the collision. But it was programmed not to behave in the way that the
51:18driver thought it was programmed to behave. Did the car behave as people expected and believed it should,
51:24based off of the Tesla's marketing and Elon Musk's statements and kind of hyping up the capabilities of
51:32this car. When you take that backdrop and you put it against this case, the car did nothing that anybody
51:40thought it would or should. So this is almost like become a case of like misadvertising or like misrepresentation then?
51:49They make the consumers and drivers believe that the car is more capable, creating false expectations
51:57in their drivers. In the Uber crash, the blame had been laid firmly on Raffaella's shoulders.
52:07With Tesla, would this now be the first time the car itself would be considered at fault?
52:33Tonight, a Florida jury forcing Tesla to pay $243 million to victims in a deadly 2019 crash.
52:41The jury finding flaws in Tesla's self-driving software were partly to blame.
52:46We can be proud that we stood up and that we did everything in our power
52:51to help shine light on what's going on.
52:56Against all odds, Dylan emerged victorious in court, forcing Tesla to take some responsibility
53:04for the first time ever for a crash involving its autopilot system.
53:11It was a shocking landmark verdict that could well reshape the future of self-driving cars.
53:20When they gave the verdict, how was it?
53:24It was very emotional, you know, I mean, like me and my dad, we started hugging each other and,
53:29you know, praying that, that we would get justice. You know, from the beginning,
53:35we knew this was a joint liability case and the jury decided to hold Tesla accountable.
53:42And I'm grateful for, for that. And I'm grateful that people heard all the evidence and, and saw
53:49that Tesla made a mistake.
53:51I really get the impression that this has never been about money for you.
53:55It was more important for us to, to shine light and to show the world that this technology is not
54:03safe.
54:06People will come to me and, you know, tell me congratulations, but I don't feel like
54:12it's something to celebrate. Like, uh, you're not walking away from this a winner.
54:17I would say it's more of just, uh, justice was served.
54:22Yeah.
54:25Tesla planned to appeal the ruling.
54:43Meanwhile, in the UK, British firm Wave will launch their self-driving cars in London later this year.
54:54This feels like so much more high stakes than it did when I was in Phoenix.
55:00I mean, this is slightly wild for me, right? Like, I've lived in London for 20 years
55:04and we're driving in a driverless car down Camden High Street.
55:08This is honestly something I thought was much further in the future.
55:12It never gets old.
55:14Wave CEO Alex Kendall points out that the AI in driverless cars has come on leaps and bounds
55:20in the past few years.
55:22And this is interesting. This person's body language was sort of turning backwards and
55:26forwards away from the crossing.
55:27So that was noticeable, actually. The car sort of changed its mind a couple of times.
55:31Yeah, we saw that person turn their body back and forth.
55:33The AI has got quite good at learning that kind of intent.
55:36So it's like super, super fancy cruise control.
55:39It's a completely different experience from cruise control.
55:43Have I just undercut your product balance?
55:45You're comparing a floppy disk with a quantum computer.
55:51Think of AI as the next evolution of tooling.
55:55When you think about the wheel, the calculator, the computer, different tools that as humanity
56:00we've invented that push forward society.
56:03I think intelligent machines are going to be that next evolution of this.
56:13Whether you like it or not, driverless cars are coming.
56:16I mean, they're here already.
56:19And I still think there's lots of good to come from that.
56:21But to get to this point, we had to go through that difficult phase.
56:26We had to go through the learning, which, I mean, by definition, learning involves making mistakes.
56:34It's just that these mistakes, you know, these deaths, these lives that were ruined,
56:39I don't think that they were inevitable.
56:41I don't think that they were an acceptable price to pay for a new technology.
56:48I think different choices could have been made.
56:51I think there were different ways to balance the rollout of the new product with the safety of the public.
56:58And I just hope now that these are lessons from the past, you know.
57:02I hope that the steepest part of this learning curve is now behind us.
57:54The chief executive of one of the United States' largest health insurance companies
57:59has been shot and killed in New York.
58:01The alleged killer has been identified as Luigi Mangione.
58:05The suspect had something to say about the insurance industry.
58:09The insurance companies are relying on algorithms to make decisions to deny patient care.
58:15Stop denials with AI!
58:17How many people have denied?
58:19The anger and the upset is real.
58:22But there is a human right!
58:25To discover more about AI and how it can shape our future,
58:29go to connect.open.ac.uk
58:33or scan the QR code on the screen now.
58:39What których whooshy upfront mają wants to get it?
58:58Go to the서� pratkom или you can use at any kind of tricks and check.
58:59So then read that.
58:59Go to the authority orключ in the Position Всuoles Find depths.
Comments