Moderators: richierich, ua900, PanAm_DC10, hOMSaR

 
User avatar
flyingturtle
Topic Author
Posts: 5881
Joined: Mon Oct 31, 2011 1:39 pm

Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 8:20 am

In March 2018, a self-driving Uber car fatally hit a woman walking across a road, with her bicycle.

The NTSB report states that the car's systems detected the woman more than five seconds before the collision. But the car's systems weren't designed to reliably detect people who cross a road in the absence of a crosswalk.

Well, couldn't the system have noticed that an object was *crossing* the road, and that it should be avoided?

"The computer-vision systems in self-driving cars are trained to identify things, such as other vehicles, trees, sign posts, bicycles, and so on, and make decisions on what to do next using that information."

But each time the system re-classified the object as "vehicle", "bicycle" and "other", the system forgot the movement history of that object, and thought the object was static and thus not crossing the road.

What a fustercluck of system design. :eyepopping: :eyepopping: :eyepopping:

https://www.theregister.co.uk/2019/11/0 ... car_death/
Reading accident reports is what calms me down
 
tommy1808
Posts: 13480
Joined: Thu Nov 21, 2013 3:24 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 8:49 am

flyingturtle wrote:
But the car's systems weren't designed to reliably detect people who cross a road in the absence of a crosswalk.


Clusterfuck in deed. Jaywalking isn't illegal everywhere after all.

Best regards
Thomas
Well, there is prophecy in the bible after all: 2 Timothy 3:1-6
 
User avatar
casinterest
Posts: 11784
Joined: Sat Feb 12, 2005 5:30 am

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 11:06 am

Put simply, it is one of the many complex system design elements that has to be found as we move towards automatic driving cars.
The car didn't have an algorithm in place for tracking object movements in memory.

Subjectively, it is painful that someone died because of code, but as is the case in other areas, onward we move with a lesson in design.
Where ever you go, there you are.
 
vikkyvik
Posts: 12592
Joined: Thu Jul 31, 2003 1:58 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:22 pm

I mean, I know design certainly can't account for everything, but this just seems like an obvious DFMEA/PFMEA item.

It's designed to detect and avoid pedestrians crossing at crosswalks.

Possible failure: what if the pedestrian is not at a crosswalk?

Ah well, hindsight is 20/20.
I'm watching Jeopardy. The category is worst Madonna songs. "This one from 1987 is terrible".
 
Kiwirob
Posts: 13031
Joined: Mon Jun 13, 2005 2:16 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:28 pm

The next question is how much will this cost UBER, her families lawyer must be rubbing his hands together because surely this has massive payout potential.
 
seat64k
Posts: 611
Joined: Fri Mar 14, 2014 11:48 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:33 pm

vikkyvik wrote:
Possible failure: what if the pedestrian is not at a crosswalk?


I remember either Elon Musk or Lex Fridman explaining why this is difficult. I don't remember what they said though - might have been something to do with with separating signal from noise - but I do remember thinking, "huh, this is much more complicated than it seems."

Either ways, maybe this is an issue of context: What if the car depends on the pedestrian crossing (a fixed-state stationary object) to identify the pedestrian (a shape-shifting, moving object) as an object to avoid?
 
User avatar
Tugger
Posts: 10712
Joined: Tue Apr 18, 2006 8:38 am

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:45 pm

Kiwirob wrote:
The next question is how much will this cost UBER, her families lawyer must be rubbing his hands together because surely this has massive payout potential.
Unlikely worth all that much. As I recall she was a homeless person afflicted by drugs and mental illness with limited family or community connections.

Tugg
(And yes, I am being horribly disconnected acting in my comment as that is the reality and want that to be seen.)
I don’t know that I am unafraid to be myself, but it is hard to be somebody else. - W. Shatner
There are many kinds of sentences that we think state facts about the world but that are really just expressions of our attitudes. - F. Ramsey
 
Okie
Posts: 4156
Joined: Wed Jul 16, 2003 11:30 am

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:45 pm

MCAS on wheels. Apparently.

Still reliant on operator.

We will have to wait for AI whose first applications will be to replace the high end cost of all the programmers and high paying tech jobs.

Okie
 
frmrCapCadet
Posts: 4316
Joined: Thu May 29, 2008 8:24 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 3:48 pm

I believe my late model Toyota has both pedestrian and bicycle recognition. Crosswalks are not related. Toyota also says, and emphatically, do not test for these things. So I brake early, but suspect the car may do it.
Buffet: the airline business...has eaten up capital...like..no other (business)
 
User avatar
Aesma
Posts: 13230
Joined: Sat Nov 14, 2009 6:14 am

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 5:14 pm

Well the Volvo used by Uber also has such protection and would probably have at least attempted to brake to avoid the collision, however this was disabled by Uber.

Putting in a safety driver has led to shortcuts being taken it seems.

The thing is, if someone had been driving and had the same accident, it would have simply been a tragic accident, I doubt the driver would have been blamed. So we're asking more from the computer than from human drivers.
New Technology is the name we give to stuff that doesn't work yet. Douglas Adams
 
winginit
Posts: 2931
Joined: Sat Feb 23, 2013 9:23 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 5:23 pm

Aesma wrote:
The thing is, if someone had been driving and had the same accident, it would have simply been a tragic accident, I doubt the driver would have been blamed. So we're asking more from the computer than from human drivers.


This.

A Self-Driving car killing a pedestrian is tragic, but how many human drivers kill how many pedestrians every hour? A lot.

We should strive for perfection with self-driving automobiles, but be perfectly content with an outcome that is incrementally superior to the status quo of human drivers.I would argue in many ways we're already there with the fully autonomous driving function in Teslas, etc.
 
Kiwirob
Posts: 13031
Joined: Mon Jun 13, 2005 2:16 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 8:45 pm

Tugger wrote:
Kiwirob wrote:
The next question is how much will this cost UBER, her families lawyer must be rubbing his hands together because surely this has massive payout potential.
Unlikely worth all that much. As I recall she was a homeless person afflicted by drugs and mental illness with limited family or community connections.

Tugg
(And yes, I am being horribly disconnected acting in my comment as that is the reality and want that to be seen.)


Ah I see that makes it ok, a person of no fixed abode isn’t worth worrying about.
 
vikkyvik
Posts: 12592
Joined: Thu Jul 31, 2003 1:58 pm

Re: Self-driving Uber car killing a woman: The cause

Wed Nov 06, 2019 10:18 pm

seat64k wrote:
I remember either Elon Musk or Lex Fridman explaining why this is difficult. I don't remember what they said though - might have been something to do with with separating signal from noise - but I do remember thinking, "huh, this is much more complicated than it seems."


I have no doubt it's difficult. Nevertheless, seems like something that should have been addressed - even if the answer is that the risk is low enough that no mitigation is necessary.

seat64k wrote:
Either ways, maybe this is an issue of context: What if the car depends on the pedestrian crossing (a fixed-state stationary object) to identify the pedestrian (a shape-shifting, moving object) as an object to avoid?


That's my point. Seems like something that should have been addressed during a DFMEA. One can't assume that pedestrians will always be at crosswalks.

winginit wrote:
Aesma wrote:The thing is, if someone had been driving and had the same accident, it would have simply been a tragic accident, I doubt the driver would have been blamed. So we're asking more from the computer than from human drivers.This. A Self-Driving car killing a pedestrian is tragic, but how many human drivers kill how many pedestrians every hour? A lot. We should strive for perfection with self-driving automobiles, but be perfectly content with an outcome that is incrementally superior to the status quo of human drivers.


Agree with both of you. Unfortunately, any incident with a self-driving car will be immediately called to the public's attention.
I'm watching Jeopardy. The category is worst Madonna songs. "This one from 1987 is terrible".
 
User avatar
Aesma
Posts: 13230
Joined: Sat Nov 14, 2009 6:14 am

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 1:27 am

Kiwirob wrote:
Tugger wrote:
Kiwirob wrote:
The next question is how much will this cost UBER, her families lawyer must be rubbing his hands together because surely this has massive payout potential.
Unlikely worth all that much. As I recall she was a homeless person afflicted by drugs and mental illness with limited family or community connections.

Tugg
(And yes, I am being horribly disconnected acting in my comment as that is the reality and want that to be seen.)


Ah I see that makes it ok, a person of no fixed abode isn’t worth worrying about.


Well presumably her family if she had any didn't care about her, why should they get money ?
New Technology is the name we give to stuff that doesn't work yet. Douglas Adams
 
User avatar
zkojq
Posts: 4353
Joined: Fri Sep 02, 2011 12:42 am

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 5:01 am

Aesma wrote:
Well the Volvo used by Uber also has such protection and would probably have at least attempted to brake to avoid the collision, however this was disabled by Uber.

That's just ridiculous. For self driving cars I'd have thought it would be pretty basic to use the Collision Avoidance/Automatic Braking system. Why on earth wouldn't you?
First to fly the 787-9
 
tommy1808
Posts: 13480
Joined: Thu Nov 21, 2013 3:24 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 5:38 am

Aesma wrote:
Well the Volvo used by Uber also has such protection and would probably have at least attempted to brake to avoid the collision, however this was disabled by Uber..


And Volvo has one of the best systems:
https://www.iihs.org/news/detail/perfor ... dsize-cars

IIHS wrote:
The performance of the systems varies widely, the latest IIHS ratings show.

The Audi A4, BMW 3 series, Mercedes-Benz C-Class, Nissan Maxima, Subaru Outback and Volvo S60 have systems that earn superior ratings. They avoided collisions or slowed substantially in track tests.

"Car makers often roll out these kinds of advanced systems in more expensive luxury lines or as expensive options, so the superior performance of the standard systems on the more mainstream Maxima and Outback is noteworthy," Harkey says.

At the other end of the spectrum, the Ford Fusion, Hyundai Sonata and Kia Optima earn no credit because they failed to slow significantly in multiple scenarios.

The C-Class, 3 series and Chevrolet Malibu each offer two different pedestrian crash prevention systems. On the C-Class, the optional system earns a superior rating, while the standard one earns a basic rating. On the 3 series, the standard system surprisingly earns a higher rating of superior, while the more costly optional system earns an advanced rating. Both of the Malibu's optional systems — a camera only and a camera plus radar — earn basic ratings.


Best regards
Thomas
Well, there is prophecy in the bible after all: 2 Timothy 3:1-6
 
olle
Posts: 2348
Joined: Tue Feb 06, 2007 3:38 am

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 1:25 pm

For a reason most of these systems is defined as support to the driver.

The misstake is to consider that they are ready for self driving. Som brands like Tesla present these cars as self driving.

Until the legal aspect of autonomous vehicles is defined to such extend that law and insurance says the driver do not have responibility there is not any self driving vehicles only PR and sales pitch.
 
910A
Posts: 1888
Joined: Sat Apr 04, 2015 2:11 am

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 1:56 pm

Maybe I missed it in the report, but the driver was watching an episode of “The Voice” while operating the test vehicle at the time of the incident according to the Tempe Police Department (AZ) which investigated the crash.
 
User avatar
flyingturtle
Topic Author
Posts: 5881
Joined: Mon Oct 31, 2011 1:39 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 2:41 pm

910A wrote:
Maybe I missed it in the report, but the driver was watching an episode of “The Voice” while operating the test vehicle at the time of the incident according to the Tempe Police Department (AZ) which investigated the crash.


Well, the thing is that the autonomous systems had a much better chance of spotting the pedestrian. It was dark. Do you also know what was communicated to her? "You have to watch out for x and y, and you have an obligation to..."

It all boils down to which information a user/customer can reasonably trust. If the ride is advertised as "autonomous", it's an invitation to sleep or to read a book. If the paying customer is an essential safety feature, then it would be a WTF on its own.

In Switzerland, we had a cable car accident a few years ago. A family of hikers was killed when they used a cable car intended for goods. The gondola had a placard saying "Not for passengers", but the farm worker who happened to be present allowed the hikers to ride and said it was safe...
Reading accident reports is what calms me down
 
User avatar
DeltaMD90
Posts: 8928
Joined: Tue Apr 15, 2008 11:25 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 2:58 pm

Too bad, for every rare instance of a driverless car killing someone there are probably hundreds of instances of manned cars hitting people.

Of course manufacturers need to be super careful and responsible for their technology, but I hope one or two cases like this doesn't prompt Congress to do something stupid like ban them.

Almost could mostly / completely (?) blame the driver... The companies are taking reasonable precautions with having the driver that should be catching things like this. Yeah I'm sure the job is boring but I mean, it's your job, quit it if you don't like it...
 
User avatar
Aesma
Posts: 13230
Joined: Sat Nov 14, 2009 6:14 am

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 3:13 pm

Driving can already be boring, so not driving but having to just check... Single pilot airliner anyone ?
New Technology is the name we give to stuff that doesn't work yet. Douglas Adams
 
User avatar
DeltaMD90
Posts: 8928
Joined: Tue Apr 15, 2008 11:25 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 3:28 pm

Aesma wrote:
Driving can already be boring, so not driving but having to just check... Single pilot airliner anyone ?

Sure but when it's your job to make sure technology in its infancy doesn't now someone down?

If this person is too bored to pay attention, probably shouldn't be doing this job
 
frmrCapCadet
Posts: 4316
Joined: Thu May 29, 2008 8:24 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 4:13 pm

zkojq wrote:
Aesma wrote:
Well the Volvo used by Uber also has such protection and would probably have at least attempted to brake to avoid the collision, however this was disabled by Uber.

That's just ridiculous. For self driving cars I'd have thought it would be pretty basic to use the Collision Avoidance/Automatic Braking system. Why on earth wouldn't you?


I suspect that the car, as outfitted, would either use Volvo's autonomous safety features, or the Uber autonomous driving but it could not integrate both.
Buffet: the airline business...has eaten up capital...like..no other (business)
 
Flaps
Posts: 1654
Joined: Sat Feb 05, 2000 1:11 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 5:56 pm

All else aside, I simply cannot understand a need for self driving vehicles in the first place, let alone the enormous investment being poured into these projects. Surely we have better uses for these funds. While I can certainly see benefits from the technology development and use of the technologies developed in other areas it just baffles me why anyone would want such a vehicle.
 
User avatar
flyingturtle
Topic Author
Posts: 5881
Joined: Mon Oct 31, 2011 1:39 pm

Re: Self-driving Uber car killing a woman: The cause

Thu Nov 07, 2019 9:19 pm

Flaps wrote:
All else aside, I simply cannot understand a need for self driving vehicles in the first place, let alone the enormous investment being poured into these projects. Surely we have better uses for these funds. While I can certainly see benefits from the technology development and use of the technologies developed in other areas it just baffles me why anyone would want such a vehicle.


There is an urgent need for autonomous vehicles. Three facts:

- cars are still expensive
- we use them for only about 5% of a typical day
- when parked, they're taking up precious space

Autonomous cars make it possible that you can call a car as soon as you need one.

If you were a city planner, you would be horrified of the very concept of personal cars.
Reading accident reports is what calms me down
 
olle
Posts: 2348
Joined: Tue Feb 06, 2007 3:38 am

Re: Self-driving Uber car killing a woman: The cause

Fri Nov 08, 2019 8:58 am

flyingturtle wrote:
Flaps wrote:
All else aside, I simply cannot understand a need for self driving vehicles in the first place, let alone the enormous investment being poured into these projects. Surely we have better uses for these funds. While I can certainly see benefits from the technology development and use of the technologies developed in other areas it just baffles me why anyone would want such a vehicle.


There is an urgent need for autonomous vehicles. Three facts:

- cars are still expensive
- we use them for only about 5% of a typical day
- when parked, they're taking up precious space

Autonomous cars make it possible that you can call a car as soon as you need one.

If you were a city planner, you would be horrified of the very concept of personal cars.



I totally agree. Look at this movie. The next generation vehicles takes things even further.

But in the start they will be used because of legal aspects not on public roads.

https://youtu.be/K8_Ybnj5nNg

V2X development shows that 2023 will be the first year that you legally will be able to let a vehicle take control on public roads. 2028 seems to be a magic year for even more autonomous development.

But the cars today have driving support features not legally correct self driving capabilities.
 
User avatar
Aesma
Posts: 13230
Joined: Sat Nov 14, 2009 6:14 am

Re: Self-driving Uber car killing a woman: The cause

Fri Nov 08, 2019 9:10 am

Flaps wrote:
All else aside, I simply cannot understand a need for self driving vehicles in the first place, let alone the enormous investment being poured into these projects. Surely we have better uses for these funds. While I can certainly see benefits from the technology development and use of the technologies developed in other areas it just baffles me why anyone would want such a vehicle.


My father is taking medication and should not be driving.

My mother gets scared of driving at night.

I could do something else while my car is driving in traffic when commuting.

My brother and sister are city dwellers and drive very occasionally, making them very bad drivers.

I could go on.
New Technology is the name we give to stuff that doesn't work yet. Douglas Adams
 
frmrCapCadet
Posts: 4316
Joined: Thu May 29, 2008 8:24 pm

Re: Self-driving Uber car killing a woman: The cause

Fri Nov 08, 2019 3:18 pm

By cold and hard logic autonomous cars should be required as soon as they show a 10% reductions in deaths, injuries, property damages. My suspicion as that the percent will need to be closer to 50% before public perception concurs.
Buffet: the airline business...has eaten up capital...like..no other (business)
 
User avatar
flyingturtle
Topic Author
Posts: 5881
Joined: Mon Oct 31, 2011 1:39 pm

Re: Self-driving Uber car killing a woman: The cause

Fri Nov 08, 2019 6:12 pm

frmrCapCadet wrote:
By cold and hard logic autonomous cars should be required as soon as they show a 10% reductions in deaths, injuries, property damages. My suspicion as that the percent will need to be closer to 50% before public perception concurs.


It would already help a lot by making RADAR/LIDAR and forced emergency braking mandatory. But without the MCAS Uber has been using in the aforementioned case.
Reading accident reports is what calms me down
 
User avatar
Classa64
Posts: 325
Joined: Fri Apr 03, 2015 5:40 pm

Re: Self-driving Uber car killing a woman: The cause

Mon Nov 11, 2019 2:16 am

As I deal with LKAS(lane keep assist) FCW( forward collision warning) CMBS(Collision Mitigation Brake System) ACC( Active cruise control) and a multitude of other systems as the lead tech at my dealer, they false alarm and go wonky all the time and this is no surprise. Diagnosing them and adjusting them is a whole another issue as to why the car slammed on the brakes for no reason.

Sad it happened, more sadly that a person had to be killed because the car did not react, but when you mix cars with and without these systems and people its its going to cause issues.

When your CMBS kicks in because you looked down to grab your coffee and you did not react to the guy in front slamming on the breaks, sadly its the car behind without the CMBS
that may not or wont stop in time causing the wreck.
"Freedom is the miles i'm rolling on"
 
kindlyfellow12
Posts: 5
Joined: Mon Aug 12, 2019 4:21 am

Re: Self-driving Uber car killing a woman: The cause

Mon Nov 11, 2019 4:53 am

seat64k wrote:
vikkyvik wrote:
Possible failure: what if the pedestrian is not at a crosswalk?


I remember either Elon Musk or Lex Fridman explaining why this is difficult. I don't remember what they said though - might have been something to do with with separating signal from noise - but I do remember thinking, "huh, this is much more complicated than it seems."


I freaking love Lex Fridman's podcast, it's great.

Cars are such a high volume operation, they are used by hundreds of millions of people in the United States alone. I wouldn't be surprised if a billion plus people know how to drive world wide. Even if the self driving algorithm has an insanely high success rate* of 99.99999%, you are still going to have a lot of deaths over time. It's an uphill battle for the self driving folks press-wise.

I'm optimistic that self-driving algorithms will be less dangerous than humans one day, but there will be a lot of kinks to work out...I hope regulators and the general public let that happen.

*you can define this however you like, not killing people is good.
 
tommy1808
Posts: 13480
Joined: Thu Nov 21, 2013 3:24 pm

Re: Self-driving Uber car killing a woman: The cause

Mon Nov 11, 2019 10:51 am

Flaps wrote:
All else aside, I simply cannot understand a need for self driving vehicles in the first place, let alone the enormous investment being poured into these projects. Surely we have better uses for these funds. While I can certainly see benefits from the technology development and use of the technologies developed in other areas it just baffles me why anyone would want such a vehicle.


Commercial driving....

Compared to the sum total of driver wages a couple of billions are peanuts. That is where the investment comes from.

Best regards
Thomas
Well, there is prophecy in the bible after all: 2 Timothy 3:1-6

Who is online

Users browsing this forum: Francoflier, NeBaNi, NYCVIE, SheikhDjibouti, Toenga and 59 guests

Popular Searches On Airliners.net

Top Photos of Last:   24 Hours  •  48 Hours  •  7 Days  •  30 Days  •  180 Days  •  365 Days  •  All Time

Military Aircraft Every type from fighters to helicopters from air forces around the globe

Classic Airliners Props and jets from the good old days

Flight Decks Views from inside the cockpit

Aircraft Cabins Passenger cabin shots showing seat arrangements as well as cargo aircraft interior

Cargo Aircraft Pictures of great freighter aircraft

Government Aircraft Aircraft flying government officials

Helicopters Our large helicopter section. Both military and civil versions

Blimps / Airships Everything from the Goodyear blimp to the Zeppelin

Night Photos Beautiful shots taken while the sun is below the horizon

Accidents Accident, incident and crash related photos

Air to Air Photos taken by airborne photographers of airborne aircraft

Special Paint Schemes Aircraft painted in beautiful and original liveries

Airport Overviews Airport overviews from the air or ground

Tails and Winglets Tail and Winglet closeups with beautiful airline logos