Safety systems are there to be gamed...

Technology in driving is becoming more dominant...
User avatar
jont-
Posts: 1043
Joined: Mon Sep 28, 2015 7:12 am
Location: Flatlands

Re: Safety systems are there to be gamed...

Postby jont- » Fri Jan 26, 2018 1:43 pm

Straw man argument. If an autonomous car is making decisions about harming humans, it's doing it wrong. Surely that was the whole point of introducing them in the first place?

User avatar
Horse
Posts: 2007
Joined: Mon Sep 28, 2015 9:20 am

Re: Safety systems are there to be gamed...

Postby Horse » Fri Jan 26, 2018 1:54 pm

Surely it's being honest and accepting that, sooner or later, this situation will occur? Your expectation might vary - but realism says that nothing is perfect, so it would be a bit remiss to not consider it.
My own views. For better or worse :)

User avatar
jont-
Posts: 1043
Joined: Mon Sep 28, 2015 7:12 am
Location: Flatlands

Re: Safety systems are there to be gamed...

Postby jont- » Fri Jan 26, 2018 2:03 pm

Horse wrote:Surely it's being honest and accepting that, sooner or later, this situation will occur? Your expectation might vary - but realism says that nothing is perfect, so it would be a bit remiss to not consider it.

If you start considering it, you then start making excuses about shoddy software being acceptable because the car can make ethical decisions.... and what's to say you won't have bugs in the ethics code too?

Sure, it's going to go wrong, but I'd rather it started with the premise that it can't be allowed to and we design to that quality.

User avatar
Horse
Posts: 2007
Joined: Mon Sep 28, 2015 9:20 am

Re: Safety systems are there to be gamed...

Postby Horse » Fri Jan 26, 2018 2:26 pm

jont- wrote:
Horse wrote: but realism says that nothing is perfect, so it would be a bit remiss to not consider it.

If you start considering it, you then start making excuses about shoddy software being acceptable because the car can make ethical decisions.... and what's to say you won't have bugs in the ethics code too?

Sure, it's going to go wrong, but I'd rather it started with the premise that it can't be allowed to and we design to that quality.


Probably more difficult to write software that incorporates those aspects! But why do you think that designers won't be trying to prevent those situations occurring? Another real-world parallel: you develop and practice mental and physicals skills to avoid bad situations - but learn how to do emergency stops for when they do happen.
My own views. For better or worse :)

User avatar
jont-
Posts: 1043
Joined: Mon Sep 28, 2015 7:12 am
Location: Flatlands

Re: Safety systems are there to be gamed...

Postby jont- » Fri Jan 26, 2018 2:37 pm

Horse wrote:
jont- wrote:
Horse wrote: but realism says that nothing is perfect, so it would be a bit remiss to not consider it.

If you start considering it, you then start making excuses about shoddy software being acceptable because the car can make ethical decisions.... and what's to say you won't have bugs in the ethics code too?

Sure, it's going to go wrong, but I'd rather it started with the premise that it can't be allowed to and we design to that quality.


Probably more difficult to write software that incorporates those aspects! But why do you think that designers won't be trying to prevent those situations occurring? Another real-world parallel: you develop and practice mental and physicals skills to avoid bad situations - but learn how to do emergency stops for when they do happen.

Engineers are lazy. If they know there's a fallback, they will start to rely on it.

Pyrolol
Posts: 72
Joined: Sun Oct 04, 2015 1:52 am
Location: San Francisco

Re: Safety systems are there to be gamed...

Postby Pyrolol » Fri Jan 26, 2018 2:59 pm

At some level we all accept that a pedestrian who dives sideways into a road at the wrong moment will get hit, or a car that runs a red into a stream of traffic, or swerves to the other side of the road in the wrong situation. It doesn't mean that an autonomous car is wrong for getting into this situation, so it may need to consider evasive action without there being any planning failure. It's rare, but happens.

User avatar
Horse
Posts: 2007
Joined: Mon Sep 28, 2015 9:20 am

Re: Safety systems are there to be gamed...

Postby Horse » Fri Jan 26, 2018 3:12 pm

jont- wrote:Engineers are lazy. If they know there's a fallback, they will start to rely on it.


Ah, that will be the 'glass half leakey' view of engineering?

Perhaps you should be more concerned about AVs being built down to a price point, with compromise as a result.
My own views. For better or worse :)

ancient
Posts: 174
Joined: Mon Dec 14, 2015 4:03 pm

Re: Safety systems are there to be gamed...

Postby ancient » Fri Jan 26, 2018 3:15 pm

We know how fast pedestrians can move sideways (it's not an infinite speed) and have automatic cars moderate their speed when passing humans at that distance, such that any impact will not do significant harm (another calculable limit that we can stay well below). This is likely to mean that vehicles will move at walking speed in close proximity to pedestrians (which is no bad thing) and be able to speed up when their trajectory puts them at significant distance from pedestrians.
Just like good human drivers of course, who slow down when passing through a crowded shopping street and speed up on motorways.

User avatar
jont-
Posts: 1043
Joined: Mon Sep 28, 2015 7:12 am
Location: Flatlands

Re: Safety systems are there to be gamed...

Postby jont- » Fri Jan 26, 2018 3:22 pm

Horse wrote:
jont- wrote:Engineers are lazy. If they know there's a fallback, they will start to rely on it.


Ah, that will be the 'glass half leakey' view of engineering?

Perhaps you should be more concerned about AVs being built down to a price point, with compromise as a result.

That too. Again, why waste expensive compute on solving difficult problems if you can just slam the brakes on and say "it did its best".

Actually, one positive outcome from dieselgate would be looking for cheating on any tests that come in too.

User avatar
akirk
Posts: 1497
Joined: Sun Sep 27, 2015 6:58 pm
Location: Cotswolds

Re: Safety systems are there to be gamed...

Postby akirk » Fri Jan 26, 2018 4:25 pm

ancient wrote:We know how fast pedestrians can move sideways (it's not an infinite speed) and have automatic cars moderate their speed when passing humans at that distance, such that any impact will not do significant harm (another calculable limit that we can stay well below). This is likely to mean that vehicles will move at walking speed in close proximity to pedestrians (which is no bad thing) and be able to speed up when their trajectory puts them at significant distance from pedestrians.
Just like good human drivers of course, who slow down when passing through a crowded shopping street and speed up on motorways.


awesome - so pedestrians will just walk across the road whenever they like, knowing that cars will stop - at present there is a balance - pedestrians take care and look etc. as they can't guarantee that the cars will stop - knowing that the cars will stop will change pedestrian habits, maybe over time, but it will happen...

Alasdair


Return to “Technology”

Who is online

Users browsing this forum: No registered users and 1 guest