The Real Issue With Self-Driving Cars
The Market Ticker - Commentary on The Capital Markets
Logging in or registering will improve your experience here
Main Navigation
Display list of topics
Sarah's Resources You Should See
Sarah's Blog Buy Sarah's Pictures
Full-Text Search & Archives
Legal Disclaimer

The content on this site is provided without any warranty, express or implied. All opinions expressed on this site are those of the author and may contain errors or omissions.

NO MATERIAL HERE CONSTITUTES "INVESTMENT ADVICE" NOR IS IT A RECOMMENDATION TO BUY OR SELL ANY FINANCIAL INSTRUMENT, INCLUDING BUT NOT LIMITED TO STOCKS, OPTIONS, BONDS OR FUTURES.

The author may have a position in any company or security mentioned herein. Actions you undertake as a consequence of any analysis, opinion or advertisement on this site are your sole responsibility.

Market charts, when present, used with permission of TD Ameritrade/ThinkOrSwim Inc. Neither TD Ameritrade or ThinkOrSwim have reviewed, approved or disapproved any content herein.

The Market Ticker content may be sent unmodified to lawmakers via print or electronic means or excerpted online for non-commercial purposes provided full attribution is given and the original article source is linked to. Please contact Karl Denninger for reprint permission in other media, to republish full articles, or for any commercial use (which includes any site where advertising is displayed.)

Submissions or tips on matters of economic or political interest may be sent "over the transom" to The Editor at any time. To be considered for publication your submission must include full and correct contact information and be related to an economic or political matter of the day. All submissions become the property of The Market Ticker.

Considering sending spam? Read this first.

2018-03-28 07:00 by Karl Denninger
in Technology , 743 references Ignore this thread
The Real Issue With Self-Driving Cars
[Comments enabled]

It's not that the car apparently didn't "see" a pedestrian walking a bicycle and hit her.  At least we think it didn't see her.  We actually don't know that yet, just that she was struck.

But you better start thinking about the real regulatory issue, and one that had better get center-stage right damn now and be part of the debate and requirements if these things are going to run around on our roads.

It's this:

The car is on a two-lane road with a hard and fixed barrier on the right side, and on-coming traffic on the other side.  A child runs out in front of the vehicle inside of the stopping distance.  The car detects the child instantly, but is unable to stop.

The vehicle is able to compute the probability of your death (and everyone else in your vehicle) with a fair degree of certainty if it intentionally crashes into the fixed object to avoid the child.  If the oncoming vehicle is also self-driving it may also be able to compute the risk of death or serious injury for those occupants if it hits that car intentionally since the vehicles are probably communicating.  If it hits the child it also probably can compute the odds (very high, perhaps 100%) that the child will die as well.

The vehicle must strike something due to the physics of the situation.

What decision does the car make and who or what does it hit?

If you are in this situation as a human driver you cannot compute the risk of death for various parties, since you don't have the mass of the vehicles, the energy each carriers, the presence or absence of wearing seat belts, where the impact will be taken, etc.  The car can make that computation in the milliseconds available, you cannot.

However, you still can choose to intentionally decide to hit the solid abutment, or the oncoming car to avoid hitting the child.

You as someone buying or riding in a self-driving vehicle must be able to know the decision tree on this situation in advance because it does happen.

Now let's take another example.  Uber is claimed to have something like 2 million road miles on its self-driving cars but there have been ~60 accidents, most minor and nearly all the fault of the other driver.  This may sound to you like a good record.

It's not.  It in fact sucks big fat donkey balls; a human driver with that record would be considered a terrible risk (and pay an astronomical insurance premium) no matter who was technically "at fault."

I have well over 750,000 lifetime miles on the road by my best guess.  My current car has 130,000.  The truck in the driveway has about 60,000.  My Jetta, which my kid now has, was given to her just short of 200,000.  That's nearly 400,000 miles just between these last three vehicles, and there were years when I lived in Chicago where I put 50k on a car because I was doing contract work and in the damn thing all the time.

My lifetime accident record on the road?  Zero.

But I have, several times (including fairly recently) intentionally violating a traffic law to avoid an accident, the most-common incident being intentionally running a light that has either just turned red or is about to and I detect that the vehicle behind me is not going to stop before striking me, yet the crossing road is clear.  The law says I must stop if I can do so safely.  I will take and fight any ticket ever given to me where I run such a light for that reason, because if I'm about to be hit then "stopping safely" isn't going to occur.  I've yet to be ticketed for this but I've avoided several accidents this way -- none of which would have been "my fault", but all of which would have damaged my car and maybe myself or others in the car.

How are these self-driving cars programmed and why would anyone get into one without knowing that first -- or buy one, for that matter?

You have every right to know if the vehicle will obey traffic laws even if it means getting struck and you potentially being injured or worse as a result, and you also have the right to know if the vehicle will kill or injure you preferentially and intentionally to avoid killing someone else who is not in the car.  Since you are not driving you are never "at fault" for hitting someone else, which means you personally will never be tagged in such a lawsuit -- the company that made the car will be!

This provides a powerful incentive for the vehicle designer to avoid hitting the other party even if they have to injure or kill you in order to do so since you have some contractual relationship with said firm (in which they might, and probably will, try to limit their liability) but the other party does not and thus can't be bound by same.

In short you have every right to know what the vehicle's "prime directive" is but we're not even talking about this!

This is the issue with these self-driving cars.  The machine may have limitations on what it can see and sense but it is always faster in making the decision and acting on it than you can be.  The question here first is did the car see the woman but second, if it did then the next question is "did it deliberately hit her because the other options were worse?"

If that's not the case here it will be in the future and in fact it has already happened with less-dire consequences, it would appear, if these vehicles have managed to rack up 60 accidents in 2 million miles where I, as a human, have yet to have one in more than 25% of that mileage -- but I have had to break traffic laws and take anticipatory actions for that to be the case.

These self-driving cars clearly are doing neither and you have every right to know what their decision matrix is and what preference it will take both in the event that a traffic law conflicts with an impending accident (that is otherwise avoidable) and what it will do in a "no-win" scenario -- before you get into one.

Never mind the data security and spying that will go on, with every trip you take in one of these things being data that does not belong to you.

Go to responses (registration required to post)
 

 
Comments.......
User: Not logged on
Login Register Top Blog Top Blog Topics FAQ
Showing Page 1 of 3  First123Last
User Info The Real Issue With Self-Driving Cars in forum [Market-Ticker]
Tomusher
Posts: 2
Incept: 2018-03-07

Robertsville
Report This As A Bad Post Add To Your Ignored User List
This brings to mind Asimov's Three Laws of Robotics:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

#1 would come into play in the scenario you lay out and who knows what the decision would be. In Asimov's stories #1 eventually leads to robots dominating mankind in the name of "safety".

I guess it's the sum total of too many science fiction books when I was younger but the current push towards eliminating the human element in the decision trees of life scares the hell out of me. By and large, I'll trust the ponderous and problematic human response just about every time as opposed to the mechanical certainty provided by technology.
Bodhi
Posts: 364
Incept: 2008-02-23

Georgia
Report This As A Bad Post Add To Your Ignored User List
Quote:
But I have, several times (including fairly recently) intentionally violating a traffic law to avoid an accident, the most-common incident being intentionally running a light that has either just turned red or is about to and I detect that the vehicle behind me is not going to stop before striking me, yet the crossing road is clear.


When my son was about 10 years old I was driving us home from breakfast. A traffic signal turned red before I went through the intersection, but I could see there was no cross traffic. My son exclaimed from the backseat, "Dad! You ran the red light." I responded, "Yep. So did the car right behind us." He thought for a few seconds, then replied, "Oh ..."

It would be good to know if self-driving cars also calculate for this type of situation.
Shannonlk1
Posts: 195
Incept: 2008-12-02

Raleigh
Report This As A Bad Post Add To Your Ignored User List
I've been waiting for you to write a article on this. I've seen the video of the incident. At first glance (nor more than what the cops did). It appears that it could not have been avoided. However, Notice how much angle the camera actually has. It has what amounts to hallway vision compared to just how wide a human can view things. Granted, it may have sensors on it as well. But if it did, they didn't work. Not just that, I would bet $10 to a donut, that a human could have seen something - movement, a silhouette, or something and slowed down and avoided this person. I have lost count of the number of times I saw "something" in the distance at night long before I got to it. I've said it 1000 times or more. There are 10s if not 100s of variables in play when behind the wheel of a car. when I'm driving, not only am i looking at the car in front of me, I'm looking at EVERY car in front of me and coming in my direction. I'm watching the adjacent parking lots for cars as well as anything else including shopping carts. There is no freaking way that a car can handle that many variables without having 20 plus sensors, radar and infrared. not only that if it has radar, is another car's radar going to interfere with it. then there is GPS, which needs to be correct 100% of the time, which is never the case. what happens if there is a foot deep pot hole in the road? what if debris falls off a vehicle. What about standing water, what about water running across the road. Standing water can be treacherous if you don't know how deep it is. Then there is cyber security, You know they won't be secure. We can't get metro trains accident free, but they want to put 1000s of these on the road? Expect a lot of deaths.

----------
Criminals thrive on the indulgence of Society's understanding.

Reason: couple of mispelled words
Goforbroke
Posts: 7338
Incept: 2007-11-30
A True American Patriot!
Time to feed the chickens.
Online
Report This As A Bad Post Add To Your Ignored User List
Could someone please tell me what problem we're trying to solve with self-driving cars?

----------
Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure. It is our Light, and not our Darkness, that most frightens us. -- Marianne Williamson
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
Uber's inability to make money because they have to pay all those drivers.

----------
Winding it down.

Redjack
Posts: 25
Incept: 2018-01-29

Iowa
Report This As A Bad Post Add To Your Ignored User List
I have been blessed to work in many different fields as a process engineer. From Food, to pharma feed stocks, to manufacturing of raw materials I have been using automation of some sort since I graduated.

I don't trust "self driving" cars. I have bots that smash into racks all the time. This is in a controlled environment, with limited inputs (no random people or objects entering the cell other than defined areas.

We use some of the same sensors these cars do. If the ultrasonic sensor gets dirty, it can lose position. I am not talking massive dirt, I am talking a thin layer of dust. The laser can be "faked" in similar conditions.

For some of the sensors outside, we have PM's to clean them every few hours depending on the weather. Rain plays havoc with them. Now you can program some of that out, but that just makes the sensor less able to "see" small objects.

Again, this is in a controlled environment, not a road. My coworkers and vendors (some of which supply Uber) have little faith in this.

But it will be forced through because the ability to totally control transportation is one that no State will pass up.
Goforbroke
Posts: 7338
Incept: 2007-11-30
A True American Patriot!
Time to feed the chickens.
Online
Report This As A Bad Post Add To Your Ignored User List
Oh, OK. Thanks.

----------
Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure. It is our Light, and not our Darkness, that most frightens us. -- Marianne Williamson
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
@Redjack -
Quote:
But it will be forced through because the ability to totally control transportation is one that no State will pass up.

Yep. That too.

But the real "driver" on this is the handful of "new tech" firms that "have to" succeed, as they can be "greener", they feed the insanity of certain political views, and they play "utopia."

Then the screwing REALLY begins as the cost-shift starts onto those who say "no ****ing way", including myself.

With the "driving record" these things have to date I would rather WALK than use one when I finally get old enough that I cannot safely drive. I can't believe NOBODY is talking about how ****TY 60 accidents in 2 million road miles actually is.

If you had that record -- which is about one accident every two years for the average driver's road mileage (that is, about one wreck for 30,000 miles) your insurance costs would be so astronomical you couldn't afford to own a car! ****, my KID has a MUCH better record than that (zero wrecks thus far) and she's just 21 and thus the only reason her record isn't MUCH better is that she hasn't been licensed for all that long, NEVER MIND THAT WHEN YOU'RE A NEW DRIVER YOUR WRECK RISK IS MUCH HIGHER SIMPLY DUE TO LACK OF EXPERIENCE!

----------
Winding it down.
Elkad
Posts: 386
Incept: 2009-09-04

Report This As A Bad Post Add To Your Ignored User List
Driving through a residential area. I spot some kids near the road a couple houses up, and I'm letting the car coast down, fully alert, covering the brake, doing maybe 20 mph.

Toddler came out from between parked cars. Not at that house, but right in front of me. Way inside my braking distance. Traffic in the oncoming lane.

Now I may not have a table of information that tells me the odds of each participant being injured/killed, but anyone can solve that problem. I delayed my brake stab to retain full steering, wrenched the wheel left, and only after the car had turned did I try to hit the brake. And hit an oncoming car. Combined impact speed maybe 30mph. I pushed her car into another one parked on the other side of the road and ssquashed it as well.

Lady jumped out of the other car and said "Oh **** that hurt. Thank you for hitting me."

I would have been "not at fault" for the accident if I'd killed that little girl. I ruined 3 cars, at great detriment to all the insurance companies. I never learned who paid what, but they jacked my rates into the sky.
Wifi
Posts: 635
Incept: 2013-02-13

Seagrove Beach
Report This As A Bad Post Add To Your Ignored User List


Quote:
Uber's inability to make money because they have to pay all those drivers.

Yep

----------
Hurricane Evacuation Plan
1.Grab Beer
2.Run Like Hell
Aquapura
Posts: 718
Incept: 2012-04-19

South of Canada
Report This As A Bad Post Add To Your Ignored User List
I have a perfect example from my commute yesterday evening. Coming up to a stoplight in a 45mph zone where traffic was flowing 55mph or better. I can tell the person behind me was not slowing down. I start pumping my brakes hoping the strobe of the brake lights will wake her up. As I'm doing this I pull over into a painted median zone. She slams on the brakes and stops but not before sliding into the zone where my vehicle would have been. This all happened over a matter of seconds but was second nature to me from several decades and hundreds of thousands of driving miles experience. I'm sure they can program a computer to do that sort of thing but I'm skeptical a computer can read the situation like I did. And this was someone coming from behind me. I'm sure the autonomous vehicles would've just let themselves get hit. Still remember from my drivers ed. training that you need to be situationally aware of everything happening around you when driving. Can a robot do that like a human? I highly doubt it.
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
In that specific situation from the various videos I've seen there was (1) no oncoming traffic, (2) the pedestrian was crossing into the adjacent curb (that is, time HELPS miss) and (3) there was sufficient time to BOTH slow AND go around her. She PROBABLY assumed the vehicle would slow somewhat (long before it got that close) and there isn't much of a mis-judge on approach distance (especially at night) required on her part to make the (reasonable) assumption that from a materially further distance the car will slow somewhat and EASILY clear her.

The car basically ran her over without warning; it either didn't "see" her AT ALL (there appears to have been NO attempt to slow down!) which boggles the mind or it DID see her and IGNORED the impending collision.

----------
Winding it down.
Goforbroke
Posts: 7338
Incept: 2007-11-30
A True American Patriot!
Time to feed the chickens.
Online
Report This As A Bad Post Add To Your Ignored User List
Quote:
I would rather WALK
Just don't walk on a street where there are driverless cars around.

----------
Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure. It is our Light, and not our Darkness, that most frightens us. -- Marianne Williamson
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
@Goforbroke
Quote:
Just don't walk on a street where there are driverless cars around.

That's what this is for: https://www.magnumresearch.com/desert-ea....

It may get me, but I'll get it and whoever is in it first.

----------
Winding it down.
Superdude
Posts: 296
Incept: 2009-06-16

Report This As A Bad Post Add To Your Ignored User List
If you get into an accident with one of these robots and it was their negligence, who do you press charges against or sue? (The car manufacturer? the company who uses them? Ceo?)
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
Whoever made the robot.

Good luck tagging them criminally, which is the real problem -- you will eat the cost of the suit indirectly, and that assumes you live. There is no standard for "gross negligence" by a robot.

----------
Winding it down.
Xqqme
Posts: 720
Incept: 2009-01-09

Ohio
Report This As A Bad Post Add To Your Ignored User List
Cars are the least of highway worries....
Fed Ex "is working with the startup Peloton Technology, whose semi-autonomous technology electronically links trucks into small caravan groups called platoons. The system, which uses wireless vehicle-to-vehicle communication to enable the driver of a lead truck to control the gas and brakes of a truck following closely behind him, is designed to reduce wind resistance and save fuel. The technology is considered a significant step toward fully autonomous trucks, and Peloton has said it will release it in late 2017."
https://www.technologyreview.com/s/60289....
Platoons of autonomous heavy vehicles! Delivery vans roaming neighborhoods with deliveries. What'll they think of next?
Thorvold
Posts: 218
Incept: 2013-09-12

NY
Report This As A Bad Post Add To Your Ignored User List
They train humans to deal with inevitable crashes so it must be expected for self-drivers to have built-in training. A friend of mine was trained to avoid slamming the brakes of a city bus if cut off by a car as the injury risk to many passengers in a bus exceeded that of injury to the fewer occupants of a car. I would expect a self-driving bus to have similar programming, but who knows. The programming decisions for cars are certainly more complicated.
Skybluepink
Posts: 406
Incept: 2007-10-20

RI
Report This As A Bad Post Add To Your Ignored User List
One day I was driving in the middle lane of the highway on a windy day. A large black garbage bag blew across, right in front of me. I drove right into it, because it was just an empty garbage bag, and swerving could have caused an accident. It made me wonder what a self driving car would do in that situation. Could it tell the difference between a garbage bag and a solid object? Would it jam on the brakes in traffic causing a pile up? Would it swerve and create a potentially dangerous situation?
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
On the way back from my CO trip I encountered (for the first time in my driving) tumbleweeds blowing across the road. REAL ONES.

I ran a couple of them over. While it was certainly POSSIBLE to avoid some of them (and I did) it wasn't possible to avoid ALL of them. On the road I was on there was little to hit if I had swerved violently, but there was utterly NO REASON to swerve violently to avoid hitting what amounted to a dried ball of vegetable matter.

Would a self-driving car have run them over? WHAT IF ONE OF THE WEEDS WAS A KID? Can IT tell the difference?

----------
Winding it down.
Asimov
Posts: 109819
Incept: 2007-08-26

East Tennessee Eastern Time
Report This As A Bad Post Add To Your Ignored User List
Any situation unknown to the car's programing is going to have an unpredictable outcome.

I don't think that's going to change anytime soon.

Imagine what the reaction is going to be when a self driving car ends up driving through a crowd, or killing a busload of kids in an OBVIOUSLY avoidable accident.

I don't even want to think about hacking these things. Anybody who's seen the last fast and furious movie will have some concept of why I don't want to consider it.

----------
It's justifiably immoral to deal morally with an immoral entity.

Festina lente.
Emg
Posts: 179
Incept: 2012-11-20

Canada
Report This As A Bad Post Add To Your Ignored User List
Researchers have found that changing a single pixel in an image can often convince a neural-network-based image recognition system to see the image as something completely different.

https://arxiv.org/abs/1710.08864

One pixel.

If I remember correctly, another team put a few lines on a stop sign and that convinced the computer that it was an animal, instead.

Driverless cars are a long way from reality, but super-hyped. Not just because so many companies want to eliminate those pesky drivers, but because governments want to ensure that individuals won't be able to travel where they want to travel.
Tickerguy
Posts: 152455
Incept: 2007-06-26
A True American Patriot!
Report This As A Bad Post Add To Your Ignored User List
The problem with "artificial intelligence" is that it is not intelligence. All such claims thus far are a LIE.

NEVER ONCE has a machine demonstrated "out of scope" operation which is the very definition of intelligence. That is, constructing out of whole cloth a theory or operative statement of fact that was beyond the range of what it was programmed to do.

Humans do this every single day. It is why I can see a tumbleweed rolling across the road for the first time in 40 years of operation of a vehicle and without conscious contemplation determine it is not dangerous -- I do not need to avoid it as running over it is of no importance, other than the possibility of having to stop and clear it if it hangs up on the undercarriage of the car and the resulting noise annoys me.

So-called "artificial intelligence" is nothing more than very fast pattern recognition. That's all the "Deep Blue" win was as a chess master; from LOTS of games it was able to analyze lots of patterns, very fast, and determine outcomes from them.

It could NOT determine that the player it was competing against was about to pull a pistol and shoot it, yet a human player might well figure that out BEFORE THE GUN CAME OUT from the body language and other cues that his opponent displayed.

Deep Blue couldn't figure this out because it cannot actually think; it lacks the essence of actual intelligence, which is the ability to form an out-of-scope result.

EVEN THE MOST-MENTALLY-RETARDED INDIVIDUAL can do this on at least a LIMITED basis. Hell, even some animals display a rudimentary ability to do that -- such as a crow. Lock a crow in a cage and place a stick outside but which it can reach that, with some out-of-scope thought it is possible to determine that the cage can be unlocked with it, and there's a very good chance the crow will figure out how to escape. It'll take a while but the crow will obtain its own freedom.

The computer never will unless you teach it what a stick is and that it can be used as a tool, yet the crow in question HAS NEVER BEEN CONFINED IN A CAGE BEFORE, has NEVER seen a lock and has NEVER manipulated a metal device using a tool it an acquire from the environment to release itself from said cage and thus its "solution" is 100% out-of-scope with its prior life experience.

----------
Winding it down.

Click
Posts: 168
Incept: 2017-06-26

Report This As A Bad Post Add To Your Ignored User List
"prime directive"

This is where the government comes in. It's a foregone conclusion.

As already duly noted by other commenters the government will use force and coercion to fully automate American Highways... Why? "Because the ability to totally control transportation is one that no State will pass up."

Control the guns.
Control the cars.
Control the ______ (you fill in the blank).

That's how Marxists roll...
Login Register Top Blog Top Blog Topics FAQ
Showing Page 1 of 3  First123Last