Today’s Autonomous Driving Systems (ADS) are not as much independent as to allow the driver to have their mind off the road, or at least to have their eyes off the road, e.g. to read a newspaper or play video games. ADS are not safe enough for real autonomous driving due to the fact that they use Artificial Intelligence (AI), and the safety of this technology is still a research topic.
Understanding safety of ADS
2016, a Tesla Model S collided with an upright semi-trailer that it didn’t recognize while on AutoPilot. The system predicted the large reflecting metal surface of the crossing truck to be clear blue sky. On June 1, 2020, a Tesla Model 3 crashed into an overturned box truck on a Taiwan highway.
Let’s watch this crash in the video below!
The fundamental problem of AI
So why do these accidents keep happening? AI is great in analysing vast amounts of data. It can identify patterns and structure far better than humans could ever do. So why is it not safe? What seems to be the problem with AI based object detection? Let’s say we want to train an AI to identify bears and zebras. That should be easy, right? We show it a dataset of thousands of pictures of bears and zebras.
The unknown in theory
Now, here is the catch! What happens if something appears that isn’t part of the training dataset? We show the AI the picture of a panda bear. The AI might tell you with extremely high confidence: Classification: Zebra! Or it might tell you: Bear! But it will either be bear or zebra – either one of its known classes. AI cannot recognize the unknown.
The unknown out there
In the real world, the unknown could be anything. Do you recognize the scene on the left? It’s a rickshaw, which an AI did not even recognize as a whole thing but as a heap of a bicycle, a person, a car, a traffic light, and a traffic sign.
A constantly changing world
We cannot train an AI to recognize everything but only certain classes of objects, e.g. other cars, pedestrians, traffic lights, or lane markings – objects which the ADS is most likely to encounter in traffic. Environments change over time, too. We all know, that hoverboards just arrived in 2015, and E-Scooters in 2019. We need AI that can say “I’m not sure what this thing is!”.
Let’s watch a movie!
We have developed AI that can say “I’m not sure what this thing is!”. Let’s make an experiment. We’ve trained an AI with 20 classes, e.g. pedestrians, cyclists, buildings, road surface, traffic signs, and pavement. As an obstacle to the AI, we’ve put boxes on the road as if unloading a car.
Let’s watch the experiment in the video below!