By Tom Fleischman
An autonomous vehicle is able to navigate city streets and other less-busy environments by recognizing pedestrians, other vehicles and potential obstacles through artificial intelligence. This is achieved with the help of artificial neural networks, which are trained to “see” the car’s surroundings, mimicking the human visual perception system.
But unlike humans, cars using artificial neural networks have no memory of the past and are in a constant state of seeing the world for the first time—no matter how many times they’ve driven down a particular road before. This is particularly problematic in adverse weather conditions, when the car cannot safely rely on its sensors.
Researchers at the Cornell Ann S. Bowers College of Computing and Information Science and the College of Engineering have produced three concurrent research papers with the goal of overcoming this limitation by providing the car with the ability to create “memories” of previous experiences and use them in future navigation.
“The fundamental question is, can we learn from repeated traversals?” said senior author Kilian Weinberger, professor of computer science in Cornell Bowers CIS. “For example, a car may mistake a weirdly shaped tree for a pedestrian the first time its laser scanner perceives it from a distance, but once it is close enough, the object category will become clear. So the second time you drive past the very same tree, even in fog or snow, you would hope that the car has now learned to recognize it correctly.”
“In reality, you rarely drive a route for the very first time,” said co-author Katie Luo, a doctoral student in the research group. “Either you yourself or someone else has driven it before recently, so it seems only natural to collect that experience and utilize it.”
Spearheaded by doctoral student Carlos Diaz-Ruiz, the group compiled a dataset by driving a car equipped with LiDAR (Light Detection and Ranging) sensors repeatedly along a 15-kilometer loop in and around Ithaca, 40 times over an 18-month period. The traversals capture varying environments (highway, urban, campus), weather conditions (sunny, rainy, snowy) and times of day.
This resulting dataset—which the group refers to as Ithaca365, and which is the subject of one of the other two papers—has more than 600,000 scenes.
“It deliberately exposes one of the key challenges in self-driving cars: poor weather conditions,” said Diaz-Ruiz, a co-author of the Ithaca365 paper. “If the street is covered by snow, humans can rely on memories, but without memories a neural network is heavily disadvantaged.”
The research was presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022), held in New Orleans in June.
https://news.cornell.edu/stories/2022/06/technology-helps-self-driving-cars-learn-own-memories
AR #129
“Giant City Discovered in Guatemalan Jungle”
