Cameras on cars are becoming more common as part of driver assistance systems. They serve as an extra set of eyes to spot and warn of hazards around a vehicle and will be absolutely essential to self-driving cars. As drivers begin to let go of the wheel and have the car take control, one of the biggest hurdles will be when (and how) to warn humans they need to take the helm. In this case, cameras can also come in handy by being turned inward.
Automotive suppliers Continental and Delphi have developed prototypes utilizing interior cameras as part of “driver monitoring” systems. Now the German research organization Fraunhofer is developing cameras and software that not only watch the driver, but everyone and everything in the car.
“Using depth-perception cameras, we capture the vehicle’s interior, identify the number of people, their size and their posture,” Michael Voit, manager of Fraunhofer’s Optronics, System Technologies, and Image Exploitation research group, said in a blog post. “From this we can deduce their activities.”
According to Fraunhofer, interior cameras can play a crucial role in managing the hand-off between man and machine by checking on the status of the person responsible for taking control of a self-driving car. “Using [cameras], the system can estimate how long the driver will need to resume full control of the vehicle following a period of autonomous driving,” said Frederik Diederichs, a scientist and project manager at Fraunhofer.
While automakers use weight sensors in seats to detect the presence of a driver or passengers to arm airbags (and pester occupants with seat-belt warnings) cameras can capture much more detail. For example, Fraunhofer noted that cameras could be used to fine-tune the deployment of airbags to individual body sizes and even the placement of passenger in a vehicle.
The company added that with cameras “analyzing the position of a passenger’s limbs, airbags could recognize special situations, such as when a passenger has their feet on the dashboard” In these instances, the system could warn passengers of the dangers of having body parts in the way of an exploding airbag—and I wouldn’t have to yell at my teens to keep their feet on the floor where they belong. Fraunhofer said that an interior camera could even allow a self-driving car to steer itself in an instant before a collision to mitigate the impact to occupants.
But as with exterior-facing cameras, one of the biggest hurdles is the lag between the hardware that’s capturing images inside the car and software used to analyze the data. Just this week Tesla CEO Elon Musk remarked, “The hardware exists to create full autonomy,” but the greatest barrier “is really a software limitation.”
Fraunhofer likewise noted that the “main challenge lies in evaluating the recorded data” from its interior camera system. “The software can already detect people and their limbs, but how can the computer be taught to recognize what passengers are doing?”
Fraunhofer researchers are currently testing the interior camera system using a driving simulator and next will integrate it into a Volkswagen van for real-world validation, which “will form the basis for new vehicle concepts for the next five to 10 years,” the company said. Fraunhofer is already working with Volkswagen, Bosch, Visteon, and others on its “Intelligent Car Interior” project, which includes the in-car camera.
This technology would likely be a boon to trucking and delivery companies to keep tabs on their drivers, as well as to car insurers to get accurate accident data. Still, many drivers may feel uncomfortable with the idea of cameras and computers knowing everything they’re doing while in their cars. With some justification.
One untended consequence of cockpit cameras, especially as cars become increasingly connected, will be privacy issues. In 2013, it was discovered, for instance, that built-in cameras in Samsung Smart TVs could be remotely hacked. And the recent rash of vehicle hacks will give the paranoid pause about having a camera inside the car.
But look at the bright side: In addition to enabling self-driving, it could launch a whole new genre of amateur Carpool Karaoke.
Originally Published by PCMag.com