Thatcham on Thursday announced it and Euro NCAP had developed a means of grading OEMs’ “assisted driving” technology, with the initial results highlighting gaps in elements like performance and driver engagement.

The term “assisted driving” refers to technology along the lines of Tesla’s Autopilot. The vehicle can control speed and steering itself, but some level of driver engagement is necessary.

In the U.S., all such “active driving assistance” systems (to use the SAE- and AAA-endorsed term) are advanced versions of SAE autonomy Level 2. The vehicle drives itself, and the driver can even take their hands off the wheel in some cases. However, the motorist must still pay attention to the road at all times and be ready to take over if the car asks or starts to do something stupid.

It’s unclear if any “assisted driving” in Europe reaches the SAE Level 3 stage of autonomy, which involves a vehicle able to drive itself in certain conditions well enough that the driver can actually quit watching the road. However, the driver must still be ready to retake the wheel if the car demands it. (One potential example of the tech would involve a vehicle able to drive itself on a freeway but handing control back to the driver when an exit is reached or severe weather arises.)

But it appears that Thatcham defines the concept along the lines of SAE Level 2, where the motorist must always pay attention.

As a collision repairer, active driving assistance technology matters for multiple reasons. First, it has the potential to cut crashes. However, it could also encourage them if drivers mistakenly or intentionally see the tech as a reason to quit paying attention to the road.

It also could place a shop’s repairs squarely in the crosshairs of attorneys or regulators if the vehicle gets in a crash. Even if the shop fixed the car correctly and the error lies in the vehicle technology, the driver, or the handoff between the AI and human, it still could be an uncomfortable experience.

For example, Heather Lommatzsch sued Service King and Tesla in 2018 after her 2016 Tesla Model S on Autopilot plowed into a truck at 60 mph. Police said Lommatzsch admitted looking at her phone, and Tesla data showed she had her hands off the wheel.

Nevertheless, Lommatzsch sued the OEM and repairer anyway.

Lommatzsch said Service King “provided service to the Tesla Model S and had replaced a sensor on the Tesla Model S” She alleged Tesla’s sales force told her the vehicle would stop on its own and she only needed to touch the wheel once in a while while in Autopilot. The OEM should have communicated Autopilot’s limitations better, Lommatzsch argued.

She ended the case against the body shop in July 2019, with both sides agreeing to pay their own costs in a joint motion for dismissal with prejudice. She dropped the case against Tesla in February with a similar arrangement.

So Thatcham and Euro NCAP’s work is certainly relevant. The quantification of these vehicles’ performance could spur OEMs to rapid improvements on the systems; this certainly has occurred with Insurance Institute for Highway Safety scores in the U.S.

“The systems that are currently allowed on our roads are there to assist the driver – but do not replace them,” Thatcham research director Matthew Avery said in a statement. “Unfortunately, there are motorists that believe they can purchase a self-driving car today. This is a dangerous misconception that sees too much control handed to vehicles that are not ready to cope with all situations.

“Clarity is therefore required to make sure drivers understand the capability and performance of current assisted systems. It’s crucial today’s technology is adopted safely before we take the next step on the road to automation. There are safety and insurance implications that must be considered seriously.”

Thatcham and Euro NCAP grade “Vehicle Assistance,” “Driver Engagement” and “Safety Back-up” on what appeared to be a 100-point scale. They then add the lesser of the assistance or engagement score to the backup score. Anybody above 160 points earned the top “very good” rating, and 141-160 points merited a “good.” A score between 121 and 140 was merely “moderate.” The lowest score was “entry” for cars scoring between 101 and 120.

“Vehicle Assistance” tests how well the car drives itself. “Driver Engagement” studies factors like how well the customer is informed about the system and the quality of the vehicle’s driver monitoring. (Some OEMs, for example, will actually point a camera at the motorist and have the AI track their attentiveness.)  “Safety Back-up” examines what the car will do in an emergency situation like an unresponsive driver or imminent crash.

“The best systems strike a good balance between the amount of assistance they give to the driver and how much they do to ensure drivers are engaged and aware of their responsibilities behind the wheel,” Avery said in a statement.

Thatcham’s first round of testing revealed some fascinating contrasts within individual vehicles.

The Tesla Model 3 beat everyone in the assistance and backup categories with scores above 90 — but lost to everyone on the engagement front with a 36.

“The first batch of results show some car makers have developed robust assisted driving systems and that’s good to see. But there are also significant gaps in capability on other vehicles,” Avery said in a statement.

“For instance, the Tesla Model 3 was the best for vehicle assistance and safety back-up. But lost ground for over selling what its ‘Autopilot’ system is capable of, while actively discouraging drivers from engaging when behind the wheel.

“Tesla should however be recognised for its ability to update vehicles ‘Over the Air’. Two years ago, it’s safety back-up results would not have been market leading. This unique capability has seen it move the safety game on, across its whole fleet of vehicles.”

The Nissan Juke did pretty well with engagement and backup, scoring in the 70s, but the vehicle’s actual driving only scored a 52.

“Another small SUV with quite impressive performance,” Avery said in a statement. “The ProPilot name is not ideal, but it still has good driver engagement and safety back-up systems for the price-point.”

The Volvo V60, meanwhile, hit the 70s in assistance and engagement but only took home a 49 in backup performance.

“A high level of vehicle assistance and good, well-balanced driver engagement,” Avery said in a statement. It’s a shame, but the vehicle platform and technology have aged quickly and are no longer state of the art.”

Only the Mercedes GLE, BMW 3 Series and Audi Q8 did well enough across the board to earn a “Very good.”

Avery called the GLE the “overall top scorer with consistently high scores across all testing categories. Keeps the driver engaged with plenty of clear communication regarding the assistance offered. Provides really useful assistance, but not so much that drivers will believe the car can drive itself.”

More information:

“​World’s first Assisted Driving Grading unveiled”

Thatcham, Oct. 1, 2020

Thatcham assisted driving grading webpage

Images:

Thatcham tested and scored aspects of the “assisted driving” featured on the Mercedes GLE. (Provided by Thatcham)

Thatcham tested and scored aspects of the “assisted driving” experience on multiple European vehicles. (Provided by Thatcham)

Thatcham tested and scored aspects of the “assisted driving” experience on multiple European vehicles. A Mercedes GLE is at left, and a Tesla Model 3 is second from right. (Provided by Thatcham)