The extrinsic calibration needed to marry feeds from three lidar sensors or a stereo camera system on Mississippi State University’s experimental autonomous Subaru Forester can be affected by vehicle disassembly or alterations, according to one of its engineers.
Will Meadows, an MSU electrical engineering masters student and the electrical lead for the vehicle, discussed the calibration process for the electric, self-driving off-road vehicle in an exclusive interview with Repairer Driven News on Oct. 29, the day before MSU unveiled the SUV in SCRS’ SEMA booth. Highlights from the conversation spanning this and numerous other interesting elements of the MSU Center for Advanced Vehicular Studies vehicle are now available in a Society of Collision Repair Specialists video posted Wednesday. (The calibration discussion starts at about the 10:23-minute mark.)
MSU retained the Subaru EyeSight system on the vehicle, which uses two cameras behind the windshield to provide a steroscopic view of the road for advanced driver assistance system features.
“We listen to it first,” Meadows said.
CAVS added two 190-degree cameras to the mirror — each can see “a little bit behind itself,” according to Meadows — and installed two more in the rear of the vehicle for another stereoscopic view of the terrain.
Meadows said MSU’s modified 2014 Forester also uses two lidar sensors angled toward the ground and combining their output with a third above the car for a single, wider sensory view. The lidar sees the world by reflecting lasers off of objects, which Meadows said provided CAVS with a convenient way to classify various potential obstacles based upon the intensity of their reflectivity.
Meadows said all the sensors are “very finely calibrated” with “very small differences between them.” CAVS uses intrinsic calibration to set unique views for each sensory device.
Intrinsic calibration is “pretty straightforward,” Meadows said. The sensor manufacturer provides software and a simplified way to handle it, he said.
Meadows described that calibration process for the camera on a side-view mirror using a black-and-white checkerboard target — something that ought to sound familiar to collision repairers. The software understands that it’s supposed to be looking at a square but that through the mirror-mounted fisheye lens, “it looks horribly distorted,” according to Meadows. The calibration is “definitely necessary” to get the visual feed converted into the square it’s actually perceiving, he said.
However, the stereo cameras and lidar are extrinsically calibrated to work together and deliver a single image, according to Meadows. For example, the view from the lidar atop the vehicle is paired with the two views from the side lidars to create a single, larger image showing a wider field of view than would be possible with a single camera, he said. Stereo cameras work like your eyes do — two visual feeds are merged into a single, wider field of view.
“In terms of service, that complicates things,” Meadows said of extrinsic calibration, calling it “a little bit more tricky.”
Intrinsic calibration on the MSU autonomous vehicle only has to be done once, Meadows said. But extrinsic calibration needs to be reapplied every time the car changes, he said.
Since every sensor or camera requiring extrinsic calibration is designed to work with another sensor, every time the device is moved or tilted too far, it “completely messes up our system,” Meadows said.
CAVS must recalibrate it extrinsically again or risk the consequences.
“I don’t know what would happen, but it probably wouldn’t be good,” Meadows said.
If CAVS plans on doing a lot of testing, it’s careful to rigidly mount sensors to minimize movement.
“Like I said, once they get knocked too out of whack … they’re not gonna perform like they should,” he said.
Calibration becomes a constant burden during the disassembly and alterations of the vehicle development stage, according to Meadows.
“We’re always tweaking things, changing things, taking apart the car … calibration is a huge time sink for us,” he said.
Meadows said a “fine-tooth comb” was necessary for many sensory systems, and such expertise would be required for servicing or rebuilding them.
Our interview discussed a heavily modified and experimental self-driving SUV; the capabilities and OEM requirements on the actual vehicles coming into your shop — including normal 2014 Subaru Foresters — could be far different. Nevertheless, the discussion provides good food for thought and perspective for shops as ADAS and the associated calibration spreads through the U.S. fleet.
Other video highlights
Here’s some of the other highlights from the video of the interview with Meadows and Amanda Meeler, CAVS marketing and communication coordinator, and the vehicle unveiling at the SCRS booth:
0:10: The vehicle unveiling by CAVS associate director Matthew Doude.
4:45: Converting the Forester to an electric offroad vehicle.
5:47: The powertrain conversion and why a “skateboard” electric vehicle design wouldn’t work.
7:51: Aluminum’s role in subframe lightweighting and cooling, as well as electric vehicle battery cooling in general.
9:37: The weight of the vehicle and suspension modifications to address it.
16:37: Use cases for an autonomous offroad vehicle.
18:47: Autonomous cars and the grade of the terrain.
20:55: Speed and autonomous travel.
22:00: Human backup drivers and switching vehicle control.
24:02: User and developer interfaces.
26:22: Power demands.
27:44: The difficulty of converting the Forester body to one for an electric car.
28:48: The vehicle interior.
32:38: The engine compartment battery pack.
35:30: The rear battery pack.
36:48: Range and battery chemistry, voltage and charging.
Society of Collision Repair Specialists YouTube Channel, Dec. 18, 2018
SCRS newsletter, Dec. 27, 2018
Mississippi State University Center for Advanced Vehicular Studies, Oct. 30, 2018
Lidar sensors can be found on the top and sides of the front bumper on the Mississippi State University Center for Advanced Vehicular Studies “Halo Project” autonomous 2014 Subaru Forester. (John Huetter/Repairer Driven News)
A fisheye camera on the side-view mirror of the Mississippi State University Center for Advanced Vehicular Studies “Halo Project” autonomous 2014 Subaru Forester. (John Huetter/Repairer Driven News)
Will Meadows, an MSU electrical engineering masters student and the electrical lead for the vehicle, is seen next to the Mississippi State University Center for Advanced Vehicular Studies “Halo Project” autonomous Subaru Forester. The engine compartment contains a battery pack with 15 modules. Other battery packs can be found in the transmission tunnel and rear of the vehicle. (John Huetter/Repairer Driven News)