NODAR patents self-calibrating, multi-camera ADAS technology
By onAnnouncements | Technology
Boston tech startup NODAR has been awarded a patent for multi-camera 3D vision technology for autonomous driving that it claims will be self-calibrating, even after a minor collision, and much less expensive than comparable lidar technology.
NODAR was awarded U.S. Patent No. 11,282,234 for its “Non-Rigid Stereo Vision Camera System,” which uses high-performance computer vision software to maintain alignment between multiple cameras mounted independently on the vehicle.
Leaf Jiang
Using industry standard hardware, this technology “make[s] self-driving far safer for Advanced Driver Assistance Systems (ADAS) in SAE L2+ and L3 passenger vehicles, as well as L4 autonomous trucks and robotaxis,” the company said in a news release.
NODAR believes that the technology “will accelerate the adoption of autonomous vehicles, and specifically that will make higher levels of autonomy, Level 2 or above, accessible to vehicles at all price points,” company founder and CEO Leaf Jiang said in a video. The company put the cost at a tenth of current lidar units, which are currently considered a necessary component of Level 3 and up systems.
Brad Rosen
Brad Rosen, chief operating officer and co-founder of NODAR, told RDN that the company is “in talks with several tier 1 providers as well as global OEMs about integrating our technology.”
“Some 250 million partially or fully self-driving vehicles are slated for production this decade, all of which require reliable 3D information to make safe driving decisions. It’s important that 3D sensors work in real-time and with sufficient resolution to identify any hazardous obstacle, and also satisfy the pricing demands of this market,” Jiang said in announcing the patent.
NODAR said the system, called “Hammerhead 3D Vision,” can compare images from each camera and triangulate the distance to every pixel in the scene, in real-time, at a distance of up to 1,000 meters, or 3,280 feet.
“Not only is the range of the system unprecedented in the industry, but Hammerhead also provides ultra-precise range measurements and has been proven to detect obstacles as small as 10 centimeters at 150 meters from the car,” the company said. “Together, Hammerhead’s long-range and precise distance measurement capabilities provide the vehicle with the time and assurance to make safe routing decisions.”
The system can accommodate manufacturing tolerances as great as plus or minus 3 degrees, and actively compensates for agitations from road shock, vibration, and temperature variations.
“Factory calibration of the system is not needed, and manual calibration during regular operation is not needed, thus simplifying manufacturing of the system,” the company said in its patent application.
Rosen confirmed to RDN that “no calibration would be needed after a minor accident (cameras still need to be pointed generally in the same direction) or component replacement.”
“Our system recalibrates every video frame, and adjusts for vibrations and/or shocks such as those you describe,” he said.
With existing stereo camera systems, “individual camera modules may shift and/or rotate with respect to each other over time, which may make initial calibration parameters become inaccurate over time,” the patent states.
“It is desirable for the fields of view of the camera modules of a stereo camera system not to move relative to each other in order for the stereo camera system to remain properly calibrated. For some stereo camera systems, a relative camera orientation rotation of only 0.05 degrees can ruin the calibration of the system. Moreover, in a conventional stereo camera system, there is potential for movement of, e.g., lens holders relative to circuit boards and/or other camera components, for relative pointing between camera modules to change, and also for positions of the circuit boards and the frame itself to change over time.
“The multiple interfaces between components in a conventional stereo camera system make it likely that vibration, shock, and even thermal expansion between components will cause the field of view of the cameras in a stereo camera system to shift over time. The relative movement of the camera components may invalidate the initial factory calibration of the system, which may make stereo vision data obtained by the system unreliable.”
OEMs that use stereo camera systems have issued specific instructions on how and when they should be calibrated. Subaru, for instance, requires calibration of its EyeSight system if a camera is removed/installed or replaced, if suspension-related components are removed/installed or replaced, if wheel alignment is performed, and if the windshield is removed/installed or replaced.
NODAR’s technology appears to sidestep that issue, with a processor that continuously tracks the relative motion between two or more cameras and keeps them calibrated with respect to one another, even if the camera positions vibrate or shift rapidly or slowly.
Because they do not need to be connected by a rigid structure, the cameras can be mounted far apart – for instance, incorporated into the door mirrors, or the upper corners of the windshield. This improves accuracy and range, the company said.
This is not possible with current technology, because “these locations are not sufficiently rigid to maintain calibration over hours or days, much less a 15-year lifetime of a typical vehicle,” NODAR said.
“In some cases, it may not be realistically possible to add rigid mechanical support structures between the headlights, the upper corners of the windshield, and the side-view mirrors, because a shortest path for these support structures may be blocked by an engine block, may block a driver’s view, and may be blocked by the driver himself/herself, respectively,” the company said.
Even if that could be done, NODAR said, it would probably be impractical to add a beam thick enough to support the cameras. A beam’s deflection displacement scales as the cube of the beam’s length, which in practical terms means stereo cameras must be located no more than 20mm, or about 9 inches, apart with existing systems.
“The NODAR system marks a significant step toward safer autonomous driving,” board member Klaus Kompass, an automotive safety expert and former vice president of vehicle safety at BMW, said in a statement. “Extremely accurate long-range sensing enables AVs to detect obstacles that other systems would miss. Most importantly, it detects dangerous objects at such a distance that even trucks traveling at highway speeds would have plenty of time to avoid a collision. It’s an advance that could undoubtedly save lives.”
More information
NODAR Awarded Patent for Multi-Camera 3D Vision Technology
http://www.prweb.com/releases/2022/3/prweb18579315.htm
A patent for a non-rigid stereo vision camera system
Images
Featured image: The logo of NODAR’s Hammerhead 3D Vision system. (Provided by NODAR)
Other images provided by NODAR