Apple has filed a patent application for the creation of a camera and sensor system to detect and correct defects in windshields and windows to more accurately relay the surrounding environment of partially autonomous and autonomous vehicles (AVs).

The patent is thought to be part of Apple’s “Project Titan,” the company’s rumored AV/EV project slated to begin production by 2025, though the patent doesn’t mention that project.

Apple also suggests in the application that its system could detect repair or replacement status for vehicle windshields and windows, shown in the diagram below.

“The control system… may use the parameter evaluation data to make the decision of whether to designate a repair status and/or a replace status to the window. Designation of a repair status may indicate a suggestion to repair at least a portion of the window. Likewise, designation of a replace status may indicate a suggestion to replace at least a portion of the window. Such suggestions may be provided via an audible and/or a visible output.”

The patent notes that AV and self-driving technologies are increasingly being researched and developed but “such vehicles typically include windows that interfere with the extent to which they are capable of sensing accurate representations of their environment.” To remedy that, Apple’s technology would use a camera and image processing “to produce a corrected image of a scene based at least partly on data corresponding to the detected defects.”

“For instance, a camera may be used to capture images to obtain information about an environment,” the patent application states. “Sometimes obstructions interfere with imaging. As an example, a dirty window may be located within a field of view of the camera and may negatively impact images captured by the camera of a scene.”

Shadowing, scattering, distortion, and glint are also listed as possible “image-altering effects.” The system, in some cases, may also include one or multiple sensor devices including radar and/or lidar, lighting modules, and/or processors.

“The lighting module(s) may be configured to illuminate the window to facilitate detection of the defects associated with the window,” the application states. “For instance, illumination of the window by the lighting module(s) may cause the defects to glow or otherwise act as secondary light sources, thereby making the defects easier to detect by a sensor device.

“…In some examples, the processor(s) may be configured to receive signals corresponding to images captured by the sensor devices, at least one of which may include an altered representation of the scene based on the image altering effects induced by the defects associated with the window. Furthermore, the processor(s) may perform image processing to compensate for the image altering effects induced by the defects and produce a corrected image of the scene.”

An individual sensor may also be configured with adaptive focus functionality, allowing it to switch between focusing on the window and on the scene, according to the application.

“In some examples, the operations may include determining to modify a state of operation of the vehicle based at least in part on the parameter evaluation data. In some embodiments, the operations may include deconvolving the second data to produce a corrected image of the scene. For instance, the second data (which may include the altered representation of the scene) may be deconvolved based at least in part on the first data (which may include the representation of the defects). To deconvolve the second data, the processor(s) may perform image processing to compensate for the image altering effects induced by the defects.”

IMAGES

Featured image credit: Bangyu Wang/Unsplash

All diagrams from Apple’s patent application filed with the U.S. Patent and Trademark Office.