US 12,169,949 B2
Camera to camera calibration
Luca del Pero, London (GB); and Karim Tarek Mahmoud Elsayed Ahmed Shaban, London (GB)
Assigned to Lyft, Inc., San Francisco, CA (US)
Filed by Lyft, Inc., San Francisco, CA (US)
Filed on Dec. 23, 2020, as Appl. No. 17/133,041.
Prior Publication US 2022/0198714 A1, Jun. 23, 2022
Int. Cl. G06T 7/80 (2017.01); B60R 1/23 (2022.01); G06T 7/73 (2017.01); G06V 20/56 (2022.01); H04W 4/46 (2018.01)
CPC G06T 7/85 (2017.01) [B60R 1/23 (2022.01); G06T 7/73 (2017.01); G06V 20/56 (2022.01); H04W 4/46 (2018.02); B60R 2300/105 (2013.01); B60R 2300/402 (2013.01); G06T 2207/30248 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
obtaining first image data captured by a first camera of a vehicle during a given period of operation of the vehicle;
obtaining second image data captured by a second camera of the vehicle during the given period of operation of the vehicle; and
applying optimization to determine a combination of an extrinsics transformation and a time offset that jointly minimizes an aggregated reprojection error determined based on the first and second image data, wherein the optimization involves iteratively:
determining a combination of a respective candidate extrinsics transformation and a respective candidate time offset for which to determine an aggregated reprojection error;
determining a respective reprojection error for each respective first image in a given set of first images from the first image data by (i) identifying a corresponding pair of second images from the second image data that bound the respective first image, (ii) determining a candidate pose for the first camera based on the respective candidate extrinsics transformation, the respective candidate time offset, and estimated poses of the second camera associated with the corresponding pair of second images, and (iii) determining the respective reprojection error for the respective first image based on at least one landmark that is reprojected from at least one of the corresponding pair of second images into the respective first image using the candidate pose for the first camera; and
based on the respective reprojection errors that are determined for the respective first images in the given set of first images, determining the aggregated reprojection error produced by the combination of the respective candidate extrinsics transformation and the respective time offset;
wherein the determined combination of the extrinsics transformation and the time offset is utilized to calibrate a first sensor system comprising the first camera with a second sensor system comprising the second camera.