US 12,169,944 B2
Image reconstruction for virtual 3D
James A. McCombe, San Francisco, CA (US); and Christoph Birkhold, San Francisco, CA (US)
Assigned to MINE ONE GMBH, Berlin (DE)
Filed by MINE ONE GmbH, Berlin (DE)
Filed on Nov. 29, 2021, as Appl. No. 17/537,286.
Application 17/537,286 is a continuation of application No. 16/487,712, granted, now 11,189,043, previously published as PCT/US2018/019243, filed on Feb. 22, 2018.
Application 16/487,712 is a continuation in part of application No. PCT/US2016/023433, filed on Mar. 21, 2016.
Application 17/537,286 is a continuation in part of application No. 17/460,246, filed on Aug. 29, 2021, granted, now 11,960,639.
Application 17/460,246 is a continuation of application No. 16/749,989, filed on Jan. 22, 2020, granted, now 11,106,275, issued on Aug. 31, 2021.
Application 16/749,989 is a continuation of application No. 15/560,019, granted, now 10,551,913, issued on Feb. 4, 2020, previously published as PCT/US2016/023433, filed on Mar. 21, 2016.
Claims priority of provisional application 62/462,307, filed on Feb. 22, 2017.
Claims priority of provisional application 62/136,494, filed on Mar. 21, 2015.
Prior Publication US 2022/0222842 A1, Jul. 14, 2022
Int. Cl. G06K 9/00 (2022.01); G06T 7/593 (2017.01); G06T 7/73 (2017.01); G06T 7/80 (2017.01); H04N 13/128 (2018.01); H04N 13/194 (2018.01); H04N 13/00 (2018.01)
CPC G06T 7/593 (2017.01) [G06T 7/73 (2017.01); G06T 7/85 (2017.01); H04N 13/128 (2018.05); H04N 13/194 (2018.05); G06T 2207/10024 (2013.01); H04N 2013/0081 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A mesh-based image reconstruction method adapted for reconstructing a synthetic image of a scene from a perspective of a virtual camera having a selected virtual camera position, based on images of the scene captured by at least first and second physical cameras each having a respective physical view of the scene, the method comprising:
obtaining a native disparity map for the first physical camera having a physical view of a scene, the native disparity map containing disparity values stored therein, the native disparity map representing a computational solution to a stereo correspondence problem between the at least first and second physical cameras in a shared epipolar plane; and
utilizing at least the native disparity map for the first physical camera, executing the following:
(A) generating vertex values to form a vertex array, wherein each vertex value has (a) an associated coordinate within the native disparity map for the first physical camera, and (b) a disparity value, wherein the disparity value can be used to re-project the respective vertex value to a new location based on a selected projection function;
(B) executing a selected transformation of the vertex values;
(C) generating per-fragment disparity reference values and two dimensional image space (U, V) coordinate values corresponding to the first physical camera;
(D) generating U, V coordinates for an image obtained by the second physical camera;
(E) executing a weighted summation of samples from camera images captured by each of the first and second physical cameras; and
(F) generating an input to a rasterization depth test;
thereby reconstructing a synthetic image from the perspective of the virtual camera.