US 12,169,975 B2
Scene reconstruction in three-dimensions from two-dimensional images
Riza Alp Guler, London (GB); Georgios Papandreou, London (GB); and Iason Kokkinos, London (GB)
Assigned to SNAP INC., Santa Monica, CA (US)
Appl. No. 17/596,697
Filed by Snap Inc., Santa Monica, CA (US)
PCT Filed Jun. 17, 2020, PCT No. PCT/EP2020/066831
§ 371(c)(1), (2) Date Dec. 16, 2021,
PCT Pub. No. WO2020/254448, PCT Pub. Date Dec. 24, 2020.
Claims priority of provisional application 62/862,139, filed on Jun. 17, 2019.
Prior Publication US 2022/0358770 A1, Nov. 10, 2022
Int. Cl. G06V 20/64 (2022.01); G06T 7/73 (2017.01); G06V 10/80 (2022.01); G06V 10/82 (2022.01)
CPC G06V 20/647 (2022.01) [G06T 7/73 (2017.01); G06V 10/806 (2022.01); G06V 10/82 (2022.01); G06T 2207/20084 (2013.01); G06V 2201/07 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A method for creating a three-dimensional reconstruction of a scene with multiple objects from a single two-dimensional image, the method comprising:
receiving the single two-dimensional image comprising a representation of the scene with the multiple objects;
identifying the multiple objects in the single two-dimensional image to be reconstructed and identifying types of the multiple objects;
estimating a three-dimensional representation of each identified object using a deep machine learning model;
estimating a three-dimensional plane physically supporting the multiple objects based on three-dimensional positions of the three-dimensional representation;
reconstructing the three-dimensional plane that passes through the three-dimensional positions;
fitting the three-dimensional plane in the single two-dimensional image;
measuring an error value based on comparing the three-dimensional plane with a two-dimensional plane shown in the single two-dimensional image;
adjusting the three-dimensional plane based on the error value, wherein the fitting, measuring, and adjusting are repeated iteratively until a threshold condition is satisfied; and
displaying, on a display of a computing device, the three-dimensional representation of the each identified object relative to the two-dimensional plane.