US 12,170,910 B2
Cross reality system with wireless fingerprints
Gilboa Shveki, Shoham (IL); Ben Weisbih, Rehovot (IL); Ofer Kapota, Plantation, FL (US); Rafael Domingos Torres, Boca Raton, FL (US); Daniel Olshansky, Mountain View, CA (US); and Joel David Holder, Austin, TX (US)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Mar. 6, 2023, as Appl. No. 18/179,275.
Application 18/179,275 is a continuation of application No. 17/071,879, filed on Oct. 15, 2020.
Claims priority of provisional application 62/915,559, filed on Oct. 15, 2019.
Prior Publication US 2023/0209373 A1, Jun. 29, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. H04W 24/02 (2009.01); G01S 5/02 (2010.01); G06T 19/00 (2011.01); H04W 4/02 (2018.01); H04W 4/029 (2018.01)
CPC H04W 24/02 (2013.01) [G01S 5/0252 (2013.01); G06T 19/006 (2013.01); H04W 4/029 (2018.02); H04W 4/02 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A portable device configured to operate within a three-dimensional (3D) environment and display virtual content of a cross reality system, the portable device comprising:
at least one processor;
an image sensor configured to output image data; and
computer executable instructions configured to, when executed by the at least one processor, perform a method comprising:
extracting a plurality of features from the image data;
forming a map of the 3D environment based on the image data produced by the image sensor as the portable device moves in the 3D environment, the map of the 3D environment
comprising a first coordinate frame and a plurality of persistent poses associated with feature descriptors of features of the plurality of features extracted from the image data and associated with locations within the map of the 3D environment;
maintaining a wireless fingerprint associated with the map of the 3D environment by repeatedly:
obtaining network access point information from network access points sending wireless signals received by the portable device at a location within the 3D environment;
selecting a persistent pose of the plurality of persistent poses at a location within the map of the 3D environment based on correspondence between features of the plurality of features and features associated with the persistent pose; and
based on selecting the persistent pose, updating a wireless fingerprint stored in association with the location within the map of the 3D environment by merging network access point information obtained at a second time with network access point information obtained at a first time and linked with the selected pose to produce a merged wireless fingerprint linked with the selected persistent pose;
for a set of features of the plurality of features associated with a position of the portable device, determining the location within the map associated with the features and sending to a localization service the merged wireless fingerprint stored in association with the determined location within the map and a plurality of feature descriptors associated with features of the set of features; and
based on sending the merged wireless fingerprint associated with the determined location and the plurality of feature descriptors to the localization service, receiving, from the localization service, a coordinate transformation between the first coordinate frame of the map of the 3D environment and a coordinate frame of a stored map.