US 12,169,916 B2
System and method for image inpainting based on layout-guided pre-processing attention mechanism
Wenbo Li, Santa Clara, CA (US); and Hongxia Jin, San Jose, CA (US)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Oct. 15, 2021, as Appl. No. 17/503,169.
Claims priority of provisional application 63/115,428, filed on Nov. 18, 2020.
Prior Publication US 2022/0156896 A1, May 19, 2022
Int. Cl. G06T 5/77 (2024.01); G06T 5/50 (2006.01); G06T 5/75 (2024.01)
CPC G06T 5/77 (2024.01) [G06T 5/50 (2013.01); G06T 5/75 (2024.01)] 20 Claims
OG exemplary drawing
 
1. An electronic device comprising:
a display; and
at least one processor configured to:
obtain an image including an object having a delicate shape;
identify a target region within the image, the target region adjacent to the object;
use a mask to separate the image into multiple semantic categories each associated with a different type or class of image data captured in the image;
aggregate neighboring contexts for the target region based on the semantic categories;
restore, based on the aggregated contexts, textures in the target region without affecting the delicate shape of the object; and
cause the display to display a refined image including the restored textures in the target region and the object;
wherein, to aggregate the neighboring contexts for the target region, the at least one processor is configured to:
split the image into multiple split contexts based on at least two of the semantic categories;
split the mask into multiple split binary layout masks based on the at least two sematic categories; and
apply a Gaussian blur to the split contexts and the split binary layout masks using a two-dimensional (2D) Gaussian kernel configured to perform a 2D convolution.