US 12,169,964 B2
System and method for providing weakly-supervised online action segmentation
Reza Ghoddoosian, Milpitas, CA (US); Isht Dwivedi, Mountain View, CA (US); Nakul Agarwal, San Jose, CA (US); Chiho Choi, San Jose, CA (US); and Behzad Dariush, San Ramon, CA (US)
Assigned to HONDA MOTOR CO., LTD., Tokyo (JP)
Filed by Honda Motor Co., Ltd., Tokyo (JP)
Filed on Feb. 1, 2022, as Appl. No. 17/590,379.
Claims priority of provisional application 63/278,001, filed on Nov. 10, 2021.
Prior Publication US 2023/0141037 A1, May 11, 2023
Int. Cl. G06V 10/778 (2022.01); G06V 10/26 (2022.01); G06V 10/82 (2022.01); G06V 20/40 (2022.01); G06V 40/20 (2022.01)
CPC G06V 10/7792 (2022.01) [G06V 10/26 (2022.01); G06V 10/82 (2022.01); G06V 20/49 (2022.01); G06V 40/20 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for providing weakly-supervised online action segmentation comprising:
receiving image data associated with multi-view videos of a procedure, wherein the procedure involves a plurality of atomic actions;
analyzing the image data using weakly-supervised action segmentation to identify each of the plurality of atomic actions by using an ordered sequence of action labels;
training a neural network with data pertaining to the plurality of atomic actions based on the weakly-supervised action segmentation; and
executing online action segmentation to label atomic actions that are occurring in real-time based on the plurality of atomic actions trained to the neural network, wherein at least one computing system is controlled to provide automation or feedback with respect to real-time atomic actions involved in completing the procedure based on the online action segmentation.