CPC G06F 9/451 (2018.02) [G06F 21/554 (2013.01); G06F 21/645 (2013.01); G11B 27/34 (2013.01); H04N 5/76 (2013.01); G06F 2221/034 (2013.01)] | 16 Claims |
1. A method for simultaneously recording video and capturing structural data to provide a view of user interface elements exposed on a user's screen of a user device, the method comprising:
recording a video of pixels of the user's screen on a user device and entering a recording state;
marking the video with a timestamp with the video start time;
simultaneously capturing at least one snapshot of hierarchical data structure of user interface elements via an accessibility application programming interface during the recording of the video pixels in the recording state;
marking the at least one snapshot with a timestamp;
continuously polling the accessibility application programming interface, wherein the at least one snapshot is queried recursively starting at an exposed user interface element, wherein the exposed user interface element has zero or more child user interface elements, providing a hierarchy of user interface elements and a data structure of the exposed user interface at a moment in time;
comparing the data structure to a previously stored data structure and storing the data structure when there is a change from the previously stored data structure;
simultaneously recording the video and the at least one snapshot of hierarchical data structure of user interface elements until the user selects to stop recording;
exiting the recording state;
storing the video and the at least one snapshot of hierarchical data structure of user interface elements as a session; and
synchronizing the recorded video and the at least one snapshot of hierarchical data structure of user interface elements for playback.
|