US 12,169,794 B1
Modeling of information technology failures of enterprise computing systems
Michael Goodwin, Denver, CO (US); Stacy R. Henryson, Clive, IA (US); Brian Karp, San Francisco, CA (US); Manoranjan Kumar, San Francisco, CA (US); and Monte Nash, San Francisco, CA (US)
Assigned to Wells Fargo Bank, N.A., San Francisco, CA (US)
Filed by Wells Fargo Bank, N.A., San Francisco, CA (US)
Filed on Dec. 11, 2020, as Appl. No. 17/119,725.
Claims priority of provisional application 63/052,832, filed on Jul. 16, 2020.
Int. Cl. G06N 7/01 (2023.01); G06F 16/25 (2019.01); G06N 20/00 (2019.01); G06F 3/04842 (2022.01)
CPC G06N 7/01 (2023.01) [G06F 16/252 (2019.01); G06N 20/00 (2019.01); G06F 3/04842 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
retrieving an application dataset comprising application data for a plurality of applications in a first application set;
extracting a first feature set comprising features for each of the applications in the first application set;
identifying a feature subset comprising one or more of the features that are independent variables indicative of high severity events, wherein the independent variables include a number of changes feature, and wherein the number of changes feature comprises a number of changes affecting each of the applications in the first application set and a number of changes affecting one or more downstream and upstream applications;
determining one or more parameters for each of the independent variables in the feature subset;
generating a first training dataset comprising the first application set and, for each corresponding application in the first application set, the one or more parameters determined for each independent variable in the feature subset, wherein generating the first training dataset comprises random undersampling in which majority class applications that are not associated with the high severity events are randomly sampled and concatenated with minority class applications that are associated with the high severity events, the majority class applications amounting to a greater number of applications than the minority class applications;
training a first predictive machine learning model based on the first training dataset by applying one or more supervised learning techniques such that the first predictive machine learning model is tuned to receive, as input, application features for an application and provide, as output, a probability of a high severity event for the application;
determining the probability of the high severity event for each application in a second application set based on one or more extracted features corresponding to each application in the second application set, wherein determining the probability of the high severity event for each application in the second application set comprises feeding the extracted features to the first predictive machine learning model and obtaining the corresponding probability of the high severity event for the application;
retraining the first predictive machine learning model using a second training dataset to obtain a second predictive machine learning model, wherein the second training dataset is generated based on actual outcomes for the second application set as compared with predicted probabilities for applications in the second application set;
determining the probability of the high severity event for each application in a third application set based on one or more extracted features corresponding to each application in the third application set, wherein determining the probability of the high severity event for each application in the third application set comprises feeding the extracted features to the second predictive machine learning model and obtaining the corresponding probability of the high severity event for the application;
storing and displaying, in a graphical user interface enabling selection of one or more ranking criteria for a prediction report, the probability associated with each application in the third application set; and
mitigating one or more applications in the third application set based on the prediction report.