US 12,169,789 B2
Selection of machine learning algorithms
Charles Parker, Corvallis, OR (US)
Assigned to BIGML, INC., Corvallis, OR (US)
Filed by BigML, Inc., Corvallis, OR (US)
Filed on Jan. 20, 2023, as Appl. No. 18/099,904.
Application 18/099,904 is a continuation of application No. 16/915,551, filed on Jun. 29, 2020, granted, now 11,586,953.
Application 16/915,551 is a continuation of application No. 15/478,097, filed on Apr. 3, 2017, abandoned.
Claims priority of provisional application 62/318,672, filed on Apr. 5, 2016.
Prior Publication US 2023/0153654 A1, May 18, 2023
Int. Cl. G06N 5/04 (2023.01); G06F 16/22 (2019.01); G06N 20/00 (2019.01); G06N 20/10 (2019.01); G06N 20/20 (2019.01); G06N 5/01 (2023.01); G06N 7/01 (2023.01)
CPC G06N 5/04 (2013.01) [G06F 16/22 (2019.01); G06N 20/00 (2019.01); G06N 20/20 (2019.01); G06N 5/01 (2023.01); G06N 7/01 (2023.01); G06N 20/10 (2019.01)] 3 Claims
OG exemplary drawing
 
1. A computing system comprising:
a prediction server to:
obtain an input dataset;
select a first algorithm from a set of algorithms;
apply the first algorithm to the input dataset to create a first model of the input dataset;
evaluate and control storage of results of applying the first algorithm to the input dataset, by:
controlling performance of a holdout procedure using the set of predictions; or
controlling performance of a two-fold cross-validation procedure using the set of predictions;
add the first algorithm into a set of tried algorithms;
for each algorithm in the set of algorithms other than the first algorithm and until a termination condition is met:
generate a submodular function, the submodular function configured to determine a likelihood that an optimum algorithm has already been tried;
select, via the submodular function, a next algorithm of the set of algorithms to apply to the input dataset based on accessed historical performance data in a benchmark database, and the set of tried algorithms,
evaluate and control storage of results of applying the next algorithm to the input dataset,
add the next algorithm to the set of tried algorithms; and
identify an optimum algorithm from the set of algorithms, wherein the optimum algorithm is an individual algorithm of the set of algorithms that is closer to fulfilling a predetermined criterion than other algorithms of the set of algorithms;
generate a report indicating the optimum algorithm; and
an application server to implement a user interface, the application server to obtain the report, and transmit the report to a user system.