US 12,169,782 B2
Dynamic precision scaling at epoch granularity in neural networks
Shomit N. Das, Austin, TX (US); and Abhinav Vishnu, Austin, TX (US)
Assigned to Advanced Micro Devices, Inc., Santa Clara, CA (US)
Filed by ADVANCED MICRO DEVICES, INC., Santa Clara, CA (US)
Filed on May 29, 2019, as Appl. No. 16/425,403.
Claims priority of provisional application 62/758,853, filed on Nov. 12, 2018.
Prior Publication US 2020/0151573 A1, May 14, 2020
Int. Cl. G06N 3/084 (2023.01); G06N 3/04 (2023.01)
CPC G06N 3/084 (2013.01) [G06N 3/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
determining losses of samples within an input volume that is provided to a neural network executed by a processor, during a first epoch, the losses being based on a comparison of output values of the neural network to labeled output values in a known training data set;
grouping, by the processor, the samples into subsets based on the losses;
assigning, by the processor, the subsets to operands in the neural network that represent the samples at different precisions that correspond to each of the subsets associated with a different precision; and
training the neural network by processing, by the processor, the subsets in the neural network at the different precisions during the first epoch.