TY - JOUR
T1 - Stochastic Rounding: Implementation, Error Analysis, and Applications
AU - Croci, Matteo
AU - Fasi, Massimiliano
AU - Higham, Nicholas
AU - Mary, Theo
AU - Mikaitis, Mantas
PY - 2022/2/4
Y1 - 2022/2/4
N2 - Stochastic rounding randomly maps a real number x to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance to x. This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of length n in floating-point arithmetic, it yields an error bound with constant √nu with high probability, where u is the unit roundoff. This is not necessarily the case for round to nearest, for which the worstcase error bound has constant nu. A particular attraction of stochastic rounding is that, unlike round to nearest, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. We survey stochastic rounding by discussing its mathematical properties and probabilistic error analysis, its implementation, and its use in applications, with a focus on machine learning and the numerical solution of differentialequations.
AB - Stochastic rounding randomly maps a real number x to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance to x. This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of length n in floating-point arithmetic, it yields an error bound with constant √nu with high probability, where u is the unit roundoff. This is not necessarily the case for round to nearest, for which the worstcase error bound has constant nu. A particular attraction of stochastic rounding is that, unlike round to nearest, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. We survey stochastic rounding by discussing its mathematical properties and probabilistic error analysis, its implementation, and its use in applications, with a focus on machine learning and the numerical solution of differentialequations.
M3 - Article
SN - 2054-5703
JO - Royal Society Open Science
JF - Royal Society Open Science
ER -