A Newton-Raphson version of the multivariate Robbins-Monro procedure
D Ruppert - The Annals of Statistics, 1985 - projecteuclid.org
The Annals of Statistics, 1985•projecteuclid.org
Suppose that $ f $ is a function from $\mathbb {R}^ k $ to $\mathbb {R}^ k $ and for some
$\theta, f (\theta)= 0$. Initially $ f $ is unknown, but for any $ x $ in $\mathbb {R}^ k $ we can
observe a random vector $ Y (x) $ with expectation $ f (x) $. The unknown $\theta $ can be
estimated recursively by Blum's (1954) multivariate version of the Robbins-Monro
procedure. Blum's procedure requires the rather restrictive assumption that infimum of the
inner product $(x-\theta)^ tf (x) $ over any compact set not containing $\theta $ be positive …
$\theta, f (\theta)= 0$. Initially $ f $ is unknown, but for any $ x $ in $\mathbb {R}^ k $ we can
observe a random vector $ Y (x) $ with expectation $ f (x) $. The unknown $\theta $ can be
estimated recursively by Blum's (1954) multivariate version of the Robbins-Monro
procedure. Blum's procedure requires the rather restrictive assumption that infimum of the
inner product $(x-\theta)^ tf (x) $ over any compact set not containing $\theta $ be positive …
Suppose that is a function from to and for some . Initially is unknown, but for any in we can observe a random vector with expectation . The unknown can be estimated recursively by Blum's (1954) multivariate version of the Robbins-Monro procedure. Blum's procedure requires the rather restrictive assumption that infimum of the inner product over any compact set not containing be positive. Thus at each gives information about the direction towards . Blum's recursion is where the conditional expectation of given is and . Unlike Blum's method, the procedure introduced in this paper does not necessarily attempt to move in a direction that decreases , at least not during the initial stage of the procedure. Rather, except for random fluctuations it moves in a direction which decreases , and it may follow a circuitous route to . Consequently, it does not require that have a constant signum. This new procedure is somewhat similar to the multivariate Kiefer-Wolfowitz procedure applied to , but unlike the latter it converges to at rate . Deterministic root finding methods are briefly discussed. The method of this paper is a stochastic analog of the Newton-Raphson and Gauss-Newton techniques.
Project Euclid