In order to demonstrate the use of the derivative method for computing
the RGCV criterion, we will derive the update for for the
steepest-descent algorithm. We have presented this method in a previous paper,
but we did not present the vigorous justification at that time (Reeves and Perry
1992).
The basic form of the steepest-descent method is
Note that , the relaxation parameter, can be a fixed scalar, or
it can be computed as a scalar function of the current estimate
.
In any case, it is considered to be a constant with respect to
.
The first step is to take the derivative of with respect
to
. The result is
Then, multiplying both sides by and taking the summation over
you have
Finally, making the substitution for the definition of , we get
which is the steepest-descent algorithm operating on , but using the
same
that was used for
.