Newton's method
Newton's method uses the derivative of a differentiable function to approximate its real or complex roots. For a function , the approximations are defined recursively by To begin the recursion, an initial guess must be chosen. Often the choice of determines which of several possible roots is found, and in some cases the initial guess can cause the recursion to cycle or diverge instead of converging to a root.
Contents
Derivation
At each step of the recursion, we have and seek a root of . Since all nonconstant linear functions have exactly one root, as long as we can construct a tangent-line approximation to and find its root as an approximation. The tangent-line approximation of about is We seek the value such that the above expression equals ; hence, Dividing by (as long as ), Therefore, the desired value of is
Worked example
Problem: Find the values of the roots of .
Solution: We could use the quadratic formula to find that the roots are , but this approach does not immediately yield a decimal value.
To form a guess, we note that is slightly larger than , so a suitable guess for the greater root is .
The derivative of is by the power rule for derivatives. Therefore, the recursive formula is
Starting at , we apply this formula repeatedly:
Because as calculated, the ratio must be very close to . Since is not too large, must be quite close to , so is a very good estimate of the greater root.
The sum of the roots is by Vieta's formulas, so the lesser root is simply .
Failure cases
Although powerful, Newton's method is delicate and can be very sensitive to the initial guess and type of function, even when the function is differentiable everywhere.
Zero derivative
Suppose in the above example of finding a root of , we had started with . In the process of calculating , then, we would have to divide by , creating an undefined result.
Wrong root
Suppose we try to estimate the greater root as before, but start with . Applying Newton's method to yields the following sequence of estimates:
These estimates converge to the lesser root, in this case because the initial estimate was closer to the lesser root than the greater root. In general, predicting which root Newton's method will finally converge to is difficult, and Newton's method may converge to a farther root even if a closer root is available within the interval between the initial estimate and the root.
Periodic behavior without finding a root
When finding roots of as above, the estimates eventually stabilize into a predictable pattern: in the case, , so every subsequent estimate will equal ; in the case, , so the subsequent estimates will oscillate, with every even estimate equal to and every odd estimate equal to . These behaviors are due to the limited precision of the calculations; with indefinite precision, the estimates would continue to approach the root without becoming periodic.
However, for some functions, Newton's method can cycle without coming close to a root.
Divergence
The function is always positive, so it has no roots. Since satisfies the differential equation for all , the ratio is always , so the sequence of estimates produced by Newton's method becomes , diverging in the negative direction.
Applications in Analytic Number Theory (Advanced)
The applications of Newton's Method actually apply to Hensel's Lemma, specifically in the adic integers . In particular, take some polynomial and an integer such that . Our goal is to approximate some such that . However, in the adics, we can't just ``lift" arbitrarily. We actually need to approximate such a solution in the next power level, and then refine it iteratively.
Here's what that looks like realistically. WLOG we assume that is some between and . We search for such that . Note the following critical step:
for some . By assumption we know that so let for some . The desired congruence holds modulo when . When , we see that which means that
This is exactly the statement of the Newtonian Algorithm! Our root is approximated, so we found some such that . But this isn't the whole story.
Theorem: Let and such that . If then there exists an such that (1) , (2) and (3) .
Proof: Let for and where . By definition we see that
This proves (2). Notice, however, that in the Taylor expansion of about we see that
where . Also notice that
but we finish (1) since . To prove (3) we just compute the order of . Taking the Taylor expansion of at point gives
But since and we have
which proves that
This sets up the entire motivation behind Hensel's Lemma in a more formal way. In addition, the proof of the lemma is just an extension of what we previously noted, which actually makes all of our previous work worth it in the long run.
Hensel's Lemma: Assume and satisfies . If , then there exists a unique root of such that and .
Proof: We first prove the existence of . Let be given. We can construct such that We can do this for and such that we have a sequence where as where and . Now we prove that is unique. For the sake of contradiction, suppose that there are two roots and satisfying
Since it was given that we have . This means that so we have at the very least
Now we have,
for some . Since and were assumed to be roots, we factor to get
but since and
which means that , so uniqueness holds