Newton's Method

Let's suppose that we want to approximate the solution to $f(x)=0$ and let's also suppose that we have somehow found an initial approximation to this solution say, $x_0$. This initial approximation is probably not all that good, in fact it may be nothing more than a quick guess we made, and so we'd like to find a better approximation. This is easy enough to do. First, we will get the tangent line to $f(x)$ at $x_0$.

Now look at the graph below.

The blue line (if you’re reading this in color anyway…) is the tangent line at $x_0$. We can see that this line will cross the $x$-axis much closer to the actual solution to the equation than $x_0$ does (since the tangent line, or linear approximation, is a good estimate for $f(x)$ near $x=x_0$). Let’s call this point where the tangent at $x_0$ crosses the $x$-axis $x_1$ and we'll use this point as our new approximation to the solution.

So, how do we find this point? Well we know it’s coordinates, $(x_1,0)$, and we know that it's on the tangent line so plug this point into the tangent line and solve for $x_1$ as follows,

So, we can find the new approximation provided the derivative isn’t zero at the original approximation. But this $x_1$ is also (likely) still just an approximation of where $f(x)=0$. So we may want an even better approximation.

Now we repeat the whole process to find an even better approximation. We form up the tangent line to $f(x)$ at $x_1$ and use its root, which we’ll call $x_2$, as a new approximation to the actual solution. If we do this we will arrive at the following formula.

This point is also shown on the graph above and we can see from this graph that if we continue following this process will get a sequence of numbers that are getting very close the actual solution. This process is called Newton’s Method.

To experience this process in action, check out this interactive example. Click on the checkboxes to see each step in the iterative process.

Okay, let's work through and example using Newton's Method.

In this last example we saw that we didn’t have to do too many computations in order for Newton's Method to give us an approximation in the desired range of accuracy. This will not always be the case. Sometimes it will take many iterations through the process to get to the desired accuracy and on occasion it can fail completely.

See if you can work out some on your own.

The following example is a little silly but it makes the point about the method failing.

So, we need to be a little careful with Newton's method. It will usually quickly find an approximation to an equation. However, there are times when it will take a lot of work or when it won't work at all.

Practice Problems

For problems 1 & 2 use Newton's Method to determine $x_2$ for the given function and given value of $x_0$.

1. $f(x)=x^3-7x^2+8x-3$, $x_0=5$
   Show Solution

   We know that the basic formula for Newton's Method is,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   so all we need to do is run through this twice. Here is the derivative of the function since we'll need that.

   $$f'(x)=3x^2-14x+8$$

   We just now need to run through the formula above twice. The first iteration through the formula for $x_1$ is,

   $${x_{\,1}} = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 5 - \frac{{f\left( 5 \right)}}{{f'\left( 5 \right)}} = 5 - \frac{{ - 13}}{{13}} = 6$$

   The second iteration through the formula for $x_2$ is,

   $${x_{\,2}} = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 6 - \frac{{f\left( 6 \right)}}{{f'\left( 6 \right)}} = 6 - \frac{9}{{32}} = 5.71875$$

   So, the answer for this problem is $x_2=5.71875$.

   Although it was not asked for in the problem statement the actual root is $5.68577952608963$. Note as well that this did require some computational aid to get and it not something that you can, in general, get by hand.

2. $f(x)=x\cos(x)-x^2$, $x_0=1$
   Show Solution

   We know that the basic formula for Newton's Method is,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   so all we need to do is run through this twice. Here is the derivative of the function since we'll need that.

   $$f'(x)=\cos(x)-x\sin(x)-2x$$

   We just now need to run through the formula above twice. The first iteration through the formula for $x_1$ is,

   $${x_{\,1}} = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 1 - \frac{{f\left( 1 \right)}}{{f'\left( 1 \right)}} = 1 - \frac{{ - 0.4596976941}}{{ - 2.301168679}} = 0.8002329432$$

   Don't forget that for us angles are always in radians so make sure your calculator is set to compute in radians.

   The second iteration through the formula for $x_2$ is,

   $$\begin{align*} {x_{\,2}} = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} & = 0.8002329432 - \frac{{f\left( {0.8002329432} \right)}}{{f'\left( {0.8002329432} \right)}}\\ & = 0.8002329432 - \frac{{ - 0.08297883948}}{{ - 1.478108132}} = 0.7440943985 \end{align*}$$

   So, the answer for this problem is $x_2=0.7440943985$.

   Although it was not asked for in the problem statement the actual root is $0.739085133215161$. Note as well that this did require some computational aid to get and it not something that you can, in general, get by hand.

For problems 3 & 4 use Newton's Method to find the root of the given equation, accurate to six decimal places, that lies in the given interval.

3. $x^4-5x^3+9x+3$ in $[4,6]$
   Show Solution

   First, recall that Newton's Method solves equation in the form $f(x)=0$ and so it is (hopefully) fairly clear that we have,

   $$f(x)=x^4-5x^3+9x+3$$

   Next, we are not given a starting value, $x_0$, but we were given an interval in which the root exists so we may as well use the midpoint of this interval as our starting point or, $x_0=5$. Note that this is not the only value we could use and if you use a different one (which is perfectly acceptable) then your values will be different from those here.

   At this point all we need to do is run through Newton's Method,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   until the answers agree to six decimal places.

   The first iteration through the formula for $x_1$ is,

   $${x_{\,1}} = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 5 - \frac{{48}}{{134}} = 4.641791045$$

   We will need to continue this process until our $x_n$'s start agreeing to six decimal places. Here are those calculations.

   $${x_{\,2}} = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 4.641791045 - \frac{{8.950542057}}{{85.85891882}} = 4.537543959$$ $${x_{\,3}} = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = 4.537543959 - \frac{{0.6329967413}}{{73.85993168}} = 4.528973727$$ $${x_{\,4}} = {x_{\,3}} - \frac{{f\left( {{x_{\,3}}} \right)}}{{f'\left( {{x_{\,3}}} \right)}} = 4.528973727 - \frac{{0.004066133005}}{{72.91199944}} = 4.52891796$$ $${x_{\,5}} = {x_{\,4}} - \frac{{f\left( {{x_{\,4}}} \right)}}{{f'\left( {{x_{\,4}}} \right)}} = 4.52891796 - \frac{{1.714694911*{{10}^{ - 7}}}}{{72.90585006}} = 4.52891796$$

   At this point we are accurate to 8 decimal places which is actually better than we asked and so we can officially stop and we can estimate that the root in the interval is,

   $$x\approx4.52891796.$$

   Using computational aids we can find that the actual root in this interval is $4.52891795729$. Note that this wasn’t actually asked for in the problem and is only given for comparison purposes.

4. $2x^2+5=e^x$ in $[3,4]$
   Show Solution

   First, recall that Newton's Method solves equation in the form $f(x)=0$ and so we'll need move everything to one side. Doing this gives,

   $$f(x)=2x^2+5-e^x.$$

   Note that we could have just as easily gone the other direction. All that would have done was change the signs on the function and derivative evaluations in the work below. The final answers however would not be changed.

   Next, we are not given a starting value, $x_0$, but we were given an interval in which the root exists so we may as well use the midpoint of this interval as our starting point or, $x_0=3.5$. Note that this is not the only value we could use and if you use a different one (which is perfectly acceptable) then your values will be different that those here.

   At this point all we need to do is run through Newton's Method,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   until the answers agree to six decimal places.

   The first iteration through the formula for $x_1$ is,

   $${x_{\,1}} = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 3.5 - \frac{{ - 3.615451959}}{{ - 19.11545196}} = 3.310862334$$

   We will need to continue this process until our $x_n$'s start agreeing to six decimal places. Here are those calculations.

   $${x_{\,2}} = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 3.310862334 - \frac{{ - 0.4851319992}}{{ - 14.16530146}} = 3.276614422$$ $${x_{\,3}} = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = 3.276614422 - \frac{{ - 0.0135463486}}{{ - 13.37949281}} = 3.275601951$$ $${x_{\,4}} = {x_{\,3}} - \frac{{f\left( {{x_{\,3}}} \right)}}{{f'\left( {{x_{\,3}}} \right)}} = 3.275601951 - \frac{{ - 0.00001152056596}}{{ - 13.356740003}} = 3.275601089$$

   At this point we are accurate to 6 decimal places which is what we were asked to do and so we can officially stop and we can estimate that the root in the interval is,

   $$x\approx3.275601089$$

   Using computational aids we found that the actual root in this interval is $3.27560108884732$. Note that this wasn't actually asked for in the problem and is only given for comparison purposes and it does look like Newton's Method did a pretty good job as this is identical to the final iteration that we did.

For problems 5 & 6 use Newton's Method to find all the roots of the given equation accurate to six decimal places.

5. $x^3-x^2-15x+1=0$
   Show Solution

   First, recall that Newton’s Method solves equation in the form $f(x)=0$ and so it is (hopefully) fairly clear that we have,

   $$f(x)=x^3-x^2-15x+1.$$

   Next, we are not given a starting value, $x_0$ and unlike Problems 3 & 4 above we are not even given an interval to use as a way to determine a good possible value of $x_0$. We are also not even told how many roots we need to find.

   Of course, if we recall our algebra skills we can see that we have a cubic polynomial and so there should be at most three distinct roots of the equation (there may be some that repeat and so we may not have three distinct roots…). Knowing this all we really need to do to get potential starting values is to do a quick sketch of the function.

   In determining a proper range of $x$ values just keep in mind what we know about limits at infinity. We can check that as $x\to\infty$ the graph should also be approaching positive infinity and as $x\to-\infty$ the graph should be approaching negative infinity. So, we can start with a large range of $x$'s that gives the behavior we expect at the right/left ends of the graph and then narrow it down until we see the actual roots showing up on the graph.

   Doing this gives,

   

   So, it looks like we are going to have three roots here (i.e. the graph crosses the $x$-axis three times and so three roots…). For each root we’ll use the graph to pick a value of $x_0$ that is close to the root we are after (we’ll go from left to right for the problem) and then run through Newton’s Method,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   until the answers agree to six decimal places. Note as well that unlike Problems 3 & 4 we are not going to put in all the function evaluations for this problem. We’ll leave that to you to check and verify our final answers for each iteration.

   For the left most root let’s start with $x_0=-3.5$. Here are the results of iterating through Newton’s Method for this root.

   $$\begin{align*} {x_{\,1}} & = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = - 3.443478261 & \hspace{0.5in} & {\mbox{No decimal places agree}}\\ {x_{\,2}} & = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = - 3.442146902 & \hspace{0.5in} & {\mbox{Accurate to two decimal places}}\\ {x_{\,3}} & = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = - 3.44214617 & \hspace{0.5in} & {\mbox{Accurate to six decimal places}} \end{align*}$$

   So, it looks like the estimate of the left most root is: $x\approx-3.44214617$.

   For the middle root let’s start with $x_0=0$. Be careful with this root. From the graph we may be tempted to just say the root is zero. However, as we’ll see the root is not zero. It is close to zero, but is not exactly zero! Here are the results of iterating through Newton’s Method for this root.

   $$\begin{align*} {x_{\,1}} & = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 0.06666666667 & \hspace{0.5in} & {\mbox{No decimal places agree}}\\ {x_{\,2}} & = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 0.06639231824 & \hspace{0.5in} & {\mbox{Accurate to three decimal places}}\\ {x_{\,3}} & = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = 0.06639231426 & \hspace{0.5in} & {\mbox{Accurate to eight decimal places}} \end{align*}$$

   So, it looks like the estimate of the middle root is: $x\approx0.06639231426$.

   For the right most root let’s start with $x_0=4.5$. Here are the results of iterating through Newton’s Method for this root.

   $$\begin{align*} {x_{\,1}} & = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 4.380952381 & \hspace{0.5in} & {\mbox{No decimal places agree}}\\ {x_{\,2}} & = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 4.375763556 & \hspace{0.5in} & {\mbox{Accurate to one decimal place}}\\ {x_{\,3}} & = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = 4.375753856 & \hspace{0.5in} & {\mbox{Accurate to four decimal places}}\\ {x_{\,4}} & = {x_{\,3}} - \frac{{f\left( {{x_{\,3}}} \right)}}{{f'\left( {{x_{\,3}}} \right)}} = 4.375753856 & \hspace{0.5in} & {\mbox{Accurate to nine decimal places}} \end{align*}$$

   So, it looks like the estimate of the right most root is: $x\approx4.375753856$.

   Using computational aids we can find that the actual roots of this equation to be,

   $$x = - 3.44214616993\hspace{0.5in}x = 0.0663923142603\hspace{0.5in}x = 4.37575385567$$

   Note that these weren’t actually asked for in the problem and are only given for comparison purposes.

   As a final warning about Newton’s Method, be careful to not assume that you’ll get six (or better in some cases) decimal places of accuracy with just a few iterations.

   These problems were chosen with the understanding that it would only take a few iterations of the method. There are problems and/or choices of $x_0$ for which it will take significantly more iterations to get any kind of real accuracy, provided the method even works for that equation and/or choice of $x_0$. Recall that we saw an example in the notes in which the method failed spectacularly.

6. $2-x^2=\sin(x)$
   Show Solution

   First, recall that Newton’s Method solves equation in the form $f(x)=0$ and so we’ll need move everything to one side. Doing this gives,

   $$f(x)=2-x^2-\sin(x)$$

   Note that we could have just as easily gone the other direction. All that would have done was change the signs on the function and derivative evaluations in the work below. The final answers however would not be changed.

   Next, we are not given a starting value, $x_0$ and unlike Problems 3 & 4 above we are not even given an interval to use as a way to determine a good possible value of $x_0$. We are also not even told how many roots we need to find.

   So, to estimate the number of roots of the equation let’s take a look at each side of the equation and realize that each root will in fact be the point of intersection of the two curves on the left and right of the equal sign.

   The left side of the original equation is a quadratic that will have its vertex at $x=2$ and open downward while the right side is the sine function. Given what we know of these two functions we should expect there to be at most two roots where the quadratic, on its way down, intersects with the sine function. Because the quadratic will never turn around and start moving back upwards it should never intersect with the sine function again after those points.

   So, let’s graph both the quadratic and sine function to see if our intuition on this is correct. Doing this gives,

   

   So, it looks like we guessed correctly and should have two roots here.

   For each root we’ll use the graph to pick a value of $x_0$ that is close to the root we are after (we’ll go from left to right for the problem) and then run through Newton’s Method,

   $${x_{n + 1}} = {x_n} - \frac{{f\left( {{x_n}} \right)}}{{f'\left( {{x_n}} \right)}}$$

   until the answers agree to six decimal places.

   Note as well that unlike Problems 3 & 4 we are not going to put in all the function evaluations for this problem. We’ll leave that to you to check and verify our final answers for each iteration.

   Also note that the analysis that we had to do to estimate the number of roots is something that does need to be done for these kinds of problems and it will differ for each equation. However, if you do have a basic knowledge of how most of the basic functions behave you can do this for most equations you’ll be asked to deal with.

   For the left most root let’s start with $x_0=−1.5$. Here are the results of iterating through Newton’s Method for this root.

   $$\begin{align*} {x_{\,1}} & = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = - 1.755181948 & \hspace{0.5in} & {\mbox{No decimal places agree}}\\ {x_{\,2}} & = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = - 1.728754674 & \hspace{0.5in} & {\mbox{Accurate to one decimal place}}\\ {x_{\,3}} & = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = - 1.728466353 & \hspace{0.5in} & {\mbox{Accurate to three decimal places}}\\ {x_{\,4}} & = {x_{\,3}} - \frac{{f\left( {{x_{\,3}}} \right)}}{{f'\left( {{x_{\,3}}} \right)}} = -1.728466319 & \hspace{0.5in} & {\mbox{Accurate to seven decimal places}} \end{align*}$$

   So, it looks like the estimate of the left most root is: $x\approx−1.728466319$.

   For the right most root let’s start with $x_0=1$. Here are the results of iterating through Newton’s Method for this root.

   $$\begin{align*} {x_{\,1}} & = {x_{\,0}} - \frac{{f\left( {{x_{\,0}}} \right)}}{{f'\left( {{x_{\,0}}} \right)}} = 1.062405571 & \hspace{0.5in} & {\mbox{No decimal places agree}}\\ {x_{\,2}} & = {x_{\,1}} - \frac{{f\left( {{x_{\,1}}} \right)}}{{f'\left( {{x_{\,1}}} \right)}} = 1.061549933 & \hspace{0.5in} & {\mbox{Accurate to two decimal places}}\\ {x_{\,3}} & = {x_{\,2}} - \frac{{f\left( {{x_{\,2}}} \right)}}{{f'\left( {{x_{\,2}}} \right)}} = 1.061549775 & \hspace{0.5in} & {\mbox{Accurate to six decimal places}} \end{align*}$$

   So, it looks like the estimate of the right most root is: $x\approx1.061549775$.

   Using computational aids we found that the actual roots of this equation to be,

   $$x = - 1.72846631899718\hspace{0.5in}x = 1.06154977463138$$

   Note that these weren’t actually asked for in the problem and are only given for comparison purposes.

   As a final warning about Newton’s Method, be careful to not assume that you’ll get six (or better in some cases) decimal places of accuracy with just a few iterations.

   These problems were chosen with the understanding that it would only take a few iterations of the method. There are problems and/or choices of $x_0$ for which it will take significantly more iterations to get any kind of real accuracy, provided the method even works for that equation and/or choice of $x_0$. Recall that we saw an example in the notes in which the method failed spectacularly.

Assignment Problems

For problems 1 - 3 use Newton's Method to determine $x_2$ for the given function and given value of $x_0$.

1. $f(x)=7x^3-8x+4x$, $x_0=-1$
2. $f(x)=\cos(3x)-\sin(x)$, $x_0=0$
3. $f(x)=7-e^{2x-3}$, $x_0=5$

For problems 4 - 8 use Newton's Method to find the root of the given equation, accurate to six decimal places, that lies in the given interval.

4. $x^5=6$ in $[1,2]$
5. $2x^3-9x^2+17x+20=0$ in $[-1,1]$
6. $3-12x-4x^2-3x^4=0$ in $[-3,1]$
7. $e^x=4\cos(x)$ in $[-1,1]$
8. $x^2=e^{2-x^2}$ in $[0,2]$

For problems 9 - 12 use Newton's Method to find all the roots of the given equation accurate to six decimal places.

9. $2x^3+5x^2-10x-4=0$
10. $x^4+4x^3-54x^2-92x+105=0$
11. $\dfrac{3}{2}-e^{-x^2}=\cos(x)$
12. $\ln(x)=2\cos(x)$

13. Suppose that we want to find the root to $x^3-7x^2+8x-3=0$.Is it possible to use $x_0=4$ as the initial point? What can you conclude about using Newton's Method to approximate roots from this example?
14. Use the function $f(x)=\cos^2(x)-\sin(x)$ for this problem.
1. Plot the function on the interval $[0,9]$.
2. Use $x_0=4$ to find one of the roots of this function to six decimal places. Did you get the root you expected to?
3. Use $x_0=5$ to find one of the roots of this function to six decimal places. Did you get the root you expected to?
4. Use $x_0=6$ to find one of the roots of this function to six decimal places. Did you get the root you expected to?
5. What can you conclude about choosing values of $x_0$ to find roots of equations using Newton's Method?
15. Use $x_0=0$ to find one of the roots of $2x^5−7x^3+3x−1=0$ accurate to six decimal places. Did we choose a good value of $x_0$ for this problem?