Convergence Analysis of Newton's Method

1. Case of Non-Convergence

Newton's method does not always converge. For example, consider:
\[ f(x) = \arctan(x) \] Starting from \( x_0 = 1.3917 \), the iterations may oscillate or diverge instead of converging.

2. Error Analysis

Let \( r \) be a simple root of \( f(x) \) (\( f(r) = 0 \) and \( f'(r) \neq 0 \)). Denote the error at iteration \( n \) by:
\[ e_n = x_n - r \] If the method converges, it does so quadratically:
\[ e_{n+1} \approx C e_n^2 \] Expanding \( f \) around \( r \) with \( x_n = r + e_n \):
\[ f(x_n) = f(r) + e_n f'(r) + \frac{1}{2} e_n^2 f''(\xi_n), \quad \xi_n \in [r, x_n] \] Since \( f(r) = 0 \):
\[ f(x_n) = e_n f'(r) + \frac{1}{2} e_n^2 f''(\xi_n) \] Similarly:
\[ f'(x_n) = f'(r) + e_n f''(\eta_n), \quad \eta_n \in [r, x_n] \] Thus:
\[ e_{n+1} = e_n - \frac{f(x_n)}{f'(x_n)} \approx \frac{1}{2} \cdot \frac{f''(\xi_n)}{f'(r)} \cdot e_n^2 \] Therefore, Newton's method converges quadratically for smooth functions with \( f'(r) \neq 0 \):
\[ e_{n+1} \approx C e_n^2, \quad C = \frac{1}{2} \cdot \frac{f''(r)}{f'(r)} \]

3. Local Convergence Theorem

Let \( f \in C^2([a, b]) \) satisfy:

  • \( f(a) f(b) < 0 \) (sign change on \([a, b]\))
  • \( f'(x) \neq 0 \) for all \( x \in [a, b] \) (strictly monotonic)
  • \( f''(x) \) does not change sign on \([a, b]\) (function is convex or concave)

Then:

  1. \( f \) has a unique root \( r \in [a, b] \).
  2. For any \( x_0 \in [a, b] \) such that \( f(x_0) f''(x_0) > 0 \), Newton's method converges to \( r \).
  3. Convergence is quadratic:
    \[ \lim_{n \to \infty} \frac{e_{n+1}}{e_n^2} = \frac{1}{2} \cdot \frac{f''(r)}{f'(r)} \]

This theorem guarantees rapid convergence provided the conditions are satisfied and the initial guess is chosen appropriately.