*Engineering, Tech, TeX, and Numbers. Lots of Numbers.*

- 2014.01.31: Latent Semantic Analysis
- 2014.01.25: On Cutting Firewood
- 2014.01.08: Dragons!
- 2013.10.13: A Haiku For Winter
- 2013.09.28: ShareLaTeX: TeX from anywhere!
- 2013.09.23: Google Trends Scipt

Numerical differentation has a cornucopia of uses in engineering. However, when you're first learning it may appear daunting - especially the form of the equations. For instance, these are the forward difference equations:

f'(x_i) \approx \Delta^1 = \frac{f(x_{i+1})-f(x_i)}{h} + O(h)

f''(x_i) \approx \Delta^2 = \frac{f(x_{i+2}) - 2f(x_{i+1}) + f(x_i)}{h^2}} + O(h)

f'''(x_i) \approx \Delta^3 = \frac{f(x_{i+3})-3f(x_{i+2}) + 3f(x_{i+1}) - f(x_i)}{h^3}} + O(h)

f'''(x_i) \approx \Delta^4 = \frac{f(x_{i+4} - 4f(x_{i+3}) + 6f(x_{i+2}) - 4f(x_{i+1}) + f(x_i)}{h^4}} + O(h)

There are a few patterns here that make memorizing these formulas pretty easy. The first, and most obvious, pattern is that you always start with a positive term, and alternate the sign of the remaining terms. Another pattern is that order of the h term in denominator increases by 1 every time.

The most important pattern is that the coefficients (if we disregard the sign) are the binomial coefficients. We can pull out the coefficients of the function evaluations and the denominator, D, and put them in a table to make this a little more obvious.

f(x_{i+4}) | f(x_{i+3}) | f(x_{i+2}) | f(x_{i+1}) | f(x_{i}) | D | |

\Delta^1 | +1 | -1 | h | |||

\Delta^2 | +1 | -2 | +1 | h^2 | ||

\Delta^3 | +1 | -3 | +3 | -1 | h^3 | |

\Delta^4 | +1 | -4 | +6 | -4 | +1 | h^4 |

This is great, but we have to remember that this pattern is only for the forward difference formulas. What about the backward difference formulas, denoted by \nabla^n? We're in luck! The coefficients follow a very similar pattern, as might be expected. There methods are also all O(h).

f(x_{i}) | f(x_{i-1}) | f(x_{i-2}) | f(x_{i-3}) | f(x_{i-4}) | D | |

\nabla^1 | +1 | -1 | h | |||

\nabla^2 | +1 | -2 | +1 | h^2 | ||

\nabla^3 | +1 | -3 | +3 | -1 | h^3 | |

\nabla^4 | +1 | -4 | +6 | -4 | +1 | h^4 |

Unforunately, the centered difference fomulas, denoted by \delta^n, don't follow exactly the same patterns. The terms still alternate in sign, but the coefficients aren't binomial. Also, all of these methods are O(h^2).

f(x_{i+2}) | f(x_{i+1}) | f(x_{i}) | f(x_{i-1}) | f(x_{i-2}) | D | |

\delta^1 | +1 | -1 | 2h | |||

\delta^2 | +1 | -2 | +1 | h^2 | ||

\delta^3 | +1 | -2 | +2 | -1 | 2h^3 | |

\delta^4 | +1 | -4 | +6 | -4 | +1 | h^4 |

Hopefully these patterns will help you as your begin to work with numerical differentiation!

by Chris McComb