Why numerical differentiation may be trickier than you think?

Here is a link to a fascinating presentation by Harvey Stein ("Risky Measures of Risk: Error Analysis of Numerical Differentiation").

He makes a very "visual" case for why one needs to think carefully before using large (convexity error) or small step sizes (cancellation error).