Thursday, June 12, 2014

Logarithm of Sum of Exponentials: Compendium

This post is just to collect a few recent entries on this topic under one umbrella.

1. This post set up the general problem

Numerically compute the following sum, for arbitrary \(x_i\) ,\[F = \log \left(e^{x_1} + e^{x_2} + ... + e^{x_N} \right).\] It also briefly discussed the major problem with doing this by brute force (overflow/underflow).

2.  We then made the problem specific, whose answer could be analytically computed. \[F = \log \left(e^{-1000} + e^{-1001} + ... + e^{-1100} \right) = -999.54.\]
3. We briefly looked at numerically evaluating  an even simpler model problem \[f(y) = \log(1 + e^y).\] While much simpler, this problem reflects all of the complexity in the original problem.

4. Equipped with the solution, we went back and solved our specific problem.

No comments: