## Thursday, June 12, 2014

### Logarithm of Sum of Exponentials: Compendium

This post is just to collect a few recent entries on this topic under one umbrella.

1. This post set up the general problem

Numerically compute the following sum, for arbitrary $x_i$ ,$F = \log \left(e^{x_1} + e^{x_2} + ... + e^{x_N} \right).$ It also briefly discussed the major problem with doing this by brute force (overflow/underflow).

2.  We then made the problem specific, whose answer could be analytically computed. $F = \log \left(e^{-1000} + e^{-1001} + ... + e^{-1100} \right) = -999.54.$
3. We briefly looked at numerically evaluating  an even simpler model problem $f(y) = \log(1 + e^y).$ While much simpler, this problem reflects all of the complexity in the original problem.

4. Equipped with the solution, we went back and solved our specific problem.