Thursday, March 10, 2011

Domain-based overconfidence

I was reading a financial blog, that I peruse occasionally. In a recent post, there were two "puzzles". Usually, this is enough to stop me from passing on.
  1. The Dow Jones Price Index does not include the effects of dividend re-investment. If dividends had been considered re-invested in the index since its inception in 1896, at what price level would the index be at today? Provide a 90% confidence interval around your answer (i.e. you are 90% confident that your interval includes the right answer).
  2. There are 100 bags, each containing 1000 poker chips. 45 bags have 700 black chips and 300 red chips, while 55 bags have 700 red chips and 300 black chips. If you select a bag, what is the probability that most of the chips are black? If you pulled out 12 chips from that bag, and 8 of them are black and 4 of them are red, now what is the probability that most of the chips in the bag are black?
I have seen questions of this type before (a short rant, later), and so I was not caught off-guard. (Sidenote: Puzzles and jokes aren't quite the same when you hear them the second time) The first question I answered 1 trillion (actual answer is 650,000, the DJIA is currently around 12,000), and the second one I said 0.45 and 0.96 (thanks to Bayes theorem), which are the correct answers.

Typically, people guess much smaller than 650,000 for Q1, and for the second part of Q2, people typically guess between 45-75%.

To be perfectly honest, for Q1, I had two numbers in mind. I thought of 1 trillion as the "correct answer" (based on having seen the type of questions before), and a much lower (and wrong) 100,000 as a plausible tight upper-bound. So in some sense I flunked Q1.

Here's my rationalization.

The disturbing part is the "90% confidence", which probably implies that if one was asked such questions 10 times, one should flunk once on average. The problem with gross overestimations (like 1 trillion) are that the odds of getting that occasional (required) wrong answer diminish.

No comments: