Metropolis-Hastings

As I mentioned previously, one of my colleagues is running an MCMC seminar this semester, which is essentially a journal club that is trying to read and discuss some of the more seminal papers in the field.

The second paper we discussed was Hastings famous 1970 paper, which generalized the "Metropolis" algorithm and couched it in a more general form (the Metropolis-Hastings algorithm).

The story behind this man and his famous paper turns out to be every bit as fascinating as that behind the original paper.

From 1966 to 1971, Hastings was an Associate Professor in the Department of Mathematics at the University of Toronto. During this period, he wrote the famous paper listed above (which generalised the work of N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller (1953), "Equations of state calculations by fast computing machines", J. Chem. Phys. 21, 1087-1091). Hastings explains: 

When I returned to the University of Toronto, after my time at Bell Labs, I focused on Monte Carlo methods and at first on methods of sampling from probability distributions with no particular area of application in mind. [University of Toronto Chemistry professor] John Valleau and his associates consulted me concerning their work. They were using Metropolis's method to estimate the mean energy of a system of particles in a defined potential field. With 6 coordinates per particle, a system of just 100 particles involved a dimension of 600. When I learned how easy it was to generate samples from high dimensional distributions using Markov chains, I realised how important this was for Statistics, and I devoted all my time to this method and its variants which resulted in the 1970 paper.

He seems to have been a very fascinating character.

He wrote only three peer-reviewed papers all this life, and supervised only a single PhD student.