One of the useful things about Bayesian methods is that they have an built-in Occam's razor, which naturally prefers simple models over more complex models. I was looking at some work done by David Mackay as part of his PhD thesis in 1991, when out of boredom I just googled his name to find out what he was up to now.
One thing I found was this fabulous resource. So far I have only read the 10-page synopsis and skimmed through the rest of the introduction, and wish I knew about it before. As it turns out, I taught undergrad thermodynamics three years in a row, and initially struggled with what I should assign for the first homework. These days I make them do a survey on recent topics relating to the energy crisis we face. While there are a tons of resources, and a plethora of information, it is nice to have some of that distilled into knowledge and wisdom, which is what this book (freely available as pdf) does.