
Bayes' Theorem describes what makes something "evidence" for something else, and tells you precisely how much evidence it actually is. Statistical models are judged by comparison to the Bayesian method because, in statistics, the Bayesian method is as good as it gets  the Bayesian method defines the maximum amount of mileage you can get out of a given piece of evidence, in the same way that thermodynamics defines the maximum amount of work you can get out of a temperature differential. This is why you hear cognitive scientists talking about Bayesian reasoners. In cognitive science, Bayesian reasoner is the technically precise codeword that we use to mean rational mind.
http://www.yudkowsky.net/bayes/bayes.html?repost
This page is long and ponderous, but it's worth slogging through. The most important thing about Bayes Theorem is that it gives you a formula to correctly compute conditional probabilities. In other words, to check your intuition about the probability of some event given the probability of some other event. You will often find your intuition is wrong when it comes to conditional probability, because the probability of A given B plus the probability of B given A doesn't have to add up to 100%, and in fact often does not. My favorite bit in the page is the mechanical gizmo problem, but the breast cancer test example is probably more valuable as a real world example. (The egg/pearl problems are too abstract for my little mind, I skipped them entirely.)