Check out my textbook from Cambridge University Press:
Bayesian Logical Data Analysis for the Physical Sciences
Check out the free resources that accompany the book which includes solution sets, errata, supporting Mathematica software, and a Mathematica free interactive player,
plus, New Supplement that includes two additional chapters (updated April 2016)
plus, Fusion Markov chain Monte Carlo code for Mathematica (updated April 2016)
Excerpt from the book preface: "We are currently in the throes of a major paradigm shift in our understanding of statistical inference based on a powerful generalization of Aristotelian logic. For historical reasons, it is referred to as Bayesian Probability Theory or Bayesian statistic. To get a taste of how significant this development is, consider the following: probabilities are commonly quantified by a real number between 0 and 1. The end-points, corresponding to absolutely false and absolutely true, are simply the extreme limits of this infinity of real numbers. Deductive logic, which is based on axiomatic knowledge, corresponds to these two extremes of 0 and 1. Now try to imagine what you might achieve with a theory of extended logic that encompassed the whole range from 0 to 1. This is exactly what is needed in science and real life where we never know anything is absolutely true or false. Of course, the field of probability has been around for years, but what is new is the appreciation that the rules of probability are not merely rules for manipulating random variables. They are now recognized as uniquely valid principles of logic, for conducting inference about any proposition or hypothesis of interest. It is thus a mathematical theory that encompasses both inductive and deductive logic. Ordinary deductive logic is just a special case in the idealized limit of complete information."