Mathematical Theory of Bayesian Statistics introduces the mathematical foundation of Bayesian inference which is well-known to be more accurate in many real-world problems than the maximum likelihood method. Recent research has uncovered several mathematical laws in Bayesian statistics, by which both the generalization loss and the marginal likelihood are estimated even if the posterior distribution cannot be approximated by any normal distribution. Features Explains Bayesian inference not subjectively but objectively. Provides a mathematical framework for conventional Bayesian theorems. Introduces and proves new theorems. Cross validation and information criteria of Bayesian statistics are studied from the mathematical point of view. Illustrates applications to several statistical problems, for example, model selection, hyperparameter optimization, and hypothesis tests. This book provides basic introductions for students, researchers, and users of Bayesian statistics, as well as applied mathematicians. Author Sumio Watanabe is a professor of Department of Mathematical and Computing Science at Tokyo Institute of Technology. He studies the relationship between algebraic geometry and mathematical statistics.

Sure to be influential, this book lays the foundations for the use of algebraic geometry in statistical learning theory. Many widely used statistical models and learning machines applied to information science have a parameter space that is singular: mixture models, neural networks, HMMs, Bayesian networks, and stochastic context-free grammars are major examples. Algebraic geometry and singularity theory provide the necessary tools for studying such non-smooth models. Four main formulas are established: 1. the log likelihood function can be given a common standard form using resolution of singularities, even applied to more complex models; 2. the asymptotic behaviour of the marginal likelihood or 'the evidence' is derived based on zeta function theory; 3. new methods are derived to estimate the generalization errors in Bayes and Gibbs estimations from training errors; 4. the generalization errors of maximum likelihood and a posteriori methods are clarified by empirical process theory on algebraic varieties.

Mathematical Theory of Bayesian Statistics introduces the mathematical foundation of Bayesian inference which is well-known to be more accurate in many real-world problems than the maximum likelihood method. Recent research has uncovered several mathematical laws in Bayesian statistics, by which both the generalization loss and the marginal likelihood are estimated even if the posterior distribution cannot be approximated by any normal distribution. Features Explains Bayesian inference not subjectively but objectively. Provides a mathematical framework for conventional Bayesian theorems. Introduces and proves new theorems. Cross validation and information criteria of Bayesian statistics are studied from the mathematical point of view. Illustrates applications to several statistical problems, for example, model selection, hyperparameter optimization, and hypothesis tests. This book provides basic introductions for students, researchers, and users of Bayesian statistics, as well as applied mathematicians. Author Sumio Watanabe is a professor of Department of Mathematical and Computing Science at Tokyo Institute of Technology. He studies the relationship between algebraic geometry and mathematical statistics.