An Introduction to Universal Artificial Intelligence provides a gentle introduction to Universal Artificial Intelligence (UAI), a theory that provides a formal underpinning of what it means for an agent to act intelligently in a general class of environments. First presented in Universal Artificial Intelligence (Hutter, 2004), UAI presents a model in which most other problems in AI can be presented, and unifies ideas from sequential decision theory, Bayesian inference and information theory to construct AIXI, an optimal reinforcement learning agent that learns to act optimally in unknown environments. AIXI represents a theoretical bound on intelligent behaviour, and so we also discuss tractable approximations of this optimal agent. The book covers important practical approaches including efficient Bayesian updating with context tree weighting, and stochastic planning, approximated by sampling with Monte Carlo tree search. Algorithms are also included for the reader to implement, along with experimental results to compare against. This serves to approximate AIXI, as well as being used in state-of-the-art approaches in AI today. The book ends with a philosophical discussion of AGI covering the following key questions: Should intelligent agents be constructed at all, is it inevitable that they will be constructed, and is it dangerous to do so? This text is suitable for late undergraduates and includes an extensive background chapter to fill in the assumed mathematical background.

An Introduction to Universal Artificial Intelligence provides a gentle introduction to Universal Artificial Intelligence (UAI), a theory that provides a formal underpinning of what it means for an agent to act intelligently in a general class of environments. First presented in Universal Artificial Intelligence (Hutter, 2004), UAI presents a model in which most other problems in AI can be presented, and unifies ideas from sequential decision theory, Bayesian inference and information theory to construct AIXI, an optimal reinforcement learning agent that learns to act optimally in unknown environments. AIXI represents a theoretical bound on intelligent behaviour, and so we also discuss tractable approximations of this optimal agent. The book covers important practical approaches including efficient Bayesian updating with context tree weighting, and stochastic planning, approximated by sampling with Monte Carlo tree search. Algorithms are also included for the reader to implement, along with experimental results to compare against. This serves to approximate AIXI, as well as being used in state-of-the-art approaches in AI today. The book ends with a philosophical discussion of AGI covering the following key questions: Should intelligent agents be constructed at all, is it inevitable that they will be constructed, and is it dangerous to do so? This text is suitable for late undergraduates and includes an extensive background chapter to fill in the assumed mathematical background.

This book constitutes revised and selected papers of the 9th European Workshop on Reinforcement Learning, EWRL 2011, which took place in Athens, Greece in September 2011. The papers presented were carefully reviewed and selected from 40 submissions. The papers are organized in topical sections online reinforcement learning, learning and exploring MDPs, function approximation methods for reinforcement learning, macro-actions in reinforcement learning, policy search and bounds, multi-task and transfer reinforcement learning, multi-agent reinforcement learning, apprenticeship and inverse reinforcement learning and real-world reinforcement learning.

Personal motivation. The dream of creating artificial devices that reach or outperform human inteUigence is an old one. It is also one of the dreams of my youth, which have never left me. What makes this challenge so interesting? A solution would have enormous implications on our society, and there are reasons to believe that the AI problem can be solved in my expected lifetime. So, it's worth sticking to it for a lifetime, even if it takes 30 years or so to reap the benefits. The AI problem. The science of artificial intelligence (AI) may be defined as the construction of intelligent systems and their analysis. A natural definition of a system is anything that has an input and an output stream. Intelligence is more complicated. It can have many faces like creativity, solving prob lems, pattern recognition, classification, learning, induction, deduction, build ing analogies, optimization, surviving in an environment, language processing, and knowledge. A formal definition incorporating every aspect of intelligence, however, seems difficult. Most, if not all known facets of intelligence can be formulated as goal driven or, more precisely, as maximizing some utility func tion. It is, therefore, sufficient to study goal-driven AI; e. g. the (biological) goal of animals and humans is to survive and spread. The goal of AI systems should be to be useful to humans."

This book constitutes the refereed proceedings of the 18th International Conference on Algorithmic Learning Theory, ALT 2007, held in Sendai, Japan, October 1-4, 2007, colocated with the 10th International Conference on Discovery Science, DS 2007.

The 25 revised full papers presented together with the abstracts of 5 invited papers were carefully reviewed and selected from 50 submissions. The papers are dedicated to the theoretical foundations of machine learning; they address topics such as query models, on-line learning, inductive inference, algorithmic forecasting, boosting, support vector machines, kernel methods, complexity and learning, reinforcement learning, unsupervised learning and grammatical inference.

Personal motivation. The dream of creating artificial devices that reach or outperform human inteUigence is an old one. It is also one of the dreams of my youth, which have never left me. What makes this challenge so interesting? A solution would have enormous implications on our society, and there are reasons to believe that the AI problem can be solved in my expected lifetime. So, it's worth sticking to it for a lifetime, even if it takes 30 years or so to reap the benefits. The AI problem. The science of artificial intelligence (AI) may be defined as the construction of intelligent systems and their analysis. A natural definition of a system is anything that has an input and an output stream. Intelligence is more complicated. It can have many faces like creativity, solving prob lems, pattern recognition, classification, learning, induction, deduction, build ing analogies, optimization, surviving in an environment, language processing, and knowledge. A formal definition incorporating every aspect of intelligence, however, seems difficult. Most, if not all known facets of intelligence can be formulated as goal driven or, more precisely, as maximizing some utility func tion. It is, therefore, sufficient to study goal-driven AI; e. g. the (biological) goal of animals and humans is to survive and spread. The goal of AI systems should be to be useful to humans."