This book constitutes the proceedings of the 24th International Conference on Algorithmic Learning Theory, ALT 2013, held in Singapore in October 2013, and co-located with the 16th International Conference on Discovery Science, DS 2013. The 23 papers presented in this volume were carefully reviewed and selected from 39 submissions. In addition the book contains 3 full papers of invited talks. The papers are organized in topical sections named: online learning, inductive inference and grammatical inference, teaching and learning from queries, bandit theory, statistical learning theory, Bayesian/stochastic learning, and unsupervised/semi-supervised learning.

Inthesummerof2008, reinforcementlearningresearchersfromaroundtheworld gathered in the north of France for a week of talks and discussions on reinfor- ment learning, on how it could be made more e?cient, applied to a broader range of applications, and utilized at more abstract and symbolic levels. As a participant in this 8th European Workshop on Reinforcement Learning, I was struck by both the quality and quantity of the presentations. There were four full days of short talks, over 50 in all, far more than there have been at any p- vious meeting on reinforcement learning in Europe, or indeed, anywhere else in the world. There was an air of excitement as substantial progress was reported in many areas including Computer Go, robotics, and ?tted methods. Overall, the work reported seemed to me to be an excellent, broad, and representative sample of cutting-edge reinforcement learning research. Some of the best of it is collected and published in this volume. The workshopandthe paperscollectedhere provideevidence thatthe ?eldof reinforcement learning remains vigorous and varied. It is appropriate to re?ect on some of the reasons for this. One is that the ?eld remains focused on a pr- lem - sequential decision making - without prejudice as to solution methods. Another is the existence of a common terminology and body of theory

From Bandits to Monte-Carlo Tree Search covers several aspects of the ""optimism in the face of uncertainty"" principle for large scale optimization problems under finite numerical budget. The monograph's initial motivation came from the empirical success of the so-called ""Monte-Carlo Tree Search"" method popularized in Computer Go and further extended to many other games as well as optimization and planning problems. It lays out the theoretical foundations of the field by characterizing the complexity of the optimization problems and designing efficient algorithms with performance guarantees. The main direction followed in this monograph consists in decomposing a complex decision making problem (such as an optimization problem in a large search space) into a sequence of elementary decisions, where each decision of the sequence is solved using a stochastic ""multi-armed bandit"" (mathematical model for decision making in stochastic environments). This defines a hierarchical search which possesses the nice feature of starting the exploration by a quasi-uniform sampling of the space and then focusing, at different scales, on the most promising areas (using the optimistic principle) until eventually performing a local search around the global optima of the function. This monograph considers the problem of function optimization in general search spaces (such as metric spaces, structured spaces, trees, and graphs) as well as the problem of planning in Markov decision processes. Its main contribution is a class of hierarchical optimistic algorithms with different algorithmic instantiations depending on whether the evaluations are noisy or noiseless and whether some measure of the local ''smoothness'' of the function around the global maximum is known or unknown.