It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and interest in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish tness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem.

This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations.

The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.

The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots."

Current robots and other artificial systems are typically able to accomplish only one single task. Overcoming this limitation requires the development of control architectures and learning algorithms that can support the acquisition and deployment of several different skills, which in turn seems to require a modular and hierarchical organization. In this way, different modules can acquire different skills without catastrophic interference, and higher-level components of the system can solve complex tasks by exploiting the skills encapsulated in the lower-level modules. While machine learning and robotics recognize the fundamental importance of the hierarchical organization of behavior for building robots that scale up to solve complex tasks, research in psychology and neuroscience shows increasing evidence that modularity and hierarchy are pivotal organization principles of behavior and of the brain. They might even lead to the cumulative acquisition of an ever-increasing number of skills, which seems to be a characteristic of mammals, and humans in particular. This book is a comprehensive overview of the state of the art on the modeling of the hierarchical organization of behavior in animals, and on its exploitation in robot controllers. The book perspective is highly interdisciplinary, featuring models belonging to all relevant areas, including machine learning, robotics, neural networks, and computational modeling in psychology and neuroscience. The book chapters review the authors' most recent contributions to the investigation of hierarchical behavior, and highlight the open questions and most promising research directions. As the contributing authors are among the pioneers carrying out fundamental work on this topic, the book covers the most important and topical issues in the field from a computationally informed, theoretically oriented perspective. The book will be of benefit to academic and industrial researchers and graduate students in related disciplines.

It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and inter est in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish fitness-enhanc ing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem. This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.

Anticipatory behavior in adaptive learning systems continues attracting attention of researchers in many areas, including cognitive systems, neuroscience, psychology, and machine learning. This book constitutes the thoroughly refereed post-workshop proceedings of the 4th International Workshop on Anticipatory Behavior in Adaptive Learning Systems, ABiALS 2008, held in Munich, Germany, in June 2008, in collaboration with the six-monthly Meeting of euCognition 'The Role of Anticipation in Cognition'. The 18 revised full papers presented were carefully selected during two rounds of reviewing and improvement for inclusion in the book. The introductory chapter of this state-of-the-art survey not only provides an overview of the contributions included in this volume but also revisits the current available terminology on anticipatory behavior and relates it to the available system approaches. The papers are organized in topical sections on anticipation in psychology with focus on the ideomotor view, conceptualizations, anticipation and dynamical systems, computational modeling of psychological processes in the individual and social domains, behavioral and cognitive capabilities based on anticipation, and computational frameworks and algorithms for anticipation, and their evaluation.

This book presents the refereed post-proceedings of the Third International Workshop on Anticipatory Behavior in Adaptive Learning Systems. Twenty full papers were chosen from among the many submissions. Papers are organized into sections covering anticipatory aspects in brains, language, and cognition; individual anticipatory frameworks; learning predictions and anticipations; anticipatory individual behavior; and anticipatory social behavior.