This book reviews research developments in diverse areas of reinforcement learning such as model-free actor-critic methods, model-based learning and control, information geometry of policy searches, reward design, and exploration in biology and the behavioral sciences. Special emphasis is placed on advanced ideas, algorithms, methods, and applications. The contributed papers gathered here grew out of a lecture course on reinforcement learning held by Prof. Jan Peters in the winter semester 2018/2019 at Technische Universitat Darmstadt. The book is intended for reinforcement learning students and researchers with a firm grasp of linear algebra, statistics, and optimization. Nevertheless, all key concepts are introduced in each chapter, making the content self-contained and accessible to a broader audience.

This book identifies, analyzes and discusses the current trends of digitalized, decentralized, and networked physical value creation by focusing on the particular example of 3D printing. In addition to evaluating 3D printing's disruptive potentials against a broader economic background, it also addresses the technology's potential impacts on sustainability and emerging modes of bottom-up and community-based innovation. Emphasizing these topics from economic, technical, social and environmental perspectives, the book offers a multifaceted overview that scrutinizes the scenario of a fundamental transition: from a centralized to a far more decentralized system of value creation.

This book presents the state of the art in reinforcement learning applied to robotics both in terms of novel algorithms and applications. It discusses recent approaches that allow robots to learn motor.

skills and presents tasks that need to take into account the dynamic behavior of the robot and its environment, where a kinematic movement plan is not sufficient. The book illustrates a method that learns to generalize parameterized motor plans which is obtained by imitation or reinforcement learning, by adapting a small set of global parameters and appropriate kernel-based reinforcement learning algorithms. The presented applications explore highly dynamic tasks and exhibit a very efficient learning process. All proposed approaches have been extensively validated with benchmarks tasks, in simulation and on real robots. These tasks correspond to sports and games but the presented techniques are also applicable to more mundane household tasks. The book is based on the first author s doctoral thesis, which won the 2013 EURON Georges Giralt PhD Award."

Experimental advances in helium atom scattering spectroscopy over the last forty years have allowed the measurement of surface phonon dispersion curves of more than 200 different crystal surfaces and overlayers of insulators, semiconductors and metals. The first part of the book presents, at a tutorial level, the fundamental concepts and methods in surface lattice dynamics, and the theory of atom-surface interaction and inelastic scattering in their various approximations, up to the recent electron-phonon theory of helium atom scattering from conducting surfaces. The second part of the book, after introducing the experimentalist to He-atom spectrometers and the rich phenomenology of helium atom scattering from corrugated surfaces, illustrates the most significant experimental results on the surface phonon dispersion curves of various classes of insulators, semiconductors, metals, layered crystals, topological insulators, complex surfaces, adsorbates, ultra-thin films and clusters. The great potential of helium atom scattering for the study of atomic scale diffusion, THz surface collective excitations, including acoustic surface plasmons, and the future prospects of helium atom scattering are presented in the concluding chapters. The book will be valuable reading for all researchers and graduate students interested in dynamical processes at surfaces.

From an engineering standpoint, the increasing complexity of robotic systems and the increasing demand for more autonomously learning robots, has become essential. This book is largely based on the successful workshop "From motor to interaction learning in robots" held at the IEEE/RSJ International Conference on Intelligent Robot Systems. The major aim of the book is to give students interested the topics described above a chance to get started faster and researchers a helpful compandium.

Drawing on empirical insights from the field of desktop 3D printing, this book elaborates the concept of innovation communities as a pattern of open and distributed innovation. As these communities spur a fruitful exchange of explorative, open source knowledge, they represent a novel mode of "doing innovation", which considerably differs from established practices in market and business realms. Hence, the people that participate in these collective endeavors often develop entrepreneurial ambitions and start to exploit community-based innovations commercially. The book presents deep insights on the institutional idiosyncrasies of innovation communities, the associated dilemma of entrepreneurship and the strategies of 3D-printing startups to face the corresponding challenges.

Knowing Governance sets out to understand governance through the design and making of its models and instruments. What kinds of knowledge do they require and reproduce? How are new understandings of governance produced in practice, by scientists and policy makers and by the publics with whom they engage?

The introduction of micro CHP - the simultaneous production of heat and power in a single building based on small energy conversion units such as Stirling and reciprocating engines or fuel cells - is of increasing political and public interest. This book introduces into micro CHP systems and technologies, and presents the results of the first such investigation carried out by four German research bodies.

This open access book covers recent advances in experiments using the ultra-cold, very weakly perturbing superfluid environment provided by helium nanodroplets for high resolution spectroscopic, structural and dynamic studies of molecules and synthetic clusters. The recent infra-red, UV-Vis studies of radicals, molecules, clusters, ions and biomolecules, as well as laser dynamical and laser orientational studies, are reviewed. The Coulomb explosion studies of the uniquely quantum structures of small helium clusters, X-ray imaging of large droplets and electron diffraction of embedded molecules are also described. Particular emphasis is given to the synthesis and detection of new species by mass spectrometry and deposition electron microscopy.

The harmful consequences of alcoholism for the unborn child are still largely unknown. This book presents a detailed description of fetal alcohol syndrome as well as fetal alcohol spectrum disorders with respect to their clinical presentation, diagnosis, epidemiology, and pathogenesis. It also includes detailed considerations of underlying psychopathology, prevention, and therapy as well as the social consequences and impacts to patients.

Since its beginnings more than fifty years ago, metal music has grown in popularity worldwide, not only as a musical culture but increasingly as a recognised field of study. This Cambridge Companion reflects the maturing field of 'metal music studies' by introducing the music and its cultures, as well as recent research perspectives from disciplines ranging from musicology and music technology to religious studies, Classics, and Scandinavian and African studies. Topics covered include technology and practice, identity and culture, modern metal genres, and global metal, with reference to performers including Black Sabbath, Metallica and Amon Amarth. Designed for students and their teachers, contributions explore the various musical styles and cultures of metal, providing an informative introduction for those new to the field and an up-to-date resource for readers familiar with the academic metal literature.

Marxist discourse around automation has recently become waylaid with breathless techno-pessimist dystopias and fanciful imaginations of automated luxury communism. This collection of essays by both established veterans of the field and new voices is a refreshingly sober materialist reflection on recent technological developments within capitalist production. It covers a broad range of digital aspects now proliferating across our work and lives, including chapters on the digitalisation of agriculture, robotics in the factory and the labour process on crowdworking platforms. It looks to how 20th century Marxist predictions of the 'workerless factory' are, or are not, coming true, and how 'Platform Capitalism' should be understood and critiqued. Through rich empirical, theoretical and historical material, this book is necessary reading for those wanting a clear overview of our digital world.

Dogs in the North offers an interdisciplinary in-depth consideration of the multiple roles that dogs have played in the North. Spanning the deep history of humans and dogs in the North, the volume examines a variety of contexts in North America and Eurasia. The case studies build on archaeological, ethnohistorical, ethnographic, and anthropological research to illuminate the diversity and similarities in canine-human relationships across this vast region. The book sheds additional light on how dogs figure in the story of domestication, and how they have participated in partnerships with people across time. With contributions from a wide selection of authors, Dogs in the North is aimed at students and scholars of anthropology, archaeology, and history, as well as all those with interests in human-animal studies and northern societies.

This book is based upon more than two years of ethnographic fieldwork and personal experiences with the Teetl'it Gwich'in community in northern Canada. The author provides insight into Gwich'in understandings of life as well as into historical and political processes that have taken place in the North. He outlines the development of an educational approach towards conducting ethnography and writing anthropological literature, starting with the premise 'you have to live it'. The book focuses on ways of knowing and collaboration through learning and being taught by interlocutors. Building on the work of Tim Ingold, Loovers investigates the notion of reading life - land, water and weather as well as texts - and analyses the reading of texts as acts of conversations or correspondences.

This Element examines the development and current state of virtuosic rock guitar in terms of playing, technology, and culture. Supported by technological advances such as extended-range guitars, virtuosos in the 21st century are exploring ways to expand standard playing techniques in a climate expecting ever-higher levels of perfection. As musician-entrepreneurs, contemporary rock guitar virtuosos record, produce, and market their music themselves, operate equipment companies, and sell merchandise, tablature, and lessons online. For their social media channels, they regularly create videos and interact with their followers while having to balance building their tribe and finding the time to develop their craft to stay competitive. For a virtuoso, the working situations have changed considerably since the last century; the aloof rock star has been replaced by the approachable virtuoso-guitarist-composer-innovator-producer-promoter-YouTuber-teacher-entrepreneur.

Translated from the original German Lenin Neuentdecken and available in English for the first time, this volume rediscovers Lenin as a strategic socialist thinker through close examination of his collected works and correspondence. Brie opens with an analysis of Lenin's theoretical development between 1914 and 1917, in preparation for his critical decision to dissolve the Constituent Assembly in January 1918 in a struggle for power. This led from the dialectics of revolutionary practice and social analysis to a new understanding of socialism, which is compared and contrasted to the alternative Marxist ideas and conceptions of the state posited by Karl Kautsky and Rosa Luxemburg. Rediscovering Lenin then moves to 1921, when Lenin begins a new stage of his theoretical development concerned with resolving the reversal of the revolution's aims and its results. This process remains unfinished, and the questions raised a hundred years ago remain: How can one intervene successfully and responsibly in social and political crises? What role do social science theories, ideological frameworks, and other practices play in transforming the economic, political and cultural power structures of a society? Brie concludes with a retrospective on the ideas developed by Marx and in the Second International, and their impact on Lenin's strategic thinking. Placing Lenin's writing itself in the foreground and arguing from inside his own self-learning, Rediscovering Lenin focuses on the reflective relationship between ideology, theory, and practice.

This book deals with the issue of sustainable development in a novel and innovative way. It examines the governance implications of reflexive modernisation - the condition that societal development is endangered by its own side-effects. With conceptualising reflexive governance the book leads a way out of endless quarrels about the definition of sustainability and into a new mode of collective action. The authors assert that sustainability is not a defined end-state, but should be understood as the capacity of society to learn about the conditions of its future existence and wants. This requires, in their view, a specific kind of problem solving framework which emphasises the interlinkage of problems and scales, as well as long-term and indirect effects of various actions. Sustainability calls for new forms of governance with attention given to uncertainty, ambivalence about multiple goals and distributed power. The book develops an alternative framework with which to address the challenge of sustainability and derives a set of strategy elements for dealing with sustainability in practice. These are discussed from conceptual as well as practical perspectives. Bringing recent insights from innovation research, governance studies and complexity theory in common focus, Reflexive Governance for Sustainable Development will be of great interest to researchers in social change, innovation and governance studies, as well as policymakers confronted with sustainable development issues.

Courts and the complex phenomenon of the courtly society have received intensified interest in academic research over recent decades, however, the field of Islamic court culture has so far been overlooked. This book provides a comparative perspective on the history of courtly culture in Muslim societies from the earliest times to the nineteenth century, and presents an extensive collection of images of courtly life and architecture within the Muslim realm. The thematic methodology employed by the contributors underlines their interdisciplinary and comprehensive approach to issues of politics and patronage from across the Islamic world stretching from Cordoba to India. Themes range from the religious legitimacy of Muslim rulers, terminologies for court culture in Oriental languages, Muslim concepts of space for royal representation, accessibility of rulers, the role of royal patronage for Muslim scholars and artists to the growing influence of European courts as role models from the eighteenth century onwards. Discussing specific terminologies for courts in Oriental languages and explaining them to the non specialist, chapters describe the specific features of Muslim courts and point towards future research areas. As such, it fills this important gap in the existing literature in the areas of Islamic history, religion, and Islam in particular.

Courts and the complex phenomenon of the courtly society have received intensified interest in academic research over recent decades, however, the field of Islamic court culture has so far been overlooked. This book provides a comparative perspective on the history of courtly culture in Muslim societies from the earliest times to the nineteenth century, and presents an extensive collection of images of courtly life and architecture within the Muslim realm. The thematic methodology employed by the contributors underlines their interdisciplinary and comprehensive approach to issues of politics and patronage from across the Islamic world stretching from Cordoba to India. Themes range from the religious legitimacy of Muslim rulers, terminologies for court culture in Oriental languages, Muslim concepts of space for royal representation, accessibility of rulers, the role of royal patronage for Muslim scholars and artists to the growing influence of European courts as role models from the eighteenth century onwards. Discussing specific terminologies for courts in Oriental languages and explaining them to the non specialist, chapters describe the specific features of Muslim courts and point towards future research areas. As such, it fills this important gap in the existing literature in the areas of Islamic history, religion, and Islam in particular.

Sustainable development stirs up debate about the capacities of political steering and governance. The complexity of the task expounds limits of steering in three dimensions: goals, knowledge, and power: Sustainability goals are subject to changing and controversial risk perceptions, values and interests. Moreover, knowledge of the coupled dynamics of society, technology and nature is limited. Finally, the power to shape structural change in society and technology is distributed across a multitude of actors and societal subsystems. Steering attempts therefore have to cope with conflict and ambivalence, with uncertainty, and with a lack of central control; and they have to face the necessity of coordinating different actor groups and social networks. This volume explores steering strategies and governance arrangements for sustainable development with a view to these problem dimensions. The contributions by authors from various disciplines approach these challenges from different conceptual angles, ranging from positivist, managerial up to post-modern, constructivist perspectives. By combining theoretical reflections with insights from empirical research in European and American contexts, the volume maps out conditions and identifies approaches which both reflect the limits of steering and reveal options for constructively taking up the task of sustainable development in science and practice.

Since its beginnings more than fifty years ago, metal music has grown in popularity worldwide, not only as a musical culture but increasingly as a recognised field of study. This Cambridge Companion reflects the maturing field of 'metal music studies' by introducing the music and its cultures, as well as recent research perspectives from disciplines ranging from musicology and music technology to religious studies, Classics, and Scandinavian and African studies. Topics covered include technology and practice, identity and culture, modern metal genres, and global metal, with reference to performers including Black Sabbath, Metallica and Amon Amarth. Designed for students and their teachers, contributions explore the various musical styles and cultures of metal, providing an informative introduction for those new to the field and an up-to-date resource for readers familiar with the academic metal literature.

Sustainable development stirs up debate about the capacities of political steering and governance. The complexity of the task expounds limits of steering in three dimensions: goals, knowledge, and power: Sustainability goals are subject to changing and controversial risk perceptions, values and interests. Moreover, knowledge of the coupled dynamics of society, technology and nature is limited. Finally, the power to shape structural change in society and technology is distributed across a multitude of actors and societal subsystems. Steering attempts therefore have to cope with conflict and ambivalence, with uncertainty, and with a lack of central control; and they have to face the necessity of coordinating different actor groups and social networks. This volume explores steering strategies and governance arrangements for sustainable development with a view to these problem dimensions. The contributions by authors from various disciplines approach these challenges from different conceptual angles, ranging from positivist, managerial up to post-modern, constructivist perspectives. By combining theoretical reflections with insights from empirical research in European and American contexts, the volume maps out conditions and identifies approaches which both reflect the limits of steering and reveal options for constructively taking up the task of sustainable development in science and practice.