This handbook presents state-of-the-art research in reinforcement learning, focusing on its applications in the control and game theory of dynamic systems and future directions for related research and technology. The contributions gathered in this book deal with challenges faced when using learning and adaptation methods to solve academic and industrial problems, such as optimization in dynamic environments with single and multiple agents, convergence and performance analysis, and online implementation. They explore means by which these difficulties can be solved, and cover a wide range of related topics including: deep learning; artificial intelligence; applications of game theory; mixed modality learning; and multi-agent reinforcement learning. Practicing engineers and scholars in the field of machine learning, game theory, and autonomous control will find the Handbook of Reinforcement Learning and Control to be thought-provoking, instructive and informative.

A field bus is a two-way link between a programmable controller or operations monitor and an industrial device like a sensor, an electric motor, or a switch. It is a critical part of any automated industrial process - whether for factory automation (discrete processes like an assembly line) or process automation (continuous flow of materials being mixed, treated, or processed). PROFIBUS is a widely established program that allows for communication among and between controllers, fieldbuses, and actuator devices. This very concise introduction for industrial engineers, controls engineers, and manufacturing technicians covers the basics of field bus architecture and communication and the fundamentals of the PROFIBUS language protocol.

This book establishes an important mathematical connection between cooperative control problems and network optimization problems. It shows that many cooperative control problems can in fact be understood, under certain passivity assumptions, using a pair of static network optimization problems. Merging notions from passivity theory and network optimization, it describes a novel network optimization approach that can be applied to the synthesis of controllers for diffusively-coupled networks of passive (or passivity-short) dynamical systems. It also introduces a data-based, model-free approach for the synthesis of network controllers for multi-agent systems with passivity-short agents. Further, the book describes a method for monitoring link faults in multi-agent systems using passivity theory and graph connectivity. It reports on some practical case studies describing the effectivity of the developed approaches in vehicle networks. All in all, this book offers an extensive source of information and novel methods in the emerging field of multi-agent cooperative control, paving the way to future developments of autonomous systems for various application domains

This book presents the select peer-reviewed proceedings of the International Conference on Futuristic Trends in Mechanical Engineering (ICOFTIME 2020). The contents focus on latest research in different areas of mechanical engineering such as additive manufacturing, vibrations, robotics and automation, nano and smart materials, green energy, supply chain management, aviation, and biomechanics. The book also includes numerical and optimization methods relevant for several real-life mechanical engineering problems. Given its contents, this book will prove useful for researchers and professionals alike.

This book presents recent results on fault diagnosis and condition monitoring of airborne electromechanical actuators, illustrating both algorithmic and hardware design solutions to enhance the reliability of onboard more electric aircraft. The book begins with an introduction to the current trends in the development of electrically powered actuation systems for aerospace applications. Practical examples are proposed to help present approaches to reliability, availability, maintainability and safety analysis of airborne equipment. The terminology and main strategies for fault diagnosis and condition monitoring are then reviewed. The core of the book focuses on the presentation of relevant case studies of fault diagnosis and monitoring design for airborne electromechanical actuators, using different techniques. The last part of the book is devoted to a summary of lessons learned and practical suggestions for the design of fault diagnosis solutions of complex airborne systems. The book is written with the idea of providing practical guidelines on the development of fault diagnosis and monitoring algorithms for airborne electromechanical actuators. It will be of interest to practitioners in aerospace, mechanical, electronic, reliability and systems engineering, as well as researchers and postgraduates interested in dynamical systems, automatic control and safety-critical systems. Advances in Industrial Control reports and encourages the transfer of technology in control engineering. The rapid development of control technology has an impact on all areas of the control discipline. The series offers an opportunity for researchers to present an extended exposition of new work in all aspects of industrial control.

This book is the first comprehensive book about reservoir computing (RC). RC is a powerful and broadly applicable computational framework based on recurrent neural networks. Its advantages lie in small training data set requirements, fast training, inherent memory and high flexibility for various hardware implementations. It originated from computational neuroscience and machine learning but has, in recent years, spread dramatically, and has been introduced into a wide variety of fields, including complex systems science, physics, material science, biological science, quantum machine learning, optical communication systems, and robotics. Reviewing the current state of the art and providing a concise guide to the field, this book introduces readers to its basic concepts, theory, techniques, physical implementations and applications. The book is sub-structured into two major parts: theory and physical implementations. Both parts consist of a compilation of chapters, authored by leading experts in their respective fields. The first part is devoted to theoretical developments of RC, extending the framework from the conventional recurrent neural network context to a more general dynamical systems context. With this broadened perspective, RC is not restricted to the area of machine learning but is being connected to a much wider class of systems. The second part of the book focuses on the utilization of physical dynamical systems as reservoirs, a framework referred to as physical reservoir computing. A variety of physical systems and substrates have already been suggested and used for the implementation of reservoir computing. Among these physical systems which cover a wide range of spatial and temporal scales, are mechanical and optical systems, nanomaterials, spintronics, and quantum many body systems. This book offers a valuable resource for researchers (Ph.D. students and experts alike) and practitioners working in the field of machine learning, artificial intelligence, robotics, neuromorphic computing, complex systems, and physics.

This volume gathers the latest advances, innovations and applications in the field of cable robots, as presented by leading international researchers and engineers at the 5th International Conference on Cable-Driven Parallel Robots (CableCon 2021), held as virtual event on July 7-9, 2021. It covers the theory and applications of cable-driven parallel robots, including their classification, kinematics and singularity analysis, workspace, statics and dynamics, cable modeling and technologies, control and calibration, design methodologies, hardware development, experimental evaluation and prototypes, as well as application reports and new application concepts. The contributions, which were selected through a rigorous international peer-review process, share exciting ideas that will spur novel research directions and foster new multidisciplinary collaborations.

This volume gathers the latest advances, innovations and applications in the field of vibration and technology of machinery, as presented by leading international researchers and engineers at the XV International Conference on Vibration Engineering and Technology of Machinery (VETOMAC), held in Curitiba, Brazil on November 10-15, 2019. Topics include concepts and methods in dynamics, dynamics of mechanical and structural systems, dynamics and control, condition monitoring, machinery and structural dynamics, rotor dynamics, experimental techniques, finite element model updating, industrial case studies, vibration control and energy harvesting, and MEMS. The contributions, which were selected through a rigorous international peer-review process, share exciting ideas that will spur novel research directions and foster new multidisciplinary collaborations.

This monograph presents innovative research regarding the body experience of human individuals who are using assistive robotic devices such as wearable robots or teleoperation systems. The focus is set on human-in-the-loop experiments that help to empirically evaluate how users experience devices. Moreover, these experiments allow for further examination of the underlying mechanisms of body experience through extending existing psychological paradigms, e.g., by disentangling tactile feedback from contacts. Besides reporting and discussing psychological examinations, the influence of various aspects of engineering design is investigated, e.g., different implementations of haptic interfaces or robot control. As haptics are of paramount importance in this tight type of human-robot interaction, it is explored with respect to modality as well as temporal and spatial effects. The first part of the book motivates the research topic and gives an in-depth analysis of the experimental requirements. The second and third part present experimental designs and studies of human-robot body experience regarding the upper and lower limbs as well as cognitive models to predict them. The fourth part discusses a multitude of design considerations and provides directions to guide future research on bidirectional human-machine interfaces and non-functional haptic feedback.

The past few years have seen the attention and rapid developments in event-triggered sampled-data systems, in which the effect of event-triggered sensor measurements and controller updates is explored in controller analysis and design. This book offers the first systematic treatment of event-triggered sampled-data control system design using active disturbance rejection control (ADRC), an effective approach that is popular in both theoretic research and industrial applications. Extensive application examples with numerous illustrations are included to show how the event-triggered ADRC with theoretic performance guarantees can be implemented in engineering systems and how the performance can be actually achieved. For theoretic researchers and graduate students, the presented results provide new directions in theoretic research on event-triggered sampled-data systems; for control practitioners, the book offers an effective approach to achieving satisfactory performance with limited sampling rates.

This book presents mathematical models of demand-side management programs, together with operational and control problems for power and renewable energy systems. It reflects the need for optimal operation and control of today's electricity grid at both the supply and demand spectrum of the grid. This need is further compounded by the advent of smart grids, which has led to increased customer/consumer participation in power and renewable energy system operations. The book begins by giving an overview of power and renewable energy systems, demand-side management programs and algebraic modeling languages. The overview includes detailed consideration of appliance scheduling algorithms, price elasticity matrices and demand response incentives. Furthermore, the book presents various power system operational and control mathematical formulations, incorporating demand-side management programs. The mathematical formulations developed are modeled and solved using the Advanced Interactive Multidimensional Modeling System (AIMMS) software, which offers a powerful yet simple algebraic modeling language for solving optimization problems. The book is extremely useful for all power system operators and planners who are concerned with optimal operational procedures for managing today's complex grids, a context in which customers are active participants and can curb/control their demand. The book details how AIMMS can be a useful tool in optimizing power grids and also offers a valuable research aid for students and academics alike.

This book provides a thorough guide to the use of numerical methods in energy systems and applications. It presents methods for analysing engineering applications for energy systems, discussing finite difference, finite element, and other advanced numerical methods. Solutions to technical problems relating the application of these methods to energy systems are also thoroughly explored. Readers will discover diverse perspectives of the contributing authors and extensive discussions of issues including: * a wide variety of numerical methods concepts and related energy systems applications;* systems equations and optimization, partial differential equations, and finite difference method;* methods for solving nonlinear equations, special methods, and their mathematical implementation in multi-energy sources;* numerical investigations of electrochemical fields and devices; and* issues related to numerical approaches and optimal integration of energy consumption. This is a highly informative and carefully presented book, providing scientific and academic insight for readers with an interest in numerical methods and energy systems.

The manufacturing industry has been optimized in recent years due to the rise of new technologies. These advances have paved the way for the development of intelligent vehicles. Intelligent Vehicles and Materials Transportation in the Manufacturing Sector: Emerging Research and Opportunities is a pivotal source of scholarly research on the various aspects of manufacturing vehicles with intelligent technology components. Including a range of perspectives on topics such as material handling, automated guided vehicles, and industrial robots, this book is ideally designed for engineers, academics, professionals, and practitioners actively involved in the manufacturing sector.

This book shares the latest findings on this topic, systematically introduces readers to advances made in robotic harvesting around the globe, and explores the relations between the development of robotic harvesting and the respective social/economic conditions and agricultural business patterns in various countries/regions. Due to the unstructured setting it is used in, and to the significant differences between individual fruit and vegetable targets, robotic harvesting is currently considered to be one of the most challenging robotics technologies. Accordingly, research into this area involves the integration of various aspects, including biomechanics, optimization design, advanced perception and intelligent control. In addition to rapid and damage-free robotic harvesting, which reflects the multidisciplinary nature of the topic, further aspects addressed include gripping collisions with viscoelastic objects, using lasers to cut plant material, plant-fruit response to vacuum sucking and pulling, and performance probability distribution. Highlighting outstanding innovations and reflecting the latest advances in intelligent agricultural equipment in China, the book offers a unique and valuable resource.

This book examines the problems in the field of energy and related fields (chemical, transport, aerospace, construction, metallurgy, engineering, etc.) and consists of 4 subsections: Electrical Engineering, Heat Power Engineering, Cybersecurity and Computer Science & Environmental Safety. In the first section, authors pay attention to contemporary issues related to the development of the electric power industry, electrical engineering, the physics of electrical phenomena and renewable energy sources (such as solar energy and wind energy). The second section is devoted to modern problems in heat power engineering and considers modern means and methods that increase the efficiency and reliability of the functioning of heat power facilities. The third section is devoted to issues of cybersecurity of critical facilities, in particular energy facilities, as well as the development of computer science and the introduction of modern information and measurement systems in the energy sector. The fourth subsection deals with the problems of rational use of natural resources, accounting for emissions of harmful substances, environmental issues at energy facilities, as well as the development of a methodology for environmental safety. The book includes 21 chapters. A book is for researchers, engineers, as well as lecturers and postgraduates of higher education institutions dealing with issues of control, diagnosis and monitoring of energy facilities.

Force and Position Control of Mechatronic Systems provides an overview of the general concepts and technologies in the area of force and position control. Novel ideas and innovations related to this area are presented and reported in detail, and examples of applications in medical technology are given. The book begins by introducing force sensing, and modelling of contacting objects. In then moves steadily through a variety of topics, including: * disturbance observer-based force estimation; * force-based supervisory control; * stabilization systems; * controller design; and * control of tube insertion procedures. This book will be of interest to researchers, engineers and students interested in force control, particularly those with a focus on medical applications of these ideas. Advances in Industrial Control reports and encourages the transfer of technology in control engineering. The rapid development of control technology has an impact on all areas of the control discipline. The series offers an opportunity for researchers to present an extended exposition of new work in all aspects of industrial control.

This book focuses on how to implement optimal control problems via the variational method. It studies how to implement the extrema of functional by applying the variational method and covers the extrema of functional with different boundary conditions, involving multiple functions and with certain constraints etc. It gives the necessary and sufficient condition for the (continuous-time) optimal control solution via the variational method, solves the optimal control problems with different boundary conditions, analyzes the linear quadratic regulator & tracking problems respectively in detail, and provides the solution of optimal control problems with state constraints by applying the Pontryagin's minimum principle which is developed based upon the calculus of variations. And the developed results are applied to implement several classes of popular optimal control problems and say minimum-time, minimum-fuel and minimum-energy problems and so on. As another key branch of optimal control methods, it also presents how to solve the optimal control problems via dynamic programming and discusses the relationship between the variational method and dynamic programming for comparison. Concerning the system involving individual agents, it is also worth to study how to implement the decentralized solution for the underlying optimal control problems in the framework of differential games. The equilibrium is implemented by applying both Pontryagin's minimum principle and dynamic programming. The book also analyzes the discrete-time version for all the above materials as well since the discrete-time optimal control problems are very popular in many fields.

This book presents several novel constructive methodologies for global stabilization and H-infinity control in switched dynamic systems by using the systems' structure information. The main features of these new approaches are twofold: i) Novel Lyapunov functions are constructed and new switching strategies are designed to guarantee global finite-time stabilization of the closed-loop switched dynamic systems,while ii) without posing any internal stability requirements on subsystems, the standard H-infinity control problem of the switched dynamic systems is solved by means of dwell-time switching techniques. Systematically presenting constructive methods for analyzing and synthesizing switched systems, the content is of great significance to theoretical research and practical applications involving switched systems alike. The book provides a unified framework for stability analysis, stabilization and H-infinity control of switched systems, making it a valuable resource for researchers and graduate students who want to learn about the state of the art in the analysis and synthesis of switched systems, as well as recent advances in switched linear systems. In addition, it offers a wealth of cutting-edge constructive methods and algorithm designs for researchers who work with switched dynamic systems and graduate students of control theory and control engineering.

This unique textbook comprehensively introduces the field of discrete event systems, offering a breadth of coverage that makes the material accessible to readers of varied backgrounds. The book emphasizes a unified modeling framework that transcends specific application areas, linking the following topics in a coherent manner: language and automata theory, supervisory control, Petri net theory, Markov chains and queueing theory, discrete-event simulation, and concurrent estimation techniques. Topics and features: detailed treatment of automata and language theory in the context of discrete event systems, including application to state estimation and diagnosis comprehensive coverage of centralized and decentralized supervisory control of partially-observed systems timed models, including timed automata and hybrid automata stochastic models for discrete event systems and controlled Markov chains discrete event simulation an introduction to stochastic hybrid systems sensitivity analysis and optimization of discrete event and hybrid systems new in the third edition: opacity properties, enhanced coverage of supervisory control, overview of latest software tools This proven textbook is essential to advanced-level students and researchers in a variety of disciplines where the study of discrete event systems is relevant: control, communications, computer engineering, computer science, manufacturing engineering, transportation networks, operations research, and industrial engineering. Christos G. Cassandras is Distinguished Professor of Engineering, Professor of Systems Engineering, and Professor of Electrical and Computer Engineering at Boston University. Stephane Lafortune is Professor of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor.

This book tackles the latest research trends in technology acceptance models and theories. It presents high-quality empirical and review studies focusing on the main theoretical models and their applications across various technologies and contexts. It also provides insights into the theoretical and practical aspects of different technological innovations that assist decision-makers in formulating the required policies and procedures for adopting a specific technology.

This book introduces the technological innovations of robotic vehicles. It presents the concepts required for self-driving cars on the road. Besides, readers can gain invaluable knowledge in the construction, programming, and control of the six-legged robot. The book also presents the controllers and aerodynamics of several different types of rotorcrafts. It includes the simulation and flight of the various kinds of rotor-propelled air vehicles under each of their different aerodynamics environment. The book is suitable for academia, educators, students, and researchers who are interested in autonomous vehicles, robotics, and rotor-propelled vehicles.

Since the 1980s, scientists have been researching adaptive structures for materials, for multifunctional elements or even for complete systems. Adaptronics (smart materials, smart structures, smart systems) is a field of distinct interdisciplinarity. The book therefore offers an interdisciplinary view of adaptronic systems, materials and functional elements and their applications. The subject matter integrates various engineering disciplines, from electrical engineering and information technology to manufacturing and control engineering, materials engineering and structural mechanics - to name but a few of the relevant subject areas. Starting from the basic principles and variants of adaptronic systems and functional materials, the textbook explains the different construction methods of functional elements. Building on this, readers learn how to apply this knowledge to active shape control, active vibration control and active vibroacoustics. For each of these topics the author presents current examples from research, discusses research results and future research questions. Each of the nine chapters closes with references to further literature. An index of the mathematical symbols used and a keyword index facilitate learning for readers.The book is aimed at Master's students in engineering courses such as mechanical engineering, aerospace engineering, mechatronics, automotive engineering and related courses. The book provides a comprehensive overview for industrial practitioners who want to familiarize themselves with the field of adaptronics and also serves as a reliable reference book.

This book is about algebraic and differential methods, as well as fractional calculus, applied to diagnose and reject faults in nonlinear systems, which are of integer or fractional order. This represents an extension of a very important and widely studied problem in control theory, namely fault diagnosis and rejection (using differential algebraic approaches), to systems presenting fractional dynamics, i.e. systems whose dynamics are represented by derivatives and integrals of non-integer order. The authors offer a thorough overview devoted to fault diagnosis and fault-tolerant control applied to fractional-order and integer-order dynamical systems, and they introduce new methodologies for control and observation described by fractional and integer models, together with successful simulations and real-time applications. The basic concepts and tools of mathematics required to understand the methodologies proposed are all clearly introduced and explained. Consequently, the book is useful as supplementary reading in courses of applied mathematics and nonlinear control theory. This book is meant for engineers, mathematicians, physicists and, in general, to researchers and postgraduate students in diverse areas who have a minimum knowledge of calculus. It also contains advanced topics for researchers and professionals interested in the area of states and faults estimation.