In the last few years, Algorithms for Convex Optimization have revolutionized algorithm design, both for discrete and continuous optimization problems. For problems like maximum flow, maximum matching, and submodular function minimization, the fastest algorithms involve essential methods such as gradient descent, mirror descent, interior point methods, and ellipsoid methods. The goal of this self-contained book is to enable researchers and professionals in computer science, data science, and machine learning to gain an in-depth understanding of these algorithms. The text emphasizes how to derive key algorithms for convex optimization from first principles and how to establish precise running time bounds. This modern text explains the success of these algorithms in problems of discrete optimization, as well as how these methods have significantly pushed the state of the art of convex optimization itself.

Many problems in statistics and econometrics offer themselves naturally to
the use of optimization heuristics. Standard methods applied to highly
complex problems often produce approximate results, of unknown quality,
based on heavy assumptions. Optimization heuristic methods provide
powerful results to many complex problems. combined with relatively
simple implementation.

• Offers a self-contained introduction to optimization heuristics in econometrics and statistics
• Features many examples of optimization heuristic methods applied to real problems
• Includes detailed coverage of the threshold accepting heuristic
• Provides suggestions for further reading

This book constitutes the conference proceedings of the 10th International Conference on Network Games, Control and Optimization, NETGCOOP 2020, held in Cargese, Corsica, France, in September 2021*.The 12 full papers and 16 short papers were carefully reviewed and selected from 44 submissions. The papers are organized in the following topical sections: game theory and iterative algorithms applied to wireless communication; stochastic models for network performance analysis; game theory in mobile and wireless networks; scheduling and resource allocation problems in networks; advance in game theory; social network; electrical network. * The conference was postponed to 2021 due to the COVID-19 pandemic.

Anja Schedel analyzes two models in the field of algorithmic game theory which both constitute bilevel problems in networks. The first model is a game-theoretic variant of the well-known Steiner forest problem, and one is interested in an optimal sharing of the cost of the Steiner forest. The author provides (and partially exactly characterizes) network structures which allow for cost-minimal pure Nash equilibria. The second model is motivated from privatized public roads, in which private, selfishly acting firms build roads, and as compensation for their investment, are allowed to set prices for using the roads. For a basic model of this situation, the author shows existence and uniqueness of pure Nash equilibria. The existence result requires a non-standard proof approach since techniques like Kakutani's fixed point theorem cannot be applied directly.

The goal of this book is to gather in a single work the most relevant concepts related in optimization methods, showing how such theories and methods can be addressed using the open source, multi-platform R tool. Modern optimization methods, also known as metaheuristics, are particularly useful for solving complex problems for which no specialized optimization algorithm has been developed. These methods often yield high quality solutions with a more reasonable use of computational resources (e.g. memory and processing effort). Examples of popular modern methods discussed in this book are: simulated annealing; tabu search; genetic algorithms; differential evolution; and particle swarm optimization. This book is suitable for undergraduate and graduate students in computer science, information technology, and related areas, as well as data analysts interested in exploring modern optimization methods using R. This new edition integrates the latest R packages through text and code examples. It also discusses new topics, such as: the impact of artificial intelligence and business analytics in modern optimization tasks; the creation of interactive Web applications; usage of parallel computing; and more modern optimization algorithms (e.g., iterated racing, ant colony optimization, grammatical evolution).

This book is one of the first to include an extensive discussion of integrated public transport planning. In times of growing urban populations and increasing environmental awareness, the importance of optimizing public transport systems is ever-developing. Three different aspects are presented: line planning, timetabling, and vehicle scheduling. Classically, challenges concerning these three aspects of planning are solved sequentially. Due to their high interdependence, the author presents a clear and detailed analysis of innovative, integrated models with accompanied numerical experiments performed to assess, and often support, the benefits of integration. The book will appeal to a wide readership ranging from graduate students to researchers.

This is a comprehensive study of various time-dependent scheduling problems in single-, parallel- and dedicated-machine environments. In addition to complexity issues and exact or heuristic algorithms which are typically presented in scheduling books, the author also includes more advanced topics such as matrix methods in time-dependent scheduling, time-dependent scheduling with two criteria and time-dependent two-agent scheduling. The reader should be familiar with the basic notions of calculus, discrete mathematics and combinatorial optimization theory, while the book offers introductory material on theory of algorithms, NP-complete problems, and the basics of scheduling theory. The author includes numerous examples, figures and tables, he presents different classes of algorithms using pseudocode, he completes all chapters with extensive bibliographies, and he closes the book with comprehensive symbol and subject indexes. The previous edition of the book focused on computational complexity of time-dependent scheduling problems. In this edition, the author concentrates on models of time-dependent job processing times and algorithms for solving time-dependent scheduling problems. The book is suitable for researchers working on scheduling, problem complexity, optimization, heuristics and local search algorithms.

This monograph collects cutting-edge results and techniques for solving nonlinear partial differential equations using critical points. Including many of the author's own contributions, a range of proofs are conveniently collected here, Because the material is approached with rigor, this book will serve as an invaluable resource for exploring recent developments in this active area of research, as well as the numerous ways in which critical point theory can be applied. Different methods for finding critical points are presented in the first six chapters. The specific situations in which these methods are applicable is explained in detail. Focus then shifts toward the book's main subject: applications to problems in mathematics and physics. These include topics such as Schroedinger equations, Hamiltonian systems, elliptic systems, nonlinear wave equations, nonlinear optics, semilinear PDEs, boundary value problems, and equations with multiple solutions. Readers will find this collection of applications convenient and thorough, with detailed proofs appearing throughout. Critical Point Theory will be ideal for graduate students and researchers interested in solving differential equations, and for those studying variational methods. An understanding of fundamental mathematical analysis is assumed. In particular, the basic properties of Hilbert and Banach spaces are used.

This book provides energy efficiency quantitative analysis and optimal methods for discrete manufacturing systems from the perspective of global optimization. In order to analyze and optimize energy efficiency for discrete manufacturing systems, it uses real-time access to energy consumption information and models of the energy consumption, and constructs an energy efficiency quantitative index system. Based on the rough set and analytic hierarchy process, it also proposes a principal component quantitative analysis and a combined energy efficiency quantitative analysis. In turn, the book addresses the design and development of quantitative analysis systems. To save energy consumption on the basis of energy efficiency analysis, it presents several optimal control strategies, including one for single-machine equipment, an integrated approach based on RWA-MOPSO, and one for production energy efficiency based on a teaching and learning optimal algorithm. Given its scope, the book offers a valuable guide for students, teachers, engineers and researchers in the field of discrete manufacturing systems.

This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming. The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization. Among the more important novel considerations presented are: the extension of the Hamilton-Jacobi-Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes; proof of semi-smoothness of the value function at degenerate points; attention to the under-selectivity issue for the long-run average and bias optimality; discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points. The book will be of interest to researchers and students in the field of stochastic control and performance optimization alike.

There are three fundamental components in Control-Flow Integrity (CFI) enforcement. The first component is accurately recovering the policy (CFG). Usually, the more precise the policy is, the more security CFI improves, but precise CFG generation was considered hard without the support of source code. The second component is embedding the CFI policy securely. Current CFI enforcement usually inserts checks before indirect branches to consult a read-only table which stores the valid CFG information. However, this kind of read-only table can be overwritten by some kinds of attacks (e.g., the Rowhammer attack and data-oriented programming). The third component is to efficiently enforce the CFI policy. In current approaches CFI checks are always executed whenever there is an indirect control flow transfer. Therefore, it is critical to minimize the performance impact of CFI checks. In this book, we propose novel solutions to handle these three fundamental components. To generate a precise CFI policy without the support of the source code, we systematically study two methods which recover CFI policy based on function signature matching at the binary level and propose our novel rule- and heuristic-based mechanism to more accurately recover function signature. To embed CFI policy securely, we design a novel platform which encodes the policy into the machine instructions directly without relying on consulting any read-only data structure, by making use of the idea of instruction-set randomization. Each basic block is encrypted with a key derived from the CFG. To efficiently enforce CFI policy, we make use of a mature dynamic code optimization platform called DynamoRIO to enforce the policy so that we are only required to do the CFI check when needed.

An up-close look at the theory behind and application of extremum seeking

Originally developed as a method of adaptive control for hard-to-model systems, extremum seeking solves some of the same problems as today's neural network techniques, but in a more rigorous and practical way. Following the resurgence in popularity of extremum-seeking control in aerospace and automotive engineering, Real-Time Optimization by Extremum-Seeking Control presents the theoretical foundations and selected applications of this method of real-time optimization.

Written by authorities in the field and pioneers in adaptive nonlinear control systems, this book presents both significant theoretic value and important practical potential. Filled with in-depth insight and expert advice, Real-Time Optimization by Extremum-Seeking Control:

• Develops optimization theory from the points of dynamic feedback and adaptation
• Builds a solid bridge between the classical optimization theory and modern feedback and adaptation techniques
• Provides a collection of useful tools for problems in this complex area
• Presents numerous applications of this powerful methodology
• Demonstrates the immense potential of this methodology for future theory development and applications

Real-Time Optimization by Extremum-Seeking Control is an important resource for both students and professionals in all areas of engineering-electrical, mechanical, aerospace, chemical, biomedical-and is also a valuable reference for practicing control engineers.

This book on optimization includes forewords by Michael I. Jordan, Zongben Xu and Zhi-Quan Luo. Machine learning relies heavily on optimization to solve problems with its learning models, and first-order optimization algorithms are the mainstream approaches. The acceleration of first-order optimization algorithms is crucial for the efficiency of machine learning. Written by leading experts in the field, this book provides a comprehensive introduction to, and state-of-the-art review of accelerated first-order optimization algorithms for machine learning. It discusses a variety of methods, including deterministic and stochastic algorithms, where the algorithms can be synchronous or asynchronous, for unconstrained and constrained problems, which can be convex or non-convex. Offering a rich blend of ideas, theories and proofs, the book is up-to-date and self-contained. It is an excellent reference resource for users who are seeking faster optimization algorithms, as well as for graduate students and researchers wanting to grasp the frontiers of optimization in machine learning in a short time.

This book covers not only foundational materials but also the most recent progresses made during the past few years on the area of machine learning algorithms. In spite of the intensive research and development in this area, there does not exist a systematic treatment to introduce the fundamental concepts and recent progresses on machine learning algorithms, especially on those based on stochastic optimization methods, randomized algorithms, nonconvex optimization, distributed and online learning, and projection free methods. This book will benefit the broad audience in the area of machine learning, artificial intelligence and mathematical programming community by presenting these recent developments in a tutorial style, starting from the basic building blocks to the most carefully designed and complicated algorithms for machine learning.

Two approaches are known for solving large-scale unconstrained optimization problems-the limited-memory quasi-Newton method (truncated Newton method) and the conjugate gradient method. This is the first book to detail conjugate gradient methods, showing their properties and convergence characteristics as well as their performance in solving large-scale unconstrained optimization problems and applications. Comparisons to the limited-memory and truncated Newton methods are also discussed. Topics studied in detail include: linear conjugate gradient methods, standard conjugate gradient methods, acceleration of conjugate gradient methods, hybrid, modifications of the standard scheme, memoryless BFGS preconditioned, and three-term. Other conjugate gradient methods with clustering the eigenvalues or with the minimization of the condition number of the iteration matrix, are also treated. For each method, the convergence analysis, the computational performances and the comparisons versus other conjugate gradient methods are given. The theory behind the conjugate gradient algorithms presented as a methodology is developed with a clear, rigorous, and friendly exposition; the reader will gain an understanding of their properties and their convergence and will learn to develop and prove the convergence of his/her own methods. Numerous numerical studies are supplied with comparisons and comments on the behavior of conjugate gradient algorithms for solving a collection of 800 unconstrained optimization problems of different structures and complexities with the number of variables in the range [1000,10000]. The book is addressed to all those interested in developing and using new advanced techniques for solving unconstrained optimization complex problems. Mathematical programming researchers, theoreticians and practitioners in operations research, practitioners in engineering and industry researchers, as well as graduate students in mathematics, Ph.D. and master students in mathematical programming, will find plenty of information and practical applications for solving large-scale unconstrained optimization problems and applications by conjugate gradient methods.

This book serves as an introductory text to optimization theory in normed spaces and covers all areas of nonlinear optimization. It presents fundamentals with particular emphasis on the application to problems in the calculus of variations, approximation and optimal control theory. The reader is expected to have a basic knowledge of linear functional analysis.

This book presents select peer-reviewed papers presented at the International Conference on Numerical Optimization in Engineering and Sciences (NOIEAS) 2019. The book covers a wide variety of numerical optimization techniques across all major engineering disciplines like mechanical, manufacturing, civil, electrical, chemical, computer, and electronics engineering. The major focus is on innovative ideas, current methods and latest results involving advanced optimization techniques. The contents provide a good balance between numerical models and analytical results obtained for different engineering problems and challenges. This book will be useful for students, researchers, and professionals interested in engineering optimization techniques.

Understand common scheduling as well as other advanced operational problems with this valuable reference from a recognized leader in the field. Beginning with basic principles and an overview of linear and mixed-integer programming, this unified treatment introduces the fundamental ideas underpinning most modeling approaches, and will allow you to easily develop your own models. With more than 150 figures, the basic concepts and ideas behind the development of different approaches are clearly illustrated. Addresses a wide range of problems arising in diverse industrial sectors, from oil and gas to fine chemicals, and from commodity chemicals to food manufacturing. A perfect resource for engineering and computer science students, researchers working in the area, and industrial practitioners.

The book is intended for graduate students and researchers in mathematics, computer science, and operational research. The book presents a new derivative-free optimization method/algorithm based on randomly generated trial points in specified domains and where the best ones are selected at each iteration by using a number of rules. This method is different from many other well established methods presented in the literature and proves to be competitive for solving many unconstrained optimization problems with different structures and complexities, with a relative large number of variables. Intensive numerical experiments with 140 unconstrained optimization problems, with up to 500 variables, have shown that this approach is efficient and robust. Structured into 4 chapters, Chapter 1 is introductory. Chapter 2 is dedicated to presenting a two level derivative-free random search method for unconstrained optimization. It is assumed that the minimizing function is continuous, lower bounded and its minimum value is known. Chapter 3 proves the convergence of the algorithm. In Chapter 4, the numerical performances of the algorithm are shown for solving 140 unconstrained optimization problems, out of which 16 are real applications. This shows that the optimization process has two phases: the reduction phase and the stalling one. Finally, the performances of the algorithm for solving a number of 30 large-scale unconstrained optimization problems up to 500 variables are presented. These numerical results show that this approach based on the two level random search method for unconstrained optimization is able to solve a large diversity of problems with different structures and complexities. There are a number of open problems which refer to the following aspects: the selection of the number of trial or the number of the local trial points, the selection of the bounds of the domains where the trial points and the local trial points are randomly generated and a criterion for initiating the line search.

This book provides a literature review of techniques used to pass from continuous to combinatorial space, before discussing a detailed example with individual steps of how cuckoo search (CS) can be adapted to solve combinatorial optimization problems. It demonstrates the application of CS to three different problems and describes their source code. The content is divided into five chapters, the first of which provides a technical description, together with examples of combinatorial search spaces. The second chapter summarizes a diverse range of methods used to solve combinatorial optimization problems. In turn, the third chapter presents a description of CS, its formulation and characteristics. In the fourth chapter, the application of discrete cuckoo search (DCS) to solve three POCs (the traveling salesman problem, quadratic assignment problem and job shop scheduling problem) is explained, focusing mainly on a reinterpretation of the terminology used in CS and its source of inspiration. In closing, the fifth chapter discusses random-key cuckoo search (RKCS) using random keys to represent positions found by cuckoo search in the TSP and QAP solution space.

This book addresses the frontier advances in the theory and application of nature-inspired optimization techniques, including solving the quadratic assignment problem, prediction in nature-inspired dynamic optimization, the lion algorithm and its applications, optimizing the operation scheduling of microgrids, PID controllers for two-legged robots, optimizing crane operating times, planning electrical energy distribution systems, automatic design and evaluation of classification pipelines, and optimizing wind-energy power generation plants. The book also presents a variety of nature-inspired methods and illustrates methods of adapting these to said applications. Nature-inspired computation, developed by mimicking natural phenomena, makes a significant contribution toward the solution of non-convex optimization problems that normal mathematical optimizers fail to solve. As such, a wide range of nature-inspired computing approaches has been used in multidisciplinary engineering applications. Written by researchers and developers from a variety of fields, this book presents the latest findings, novel techniques and pioneering applications.

Solving nonsmooth optimization (NSO) problems is critical in many practical applications and real-world modeling systems. The aim of this book is to survey various numerical methods for solving NSO problems and to provide an overview of the latest developments in the field. Experts from around the world share their perspectives on specific aspects of numerical NSO. The book is divided into four parts, the first of which considers general methods including subgradient, bundle and gradient sampling methods. In turn, the second focuses on methods that exploit the problem's special structure, e.g. algorithms for nonsmooth DC programming, VU decomposition techniques, and algorithms for minimax and piecewise differentiable problems. The third part considers methods for special problems like multiobjective and mixed integer NSO, and problems involving inexact data, while the last part highlights the latest advancements in derivative-free NSO. Given its scope, the book is ideal for students attending courses on numerical nonsmooth optimization, for lecturers who teach optimization courses, and for practitioners who apply nonsmooth optimization methods in engineering, artificial intelligence, machine learning, and business. Furthermore, it can serve as a reference text for experts dealing with nonsmooth optimization.

This book collects some basic results on the null controllability for degenerate and singular parabolic problems. It aims to provide postgraduate students and senior researchers with a useful text, where they can find the desired statements and the related bibliography. For these reasons, the authors will not give all the detailed proofs of the given theorems, but just some of them, in order to show the underlying strategy in this area.

This proceedings presents the result of the 8th International Conference in Network Analysis, held at the Higher School of Economics, Moscow, in May 2018. The conference brought together scientists, engineers, and researchers from academia, industry, and government. Contributions in this book focus on the development of network algorithms for data mining and its applications. Researchers and students in mathematics, economics, statistics, computer science, and engineering find this collection a valuable resource filled with the latest research in network analysis. Computational aspects and applications of large-scale networks in market models, neural networks, social networks, power transmission grids, maximum clique problem, telecommunication networks, and complexity graphs are included with new tools for efficient network analysis of large-scale networks. Machine learning techniques in network settings including community detection, clustering, and biclustering algorithms are presented with applications to social network analysis.

This volume covers contemporary aspects of geometric measure theory with a focus on applications to partial differential equations, free boundary problems and water waves. It is based on lectures given at the 2019 CIME summer school "Geometric Measure Theory and Applications - From Geometric Analysis to Free Boundary Problems" which took place in Cetraro, Italy, under the scientific direction of Matteo Focardi and Emanuele Spadaro. Providing a description of the structure of measures satisfying certain differential constraints, and covering regularity theory for Bernoulli type free boundary problems and water waves as well as regularity theory for the obstacle problems and the developments leading to applications to the Stefan problem, this volume will be of interest to students and researchers in mathematical analysis and its applications.