Take a journey toward discovering, learning, and using Apache Spark 3.0. In this book, you will gain expertise on the powerful and efficient distributed data processing engine inside of Apache Spark; its user-friendly, comprehensive, and flexible programming model for processing data in batch and streaming; and the scalable machine learning algorithms and practical utilities to build machine learning applications. Beginning Apache Spark 3 begins by explaining different ways of interacting with Apache Spark, such as Spark Concepts and Architecture, and Spark Unified Stack. Next, it offers an overview of Spark SQL before moving on to its advanced features. It covers tips and techniques for dealing with performance issues, followed by an overview of the structured streaming processing engine. It concludes with a demonstration of how to develop machine learning applications using Spark MLlib and how to manage the machine learning development lifecycle. This book is packed with practical examples and code snippets to help you master concepts and features immediately after they are covered in each section. After reading this book, you will have the knowledge required to build your own big data pipelines, applications, and machine learning applications. What You Will Learn Master the Spark unified data analytics engine and its various components Work in tandem to provide a scalable, fault tolerant and performant data processing engine Leverage the user-friendly and flexible programming model to perform simple to complex data analytics using dataframe and Spark SQL Develop machine learning applications using Spark MLlib Manage the machine learning development lifecycle using MLflow Who This Book Is For Data scientists, data engineers and software developers.

The three-volume set CCIS 1467, CCIS 1468, and CCIS 1469 constitutes the thoroughly refereed proceedings of the 7th International Conference on Life System Modeling and Simulation, LSMS 2021, and of the 7th International Conference on Intelligent Computing for Sustainable Energy and Environment, ICSEE 2021, held in Hangzhou, China, in October 2021. The 159 revised papers presented were carefully reviewed and selected from over 430 submissions. The papers of this volume are organized in topical sections on: Medical Imaging and Analysis Using Intelligence Computing; Biomedical signal processing, imaging, visualization and surgical robotics; Computational method in taxonomy study and neural dynamics; Intelligent medical apparatus, clinical applications and intelligent design of biochips; Power and Energy Systems; Computational Intelligence in Utilization of Clean and Renewable Energy Resources, and Intelligent Modelling, Control and Supervision for Energy Saving and Pollution Reduction; Intelligent Methods in Developing Electric Vehicles, Engines and Equipment; Intelligent Control Methods in Energy Infrastructure Development and Distributed Power Generation Systems; Intelligent Modeling, Simulation and Control of Power Electronics and Power Networks; Intelligent Techniques for Sustainable Energy and Green Built Environment, Water Treatment and Waste Management; Intelligent Robot and Simulation; Intelligent Data Processing, Analysis and Control in Complex Systems; Advanced Neural Network Theory and Algorithms; Advanced Computational Methods and Applications; Fuzzy, Neural, and Fuzzy-neuro Hybrids; Intelligent Modelling, Monitoring, and Control of Complex Nonlinear Systems; Intelligent manufacturing, autonomous systems, intelligent robotic systems; Computational Intelligence and Applications.

This book provides an introduction to next generation smart screening technology for medical image analysis that combines artificial intelligence (AI) techniques with digital screening to develop innovative methods for detecting breast cancer. The authors begin with a discussion of breast cancer, its characteristics and symptoms, and the importance of early screening.They then provide insight on the role of artificial intelligence in global healthcare, screening methods for breast cancer using mammogram, ultrasound, and thermogram images, and the potential benefits of using AI-based systems for clinical screening to more accurately detect, diagnose, and treat breast cancer. Discusses various existing screening methods for breast cancer Presents deep information on artificial intelligence-based screening methods Discusses cancer treatment based on geographical differences and cultural characteristics

The book first explores the cybersecurity's landscape and the inherent susceptibility of online communication system such as e-mail, chat conversation and social media in cybercrimes. Common sources and resources of digital crimes, their causes and effects together with the emerging threats for society are illustrated in this book. This book not only explores the growing needs of cybersecurity and digital forensics but also investigates relevant technologies and methods to meet the said needs. Knowledge discovery, machine learning and data analytics are explored for collecting cyber-intelligence and forensics evidence on cybercrimes. Online communication documents, which are the main source of cybercrimes are investigated from two perspectives: the crime and the criminal. AI and machine learning methods are applied to detect illegal and criminal activities such as bot distribution, drug trafficking and child pornography. Authorship analysis is applied to identify the potential suspects and their social linguistics characteristics. Deep learning together with frequent pattern mining and link mining techniques are applied to trace the potential collaborators of the identified criminals. Finally, the aim of the book is not only to investigate the crimes and identify the potential suspects but, as well, to collect solid and precise forensics evidence to prosecute the suspects in the court of law.

This book constitutes the thoroughly refereed papers of the 4th International Conference on Applied Informatics, ICAI 2021, held in Buenos Aires, Argentina, in October, 2021.The 35 full papers were carefully reviewed and selected from 89 submissions. The papers are organized in topical sections on artificial intelligence; data analysis; decision systems; health care information systems; image processing; security services; simulation and emulation; smart cities; software and systems modeling; software design engineering.

Algorithms are now widely employed to make decisions that have increasingly far-reaching impacts on individuals and society as a whole ("algorithmic governance"), which could potentially lead to manipulation, biases, censorship, social discrimination, violations of privacy, property rights, and more. This has sparked a global debate on how to regulate AI and robotics ("governance of algorithms"). This book discusses both of these key aspects: the impact of algorithms, and the possibilities for future regulation.

This book describes new theories and applications of artificial neural networks, with a special focus on answering questions in neuroscience, biology and biophysics and cognitive research. It covers a wide range of methods and technologies, including deep neural networks, large scale neural models, brain computer interface, signal processing methods, as well as models of perception, studies on emotion recognition, self-organization and many more. The book includes both selected and invited papers presented at the XXII International Conference on Neuroinformatics, held on October 12-16, 2020, Moscow, Russia.

This text provides deep and comprehensive coverage of the mathematical background for data science, including machine learning, optimal recovery, compressed sensing, optimization, and neural networks. In the past few decades, heuristic methods adopted by big tech companies have complemented existing scientific disciplines to form the new field of Data Science. This text embarks the readers on an engaging itinerary through the theory supporting the field. Altogether, twenty-seven lecture-length chapters with exercises provide all the details necessary for a solid understanding of key topics in data science. While the book covers standard material on machine learning and optimization, it also includes distinctive presentations of topics such as reproducing kernel Hilbert spaces, spectral clustering, optimal recovery, compressed sensing, group testing, and applications of semidefinite programming. Students and data scientists with less mathematical background will appreciate the appendices that provide more background on some of the more abstract concepts.

This book presents high-quality papers from the Fourth International Conference on Microelectronics, Computing & Communication Systems (MCCS 2019). It discusses the latest technological trends and advances in MEMS and nanoelectronics, wireless communication, optical communication, instrumentation, signal processing, image processing, bioengineering, green energy, hybrid vehicles, environmental science, weather forecasting, cloud computing, renewable energy, RFID, CMOS sensors, actuators, transducers, telemetry systems, embedded systems and sensor network applications. It includes papers based on original theoretical, practical and experimental simulations, development, applications, measurements and testing. The applications and solutions discussed here provide excellent reference material for future product development.

This book features selected papers presented at the International Conference on Information Management and Machine Intelligence (ICIMMI 2019), held at the Poornima Institute of Engineering & Technology, Jaipur, Rajasthan, India, on December 14-15, 2019. It covers a range of topics, including data analytics; AI; machine and deep learning; information management, security, processing techniques and interpretation; applications of artificial intelligence in soft computing and pattern recognition; cloud-based applications for machine learning; application of IoT in power distribution systems; as well as wireless sensor networks and adaptive wireless communication.

This book features a collection of high-quality research papers presented at the International Conference on Intelligent and Cloud Computing (ICICC 2019), held at Siksha 'O' Anusandhan (Deemed to be University), Bhubaneswar, India, on December 20, 2019. Including contributions on system and network design that can support existing and future applications and services, it covers topics such as cloud computing system and network design, optimization for cloud computing, networking, and applications, green cloud system design, cloud storage design and networking, storage security, cloud system models, big data storage, intra-cloud computing, mobile cloud system design, real-time resource reporting and monitoring for cloud management, machine learning, data mining for cloud computing, data-driven methodology and architecture, and networking for machine learning systems.

This book discusses various machine learning applications and models, developed using heterogeneous data, which helps in a comprehensive prediction, optimization, association analysis, cluster analysis and classification-related applications for various activities in urban area. It details multiple types of data generating from urban activities and suitability of various machine learning algorithms for handling urban data. The book is helpful for researchers, academicians, faculties, scientists and geospatial industry professionals for their research work and sets new ideas in the field of urban computing.

This book explores various applications of deep learning to the diagnosis of cancer,while also outlining the future face of deep learning-assisted cancer diagnostics. As is commonly known, artificial intelligence has paved the way for countless new solutions in the field of medicine. In this context, deep learning is a recent and remarkable sub-field, which can effectively cope with huge amounts of data and deliver more accurate results. As a vital research area, medical diagnosis is among those in which deep learning-oriented solutions are often employed. Accordingly, the objective of this book is to highlight recent advanced applications of deep learning for diagnosing different types of cancer. The target audience includes scientists, experts, MSc and PhD students, postdocs, and anyone interested in the subjects discussed. The book can be used as a reference work to support courses on artificial intelligence, medical and biomedicaleducation.

This two-volume book constitutes the post-conference proceedings of the 5th International Conference on Advances in Computing and Data Sciences, ICACDS 2021, held in Nashik, India, in April 2021.* The 103 full papers were carefully reviewed and selected from 781 submissions. The papers in Part I and II are centered around topics like distributed systems organizing principles, development frameworks and environments, software verification and validation, computational complexity and cryptography, machine learning theory, database theory, probabilistic representations database management system engines, data mining, information retrieval query processing, database and storage security, ubiquitous and mobile computing, parallel computing methodologies, and others. *The conference was held virtually due to the COVID-19 pandemic.

This book describes active illumination techniques in computer vision. We can classify computer vision techniques into two classes: passive and active techniques. Passive techniques observe the scene statically and analyse it as is. Active techniques give the scene some actions and try to facilitate the analysis. In particular, active illumination techniques project specific light, for which the characteristics are known beforehand, to a target scene to enable stable and accurate analysis of the scene. Traditional passive techniques have a fundamental limitation. The external world surrounding us is three-dimensional; the image projected on a retina or an imaging device is two-dimensional. That is, reduction of one dimension has occurred. Active illumination techniques compensate for the dimensional reduction by actively controlling the illumination. The demand for reliable vision sensors is rapidly increasing in many application areas, such as robotics and medical image analysis. This book explains this new endeavour to explore the augmentation of reduced dimensions in computer vision. This book consists of three parts: basic concepts, techniques, and applications. The first part explains the basic concepts for understanding active illumination techniques. In particular, the basic concepts of optics are explained so that researchers and engineers outside the field can understand the later chapters. The second part explains currently available active illumination techniques, covering many techniques developed by the authors. The final part shows how such active illumination techniques can be applied to various domains, describing the issue to be overcome by active illumination techniques and the advantages of using these techniques. This book is primarily aimed at 4th year undergraduate and 1st year graduate students, and will also help engineers from fields beyond computer vision to use active illumination techniques. Additionally, the book is suitable as course material for technical seminars.

This book gathers the latest quality research work of Ph.D. students working on the current areas presented in the Doctoral Symposium on Computational Intelligence (DoSCI 2020). The book includes works in the areas of artificial intelligence, deep learning, evolutionary algorithms, swarm intelligence, fuzzy sets and vague sets, rough set theoretic approaches, quantum-inspired computational intelligence, hybrid computational intelligence, machine learning, computer vision, soft computing, distributed computing, parallel and grid computing, cloud computing, high-performance computing, biomedical computing, decision support and decision making. The book is useful for researchers, students, engineers, practitioners and academicians in their advance studies.

This book gathers papers addressing state-of-the-art research in the areas of machine learning and predictive analysis, presented virtually at the Fourth International Conference on Information and Communication Technology for Intelligent Systems (ICTIS 2020), India. It covers topics such as intelligent agent and multi-agent systems in various domains, machine learning, intelligent information retrieval and business intelligence, intelligent information system development using design science principles, intelligent web mining and knowledge discovery systems.

This book addresses the principles and applications of metaheuristic approaches in engineering and related fields. The first part covers metaheuristics tools and techniques such as ant colony optimization and Tabu search, and their applications to several classes of optimization problems. In turn, the book's second part focuses on a wide variety of metaheuristics applications in engineering and/or the applied sciences, e.g. in smart grids and renewable energy. In addition, the simulation codes for the problems discussed are included in an appendix for ready reference. Intended for researchers aspiring to learn and apply metaheuristic techniques, and gathering contributions by prominent experts in the field, the book offers readers an essential introduction to metaheuristics, its theoretical aspects and applications.

The unification of symbolist and connectionist models is a major trend in AI. The key is to keep the symbolic semantics unchanged. Unfortunately, present embedding approaches cannot. The approach in this book makes the unification possible. It is indeed a new and promising approach in AI. -Bo Zhang, Director of AI Institute, Tsinghua It is indeed wonderful to see the reviving of the important theme Nural Symbolic Model. Given the popularity and prevalence of deep learning, symbolic processing is often neglected or downplayed. This book confronts this old issue head on, with a historical look, incorporating recent advances and new perspectives, thus leading to promising new methods and approaches. -Ron Sun (RPI), on Governing Board of Cognitive Science Society Both for language and humor, approaches like those described in this book are the way to snickerdoodle wombats. -Christian F. Hempelmann (Texas A&M-Commerce) on Executive Board of International Society for Humor Studies

This unique volume reviews the latest advances in domain adaptation in the training of machine learning algorithms for visual understanding, offering valuable insights from an international selection of experts in the field. The text presents a diverse selection of novel techniques, covering applications of object recognition, face recognition, and action and event recognition. Topics and features: reviews the domain adaptation-based machine learning algorithms available for visual understanding, and provides a deep metric learning approach; introduces a novel unsupervised method for image-to-image translation, and a video segment retrieval model that utilizes ensemble learning; proposes a unique way to determine which dataset is most useful in the base training, in order to improve the transferability of deep neural networks; describes a quantitative method for estimating the discrepancy between the source and target data to enhance image classification performance; presents a technique for multi-modal fusion that enhances facial action recognition, and a framework for intuition learning in domain adaptation; examines an original interpolation-based approach to address the issue of tracking model degradation in correlation filter-based methods. This authoritative work will serve as an invaluable reference for researchers and practitioners interested in machine learning-based visual recognition and understanding.

This book comprehensively reviews the various automated and semi-automated signal and image processing techniques, as well as deep-learning-based image analysis techniques, used in healthcare diagnostics. It highlights a range of data pre-processing methods used in signal processing for effective data mining in remote healthcare, and discusses pre-processing using filter techniques, noise removal, and contrast-enhanced methods for improving image quality. The book discusses the status quo of artificial intelligence in medical applications, as well as its future. Further, it offers a glimpse of feature extraction methods for reducing dimensionality and extracting discriminatory information hidden in biomedical signals. Given its scope, the book is intended for academics, researchers and practitioners interested in the latest real-world technological innovations.

This book provides a survey of deep learning approaches to domain adaptation in computer vision. It gives the reader an overview of the state-of-the-art research in deep learning based domain adaptation. This book also discusses the various approaches to deep learning based domain adaptation in recent years. It outlines the importance of domain adaptation for the advancement of computer vision, consolidates the research in the area and provides the reader with promising directions for future research in domain adaptation. Divided into four parts, the first part of this book begins with an introduction to domain adaptation, which outlines the problem statement, the role of domain adaptation and the motivation for research in this area. It includes a chapter outlining pre-deep learning era domain adaptation techniques. The second part of this book highlights feature alignment based approaches to domain adaptation. The third part of this book outlines image alignment procedures for domain adaptation. The final section of this book presents novel directions for research in domain adaptation. This book targets researchers working in artificial intelligence, machine learning, deep learning and computer vision. Industry professionals and entrepreneurs seeking to adopt deep learning into their applications will also be interested in this book.

Deep reinforcement learning has attracted considerable attention recently. Impressive results have been achieved in such diverse fields as autonomous driving, game playing, molecular recombination, and robotics. In all these fields, computer programs have taught themselves to understand problems that were previously considered to be very difficult. In the game of Go, the program AlphaGo has even learned to outmatch three of the world's leading players.Deep reinforcement learning takes its inspiration from the fields of biology and psychology. Biology has inspired the creation of artificial neural networks and deep learning, while psychology studies how animals and humans learn, and how subjects' desired behavior can be reinforced with positive and negative stimuli. When we see how reinforcement learning teaches a simulated robot to walk, we are reminded of how children learn, through playful exploration. Techniques that are inspired by biology and psychology work amazingly well in computers: animal behavior and the structure of the brain as new blueprints for science and engineering. In fact, computers truly seem to possess aspects of human behavior; as such, this field goes to the heart of the dream of artificial intelligence. These research advances have not gone unnoticed by educators. Many universities have begun offering courses on the subject of deep reinforcement learning. The aim of this book is to provide an overview of the field, at the proper level of detail for a graduate course in artificial intelligence. It covers the complete field, from the basic algorithms of Deep Q-learning, to advanced topics such as multi-agent reinforcement learning and meta learning.

This book highlights selected papers from the Mechanical Engineering track, with a focus on mechatronics and manufacturing, presented at the "Malaysian Technical Universities Conference on Engineering and Technology" (MUCET 2019). The conference brings together researchers and professionals in the fields of engineering, research and technology, providing a platform for future collaborations and the exchange of ideas.

Making use of data is not anymore a niche project but central to almost every project. With access to massive compute resources and vast amounts of data, it seems at least in principle possible to solve any problem. However, successful data science projects result from the intelligent application of: human intuition in combination with computational power; sound background knowledge with computer-aided modelling; and critical reflection of the obtained insights and results. Substantially updating the previous edition, then entitled Guide to Intelligent Data Analysis, this core textbook continues to provide a hands-on instructional approach to many data science techniques, and explains how these are used to solve real world problems. The work balances the practical aspects of applying and using data science techniques with the theoretical and algorithmic underpinnings from mathematics and statistics. Major updates on techniques and subject coverage (including deep learning) are included. Topics and features: guides the reader through the process of data science, following the interdependent steps of project understanding, data understanding, data blending and transformation, modeling, as well as deployment and monitoring; includes numerous examples using the open source KNIME Analytics Platform, together with an introductory appendix; provides a review of the basics of classical statistics that support and justify many data analysis methods, and a glossary of statistical terms; integrates illustrations and case-study-style examples to support pedagogical exposition; supplies further tools and information at an associated website. This practical and systematic textbook/reference is a "need-to-have" tool for graduate and advanced undergraduate students and essential reading for all professionals who face data science problems. Moreover, it is a "need to use, need to keep" resource following one's exploration of the subject.