Data engineering has grown rapidly in the past decade, leaving many software engineers, data scientists, and analysts looking for a comprehensive view of this practice. With this practical book, you will learn how to plan and build systems to serve the needs of your organization and customers by evaluating the best technologies available in the framework of the data engineering lifecycle. Authors Joe Reis and Matt Housley walk you through the data engineering lifecycle and show you how to stitch together a variety of cloud technologies to serve the needs of downstream data consumers. You will understand how to apply the concepts of data generation, ingestion, orchestration, transformation, storage, governance, and deployment that are critical in any data environment regardless of the underlying technology. This book will help you: Assess data engineering problems using an end-to-end data framework of best practices Cut through marketing hype when choosing data technologies, architecture, and processes Use the data engineering lifecycle to design and build a robust architecture Incorporate data governance and security across the data engineering lifecycle

* This book is an updated version of a well-received book previously published in Chinese by Science Press of China (the first edition in 2006 and the second in 2013). It offers a systematic and practical overview of spatial data mining, which combines computer science and geo-spatial information science, allowing each field to profit from the knowledge and techniques of the other. To address the spatiotemporal specialties of spatial data, the authors introduce the key concepts and algorithms of the data field, cloud model, mining view, and Deren Li methods. The data field method captures the interactions between spatial objects by diffusing the data contribution from a universe of samples to a universe of population, thereby bridging the gap between the data model and the recognition model. The cloud model is a qualitative method that utilizes quantitative numerical characters to bridge the gap between pure data and linguistic concepts. The mining view method discriminates the different requirements by using scale, hierarchy, and granularity in order to uncover the anisotropy of spatial data mining. The Deren Li method performs data preprocessing to prepare it for further knowledge discovery by selecting a weight for iteration in order to clean the observed spatial data as much as possible. In addition to the essential algorithms and techniques, the book provides application examples of spatial data mining in geographic information science and remote sensing. The practical projects include spatiotemporal video data mining for protecting public security, serial image mining on nighttime lights for assessing the severity of the Syrian Crisis, and the applications in the government project 'the Belt and Road Initiatives'.

This volume gathers selected peer-reviewed papers presented at the XXVI International Joint Conference on Industrial Engineering and Operations Management (IJCIEOM), held on July 8-11, 2020 in Rio de Janeiro, Brazil. The respective chapters address a range of timely topics in industrial engineering, including operations and process management, global operations, managerial economics, data science and stochastic optimization, logistics and supply chain management, quality management, product development, strategy and organizational engineering, knowledge and information management, work and human factors, sustainability, production engineering education, healthcare operations management, disaster management, and more. These topics broadly involve fields like operations, manufacturing, industrial and production engineering, and management. Given its scope, the book offers a valuable resource for those engaged in optimization research, operations research, and practitioners alike.

NVIDIA's Full-Color Guide to Deep Learning: All You Need to Get Started and Get Results "To enable everyone to be part of this historic revolution requires the democratization of AI knowledge and resources. This book is timely and relevant towards accomplishing these lofty goals." -- From the foreword by Dr. Anima Anandkumar, Bren Professor, Caltech, and Director of ML Research, NVIDIA "Ekman uses a learning technique that in our experience has proven pivotal to success-asking the reader to think about using DL techniques in practice. His straightforward approach is refreshing, and he permits the reader to dream, just a bit, about where DL may yet take us." -- From the foreword by Dr. Craig Clawson, Director, NVIDIA Deep Learning Institute Deep learning (DL) is a key component of today's exciting advances in machine learning and artificial intelligence. Learning Deep Learning is a complete guide to DL. Illuminating both the core concepts and the hands-on programming techniques needed to succeed, this book is ideal for developers, data scientists, analysts, and others--including those with no prior machine learning or statistics experience. After introducing the essential building blocks of deep neural networks, such as artificial neurons and fully connected, convolutional, and recurrent layers, Magnus Ekman shows how to use them to build advanced architectures, including the Transformer. He describes how these concepts are used to build modern networks for computer vision and natural language processing (NLP), including Mask R-CNN, GPT, and BERT. And he explains how a natural language translator and a system generating natural language descriptions of images. Throughout, Ekman provides concise, well-annotated code examples using TensorFlow with Keras. Corresponding PyTorch examples are provided online, and the book thereby covers the two dominating Python libraries for DL used in industry and academia. He concludes with an introduction to neural architecture search (NAS), exploring important ethical issues and providing resources for further learning. Explore and master core concepts: perceptrons, gradient-based learning, sigmoid neurons, and back propagation See how DL frameworks make it easier to develop more complicated and useful neural networks Discover how convolutional neural networks (CNNs) revolutionize image classification and analysis Apply recurrent neural networks (RNNs) and long short-term memory (LSTM) to text and other variable-length sequences Master NLP with sequence-to-sequence networks and the Transformer architecture Build applications for natural language translation and image captioning NVIDIA's invention of the GPU sparked the PC gaming market. The company's pioneering work in accelerated computing--a supercharged form of computing at the intersection of computer graphics, high-performance computing, and AI--is reshaping trillion-dollar industries, such as transportation, healthcare, and manufacturing, and fueling the growth of many others. Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.

Data is constantly increasing and data analysts are in higher demand than ever. This book is an essential guide to the role of data analyst. Aspiring data analysts will discover what data analysts do all day, what skills they will need for the role, and what regulations they will be required to adhere to. Practising data analysts can explore useful data analysis tools, methods and techniques, brush up on best practices and look at how they can advance their career.

This book, drawing on recent literature, highlights several methodologies for the detection of outliers and explains how to apply them to solve several interesting real-life problems. The detection of objects that deviate from the norm in a data set is an essential task in data mining due to its significance in many contemporary applications. More specifically, the detection of fraud in e-commerce transactions and discovering anomalies in network data have become prominent tasks, given recent developments in the field of information and communication technologies and security. Accordingly, the book sheds light on specific state-of-the-art algorithmic approaches such as the community-based analysis of networks and characterization of temporal outliers present in dynamic networks. It offers a valuable resource for young researchers working in data mining, helping them understand the technical depth of the outlier detection problem and devise innovative solutions to address related challenges.

Online social networking sites like Facebook, LinkedIn, and Twitter, offer millions of members the opportunity to befriend one another, send messages to each other, and post content on the site - actions which generate mind-boggling amounts of data every day.To make sense of the massive data from these sites, we resort to social media mining to answer questions like the following:

This book enriches unsupervised outlier detection research by proposing several new distance-based and density-based outlier scores in a k-nearest neighbors' setting. The respective chapters highlight the latest developments in k-nearest neighbor-based outlier detection research and cover such topics as our present understanding of unsupervised outlier detection in general; distance-based and density-based outlier detection in particular; and the applications of the latest findings to boundary point detection and novel object detection. The book also offers a new perspective on bridging the gap between k-nearest neighbor-based outlier detection and clustering-based outlier detection, laying the groundwork for future advances in unsupervised outlier detection research. The authors hope the algorithms and applications proposed here will serve as valuable resources for outlier detection researchers for years to come.

This book features multi-omics big-data integration and data-mining techniques. In the omics age, paramount of multi-omics data from various sources is the new challenge we are facing, but it also provides clues for several biomedical or clinical applications. This book focuses on data integration and data mining methods for multi-omics research, which explains in detail and with supportive examples the “What�, “Why� and “How� of the topic. The contents are organized into eight chapters, out of which one is for the introduction, followed by four chapters dedicated for omics integration techniques focusing on several omics data resources and data-mining methods, and three chapters dedicated for applications of multi-omics analyses with application being demonstrated by several data mining methods. This book is an attempt to bridge the gap between the biomedical multi-omics big data and the data-mining techniques for the best practice of contemporary bioinformatics and the in-depth insights for the biomedical questions. It would be of interests for the researchers and practitioners who want to conduct the multi-omics studies in cancer, inflammation disease, and microbiome researches.

Algorithms and Applications for Academic Search, Recommendation and Quantitative Association Rule Mining presents novel algorithms for academic search, recommendation and association rule mining that have been developed and optimized for different commercial as well as academic purpose systems. Along with the design and implementation of algorithms, a major part of the work presented in the book involves the development of new systems both for commercial as well as for academic use. In the first part of the book the author introduces a novel hierarchical heuristic scheme for re-ranking academic publications retrieved from standard digital libraries. The scheme is based on the hierarchical combination of a custom implementation of the term frequency heuristic, a time-depreciated citation score and a graph-theoretic computed score that relates the paper's index terms with each other. In order to evaluate the performance of the introduced algorithms, a meta-search engine has been designed and developed that submits user queries to standard digital repositories of academic publications and re-ranks the top-n results using the introduced hierarchical heuristic scheme. In the second part of the book the design of novel recommendation algorithms with application in different types of e-commerce systems are described. The newly introduced algorithms are a part of a developed Movie Recommendation system, the first such system to be commercially deployed in Greece by a major Triple Play services provider. The initial version of the system uses a novel hybrid recommender (user, item and content based) and provides daily recommendations to all active subscribers of the provider (currently more than 30,000). The recommenders that we are presenting are hybrid by nature, using an ensemble configuration of different content, user as well as item-based recommenders in order to provide more accurate recommendation results. The final part of the book presents the design of a quantitative association rule mining algorithm. Quantitative association rules refer to a special type of association rules of the form that antecedent implies consequent consisting of a set of numerical or quantitative attributes. The introduced mining algorithm processes a specific number of user histories in order to generate a set of association rules with a minimally required support and confidence value. The generated rules show strong relationships that exist between the consequent and the antecedent of each rule, representing different items that have been consumed at specific price levels. This research book will be of appeal to researchers, graduate students, professionals, engineers and computer programmers.

In the era of big data, this book explores the new challenges of urban-rural planning and management from a practical perspective based on a multidisciplinary project. Researchers as contributors to this book have accomplished their projects by using big data and relevant data mining technologies for investigating the possibilities of big data, such as that obtained through cell phones, social network systems and smart cards instead of conventional survey data for urban planning support. This book showcases active researchers who share their experiences and ideas on human mobility, accessibility and recognition of places, connectivity of transportation and urban structure in order to provide effective analytic and forecasting tools for smart city planning and design solutions in China.

This laboratory manual is intended for business analysts who wish to increase their skills in the use of statistical analysis to support business decisions. Most of the case studies use Excel,today's most common analysis tool. They range from the most basic descriptive analytical techniques to more advanced techniques such as linear regression and forecasting. Advanced projects cover inferential statistics for continuous variables (t-Test) and categorical variables (chi-square), as well as A/B testing. The manual ends with techniques to deal with the analysis of text data and tools to manage the analysis of large data sets (Big Data) using Excel. Includes companion files with solution spreadsheets, sample files, data sets, etc. from the book. Features: Teaches the statistical analysis skills needed to support business decisions Provides projects ranging from the most basic descriptive analytical techniques to more advanced techniques such as linear regression, forecasting, inferential statistics, and analyzing big data sets Includes companion files with solution spreadsheets, sample files, data sets, etc. used in the book's case studies

This compendium is a completely revised version of an earlier book, Data Mining in Time Series Databases, by the same editors. It provides a unique collection of new articles written by leading experts that account for the latest developments in the field of time series and data stream mining.The emerging topics covered by the book include weightless neural modeling for mining data streams, using ensemble classifiers for imbalanced and evolving data streams, document stream mining with active learning, and many more. In particular, it addresses the domain of streaming data, which has recently become one of the emerging topics in Data Science, Big Data, and related areas. Existing titles do not provide sufficient information on this topic.

This book presents a series of studies that demonstrate the value of interactions between knowledge management with the arts and humanities. The carefully compiled chapters show, on the one hand, how traditional methods from the arts and humanities - e.g. theatrical improvisation, clay modelling, theory of aesthetics - can be used to enhance knowledge creation and evolution. On the other, the chapters discuss knowledge management models and practices such as virtual knowledge space (BA) design, social networking and knowledge sharing, data mining and knowledge discovery tools. The book also demonstrates how these practices can yield valuable benefits in terms of organizing and analyzing big arts and humanities data in a digital environment.

Data analysis is of upmost importance in the mining of big data, where knowledge discovery and inference are the basis for intelligent systems to support the real world applications. However, the process involves knowledge acquisition, representation, inference and data, Bayesian network (BN) is the key technology plays a key role in knowledge representation, in order to pave way to cope with incomplete, fuzzy data to solve the real-life problems.This book presents Bayesian network as a technology to support data-intensive and incremental learning in knowledge discovery, inference and data fusion in uncertain environment.

This book has two main goals: to define data science through the work of data scientists and their results, namely data products, while simultaneously providing the reader with relevant lessons learned from applied data science projects at the intersection of academia and industry. As such, it is not a replacement for a classical textbook (i.e., it does not elaborate on fundamentals of methods and principles described elsewhere), but systematically highlights the connection between theory, on the one hand, and its application in specific use cases, on the other. With these goals in mind, the book is divided into three parts: Part I pays tribute to the interdisciplinary nature of data science and provides a common understanding of data science terminology for readers with different backgrounds. These six chapters are geared towards drawing a consistent picture of data science and were predominantly written by the editors themselves. Part II then broadens the spectrum by presenting views and insights from diverse authors - some from academia and some from industry, ranging from financial to health and from manufacturing to e-commerce. Each of these chapters describes a fundamental principle, method or tool in data science by analyzing specific use cases and drawing concrete conclusions from them. The case studies presented, and the methods and tools applied, represent the nuts and bolts of data science. Finally, Part III was again written from the perspective of the editors and summarizes the lessons learned that have been distilled from the case studies in Part II. The section can be viewed as a meta-study on data science across a broad range of domains, viewpoints and fields. Moreover, it provides answers to the question of what the mission-critical factors for success in different data science undertakings are. The book targets professionals as well as students of data science: first, practicing data scientists in industry and academia who want to broaden their scope and expand their knowledge by drawing on the authors' combined experience. Second, decision makers in businesses who face the challenge of creating or implementing a data-driven strategy and who want to learn from success stories spanning a range of industries. Third, students of data science who want to understand both the theoretical and practical aspects of data science, vetted by real-world case studies at the intersection of academia and industry.

A visual approach to data mining. Data mining has been defined as the search for useful and previously unknown patterns in large datasets, yet when faced with the task of mining a large dataset, it is not always obvious where to start and how to proceed. This book introduces a visual methodology for data mining demonstrating the application of methodology along with a sequence of exercises using VisMiner. VisMiner has been developed by the author and provides a powerful visual data mining tool enabling the reader to see the data that they are working on and to visually evaluate the models created from the data. Key features: * Presents visual support for all phases of data mining including dataset preparation. * Provides a comprehensive set of non-trivial datasets and problems with accompanying software. * Features 3-D visualizations of multi-dimensional datasets. * Gives support for spatial data analysis with GIS like features. * Describes data mining algorithms with guidance on when and how to use. * Accompanied by VisMiner, a visual software tool for data mining, developed specifically to bridge the gap between theory and practice. Visual Data Mining: The VisMiner Approach is designed as a hands-on work book to introduce the methodologies to students in data mining, advanced statistics, and business intelligence courses. This book provides a set of tutorials, exercises, and case studies that support students in learning data mining processes. In praise of the VisMiner approach: "What we discovered among students was that the visualization concepts and tools brought the analysis alive in a way that was broadly understood and could be used to make sound decisions with greater certainty about the outcomes" Dr. James V. Hansen, J. Owen Cherrington Professor, Marriott School, Brigham Young University, USA "Students learn best when they are able to visualize relationships between data and results during the data mining process. VisMiner is easy to learn and yet offers great visualization capabilities throughout the data mining process. My students liked it very much and so did I." Dr. Douglas Dean, Assoc. Professor of Information Systems, Marriott School, Brigham Young University, USA

This book presents a broad range of deep-learning applications related to vision, natural language processing, gene expression, arbitrary object recognition, driverless cars, semantic image segmentation, deep visual residual abstraction, brain-computer interfaces, big data processing, hierarchical deep learning networks as game-playing artefacts using regret matching, and building GPU-accelerated deep learning frameworks. Deep learning, an advanced level of machine learning technique that combines class of learning algorithms with the use of many layers of nonlinear units, has gained considerable attention in recent times. Unlike other books on the market, this volume addresses the challenges of deep learning implementation, computation time, and the complexity of reasoning and modeling different type of data. As such, it is a valuable and comprehensive resource for engineers, researchers, graduate students and Ph.D. scholars.

This book explores an approach to social robotics based solely on autonomous unsupervised techniques and positions it within a structured exposition of related research in psychology, neuroscience, HRI, and data mining. The authors present an autonomous and developmental approach that allows the robot to learn interactive behavior by imitating humans using algorithms from time-series analysis and machine learning. The first part provides a comprehensive and structured introduction to time-series analysis, change point discovery, motif discovery and causality analysis focusing on possible applicability to HRI problems. Detailed explanations of all the algorithms involved are provided with open-source implementations in MATLAB enabling the reader to experiment with them. Imitation and simulation are the key technologies used to attain social behavior autonomously in the proposed approach. Part two gives the reader a wide overview of research in these areas in psychology, and ethology. Based on this background, the authors discuss approaches to endow robots with the ability to autonomously learn how to be social. Data Mining for Social Robots will be essential reading for graduate students and practitioners interested in social and developmental robotics.

This book provides a readable and elegant presentation of the principles of anomaly detection,providing an easy introduction for newcomers to the field. A large number of algorithms are succinctly described, along with a presentation of their strengths and weaknesses. The authors also cover algorithms that address different kinds of problems of interest with single and multiple time series data and multi-dimensional data. New ensemble anomaly detection algorithms are described, utilizing the benefits provided by diverse algorithms, each of which work well on some kinds of data. With advancements in technology and the extensive use of the internet as a medium for communications and commerce, there has been a tremendous increase in the threats faced by individuals and organizations from attackers and criminal entities. Variations in the observable behaviors of individuals (from others and from their own past behaviors) have been found to be useful in predicting potential problems of various kinds. Hence computer scientists and statisticians have been conducting research on automatically identifying anomalies in large datasets. This book will primarily target practitioners and researchers who are newcomers to the area of modern anomaly detection techniques. Advanced-level students in computer science will also find this book helpful with their studies.

Managing and Mining Graph Data is a comprehensive survey book in graph management and mining. It contains extensive surveys on a variety of important graph topics such as graph languages, indexing, clustering, data generation, pattern mining, classification, keyword search, pattern matching, and privacy. It also studies a number of domain-specific scenarios such as stream mining, web graphs, social networks, chemical and biological data. The chapters are written by well known researchers in the field, and provide a broad perspective of the area. This is the first comprehensive survey book in the emerging topic of graph data processing.
Managing and Mining Graph Data is designed for a varied audience composed of professors, researchers and practitioners in industry. This volume is also suitable as a reference book for advanced-level database students in computer science and engineering.

Multi-database mining has been recognized recently as an important and strategically essential area of research in data mining. In this book, we discuss various issues regarding the systematic and efficient development of multi-database mining applications. It explains how systematically one could prepare data warehouses at different branches. As appropriate multi-database mining technique is essential to develop better applications. Also, the efficiency of a multi-database mining application could be improved by processing more patterns in the application. A faster algorithm could also play an important role in developing a better application. Thus the efficiency of a multi-database mining application could be enhanced by choosing an appropriate multi-database mining model, an appropriate pattern synthesizing technique, a better pattern representation technique, and an efficient algorithm for solving the problem. This book illustrates each of these issues either in the context of a specific problem, or in general.

Cognitive Information Systems in Management Sciences summarizes the body of work in this area, taking an analytical approach to interpreting the data, while also providing an approach that can be used for practical implementation in the fields of computing, economics, and engineering. Using numerous illustrative examples, and following both theoretical and practical results, Dr. Lidia Ogiela discusses the concepts and principles of cognitive information systems, the relationship between intelligent computer data analysis, and how to utilize computational intelligent approaches to enhance information retrieval. Real world implantation use cases round out the book, with valuable scenarios covering management science, computer science, and engineering. Indexing: The books of this series are submitted to EI-Compendex and SCOPUS

This volume features selected, refereed papers on various aspects of statistics, matrix theory and its applications to statistics, as well as related numerical linear algebra topics and numerical solution methods, which are relevant for problems arising in statistics and in big data. The contributions were originally presented at the 25th International Workshop on Matrices and Statistics (IWMS 2016), held in Funchal (Madeira), Portugal on June 6-9, 2016. The IWMS workshop series brings together statisticians, computer scientists, data scientists and mathematicians, helping them better understand each other's tools, and fostering new collaborations at the interface of matrix theory and statistics.

Observational calculi were introduced in the 1960's as a tool of logic of discovery. Formulas of observational calculi correspond to assertions on analysed data. Truthfulness of suitable assertions can lead to acceptance of new scientific hypotheses. The general goal was to automate the process of discovery of scientific knowledge using mathematical logic and statistics. The GUHA method for producing true formulas of observational calculi relevant to the given problem of scientific discovery was developed. Theoretically interesting and practically important results on observational calculi were achieved. Special attention was paid to formulas - couples of Boolean attributes derived from columns of the analysed data matrix. Association rules introduced in the 1990's can be seen as a special case of such formulas. New results on logical calculi and association rules were achieved. They can be seen as a logic of association rules. This can contribute to solving contemporary challenging problems of data mining research and practice. The book covers thoroughly the logic of association rules and puts it into the context of current research in data mining. Examples of applications of theoretical results to real problems are presented. New open problems and challenges are listed. Overall, the book is a valuable source of information for researchers as well as for teachers and students interested in data mining.