Carry out data analysis with PySpark SQL, graphframes, and graph data processing using a problem-solution approach. This book provides solutions to problems related to dataframes, data manipulation summarization, and exploratory analysis. You will improve your skills in graph data analysis using graphframes and see how to optimize your PySpark SQL code. PySpark SQL Recipes starts with recipes on creating dataframes from different types of data source, data aggregation and summarization, and exploratory data analysis using PySpark SQL. You'll also discover how to solve problems in graph analysis using graphframes. On completing this book, you'll have ready-made code for all your PySpark SQL tasks, including creating dataframes using data from different file formats as well as from SQL or NoSQL databases. What You Will Learn Understand PySpark SQL and its advanced features Use SQL and HiveQL with PySpark SQL Work with structured streaming Optimize PySpark SQL Master graphframes and graph processing Who This Book Is ForData scientists, Python programmers, and SQL programmers.

Learn functional data structures and algorithms for your applications and bring their benefits to your work now About This Book * Moving from object-oriented programming to functional programming? This book will help you get started with functional programming. * Easy-to-understand explanations of practical topics will help you get started with functional data structures. * Illustrative diagrams to explain the algorithms in detail. * Get hands-on practice of Scala to get the most out of functional programming. Who This Book Is For This book is for those who have some experience in functional programming languages. The data structures in this book are primarily written in Scala, however implementing the algorithms in other functional languages should be straight forward. What You Will Learn * Learn to think in the functional paradigm * Understand common data structures and the associated algorithms, as well as the context in which they are commonly used * Take a look at the runtime and space complexities with the O notation * See how ADTs are implemented in a functional setting * Explore the basic theme of immutability and persistent data structures * Find out how the internal algorithms are redesigned to exploit structural sharing, so that the persistent data structures perform well, avoiding needless copying. * Get to know functional features like lazy evaluation and recursion used to implement efficient algorithms * Gain Scala best practices and idioms In Detail Functional data structures have the power to improve the codebase of an application and improve efficiency. With the advent of functional programming and with powerful functional languages such as Scala, Clojure and Elixir becoming part of important enterprise applications, functional data structures have gained an important place in the developer toolkit. Immutability is a cornerstone of functional programming. Immutable and persistent data structures are thread safe by definition and hence very appealing for writing robust concurrent programs. How do we express traditional algorithms in functional setting? Won't we end up copying too much? Do we trade performance for versioned data structures? This book attempts to answer these questions by looking at functional implementations of traditional algorithms. It begins with a refresher and consolidation of what functional programming is all about. Next, you'll get to know about Lists, the work horse data type for most functional languages. We show what structural sharing means and how it helps to make immutable data structures efficient and practical. Scala is the primary implementation languages for most of the examples. At times, we also present Clojure snippets to illustrate the underlying fundamental theme. While writing code, we use ADTs (abstract data types). Stacks, Queues, Trees and Graphs are all familiar ADTs. You will see how these ADTs are implemented in a functional setting. We look at implementation techniques like amortization and lazy evaluation to ensure efficiency. By the end of the book, you will be able to write efficient functional data structures and algorithms for your applications. Style and approach Step-by-step topics will help you get started with functional programming. Learn by doing with hands-on code snippets that give you practical experience of the subject.