Work with petabyte-scale datasets while building a collaborative, agile workplace in the process. This practical book is the canonical reference to Google BigQuery, the query engine that lets you conduct interactive analysis of large datasets. BigQuery enables enterprises to efficiently store, query, ingest, and learn from their data in a convenient framework. With this book, you’ll examine how to analyze data at scale to derive insights from large datasets efficiently. Valliappa Lakshmanan, tech lead for Google Cloud Platform, and Jordan Tigani, engineering director for the BigQuery team, provide best practices for modern data warehousing within an autoscaled, serverless public cloud. Whether you want to explore parts of BigQuery you’re not familiar with or prefer to focus on specific tasks, this reference is indispensable.

This book shows business and data analysts how to use BigQuery most effectively, avoid common pitfalls, and ultimately execute sophisticated queries against large, complex data sets. The authors will share tips and recipes for running complex queries. And they will also show how to write code to communicate with the BigQuery API. The authors will demonstrate best practices and techniques against an extended real-world example -- a web application that collects sensor data from mobile devices and displays a dashboard visualizing the data in real-time. Along the way, the authors will use examples to demonstrate streaming ingestion, transformation via Hadoop in Google Compute engine, AppEngine datastore integration, and using GViz with Tableau to generate charts of query results.The authors will not just cover the mechanics of using BigQuery; they will also cover the architecture of the underlying Dremel query engine: understanding how a query will execute is a key to getting good results from BigQuery. The book describes how Dremel works, and pairs it with concrete query examples showing how to work around limitations in the architecture. The query samples will be in BigQuery's variant of SQL. And the web application examples will be in Python, the most popular language for analytics. Where the Java analogue of the Python samples would differ significantly, Java samples will be given as well. All code and data sets will be available on the book's companion website.