Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Apache Hive Cookbook
  • Toc
  • feedback
Apache Hive Cookbook

Apache Hive Cookbook

By : Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra
3 (4)
close
Apache Hive Cookbook

Apache Hive Cookbook

3 (4)
By: Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra

Overview of this book

Hive was developed by Facebook and later open sourced in Apache community. Hive provides SQL like interface to run queries on Big Data frameworks. Hive provides SQL like syntax also called as HiveQL that includes all SQL capabilities like analytical functions which are the need of the hour in today’s Big Data world. This book provides you easy installation steps with different types of metastores supported by Hive. This book has simple and easy to learn recipes for configuring Hive clients and services. You would also learn different Hive optimizations including Partitions and Bucketing. The book also covers the source code explanation of latest Hive version. Hive Query Language is being used by other frameworks including spark. Towards the end you will cover integration of Hive with these frameworks.
Table of Contents (14 chapters)
close
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Developing Hive
1. Developing Hive
Introduction
Deploying Hive on a Hadoop cluster
Deploying Hive Metastore
Installing Hive
Configuring HCatalog
Understanding different components of Hive
Compiling Hive from source
Hive packages
Debugging Hive
Running Hive
Changing configurations at runtime
2
2. Services in Hive
2. Services in Hive
Introducing HiveServer2
Understanding HiveServer2 properties
Configuring HiveServer2 high availability
Using HiveServer2 clients
Introducing the Hive metastore service
Configuring high availability of metastore service
Introducing Hue
3
3. Understanding the Hive Data Model
3. Understanding the Hive Data Model
Introduction
Using numeric data types
Using string data types
Using Date/Time data types
Using miscellaneous data types
Using complex data types
Using operators
Partitioning
Partitioning a managed table
Partitioning an external table
Bucketing
4
4. Hive Data Definition Language
4. Hive Data Definition Language
Introduction
Creating a database schema
Dropping a database schema
Altering a database schema
Using a database schema
Showing database schemas
Describing a database schema
Creating tables
Dropping tables
Truncating tables
Renaming tables
Altering table properties
Creating views
Dropping views
Altering the view properties
Altering the view as select
Showing tables
Showing partitions
Show the table properties
Showing create table
HCatalog
WebHCat
5
5. Hive Data Manipulation Language
5. Hive Data Manipulation Language
Introduction
Loading files into tables
Inserting data into Hive tables from queries
Inserting data into dynamic partitions
Writing data into files from queries
Enabling transactions in Hive
Inserting values into tables from SQL
Updating data
Deleting data
6
6. Hive Extensibility Features
6. Hive Extensibility Features
Introduction
Serialization and deserialization formats and data types
Exploring views
Exploring indexes
Hive partitioning
Creating buckets in Hive
Analytics functions in Hive
Windowing in Hive
File formats
7
7. Joins and Join Optimization
7. Joins and Join Optimization
Understanding the joins concept
Using a left/right/full outer join
Using a left semi join
Using a cross join
Using a map-side join
Using a bucket map join
Using a bucket sort merge map join
Using a skew join
8
8. Statistics in Hive
chevron up
8. Statistics in Hive
Bringing statistics in to Hive
Table and partition statistics in Hive
Column statistics in Hive
Top K statistics in Hive
9
9. Functions in Hive
9. Functions in Hive
Using built-in functions
Using the built-in User-defined Aggregation Function (UDAF)
Using the built-in User Defined Table Function (UDTF)
Creating custom User-Defined Functions (UDF)
10
10. Hive Tuning
10. Hive Tuning
Enabling predicate pushdown optimizations in Hive
Optimizations to reduce the number of map
Sampling
11
11. Hive Security
11. Hive Security
Securing Hadoop
Authorizing Hive
Configuring the SQL standards-based authorization
Authenticating Hive
12
12. Hive Integration with Other Frameworks
12. Hive Integration with Other Frameworks
Working with Apache Spark
Working with Accumulo
Working with HBase
Working with Google Drill
13
Index
Index

Table and partition statistics in Hive

The first development in statistical computation is to support tables and partition-level statistics. With other metadata, the table and partition statistics are also stored in a configured metastore. The statistics are supported for both existing and new tables. The following are the statistics currently supported for tables and partitions:

  • The number of rows

  • The number of files

  • Size in bytes

  • Max, min, and average row sizes

  • Max, min, and average file sizes

  • The number of partitions (in the case of tables)

Getting ready

This recipe requires Hive installed as described in the Installing Hive recipe of Chapter 1, Developing Hive. You will also need Hive CLI or the beeline client to run the commands.

How to do it…

For newly created table or partitions using the INSERT OVERWRITE command, statistics are computed automatically at table level. If you want to disable statistics calculations for a table, you need to set hive.stats.autogather to false either for the session...

Unlock full access

Continue reading for free

A Packt free trial gives you instant online access to our library of over 7000 practical eBooks and videos, constantly updated with the latest in tech
Start a 7-day FREE trial
End of Section 3
Next Section
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete