Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Mastering Hadoop
  • Toc
  • feedback
Mastering Hadoop

Mastering Hadoop

By : Karanth
4 (3)
close
Mastering Hadoop

Mastering Hadoop

4 (3)
By: Karanth

Overview of this book

Do you want to broaden your Hadoop skill set and take your knowledge to the next level? Do you wish to enhance your knowledge of Hadoop to solve challenging data processing problems? Are your Hadoop jobs, Pig scripts, or Hive queries not working as fast as you intend? Are you looking to understand the benefits of upgrading Hadoop? If the answer is yes to any of these, this book is for you. It assumes novice-level familiarity with Hadoop.
Table of Contents (15 chapters)
close
close
Preface
Preface
What this book covers
What you need for this book?
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Hadoop 2.X
1. Hadoop 2.X
The inception of Hadoop
The evolution of Hadoop
Hadoop 2.X
Hadoop distributions
Summary
2
2. Advanced MapReduce
2. Advanced MapReduce
MapReduce input
The RecordReader class
Hadoop's "small files" problem
Filtering inputs
The Map task
The Reduce task
MapReduce output
MapReduce job counters
Handling data joins
Summary
3
3. Advanced Pig
3. Advanced Pig
Pig versus SQL
Different modes of execution
Complex data types in Pig
Compiling Pig scripts
Development and debugging aids
The advanced Pig operators
User-defined functions
Pig performance optimizations
Best practices
Summary
4
4. Advanced Hive
chevron up
4. Advanced Hive
The Hive architecture
Data types
File formats
The data model
Hive query optimizers
Advanced DML
UDF, UDAF, and UDTF
Summary
5
5. Serialization and Hadoop I/O
5. Serialization and Hadoop I/O
Data serialization in Hadoop
Avro serialization
File formats
Compression
Summary
6
6. YARN – Bringing Other Paradigms to Hadoop
6. YARN – Bringing Other Paradigms to Hadoop
The YARN architecture
Developing YARN applications
Monitoring YARN
Job scheduling in YARN
YARN commands
Summary
7
7. Storm on YARN – Low Latency Processing in Hadoop
7. Storm on YARN – Low Latency Processing in Hadoop
Batch processing versus streaming
Apache Storm
Storm on YARN
Summary
8
8. Hadoop on the Cloud
8. Hadoop on the Cloud
Cloud computing characteristics
Hadoop on the cloud
Amazon Elastic MapReduce (EMR)
Summary
9
9. HDFS Replacements
9. HDFS Replacements
HDFS – advantages and drawbacks
Amazon AWS S3
Implementing a filesystem in Hadoop
Implementing an S3 native filesystem in Hadoop
Summary
10
10. HDFS Federation
10. HDFS Federation
Limitations of the older HDFS architecture
Architecture of HDFS Federation
HDFS high availability
HDFS block placement
Summary
11
11. Hadoop Security
11. Hadoop Security
The security pillars
Authentication in Hadoop
Authorization in Hadoop
Data confidentiality in Hadoop
Audit logging in Hadoop
Summary
12
12. Analytics Using Hadoop
12. Analytics Using Hadoop
Data analytics workflow
Machine learning
Apache Mahout
Document analysis using Hadoop and Mahout
RHadoop
Summary
13
A. Hadoop for Microsoft Windows
A. Hadoop for Microsoft Windows
Deploying Hadoop on Microsoft Windows
Summary
14
Index
Index

Data types

Hive supports all the primitive numeric data types such as TINYINT, SMALLINT, INT, BIGINT, FLOAT, DOUBLE, and DECIMAL. In addition to these primitive data types, Hive also supports string types such as CHAR, VARCHAR, and STRING data types. Like SQL, the time indicator data types such as TIMESTAMP and DATE are present. BOOLEAN and BINARY miscellaneous types are available too.

A number of complex types are also available. Complex types can be composed from other primitive or complex types. The complex types available are as follows:

  • STRUCTS: These are groupings of data elements similar to a C-struct. The dot notation is used to dereference elements within a struct. A field within column C defined as STRUCT {x INT, y STRING} can be accessed as A.x or A.y.

    The syntax for this is STRUCT<field_name : data_type>

  • MAPS: These are key-value data types; providing the key within square braces can access a value. A value of a map column M that maps from key x to value y can be accessed...

End of Section 4
Next Section
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete