Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Apache Hadoop 3 Quick Start Guide
  • Toc
  • feedback
Apache Hadoop 3 Quick Start Guide

Apache Hadoop 3 Quick Start Guide

By : Vijay Karambelkar
close
Apache Hadoop 3 Quick Start Guide

Apache Hadoop 3 Quick Start Guide

By: Vijay Karambelkar

Overview of this book

Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics, including MapReduce, YARN, and HDFS. The book begins with an overview of big data and Apache Hadoop. Then, you will set up a pseudo Hadoop development environment and a multi-node enterprise Hadoop cluster. You will see how the parallel programming paradigm, such as MapReduce, can solve many complex data processing problems. The book also covers the important aspects of the big data software development lifecycle, including quality assurance and control, performance, administration, and monitoring. You will then learn about the Hadoop ecosystem, and tools such as Kafka, Sqoop, Flume, Pig, Hive, and HBase. Finally, you will look at advanced topics, including real time streaming using Apache Storm, and data analytics using Apache Spark. By the end of the book, you will be well versed with different configurations of the Hadoop 3 cluster.
Table of Contents (10 chapters)
close
close
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Hadoop 3.0 - Background and Introduction
Hadoop 3.0 - Background and Introduction
How it all started
What Hadoop is and why it is important
How Apache Hadoop works
Hadoop 3.0 releases and new features
Choosing the right Hadoop distribution
Summary
2
Planning and Setting Up Hadoop Clusters
Planning and Setting Up Hadoop Clusters
Technical requirements
Prerequisites for Hadoop setup
Running Hadoop in standalone mode
Setting up a pseudo Hadoop cluster
Planning and sizing clusters
Setting up Hadoop in cluster mode
Diagnosing the Hadoop cluster
Summary
3
Deep Dive into the Hadoop Distributed File System
Deep Dive into the Hadoop Distributed File System
Technical requirements
How HDFS works
Key features of HDFS
Data flow patterns of HDFS
HDFS configuration files
Hadoop filesystem CLIs
Working with data structures in HDFS
Summary
4
Developing MapReduce Applications
Developing MapReduce Applications
Technical requirements
How MapReduce works
Configuring a MapReduce environment
Understanding Hadoop APIs and packages
Setting up a MapReduce project
Deep diving into MapReduce APIs
Compiling and running MapReduce jobs
Streaming in MapReduce programming
Summary
5
Building Rich YARN Applications
Building Rich YARN Applications
Technical requirements
Understanding YARN architecture
Key features of YARN
Configuring the YARN environment in a cluster
Working with YARN distributed CLI
Deep dive with YARN application framework
Building and monitoring a YARN application on a cluster
Summary
6
Monitoring and Administration of a Hadoop Cluster
Monitoring and Administration of a Hadoop Cluster
Roles and responsibilities of Hadoop administrators
Planning your distributed cluster
Resource management in Hadoop
High availability of Hadoop
Securing Hadoop clusters
Performing routine tasks
Summary
7
Demystifying Hadoop Ecosystem Components
Demystifying Hadoop Ecosystem Components
Technical requirements
Understanding Hadoop's Ecosystem
Working with Apache Kafka
Writing Apache Pig scripts
Transferring data with Sqoop
Writing Flume jobs
Understanding Hive
Using HBase for NoSQL storage
Summary
8
Advanced Topics in Apache Hadoop
chevron up
Advanced Topics in Apache Hadoop
Technical requirements
Hadoop use cases in industries
Advanced Hadoop data storage file formats
Real-time streaming with Apache Storm
Data analytics with Apache Spark
Summary
9
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Data analytics with Apache Spark

Apache Spark offers a blazing fast processing engine based out of Apache Hadoop. It provides in-memory cluster processing of the data, thereby providing analytics at high speeds. Apache Spark evolved in AMPLab (U. C. Berkeley) in 2009 and it was made open source through the Apache Software Foundation. Apache Spark is based out of YARN. Following are key features of Apache Spark:

  • Fast: Due to in-memory processing capability, Spark is fast in processing
  • Multiple language support: You can write Spark programs in Java, Scala, R, and Python
  • Deep analytics: It provides truly distributed analytics, which includes machine learning, streaming data processing, and data querying
  • Rich API support: It provides a rich API library for interaction in multiple languages
  • Multi-processing engine support: Apache Spark can be deployed on MapReduce, YARN, and Mesos...
End of Section 6
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete