Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Mastering Apache Storm
  • Toc
  • feedback
Mastering Apache Storm

Mastering Apache Storm

By : Jain
1 (1)
close
Mastering Apache Storm

Mastering Apache Storm

1 (1)
By: Jain

Overview of this book

Apache Storm is a real-time Big Data processing framework that processes large amounts of data reliably, guaranteeing that every message will be processed. Storm allows you to scale your data as it grows, making it an excellent platform to solve your big data problems. This extensive guide will help you understand right from the basics to the advanced topics of Storm. The book begins with a detailed introduction to real-time processing and where Storm fits in to solve these problems. You’ll get an understanding of deploying Storm on clusters by writing a basic Storm Hello World example. Next we’ll introduce you to Trident and you’ll get a clear understanding of how you can develop and deploy a trident topology. We cover topics such as monitoring, Storm Parallelism, scheduler and log processing, in a very easy to understand manner. You will also learn how to integrate Storm with other well-known Big Data technologies such as HBase, Redis, Kafka, and Hadoop to realize the full potential of Storm. With real-world examples and clear explanations, this book will ensure you will have a thorough mastery of Apache Storm. You will be able to use this knowledge to develop efficient, distributed real-time applications to cater to your business needs.
Table of Contents (13 chapters)
close
close
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
Real-Time Processing and Storm Introduction
Real-Time Processing and Storm Introduction
Apache Storm
Features of Storm
Storm components
The Storm data model
Programming languages
Summary
2
Storm Deployment, Topology Development, and Topology Options
Storm Deployment, Topology Development, and Topology Options
Storm prerequisites
Setting up the Storm cluster
Developing the hello world example
The different options of the Storm topology
Walkthrough of the Storm UI
Dynamic log level settings
Summary
3
Storm Parallelism and Data Partitioning
chevron up
Storm Parallelism and Data Partitioning
Parallelism of a topology
Rebalance the parallelism of a topology
Different types of stream grouping in the Storm cluster
Guaranteed message processing
Tick tuple
Summary
4
Trident Introduction
Trident Introduction
Trident introduction
Understanding Trident's data model
Writing Trident functions, filters, and projections
Trident repartitioning operations
Trident aggregator
Utilizing the groupBy operation
When to use Trident
Summary
5
Trident Topology and Uses
Trident Topology and Uses
Trident groupBy operation
Non-transactional topology
Trident hello world topology
Trident state
Distributed RPC
When to use Trident
Summary
6
Storm Scheduler
Storm Scheduler
Introduction to Storm scheduler
Default scheduler
Isolation scheduler
Resource-aware scheduler
Custom scheduler
Summary
7
Monitoring of Storm Cluster
Monitoring of Storm Cluster
Cluster statistics using the Nimbus thrift client
Monitoring the Storm cluster using JMX
Monitoring the Storm cluster using Ganglia
Summary
8
Integration of Storm and Kafka
Integration of Storm and Kafka
Introduction to Kafka
Kafka architecture
Installation of Kafka brokers
Share ZooKeeper between Storm and Kafka
Kafka producers and publishing data into Kafka
Kafka Storm integration
Deploy the Kafka topology on Storm cluster
Summary
9
Storm and Hadoop Integration
Storm and Hadoop Integration
Introduction to Hadoop
Installation of Hadoop
Write Storm topology to persist data into HDFS
Integration of Storm with Hadoop
Setting up Storm-YARN
Storm-Starter topologies on Storm-YARN
Summary
10
Storm Integration with Redis, Elasticsearch, and HBase
Storm Integration with Redis, Elasticsearch, and HBase
Integrating Storm with HBase
Integrating Storm with Redis
Integrating Storm with Elasticsearch
Integrating Storm with Esper
Summary
11
Apache Log Processing with Storm
Apache Log Processing with Storm
Apache log processing elements
Producing Apache log in Kafka using Logstash
Splitting the Apache log line
Identifying country, operating system type, and browser type from the log file
Calculate the search keyword
Persisting the process data
Kafka spout and define topology
Deploy topology
MySQL queries
Summary
12
Twitter Tweet Collection and Machine Learning
Twitter Tweet Collection and Machine Learning
Exploring machine learning
Twitter sentiment analysis
Kafka spout, sentiments bolt, and HDFS bolt
Summary

Storm Parallelism and Data Partitioning

In the first two chapters, we have covered the introduction to Storm, the installation of Storm, and developing a sample topology. In this chapter, we are focusing on distribution of the topology on multiple Storm machines/nodes. This chapter covers the following points:

  • Parallelism of topology
  • How to configure parallelism at the code level
  • Different types of stream groupings in a Storm cluster
  • Guaranteed message processing
  • Tick tuple
End of Section 1
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete