Splunk Operational Intelligence Cookbook

By : Josh Diakun, Raheja, Paul R. Johnson, Derek Mock

4.7 (3)

Buy this Book

Splunk Operational Intelligence Cookbook

4.7 (3)

By: Josh Diakun, Raheja, Paul R. Johnson, Derek Mock

Buy this Book

Overview of this book

Splunk makes it easy for you to take control of your data, and with Splunk Operational Cookbook, you can be confident that you are taking advantage of the Big Data revolution and driving your business with the cutting edge of operational intelligence and business analytics. With more than 80 recipes that demonstrate all of Splunk’s features, not only will you find quick solutions to common problems, but you’ll also learn a wide range of strategies and uncover new ideas that will make you rethink what operational intelligence means to you and your organization. You’ll discover recipes on data processing, searching and reporting, dashboards, and visualizations to make data shareable, communicable, and most importantly meaningful. You’ll also find step-by-step demonstrations that walk you through building an operational intelligence application containing vital features essential to understanding data and to help you successfully integrate a data-driven way of thinking in your organization. Throughout the book, you’ll dive deeper into Splunk, explore data models and pivots to extend your intelligence capabilities, and perform advanced searching with machine learning to explore your data in even more sophisticated ways. Splunk is changing the business landscape, so make sure you’re taking advantage of it.

Preface

Who this book is for

What this book covers

To get the most out of this book

Sections

Get in touch

Free Chapter

Play Time – Getting Data In

Introduction

Indexing files and directories

Getting data through network ports

Using scripted inputs

Using modular inputs

Using the Universal Forwarder to gather data

Receiving data using the HTTP Event Collector

Getting data from databases using DB Connect

Loading the sample data for this book

Data onboarding – defining field extractions

Data onboarding - defining event types and tags

Installing the Machine Learning Toolkit

Diving into Data – Search and Report

Introduction

Making raw event data readable

Finding the most accessed web pages

Finding the most used web browsers

Identifying the top-referring websites

Charting web page response codes

Displaying web page response time statistics

Listing the top-viewed products

Charting the application's functional performance

Charting the application's memory usage

Counting the total number of database connections

Dashboards and Visualizations - Make Data Shine

Introduction

Creating an Operational Intelligence dashboard

Using a pie chart to show the most accessed web pages

Displaying the unique number of visitors

Using a gauge to display the number of errors

Charting the number of method requests by type and host

Creating a timechart of method requests, views, and response times

Using a scatter chart to identify discrete requests by size and response time

Creating an area chart of the application's functional statistics

Using metrics data and a trellis layout to monitor physical environment operating conditions

Using a bar chart to show the average amount spent by category

Creating a line chart of item views and purchases over time

Building an Operational Intelligence Application

Introduction

Creating an Operational Intelligence application

Adding dashboards and reports

Organizing the dashboards more efficiently

Dynamically drilling down on activity reports

Creating a form for searching web activity

Linking web page activity reports to the form

Displaying a geographical map of visitors

Highlighting average product price

Scheduling the PDF delivery of a dashboard

Extending Intelligence – Datasets, Modeling and Pivoting

Introduction

Creating a data model for web access logs

Creating a data model for application logs

Accelerating data models

Pivoting total sales transactions

Pivoting purchases by geographic location

Pivoting slowest responding web pages

Pivot charting top error codes

Diving Deeper – Advanced Searching, Machine Learning and Predictive Analytics

Introduction

Calculating the average session time on a website

Calculating the average execution time for multi-tier web requests

Displaying the maximum concurrent checkouts

Analyzing the relationship of web requests

Predicting website traffic volumes

Finding abnormally-sized web requests

Identifying potential session spoofing

Detecting outliers in server response times

Forecasting weekly sales

Summary

Enriching Data – Lookups and Workflows

Introduction

Looking up product code descriptions

Flagging suspect IP addresses

Creating a session state table

Adding hostnames to IP addresses

Searching ARIN for a given IP address

Triggering a Google search for a given error

Generating a chat notification for application errors

Looking up inventory from an external database

Being Proactive – Creating Alerts

Introduction

Alerting on abnormal web page response times

Alerting on errors during checkout in real time

Alerting on abnormal user behavior

Alerting on failure and triggering a chat notification

Alerting when predicted sales exceed inventory

Generating alert events for high sensor readings

Speeding Up Intelligence – Data Summarization

Introduction

Calculating an hourly count of sessions versus completed transactions

Backfilling the number of purchases by city

Displaying the maximum number of concurrent sessions over time

Above and Beyond – Customization, Web Framework, HTTP Event Collector, REST API, and SDKs

Introduction

Customizing the application navigation

Adding a Sankey diagram of web hits

Developing a tag cloud of purchases by country

Adding Cell Icons to Highlight Average Product Price

Remotely querying Splunk's REST API for unique page views

Creating a Python application to return unique IP addresses

Creating a custom search command to format product names

Collecting data from remote scanning devices

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

4.7 (3)

5 star

66.7%

4 star

33.3%

3 star

2 star

1 star

Charting the application's functional performance

Another of the data samples we loaded in Chapter 1, Play Time - Getting Data In, contained application logs from our application server. These have a Splunk source type of log4j and detail the various calls that our application makes to the backend database in response to user web requests, in addition to providing insight into memory utilization and other health-related information. We are particularly interested in tracking how our application is performing in relation to the time taken to process user-driven requests for information.

In this recipe, we will write a Splunk search to find out how our application is performing. To do this, we will analyze database call transactions and chart the maximum, mean, and minimum transaction durations over the past week.

...