Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Elastic Stack 8.x Cookbook
  • Toc
  • feedback
Elastic Stack 8.x Cookbook

Elastic Stack 8.x Cookbook

By : Huage Chen, Yazid Akadiri
5 (3)
close
Elastic Stack 8.x Cookbook

Elastic Stack 8.x Cookbook

5 (3)
By: Huage Chen, Yazid Akadiri

Overview of this book

Learn how to make the most of the Elastic Stack (ELK Stack) products—including Elasticsearch, Kibana, Elastic Agent, and Logstash—to take data reliably and securely from any source, in any format, and then search, analyze, and visualize it in real-time. This cookbook takes a practical approach to unlocking the full potential of Elastic Stack through detailed recipes step by step. Starting with installing and ingesting data using Elastic Agent and Beats, this book guides you through data transformation and enrichment with various Elastic components and explores the latest advancements in search applications, including semantic search and Generative AI. You'll then visualize and explore your data and create dashboards using Kibana. As you progress, you'll advance your skills with machine learning for data science, get to grips with natural language processing, and discover the power of vector search. The book covers Elastic Observability use cases for log, infrastructure, and synthetics monitoring, along with essential strategies for securing the Elastic Stack. Finally, you'll gain expertise in Elastic Stack operations to effectively monitor and manage your system.
Table of Contents (16 chapters)
close
close
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Conventions used
Sections
Get in touch
Share Your Thoughts
Download a free PDF copy of this book
1
Chapter 1: Getting Started – Installing the Elastic Stack
Chapter 1: Getting Started – Installing the Elastic Stack
Deploying the Elastic Stack on Elastic Cloud
Installing the Elastic Stack with ECK
Installing a self-managed Elastic Stack
Creating and setting up data tiering
Creating and setting up additional Elasticsearch nodes
Creating and setting up Fleet Server
Setting up snapshot repository
Free Chapter
2
Chapter 2: Ingesting General Content Data
chevron up
Chapter 2: Ingesting General Content Data
Introducing the Wikipedia Movie Plots dataset
Technical requirements
Adding data from the Elasticsearch client
Updating data in Elasticsearch
Deleting data in Elasticsearch
Using an analyzer
Defining index mapping
Using dynamic templates in document mapping
Creating an index template
Indexing multiple documents using Bulk API
3
Chapter 3: Building Search Applications
Chapter 3: Building Search Applications
Technical requirements
Searching with Query DSL
Building advanced search queries with Query DSL
Using search templates to pre-render search requests
Getting started with Search Applications for your Elasticsearch index
Building a search experience with the Search Application client
Measuring the performance of your Search Applications with Behavioral Analytics
4
Chapter 4: Timestamped Data Ingestion
Chapter 4: Timestamped Data Ingestion
Technical requirements
Deploying Elastic Agent with Fleet
Monitoring Apache HTTP logs and metrics using the Apache integration
Deploying standalone Elastic Agent
Adding data using Beats
Setting up a data stream manually
Setting up a time series data stream manually
5
Chapter 5: Transform Data
Chapter 5: Transform Data
Technical requirements
Creating an ingest pipeline
Enriching data with a custom ingest pipeline for an existing Elastic Agent integration
Using a processor to enrich your data before ingesting with Elastic Agent
Installing self-managed Logstash
Creating a Logstash pipeline
Setting up pivot data transform
Setting up the latest data transform
Downsampling your time series data
6
Chapter 6: Visualize and Explore Data
Chapter 6: Visualize and Explore Data
Technical requirements
Exploring your data in Discover
Exploring your data with ES|QL
Creating visualizations with Kibana Lens
Creating visualizations from runtime fields
Creating Kibana maps
Creating and using Kibana dashboards
Creating Canvas workpads
7
Chapter 7: Alerting and Anomaly Detection
Chapter 7: Alerting and Anomaly Detection
Technical requirements
Creating alerts in Kibana
Monitoring alert rules
Investigating data with log rate analysis
Investigating data with log pattern analysis
Investigating data with change point detection
Detecting anomalies in your data with unsupervised machine learning jobs
Creating anomaly detection jobs from a Lens visualization
8
Chapter 8: Advanced Data Analysis and Processing
Chapter 8: Advanced Data Analysis and Processing
Technical requirements
Finding deviations in your data with outlier detection
Building a model to perform regression analysis
Building a model for classification
Using a trained model for inference
Deploying third-party NLP models and testing via the UI
Running advanced data processing with trained models
9
Chapter 9: Vector Search and Generative AI Integration
Chapter 9: Vector Search and Generative AI Integration
Technical requirements
Implementing semantic search with dense vectors
Implementing semantic search with sparse vectors
Using hybrid search to build advanced search applications
Developing question-answering applications with Generative AI
Using advanced techniques for RAG applications
10
Chapter 10: Elastic Observability Solution
Chapter 10: Elastic Observability Solution
Technical requirements
Instrumenting your application with Elastic APM
Setting up RUM
Instrumenting and monitoring with OpenTelemetry
Monitoring Kubernetes environments with Elastic Agent
Managing synthetics monitoring
Gaining comprehensive system visibility with Elastic Universal Profiling
Detecting incidents with alerting and machine learning
Gaining insights with the AI Assistant
11
Chapter 11: Managing Access Control
Chapter 11: Managing Access Control
Technical requirements
Using built-in roles
Defining custom roles
Granting additional privileges
Managing and securing access to Kibana spaces
Managing access with API keys
Configuring single sign-on
Mapping users and groups to roles
12
Chapter 12: Elastic Stack Operation
Chapter 12: Elastic Stack Operation
Technical requirements
Setting up an index lifecycle policy
Optimizing time series data streams with downsampling
Managing the snapshot lifecycle
Configuring Elastic Stack components with Terraform
Enabling and configuring cross-cluster search
13
Chapter 13: Elastic Stack Monitoring
Chapter 13: Elastic Stack Monitoring
Technical requirements
Setting up Stack Monitoring
Building custom visualizations for monitoring data
Monitoring cluster health via an API
Enabling audit logging
14
Index
Index
Why subscribe?
15
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book

Ingesting General Content Data

This chapter, along with Chapter 4, will focus on data ingestion. Generally, we can categorize data into two groups – general content (data from APIs, HTML pages, catalogs, data from Relational Database Management System (RDBMS), PDFs, spreadsheets, etc.), and time series (data indexed in chronological order, such as logs, metrics, traces, and security events). In this chapter, we will ingest general content to illustrate the basic concepts of data ingestion, including fundamental data operations (index, delete, and update), analyzers, static and dynamic index mappings, and index templates.

Figure 2.1 illustrates the connections between various components, and in this chapter, we will explore recipes dedicated to the Client APP, Analyzer, Mapping, and Index template components (you can view the color image when you download the free PDF version of this book):

Figure 2.1 – Elasticsearch index management components

Figure 2.1 – Elasticsearch index management components

In this chapter, we are going to cover the following main topics:

  • Adding data from the Elasticsearch client
  • Updating data in Elasticsearch
  • Deleting data in Elasticsearch
  • Using an analyzer
  • Defining index mapping
  • Using dynamic templates in document mapping
  • Creating an index template
  • Indexing multiple documents using Bulk API
End of Section 1
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete