Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Elasticsearch 8.x Cookbook
  • Toc
  • feedback
Elasticsearch 8.x Cookbook

Elasticsearch 8.x Cookbook

By : Alberto Paro
4 (6)
close
Elasticsearch 8.x Cookbook

Elasticsearch 8.x Cookbook

4 (6)
By: Alberto Paro

Overview of this book

Elasticsearch is a Lucene-based distributed search engine at the heart of the Elastic Stack that allows you to index and search unstructured content with petabytes of data. With this updated fifth edition, you'll cover comprehensive recipes relating to what's new in Elasticsearch 8.x and see how to create and run complex queries and analytics. The recipes will guide you through performing index mapping, aggregation, working with queries, and scripting using Elasticsearch. You'll focus on numerous solutions and quick techniques for performing both common and uncommon tasks such as deploying Elasticsearch nodes, using the ingest module, working with X-Pack, and creating different visualizations. As you advance, you'll learn how to manage various clusters, restore data, and install Kibana to monitor a cluster and extend it using a variety of plugins. Furthermore, you'll understand how to integrate your Java, Scala, Python, and big data applications such as Apache Spark and Pig with Elasticsearch and create efficient data applications powered by enhanced functionalities and custom plugins. By the end of this Elasticsearch cookbook, you'll have gained in-depth knowledge of implementing the Elasticsearch architecture and be able to manage, search, and store data efficiently and effectively using Elasticsearch.
Table of Contents (20 chapters)
close
close
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the color images
Conventions used
Sections
Get in touch
Share Your Thoughts
1
Chapter 1: Getting Started
Chapter 1: Getting Started
Technical requirements
Downloading and installing Elasticsearch
Setting up networking
Setting up a node
Setting up Linux systems
Setting up different node roles
Setting up a coordinating-only node
Setting up an ingestion node
Installing plugins in Elasticsearch
Removing a plugin
Changing logging settings
Setting up a node via Docker
Deploying on Elastic Cloud Enterprise
Free Chapter
2
Chapter 2: Managing Mappings
chevron up
Chapter 2: Managing Mappings
Technical requirements
Using explicit mapping creation
Mapping base types
Mapping arrays
Mapping an object
Mapping a document
Using dynamic templates in document mapping
Managing nested objects
Managing a child document with a join field
Adding a field with multiple mappings
Mapping a GeoPoint field
Mapping a GeoShape field
Mapping an IP field
Mapping an Alias field
Mapping a Percolator field
Mapping the Rank Feature and Feature Vector fields
Mapping the Search as you type field
Using the Range Fields type
Using the Flattened field type
Using the Point and Shape field types
Using the Dense Vector field type
Using the Histogram field type
Adding metadata to a mapping
Specifying different analyzers
Using index components and templates
3
Chapter 3: Basic Operations
Chapter 3: Basic Operations
Technical requirements
Creating an index
Deleting an index
Opening or closing an index
Putting a mapping in an index
Getting a mapping
Reindexing an index
Refreshing an index
Flushing an index
Using ForceMerge on an index
Shrinking an index
Checking whether an index exists
Managing index settings
Using index aliases
Managing dangling indices
Resolving index names
Rolling over an index
Indexing a document
Getting a document
Deleting a document
Updating a document
Speeding up atomic operations (bulk operations)
Speeding up GET operations (multi-GET)
4
Chapter 4: Exploring Search Capabilities
Chapter 4: Exploring Search Capabilities
Technical requirements
Executing a search
Sorting results
Highlighting results
Executing a scrolling query
Using the search_after functionality
Returning inner hits in results
Suggesting a correct query
Counting matched results
Explaining a query
Query profiling
Deleting by query
Updating by query
Matching all of the documents
Using a Boolean query
Using the search template
5
Chapter 5: Text and Numeric Queries
Chapter 5: Text and Numeric Queries
Technical requirements
Using a term query
Using a terms query
Using a terms set query
Using a prefix query
Using a wildcard query
Using a regexp query
Using span queries
Using a match query
Using a query string query
Using a simple query string query
Using the range query
Using an IDs query
Using the function score query
Using the exists query
Using a pinned query (XPACK)
6
Chapter 6: Relationships and Geo Queries
Chapter 6: Relationships and Geo Queries
Technical requirements
Using the has_child query
Using the has_parent query
Using the nested query
Using the geo_bounding_box query
Using the geo_shape query
Using the geo_distance query
7
Chapter 7: Aggregations
Chapter 7: Aggregations
Executing an aggregation
Executing a stats aggregation
Executing a terms aggregation
Executing a significant terms aggregation
Executing a range aggregation
Executing a histogram aggregation
Executing a date histogram aggregation
Executing a filter aggregation
Executing a filters aggregation
Executing a global aggregation
Executing a geo distance aggregation
Executing a children aggregation
Executing a nested aggregation
Executing a top hit aggregation
Executing a matrix stats aggregation
Executing a geo bounds aggregation
Executing a geo centroid aggregation
Executing a geotile grid aggregation
Executing a sampler aggregation
Executing a pipeline aggregation
8
Chapter 8: Scripting in Elasticsearch
Chapter 8: Scripting in Elasticsearch
Painless scripting
Installing additional scripting languages
Managing scripts
Sorting data using scripts
Computing return fields with scripting
Filtering a search using scripting
Using scripting in aggregations
Updating a document using scripts
Reindexing with a script
Scripting in ingest processors
9
Chapter 9: Managing Clusters
Chapter 9: Managing Clusters
Controlling the cluster health using the health API
Controlling the cluster state using the API
Getting cluster node information using the API
Getting node statistics using the API
Using the task management API
Using the hot threads API
Managing the shard allocation
Monitoring segments with the segment API
Cleaning the cache
10
Chapter 10: Backups and Restoring Data
Chapter 10: Backups and Restoring Data
Managing repositories
Executing a snapshot
Restoring a snapshot
Setting up an NFS share for backups
Reindexing from a remote cluster
11
Chapter 11: User Interfaces
Chapter 11: User Interfaces
Installing Kibana
Managing Kibana Discover
Visualizing data with Kibana
Using Kibana Dev Tools
12
Chapter 12: Using the Ingest Module
Chapter 12: Using the Ingest Module
Pipeline definition
Inserting an ingest pipeline
Getting an ingest pipeline
Deleting an ingest pipeline
Simulating an ingest pipeline
Built-in processors
The grok processor
Using the ingest attachment plugin
Using the ingest GeoIP processor
Using the enrichment processor
13
Chapter 13: Java Integration
Chapter 13: Java Integration
Creating a standard Java HTTP client
Creating a low-level Elasticsearch client
Using the Elasticsearch official Java client
Managing indices
Managing mappings
Managing documents
Managing bulk actions
Building a query
Executing a standard search
Executing a search with aggregations
Executing a scroll search
Integrating with DeepLearning4j
14
Chapter 14: Scala Integration
Chapter 14: Scala Integration
Creating a client in Scala
Managing indices
Managing mappings
Managing documents
Executing a standard search
Executing a search with aggregations
Integrating with DeepLearning.scala
15
Chapter 15: Python Integration
Chapter 15: Python Integration
Creating a client
Managing indices
Managing mappings
Managing documents
Executing a standard search
Executing a search with aggregations
Integrating with NumPy and scikit-learn
Using AsyncElasticsearch
Using Elasticsearch with FastAPI
16
Chapter 16: Plugin Development
Chapter 16: Plugin Development
Creating a plugin
Creating an analyzer plugin
Creating a REST plugin
Creating a cluster action
Creating an ingest plugin
17
Chapter 17: Big Data Integration
Chapter 17: Big Data Integration
Installing Apache Spark
Indexing data using Apache Spark
Indexing data with meta using Apache Spark
Reading data with Apache Spark
Reading data using Spark SQL
Indexing data with Apache Pig
Using Elasticsearch with Alpakka
Using Elasticsearch with MongoDB
18
Chapter 18: X-Pack
Chapter 18: X-Pack
ILM – managing the index life cycle
ILM – automating rollover
Using the SQL Rest API
Using SQL via JDBC
Using X-Pack Security
Using alerting to monitor data events
Why subscribe?
19
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Mapping an IP field

Elasticsearch is used in a lot of systems to collect and search logs, such as Kibana (https://www.elastic.co/products/kibana) and LogStash (https://www.elastic.co/products/logstash). To improve search when using IP addresses, Elasticsearch provides the IPv4 and IPv6 types, which can be used to store IP addresses in an optimized way.

Getting ready

You will need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe of Chapter 1, Getting Started.

How to do it…

You need to define the type of field that contains an IP address as ip.

Regarding the preceding order example, we can extend it by adding the customer IP, like so:

"customer_ip": { "type": "ip" }

The IP must be in the standard point notation form, as follows:

"customer_ip":"19.18.200.201"

How it works…

When Elasticsearch is processing a document and if a field is an IP one, it tries to convert its value into a numerical form and generates tokens for fast value searching.

The IP has special properties:

  • index (the default is true): This defines whether the field must be indexed. If not, false must be used.
  • doc_values (the default is true): This defines whether the field values should be stored in a column-stride fashion to speed up sorting and aggregations.

The other properties (store, boost, null_value, and include_in_all) work as other base types.

The advantage of using IP fields over strings is more speed in every range and filter and lower resource usage (disk and memory).

End of Section 14
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete