Sign In Start Free Trial

Lucene 4 Cookbook

By : Edwood Ng, Vineeth Mohan

3.2 (5)

Lucene 4 Cookbook

3.2 (5)

By: Edwood Ng, Vineeth Mohan

Overview of this book

This book is for software developers who are new to Lucene and who want to explore the more advanced topics to build a search engine. Knowledge of Java is necessary to follow the code samples. You will learn core concepts, best practices, and also advanced features, in order to build an effective search application.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

1. Introducing Lucene

1. Introducing Lucene

Introduction

Installing Lucene

Setting up a simple Java Lucene project

Obtaining an IndexWriter

Creating an analyzer

Creating fields

Creating and writing documents to an index

Deleting documents

Obtaining an IndexSearcher

Creating queries with the Lucene QueryParser

Performing a search

Enumerating results

2. Analyzing Your Text

2. Analyzing Your Text

Introduction

Obtaining a common analyzer

Obtaining a TokenStream

Obtaining TokenAttribute values

Using PositionIncrementAttribute

Using PerFieldAnalyzerWrapper

Defining custom TokenFilters

Defining custom analyzers

Defining custom tokenizers

Defining custom attributes

3. Indexing Your Data

3. Indexing Your Data

Introduction

Obtaining an IndexWriter

Creating a StringField

Creating a TextField

Creating a numeric field

Creating a DocValue Field

Transactional commits and index versioning

Reusing field and document objects per thread

Delving into field norms

Changing similarity implementation used during indexing

4. Searching Your Indexes

4. Searching Your Indexes

Introduction

Obtaining IndexReaders

Un-inverting single-valued fields in memory with FieldCache

TermVectors

IndexSearcher

Constructing queries

Specifying sort logic

Forming a search result

Pagination

Using Collectors

Sorting with custom FieldComparator

5. Near Real-time Searching

5. Near Real-time Searching

Introduction

Using the DirectoryReader to open index in Near Real-Time

Using the SearcherManager to refresh IndexSearcher

Generational indexing with TrackingIndexWriter

Maintaining search sessions with SearcherLifetimeManager

Performance tuning: latency and throughput

6. Querying and Filtering Data

6. Querying and Filtering Data

Introduction

Performing advanced filtering

Creating a custom filter

Searching with QueryParser

TermQuery and TermRangeQuery

BooleanQuery

PrefixQuery and WildcardQuery

PhraseQuery and MultiPhraseQuery

FuzzyQuery

NumericRangeQuery

DisjunctionMaxQuery

RegexpQuery

SpanQuery

CustomScoreQuery

7. Flexible Scoring

7. Flexible Scoring

Introduction

Overriding similarity

Implementing the BM25 model

Implementing the language model

Implementing the divergence from randomness model

Implementing the information-based model

8. Introducing Elasticsearch

8. Introducing Elasticsearch

Introduction

Getting Elasticsearch

Creating a new index

Predefine field mappings

Adding a document

Deleting a document

Updating a document

Performing bulk indexing

Searching the index

Scaling Elasticsearch

9. Extending Lucene with Modules

9. Extending Lucene with Modules

Introduction

Exploring spatial search

Implementing joins

Performing faceting

Implementing grouping

Employing autosuggest

Implementing highlighting

Index

Index

Customer Reviews

3.2 (5)

5 star

20%

4 star

20%

3 star

40%

2 star

0

1 star

20%

Overriding similarity

The Similarity class is an abstract class that defines a set of components for score calculation. To steer away from default scoring, we can create a new class extending from the DefaultSimilarity (TFIDFSimilarity) or one of the other Similarity classes. We will perform some experimentation in this section to see how each scoring components affect the overall score.

Let's begin by reviewing Similarity's methods:

computeNorm(FieldInvertState): This calculates a normalization value for a Field at indexing time.
computeWeight(float, CollectionStatics, TermStatistics): This returns a SimWeight object to calculate a score. It accepts a boost (float) value for query-time boosting.
coord(int, int): This returns a score factor based on term overlap in a query. This value helps to integrate coordinate-level matching. The default is disabled with the returning value 1.
queryNorm(float): This generates a normalization value for a query. The value is also passed back to the Weight...

Search

Your notes and bookmarks