Defining custom TokenFilters
Sometimes, search behaviors may be so specific that we need to create a custom TokenFilter to achieve those behaviors. To create a custom filter, we will extend from the TokenFilter class and override the incrementToken() method.
We will create a simple word-expanding TokenFilter that expands courtesy titles from the short form to the full word. For example, Dr expands to doctor.
How to do it…
Here is the sample code:
public class CourtesyTitleFilter extends TokenFilter {
Map<String,String> courtesyTitleMap = new HashMap<String,String>();
private CharTermAttribute termAttr;
public CourtesyTitleFilter(TokenStream input) {
super(input);
termAttr = addAttribute(CharTermAttribute.class);
courtesyTitleMap.put("Dr", "doctor");
courtesyTitleMap.put("Mr", "mister");
courtesyTitleMap.put("Mrs", "miss");
}
public boolean incrementToken() throws IOException {
if (!input.incrementToken())
...