Hibernate lucene search include "the" and "a" as part of search

Question

I'm using Hibernate Lucene to index a table MOVIE. I want to index and search by the title column, including "the" and "a" as part of the search. Ex, "the last man standing" "A Most Wanted Man". "The" and "A" are relevant as part of the search results.

The problem is if I used index=Index.UN_TOKENIZED in @Field, then no search can be done. If I used Index.TOKENIZED, then I can't search with "the" and "a".

Can someone give me some guidance? Thanks in advance.

Below is the code snippet:

@Field(index = Index.UN_TOKENIZED, store = Store.NO) <<< I've tried Index.TOKENIZED also. @Column(name = "TITLE") private String title;

Code to search:

@Field(index = Index.UN_TOKENIZED, store = Store.NO)  <<< I've tried Index.TOKENIZED also.

@Column(name = "TITLE") private String title;

Code to search: FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(entityManager);

QueryBuilder queryBuilder = fullTextEntityManager.getSearchFactory().buildQueryBuilder().forEntity(Movie.class).get(); Query luceneQuery = queryBuilder.keyword().onField("title") .matching(title.toLowerCase()).createQuery();

return fullTextEntityManager.createFullTextQuery(luceneQuery, Movie.class) .getResultList();

Answer 1

I think this is one solution, by defining the Analyzer. This will tokenize, but will not exclude the common words "the". Also, in the search code, use method QueryBuilder.phrase() instead of keyword() so the search will be performed based on the search phrase.

@AnalyzerDef(name = "custom", tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = LowerCaseFilterFactory.class), @TokenFilterDef(factory = SnowballPorterFilterFactory.class, params = { @Parameter(name = "language", value = "English") }) }) public class Movie {

@Field(index = Index.YES, analyze = Analyze.YES, store = Store.NO)
@Analyzer(definition="custom")
private String title;

... }

Code to search: QueryBuilder qb = fullTextEntityMgr.getSearchFactory() .buildQueryBuilder().forEntity(Movie.class).get();

Query luceneQuery = qb.phrase().onField("title").sentence(term.toLowerCase()).createQuery(); javax.persistence.Query query = fullTextEntityMgr.createFullTextQuery(luceneQuery, Movie.class);