Posted by Kelvin on 03 Mar 2012 at 06:23 pm | Tagged as: Lucene / Solr / Elasticsearch / Nutch
Context-less multi-term autocomplete is difficult.
Given the term "di", we can look at our index and rank terms starting with "di" by frequency and return the n most frequent terms. Solr's TSTLookup and FSTLookup do this very well.
However, given the term "walt di", we can no longer do what we did above for each term and not look silly, especially if the corpus in question is a list of US companies (hint: think mickey mouse". There's little excuse to suggesting "walt discovery" or "walt diners" when our corpus does not contain any documents with that combination of terms.
In the absence of a large number of historical user queries to augment the autocomplete, context is king when it comes to multi-term queries.
The simplest way I can think of doing this, if it is feasible to do so memory-wise, is to store a list of terms and the term that immediately follows it. For example, given the field value "international business machines", mappings would be created for
Out-of-order queries wouldn't be supported with this system, nor would term skips (e.g. international machines).
Here's a method fragment that does just this:
Guava's Multimap is perfect for this, and Solr already has a Guava dependency, so we might as well make full use of it.