Quickly compare a string against a Collection in Java

Question

I am trying to calculate edit distances of a string against a collection to find the closest match. My current problem is that the collection is very large (about 25000 items), so I had to narrow down the set to just strings of similar lengths but that still would only narrow it down to a few thousand strings and this still is very slow. Is there a datastructure that allows for a quick lookup of similar strings or is there another way I could address this problem?

Answer 1

Sounds like a BK-tree might be what you want. Here's an article discussing them: http://blog.notdot.net/2007/4/Damn-Cool-Algorithms-Part-1-BK-Trees. A quick Google yields some Java implementations.

Answer 2

If your criteria for 'similar' define a total ordering, you should be able to define a Comparator and use a TreeSet to find the closest matches (eg using the ceiling and floor methods).

Answer 3

Levenshtein Automata allow for fast selection of a set of words from a large dictionary such that they are within the given Levenshtein distance from a given word.

See: Schulz K, Mihov S. (2002) Fast String Correction with Levenshtein-Automata.