如何在使用lucene.net进行搜索时启用词干？

How to enable stemming when searching using lucene.net?

如何在使用lucene.net进行搜索时启用词干?

2 个解决方案

#1

To do this you need to write your own analyzer class. This is relatively straightforward. Here is the one that I am using. It combines stop word filtering. Porter stemming and (this may be too much for your needs) stripping of accents from characters.

为此,您需要编写自己的分析器类。这是相对简单的。这是我正在使用的那个。它结合了停用词过滤功能。 Porter阻止和(这可能对您的需求来说太多)从角色中剥离重音符号。

/// <summary>
/// An analyzer that implements a number of filters. Including porter stemming,
/// Diacritic stripping, and stop word filtering.
/// </summary>
public class CustomAnalyzer : Analyzer
{
    /// <summary>
    /// A rather short list of stop words that is fine for basic search use.
    /// </summary>
    private static readonly string[] stopWords = new[]
    {
        "0", "1", "2", "3", "4", "5", "6", "7", "8",
        "9", "000", "$", "£",
        "about", "after", "all", "also", "an", "and",
        "another", "any", "are", "as", "at", "be",
        "because", "been", "before", "being", "between",
        "both", "but", "by", "came", "can", "come",
        "could", "did", "do", "does", "each", "else",
        "for", "from", "get", "got", "has", "had",
        "he", "have", "her", "here", "him", "himself",
        "his", "how","if", "in", "into", "is", "it",
        "its", "just", "like", "make", "many", "me",
        "might", "more", "most", "much", "must", "my",
        "never", "now", "of", "on", "only", "or",
        "other", "our", "out", "over", "re", "said",
        "same", "see", "should", "since", "so", "some",
        "still", "such", "take", "than", "that", "the",
        "their", "them", "then", "there", "these",
        "they", "this", "those", "through", "to", "too",
        "under", "up", "use", "very", "want", "was",
        "way", "we", "well", "were", "what", "when",
        "where", "which", "while", "who", "will",
        "with", "would", "you", "your",
        "a", "b", "c", "d", "e", "f", "g", "h", "i",
        "j", "k", "l", "m", "n", "o", "p", "q", "r",
        "s", "t", "u", "v", "w", "x", "y", "z"
    };

    private Hashtable stopTable;

    /// <summary>
    /// Creates an analyzer with the default stop word list.
    /// </summary>
    public CustomAnalyzer() : this(stopWords) {}

    /// <summary>
    /// Creates an analyzer with the passed in stop words list.
    /// </summary>
    public CustomAnalyzer(string[] stopWords)
    {
        stopTable = StopFilter.MakeStopSet(stopWords);       
    }

    public override TokenStream TokenStream(string fieldName, System.IO.TextReader reader)
    {
        return new PorterStemFilter(new ISOLatin1AccentFilter(new StopFilter(new LowerCaseTokenizer(reader), stopWords)));
    }
}

#2

You can use Snowball or PorterStemFilter. See the Java Analyzer documentation as a guide to combining different Filters/Tokenizers/Analyzers. Note you have to use the same analyzer for indexing and retrieval, so that handling stemming should start at indexing time.

您可以使用Snowball或PorterStemFilter。有关组合不同过滤器/标记器/分析器的指南,请参阅Java Analyzer文档。请注意,您必须使用相同的分析器进行索引和检索,因此处理词干应该从索引时开始。