1996-023 - A Detailed Analysis of English Stemming Algorithms
- David Hull,Greg Grefenstette
Xerox technical report
We present a study comparing the performance of traditional stemming algorithms based on suffic removal to linguistic methods performing morphological analysis. The results indicate that most conflation algorithms perform about 5% better than no stemming, and there is little difference between methods in terms of average performance. However, a detailed analysis of individual queries indicates that performance on this level is often highly sensitive to the choice of stemming technique. From this analysis, we can suggest a number of different ways to modify linguistic approaches so that they will be better suited to the stemming problem.