Publication Search Form

Keywords

Authors

Year

We found publication with these paramters.

Coreference Resolution Evaluation Based on Descriptive Specificity

François Trouilleux, Eric Gaussier, Gabriel Bès, Annie Zaenen
This paper introduces a new evaluation method for the coreference resolution task. Considering that coreference resolution is a matter of linking expressions to discourse referents, we set our evaluation criteron in terms of an evaluation of the denotations assigned to the expressions. This criterion requires that the coreference chains identified in one annotation stand in a one-to-one correspondence with the coreference chains in the other. To determine this correspondence and with a view to keep closer to what human interpretation of the coreference chains would be, we take into account the fact that, in a coreference chain, some expressions are more specific to their referent than others. With this observation in mind, we measure the similarity between the chains in one annotation and the chains in the other, and then compute the optimal similarity between the two annotations. Evaluation then consists in checking whether the denotations assigned to the expressions are correct or not. New measures to analyse errors are also introduced. A comparison with other methods is given at the end of the paper.
Proceedings of the Second International Conference on Language Resources and Evaluation (LREC 2000). Athens, Greece. 2000.
2000
1999/044

Attachments

lrec2000.pdf (87.03 kB)