Problèmes d'intersubjectivité dans l'évaluation des analyseurs syntaxiques
Salah Ait-Mokhtar, Caroline Hagege, Agnes Sandor
In this article we discuss basic questions concerning the evaluation of syntactic parsers. We pay special
attention to the definition of a reference annotation scheme with syntactic dependency structures for a "black
box" comparative evaluation over real-world corpora. We raise the question of intersubjectivity that is little
studied in the literqture. This problem does not only concern the definition of the dependencies (e.g. What's a
subject ? What's a modifier ? What should be the relationship between dependency types and parts of
speech ? etc.), but also the segmentation of texts into words and sentences. We give a number of examples
that illustrate these difficulties and propose some solutions.
TALN Conference, Batz-sur-mer, France, June 11-14, 2003.