EMMA: A Novel Evaluation Metric for Morphological Analysis - Experimental Results in DetailSebastian Spiegler, EMMA: A Novel Evaluation Metric for Morphological Analysis - Experimental Results in Detail. CSTR-10-004, University of Bristol. July 2010. PDF, 121 Kbytes.
This technical report is intended to be a supplement to the conference paper: "Sebastian Spiegler and Christian Monson. EMMA: A Novel Metric for Morphological Analysis. 23rd International Conference on Computational Linguistics (COLING), 2010." It details described experiments including the evaluation of 93 system-language pairs from Morpho Challenge (MC) 2007, 2008 and 2009 comparing the MC metric against the EMMA metric proposed in the paper above, the comparison against a real-world natural language task - information retrieval, and ways how the previous MC metric could be gamed which have less influence on the EMMA metric.