A Simple Lexicographic Ranker and Probability EstimatorPeter Flach, Edson Takashi Matsubara, A Simple Lexicographic Ranker and Probability Estimator. 18th European Conference on Machine Learning. Joost N. Kok, Jacek Koronacki, Ramon Lopez de Mantaras, Stan Matwin, Dunja Mladenic, Andrzej Skowron, (eds.). ISBN 978-3-540-74975-2, pp. 575–582. September 2007. PDF, 287 Kbytes. External information
Given a binary classification task, a ranker sorts a set of instances from highest to lowest expectation that the instance is positive. We propose a lexicographic ranker, LexRank, whose rankings are derived not from scores, but from a simple ranking of attribute values obtained from the training data. When using the odds ratio to rank the attribute values we obtain a restricted version of the naive Bayes ranker. We systematically develop the relationships and differences between classification, ranking, and probability estimation, which leads to a novel connection between the Brier score and ROC curves. Combining LexRank with isotonic regression, which derives probability estimates from the ROC convex hull, results in the lexicographic probability estimator LexProb. Both LexRank and LexProb are empirically evaluated on a range of data sets, and shown to be highly effective.