At the European Conference on Machine Learning and Knowledge Discovery (ECML-PKDD 2014) which was held from 15 to 19 September 2014 in Nancy, France, Dr Meelis Kull and Prof Peter Flach from the Intelligent Systems Laboratory received the award for best paper presented at the conference for their paper entitled Reliability Maps: A Tool to Enhance Probability Estimates and Improve Classification Accuracy. This gave Meelis the opportunity to present the work in a plenary session to over 500 delegates.

The paper proposes a general method to assess the reliability of probability estimates output by a classifier in an instance-wise manner. This is relevant, for instance, for obtaining well-calibrated multi-class probabilities from two-class probability scores. Calibrated probabilities are familiar from weather forecasts: if we compare 100 cases where the forecast is '40% rain', we expect it to actually rain in 40 of those 100 cases. Having access to well-calibrated probabilities allows us to make optimal decisions: for example, if the cost of getting wet is twice that of unnecessarily carrying an umbrella, one would carry an umbrella whenever the probability of rain exceeds 33%.

This work was carried out as part of the REFRAME project. Members of the Intelligent Systems Laboratory presented two other papers at the conference, two papers at the co-located Inductive Logic Programming Conference, and four papers at the workshop on Learning over Multiple Contexts.