What should a classifier system learn and how should we measure it?Tim Kovacs, What should a classifier system learn and how should we measure it?. Journal of Soft Computing, 6 (3-4). ISSN 1432-7643, pp. 171–182. June 2002. No electronic version available.
We consider the issues of how a classifier system should learn to represent a Boolean function, and how we should measure its progress in doing so. We identify four properties which may be desirable of a representation; that it be complete, accurate, minimal and non-overlapping. We distinguish two categories of learning metric, introduce new metrics and evaluate them. We demonstrate the superiority of population state metrics over performance metrics in two situations, and in the process find evidence of XCS's strong bias against overlapping rules.
[For a revised version please see the chapter in "Strength or Accuracy: Credit Assignment in Learning Classifier Systems" PhD Thesis, 2002. School of Computer Science. University of Birmingham. Birmingham, U.K. http://www.cs.bris.ac.uk/~kovacs/author.directory/thesis/thesis.html]