Skip to main content

Complete action map or best action map in accuracy-based reinforcement learning classifier systems

Masaya Nakata, Pier Luca Lanzi, Tim Kovacs, Keiki Takadama, Complete action map or best action map in accuracy-based reinforcement learning classifier systems. Genetic and Evolutionary Computation Conference, GECCO '14. ISBN 978-1-4503-2662-9, pp. 557–564. July 2014. PDF, 676 Kbytes.

Abstract

We study two existing Learning Classifier Systems (LCSs): XCS, which has a complete map (which covers all actions in each state), and XCSAM, which has a best action map (which covers only the highest-return action in each state). This allows XCSAM to learn with a smaller population size limit (but larger population size) and to learn faster than XCS on well-behaved tasks. However, many tasks have difficulties like noise and class imbalances. XCS and XCSAM have not been compared on such problems before. This pa- per aims to discover which kind of map is more robust to these difficulties. We apply them to a classification problem (the multiplexer problem) with class imbalance, Gaussian noise or alternating noise (where we return the reward for a different action). We also compare them on real-world data from the UCI repository without adding noise. We analyze how XCSAM focuses on the best action map and introduce a novel deletion mechanism that helps to evolve classifiers towards a best action map. Results show the best action map is more robust (has higher accuracy and sometimes learns faster) in all cases except small amounts of alternating noise.

Bibtex entry.

Contact details

Publication Admin