Abstract

Abstract — This paper compares the use of temporal difference learning (TDL) versus co-evolutionary learning (CEL) for acquiring position evaluation functions for the game of Othello. The paper provides important insights into the strengths and weaknesses of each approach. The main findings are that for Othello, TDL learns much faster than CEL, but that properly tuned CEL can learn better playing strategies. For CEL, it is essential to use parent-child weighted averaging in order to achieve good performance. Using this method a high quality weighted piece counter was evolved, and was shown to significantly outperform a set of standard heuristic weights.

Citations

649	Reinforcement learning - Sutton, Barto - 1998
100	A.D.: Co-evolution in the successful learning of backgammon strategy - Pollack, Blair - 1998
45	Evolving an expert checkers playing program without using human expertise - Fogel, Chellapilla - 2001
44	A pattern classification approach to evaluation function learning - Lee - 1988
21	Co-evolution versus self-play temporal difference learning for acquiring position evaluation in smallboard go - Runarsson, Lucas - 2005
8	Observing the evolution of neural networks learning to play the game of Othello - Chong, Tan, et al. - 2005
8	Strategy acquisition for the game “Othello” based on reinforcement learning - Ishii, Hayashi - 1999
2	ProbCut: An effective selective extension of the AalphaBeta algorithm - Buro - 1995
2	a strong learning othello program - “LOGISTELLO - 1997
2	Board evaluation for the Virus game - Cowling - 2005