Citation Context

... go to zero. Ideas for this kind of algorithm can be found in two fields of research: n-armed bandit problems, and discrete stochastic optimization. n-armed bandit techniques (Sutton and Barto’s book =-=[25]-=- provides an introduction) are the basis for the MonteCarlo tree search algorithm of Chang, Fu and Marcus [12]. Optimal schemes for the allocation of simulations in discrete stochastic optimization [1...

Citation Context

...so refute other moves of a node. Since formal methods seem difficult to apply, the backup operator of Crazy Stone was determined empirically, by an algorithm similar to the temporal difference method =-=[24]-=-. In the beginning, the backup method for internal nodes was the external-node method. 1,500 positions were sampled at random from ,s6 R. Coulom self-play games. For each of these 1,500 positions, the...

Citation Context

...1 Introduction When writing a program to play a two-person zero-sum game with perfect information, the traditional approach consists in combining alpha-beta search with a heuristic position evaluator =-=[20]-=-. The heuristic evaluator is based on domainspecific knowledge, and provides values at the leaves of the search tree. This technique has been very successful for games such as chess, draughts, checker...

Citation Context

...tion inaccuracies. Other algorithms with better asymptotic properties (given enough time and memory, they will find an optimal action) have been proposed in the formalism of Markov decision processes =-=[12, 19, 22]-=-. This paper presents a new algorithm for combining Monte-Carlo evaluation with tree search. Its basic structure is described in Section 2. Its selectivity and backup operators are presented in the fo...

Citation Context

...for many games, it has failed for the game of Go. Experienced human Go players still easily outplay the best programs. So, the game of Go remains an open challenge for artificialintelligence research =-=[8]-=-. Among the main difficulties in writing a Go-playing program is the creation of an accurate static position evaluator [15, 8]. When played on a 9 × 9 grid, the complexity of the game of Go, in terms ...

Citation Context

...ns is Monte-Carlo evaluation. Monte-Carlo evaluation consists in averagings2 R. Coulom the outcome of several continuations. It is an usual technique in games with randomness or partial observability =-=[5, 23, 26, 14, 17]-=-, but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. J...

Citation Context

...ccurate static position evaluator [15, 8]. When played on a 9 × 9 grid, the complexity of the game of Go, in terms of the number of legal positions, is inferior to the complexity of the game of chess =-=[2, 27]-=-, and the number of legal moves per position is similar. Nevertheless, chess-programming techniques fail to produce a player stronger than experienced humans. One reason is that tree search cannot be ...

Citation Context

...5] provides an introduction) are the basis for the MonteCarlo tree search algorithm of Chang, Fu and Marcus [12]. Optimal schemes for the allocation of simulations in discrete stochastic optimization =-=[13, 16, 3]-=-, could also be applied to Monte-Carlo tree search. Although they provide interesting sources of inspiration, the theoretical frameworks of n-armed bandit problems and discrete stochastic optimization...

Citation Context

...he maximum evaluation overestimates the best move, and generates a lot of instability in the search. Other candidates for a backup method would be algorithms that operate on probability distributions =-=[21, 4]-=-. The weakness of these methods is that they have to assume some degree of independence between probability distributions. This assumption of independence is wrong in the case of Monte-Carlo evaluatio...

Citation Context

...ns is Monte-Carlo evaluation. Monte-Carlo evaluation consists in averagings2 R. Coulom the outcome of several continuations. It is an usual technique in games with randomness or partial observability =-=[5, 23, 26, 14, 17]-=-, but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. J...

Citation Context

... usual technique in games with randomness or partial observability [5, 23, 26, 14, 17], but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached =-=[1, 9, 10]-=-. The accuracy of Monte-Carlo evaluation can be improved with tree search. Juillé [18] proposed a selective Monte-Carlo algorithm for single-agent deterministic problems, and applied it successfully t...

Citation Context

...he maximum evaluation overestimates the best move, and generates a lot of instability in the search. Other candidates for a backup method would be algorithms that operate on probability distributions =-=[21, 4]-=-. The weakness of these methods is that they have to assume some degree of independence between probability distributions. This assumption of independence is wrong in the case of Monte-Carlo evaluatio...

Citation Context

...tion inaccuracies. Other algorithms with better asymptotic properties (given enough time and memory, they will find an optimal action) have been proposed in the formalism of Markov decision processes =-=[12, 19, 22]-=-. This paper presents a new algorithm for combining Monte-Carlo evaluation with tree search. Its basic structure is described in Section 2. Its selectivity and backup operators are presented in the fo...

Citation Context

... usual technique in games with randomness or partial observability [5, 23, 26, 14, 17], but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached =-=[1, 9, 10]-=-. The accuracy of Monte-Carlo evaluation can be improved with tree search. Juillé [18] proposed a selective Monte-Carlo algorithm for single-agent deterministic problems, and applied it successfully t...

Citation Context

...can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. Juillé =-=[18]-=- proposed a selective Monte-Carlo algorithm for single-agent deterministic problems, and applied it successfully to grammar induction, sorting-network optimization and a solitaire game. Bouzy [6] also...

Citation Context

... the game of Go remains an open challenge for artificialintelligence research [8]. Among the main difficulties in writing a Go-playing program is the creation of an accurate static position evaluator =-=[15, 8]-=-. When played on a 9 × 9 grid, the complexity of the game of Go, in terms of the number of legal positions, is inferior to the complexity of the game of chess [2, 27], and the number of legal moves pe...

Citation Context

...ns is Monte-Carlo evaluation. Monte-Carlo evaluation consists in averagings2 R. Coulom the outcome of several continuations. It is an usual technique in games with randomness or partial observability =-=[5, 23, 26, 14, 17]-=-, but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. J...

Citation Context

...llé [18] proposed a selective Monte-Carlo algorithm for single-agent deterministic problems, and applied it successfully to grammar induction, sorting-network optimization and a solitaire game. Bouzy =-=[6]-=- also applied a similar method to 9 ×9 Go. The algorithms of Juillé and Bouzy grow a tree by iterative deepening, and prune it by keeping only the best-looking moves after each iteration. A problem wi...

Citation Context

...ns is Monte-Carlo evaluation. Monte-Carlo evaluation consists in averagings2 R. Coulom the outcome of several continuations. It is an usual technique in games with randomness or partial observability =-=[5, 23, 26, 14, 17]-=-, but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. J...

Citation Context

...er boards. For 19x19, an approach based on a global tree search does not seem reasonable. Generalizing the tree search with high-level tactical objectives such as Cazenave and Helmstetter’s algorithm =-=[11]-=- might be an interesting solution. Acknowledgements I thank Bruno Bouzy and Guillaume Chaslot, for introducing me to Monte-Carlo Go. A lot of the inspiration for the research presented in this paper c...

Citation Context

...tion inaccuracies. Other algorithms with better asymptotic properties (given enough time and memory, they will find an optimal action) have been proposed in the formalism of Markov decision processes =-=[12, 19, 22]-=-. This paper presents a new algorithm for combining Monte-Carlo evaluation with tree search. Its basic structure is described in Section 2. Its selectivity and backup operators are presented in the fo...

Citation Context

...compare the expected values of many random variables, this theorem allows to compute a probability that the expected value of one variable is larger than the expected value of another variable. Bouzy =-=[9, 7]-=- used this principle to propose progressive pruning. Progressive pruning cuts off moves whose probability of being best according to the distribution of the central-limit theorem falls below some thre...

Citation Context

...ccurate static position evaluator [15, 8]. When played on a 9 × 9 grid, the complexity of the game of Go, in terms of the number of legal positions, is inferior to the complexity of the game of chess =-=[2, 27]-=-, and the number of legal moves per position is similar. Nevertheless, chess-programming techniques fail to produce a player stronger than experienced humans. One reason is that tree search cannot be ...

Citation Context

...ns is Monte-Carlo evaluation. Monte-Carlo evaluation consists in averagings2 R. Coulom the outcome of several continuations. It is an usual technique in games with randomness or partial observability =-=[5, 23, 26, 14, 17]-=-, but can also be applied to deterministic games, by choosing actions at random until a terminal state is reached [1, 9, 10]. The accuracy of Monte-Carlo evaluation can be improved with tree search. J...

Citation Context

...5] provides an introduction) are the basis for the MonteCarlo tree search algorithm of Chang, Fu and Marcus [12]. Optimal schemes for the allocation of simulations in discrete stochastic optimization =-=[13, 16, 3]-=-, could also be applied to Monte-Carlo tree search. Although they provide interesting sources of inspiration, the theoretical frameworks of n-armed bandit problems and discrete stochastic optimization...

Citation Context

...5] provides an introduction) are the basis for the MonteCarlo tree search algorithm of Chang, Fu and Marcus [12]. Optimal schemes for the allocation of simulations in discrete stochastic optimization =-=[13, 16, 3]-=-, could also be applied to Monte-Carlo tree search. Although they provide interesting sources of inspiration, the theoretical frameworks of n-armed bandit problems and discrete stochastic optimization...

Citation Context

...ssible to find better methods. 5 Game Results As indicated in the abstract, Crazy Stone won the 10th KGS computer-Go tournament, ahead of 8 participants, including GNU Go, Neuro Go, Viking 5, and Aya =-=[28]-=-. This is a spectacular result, but this was only a 6-round tournament, and luck was probably one of the main factor in this victory. In order to test the strength of Crazy Stone more accurately, 100-...