## Bias-Optimal Incremental Problem Solving (2003)

### Cached

### Download Links

- [ftp.idsia.ch]
- [books.nips.cc]
- [ftp.idsia.ch]
- DBLP

### Other Repositories/Bibliography

Venue: | In Advances in Neural Information Processing Systems 15 |

Citations: | 14 - 8 self |

### BibTeX

@INPROCEEDINGS{Schmidhuber03bias-optimalincremental,

author = {Jürgen Schmidhuber},

title = {Bias-Optimal Incremental Problem Solving},

booktitle = {In Advances in Neural Information Processing Systems 15},

year = {2003},

pages = {1571--1578},

publisher = {MIT Press}

}

### OpenURL

### Abstract

Given is a problem sequence and a probability distribution (the bias) on programs computing solution candidates. We present an optimally fast way of incrementally solving each task in the sequence. Bias shifts are computed by program prefixes that modify the distribution on their suffixes by reusing successful code for previous tasks (stored in non-modifiable memory). No tested program gets more runtime than its probability times the total search time. In illustrative experiments, ours becomes the first general system to learn a universal solver for arbitrary disk Towers of Hanoi tasks (minimal solution size 2^n - 1). It demonstrates the advantages of incremental learning by profiting from previously solved, simpler tasks involving samples of a simple context free language.

### Citations

1680 | An introduction to Kolmogorov complexity and its applications
- Li, Vitányi
- 1997
(Show Context)
Citation Context ...rogram in the alphabetical ¡�� list, then LSEARCH (for Levin Search) [6] will need at ��¨ ��¨��£��������¨���¨��£��� most steps — the constant f=-=actor ¡ � may be huge but does not depend � on . Compare [11, 7, 3]-=-. Recently Hutter developed a more complex asymptotically optimal search algorithm for all well-defined problems [3]. HSEARCH (for Hutter Search) cleverly allocates part of the total search time for s... |

1302 | Reinforcement Learning: A Survey
- Kaelbling, Littman, et al.
- 1996
(Show Context)
Citation Context ...f for � any maximal total search � ����� ��© time it is guaranteed to solve any ����� problem if � it ��¨�������������¨������=-=���� �������ssatisfying . has a solution ��� Unlike reinforcement learners [4]-=- and heuristics such as Genetic Programming [2], OOPS (section 2.2) will bes-bias-optimal, wheresis a small and acceptable number, such as 8.s2.1 OOPS Prerequisites: Multitasking & Prefix Tracking Thr... |

215 |
A representation for the adaptive generation of simple sequential programs
- Cramer
- 1985
(Show Context)
Citation Context ...me it is guaranteed to solve any ����� problem if � it ��¨�������������¨���������� �������ssatisfying . has a solution �=-=�� Unlike reinforcement learners [4] and heuristics such as Genetic Programming [2], OO-=-PS (section 2.2) will bes-bias-optimal, wheresis a small and acceptable number, such as 8.s2.1 OOPS Prerequisites: Multitasking & Prefix Tracking Through Method “Try” The Turing machine-based setu... |

118 |
Universal sequential search problems
- Levin
- 1973
(Show Context)
Citation Context ...ss, if some unknown optimal program� requires ��¨��£� steps to solve a problem instance of � size , � and happens to be � the -th program in the alphabetical ¡�� list, then LS=-=EARCH (for Levin Search) [6] will need at ��¨ ��¨��£������-=-��¨���¨��£��� most steps — the constant factor ¡ � may be huge but does not depend � on . Compare [11, 7, 3]. Recently Hutter developed a more complex asymptotically opti... |

62 | Optimal ordered problem solver
- Schmidhuber
- 2004
(Show Context)
Citation Context ...ough a maze than the one found during the search for a solution to � ��� � task . depend on solutions for ����¨�������� We are searching for a single program solving=-= all tasks encountered so far (see [9] for variants of this setup).-=- Inductively suppose we have solved the firststasks through programs stored below � ��������� ¢ address , and that the most recently found program starting at address ����... |

62 | Shifting inductive bias with success-story algorithm, adaptive Levin seach, and incremental self-improvement
- Schmidhuber, Zhao, et al.
- 1997
(Show Context)
Citation Context ... overhead) with respect to the current task. Our only bias shifts are due to freezing programs once they have solved a problem. That is, unlike the learning rate-based bias shifts of ADAPTIVE LSEARCH =-=[10], th-=-ose of OOPS do not reduce probabilities of programs that were meaningful and executable before the addition of any new � � . Only formerly meaningless, interrupted programs trying to access code f... |

53 | Learning and Problem Solving with Multilayer Connectionist Systems
- Anderson
- 1986
(Show Context)
Citation Context ...aller, transfer all disks to the third peg. Remarkably, the fastest way of solving this famous problem requires ¡�¢�¤�¦ moves ¨s� ©�� . Untrained humans find it hard to solve instanc=-=ess��� . Anderson [1] appli-=-ed traditional reinforcement learning methods and was able to solve instances up tos��� , solvable within at most 7 moves. Langley [5] used learning production systems and was able to solve Hano... |

39 | Learning to Search: From Weak Methods to Domain-Specific Heuristics
- Langley
- 1985
(Show Context)
Citation Context ...ained humans find it hard to solve instancess��� . Anderson [1] applied traditional reinforcement learning methods and was able to solve instances up tos��� , solvable within at most 7 mov=-=es. Langley [5] used -=-learning production systems and was able to solve Hanoi instances up tos��� , solvable within at most 31 moves. Traditional nonlearning planning procedures systematically explore all possible mo... |

35 | The fastest and shortest algorithm for all well-defined problems
- Hutter
(Show Context)
Citation Context ...rogram in the alphabetical ¡�� list, then LSEARCH (for Levin Search) [6] will need at ��¨ ��¨��£��������¨���¨��£��� most steps — the constant f=-=actor ¡ � may be huge but does not depend � on . Compare [11, 7, 3]-=-. Recently Hutter developed a more complex asymptotically optimal search algorithm for all well-defined problems [3]. HSEARCH (for Hutter Search) cleverly allocates part of the total search time for s... |

30 | An application of algorithmic probability to problems in artificial intelligence
- Solomonoff
- 1986
(Show Context)
Citation Context ...rogram in the alphabetical ¡�� list, then LSEARCH (for Levin Search) [6] will need at ��¨ ��¨��£��������¨���¨��£��� most steps — the constant f=-=actor ¡ � may be huge but does not depend � on . Compare [11, 7, 3]-=-. Recently Hutter developed a more complex asymptotically optimal search algorithm for all well-defined problems [3]. HSEARCH (for Hutter Search) cleverly allocates part of the total search time for s... |

16 |
FORTH - a language for interactive computing
- Moore, Leach
(Show Context)
Citation Context ...d on matrix operations for neural network-like parallel architectures, etc. For the experiments we wrote an interpreter for an exemplary, stack-based, universal programming language inspired by FORTH =-=[8], -=-whose disciples praise its beauty and the compactness of its programs. Each task’s tape holds its state: various stack-like data structures represented as sequences of integers, including a data sta... |