## One decade of universal artificial intelligence (2012)

Venue: | In Theoretical Foundations of Artificial General Intelligence |

Citations: | 2 - 2 self |

### BibTeX

@INPROCEEDINGS{Hutter12onedecade,

author = {Marcus Hutter},

title = {One decade of universal artificial intelligence},

booktitle = {In Theoretical Foundations of Artificial General Intelligence},

year = {2012},

publisher = {Atlantis Press}

}

### OpenURL

### Abstract

The first decade of this century has seen the nascency of the first mathematical theory of general artificial intelligence. This theory of Universal Artificial Intelligence (UAI) has made significant contributions to many theoretical, philosophical, and practical AI questions. In a series of papers culminating in book (Hutter, 2005), an exciting sound and complete mathematical model for a super intelligent agent (AIXI) has been developed and rigorously analyzed. While nowadays most AI researchers avoid discussing intelligence, the awardwinning PhD thesis (Legg, 2008) provided the philosophical embedding and investigated the UAI-based universal measure of rational intelligence, which is formal, objective and non-anthropocentric. Recently, effective approximations of AIXI have been derived and experimentally investigated in JAIR paper (Veness et al. 2011). This practical breakthrough has resulted in some impressive applications, finally muting earlier critique that UAI is only a theory. For the first time, without providing any domain knowledge, the same

### Citations

3711 |
Artificial Intelligence: A Modern Approach (Second Edition
- Russell, Norvig
- 2003
(Show Context)
Citation Context ...en proposed in [Pan08] that samples programs according to their algorithmic probability as a way of approximating Solomonoff’s universal a-priori probability, similar to sampling from the speed prior =-=[Sch02]-=-. The most powerful sys- 1 tematic approximation, implementation, and appli- 0.8 cation of AIXI so far is the MC-AIXI-CTW algorithm 0.6 [VNHS10]. It combines Optimal award-winning ideas from 0.4 Chees... |

2621 |
Dynamic Programming
- Bellman
- 1957
(Show Context)
Citation Context ...n environment? The theory, coined UAI, developed in the last decade and explained in [Hut05] says: All you need is Ockham, Epicurus, Turing, Bayes, Solomonoff [Sol64], Kolmogorov [Kol65], and Bellman =-=[Bel57]-=-: Sequential decision theory [Ber06b] (Bellman’s equation) formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. If the enviro... |

1856 | Foundations of Logic Programming - Lloyd - 1987 |

1687 | An Introduction to Kolmogorov Complexity and its Applications - Li, Vitanyi - 1997 |

1245 |
Statistical Decision Theory and Bayesian Analysis
- Berger
- 1985
(Show Context)
Citation Context ... [Ber06b] (Bellman’s equation) formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. If the environment is unknown, Bayesians =-=[Ber93]-=- replace the true distribution by a weighted mixture of distributions from some (hypothesis) class. Using the large class of all (semi)measures that are (semi)computable on a Turing machine bears in m... |

740 |
Computing machinery and intelligence
- Turing
- 1950
(Show Context)
Citation Context ...11]. Normalised Average Reward per Cycle Measures/tests/definitions of intelligence. The history of informal definitions and measures of intelligence [LH07a] and anthropocentric tests of intelligence =-=[Tur50]-=- is long and old. In the last decade various formal definitions, mea14sures and tests have been suggested: Solomonoff induction and Kolmogorov complexity inspired the universal C-test [HO00, HOMC98],... |

524 | Probability Theory: The Logic of Science - Jaynes - 2003 |

523 |
Three approaches to the quantitative definition of information
- Kolmogorov
- 1965
(Show Context)
Citation Context ...] and the TicTacToe 0.2 Biased RPS recent highly successful (in Kuhn Poker Experience Pacman computer Go) upper confi- 0 dence bound algorithm for 100 1000 10000 100000 1000000 expectimax tree search =-=[KS06]-=-. For the first time, without any domain knowledge, the same agent is able to self-adapt to a diverse range of environments. For instance, AIXI, is able to learn from scratch how to play TicTacToe, Pa... |

333 |
The Conscious Mind
- CHALMERS
- 1996
(Show Context)
Citation Context ...AIs in general is not clear and in any case beyond my expertise. I leave this to philosophers of the mind [Cha02] like the world-renowned expert on (the hard problem of) consciousness, David Chalmers =-=[Cha96]-=-. 115 Social Questions Consider now a sophisticated physical humanoid robot like Honda’s ASIMO but equipped with an AIXI brain. The observations ok consist of camera image, microphone signal, and oth... |

278 |
Reinforcement Learning with Selective Perception and Hidden State
- Mccallum
- 1996
(Show Context)
Citation Context ...ttention (http://users.dsic.upv.es/proy/anynt/). Less related/general work. There is of course other less related, less general work, similar in spirit to or with similar aims as UAI/AIXI, e.g. UTree =-=[McC96]-=-, URL [FMRW10], PORL [SHL97, SH99], FOMDP [SB09], FacMDP [SDL07], PSR [SLJ + 03], POMDP [DV09], and others. The feature reinforcement learning approach also belongs to this category [Hut09b, Hut09a, S... |

174 |
Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability
- Hutter
- 2005
(Show Context)
Citation Context ...n which logical reasoning plays no direct role. UAI is a new paradigm to AGI via a path from universal induction to prediction to decision to action. It has been investigated in great technical depth =-=[Hut05]-=- and has already spawned promising formal definitions of rational intelligence, the optimal rational agent AIXI and practical approximations thereof, and put AI on solid mathematical foundations. It s... |

118 | Universal sequential search problems - Levin - 1973 |

116 | Language and Mind - Chomsky - 2006 |

106 | Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations
- Shoham, Leyton-Brown
- 2009
(Show Context)
Citation Context ...mation theory [CT06], sequential decision theory [Ber06b], reinforcement learning [SB98], artificial intelligence [RN10], Bayesian statistics [Ber06a], universal induction [RH11], and rational agents =-=[SLB09]-=-. Various notions of optimality have been considered. The difficulty is coming up with sufficiently strong but still satisfiable notions. Some are weaker than desirable, others are too strong for any ... |

86 | 2005): Statistical and Inductive Inference by Minimum Message - Wallace |

79 |
Elements of information theory, Wiley-Interscience
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...servations can often be clustered into different categories of similar=associated items. For AGI, a universal similarity measure is required. Kolmogorov complexity via the universal similarity metric =-=[CV05]-=- can provide such a measure, but many fundamental questions have yet to be explored: How does association function in AIXI? How can Kolmogorov complexity well-define the (inherently? so far?) ill-defi... |

79 | The context tree weighting method: Basic properties
- Willems, Shtarkov, et al.
- 1995
(Show Context)
Citation Context ...tation, and appli- 0.8 cation of AIXI so far is the MC-AIXI-CTW algorithm 0.6 [VNHS10]. It combines Optimal award-winning ideas from 0.4 Cheese Maze Tiger universal Bayesian data com4x4 Grid pression =-=[WST95]-=- and the TicTacToe 0.2 Biased RPS recent highly successful (in Kuhn Poker Experience Pacman computer Go) upper confi- 0 dence bound algorithm for 100 1000 10000 100000 1000000 expectimax tree search [... |

62 | Optimal ordered problem solver
- Schmidhuber
- 2004
(Show Context)
Citation Context ...he down-scaled still provably optimal AIXItl model [Hut05, Chp.7] based on universal search algorithms [Lev73, Hut02a, Gag07] was still computationally intractable. The Optimal Ordered Problem Solver =-=[Sch04]-=- was the first practical implementation of universal search and has been able to solve open learning tasks such as Towers-of-Hanoi for arbitrary number of disks, robotic behavior, and others. For repe... |

60 | Learning predictive state representations - Singh, Littman, et al. - 2003 |

47 | Efficient structure learning in factoredstate MDPs. AAAI (pp. 645–650
- Strehl, Diuk, et al.
- 2007
(Show Context)
Citation Context ...neral work. There is of course other less related, less general work, similar in spirit to or with similar aims as UAI/AIXI, e.g. UTree [McC96], URL [FMRW10], PORL [SHL97, SH99], FOMDP [SB09], FacMDP =-=[SDL07]-=-, PSR [SLJ + 03], POMDP [DV09], and others. The feature reinforcement learning approach also belongs to this category [Hut09b, Hut09a, SH10, NSH11]. Compression contest. The ongoing Human Knowledge Co... |

42 | Universal Intelligence: A Definition of Machine Intelligence
- Legg, Hutter
- 2007
(Show Context)
Citation Context ...intelligence trying to capture all traits informally [LH07a]. One of the more successful characterizations is: Intelligence measures an agents ability to perform well in a large range of environments =-=[LH07b]-=-. Most traits of intelligence are implicit in and emergent from this definition as these capacities enable an agent to succeed [Leg08]. Convincing formal definitions other than the ones spawned by UAI... |

35 | The fastest and shortest algorithm for all well-defined problems - Hutter |

34 | The Case for Objective Bayesian Analysis
- Berger
- 2006
(Show Context)
Citation Context ...ed in the first place: Kolmogorov complexity [LV08], information theory [CT06], sequential decision theory [Ber06b], reinforcement learning [SB98], artificial intelligence [RN10], Bayesian statistics =-=[Ber06a]-=-, universal induction [RH11], and rational agents [SLB09]. Various notions of optimality have been considered. The difficulty is coming up with sufficiently strong but still satisfiable notions. Some ... |

34 | Logic for Artificial Intelligence - Turner - 1984 |

31 | Self-optimizing and Pareto-optimal policies in general environments based on Bayes-mixtures
- Hutter
(Show Context)
Citation Context ...onments, and a variant is self-optimizing in the sense that asymptotically the accumulated reward is as high as possible, i.e. the same as the maximal reward achievable by a completely informed agent =-=[Hut02b]-=-. AIXI is likely also optimal in other/stronger senses. An axiomatic characterization has also been developed [SH11a, SH11b]. The induction problem. The induction problem is a fundamental problem in p... |

30 | Measuring Universal Intelligence: Towards an Anytime Intelligence Test - Hernandez-Orallo, Dowe - 2010 |

30 | A formal definition of intelligence based on an intensional variant of Kolmogorov complexity - Hernández-Orallo, Minaya-Collado - 1998 |

29 | Mind: A brief introduction - Searle - 2004 |

25 |
Dynamic Programming and Optimal Control, volume I
- Bertsekas
- 2007
(Show Context)
Citation Context ...I, developed in the last decade and explained in [Hut05] says: All you need is Ockham, Epicurus, Turing, Bayes, Solomonoff [Sol64], Kolmogorov [Kol65], and Bellman [Bel57]: Sequential decision theory =-=[Ber06b]-=- (Bellman’s equation) formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. If the environment is unknown, Bayesians [Ber93] r... |

23 | On universal prediction and Bayesian confirmation
- Hutter
(Show Context)
Citation Context ...es in general and the Black 13ravens paradox in particular, reparametrization invariance, the old-evidence problem and ad-hoc hypotheses, and the updating problem [Ear93]. In a series of papers (see =-=[Hut07]-=- for references) it has been shown that Solomonoff’s theory of universal induction essentially solves or circumvents all these problems [RH11]. It is also predictively optimal and has minimal regret f... |

22 |
Machine Super Intelligence
- Legg
- 2008
(Show Context)
Citation Context ...ing (artificial) intelligence. Since the theory is complete, definite answers can be obtained for a large variety of intelligence-related questions, as foreshadowed by the award winning PhD thesis of =-=[Leg08]-=-. Contents. Section 2 provides the context and background for UAI. It will summarize various last century’s paradigms for and approaches to understanding and 3building artificial intelligences, highl... |

18 | www.idsia.ch/~shane/intelligence.html. A collection of definitions of intelligence, 2006
- Legg, Hutter
(Show Context)
Citation Context .... A formal definition incorporating every aspect of intelligence, however, seems difficult. There is no lack of attempts to characterize or define intelligence trying to capture all traits informally =-=[LH07a]-=-. One of the more successful characterizations is: Intelligence measures an agents ability to perform well in a large range of environments [LH07b]. Most traits of intelligence are implicit in and eme... |

16 | Feature reinforcement learning: Part I: Unstructured MDPs - Hutter |

14 | On the computational measurement of intelligence factors - Hernández-Orallo - 2000 |

13 | A Monte Carlo AIXI approximation - Veness, Ng, et al. |

12 | The infinite partially observable Markov Decision Process
- Doshi-Velez
(Show Context)
Citation Context ...confirmation of (universal) hypotheses in general and the Black 13ravens paradox in particular, reparametrization invariance, the old-evidence problem and ad-hoc hypotheses, and the updating problem =-=[Ear93]-=-. In a series of papers (see [Hut07] for references) it has been shown that Solomonoff’s theory of universal induction essentially solves or circumvents all these problems [RH11]. It is also predictiv... |

12 | Foundations of Computational Linguistics. Human-Computer Communication in Natural Language. 2nd edition - Hausser - 2001 |

12 |
The Human Knowledge Compression Prize
- Hutter
- 2006
(Show Context)
Citation Context ...POMDP [DV09], and others. The feature reinforcement learning approach also belongs to this category [Hut09b, Hut09a, SH10, NSH11]. Compression contest. The ongoing Human Knowledge Compression Contest =-=[Hut06]-=- is another outgrowth of UAI. The contest is motivated by the fact that being able to compress well is closely related to being able to predict well and ultimately to act intelligently, thus reducing ... |

11 | Simple algorithmic principles of discovery, subjective beauty, selective attention, curiosity & creativity - Schmidhuber - 2007 |

10 | Feature dynamic Bayesian networks - Hutter |

9 | Universal reinforcement learning
- Farias, Moallemi, et al.
- 2010
(Show Context)
Citation Context ...p://users.dsic.upv.es/proy/anynt/). Less related/general work. There is of course other less related, less general work, similar in spirit to or with similar aims as UAI/AIXI, e.g. UTree [McC96], URL =-=[FMRW10]-=-, PORL [SHL97, SH99], FOMDP [SB09], FacMDP [SDL07], PSR [SLJ + 03], POMDP [DV09], and others. The feature reinforcement learning approach also belongs to this category [Hut09b, Hut09a, SH10, NSH11]. C... |

9 | A reinforcement learning algorithm in partially observable environments using short-term memory - Suematsu, Hayashi - 1999 |

9 | Reinforcement Learning via AIXI Approximation
- Veness, Ng, et al.
- 2010
(Show Context)
Citation Context ...ri probability, similar to sampling from the speed prior [Sch02]. The most powerful sys- 1 tematic approximation, implementation, and appli- 0.8 cation of AIXI so far is the MC-AIXI-CTW algorithm 0.6 =-=[VNHS10]-=-. It combines Optimal award-winning ideas from 0.4 Cheese Maze Tiger universal Bayesian data com4x4 Grid pression [WST95] and the TicTacToe 0.2 Biased RPS recent highly successful (in Kuhn Poker Exper... |

8 |
The singularity is near”, Viking
- Kurzweil
- 2005
(Show Context)
Citation Context ...nce in the sense of being able to solve a wide range of problems generally regarded to require human-level intelligence. Optimists believe that the AI problem can be solved within a couple of decades =-=[Kur05]-=-. Pessimists deny its principle feasibility on religious, philosophical, mathematical, or technical grounds (see [RN10, Chp.26] for a list of arguments). Optimists have refuted/rebutted all those argu... |

7 | Self-modification and mortality in artificial agents
- Orseau, Ring
(Show Context)
Citation Context ...consistent) hypothesis. In order not to ignore Ockham, who would select the simplest hypothesis, Solomonoff defined a universal prior that assigns high/low prior weight to simple/complex environments =-=[RH11]-=-, where Kolmogorov quantifies complexity [LV08]. Their unification constitutes the theory of UAI and resulted in the universal intelligence measure and order relation and the following model/agent AIX... |

7 | A Bayesian approach to model learning in nonMarkovian environment - Suematsu, Hayashi, et al. - 1997 |

6 | Consistency of feature Markov processes - Sunehag, Hutter - 2010 |

5 |
Mesurer l’intelligence d’une machine
- Fiévet
- 2005
(Show Context)
Citation Context ...xtremely general, objective, fundamental, and formal intelligence order relation [Hut05] and a universal intelligence measure [LH07b, Leg08], which have already attracted the popular scientific press =-=[Fié05]-=- and received the SIAI award. Practical instantiations thereof [HOD10, LV11] also received quite some media attention (http://users.dsic.upv.es/proy/anynt/). Less related/general work. There is of cou... |

5 | Open problems in universal induction & intelligence
- Hutter
(Show Context)
Citation Context ...h more coherent view of the field. Indeed, UAI seems to be the first sound and complete mathematical theory of (rational) intelligence. The next section presents a very brief introduction to UAI from =-=[Hut09c]-=-, together with an informal explanation of what the previous sentence actually means. See [Hut05] for formal definitions and results. 3 Universal Artificial Intelligence This section describes the the... |

5 | Asymptotically optimal agents
- Lattimore, Hutter
- 2011
(Show Context)
Citation Context ...uences, which the expectimax planner of AIXI should properly take into account. Particular issues of concern are the interplay of learning and planning (the infamous exploration↔exploitation tradeoff =-=[LH11]-=-). Additional complications that arise from embodied agents will be considered in the next section. Learning. There are many different forms of learning: supervised, unsupervised, semi-supervised, rei... |