Unpredictability in the running time of complete search procedures can often be explained by the phenomenon of “heavy-tailed cost distributions”, meaning that at any time during the experiment there is a non-negligible probability of hitting a problem that requires exponentially more time to solve than any that has been encountered before (Gomes et al. 1998a). We present a general method for introducing controlled randomization into complete search algorithms. The “boosted ” search methods provably eliminate heavy-tails to the right of the median. Furthermore, they can take advantage of heavy-tails to the left of the median (that is, a nonnegligible chance of very short runs) to dramatically shorten the solution time. We demonstrate speedups of several orders of magnitude for state-of-the-art complete search procedures running on hard, real-world problems.

Abstract. We study the runtime distributions of backtrack procedures for propositional satisfiability and constraint satisfaction. Such procedures often exhibit a large variability in performance. Our study reveals some intriguing properties of such distributions: They are often characterized by very long tails or “heavy tails”. We will show that these distributions are best characterized by a general class of distributions that can have infinite moments (i.e., an infinite mean, variance, etc.). Such nonstandard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We also show how random restarts can effectively eliminate heavy-tailed behavior. Furthermore, for harder problem instances, we observe long tails on the left-hand side of the distribution, which is indicative of a non-negligible fraction of relatively short, successful runs. A rapid restart strategy eliminates heavy-tailed behavior and takes advantage of short runs, significantly reducing expected solution time. We demonstrate speedups of up to two orders of magnitude on SAT and CSP encodings of hard problems in planning, scheduling, and circuit synthesis. Key words: satisfiability, constraint satisfaction, heavy tails, backtracking 1.

Abstract. Iterated random functions are used to draw pictures or simulate large Ising models, among other applications. They offer a method for studying the steady state distribution of a Markov chain, and give useful bounds on rates of convergence in a variety of examples. The present paper surveys the field and presents some new examples. There is a simple unifying idea: the iterates of random Lipschitz functions converge if the functions are contracting on the average. 1. Introduction. The

This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coherent picture of this framework. A second goal is to describe how this topic fits into the even larger field of MR methods and concepts–in particular making ties to topics such as wavelets and multigrid methods. A third is to provide several alternate viewpoints for this body of work, as the methods and concepts we describe intersect with a number of other fields. The principle focus of our presentation is the class of MR Markov processes defined on pyramidally organized trees. The attractiveness of these models stems from both the very efficient algorithms they admit and their expressive power and broad applicability. We show how a variety of methods and models relate to this framework including models for self-similar and 1/f processes. We also illustrate how these methods have been used in practice. We discuss the construction of MR models on trees and show how questions that arise in this context make contact with wavelets, state space modeling of time series, system and parameter identification, and hidden

in signal and image processing, including image denoising, coding, and super-resolution. # 2001 Academic Press 1. INTRODUCTION Stochastic models of natural images underlie a variety of applications in image processing and low-level computer vision, including image coding, denoising and 1 MW supported by NSERC 1967 fellowship; AW and MW by AFOSR Grant F49620-98-1-0349 and ONR Grant N00014-91-J-1004. Address correspondence to MW. 2 ES supported by NSF Career Grant MIP-9796040 and an Alfred P. Sloan fellowship. 89 1063-5203/01 $35.00 Copyright # 2001 by Academic Press All rights of reproduction in any form reserved. 90 WAINWRIGHT, SIMONCELLI, AND WILLSKY restoration, interpolation and synthesis. Accordingly, the past decade has witnessed an increasing amount of research devoted to developing stochastic models of images (e.g., [19, 38, 45, 48, 55]). Simultaneously, wavel

INTRODUCTION 1.1.1 Background Since the seminal study of Leland, Taqqu, Willinger and Wilson [41] which set the groundwork for considering self-similarity an important notion in the understanding of network traffic including the modeling and analysis of network performance, an explosion of work has ensued investigating the multifaceted nature of this phenomenon. 1 The long held paradigm in the communication and performance communities has been that voice traffic and, by extension, data traffic are adequately described by certain Markovian models (e.g., Poisson) which are amenable to accurate analysis and efficient control. The first property stems from the well-developed field of Markovian analysis which allows tight equilibrium bounds on performance variables such as the waiting time in various queueing systems to be found. This also 1 For a non-technical account of the discovery of the self-similar nature of network traffic, including parallel effort

Recent evidence suggests that some characteristics of computer and telecommunications systems may be well described using heavy tailed distributions — distributions whose tail declines like a power law, which means that the probability of extremely large observations is non-negligible. For example, such distributions have been found to describe the lengths of bursts in network traffic and the sizes of files in some systems. As a result, system designers are increasingly interested in employing heavy-tailed distributions in simulation workloads. Unfortunately, these distributions have properties considerably different from the kinds of distributions more commonly used in simulations; these properties make simulation stability hard to achieve. In this paper we explore the difficulty of achieving stability in such simulations, using tools from the theory of stable distributions. We show that such simulations exhibit two characteristics related to stability: slow convergence to steady state, and high variability at steady state. As a result, we argue that such simulations must be treated as effectively always in a transient condition. One way to address this problem is to introduce the notion of time scale as a parameter of the simulation, and we discuss methods for simulating such systems while explicitly incorporating time scale as a parameter. 1

Highly Optimized Tolerance (HOT) was recently introduced as a conceptual framework to study fundamental aspects of complexity. HOT is motivated primarily by systems from biology and engineering and emphasizes 1) highly structured, nongeneric, selfdissimilar internal configurations and 2) robust, yet fragile external behavior. HOT claims these are the most important features of complexity and are not accidents of evolution or artifices of engineering design, but are inevitably intertwined and mutually reinforcing. In the spirit of this collection, our paper contrasts HOT with alternative perspectives on complexity, drawing on both real world examples and also model systems, particularly those from Self-Organized Criticality (SOC).

Combinatorial search methods often exhibit a large variability in performance. We study the cost profiles of combinatorial search procedures. Our study reveals some intriguing properties of such cost profiles. The distributions are often characterized by very long tails or "heavy tails". We will show that these distributions are best characterized by a general class of distributions that have no moments (i.e., an infinite mean, variance, etc.). Such non-standard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We believe this is the first finding of these distributions in a purely computational setting. We also show how random restarts can effectively eliminate heavy-tailed behavior, thereby dramatically improving the overall performance of a search procedure.

. The aim of this note is to study the distribution and the asymptotic behavior of the exponential functional A t := R t 0 e s ds, where ( s ; s 0) denotes a L'evy process. When A1 ! 1, we show that in most cases, the law of A1 is a solution of an integrodifferential equation ; moreover, this law is characterized by its integral moments. When the process is asymptotically ff-stable, we prove that t \Gamma1=ff log A t converges in law, as t !1, to the supremum of an ff-stable L'evy process ; in particular, if E [ 1 ] ? 0, then ff = 1 and (1=t) log A t converges almost surely to E [ 1 ]. Eventually, we use Girsanov's transform to give the explicit behavior of E \Theta (a +A t ()) \Gamma1 as t ! 1, where a is a constant, and deduce from this the rate of decay of the tail of the distribution of the maximum of a diffusion process in a random L'evy environment. 1. Introduction We first describe three different sources of interest for exponential functionals of Brownian mot...