Abstract. We study the problem of computing conditional probabilities, a fundamental operation in statistics and machine learning. In the elementary discrete setting, conditional probability is defined axiomatically and the search for more constructive definitions is the subject of a rich literature in probability theory and statistics. In the discrete or dominated setting, under suitable computability hypotheses, conditional probabilities are computable. However, we show that in general one cannot compute conditional probabilities. We do this by constructing a pair of computable random variables in the unit interval whose conditional distribution encodes the halting problem at almost every point. We show that this result is tight, in the sense that given an oracle for the halting problem, one can compute this conditional distribution. On the other hand, we show that conditioning in abstract settings is computable in the presence of certain additional structure, such as independent absolutely continuous noise. 1.

Recurrences arise naturally in analyzing the complexity of recursive algorithms and in probabilistic analysis. We introduce some basic techniques for solving recurrences. A recurrence is a recursive relation for a complexity function T(n). Here are two examples: F(n) = F(n − 1) + F(n − 2) (1) and Looks famil-T(n) = n + 2T(n/2). (2) iar? The reader may recognize the first as the recurrence for Fibonacci numbers, and the second as the complexity of the Merge Sort, described in Lecture 1. These recurrences have 1 the following “separable form”: T(n) = G(n, T(n1),..., T(nk)) (3) where G(x0, x1,..., xk) is a function or expression in k + 1 variables and n1,...,nk are all strictly less than n. Each ni is a function of n. E.g., in (1), we have k = 2 and n1 = n − 1, n2 = n − 2. But in (2), we have k = 1 and n1 = n/2. What does it mean to “solve ” recurrences such as equations (1) and (2)? The Fibonacci recurrence and the Mergesort recurrence has the following well-known solutions:

We prove a uniformly computable version of de Finetti’s theorem on exchangeable sequences of real random variables. As a consequence, exchangeable stochastic processes in probabilistic functional programming languages can be automatically rewritten as procedures that do not modify non-local state. Along the way, we prove that a distribution on the unit interval is computable if and only if its moments are uniformly computable.