The two-parameter Poisson-Dirichlet distribution, denoted pd(ff; `), is a distribution on the set of decreasing positive sequences with sum 1. The usual Poisson-Dirichlet distribution with a single parameter `, introduced by Kingman, is pd(0; `). Known properties of pd(0; `), including the Markov chain description due to Vershik-Shmidt-Ignatov, are generalized to the two-parameter case. The size-biased random permutation of pd(ff; `) is a simple residual allocation model proposed by Engen in the context of species diversity, and rediscovered by Perman and the authors in the study of excursions of Brownian motion and Bessel processes. For 0 ! ff ! 1, pd(ff; 0) is the asymptotic distribution of ranked lengths of excursions of a Markov chain away from a state whose recurrence time distribution is in the domain of attraction of a stable law of index ff. Formulae in this case trace back to work of Darling, Lamperti and Wendel in the 1950's and 60's. The distribution of ranked lengths of e...

For each finite measure on [0

The Blackwell-MacQueen description of sampling from a Dirichlet random distribution on an abstract space is reviewed, and extended to a general family of random discrete distributions. Results are obtained by application of Kingman's theory of partition structures. 1 Introduction Blackwell and MacQueen [10] described the construction of a Dirichlet prior distribution by a generalization of P'olya's urn scheme. While the notion of a random discrete probability measure governed by a Dirichlet distribution was first developed in the setting of Bayesian statistics [30, 26, 27, 28], this idea has applications in other fields. The distribution of the ranked masses of atoms in a Dirichlet distribution, called the Poisson-Dirichlet (pd) distribution [45], appears as an asymptotic distribution in number theory [14, 8, 67, 16], combinatorics [65, 68, 69, 34], and population genetics [70, 24]. Though the finite dimensional distributions of the pd distribution are difficult to describe explicitl...

Abstract not found

Developing better methods for segmenting continuous text into words is important for improving the processing of Asian languages, and may shed light on how humans learn to segment speech. We propose two new Bayesian word segmentation methods that assume unigram and bigram models of word dependencies respectively. The bigram model greatly outperforms the unigram model (and previous probabilistic models), demonstrating the importance of such dependencies for word segmentation. We also show that previous probabilistic models rely crucially on suboptimal search procedures. 1

Partition-valued and measure-valued coalescent Markov processes are constructed whose state describes the decomposition of a finite total mass m into a finite or countably infinite number of masses with sum m, and whose evolution is determined by the following intuitive prescription: each pair of masses of magnitudes x and y runs the risk of a binary collision to form a single mass of magnitude x+y at rate (x; y), for some non-negative, symmetric collision rate kernel (x; y). Such processes with finitely many masses have been used to model polymerization, coagulation, condensation, and the evolution of galactic clusters by gravitational attraction. With a suitable choice of state space, and under appropriate restrictions on and the initial distribution of mass, it is shown that such processes can be constructed as Feller or Feller-like processes. A number of further results are obtained for the additive coalescent with collision kernel (x; y) = x + y. This process, which arises fro...

Explicit formulae are obtained for the distribution of various random partitions of a positive integer n, both ordered and unordered, derived from the zero set M of a Brownian motion by the following scheme: pick n points uniformly at random from [0; 1], and classify them by whether they fall in the same or different component intervals of the complement of M . Corresponding results are obtained for M the range of a stable subordinator and for bridges defined by conditioning on 1 2 M . These formulae are related to discrete renewal Research supported by N.S.F. Grants MCS91-07531 and DMS94-04345 theory by a general method of discretizing a subordinator using the points of an independent homogeneous Poisson process. Keywords: composition, excursion, local time, random set, renewal. 1 Introduction A partition of n is an unordered collection of positive integers with sum n, usually coded by the vector of counts (m j ; 1 j n), where m j is the number of j's in the partition. The n...

The Indian buffet process (IBP) is a Bayesian nonparametric distribution whereby objects are modelled using an unbounded number of latent features. In this paper we derive a stick-breaking representation for the IBP. Based on this new representation, we develop slice samplers for the IBP that are efficient, easy to implement and are more generally applicable than the currently available Gibbs sampler. This representation, along with the work of Thibaux and Jordan [17], also illuminates interesting theoretical connections between the IBP, Chinese restaurant processes, Beta processes and Dirichlet processes. 1

Abstract. We study a 2-parametric family of probability measures on an infinite– dimensional simplex (the Thoma simplex). These measures originate in harmonic analysis on the infinite symmetric group (S. Kerov, G. Olshanski and A. Vershik, Comptes Rendus Acad. Sci. Paris I 316 (1993), 773-778). Our approach is to interprete them as probability distributions on a space of point configurations, i.e., as certain point stochastic processes, and to find the correlation functions of these processes. In the present paper we relate the correlation functions to the solutions of certain multidimensional moment problems. Then we calculate the first correlation function which leads to a conclusion about the support of the initial measures. In the appendix, we discuss a parallel but more elementary theory related to the well–known Poisson–Dirichlet distribution. The higher correlation functions are explicitly calculated in the subsequent paper (A. Borodin). In the third part (A. Borodin and G. Olshanski) we discuss some applications and relationships with the random matrix theory. The goal of our work is to understand new phenomena in noncommutative harmonic analysis which arise when the irreducible representations depend on countably many continuous parameters.

Invariance of a random discrete distribution under size-biased permutation is equivalent to a conjunction of symmetry conditions on its finite-dimensional distributions. This is applied to characterize residual allocation models with independent factors that are invariant under size-biased permutation. Apart from some exceptional cases and minor modifications, such models form a two-parameter family of generalized Dirichlet distributions. RESIDUAL ALLOCATION MODEL; EXCHANGEABLE RANDOM PARTITION; GENERALIZED DIRICHLET DISTRIBUTION. 1991 Math. Subject Classification: Primary 62E10. Secondary 60G09 Research supported by N.S.F. Grants MCS91-07531 and DMS-9404345 1 Introduction Let (P n ) = (P 1 ; P 2 ; \Delta \Delta \Delta) be a random discrete distribution over f1; 2; \Delta \Delta \Deltag. That is to say, the P n are non-negative random variables with P n P n = 1. Define ( ~ P n ), a size-biased random permutation (SBP) of (P n ), by the following procedure. Given (P n ), let X(...