## The dichotomy between structure and randomness, arithmetic progressions, and the primes

Citations: | 21 - 1 self |

### BibTeX

@MISC{Tao_thedichotomy,

author = {Terence Tao},

title = {The dichotomy between structure and randomness, arithmetic progressions, and the primes},

year = {}

}

### OpenURL

### Abstract

Abstract. A famous theorem of Szemerédi asserts that all subsets of the integers with positive upper density will contain arbitrarily long arithmetic progressions. There are many different proofs of this deep theorem, but they are all based on a fundamental dichotomy between structure and randomness, which in turn leads (roughly speaking) to a decomposition of any object into a structured (low-complexity) component and a random (discorrelated) component. Important examples of these types of decompositions include the Furstenberg structure theorem and the Szemerédi regularity lemma. One recent application of this dichotomy is the result of Green and Tao establishing that the prime numbers contain arbitrarily long arithmetic progressions (despite having density zero in the integers). The power of this dichotomy is evidenced by the fact that the Green-Tao theorem requires surprisingly little technology from analytic number theory, relying instead almost exclusively on manifestations of this dichotomy such as Szemerédi’s theorem. In this paper we survey various manifestations of this dichotomy in combinatorics, harmonic analysis, ergodic theory, and number theory. As we hope to emphasize here, the underlying themes in these arguments are remarkably similar even though the contexts are radically different. 1.

### Citations

233 | On sets of integers containing no k elements in arithmetic progression
- Szemerédi
- 1975
(Show Context)
Citation Context ..., and number theory. As we hope to emphasize here, the underlying themes in these arguments are remarkably similar even though the contexts are radically different. 1. Introduction In 1975, Szemerédi =-=[53]-=- proved the following deep and enormously influential theorem: Theorem 1.1 (Szemerédi’s theorem). Let A be a subset of the integers Z of positive |A∩[−N,N]| upper density, thus limsupN→∞ |[−N,N]| > 0.... |

228 |
Recurrence in Ergodic Theory and Combinatorial Number Theory
- Furstenberg
- 1981
(Show Context)
Citation Context ...eudorandom error. The reason for the terminology “compact” to describe the limit of structured objects is in analogy to how a compact operator can be viewed as the limit of finite rank operators; see =-=[12]-=- for further discussion. In many applications, the small or pseudorandom errors in these structure theorems are negligible, and one then reduces to the study of structured objects. One then exploits t... |

206 | Szemerédi’s regularity lemma and its applications in graph theory
- Komlós, Simonovits
- 1993
(Show Context)
Citation Context ... Szemerédi regularity lemma, which was developed in [53] in the original proof of Szemerédi’s theorem, and has since proven to have many further applications in graph theory and computer science; see =-=[41]-=- for a survey. More recently, the analogous regularity lemma for hypergraphs have been developed in [21], [46], [47], [48], [49], [58]. Roughly speaking, these very useful lemmas assert that any graph... |

190 |
Harmonic Analysis. Real variable methods, orthogonality and oscillatory integrals
- Stein
- 1993
(Show Context)
Citation Context ...ems here are also widely used in harmonic analysis, in particular obtaining fundamental decompositions such as the Calderón-Zygmund decomposition or the atomic decomposition of Hardy spaces (see e.g. =-=[52]-=-), as well as the tree selection arguments used in multilinear harmonic analysis (see e.g. [43]). It may be worth investigating whether there are any concrete connections between these disparate struc... |

155 | The primes contain arbitrarily long arithmetic progressions
- Green, Tao
- 2004
(Show Context)
Citation Context ...i regularity lemma as examples - are of independent interest (beyond their immediate applications to arithmetic progressions), and have led to many further developments and insights. For instance, in =-=[27]-=- a “weighted” structure theorem (which was in some sense a hybrid of the Furstenberg structure theorem and the Szemerédi regularity lemma) was the primary new ingredient in proving that the primes P :... |

142 |
On certain sets of integers
- Roth
- 1954
(Show Context)
Citation Context ... used ergodic theory and has led to many extensions. A more quantitative proof of Gowers [19], [20] was based on Fourier analysis and arithmetic combinatorics (extending a much older argument of Roth =-=[50]-=- handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Skokan [46], [47], [48], [49] relied on the structural theory of hypergraphs. These proofs are superficially all ... |

141 |
Ergodic behavior of diagonal measures and a theorem of Szemerédi on arithmetic progressions
- Furstenberg
- 1977
(Show Context)
Citation Context ...≥ 3, A contains infinitely many arithmetic progressions of length k. Several proofs of this theorem are now known. The original proof of Szemerédi [53] was combinatorial. A later proof of Furstenberg =-=[11]-=-, [13] used ergodic theory and has led to many extensions. A more quantitative proof of Gowers [19], [20] was based on Fourier analysis and arithmetic combinatorics (extending a much older argument of... |

141 | A new proof of Szemerédi’s Theorem
- Gowers
(Show Context)
Citation Context ...ow known. The original proof of Szemerédi [53] was combinatorial. A later proof of Furstenberg [11], [13] used ergodic theory and has led to many extensions. A more quantitative proof of Gowers [19], =-=[20]-=- was based on Fourier analysis and arithmetic combinatorics (extending a much older argument of Roth [50] handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Skokan [... |

140 | Some problems of `Partitio Numerorum': III On the expression of a number as a sum of primes - Hardy, Littlewood - 1923 |

114 | Quick approximation to matrices and applications
- Frieze, Kannan
- 1999
(Show Context)
Citation Context ... Ex,y∈V f(x, y)1A(x)1B(y) has magnitude at least η4 /4, and the claim follows. □ One can iterate this to obtain a weak version of the Szemerédi regularity lemma: Theorem 4.3 (Weak structure theorem). =-=[10]-=- Let f : V ×V → R be a non-negative function bounded by 1, and let ε > 0. Then we can decompose f = fU ⊥ +fU, where fU ⊥ = E(f|Z ⊗ Z), Z is a σ-algebra of V generated by at most 2/ε sets, and ‖fU‖□2 ≤... |

110 | A new proof of Szemered́i’s theorem for arithmetic progressions of length four. Geometric and Functional Analysis
- Gowers
- 1998
(Show Context)
Citation Context ... are now known. The original proof of Szemerédi [53] was combinatorial. A later proof of Furstenberg [11], [13] used ergodic theory and has led to many extensions. A more quantitative proof of Gowers =-=[19]-=-, [20] was based on Fourier analysis and arithmetic combinatorics (extending a much older argument of Roth [50] handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Sk... |

94 |
Regularity and the multidimensional Szemerédi Theorem
- Gowers
(Show Context)
Citation Context ...nd has since proven to have many further applications in graph theory and computer science; see [41] for a survey. More recently, the analogous regularity lemma for hypergraphs have been developed in =-=[21]-=-, [46], [47], [48], [49], [58]. Roughly speaking, these very useful lemmas assert that any graph (binary relation) or hypergraph (higher order relation), no matter how complex, can be modelled effecti... |

89 | A characterization of the (natural) graph properties testable with one-sided error
- Alon, Shapira
- 2008
(Show Context)
Citation Context ...imilarities between this approach and the previously discussed approaches, we shall not use the standard formulation of this lemma, but instead use a more recent formulation from [57], [58] (see also =-=[1]-=-, [45]), which replaces graphs with functions, and then obtains a structure theorem decomposing such functions into a structured (finite complexity) component, a small component, and a pseudorandom (r... |

87 |
Triple systems with no six points carrying three triangles
- Ruzsa, Szemerédi
- 1978
(Show Context)
Citation Context ...o (10) - without using any further arithmetic structure present in these constraints. Indeed, the claim now follows from the following graph-theoretical statement. Lemma 4.1 (Triangle removal lemma). =-=[51]-=- For every 0 < δ < 1 there exists 0 < σ < 1 with the following property. Let G = (V, E) be an (undirected) graph with |V | = N vertices which contains fewer than σN 3 triangles. Then it is possible to... |

84 | On some sequences of integers
- Erdös, Turán
- 1936
(Show Context)
Citation Context ...where ck := 22k+9. In the k = 3 case it is known that c(3, δ) ≥ δC/δ2 for some absolute constant C, see known lower bounds on c(k, δ), namely c(k, δ) > 2 −21/δc k [5]. A conjecture of Erdős and Turán =-=[8]-=- is roughly equivalent to asserting that c(k, δ) > e −Ck/δ for some Ck. In the converse direction, an example of Behrend shows that c(3, δ) cannot exceed e clog2 (1/δ) for some small absolute constant... |

79 |
Nonconventional ergodic averages and nilmanifolds
- Host, Kra
- 2005
(Show Context)
Citation Context ...the notation of [14], we have just shown that the Kronecker factor is a characteristic factor for the recurrence in (1). (In fact it is essentially the universal factor for this recurrence, see [64], =-=[39]-=- for further discussion.) We have reduced the proof of (1) to the case when f is structured, in the sense of being measurable in Z2. There are two ways to obtain the desired “structured recurrence” re... |

69 | On Sets of Integers which Contain no Three Terms - Behrend - 1946 |

66 | The counting lemma for regular k-uniform hypergraphs, Random Structures and Algorithms 28
- Nagle, Rödl, et al.
- 2006
(Show Context)
Citation Context ...] was based on Fourier analysis and arithmetic combinatorics (extending a much older argument of Roth [50] handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Skokan =-=[46]-=-, [47], [48], [49] relied on the structural theory of hypergraphs. These proofs are superficially all very different (with each having their own strengths and weaknesses), but have a surprising number... |

55 | Lp estimates on the bilinear Hilbert transform for 2 < p
- Lacey, Thiele
(Show Context)
Citation Context ...sitions such as the Calderón-Zygmund decomposition or the atomic decomposition of Hardy spaces (see e.g. [52]), as well as the tree selection arguments used in multilinear harmonic analysis (see e.g. =-=[43]-=-). It may be worth investigating whether there are any concrete connections between these disparate structural theorems.STRUCTURE, RANDOMNESS, AND PROGRESSIONS IN PRIMES 5 2. Ergodic theory We now il... |

53 |
On triples in arithmetic progression
- Bourgain
- 1999
(Show Context)
Citation Context ...guments in Gowers [20] give the best where ck := 22k+9. In the k = 3 case it is known that c(3, δ) ≥ δC/δ2 for some absolute constant C, see known lower bounds on c(k, δ), namely c(k, δ) > 2 −21/δc k =-=[5]-=-. A conjecture of Erdős and Turán [8] is roughly equivalent to asserting that c(k, δ) > e −Ck/δ for some Ck. In the converse direction, an example of Behrend shows that c(3, δ) cannot exceed e clog2 (... |

47 | T.: A variant of the hypergraph removal lemma
- Tao
(Show Context)
Citation Context ...ny further applications in graph theory and computer science; see [41] for a survey. More recently, the analogous regularity lemma for hypergraphs have been developed in [21], [46], [47], [48], [49], =-=[58]-=-. Roughly speaking, these very useful lemmas assert that any graph (binary relation) or hypergraph (higher order relation), no matter how complex, can be modelled effectively as a pseudorandom sub(hyp... |

46 | Universal characteristic factors and Furstenberg averages
- Ziegler
(Show Context)
Citation Context ...1. In the notation of [14], we have just shown that the Kronecker factor is a characteristic factor for the recurrence in (1). (In fact it is essentially the universal factor for this recurrence, see =-=[64]-=-, [39] for further discussion.) We have reduced the proof of (1) to the case when f is structured, in the sense of being measurable in Z2. There are two ways to obtain the desired “structured recurren... |

45 | A Szemer¶edi-type regularity lemma in abelian groups
- Green
- 2005
(Show Context)
Citation Context ...12 TERENCE TAO quantitative version in Theorem 3.1 (but with a moderately bad bound for c(3, δ), namely c(3, δ) = 2−2C/δC for some absolute constant C). A more refined structure theorem was given in =-=[23]-=- (see also [35]), which was termed an “arithmetic regularity lemma” in analogy with the Szemerédi regularity lemma which we discuss in the next section. That theorem has similar hypotheses to Theorem ... |

45 |
On certain sets of positive density
- Varnavides
- 1959
(Show Context)
Citation Context ...although the precise nature of this relationship is still being understood.10 TERENCE TAO It is convenient to work in a cyclic group Z/NZ of prime order. It can be shown via averaging arguments (see =-=[63]-=-) that Szemerédi’s theorem is equivalent to the following quantitative version: Theorem 3.1 (Szemerédi’s theorem, quantitative version). Let N > 1 be a large prime, let k ≥ 3, and let 0 < δ < 1. Let f... |

35 |
Extremal problems on set systems, Random Structure and Algorithms 20(2
- Frankl, Rödl
- 2002
(Show Context)
Citation Context ...f H. Define a tetrahedron in H to be a quadruple (x, y, z, w) of vertices such that all four triplets (x, y, z), (y, z, w), (z, w, x), (w, x, y) are edges of H. Lemma 4.5 (Tetrahedron removal lemma). =-=[9]-=- For every 0 < δ < 1 there exists 0 < σ < 1 with the following property. Let H = (V, E) be a 3-uniform hypergraph graph with |V | = N vertices which contains fewer than σN 4 tetrahedra. Then it is pos... |

33 | Applications of the regularity lemma for uniform hypergraphs, submitted
- Rödl, Skokan
(Show Context)
Citation Context ...rier analysis and arithmetic combinatorics (extending a much older argument of Roth [50] handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Skokan [46], [47], [48], =-=[49]-=- relied on the structural theory of hypergraphs. These proofs are superficially all very different (with each having their own strengths and weaknesses), but have a surprising number of features in co... |

33 | A quantitative ergodic theory proof of Szemerédi’s theorem
- Tao
- 2006
(Show Context)
Citation Context ...Z2, analogous to the “soft” description of the Kronecker factor, is the space of all “quadratically almost periodic functions”. This concept is a bit tricky to define rigorously (see e.g. [13], [12], =-=[54]-=-), but roughly speaking, a function f is linearly almost periodic if the orbit {T n f : n ∈ Z} is precompact in L 2 (X) viewed as a Hilbert space, while a function f is quadratically almost periodic i... |

28 | Higher correlations of divisor sums related to primes. I. Triple correlations
- Goldston, Yıldırım
(Show Context)
Citation Context ...an enveloping sieve ν, but instead of using a Selberg-type sieve that enjoys good Fourier coefficient control, it turns out to be more convenient to use an enveloping sieve 8 of Goldston and Yıldırım =-=[15]-=-, [16], [17] which has good control on k-point correlations (indeed, it behaves pseudorandomly after subtracting off its mean, which is essentially 1). The next step is a generalized von Neumann theor... |

21 |
A Szemerédi type theorem for sets of positive density in
- Bourgain
- 1986
(Show Context)
Citation Context ...h the graph regularity lemmas that we discuss in the next section. Theorem 3.5 can be used to deduce the structure theorems in [23], [56], [35], while a closely related result was also established in =-=[4]-=-. It can also be used to directly derive the k = 3 case of Theorem 3.1, as follows. Let f be as in that proposition, and let ε := δ3 /100. We apply Theorem 3.5 to decompose f = fU ⊥ + fS + fU. Because... |

21 | Roth’s Theorem in the primes
- Green
- 2005
(Show Context)
Citation Context ...elative upper density, thus limsupN→∞ |P ∩[−N,N]| > 0. Then for any k ≥ 3, A contains infinitely many arithmetic progressions of length k. This result was first established in the k = 3 case by Green =-=[22]-=-, the key step again being a (Fourier-analytic) structure theorem, this time for subsets of the primes. The arguments used to prove this theorem do not directly address the important question of wheth... |

21 |
A generalization of a problem of
- Komlos
- 1967
(Show Context)
Citation Context ...ace to describe all the generalizations and refinements of these results here. However, these types of structural theorems appear in other contexts also, for instance the Komlós subsequence principle =-=[40]-=- in probability theory. The Lebesgue decomposition of a spectral measure into pure point, singular continuous, and absolutely continuous spectral components can also be viewed as a structure theorem o... |

21 | The Green-Tao theorem on arithmetic progressions in the primes: an ergodic point of view
- Kra
(Show Context)
Citation Context ...turns out to be sufficient (thanks to Szemerédi’s theorem) for the purpose of establishing the existence of arithmetic progressions. There are now several expositions of Theorem 1.2; see for instance =-=[42]-=-, [25], [55], [56], [37]. Rather than give another exposition of this result, we have chosen to take a broader view, surveying the collection of structural theorems which underlie the proof of such re... |

18 | The Gaussian primes contain arbitrarily shaped constellations
- Tao
- 2006
(Show Context)
Citation Context ...deas from all three approaches, but is closest in spirit to the ergodic approach, albeit set in the finitary context of a cyclic group Z/NZ rather than on an infinitary measure space. The argument in =-=[59]-=-, which shows that the Gaussian primes (or any dense subset thereof) contains infinitely many constellations of any prescribed shape, and can be viewed as a two-dimensional analogue of Theorem 1.2, wa... |

15 | Multiple recurrence and nilsequences - Bergelson, Host, et al. |

14 | Restriction theory of the Selberg sieve, with applications
- Green, Tao
(Show Context)
Citation Context ... of ΛW,b for some 2 < p < 3. This can be done by a more careful application of Vinogradov’s method, but can also be achieved using harmonic analysis methods arising from restriction theory; see [22], =-=[28]-=-. The key new insight here is that while the Fourier coefficients of ΛW,b are difficult to understand directly, one can majorize ΛW,b pointwise by (a constant multiple of) a much better behaved functi... |

14 | Szemerédi’s regularity lemma revisited
- Tao
(Show Context)
Citation Context ...18 TERENCE TAO the similarities between this approach and the previously discussed approaches, we shall not use the standard formulation of this lemma, but instead use a more recent formulation from =-=[57]-=-, [58] (see also [1], [45]), which replaces graphs with functions, and then obtains a structure theorem decomposing such functions into a structured (finite complexity) component, a small component, a... |

12 |
A mean ergodic theorem for (1/N) ∑N n=1 f(T nx)g(T n2x). Convergence in ergodic theory and probability 92
- Weiss
- 1993
(Show Context)
Citation Context ...RIMES 7 f by fU ⊥ if desired; in other words, for the purposes of proving (1) we may assume without loss of generality that f is measurable with respect to the Kronecker factor Z1. In the notation of =-=[14]-=-, we have just shown that the Kronecker factor is a characteristic factor for the recurrence in (1). (In fact it is essentially the universal factor for this recurrence, see [64], [39] for further dis... |

10 | New bounds for Szemeredi’s theorem, I: Progressions of length 4 in finite field geometries
- Green, Tao
(Show Context)
Citation Context ...ed by 1. Setting fU := f − f U ⊥ one obtains the claim. □ It is likely that quantitative versions of this structure theorem will improve the known bounds on Szemerédi’s theorem in the k = 4 case; see =-=[32]-=-, [33], [34]. A closely related version of this argument was also essential in establishing Theorem 1.2, see Section 5 below. 4. Graph theory We now turn to the third major line of attack to Szemerédi... |

10 |
New bounds for Szemerédi’s theorem. II. A new bound for r4(N), Analytic number theory
- Green, Tao
- 2009
(Show Context)
Citation Context ...1. Setting fU := f − f U ⊥ one obtains the claim. □ It is likely that quantitative versions of this structure theorem will improve the known bounds on Szemerédi’s theorem in the k = 4 case; see [32], =-=[33]-=-, [34]. A closely related version of this argument was also essential in establishing Theorem 1.2, see Section 5 below. 4. Graph theory We now turn to the third major line of attack to Szemerédi’s the... |

10 |
Polynomial sequences in groups
- Leibman
- 1998
(Show Context)
Citation Context ...derstanding the structure of arithmetic progressions xΓ, αxΓ, α 2 xΓ, α 3 xΓ on the nilsystem, which can be handled by algebraic arguments, for instance using the machinery of Hall-Petresco sequences =-=[44]-=-. The ergodic methods, while non-elementary and non-quantitative (though see [54]), have proven to be the most powerful and flexible approach to Szemerédi’s theorem, leading to many generalizations an... |

8 | Finite field models in arithmetic combinatorics - Green |

8 |
Regular partitions of hypergraphs, preprint
- Rödl, Schacht
(Show Context)
Citation Context ...based on Fourier analysis and arithmetic combinatorics (extending a much older argument of Roth [50] handling the k = 3 case). A fourth proof by Gowers [21] and Rödl, Nagle, Schacht, and Skokan [46], =-=[47]-=-, [48], [49] relied on the structural theory of hypergraphs. These proofs are superficially all very different (with each having their own strengths and weaknesses), but have a surprising number of fe... |

7 | Small gaps between primes
- Goldston, Yıldırım
(Show Context)
Citation Context ...g sieve ν, but instead of using a Selberg-type sieve that enjoys good Fourier coefficient control, it turns out to be more convenient to use an enveloping sieve 8 of Goldston and Yıldırım [15], [16], =-=[17]-=- which has good control on k-point correlations (indeed, it behaves pseudorandomly after subtracting off its mean, which is essentially 1). The next step is a generalized von Neumann theorem to show t... |

7 |
Convergence of Conze-Lesigne averages. Ergodic Theory Dynam
- Host, Kra
- 2005
(Show Context)
Citation Context ...rgument used to prove van der Waerden’s theorem; see [11], [13], [12], [54]. More recently, a more efficient “hard” factor Z2 was constructed by ConzeLesigne [7], Furstenberg-Weiss [14], and Host-Kra =-=[38]-=-; the analogous factors for higher k are more difficult to construct, but this was achieved by Host-Kra in [39], and also subsequently by Ziegler [64]. This factor yields more precise information, inc... |

6 |
Quadratic uniformity of the Möbius function, preprint
- Green, Tao
(Show Context)
Citation Context ...y many constellations of any prescribed shape, and can be viewed as a two-dimensional analogue of Theorem 1.2, was proven via the (hyper)graph-theoretical approach. Finally, a more recent argument in =-=[30]-=-, [31], in which precise asymptotics for the number of progressions of length four in the primes are obtained, as well as a “quadratic pseudorandomness” estimate on a renormalized counting function fo... |

6 |
Progressions arithmétiques dans les nombres premiers (d’après B. Green et T
- Host
(Show Context)
Citation Context ...nt (thanks to Szemerédi’s theorem) for the purpose of establishing the existence of arithmetic progressions. There are now several expositions of Theorem 1.2; see for instance [42], [25], [55], [56], =-=[37]-=-. Rather than give another exposition of this result, we have chosen to take a broader view, surveying the collection of structural theorems which underlie the proof of such results as Theorem 1.1 and... |

5 | The ergodic-theoretical proof of Szemerédi’s theorem - Furstenberg, Katznelson, et al. - 1982 |

5 |
An inverse theorem for the Gowers U3 norm, preprint
- Green, Tao
(Show Context)
Citation Context ...ve (8) with a reasonable bound on c(4, δ) (basically of the form 1/ exp(exp(δ −C ))); see [19], [20]. Building upon this work, a stronger dichotomy, similar in spirit to Lemma 2.5, was established in =-=[29]-=-. Here, a number of essentially equivalent formulations ofSTRUCTURE, RANDOMNESS, AND PROGRESSIONS IN PRIMES 15 quadratic structure were established, but the easiest to state (and the one which genera... |

5 | Obstructions to uniformity, and arithmetic patterns in the primes, Quart
- Tao
- 2006
(Show Context)
Citation Context ... be sufficient (thanks to Szemerédi’s theorem) for the purpose of establishing the existence of arithmetic progressions. There are now several expositions of Theorem 1.2; see for instance [42], [25], =-=[55]-=-, [56], [37]. Rather than give another exposition of this result, we have chosen to take a broader view, surveying the collection of structural theorems which underlie the proof of such results as The... |

5 |
Additive Combinatorics, book in preparation
- Tao, Vu
(Show Context)
Citation Context ...y quadruples n1, n2, n3, n4 with n1 + n2 = n3 + n4 and ξ(n1) + ξ(n2) = ξ(n3) + ξ(n4)). Methods from additive combinatorics (notably the BalogSzemerédi(-Gowers) theorem and Freiman’s theorem, see e.g. =-=[61]-=-) are then used to “linearize” ξ, in the sense that ξ(n) agrees with a (generalized) linear function of n on a large (generalized) arithmetic progression. One then “integrates” this fact to conclude t... |