## Testing For Nonlinearity Using Redundancies: Quantitative and Qualitative Aspects (1995)

Venue: | Physica D |

Citations: | 19 - 7 self |

### BibTeX

@ARTICLE{Palus95testingfor,

author = {Milan Palus},

title = {Testing For Nonlinearity Using Redundancies: Quantitative and Qualitative Aspects},

journal = {Physica D},

year = {1995},

volume = {80},

pages = {186--205}

}

### Years of Citing Articles

### OpenURL

### Abstract

A method for testing nonlinearity in time series is described based on information-theoretic functionals -- redundancies, linear and nonlinear forms of which allow either qualitative, or, after incorporating the surrogate data technique, quantitative evaluation of dynamical properties of scrutinized data. An interplay of quantitative and qualitative testing on both the linear and nonlinear levels is analyzed and robustness of this combined approach against spurious nonlinearity detection is demonstrated. Evaluation of redundancies and redundancy-based statistics as functions of time lag and embedding dimension can further enhance insight into dynamics of a system under study. Keywords: time series, nonlinearity, mutual information, redundancy, surrogate data 1 Introduction The problem of inferring the dynamics of a system from measured data is a perpetual challenge for time series analysts. Ideas and concepts from nonlinear dynamics and theory of deterministic chaos have led to a num...

### Citations

8565 |
Elements of information theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...the series to be nonlinear. 3 Mutual information and redundancies In this section we will define basic functionals from information theory, more details can be found in any book on information theory =-=[4, 15, 24, 25, 45, 6]-=-. Let X, Y be random variables with probability distribution densities pX (x) and p Y (y). The entropy of the distribution of a single variable, say X, is defined as: H(X) := \Gamma Z pX (x) log(p X (... |

1483 |
Information Theory and Reliable Communications
- Gallager
- 1968
(Show Context)
Citation Context ...the series to be nonlinear. 3 Mutual information and redundancies In this section we will define basic functionals from information theory, more details can be found in any book on information theory =-=[4, 15, 24, 25, 45, 6]-=-. Let X, Y be random variables with probability distribution densities pX (x) and p Y (y). The entropy of the distribution of a single variable, say X, is defined as: H(X) := \Gamma Z pX (x) log(p X (... |

1154 | Information theory and statistics
- Kullback
(Show Context)
Citation Context ...the series to be nonlinear. 3 Mutual information and redundancies In this section we will define basic functionals from information theory, more details can be found in any book on information theory =-=[4, 15, 24, 25, 45, 6]-=-. Let X, Y be random variables with probability distribution densities pX (x) and p Y (y). The entropy of the distribution of a single variable, say X, is defined as: H(X) := \Gamma Z pX (x) log(p X (... |

828 |
Deterministic non-periodic flow
- Lorenz
(Show Context)
Citation Context ...ta", see Sec. 5). Figures 2a and 2b present the linear redundancy L n (�� ) and the redundancy R n (�� ), respectively, computed from the time series generated by the Lorenz system in a c=-=haotic state [26]. L n (�� ) -=-decreases quickly to values close to zero and detects no dependence for �� ? 10 samples, while R n (�� ) detects nonlinear dependence, the level of which oscillates with �� and is characte... |

612 |
The jackknife, the bootstrap, and other resampling plans
- Efron
- 1982
(Show Context)
Citation Context ...ribe the underlying nonlinear dynamics, but simply to find arguments that a linear model is inadequate. This approach -- testing data against a null hypothesis -- was advocated by a number of authors =-=[11, 8, 9, 10, 54]-=-. Therefore we need to define a linear stochastic processes. Let fy(t)g be a time-series realization of a stationary stochastic process fY (t)g. Without loss of generality we can set its mean to zero.... |

507 |
Spectral Analysis and Time Series
- Priestley
- 1981
(Show Context)
Citation Context ...where b(0) = 1, P 1 i=1 ja(i)j ! 1, P 1 i=0 jb(i)j ! 1, fN(t)g is an independent, identically distributed (iid), normally distributed process with zero mean and finite variance. (For more details see =-=[39]-=-.) The requirement for fN(t)g to be an iid process is an important condition, since any stationary process with zero mean posses a Wold decomposition [2] of the form (1), when the process fN(t)g is un... |

255 |
Numerical recipes: The art of scientific computing”, Cambridge University Press, NY. 84 Sriram Venkatesan Born September 28
- Press, Flannery, et al.
- 1989
(Show Context)
Citation Context ...ent of Energy (ER-FG05-88ER25054). Appendix A The linear redundancies were computed according to eq. (9). Eigenvalues of each correlation matrix were obtained by using the SVDCMP routine described in =-=[38]-=-, p.52. The algorithm for computing the redundancy, proposed by Fraser [13, 14] or Fraser and Swinney [12], is rather complicated one. We have found a simple box-counting method is sufficient. The onl... |

252 |
Measuring the strangeness of strange attractors
- Grassberger, Procaccia
- 1983
(Show Context)
Citation Context ...concepts from nonlinear dynamics and theory of deterministic chaos have led to a number of algorithms, able, in principle, to identify and quantify underlying nonlinear deterministic/chaotic dynamics =-=[1, 16, 17, 27]-=-. After extensive use, however, many of these algorithms were found chronically unreliable, often producing spurious dimension or Lyapunov exponent estimates [28, 41, 47, 51, 7] and thus supporting fa... |

245 |
The Statistical Analysis of Time Series
- Anderson
- 1971
(Show Context)
Citation Context ...an and finite variance. (For more details see [39].) The requirement for fN(t)g to be an iid process is an important condition, since any stationary process with zero mean posses a Wold decomposition =-=[2] of the form (1), when the process -=-fN(t)g is uncorrelated. The principal difference lies in the fact, that while "independent" means also "uncorrelated ", "uncorrelated" means only "linearly independe... |

245 |
Ergodic theory and information
- Billingsley
- 1965
(Show Context)
Citation Context |

223 |
Independent coordinates for strange attractors from mutual information
- Faser, Swinney
- 1986
(Show Context)
Citation Context ...Eigenvalues of each correlation matrix were obtained by using the SVDCMP routine described in [38], p.52. The algorithm for computing the redundancy, proposed by Fraser [13, 14] or Fraser and Swinney =-=[12], is rathe-=-r complicated one. We have found a simple box-counting method is sufficient. The only "special prescriptions", based on our extensive numerical experience, concern the way of data quantizati... |

195 |
Characterization of strange attractors
- Grassberger, Procaccia
- 1983
(Show Context)
Citation Context ...concepts from nonlinear dynamics and theory of deterministic chaos have led to a number of algorithms, able, in principle, to identify and quantify underlying nonlinear deterministic/chaotic dynamics =-=[1, 16, 17, 27]-=-. After extensive use, however, many of these algorithms were found chronically unreliable, often producing spurious dimension or Lyapunov exponent estimates [28, 41, 47, 51, 7] and thus supporting fa... |

186 |
Testing for nonlinearity in time series: the method of surrogate data. Physica D: Nonlinear Phenomena
- Theiler, Eubank, et al.
- 1992
(Show Context)
Citation Context ...s in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism =-=[53, 35, 21]-=-. As it was pointed out by Theiler et al. [53], detection of nonlinearity is considerably easier goal than that of positive identification of chaotic dynamics. On the other hand, detection of nonlinea... |

147 |
Mathematical Foundations of Information Theory
- KHINCHIN
- 1957
(Show Context)
Citation Context |

129 |
An Introduction to the Bootstrap, Chapman
- Efron, Tibshirani
- 1993
(Show Context)
Citation Context ...ribe the underlying nonlinear dynamics, but simply to find arguments that a linear model is inadequate. This approach -- testing data against a null hypothesis -- was advocated by a number of authors =-=[11, 8, 9, 10, 54]-=-. Therefore we need to define a linear stochastic processes. Let fy(t)g be a time-series realization of a stationary stochastic process fY (t)g. Without loss of generality we can set its mean to zero.... |

128 |
Simultaneous Statistical Inference
- Miller
- 1981
(Show Context)
Citation Context ...l inference, which we will discuss in connection with a particular practical application of the redundancy-based nonlinearity testing (Sec. 8, [37]), rather than here. For more information see, e.g., =-=[33, 29, 31, 19]-=-.) 3 In some cases it is not true, due to numerical reasons, see Sec. 6. The usefulness of plotting a statistic versus time lag to illustrate differences between test and surrogate data was also demon... |

112 |
Detecting strange attractors in turbulence, in: Dynamical Systems and Turbulence
- Takens
- 1981
(Show Context)
Citation Context ...t)g which is stationary and ergodic. We will study the redundancies for the variables X i (t) = y(t + (i \Gamma 1)�� ); i = 1; : : : ; n; (10) where �� is a time delay and n is an embedding di=-=mension [50]. Redundancies o-=-f the type R(y(t); y(t + �� ); : : : ; y(t + (n \Gamma 1)�� )) are, due to stationarity of fY (t)g, independent of t. We introduce the notation: R n (�� ) := R(y(t); y(t + �� ); : : : ... |

85 |
The Mathematical Theory of Communication (University of Illinois
- Shannon, Weaver
- 1949
(Show Context)
Citation Context |

85 |
Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series
- SUGIHARA, MAY
- 1990
(Show Context)
Citation Context ... is generated with frequency which is not constant, but fluctuating. Measles The number of measles cases reported each month in New York City in the years 1928 -- 1963 was studied by Sugihara and May =-=[49]-=- and more recently by Kaplan and Glass [22]. We have analyzed last 256 samples using the same numerical conditions as in the case of sunspots, but for lags 1 -- 60 months. The results are presented in... |

72 |
Spurious dimensions from correlation algorithms applied to limited time series data, Phys
- Theiler
- 1986
(Show Context)
Citation Context ...rministic/chaotic dynamics [1, 16, 17, 27]. After extensive use, however, many of these algorithms were found chronically unreliable, often producing spurious dimension or Lyapunov exponent estimates =-=[28, 41, 47, 51, 7]-=- and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary cond... |

69 |
Predicting Sunspots and Exchange Rates with Connectionist Networks
- Weigend, Huberman, et al.
- 1992
(Show Context)
Citation Context ...s of static nonlinearities and thus to detect only dynamical nonlinearity. Sunspots The annual Wolf sunspot numbers have been recorded since 1700. The time series is described and displayed, e.g., in =-=[57]-=-. We have analyzed the sunspot numbers for the years 1724 -- 1979, giving 256 observations. Using such short time series we restricted the test for embedding dimension n = 2 only, and lags 1 -- 32 yea... |

46 |
Spectral Analysis and Its Applications. Holden-Day
- Jenkins, Watts
- 1968
(Show Context)
Citation Context ...absolute values of these differences are very small, of the order of 10 \Gamma3 in redundancies and 10 \Gamma4 in related autocorrelation functions (ACF) for both the unbiased and circular estimators =-=[20, 55].) The sit-=-uation is different, when we analyze only 16,384 (16K) samples from the above data, i.e., now we construct 16K-surrogates from a (16K) "test data", which is, in fact, a 16K segment of a surr... |

46 |
Finite Correlation Dimension for Stochastic Systems with Power-Law Spectra
- Osborne, Provenzale
- 1989
(Show Context)
Citation Context ...able, often producing spurious dimension or Lyapunov exponent estimates [28, 41, 47, 51, 7] and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation =-=[34, 40, 52, 51]-=-. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism [53, 35, 21]. As it was pointed out by Theiler et al. [53], detec... |

42 |
Computers and the theory of statistics: thinking the unthinkable
- Efron
- 1979
(Show Context)
Citation Context ...ribe the underlying nonlinear dynamics, but simply to find arguments that a linear model is inadequate. This approach -- testing data against a null hypothesis -- was advocated by a number of authors =-=[11, 8, 9, 10, 54]-=-. Therefore we need to define a linear stochastic processes. Let fy(t)g be a time-series realization of a stationary stochastic process fY (t)g. Without loss of generality we can set its mean to zero.... |

42 | Constrained-realization Monte-Carlo method for hypothesis testing - Theiler, Prichard - 1996 |

37 | Strange attractors, chaotic behavior, and information - Shaw - 1981 |

28 |
Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems
- Eckmann, Ruelle
- 1992
(Show Context)
Citation Context ...rministic/chaotic dynamics [1, 16, 17, 27]. After extensive use, however, many of these algorithms were found chronically unreliable, often producing spurious dimension or Lyapunov exponent estimates =-=[28, 41, 47, 51, 7]-=- and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary cond... |

28 |
M.(1989), Information and entropy in strange attractors
- Fraser
(Show Context)
Citation Context ...6. R n (�� ). This difference we again consider as important or qualitative. (In fact, this long-term decreasing trend is a specific nonlinear property, actually a signature of chaos, as discussed=-= in [35, 36, 13].) In summary, i-=-n the qualitative comparison of L n (�� ) with R n (�� ) (or, equivalently, of R n (�� ) computed from the scrutinized data and R n (�� ) from its surrogates, defined in the following ... |

26 |
Strange attractors that are not chaotic
- Grebogi, Ott, et al.
- 1984
(Show Context)
Citation Context ...of R 2 (�� )) as it is in the case of the trivial nonlinearity. In Fig. 10 we present results of the analysis of another example of "nontrivial" nonlinearity. The system, introduced by G=-=rebogi et al. [18], is an ex-=-ample of the evolution on a "strange nonchaotic attractor", i.e., the attractor is a fractal set, but the (3-dimensional) system has one zero and two negative Lyapunov exponents. (The distin... |

21 | Using surrogate data to detect nonlinearity in time series. A proceedings volume - Theiler, Galdrikian, et al. - 1992 |

21 | Detecting Nonlinearity in Data with Long Coherence Times
- Theiler, Linsay, et al.
- 1994
(Show Context)
Citation Context ...dancies as the functions of �� are the same). The surrogate data have the same spectrum as the original one, that means they should have also the same autocorrelation function 3 (for more details =-=see [55]). Recalling the-=- relation between the autocorrelation function and the linear redundancy, we can see that: L n (�� )[s] = L n (�� )[o]. This means that R n (�� )[o] = L n (�� )[o] if and only if R n (... |

19 | Reconstructing attractors from scalar time series
- Fraser
- 1989
(Show Context)
Citation Context ...ements (variances) and oe i are the eigenvalues of the n \Theta n covariance matrix C. (See, e.g., the work of Morgera [32]; two-dimensional case, i.e., mutual information, was derived also by Fraser =-=[14]-=-.) Formula (7), obviously, may be associated with any positive definite covariance matrix. Thus, we use formula (7) to define the linear redundancy L(X 1 ; : : : ; X n ) of an arbitrary n-dimensional ... |

17 |
Direct test for determinism in a time series
- Kaplan, Glass
- 1992
(Show Context)
Citation Context ...s in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism =-=[53, 35, 21]-=-. As it was pointed out by Theiler et al. [53], detection of nonlinearity is considerably easier goal than that of positive identification of chaotic dynamics. On the other hand, detection of nonlinea... |

17 |
Method to distinguish possible chaos from colored noise and to determine embedding parameters
- Kennel, Isabelle
- 1992
(Show Context)
Citation Context ...nearities, or non-Gaussian distributions of the data under study, on the nonlinearity tests and thus to detect only true dynamical nonlinearity. (This problem is also discussed by Kennel and Isabelle =-=[23], Theiler et al. [53-=-] call an equivalent procedure "amplitude adjustment", Rapp et al. [42] use the term "Gaussian-scaled surrogates".) An interesting question emerged when we consider the above AR pr... |

17 |
Chaotic Vibrations: An Introduction for Applied Scientists and Engineers
- Moon
- 1987
(Show Context)
Citation Context ...ction of trivial nonlinearity, however, does not mean that the system under study is linear. In Fig. 8 we present results of analysis of the series generated by the Rossler system in a periodic state =-=[30]-=- (again recorded with a small amount of the additive Gaussian noise). The results, again obtained from the series 128K samples long, are the same as those for the noisy sine series: no significant dif... |

12 | 1988] Deterministic Chaos: An Introduction (Physik-Verlag - Schuster |

11 |
Phase-randomized surrogates can produce spurious identifications of non-random structure, Phys. Lett. A 192
- Rapp, Albano, et al.
- 1994
(Show Context)
Citation Context ... and thus to detect only true dynamical nonlinearity. (This problem is also discussed by Kennel and Isabelle [23], Theiler et al. [53] call an equivalent procedure "amplitude adjustment", Ra=-=pp et al. [42] use the t-=-erm "Gaussian-scaled surrogates".) An interesting question emerged when we consider the above AR process con13 structed with a non-Gaussian driving noise. As was mentioned in Sec. 4 even wit... |

10 |
Tests of the overall hypothesis for arbitrary dependence structures
- HOMMEL
- 1983
(Show Context)
Citation Context ...l inference, which we will discuss in connection with a particular practical application of the redundancy-based nonlinearity testing (Sec. 8, [37]), rather than here. For more information see, e.g., =-=[33, 29, 31, 19]-=-.) 3 In some cases it is not true, due to numerical reasons, see Sec. 6. The usefulness of plotting a statistic versus time lag to illustrate differences between test and surrogate data was also demon... |

9 |
Identifying and quantifying chaos by using information-theoretic functionals, in
- Paluš
- 1993
(Show Context)
Citation Context ...R n (�� ) detect long term trends, but the trends are different (e.g., slower, close to linear decrease in R n (�� ) and faster, exponential or power-law decrease in L n (�� ) -- see, e.g.=-=, Fig. 9 in [36]). 5 Quant-=-itative testing: surrogate data The redundancy-based method, introduced in [35] and reviewed in Sec. 4, was called "the test for nonlinearity". A reader with background in mathematical stati... |

8 |
Intrinsic limits on dimension calculations, Phys
- Smith
- 1988
(Show Context)
Citation Context ...rministic/chaotic dynamics [1, 16, 17, 27]. After extensive use, however, many of these algorithms were found chronically unreliable, often producing spurious dimension or Lyapunov exponent estimates =-=[28, 41, 47, 51, 7]-=- and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary cond... |

6 |
Coarse-grained embeddings of time series: random walks, Gaussian random process, and deterministic chaos
- Kaplan, Glass
- 1993
(Show Context)
Citation Context ... true, due to numerical reasons, see Sec. 6. The usefulness of plotting a statistic versus time lag to illustrate differences between test and surrogate data was also demonstrated by Kaplan and Glass =-=[22]-=-. 6 Spurious nonlinearity - pitfalls in the surrogate data Theiler et al. [55] recently have observed that generating a surrogate data from a time series with a long coherence time, i.e., from strongl... |

6 |
Some comments on the correlation dimension of 1=f ff noise
- Theiler
- 1991
(Show Context)
Citation Context ...able, often producing spurious dimension or Lyapunov exponent estimates [28, 41, 47, 51, 7] and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation =-=[34, 40, 52, 51]-=-. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism [53, 35, 21]. As it was pointed out by Theiler et al. [53], detec... |

5 |
Information-theoretic test for nonlinearity in time series, Phys. Lett. A 175
- Palus, Albrecht, et al.
- 1993
(Show Context)
Citation Context ...s in data, consistent with a simpler explanation [34, 40, 52, 51]. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism =-=[53, 35, 21]-=-. As it was pointed out by Theiler et al. [53], detection of nonlinearity is considerably easier goal than that of positive identification of chaotic dynamics. On the other hand, detection of nonlinea... |

5 |
Analysis of nonlinear time series (and chaos) by bispectral methods
- Rao
- 1992
(Show Context)
Citation Context ...ivial nonlinearity 8 than to a specific nonlinear behavior like evolution on strange attractors. Therefore we can agree with Suba Rao's conclusion that the sunspot data is nonlinear, but not chaotic. =-=[48]-=- 7 We did not use this adjustment in the tests depicted in Figs. 3c and 3d, since the ratio k=m was close to one. In other tests the test values, considered as significant, were large enough to fulfil... |

4 |
Detecting low-dimensional chaos in population dynamics data: a critical review
- Ellner
- 1991
(Show Context)
Citation Context |

4 |
Application of dimension algorithms to experimental chaos
- Mayer-Kress
- 1988
(Show Context)
Citation Context |

4 |
Data Analysis and Regression: A Second Course
- Mosteller, JW
- 1977
(Show Context)
Citation Context ...l inference, which we will discuss in connection with a particular practical application of the redundancy-based nonlinearity testing (Sec. 8, [37]), rather than here. For more information see, e.g., =-=[33, 29, 31, 19]-=-.) 3 In some cases it is not true, due to numerical reasons, see Sec. 6. The usefulness of plotting a statistic versus time lag to illustrate differences between test and surrogate data was also demon... |

3 |
Convergence of the K2 entropy for random noises with power law spectra. Phys. D Nonlinear Phenom
- Provenzale, Osborne, et al.
- 1991
(Show Context)
Citation Context ...able, often producing spurious dimension or Lyapunov exponent estimates [28, 41, 47, 51, 7] and thus supporting false identification of chaotic dynamics in data, consistent with a simpler explanation =-=[34, 40, 52, 51]-=-. Some authors, considering these problems, proposed to test necessary conditions for chaos like nonlinearity or nonlinear determinism [53, 35, 21]. As it was pointed out by Theiler et al. [53], detec... |

2 |
Information theoretic complexity and relation to pattern recognition
- Morgera
- 1985
(Show Context)
Citation Context ... i=1 log(c ii ) \Gamma 1 2 n X i=1 log(oe i ); (7) where c ii are the diagonal elements (variances) and oe i are the eigenvalues of the n \Theta n covariance matrix C. (See, e.g., the work of Morgera =-=[32]-=-; two-dimensional case, i.e., mutual information, was derived also by Fraser [14].) Formula (7), obviously, may be associated with any positive definite covariance matrix. Thus, we use formula (7) to ... |

1 |
Berechnung des maximalen Signifikanzniveaus des Testes: "Lehne H 0 ab, wenn k unter n gegebenen Tests zur Ablehnung fuhren", Metrika 27
- Morgerstern
- 1980
(Show Context)
Citation Context |