Currently surrogate data analysis can be used to determine if data is consistent with various linear systems, or something else (a nonlinear system). In this paper we propose an extension of these methods in an attempt to make more specific classifications within the the class of nonlinear systems. In the method of surrogate data one estimates the probability distribution of values of a test statistic for a set of experimental data under the assumption that the data is consistent with a given hypothesis. If the probability distribution of the test statistic is different for different dynamical systems consistent with the hypothesis one must ensure that the surrogate generation technique generates surrogate data that are a good approximation to the data. This is often achieved with a careful choice of surrogate generation method and for noise driven linear surrogates such methods are commonly used. This paper argues that, in many cases (particularly for nonlinear hypotheses), it is ea...

We review a relatively new statistical test that may be applied to determine whether an observed time series is inconsistent with a specific class of dynamical systems. These surrogate data methods may test an observed time series against the hypotheses of: i) independent and identically distributed noise; ii) linearly filtered noise; and iii) a monotonic nonlinear transformation of linearly filtered noise. A recently suggested fourth algorithm for testing the hypothesis of a periodic orbit with uncorrelated noise is also described. We propose several novel applications of these methods for various engineering problems, including: identifying a deterministic (message) signal in a noisy time series; and separating deterministic and stochastic components. When employed to separate deterministic and noise components, we show that the application of surrogate methods to the residuals of nonlinear models is equivalent to fitting that model subject to an information theoretic model selection criteria.

Copyright c○2003 by the authors. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher bepress. Determinism in Financial Time Series The attractive possibility that financial indices may be chaotic has been the subject of much study. In this paper we address two specific questions: “Masked by stochasticity, do financial data exhibit deterministic nonlinearity?”, and “If so, so what?”. We examine daily returns from three financial indicators: the Dow Jones Industrial Average, the London gold fixings, and the USD-JPY exchange rate. For each data set we apply surrogate data methods and nonlinearity tests to quantify determinism over a wide range of time scales (from 100 to 20,000 days). We find that all three time series are distinct from linear noise or conditional heteroskedastic models and that there therefore exists detectable deterministic nonlinearity that can potentially be exploited for prediction.

The technique of surrogate data provides has been used to test for membership of particular classes of linear systems. Existing algorithms provide non-parametric methods to generate surrogates similar to the data and consistent with a given hypothesis. These non-parametric methods allow a wide range of test statistics to be utilized. We suggest an obvious extension of this to classes of nonlinear parametric models. To do so it is necessary to restrict the statistics employed to a relatively broad class. We demonstrate that correlation dimension provides a suitable statistic and apply these methods, together with existing surrogate tests to respiratory data from sleeping infants. Although our data are clearly distinct from the different classes of linear systems we are unable to distinguish between our data and surrogates generated by nonlinear models. Hence we conclude that our data cannot be explained by linearly filtered noise but is consistent with the noisy periodic orbit of a nonl...

The technique of surrogate data analysis may be employed to test the hypothesis that an observed data set was generated by one of several specific classes of dynamical system. Current algorithms for surrogate data analysis enable one, in a generic way, to test for membership of the following three classes of dynamical system: (0) independent and identically distributed noise, (1) linearly filtered noise, and (2) a monotonic nonlinear transformation of linearly filtered noise. We show that one may apply statistics from nonlinear dynamical systems theory, in particular those derived from the correlation integral, as test statistics for the hypothesis that an observed time series is consistent with each of these three linear classes of dynamical system. Using statistics based on the correlation integral we show that it is also possible to test much broader (and not necessarily linear) hypotheses. We illustrate these methods with radial basis models and an algorithm to estimate t...