### Citations

6241 |
Fuzzy sets
- Zadeh
- 1965
(Show Context)
Citation Context ...overy of the back-propagation algorithm (Rumelhart, Hinton, & Williams, 1994). The competitions also did not receive submissions using some other CI methods such as CART (Breiman, 1984), fuzzy logic (=-=Zadeh, 1965-=-) or evolutionary computation (Fogel, 1994), although these algorithms had already been developed. In 1998, the popular M3-Competition evaluated the accuracies of 24 algorithms on 3003 univariate empi... |

5968 |
Classification and regression trees
- Breiman, Friedman, et al.
- 1984
(Show Context)
Citation Context ...of NNs, through the (re-)discovery of the back-propagation algorithm (Rumelhart, Hinton, & Williams, 1994). The competitions also did not receive submissions using some other CI methods such as CART (=-=Breiman, 1984-=-), fuzzy logic (Zadeh, 1965) or evolutionary computation (Fogel, 1994), although these algorithms had already been developed. In 1998, the popular M3-Competition evaluated the accuracies of 24 algorit... |

5963 |
Neural Networks – A Comprehensive Foundation
- Haykin
- 1999
(Show Context)
Citation Context ...sis WH3) that a significant number of historic observations are a prerequisite for the sufficient initialization, training, validation, evaluation and generalisation of CI approaches (see for example =-=Haykin, 1999-=-). Furthermore, across time series patterns, more CI are ranked highly on seasonal data than on non-seasonal data, a second fundamental contradiction to prior research in the form of working hypothesi... |

3499 | A Decision-theoretic Generalization of On-line Learning and an Application to Boosting. in
- Freund, Schapire
- 1995
(Show Context)
Citation Context ...computer intensive algorithms in CI for forecasting (including new algorithms, e.g. Support Vector Regression, see Smola & Schölkopf, 2004; and methodologies, e.g. method combination by boosting, see =-=Freund & Schapire, 1997-=-). In addition, there has been substantial progress in information technology, which may facilitate the application of existing algorithms and novel extensions to large scale forecasting competitions ... |

865 | A tutorial on support vector regression,”
- Smola, Schölkopf
- 2004
(Show Context)
Citation Context ... 635–660 637 Fildes, 2005; Qi & Zhang, 2001), and the appearance of a range of novel computer intensive algorithms in CI for forecasting (including new algorithms, e.g. Support Vector Regression, see =-=Smola & Schölkopf, 2004-=-; and methodologies, e.g. method combination by boosting, see Freund & Schapire, 1997). In addition, there has been substantial progress in information technology, which may facilitate the application... |

744 | Statistical comparisons of classifiers over multiple data sets. - Demsar - 2006 |

331 |
Time Series Prediction: Forecasting the future and Understanding the Past (Addison-Wesley:
- Weigend, Gershenfeld
- 2006
(Show Context)
Citation Context ...Ns for batch forecasting fell far short of their presumed potential. At the same time, forecasting competitions conducted in computer science and machine learning (e.g., the Santa Fe competition, see =-=Weigend & Gershenfeld, 1994-=-, or the EUNITE competition, see Suykens & Vandewalle, 1998a) attracted a large number of NN and CI algorithms. Although these demonstrated the superior performance of NNs, the algorithms were often n... |

285 | Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,
- Jaeger, Haas
(Show Context)
Citation Context ...ion (see Makridakis et al., 1982): a full decade has passed since the start of the M3 competition, a decade which has seen the development of extended NN paradigms (e.g., recurrent Echo State NN, see =-=Jaeger & Haas, 2004-=-), theoretical advances in methodologies for specifying NNs (see, e.g., Crone & Kourentzes, 2010; Liao &S.F. Crone et al. / International Journal of Forecasting 27 (2011) 635–660 637 Fildes, 2005; Qi... |

245 |
An introduction to simulated evolutionary optimization”,
- Fogel
- 1994
(Show Context)
Citation Context ...umelhart, Hinton, & Williams, 1994). The competitions also did not receive submissions using some other CI methods such as CART (Breiman, 1984), fuzzy logic (Zadeh, 1965) or evolutionary computation (=-=Fogel, 1994-=-), although these algorithms had already been developed. In 1998, the popular M3-Competition evaluated the accuracies of 24 algorithms on 3003 univariate empirical time series of historical data (Makr... |

167 |
The accuracy of extrapolation (time series) methods: results of a forecasting competition,
- Makridakis, Anderson, et al.
- 1982
(Show Context)
Citation Context ...to determine the conditions under which different algorithms perform well. Our motivation for conducting yet another competition follows the same arguments as those of the original M-competition (see =-=Makridakis et al., 1982-=-): a full decade has passed since the start of the M3 competition, a decade which has seen the development of extended NN paradigms (e.g., recurrent Echo State NN, see Jaeger & Haas, 2004), theoretica... |

166 |
The M3-Competition: Results, conclusions and implications.
- Makridakis, Hibon
- 2000
(Show Context)
Citation Context ...e of their ex ante forecasting accuracy in large scale empirical evaluations in the form of forecasting competitions. The most renowned empirical investigation conducted to date — the M3 competition (=-=Makridakis & Hibon, 2000-=-) — indicated a comparatively poor performance from a single NN contestant. Thus, the performances of NNs for batch forecasting fell far short of their presumed potential. At the same time, forecastin... |

158 |
Neural Networks for Short-Term Load Forecasting:A Review and Evaluation” Power Systems,
- Hippert, Pedreira, et al.
- 2001
(Show Context)
Citation Context ...or which — following his arguments — no evidence exists. As the omitted data properties are representative of those on which NNs are regularly employed in practice (e.g., electrical load forecasting, =-=Hippert, Pedreira, & Souza, 2001-=-), this yields a possible explanation to the simultaneous skepticism and euphoria on NNs in forecasting and CI respectively. Hopefully, it will provide the motivation for the gap to be closed byS.F. ... |

128 | Long-Range Forecasting: From Crystal Ball to Computer, - Armstrong - 1985 |

117 | Another Look at Measures of Forecast Accuracy.
- Hyndman, Koehler
- 2006
(Show Context)
Citation Context ...S.F. Crone et al. / International Journal of Forecasting 27 (2011) 635–660 645 This permits us to use Armstrong’s (1985) version of sMAPE(1), as in the M3 competition, for reasons of comparison (see =-=Hyndman & Koehler, 2006-=-, for a more robust version of the sMAPE). 3.4. Methods and benchmarks The competition invited contributions from all areas of machine learning, data mining and CI, including all NN paradigms and arch... |

94 |
Bayesian estimation and prediction using asymmetric loss functions,”
- Zellner
- 1986
(Show Context)
Citation Context ...der to allow those CI methods which are capable of using alternative loss functions (i.e. non-squared costs of errors) to align their approaches with the final criterion (see, e.g., the discussion by =-=Zellner, 1986-=-, following the M3). Despite the shortcomings of the sMAPE (Goodwin & Lawton, 1999), it was chosen both because it served as the primary criterion in the M3 competition and to make the NN3 results acc... |

85 |
Experience with forecasting univariate time series and the combination of forecasts (with discussion),
- Newbold, Granger
- 1974
(Show Context)
Citation Context ... competitions have been conducted that have received a substantial amount of attention. Drawing upon the criticisms of earlier competitions on time series data (Groff, 1973; Makridakis & Hibon, 1979; =-=Newbold & Granger, 1974-=-; Reid, unpublished, 1972), Makridakis et al. conducted a series of enlarged forecasting competitions where experts could submit the predictions of their preferred algorithms: the M-Competition (Makri... |

75 | How effective are neural networks at forecasting and prediction? A review and evaluation. - Adya, Collopy - 1998 |

59 | Feedforward neural nets as models for time series forecasting.
- Tang, Fishwick
- 1993
(Show Context)
Citation Context ... competitive or even superior performance of NNs, from publications on single benchmark time series such as the popular airline passenger dataset (Faraway & Chatfield, 1998; Kolarik & Rudorfer, 1994; =-=Tang & Fishwick, 1993-=-), to representative subsets of established benchmarks from previous forecasting competitions (Foster, Collopy, & Ungar, 1992; Hill, O’Connor, & Remus, 1996; Sharda & Patil, 1992). In one of the few e... |

58 | Time Series Forecasting with Neural Networks: A Comparative Study Using Airline Data.
- Faraway, Chatfield
- 1998
(Show Context)
Citation Context ...to justify: the majority of publications indicate the competitive or even superior performance of NNs, from publications on single benchmark time series such as the popular airline passenger dataset (=-=Faraway & Chatfield, 1998-=-; Kolarik & Rudorfer, 1994; Tang & Fishwick, 1993), to representative subsets of established benchmarks from previous forecasting competitions (Foster, Collopy, & Ungar, 1992; Hill, O’Connor, & Remus,... |

54 |
Out-of-sample Tests of Forecasting Accuracy: An Analysis and Review.
- Tashman
- 2000
(Show Context)
Citation Context ...origin design), the use of multiple robust error metrics, a comparison with established (statistical) benchmark algorithms, and the analysis of the data conditions under which a method performs well (=-=Tashman, 2000-=-), in order to obtain valid and reliable results. Conclusion H6 seems to be particularly relevant, as NNs and other computer intensive methods — just like sophisticated statistical algorithms such as ... |

51 | Load forecasting using support vector machines: A study on EUNITE competition
- Chen, Chang, et al.
- 2001
(Show Context)
Citation Context ...idays (all provided by the Eastern Slovakian Electricity Corporation). Forecasts were made up to 31 days into the future from a single time origin. The best contestant used support vector regression (=-=Chen, Chang, & Lin, 2004-=-) to outperform the CI contestants and one ‘statistical’ contender using regression on decomposed time series components. Although all of the algorithms were published in a monograph (Sincák, Strackel... |

51 | Neural networks models for time series forecasts,” - Hill, Marquez, et al. - 1996 |

50 |
Neural network forecasting for seasonal and trend time series”.
- Zhang, M
- 2005
(Show Context)
Citation Context ...es (see for example de Menezes & Nikolaev, 2006, Preminger & Franck, 2007 and Terasvirta, van Dijk, & Medeiros, 2005) and representative sets of empirical time series (see, e.g., Liao & Fildes, 2005, =-=Zhang & Qi, 2005-=-), where new methodologies for fully automated applications of NN are developed. These have not yet been evaluated in an objective empirical competition. Lastly, the computational power today is far s... |

49 |
Accuracy of Forecasting: An Empirical Investigation.
- Makridakis, Hibon
- 1979
(Show Context)
Citation Context ...ting research, a series of competitions have been conducted that have received a substantial amount of attention. Drawing upon the criticisms of earlier competitions on time series data (Groff, 1973; =-=Makridakis & Hibon, 1979-=-; Newbold & Granger, 1974; Reid, unpublished, 1972), Makridakis et al. conducted a series of enlarged forecasting competitions where experts could submit the predictions of their preferred algorithms:... |

48 |
The evaluation of extrapolative forecasting methods,”
- Fildes
- 1992
(Show Context)
Citation Context ...ns which had been drawn from previous M-competitions (Makridakis et al., 1982, 1993) were confirmed in the M3-competition (see Makridakis & Hibon, 2000), verified through followup studies (see, e.g., =-=Fildes, 1992-=-), and extended to provide additional insights (Fildes et al., 1998): (H1) the characteristics of the data series are an important factor in determining the relative performances of different methods;... |

39 |
Generalising about univariate forecasting methods: further empirical evidence.
- Fildes, Hibon, et al.
- 1998
(Show Context)
Citation Context ...time origin), or a small set of heterogeneous time series. These setups ignored the evidence within the forecasting field as to how to design valid and reliable empirical evaluations (see for example =-=Fildes, Hibon, Makridakis, & Meade, 1998-=-), severely limiting the validity and reliability of their findings. As a consequence of the poor experimental designs, the forecasting community largely ignored these findings. The discrepancy betwee... |

38 | Neural networks: Forecasting breakthrough or just a passing fad - Chatfield - 1993 |

38 | Efficient Market Hypothesis and Forecasting.
- Timmermann, Granger
- 2004
(Show Context)
Citation Context ...in size, structure and heterogeneity, and the exclusion of certain performance metrics that assess the final impact on decision making, e.g., the inventory costs arising from operational forecasting (=-=Timmermann & Granger, 2004-=-). As with prior Mcompetitions, our assessment considered only the empirical accuracy of the algorithms, and neglected robustness, interpretability, and efficiency through the computational resources ... |

35 |
The M-2 Competition: a real-time judgmentally based forecasting study.
- Makridakis, Chatfield, et al.
- 1993
(Show Context)
Citation Context ...ow the participation of algorithms which required time and cost intensive manual tuning by experts (e.g., the ARIMA models required more than one hour per time series). The subsequent M2-competition (=-=Makridakis et al., 1993-=-) focussed on non-automatic, real time judgmental forecasts of 23 time series, and hence is less relevant for our quantitative competition design. None of the earlier competitions attracted any submis... |

34 | The impact of empirical accuracy studies on time series analysis and forecasting. - Fildes, Makridakis - 1995 |

32 | Findings from Evidence-Based Forecasting: Methods for Reducing Forecast Error.”
- Armstrong
- 2006
(Show Context)
Citation Context ...he lack of empirical accuracy in large scale ex ante evaluations, has raised serious concerns in the forecasting domain as to their adequacy for forecasting. As a consequence, Chatfield (as quoted by =-=Armstrong, 2006-=-) suspects a positive bias in NN publications, due to a “file-drawer problem” of negative results, leading Armstrong (2006) to conclude that too much research effort is being devoted to this method. H... |

32 | Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: a re-examination, - Terasvirta, Digk, et al. - 2005 |

31 | Automatic neural network modeling for univariate time series
- Balkin, Ord
- 2000
(Show Context)
Citation Context ... methods (computed by Hibon) in the top performers. Despite the initial interest shown by various CI researchers, only one group ended up submitting results to the competition using a NN methodology (=-=Balkin & Ord, 2000-=-). However, their fully automated methodology AutomatANN performed only moderately well relative to the majority of the twenty statistical approaches, and was not ranked among the top performers (Makr... |

30 |
Connectionist approach to time series prediction: An empirical test.”
- Sharda, Patil
- 1992
(Show Context)
Citation Context ...ik & Rudorfer, 1994; Tang & Fishwick, 1993), to representative subsets of established benchmarks from previous forecasting competitions (Foster, Collopy, & Ungar, 1992; Hill, O’Connor, & Remus, 1996; =-=Sharda & Patil, 1992-=-). In one of the few evaluative reviews, Adya and Collopy (1998) found eleven studies that met the criteria for a valid and reliable empirical evaluation, and NNs were more accurate in 8 of these (73%... |

27 | Significance Tests Harm Progress in Forecasting - Armstrong - 2007 |

26 |
Nonlinear modeling: advanced black box techniques
- Suykens, Vandewalle
- 1998
(Show Context)
Citation Context ...tential. At the same time, forecasting competitions conducted in computer science and machine learning (e.g., the Santa Fe competition, see Weigend & Gershenfeld, 1994, or the EUNITE competition, see =-=Suykens & Vandewalle, 1998-=-a) attracted a large number of NN and CI algorithms. Although these demonstrated the superior performance of NNs, the algorithms were often not evaluated against statistical methods, using only a sing... |

25 |
The Accuracy of Procedural Approach to Specifying Feedforward Neural Networks for Forecasting.
- Fildes, Liao
- 2005
(Show Context)
Citation Context ...both single time series (see for example de Menezes & Nikolaev, 2006, Preminger & Franck, 2007 and Terasvirta, van Dijk, & Medeiros, 2005) and representative sets of empirical time series (see, e.g., =-=Liao & Fildes, 2005-=-, Zhang & Qi, 2005), where new methodologies for fully automated applications of NN are developed. These have not yet been evaluated in an objective empirical competition. Lastly, the computational po... |

24 |
The Theta model: A decomposition approach to forecasting
- Assimakopoulos, Nikolopoulos
- 2000
(Show Context)
Citation Context ...nd the parameterisation of exponential smoothing (ES) and ARIMA models (Goodrich, 2000), and Theta, a decomposition approach combining exponential smoothing and regressing around a damped trend line (=-=Assimakopoulos & Nikolopoulos, 2000-=-). Further statistical analysis by Koning et al. (2005) has provided statistical evidence for a group of four methods with higher accuracies, which also includes rule based forecasting (Adya, Armstron... |

23 | Forecasting Competitions: Their Role in Improving Forecasting Practice and Research. - Fildes, Ord - 2001 |

22 |
Exponential Smoothing model Selection for forecasting, Int J. forecasting‚ 22: 239-247.Central Data base of Iran website. 1936-2005, www.cbi.ir
- Billah, King
- 2006
(Show Context)
Citation Context ...e conditions under which a given algorithm performs well. The competition was open to all NN and CI methods. To reduce potential biases, we also allowed novel statistical methodologies (e.g., that of =-=Billah, King, Snyder, & Koehler, 2006-=-) and newer software releases (e.g., the latest versions of Autobox, ForecastPro or R), which had been developed but had not yet been assessed in competitions, to participate as benchmarks. NN3 attrac... |

22 | Time series forecasting using neural networks : should the data be deseasonalized first - Nelson, Hill, et al. |

21 |
An investigation of model selection criteria for neural network time series forecasting,
- Qai, Zhang
- 2001
(Show Context)
Citation Context ...04), theoretical advances in methodologies for specifying NNs (see, e.g., Crone & Kourentzes, 2010; Liao &S.F. Crone et al. / International Journal of Forecasting 27 (2011) 635–660 637 Fildes, 2005; =-=Qi & Zhang, 2001-=-), and the appearance of a range of novel computer intensive algorithms in CI for forecasting (including new algorithms, e.g. Support Vector Regression, see Smola & Schölkopf, 2004; and methodologies,... |

17 |
An application of rule-based forecasting to a situation lacking domain knowledge,”
- Adya, Armstrong, et al.
- 2000
(Show Context)
Citation Context ...lopoulos, 2000). Further statistical analysis by Koning et al. (2005) has provided statistical evidence for a group of four methods with higher accuracies, which also includes rule based forecasting (=-=Adya, Armstrong, Collopy, & Kennedy, 2000-=-) and Comb S-H-D, an equally weighted combination of the Brown’s single, Holt’s linear trend and Gardner’s damped trend ES methods (computed by Hibon) in the top performers. Despite the initial intere... |

17 |
Neural network forecasting of short, noisy time series.
- Foster, Collopy, et al.
- 1992
(Show Context)
Citation Context ...lar airline passenger dataset (Faraway & Chatfield, 1998; Kolarik & Rudorfer, 1994; Tang & Fishwick, 1993), to representative subsets of established benchmarks from previous forecasting competitions (=-=Foster, Collopy, & Ungar, 1992-=-; Hill, O’Connor, & Remus, 1996; Sharda & Patil, 1992). In one of the few evaluative reviews, Adya and Collopy (1998) found eleven studies that met the criteria for a valid and reliable empirical eval... |

15 | The M3 competition: Statistical tests of the results - Koning, Franses, et al. - 2005 |

13 |
Forecasting Exchange Rates: A Robust Regression Approach,”
- Preminger, Franck
- 2007
(Show Context)
Citation Context ...y no longer reflect the capabilities of today’s NNs. There is evidence of substantial theoretical progress in NNs, in forecasting both single time series (see for example de Menezes & Nikolaev, 2006, =-=Preminger & Franck, 2007-=- and Terasvirta, van Dijk, & Medeiros, 2005) and representative sets of empirical time series (see, e.g., Liao & Fildes, 2005, Zhang & Qi, 2005), where new methodologies for fully automated applicatio... |

11 |
On the asymetry of the symmetric MAPE.
- Goodwin, Lawton
- 1999
(Show Context)
Citation Context ... functions (i.e. non-squared costs of errors) to align their approaches with the final criterion (see, e.g., the discussion by Zellner, 1986, following the M3). Despite the shortcomings of the sMAPE (=-=Goodwin & Lawton, 1999-=-), it was chosen both because it served as the primary criterion in the M3 competition and to make the NN3 results accessible to practitioners, whose predominant error metric is the MAPE. As the NN3 t... |

11 | Time series forecasting using neural networks
- Kolarik, Rudorfer
- 2004
(Show Context)
Citation Context ... publications indicate the competitive or even superior performance of NNs, from publications on single benchmark time series such as the popular airline passenger dataset (Faraway & Chatfield, 1998; =-=Kolarik & Rudorfer, 1994-=-; Tang & Fishwick, 1993), to representative subsets of established benchmarks from previous forecasting competitions (Foster, Collopy, & Ungar, 1992; Hill, O’Connor, & Remus, 1996; Sharda & Patil, 199... |

10 |
The forecasting journals and their contribution to forecasting research: Citation analysis and expert opinion,
- Fildes
- 2006
(Show Context)
Citation Context ...uted the largest number of publications of any area in operational research (Fildes, Nikolopoulos, Crone, & Syntetos, 2008), and they form one of the top four areas of growth in forecasting journals (=-=Fildes, 2006-=-). Their growth in prominence appears to be easy to justify: the majority of publications indicate the competitive or even superior performance of NNs, from publications on single benchmark time serie... |

9 |
Neural networks and seasonality: Some technical considerations
- Curry
- 2007
(Show Context)
Citation Context ...th n, resulting in 50 long (n > 100) and 50 short (n < 50) time series. Second, in order to evaluate recent publications which conclude that NNs cannot forecast seasonal time series (WH4) see, e.g., (=-=Curry, 2007-=-; Nelson, Hill, Remus, & O’Connor, 1999; Zhang & Qi, 2005), stratified samples were taken to reflect the time series patterns of 50 seasonal and 50 non-seasonal time series (as per the original M3 cla... |

9 |
The forecast pro methodology
- Goodrich
(Show Context)
Citation Context ..., two methods generally outperformed all other methods: the software expert system ForecastPro using automatic model selection and the parameterisation of exponential smoothing (ES) and ARIMA models (=-=Goodrich, 2000-=-), and Theta, a decomposition approach combining exponential smoothing and regressing around a damped trend line (Assimakopoulos & Nikolopoulos, 2000). Further statistical analysis by Koning et al. (2... |

8 |
The M-3 competition
- Ord, Hibon, et al.
- 2000
(Show Context)
Citation Context ...c forecasting community, opening up new areas of academic research (e.g. model selection and evaluation) and leading to improved practices on valid and reliable competitions and experimental designs (=-=Ord, Hibon, & Makridakis, 2000-=-). An overview and discussion of the impact of empirical evaluations is given by Fildes and Makridakis (1995) and Fildes and Ord (2002). In contrast, time series prediction competitions which have bee... |

8 | A comparative study of time series prediction techniques on economic data - Reid - 1969 |

7 | The tourism forecasting competition. - Athanasopoulos, Hyndman, et al. - 2011 |

7 |
Winning Entry of the K.U. Leuven TimeSeries Prediction Competition
- McNames, Suykens, et al.
- 1999
(Show Context)
Citation Context ... the KULeuven competition on synthetic data by Suykens and Vandewalle (1998a,b) held at the International Workshop on Advanced Black-Box Techniques for Nonlinear Modeling in 1998 (for the winner, see =-=McNames, Suykens, & Vandewalle, 1999-=-). Table 1 provides a structured summary of prior time series competitions, in both forecasting and CI, and points out differences in experimental designs between the two domains, to assess their cont... |

7 |
The K.U. Leuven time series prediction competition,” in Nonlinear Modeling: Advanced Black-Box Techniques
- Suykens, Vandewalle
- 1998
(Show Context)
Citation Context ...tential. At the same time, forecasting competitions conducted in computer science and machine learning (e.g., the Santa Fe competition, see Weigend & Gershenfeld, 1994, or the EUNITE competition, see =-=Suykens & Vandewalle, 1998-=-a) attracted a large number of NN and CI algorithms. Although these demonstrated the superior performance of NNs, the algorithms were often not evaluated against statistical methods, using only a sing... |

6 | 2007: Predictive uncertainty in environmental modelling
- Janacek, Haylock, et al.
(Show Context)
Citation Context ...alue of domain knowledge over agnostic prediction (Guyon, Saffari, Dror, & Cawley, 2008), or the extent to which the (in-)sample accuracy can be generalised to the out-of-sample performance accuracy (=-=Cawley, Janacek, Haylock, & Dorling, 2007-=-). In contrast, only few competitions in the CI-domain have been dedicated to time series data, as shown in the exhaustive overview in Table 1, although some CI competitions on forecasting may have el... |

6 |
Judging the judges through accuracy-implication metrics: the case of inventory forecasting
- SYNTETOS, NIKOLOPOULOS, et al.
- 2010
(Show Context)
Citation Context ...in operational benefits, and could result in manifold savings in safety stocks. Thus, accuracy results in term of average metrics should never be ignored, as they are often operationally significant (=-=Syntetos, Nikolopoulos, & Boylan, 2010-=-). It should be noted that there are more adequate tests available today for assessing significant differences between the relative performances of algorithms, see, e.g., Demsar (2006); however, they ... |

5 |
Empirical comparison of models for short-range forecasting.
- Groff
- 1973
(Show Context)
Citation Context ...1. In forecasting research, a series of competitions have been conducted that have received a substantial amount of attention. Drawing upon the criticisms of earlier competitions on time series data (=-=Groff, 1973-=-; Makridakis & Hibon, 1979; Newbold & Granger, 1974; Reid, unpublished, 1972), Makridakis et al. conducted a series of enlarged forecasting competitions where experts could submit the predictions of t... |

5 |
Analysis of the ijcnn 2007 agnostic learning vs. prior knowledge challenge. Neural Networks
- Guyon, Saffari, et al.
- 2008
(Show Context)
Citation Context ...m = minutes.? indicates undisclosed information.S.F. Crone et al. / International Journal of Forecasting 27 (2011) 635–660 641 questions, e.g. the value of domain knowledge over agnostic prediction (=-=Guyon, Saffari, Dror, & Cawley, 2008-=-), or the extent to which the (in-)sample accuracy can be generalised to the out-of-sample performance accuracy (Cawley, Janacek, Haylock, & Dorling, 2007). In contrast, only few competitions in the C... |

4 | Statistical significance tests are unnecessary even when properly done and properly interpreted. - Armstrong - 2007 |

3 |
Feature selection for time series prediction–A combined filter and wrapper approach for neural networks.
- Crone, Kourentzes
- 2010
(Show Context)
Citation Context ...tition, a decade which has seen the development of extended NN paradigms (e.g., recurrent Echo State NN, see Jaeger & Haas, 2004), theoretical advances in methodologies for specifying NNs (see, e.g., =-=Crone & Kourentzes, 2010-=-; Liao &S.F. Crone et al. / International Journal of Forecasting 27 (2011) 635–660 637 Fildes, 2005; Qi & Zhang, 2001), and the appearance of a range of novel computer intensive algorithms in CI for ... |

3 | A naive support vector regression benchmark for the NN3 forecasting competition
- Crone, Pietsch
- 2007
(Show Context)
Citation Context ...l to that of the M3 competition. In addition, we computed various CI benchmarks to provide additional levels of comparison for the entries, including a naïve support vector regression (SVR) approach (=-=Crone & Pietsch, 2007-=-, B01) and a naïve multilayer perceptron (MLP) model (B02), both of which replicate novice model building mistakes as a lower bound of errors for CI-methods. A novel NN extension of the successful The... |

3 |
A comparative study of artificial neural network techniques for river stage forecasting
- Dawson, See, et al.
- 2005
(Show Context)
Citation Context ...ime series of sugar and retail sales, organised by Richard Weber at the IEEE Latin-American Summer School on Computational Intelligence (EVIC); the 2001 ANNEXG competition on river stage forecasting (=-=Dawson et al., 2005-=-), held at the 2002 BHS National Hydrology Symposium (the 2005 re-run attracted no competitors); and the KULeuven competition on synthetic data by Suykens and Vandewalle (1998a,b) held at the Internat... |

2 | An extended evaluation framework for neural network publications in sales forecasting," presented at
- Crone, Pressmar
- 2006
(Show Context)
Citation Context ...nal of Forecasting 27 (2011) 635–660 last two decades have witnessed over 5000 publications in academic journals and conference proceedings on forecasting with NNs across a wide range of disciplines (=-=Crone & Preßmar, 2006-=-). In two recent surveys on forecasting publications, Fildes et al. note that while the last 25 years have seen rapid developments in forecasting across a broad range of topics, computer intensive met... |

2 |
Should we be using significance test in forecasting research
- Goodwin
- 2007
(Show Context)
Citation Context ... CI which were not evaluated here. 4.2. Significance of the findings Regardless of the recent and vivid discussion about statistical significance within the forecasting community (Armstrong, 2007a,b; =-=Goodwin, 2007-=-), we computed two non-parametric tests, replicating the analysis of the M3 by Koning et al. (2005): ANOM and MCB, both of which are based upon the average ranks of 41 methods (including both CI ensem... |

2 |
Learning representations by back-propagating errors (from Nature
- Rumelhart, Hinton, et al.
- 1994
(Show Context)
Citation Context ...itions attracted any submissions of NNs or CI methods, as these algorithms did not emerge until the late 1980s; e.g., in the case of NNs, through the (re-)discovery of the back-propagation algorithm (=-=Rumelhart, Hinton, & Williams, 1994-=-). The competitions also did not receive submissions using some other CI methods such as CART (Breiman, 1984), fuzzy logic (Zadeh, 1965) or evolutionary computation (Fogel, 1994), although these algor... |

1 |
international joint conference on neural networks: Vols. 1–5 (pp. 2666–2670). Crone et al
- Menezes, Nikolaev
- 2006
(Show Context)
Citation Context ...eaning that the results may no longer reflect the capabilities of today’s NNs. There is evidence of substantial theoretical progress in NNs, in forecasting both single time series (see for example de =-=Menezes & Nikolaev, 2006-=-, Preminger & Franck, 2007 and Terasvirta, van Dijk, & Medeiros, 2005) and representative sets of empirical time series (see, e.g., Liao & Fildes, 2005, Zhang & Qi, 2005), where new methodologies for ... |

1 |
Book review: “Time series predicition— forecasting the future and understanding the past” by A.S
- Makridakis
- 1994
(Show Context)
Citation Context ...the comparative work undertaken in the competition remains rudimentary and does not provide sufficient evidence to enable us to draw conclusions as to the accuracy of any of the nonlinear algorithms (=-=Makridakis, 1994-=-). The lack of rigor seems particular disappointing, considering that the authors were aware of the design and findings of the M-competitions, and given that the late Clive Granger served on the compe... |

1 | Electricity load forecast using intelligent technologies - Sincák, Strackeljan, et al. - 2002 |

1 | The K.U. Leuven competition data—a challenge for advanced neural network techniques - Suykens, Vandewalle - 2000 |

1 | is an Assistant Professor of Management Science at Lancaster University Management School, and the deputy director of the Lancaster Research Centre for Forecasting. His research focuses on forecasting, time series prediction and data mining in business ap - Crone |