Results 1  10
of
54
Rule Extraction from Recurrent Neural Networks: a Taxonomy and Review
 Neural Computation
, 2005
"... this paper, the progress of this development is reviewed and analysed in detail. In order to structure the survey and to evaluate the techniques, a taxonomy, specifically designed for this purpose, has been developed. Moreover, important open research issues are identified, that, if addressed pr ..."
Abstract

Cited by 34 (5 self)
 Add to MetaCart
this paper, the progress of this development is reviewed and analysed in detail. In order to structure the survey and to evaluate the techniques, a taxonomy, specifically designed for this purpose, has been developed. Moreover, important open research issues are identified, that, if addressed properly, possibly can give the field a significant push forward
M.N.Vrahatis, Financial forecasting through unsupervised clustering and evolutionary trained neural networks
 in: Congress on Evolutionary Computation
, 2003
"... In this paper, we review our work on a time series forecasting methodology based on the combination of unsupervised clustering and artificial neural networks. To address noise and nonstationarity, a common approach is to combine a method for the partitioning of the input space into a number of subs ..."
Abstract

Cited by 17 (8 self)
 Add to MetaCart
(Show Context)
In this paper, we review our work on a time series forecasting methodology based on the combination of unsupervised clustering and artificial neural networks. To address noise and nonstationarity, a common approach is to combine a method for the partitioning of the input space into a number of subspaces with a local approximation scheme for each subspace. Unsupervised clustering algorithms have the desirable property of deciding on the number of partitions required to accurately segment the input space during the clustering process, thus relieving the user from making this ad hoc choice. Artificial neural networks, on the other hand, are powerful computational models that have proved their capabilities on numerous hard realworld problems. The time series that we consider are all daily spot foreign exchange rates of major currencies. The experimental results reported suggest that predictability varies across different regions of the input space, irrespective of clustering algorithm. In all cases, there are regions that are associated with a particularly high forecasting performance. Evaluating the performance of the proposed methodology with respect to its profit generating capability indicates that it compares favorably with that of two other established approaches. Moving from the task of onestepahead to multiplestepahead prediction, performance deteriorates rapidly.
A dynamic neural network method for time series prediction using the KIII model
 Proceedings of the 2003 International Joint Conference on Neural Networks
, 2003
"... Abstract – In this paper, the KIII dynamic neural network is introduced and it is applied to the prediction of complex temporal sequences. In our approach, KIII gives a stepbystep prediction of the direction of the currency exchange rate change. Previously, various multiplayer perceptron (MLP) net ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
Abstract – In this paper, the KIII dynamic neural network is introduced and it is applied to the prediction of complex temporal sequences. In our approach, KIII gives a stepbystep prediction of the direction of the currency exchange rate change. Previously, various multiplayer perceptron (MLP) networks and recurrent neural networks have been successfully implemented for this application. Results obtained by KIII compare favorably with other methods. I.
Temporal Pattern Recognition in Noisy Nonstationary Time Series Based on Quantization into Symbolic Streams: Lessons Learned from Financial Volatility Trading
 URL http://citeseer.nj.nec.com/tino00temporal.html. (URL accessed on March 30
, 2000
"... In this paper we investigate the potential of the analysis of noisy nonstationary time series by quantizing it into streams of discrete symbols and applying finitememory symbolic predictors. The main argument is that careful quantization can reduce the noise in the time series to make model esti ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
In this paper we investigate the potential of the analysis of noisy nonstationary time series by quantizing it into streams of discrete symbols and applying finitememory symbolic predictors. The main argument is that careful quantization can reduce the noise in the time series to make model estimation more amenable given limited numbers of samples that can be drawn due to the nonstationarity in the time series. As a main application area we study the use of such an analysis in a realistic setting involving financial forecasting and trading. In particular, using historical data, we simulate the trading of straddles on the financial indexes DAX and FTSE 100 on a daily basis, based on predictions of the daily volatility differences in the underlying indexes. We propose a parametric, datadriven quantization scheme which transforms temporal patterns in the series of daily volatility changes into grammatical and statistical patterns in the corresponding symbolic streams. As sy...
Financial Volatility Trading using Recurrent Neural Networks
 Online]. Available: citeseer.nj.nec.com/506945.html
, 2001
"... We simulate daily trading of straddles on the financial indexes DAX and FTSE 100. The straddles are traded based on predictions of daily volatility differences in the underlying indexes. The main predictive models studied in this paper are recurrent neural networks (RNNs). In the past, applications ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
We simulate daily trading of straddles on the financial indexes DAX and FTSE 100. The straddles are traded based on predictions of daily volatility differences in the underlying indexes. The main predictive models studied in this paper are recurrent neural networks (RNNs). In the past, applications of RNNs in the financial domain were often studied in isolation. We argue against such a practice by showing that, due to the special character of daily financial timeseries, it is difficult to make full use of RNN representational power. Recurrent networks either tend to overestimate the noisy data, or behave like finitememory sources with a relatively shallow memory. In fact, they can hardly beat (rather simple) classical fixedorder Markov models. To overcome the inherent nonstationarity in the data, we use a special technique that combines "sophisticated" models fitted on a larger data set, with a fixed set of simpleminded symbolic predictors using only recent inputs, thereby avoiding older (and potentially misleading) data. Finally, we compare our predictors with the GARCH family of econometric models designed to capture timedependent volatility structure in financial returns. GARCH models have been used in the past to trade volatility. Experimental results show that while GARCH models are not able to generate any significantly positive profit, by careful use of recurrent networks or Markov models, the market makers can generate a statistically significant excess profit. However, on this type of problems, there is no reason to prefer RNNs over much more simple and straightforward Markov models. We argue that any report containing RNN results on financial tasks should be accompanied by results achieved by simple finitememory sources combined with simple tech...
Learning balls of strings from edit corrections
 Journal of Machine Learning Research
, 2008
"... When facing the question of learning languages in realistic settings, one has to tackle several problems that do not admit simple solutions. On the one hand, languages are usually defined by complex grammatical mechanisms for which the learning results are predominantly negative, as the few algorith ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
When facing the question of learning languages in realistic settings, one has to tackle several problems that do not admit simple solutions. On the one hand, languages are usually defined by complex grammatical mechanisms for which the learning results are predominantly negative, as the few algorithms are not really able to cope with noise. On the other hand, the learning settings themselves rely either on too simple information (text) or on unattainable one (query systems that do not exist in practice, nor can be simulated). We consider simple but sound classes of languages defined via the widely used edit distance: the balls of strings. We propose to learn them with the help of a new sort of queries, called the correction queries: when a string is submitted to the Oracle, either she accepts it if it belongs to the target language, or she proposes a correction, that is, a string of the language close to the query with respect to the edit distance. We show that even if the good balls are not learnable in Angluin’s MAT model, they can be learned from a polynomial number of correction queries. Moreover, experimental evidence simulating a human Expert shows that this algorithm is resistant to approximate answers.
Predictive Filtering: A LearningBased Approach to Data Stream Filtering
 In DMSN
, 2004
"... Recent years have witnessed an increasing interest in filtering of distributed data streams, such as those produced by networked sensors. The focus is to conserve bandwidth and sensor battery power by limiting the number of updates sent from the source while maintaining an acceptable approximation o ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Recent years have witnessed an increasing interest in filtering of distributed data streams, such as those produced by networked sensors. The focus is to conserve bandwidth and sensor battery power by limiting the number of updates sent from the source while maintaining an acceptable approximation of the value at the sink. We propose a novel technique called Predictive Filtering. We use matching predictors at the source and the sink simultaneously to predict the next update. The update is streamed only when the difference between the actual and the predicted value at the source increases beyond a threshold. Different predictors can be plugged into our framework, and we present a comparison of the effectiveness of various predictors. Through experiments performed on a beemotion tracking log we demonstrate the effectiveness of our algorithm in limiting the number of updates while maintaining a good approximation of the streamed data at the sink. 1.
Extracting symbolic knowledge from recurrent neural networksA fuzzy logic approach
 Fuzzy Sets and Systems, Volume 160, Issue
, 2009
"... Considerable research has been devoted to the integration of fuzzy logic (FL) tools with classic artificial intelligence (AI) paradigms. One reason for this is that FL provides powerful mechanisms for handling and processing symbolic information stated using natural language. In this respect, fuzzy ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
Considerable research has been devoted to the integration of fuzzy logic (FL) tools with classic artificial intelligence (AI) paradigms. One reason for this is that FL provides powerful mechanisms for handling and processing symbolic information stated using natural language. In this respect, fuzzy rulebased systems are whiteboxes, as they process information in a form that is easy to understand, verify and, if necessary, refine. The synergy between artificial neural networks (ANNs), which are notorious for their blackbox character, and FL proved to be particularly successful. Such a synergy allows combining the powerful learningfromexamples capability of ANNs with the highlevel symbolic information processing of FL systems. In this paper, we present a new approach for extracting symbolic information from recurrent neural networks (RNNs). The approach is based on the mathematical equivalence between a specific fuzzy rulebase and functions composed of sums of sigmoids. We show that this equivalence can be used to provide a comprehensible explanation of the RNN functioning. We demonstrate the applicability of our approach by using it to extract the knowledge embedded within an RNN trained to recognize a formal language.
Prediction of Financial Time Series with Hidden Markov Models
, 2004
"... In this thesis, we develop an extension of the Hidden Markov Model (HMM) that addresses two of the most important challenges of financial time series modeling: nonstationary and nonlinearity. Specifically, we extend the HMM to include a novel exponentially weighted ExpectationMaximization (EM) al ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
In this thesis, we develop an extension of the Hidden Markov Model (HMM) that addresses two of the most important challenges of financial time series modeling: nonstationary and nonlinearity. Specifically, we extend the HMM to include a novel exponentially weighted ExpectationMaximization (EM) algorithm to handle these two challenges. We show that this extension allows the HMM algorithm to model not only sequence data but also dynamic financial time series. We show the update rules for the HMM parameters can be written in a form of exponential moving averages of the model variables so that we can take the advantage of existing technical analysis techniques. We further propose a double weighted EM algorithm that is able to adjust training sensitivity automatically. Convergence results for the proposed algorithms are proved using techniques from the EM Theorem. Experimental results show that our models consistently beat the S&P 500 Index over five 400day testing periods from 1994 to 2002, including both bull and bear markets. Our models also consistently outperform the top 5 S&P 500 mutual funds in terms of the Sharpe Ratio.
Dynamic Neural Networks, Comparing Spiking Circuits and LSTM
 Utrecht University
, 2003
"... this memory gave rise to fundamental problems during the training phase of siginoid recurrent networks. Popular training algorithms for recurrent neural networks include BackPropagation Through Time (BPTT) and RealTime Recurrent Learning (RTRL) [9,10,12]. During the learning phase, BPTT gradually ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
this memory gave rise to fundamental problems during the training phase of siginoid recurrent networks. Popular training algorithms for recurrent neural networks include BackPropagation Through Time (BPTT) and RealTime Recurrent Learning (RTRL) [9,10,12]. During the learning phase, BPTT gradually enfolds each layer of the network into a multilayer network, in which each layer represents a snapshot of the corresponding time step. The resulting network allows the error to flow in time and is used for learning temporal correlations. The temporal error is provided in a way similar to that of the well known backpropagation algorithm [29]. A major drawback of BPTT is its need to record the whole network state, inputs, target vectors and weights during the training phase, as weight adjustment is done only after the epoch has ended. In contrast, RTRL allows for realtime weight adjustments, at the cost of losing the ability to follow the true gradient, which gives no practical limitations though [9]