Results 1 
9 of
9
System Identification, Approximation and Complexity
 International Journal of General Systems
, 1977
"... This paper is concerned with establishing broadlybased systemtheoretic foundations and practical techniques for the problem of system identification that are rigorous, intuitively clear and conceptually powerful. A general formulation is first given in which two order relations are postulated on a ..."
Abstract

Cited by 34 (23 self)
 Add to MetaCart
This paper is concerned with establishing broadlybased systemtheoretic foundations and practical techniques for the problem of system identification that are rigorous, intuitively clear and conceptually powerful. A general formulation is first given in which two order relations are postulated on a class of models: a constant one of complexity; and a variable one of approximation induced by an observed behaviour. An admissible model is such that any less complex model is a worse approximation. The general problem of identification is that of finding the admissible subspace of models induced by a given behaviour. It is proved under very general assumptions that, if deterministic models are required then nearly all behaviours require models of nearly maximum complexity. A general theory of approximation between models and behaviour is then developed based on subjective probability concepts and semantic information theory The role of structural constraints such as causality, locality, finite memory, etc., are then discussed as rules of the game. These concepts and results are applied to the specific problem or stochastic automaton, or grammar, inference. Computational results are given to demonstrate that the theory is complete and fully operational. Finally the formulation of identification proposed in this paper is analysed in terms of Klir’s epistemological hierarchy and both are discussed in terms of the rich philosophical literature on the acquisition of knowledge. 1
Discriminative Training of Hidden Markov Models
, 1998
"... vi Abbreviations vii Notation viii 1 Introduction 1 2 Hidden Markov Models 4 2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 HMM Modelling Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 HMM Topology . . . . . . . . . ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
vi Abbreviations vii Notation viii 1 Introduction 1 2 Hidden Markov Models 4 2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 HMM Modelling Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 HMM Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Finding the Best Transcription . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.5 Setting the Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3 Objective Functions 19 3.1 Properties of Maximum Likelihood Estimators . . . . . . . . . . . . . . . . . . . 19 3.2 Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3 Maximum Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4 Frame Discrimination . . . . . . . . . . . . . . . . ....
Twostage dynamic signal detection: A theory of choice, decision time, and confidence
 In
, 2010
"... The 3 most oftenused performance measures in the cognitive and decision sciences are choice, response or decision time, and confidence. We develop a random walk/diffusion theory—2stage dynamic signal detection (2DSD) theory—that accounts for all 3 measures using a common underlying process. The mo ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
The 3 most oftenused performance measures in the cognitive and decision sciences are choice, response or decision time, and confidence. We develop a random walk/diffusion theory—2stage dynamic signal detection (2DSD) theory—that accounts for all 3 measures using a common underlying process. The model uses a drift diffusion process to account for choice and decision time. To estimate confidence, we assume that evidence continues to accumulate after the choice. Judges then interrupt the process to categorize the accumulated evidence into a confidence rating. The model explains all known interrelationships between the 3 indices of performance. Furthermore, the model also accounts for the distributions of each variable in both a perceptual and general knowledge task. The dynamic nature of the model also reveals the moderating effects of time pressure on the accuracy of choice and confidence. Finally, the model specifies the optimal solution for giving the fastest choice and confidence rating for a given level of choice and confidence accuracy. Judges are found to act in a manner consistent with the optimal solution when making confidence judgments.
Abstract: Stated Beliefs Versus Empirical Beliefs: A Methodological Inquiry and Experimental Test by
, 2006
"... If asking subjects their beliefs during repeated game play changes the way those subjects play, using those stated beliefs to evaluate and compare theories of strategic behavior is problematic. We experimentally verify that belief elicitation can alter paths of play in a repeated asymmetric matching ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
If asking subjects their beliefs during repeated game play changes the way those subjects play, using those stated beliefs to evaluate and compare theories of strategic behavior is problematic. We experimentally verify that belief elicitation can alter paths of play in a repeated asymmetric matching pennies game. In this setting, belief elicitation improves the goodness of fit of structural models of belief learning, and the prior beliefs implied by such structural models are both stronger and more realistic when beliefs are elicited than when they are not. These effects are, however, confined to the player type who sees a strong asymmetry between payoff possibilities for her two strategies in the game. We conjecture that this occurs because both automatic, unconscious evaluative processes and more effortful conscious deliberative processes can conflict, and that the latter are enhanced by belief elicitation procedures. We also find that “empirical beliefs ” (beliefs estimated from past observed actions of opponents) can be better predictors of observed actions than the “stated beliefs ” resulting from belief elicitation.
Scoring The Forecaster By Mean Resulting Payoff Of A Distribution Of Decision Problems
 Maximum Entropy and Bayesian Methods. Proceedings of the Thirteenth International Workshop
, 1994
"... . We present a new way to understand and characterize the choice of scoring, objective, or loss function for a model, machine, or expert estimating (i.e. predicting) the probabilities of events or outcomes (such as class membership). The ultimate value of a probability estimate lies in the actual pa ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
. We present a new way to understand and characterize the choice of scoring, objective, or loss function for a model, machine, or expert estimating (i.e. predicting) the probabilities of events or outcomes (such as class membership). The ultimate value of a probability estimate lies in the actual payoff (utility) accruing to those who use this information to make a decision. We allow that we often cannot specify with certainty that the estimate will be used in a particular decision problem, characterized by a particular decisionoutcome payoff matrix (cost schedule), and thus by a particular decision threshold. Instead, we consider the more general case of a distribution over such matrices. The proposed scoring function is the expectation, with respect to this distribution, of the payoff that will actually be received. Squareerror loss (or Brier score) and loglikelihood (or various entropic measures) arise from specific examples of such distributions, and even common singlethreshold...
Objective Bayesianism and the Maximum Entropy Principle
, 2013
"... For Maximum Entropy and Bayes Theorem, a special issue of Entropy journal. Objective Bayesian epistemology invokes three norms: the strengths of our beliefs should be probabilities, they should be calibrated to our evidence of physical probabilities, and they should otherwise equivocate sufficiently ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
For Maximum Entropy and Bayes Theorem, a special issue of Entropy journal. Objective Bayesian epistemology invokes three norms: the strengths of our beliefs should be probabilities, they should be calibrated to our evidence of physical probabilities, and they should otherwise equivocate sufficiently between the basic propositions that we can express. The three norms are sometimes explicated by appealing to the maximum entropy principle, which says that a belief function should be a probability function, from all those that are calibrated to evidence, that has maximum entropy. However, the three norms of objective Bayesianism are usually justified in different ways. In this paper we show that the three norms can all be subsumed under a single justification in terms of minimising worstcase expected loss. This, in turn, is equivalent to maximising a generalised notion of entropy. We suggest that requiring language invariance, in addition to minimising worstcase expected loss, motivates maximisation of standard entropy as opposed to maximisation of other instances of generalised entropy. Our argument also provides a qualified justification for updating degrees of belief by Bayesian conditionalisation. However, conditional probabilities play a less central part in the objective Bayesian account than they do under the subjective view of Bayesianism, leading to a reduced role for Bayes ’ Theorem. Contents
Howgood Were Those Probability Predictions? The Expected Recommendation Loss (erl) Scoring Rule
"... . We present a new way to understand and characterize the choice of scoring rule (probability loss function) for evaluating the performance of a supplier of probabilistic predictions after the outcomes (true classes) are known. The ultimate value of a prediction (estimate) lies in the actual utility ..."
Abstract
 Add to MetaCart
. We present a new way to understand and characterize the choice of scoring rule (probability loss function) for evaluating the performance of a supplier of probabilistic predictions after the outcomes (true classes) are known. The ultimate value of a prediction (estimate) lies in the actual utility (loss reduction) accruing to one who uses this information to make some decision(s). Often we cannot specify with certainty that the prediction will be used in a particular decision problem, characterized by a particular loss matrix (indexed by outcome and decision), and thus having a particular decision threshold. Instead, we consider the more general case of a distribution over such matrices. The proposed scoring rule is the expectation, with respect to this distribution, of the loss that is actually incurred when following the decision recommendation, the latter being the decision that would be considered optimal if we were to assume the predicted probabilities. Logarithmic and quadrati...
Local Proper Scoring Rules
, 2009
"... Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order λ if the score depends on the predictive de ..."
Abstract
 Add to MetaCart
Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order λ if the score depends on the predictive density only through its value and its derivatives of order up to λ at the observation. Previously, only a single local proper scoring rule had been known, namely the logarithmic score, which is local of order λ = 0. Here we introduce the Fisher score, which is a local proper scoring rule of order λ = 2. It relates to the Fisher information in the same way that the logarithmic score relates to the KullbackLeibler information. The convex cone generated by the logarithmic score and the Fisher score exhausts the class of the local proper scoring rules of order λ ≤ 2, up to equivalence and regularity conditions. In a data example, we use local and nonlocal proper scoring rules to assess statistically postprocessed ensemble weather forecasts. Finally, we develop a multivariate version of the Fisher score. 1
Learning and Belief Elicitation: Observer Effects by
, 2004
"... This paper asks a simple methodological question about belief elicitation: Does asking subjects about their beliefs during a repeated game change the way subjects play the game? In particular, we ask if belief elicitation changes behavior in the direction of belief based learning models, away from r ..."
Abstract
 Add to MetaCart
This paper asks a simple methodological question about belief elicitation: Does asking subjects about their beliefs during a repeated game change the way subjects play the game? In particular, we ask if belief elicitation changes behavior in the direction of belief based learning models, away from reinforcement learning models. Based on a carefully designed laboratory experiment, we show that play is indeed affected by eliciting players beliefs. More importantly, however, the effect is not simply a parametric shift in favor of one or another theory. What we instead observe is that subjects behave as if they construct mental models of their opponents that are both more sophisticated and robust in the presence of belief elicitation. Finally we show that game theoretically motivated learning models fit the behavioral data much better when the action data is generated jointly with belief elicitation than when it is generated without. This supports the conjecture that eliciting beliefs make subjects behave in a more game theoretic manner. Our findings have serious implications both for the choice of econometric models when testing game theoretic and learning models, and for the ability to test these models based on stated, rather than empirical beliefs