## Information-Theoretic Analysis of Neural Coding (1998)

### Cached

### Download Links

- [www-ece.rice.edu]
- [robotics.caltech.edu]
- [www-dsp.rice.edu]
- [www.ece.rice.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | J. Comp. Neuroscience |

Citations: | 59 - 13 self |

### BibTeX

@ARTICLE{Johnson98information-theoreticanalysis,

author = {Don H. Johnson and Charlotte M. Gruner and Keith Baggerly and Chandran Seshagiri and Ran Seshagiri},

title = {Information-Theoretic Analysis of Neural Coding},

journal = {J. Comp. Neuroscience},

year = {1998},

volume = {10},

pages = {47--69}

}

### Years of Citing Articles

### OpenURL

### Abstract

We describe an approach to analyzing single- and multi-unit (ensemble) discharge patterns based on information-theoretic distance measures and on empirical theories derived from work in universal signal processing. In this approach, we quantify the difference between response patterns, be they time-varying or not, using information-theoretic distance measures. We apply these techniques to single and multiple unit processing of sound amplitude and sound location. These examples illustrate that neurons can simultaneously represent at least two kinds of information with different levels of fidelity. The fidelity can persist through a transient and a subsequent steady-state response, indicating that it is possible for an evolving neural code to represent information with constant fidelity. 1 Johnson et al. Analysis of Neural Coding 1 Introduction Neural coding has been classified into two broadly defined types: rate codes the average rate of spike discharge and timing codes the t...

### Citations

9088 |
Elements of information theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...� ��� � � �¢� � � ����� � � density ��� ��� � � �¢� � function: . The � Cramér-Rao bound still holds, but in a more complicated form. � �=-=�� §���§ � � §���§ � � � � � � � � ��� � © ��§ � � � � (5) (6) § (7) This result means that the mean-squared e-=-stimation error for any one parameter must be greater than the corresponding diagonal entry in the inverse of the Fisher � � � � � ��� � � � � � � � © ��� ��§... |

3125 | Tibshirani R. An introduction to the bootstrap
- Efron
- 1993
(Show Context)
Citation Context ... about a statistical estimate. Another method in this family is the so-called jackknife method, and it has been used for removal of bias in entropy calculations [10]. The book by Efron and Tibshirani =-=[9] provides excellent descriptions of the bootst-=-rap procedure and its theoretical properties. In a general setting, let ¨ ��� ��©�� ����� ������� denote a dataset from which we estimate the quantity ���... |

1040 |
A probabilistic Theory of Pattern Recognition
- Devroye, Gyorfi, et al.
- 1996
(Show Context)
Citation Context ...ood” classification performance to result. How large “large” is will depend on the problem and theoretical results from empirical classification theory hint that no general rule of thumb can be found =-=[10]-=-. What we have found is that moderate data and training set sizes suffice to yield exponentially decreasing error probabilities (roughly 100–1000) [19, 20]. Furthermore, Gutman showed that if the trai... |

701 |
An Introduction to Signal Detection and Estimation
- Poor
- 1994
(Show Context)
Citation Context ... optimize the other. They are judged by the error probability not constrained. For both kinds of classification rules, comparing the likelihood ratio to a threshold is the optimal classification rule =-=[27]-=-. The choice of rule defines the threshold. In Neyman-Pearson classifiers, the threshold depends on how much data are available; in optimal classifiers, the threshold is fixed. One would expect that o... |

537 |
Visual feature integration and the temporal correlation hypothesis, Annu Rev Neurosci 18
- Singer, Gray
- 1995
(Show Context)
Citation Context ... random to effectively represent sound [19]. Coordinated responses of neurons within sensory and motor nuclei have been found, with discharge timing relations among neural outputs having significance =-=[1, 3, 8, 22, 24, 27, 29]-=-. Consequently, much current work has focused on population activity, using the fundamental assumption that coordinated sequences of action potential occurrence times produced by groups of neurons col... |

426 |
Neural circuits of the cerebral cortex
- Abeles
- 1991
(Show Context)
Citation Context ... random to effectively represent sound [19]. Coordinated responses of neurons within sensory and motor nuclei have been found, with discharge timing relations among neural outputs having significance =-=[1, 3, 8, 22, 24, 27, 29]-=-. Consequently, much current work has focused on population activity, using the fundamental assumption that coordinated sequences of action potential occurrence times produced by groups of neurons col... |

192 |
Introduction to Mathematical Statistics
- Hogg, Craig
- 1978
(Show Context)
Citation Context ... which of two categories , best match the observations. Given the probabilistic descriptions of the categories, the optimal rule for classifying observations, the likelihood ratio test, is well known =-=[14]-=-. Interestingly, what is very difficult to calculate is how well this optimal rule works. Developing approximations for calculating performance leads to important notions that directly apply to the ne... |

137 |
A Spike synchronization and rate modulation differentially involved in motor cortical function. Science 278:1950--1953
- Riehle, Gruen, et al.
- 1997
(Show Context)
Citation Context ... random to effectively represent sound [19]. Coordinated responses of neurons within sensory and motor nuclei have been found, with discharge timing relations among neural outputs having significance =-=[1, 3, 8, 22, 24, 27, 29]-=-. Consequently, much current work has focused on population activity, using the fundamental assumption that coordinated sequences of action potential occurrence times produced by groups of neurons col... |

135 |
Reading a neural code
- Bialek, Rieke, et al.
- 1991
(Show Context)
Citation Context ...ulus-response relationship. Thus, today the “neural code” is taken to mean how groups of neurons, responding individually and collectively, represent sensory information with their discharge patte=-=rns [5]-=-. Knowing the code would unlock the secrets of how neurons, working in concert, process and represent information. In developing data analysis techniques that seek to elucidate the neural code, some k... |

133 |
Trofimov, “The Performance of Universal Encoding
- Krichevsky, K
- 1981
(Show Context)
Citation Context ...ro, we obtain an infinite answer, which may not be accurate (the true reference distribution has a nonzero probability for the offending letter). To alleviate this problem, the so-called K-T estimate =-=[20] is employed. Each type is modified by addin-=-g one half to the histogram estimate before it is normalized to yield a type. Thus, for the � ��� letter, the K-T estimate at bin � is ����� ��� ����� ��� �... |

112 |
Dynamics of neuronal interactions in monkey cortex in relation to behavioural events
- Vaadia, Haalman, et al.
- 1995
(Show Context)
Citation Context |

107 |
Distance measures for signal processing and pattern recognition
- Basseville
- 1989
(Show Context)
Citation Context ...ich systems can extract information no matter what response component(s) they may use. 2 Information-theoretic distance measures While the merits of one distance measure versus another can be debated =-=[4]-=-, we describe here a collection of information-theoretic distances that have a clear, intuitive mathematical foundation. The underlying theory is not rooted in the classic results of Shannon, but in m... |

88 |
Primary cortical representation of sounds by the coordination of action-potential timing, Nature 381
- deCharms, Merzenich
- 1996
(Show Context)
Citation Context |

80 |
Odour encoding by temporal sequences of firing in oscillating neural assemblies
- Wehr, Laurent
- 1996
(Show Context)
Citation Context |

76 |
Sequential Detection of Abrupt Changes
- Nikiforov
- 1983
(Show Context)
Citation Context ...dence interval for the second response is indicated by the dotted lines. Note that the confidence interval’s lower edge was less than zero for the first 30 bins. the classic change-detection solution =-=[4]-=-. Here, the traditional approach is to consider this problem as a multihypothesis hypothesis testing problem, with each hypothesis indicating the index at which the change occurs. However, this approa... |

72 |
A universal finite memory source
- Weinberger, Rissanen, et al.
- 1995
(Show Context)
Citation Context ... � � © � ����� ��� � � � ���sJohnson et al. Analysis of Neural Coding The demands placed on data collection are greatly reduced when we employ this approximation. Re=-=cent results in information theory [30] prescribe how much data are needed by any bin-ba-=-sed technique to analyze data to a given degree of dependence: ����� �¢� � ����� ��� ��� � � � � � where is the amount of averaging used in analyzing t... |

62 |
Precisely correlated firing in cells of the lateral geniculate nucleus
- Alonso, Usrey, et al.
- 1996
(Show Context)
Citation Context |

60 |
Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents
- Rieke, DA, et al.
- 1995
(Show Context)
Citation Context ... Purpura (1997) measures the distance between two responses by determining the number of steps it takes to systematically transform one discharge pattern into another. Mutual information calculations =-=[11, 23]-=- measure how effectively a spike train encodes the stimulus under Poisson assumptions. Neural network models have been trained on recorded neural responses to determine how well these responses can be... |

54 | Metric-space analysis of spike trains: theory, algorithms and application - Victor, Purpura - 1997 |

51 | Point Process Model of Single-Neuron Discharges
- Johnson
- 1996
(Show Context)
Citation Context ...tely explain information coding. For example, theoretical considerations indicate that single-neuron discharge patterns in the mammalian auditory pathway are too random to effectively represent sound =-=[19]-=-. Coordinated responses of neurons within sensory and motor nuclei have been found, with discharge timing relations among neural outputs having significance [1, 3, 8, 22, 24, 27, 29]. Consequently, mu... |

50 |
Asymptotically Optimal Classification for Multiple Tests with Empirically Observed Statistics
- Gutman
- 1989
(Show Context)
Citation Context ...et is . What we seek is the best possible empirical classifier: Which classifier produces the smallest possible classification error probabilities given only the availability of training data? Gutman =-=[17]-=- found a classifier that is not only optimal in a certain sense, but will also, given enough training and observational data, produce error probabilities having the same exponential rate as the likeli... |

45 |
Multispike train analysis
- Abeles, Goldstein
- 1977
(Show Context)
Citation Context ...ire stationary responses: The probability law governing the response must not change with time. Among these are several interspike interval statistics [19], and auto- and cross-correlation techniques =-=[2]-=-. To quantify time-varying responses, responses are averaged over several stimulus presentations that are spaced sufficiently far apart in time to prevent adaptation and sequential stimulus effects. S... |

29 |
A panoramic code for sound location by cortical neurons
- Middlebrooks, Clock, et al.
- 1994
(Show Context)
Citation Context ...encodes the stimulus under Poisson assumptions. Neural network models have been trained on recorded neural responses to determine how well these responses can be used to determine stimulus parameters =-=[21]-=-. These methods yield response metrics that are difficult to relate to how well systems can extract information from single- or multiple-neuron responses. When inputs to a neural population have been ... |

26 |
Coding of time-varying signals in spike trains of integrate-and-fire neurons with random threshold
- Gabbiani, Koch
- 1996
(Show Context)
Citation Context ... Purpura (1997) measures the distance between two responses by determining the number of steps it takes to systematically transform one discharge pattern into another. Mutual information calculations =-=[11, 23]-=- measure how effectively a spike train encodes the stimulus under Poisson assumptions. Neural network models have been trained on recorded neural responses to determine how well these responses can be... |

21 |
Reading a neural code
- Rieke, Steveninck, et al.
(Show Context)
Citation Context ...ulus-response relationship. Thus, today the “neural code” is taken to mean how groups of neurons, responding individually and collectively, represent sensory information with their discharge patte=-=rns [5]-=-. Knowing the code would unlock the secrets of how neurons, working in concert, process and represent information. In developing data analysis techniques that seek to elucidate the neural code, some k... |

13 |
An introduction to cyclostationary signals
- gardner
- 1994
(Show Context)
Citation Context ...es are averaged over several stimulus presentations that are spaced sufficiently far apart in time to prevent adaptation and sequential stimulus effects. Such responses are said to be cyclostationary =-=[12]-=-: The probability law varies with time but does so periodically, the period equaling the inter-stimulus interval. The well-known PST histogram measures, under a Poisson point process assumption, how t... |

9 |
On the bias of information estimates
- Carlton
- 1969
(Show Context)
Citation Context ...cally unbiased, experience shows that the bias is significant even for large datasets, and can lead to analysis difficulties. Analytic expressions for the bias of a related quantity entropy are known =-=[6]-=-, and they indicate that bias expressions will depend on the underlying distribution in complicated ways. Fortunately, recent work in statistics provides a way of estimating the bias and removing it f... |

9 |
The transmission of signals by auditory-nerve fiber discharge patterns
- Johnson, Swami
- 1983
(Show Context)
Citation Context ...subtracting � �¢��� � � � from one. ��� Just as in the usual PST histogram, this multiunit PST histogram does not faithfully represent temporal dependence that may be present in=-= the ensemble response [17]-=-. The multiunit PST histogram essentially assumes responses occur independently from bin to bin what amounts to a Poisson assumption because no record is kept of what preceded a particular population ... |

9 |
A geometry for detection theory
- Dabak
- 1992
(Show Context)
Citation Context ... classification problem show that no distance metric exists for it, and that the Kullback-Leibler distance is the distance-like quantity that should be used to assess how different two categories are =-=[8]-=-. The Gaussian example also indicates that both the Kullback-Leibler and Chernoff distances have the form of a squared-distance: When we have several statistically independent components, these distan... |

6 |
Single-neuron modeling of LSO unit responses
- Zacksenhouse, Johnson, et al.
- 1998
(Show Context)
Citation Context ...us. For example, we developed a point process model for the tone-burst responses of single units located in the lateral superior olive (LSO) [31, 32] and an underlying computational biophysical model =-=[33]. Wh-=-ile these models provide a notion of the response’s structure, they do not help us determine the typical LSO unit’s information processing role and what processing function the variety of LSO resp... |

4 |
The application of a point process model to the single unit responses of the cat lateral superior olive to ipsilaterally presented tones
- Johnson, Tsuchitani, et al.
- 1986
(Show Context)
Citation Context ...t revealed by the PST histogram, be it a single- or multi-unit histogram. Temporal dependence in discharge probabilities can arise in a variety of ways: among them are dependence on discharge history =-=[18, 31, 32], non-expo-=-nential interval distributions 5 , and syn-fire response patterns in ensembles [1, 22]. The Markov dependence � corresponds to a temporal analysis window ��� s long. The bound on � determine... |

4 |
The inhibition of cat lateral superior olivary unit excitatory responses to binaural tone bursts: II. The sustained discharges
- Tsuchitani
- 1988
(Show Context)
Citation Context ...0 20 e h 20 4 14 30 46 0 20 30 40 1 0.5 c 0 0 100 10 36 0 40 20 f i 20 4 14 30 46 0 20 30 40 0 10 start time (ms) 20 Figure 8: We simulated [33] the response of a single lateral superior olive neuron =-=[25, 26] to -=-high-frequency tone bursts presented at various amplitudes and azimuthal locations. The top row (panels a-c) shows the PST histogram of the simulated response at the reference condition (20 dB, ��... |

3 |
Performance of optimal non-Gaussian detectors
- Johnson, Orsak
- 1993
(Show Context)
Citation Context ...initions apply to both univariate and multivariate distributions. When the observations are not statistically independent, all these results apply to the multivariate distribution of the observations =-=[16]. For example, �©� ���¦� � fa � �©��� � � � �©� ���¦� � � � � � � � (A3) ����� ��� �����-=-�� ����� ������� � � � ��� � ����� ����� ��� � � � � � � � for fixed � � miss Stein’s Lemma (A1) is not stated dir... |

2 |
Information measures: Statistical confidence limits and inference
- Fagan
- 1978
(Show Context)
Citation Context ...ariance, bias, and confidence intervals about a statistical estimate. Another method in this family is the so-called jackknife method, and it has been used for removal of bias in entropy calculations =-=[10]. The book by Efron and Tibshirani [9] p-=-rovides excellent descriptions of the bootstrap procedure and its theoretical properties. In a general setting, let ¨ ��� ��©�� ����� ������� denote a dataset f... |

2 |
Time-varying, multiplexed feature coding in the lateral superior olive. Nature
- Gruner, Johnson
- 1998
(Show Context)
Citation Context ... to the Fisher information matrix. In our approach, we measure responses recorded in response to a reference stimulus parameterized by and a family of responses parameterized by , with a perturbation =-=[16]-=-. We compute types from ensemble responses to both stimuli and quantify the “distance” between them. We use the KullbackLeibler distance in this application since it yields larger values for a given p... |

2 |
Type-based detection for unknown channels
- Johnson, Lee, et al.
- 1996
(Show Context)
Citation Context ... hint that no general rule of thumb can be found [10]. What we have found is that moderate data and training set sizes suffice to yield exponentially decreasing error probabilities (roughly 100–1000) =-=[19, 20]-=-. Furthermore, Gutman showed that if the training set size increases more rapidly than the dataset size that the exponential rate of the empirical rule equaled that of the optimal clairvoyant classifi... |

1 |
Correlation and neural information coding efficiency
- Gruner, Johnson
- 1998
(Show Context)
Citation Context ...����� denotes the joint distribution of � the random variables. We note that from an information transfer viewpoint, statistically independent responses do not always correspond to the bes=-=t situation [13]-=-. 6.3 Uncovering feature extraction As part of developing these new techniques, we re-examined how the signal processing function of any system should be assessed. Consider a nonlinear, adaptive syste... |

1 |
Excitatory/inhibitory interaction in the LSO revealed by point process modeling
- Zacksenhouse, Johnson, et al.
- 1992
(Show Context)
Citation Context ...n, and how effective the code is in representing the stimulus. For example, we developed a point process model for the tone-burst responses of single units located in the lateral superior olive (LSO) =-=[31, 32] and-=- an underlying computational biophysical model [33]. While these models provide a notion of the response’s structure, they do not help us determine the typical LSO unit’s information processing ro... |

1 |
Excitation effects on LSO unit sustained responses: Point process characterization
- Zacksenhouse, Johnson, et al.
- 1993
(Show Context)
Citation Context ...n, and how effective the code is in representing the stimulus. For example, we developed a point process model for the tone-burst responses of single units located in the lateral superior olive (LSO) =-=[31, 32] and-=- an underlying computational biophysical model [33]. While these models provide a notion of the response’s structure, they do not help us determine the typical LSO unit’s information processing ro... |

1 |
et al. Empirical classification based on types
- Johnson
- 1997
(Show Context)
Citation Context ... hint that no general rule of thumb can be found [10]. What we have found is that moderate data and training set sizes suffice to yield exponentially decreasing error probabilities (roughly 100–1000) =-=[19, 20]-=-. Furthermore, Gutman showed that if the training set size increases more rapidly than the dataset size that the exponential rate of the empirical rule equaled that of the optimal clairvoyant classifi... |