## Information Theory and Communication Networks: An Unconsummated Union (1998)

Venue: | IEEE Trans. Inform. Theory |

Citations: | 139 - 5 self |

### BibTeX

@ARTICLE{Ephremides98informationtheory,

author = {Anthony Ephremides and Bruce Hajek},

title = {Information Theory and Communication Networks: An Unconsummated Union},

journal = {IEEE Trans. Inform. Theory},

year = {1998},

volume = {44},

pages = {2416--2434}

}

### Years of Citing Articles

### OpenURL

### Abstract

Information theory has not yet had a direct impact on networking, although there are similarities in concepts and methodologies that have consistently attracted the attention of researchers from both fields. In this paper, we review several topics that are related to communication networks and that have an information theoretic flavor, including multiaccess protocols, timing channels, effective bandwidth of bursty data sources, deterministic constraints on datastreams, queueing theory, and switching networks. Keywords--- Communication networks, multiaccess, effective bandwidth, switching I. INTRODUCTION Information theory is the conscience of the theory of communication; it has defined the "playing field" within which communication systems can be studied and understood. It has provided the spawning grounds for the fields of coding, compression, encryption, detection, and modulation and it has enabled the design and evaluation of systems whose performance is pushing the limits of wha...

### Citations

1109 | Multiuser detection
- Verdu
- 1998
(Show Context)
Citation Context ...es to ignore the intrinsic role of delay and burstiness. Nonetheless, it has spawned the rapid development of the field of multiuser detection (see [5] in this issue and, for a more thorough account, =-=[6]-=-). In a sense, both multiuser information theory and multiuser detection theory represent major forays of information theory toward the field of networks that, so far, have revealed insights but have ... |

606 |
A calculus for network delay, Part I: Network elements in isolation
- Cruz
- 1991
(Show Context)
Citation Context ...rmance guarantees. This may entail, for example, providing end-to-end delay guarantees by bounding the delay for each device or link transitted. A popular datastream constraint, introduced by Cruz in =-=[52]-=-, [53], is the (oe; ae) constraint, defined as follows. Consider a datastream described by a function (R(t) : ts0), where R(t) denotes the amount of data generated up to time t. Assume that R(0) = 0, ... |

600 | Queuing Systems
- Kleinrock
- 1975
(Show Context)
Citation Context ...ho formulated the backbone elements of packet-switched networking was Kleinrock who, first, in his original work [7] that was based on his Ph.D. dissertation and, subsequently, in his two-volume book =-=[8]-=- on queueing systems, popularized many of the innovative intricacies and challenges of communication networks. 1 A substantial volume of other work in the late sixties and early seventies [9]-[15], mo... |

418 |
The ALOHA system | another alternative for computer communications
- Abramson
- 1970
(Show Context)
Citation Context ...sion of time-division multiplexing. However, it is believed that to achieve throughput near one for very large N , the mean delay must also be large. The first consideration of this model by Abramson =-=[23]-=- made the additional natural simplification that the number of sources N is infinite. Such an assumption, unnatural though it may appear at first, is a clever and useful one in that, first of all, it ... |

368 |
A distributed algorithm for minimum-weight spanning trees
- Gallager, Spira
- 1983
(Show Context)
Citation Context ... quite natural for an information theorist to ask, is "How many messages must be passed to accomplish a task." This is known as the communication complexity. For example, the paper by Gallag=-=er et al. [31]-=- gives an efficient distributed algorithm for finding a minimum weight spanning tree given weights on the edges connecting nodes. There has been much more in the brief history of networking that can b... |

331 |
A calculus for network delay, part ii: Network analysis
- Cruz
- 1991
(Show Context)
Citation Context ... guarantees. This may entail, for example, providing end-to-end delay guarantees by bounding the delay for each device or link transitted. A popular datastream constraint, introduced by Cruz in [52], =-=[53]-=-, is the (oe; ae) constraint, defined as follows. Consider a datastream described by a function (R(t) : ts0), where R(t) denotes the amount of data generated up to time t. Assume that R(0) = 0, and th... |

297 |
Maximal Flow Through a Network
- Ford, Fulkerson
- 1956
(Show Context)
Citation Context ...o accept that it does, then, again, Shannon must be credited with the prophesy of almost all aspects of the field of communication. It is also interesting to note that in [18] (as well as in [19] and =-=[20]-=-), a first look at max-flow min-cut relationships is provided; thus, the notion of flow approximations, widely used in network studies, was again, first noted by Shannon (among others). This notion wa... |

192 | Queue Length, and Delay of Deterministic and Stochastic Queuing Networks
- Chang, Stability
- 1994
(Show Context)
Citation Context ...fluence the approximation. See [49] and [43] for extensive surveys on effective bandwidth, and [40] for a very extensive bibliographic guide to self-similar datastream models and their use. The paper =-=[50]-=- presents significant bounds and analysis related to notions of equivalent bandwidth with a different terminology. Finally, the paper [51] connects the theory of effective bandwidths to thermodynamics... |

182 |
A minimum delay routing algorithm using distributed computation
- Gallager
- 1977
(Show Context)
Citation Context ...ge delay. For clear implementation reasons (to reduce state-information latency and overhead and to ensure improved survivability and robustness) dynamic and distributed solutions were preferable. In =-=[24]-=-, Gallager presented a concise and direct formulation of the problem accompanied by an elegant solution that permitted each node, based on simple periodic information exchanges with its neighbors, to ... |

181 |
Resource Allocation for Broadband Networks“, in
- Hui
- 1988
(Show Context)
Citation Context ...s on the context) somewhere between the mean and peak rate of the stream. To illustrate the ideas in the simplest setting first, we begin by considering a bufferless communication link, following Hui =-=[41]-=-, [42]. The total offered load (measured in bits per second, for example) is given by X = J X j=1 n j X i=1 X ji where J is the number of connection types, n j is the number of connections of type j, ... |

152 |
Switching and traffic theory for integrated broadband networks
- HUI
- 1990
(Show Context)
Citation Context ...he context) somewhere between the mean and peak rate of the stream. To illustrate the ideas in the simplest setting first, we begin by considering a bufferless communication link, following Hui [41], =-=[42]-=-. The total offered load (measured in bits per second, for example) is given by X = J X j=1 n j X i=1 X ji where J is the number of connection types, n j is the number of connections of type j, and X ... |

142 |
Multi-way communication channels
- Ahlswede
- 1971
(Show Context)
Citation Context ... bias in parts of the networking community. At the same time, information theory has not done much to dispel that bias. During its early development, information theory did consider multiuser systems =-=[1]-=-, [2] and much of the subsequent work on such systems tried to capture (and did) many of the fundamental differences between the classical, stand-alone, single-channel case and that of the shared chan... |

139 | Large deviations for performance analysis
- Shwartz, Weiss
- 1995
(Show Context)
Citation Context ...d A o =C tends TO APPEAR IN IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 7 to zero as C and fl tend to infinity with C=fl fixed [43]. This follows from Cram'er's theorem (see =-=[44]-=-), to the effect that Chernoff's bound gives the correct exponent. So far, only a bufferless link confronted with demand that is constant over all time has been considered. The notion of effective ban... |

132 | Large deviations, the shape of the loss curve, and economies of scale in large multiplexers
- Botvich, Duffield
- 1995
(Show Context)
Citation Context ...onstant over all time has been considered. The notion of effective bandwidth can be extended to cover sources of data that vary in time, but that are statistically stationary and mutually independent =-=[45]-=-, [46], [47]. Let X ji [a; b] denote the amount of data generated by the ith connection of type j during an interval [a; b]. We assume that the process X is stationary in time. Set ff j (s; t) = 1 st ... |

125 | Two way communication channels
- Shannon
- 1961
(Show Context)
Citation Context ...ndipitous presence of Claude Shannon in Kleinrock's Ph.D. defense committee may have been the forecaster of the bond between the two fields. 2 We do not consider Shannon's work on the two-way channel =-=[17]-=- to be a genuine grasp of networking; this may be a debatable point, however. If we do accept that it does, then, again, Shannon must be credited with the prophesy of almost all aspects of the field o... |

96 |
A bibliographical guide to self-similar traffic and performance modeling for modern high-speed networks
- Willinger, Taqqu, et al.
- 1996
(Show Context)
Citation Context ...ective bandwidth of a self-similar Gaussian source. An extensive annotated bibliography on the subject is given in TO APPEAR IN IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 6 =-=[40]-=-. A. Effective Bandwidth of a Datastream One of the primary goals of information theory is to identify the effective information rate of a data source. The entropy or the rate-distortion function of a... |

83 | Bits through queues
- Anantharam, Verdu
- 1996
(Show Context)
Citation Context ...y information to a second party at no charge. The second party never answers the phone but only observes the times that it rings, that are controlled by the first party so as to convey a message (see =-=[33]-=- for a mathematical formulation and capacity result). A so-called two-ring, four-ring answering machine conveys information in the reverse direction as follows. It answers after four rings if it conta... |

68 |
Large deviations approximation for fluid queues fed by a large number of on/off sources
- Simonian, Guibert
- 1995
(Show Context)
Citation Context ...t over all time has been considered. The notion of effective bandwidth can be extended to cover sources of data that vary in time, but that are statistically stationary and mutually independent [45], =-=[46]-=-, [47]. Let X ji [a; b] denote the amount of data generated by the ith connection of type j during an interval [a; b]. We assume that the process X is stationary in time. Set ff j (s; t) = 1 st log E[... |

63 | Resource management in wide-area ATM networks using effective bandwidths
- Veciana, Kesidis, et al.
- 1995
(Show Context)
Citation Context ... is still finite in the limiting regime of C, B and n tending to infinity with fixed ratios among them [48]. The value of V (t) for t larger than t therefore does not influence the approximation. See =-=[49]-=- and [43] for extensive surveys on effective bandwidth, and [40] for a very extensive bibliographic guide to self-similar datastream models and their use. The paper [50] presents significant bounds an... |

53 |
Basic limits on protocol information in data communication networks
- Gallager
- 1976
(Show Context)
Citation Context ...of the natural connection between the two areas 2 . And he was the first to point out the fundamental significance of source burstiness and its relationship to information rate. In his landmark paper =-=[22]-=- on the subject, he considered a simple multiplexer of a finite number of sources, each of which was transmitting symbols from a ternary alphabet (0; 1; i), where i indicated idleness and, therefore, ... |

52 |
and R.Weber, Buffer overflow asymptotics for a switch handling many traffic sources
- Courcoubetis
- 1996
(Show Context)
Citation Context ... all time has been considered. The notion of effective bandwidth can be extended to cover sources of data that vary in time, but that are statistically stationary and mutually independent [45], [46], =-=[47]-=-. Let X ji [a; b] denote the amount of data generated by the ith connection of type j during an interval [a; b]. We assume that the process X is stationary in time. Set ff j (s; t) = 1 st log E[e sX i... |

44 | A network Pump
- Kang, Moskowitz, et al.
- 1998
(Show Context)
Citation Context ...ce of other clients that are not participating in the covert communication could be considered to cause noise on the covert channel, giving rise to subtle and complex multiuser communication channels =-=[39]-=-. To summarize this section we note that there is much to the theory and practice of timing information and timing channels which remains to be understood, especially in network scenarios. Additionall... |

41 | A comment on the confinement problem
- Lipner
- 1975
(Show Context)
Citation Context ...the system should restrict, and ideally completely prevent, the flow of information from high to low. One scenario is known as the computer processing unit (CPU) scheduling channel, and dates back to =-=[36]-=- and [37]. (See [38] for more background and citations.) Both clients submit tasks to their respective queues, one low queue and one high queue. The tasks are served by a single processor, that divide... |

29 |
Event Driven Topology Broadcast without Sequence Numbers
- Spinelli, Gallager
- 1989
(Show Context)
Citation Context ...orist, is what sorts of things are possible. For example, if nodes can enter and exit a network, and if routing tables are to be maintained, is it necessary to use sequence numbers? (The answer is no =-=[30]). Another-=- basic question, quite natural for an information theorist to ask, is "How many messages must be passed to accomplish a task." This is known as the communication complexity. For example, the... |

27 |
On the max-flow min-cut theorem of networks
- Ford, Fulkerson
- 1956
(Show Context)
Citation Context .... If we do accept that it does, then, again, Shannon must be credited with the prophesy of almost all aspects of the field of communication. It is also interesting to note that in [18] (as well as in =-=[19]-=- and [20]), a first look at max-flow min-cut relationships is provided; thus, the notion of flow approximations, widely used in network studies, was again, first noted by Shannon (among others). This ... |

23 |
Communication Nets
- Kleinrock
- 1964
(Show Context)
Citation Context ...nterconnection of the, then, so-called, interface message processors. Among the first who formulated the backbone elements of packet-switched networking was Kleinrock who, first, in his original work =-=[7]-=- that was based on his Ph.D. dissertation and, subsequently, in his two-volume book [8] on queueing systems, popularized many of the innovative intricacies and challenges of communication networks. 1 ... |

21 | The Interface Message Processor for the ARPA Computer Network - Heart, Kahn, et al. - 1970 |

19 |
Program Confinement in KVM/370
- Schaefer, Gold
- 1977
(Show Context)
Citation Context ...m should restrict, and ideally completely prevent, the flow of information from high to low. One scenario is known as the computer processing unit (CPU) scheduling channel, and dates back to [36] and =-=[37]-=-. (See [38] for more background and citations.) Both clients submit tasks to their respective queues, one low queue and one high queue. The tasks are served by a single processor, that divides its ser... |

14 |
Control and optimization methods in communication network problems
- Ephremides, Verdú
- 1989
(Show Context)
Citation Context ...AR IN IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 4 works and control system methodology (another, not fully explored and exploited relationship) that has been identified in =-=[28]-=- as well as in several subsequent publications and forums [29], was actually first pointed out, through [24], by information theorists. Information theorists played a part in the origins of the field ... |

13 |
The Capacity Region of a Multiple Access Discrete Memoryless Channel Can Increase with Feedback
- Gaarder, Wolf
- 1975
(Show Context)
Citation Context ...as realized that although feedback from the receiver to the source did not have an effect on channel capacity in single-user memoryless systems, it did have an effect in the case of multiuser systems =-=[3]-=-. But, still, the study of these systems has continued to be conducted in the restricted framework of nonbursty and delay-insensitive sources. In this paper we will not address multiuser information t... |

13 | M.H.: An Analysis of the Timed Z-channel
- Moskowitz, Greenwald, et al.
- 1996
(Show Context)
Citation Context ...strict, and ideally completely prevent, the flow of information from high to low. One scenario is known as the computer processing unit (CPU) scheduling channel, and dates back to [36] and [37]. (See =-=[38]-=- for more background and citations.) Both clients submit tasks to their respective queues, one low queue and one high queue. The tasks are served by a single processor, that divides its service among ... |

12 |
Functionoriented protocols for the ARPA computer network
- Crocker, Heafner, et al.
- 1972
(Show Context)
Citation Context ...book [8] on queueing systems, popularized many of the innovative intricacies and challenges of communication networks. 1 A substantial volume of other work in the late sixties and early seventies [9]-=-=[15]-=-, mostly by computer scientists and engineers, and the global interest on the still embryonic, but rapidly growing, field led to the formulation of the seven-layer, Open System Interconnection (OSI) f... |

11 |
Multiple user information theory
- El-Gamal, Cover
- 1980
(Show Context)
Citation Context ... in parts of the networking community. At the same time, information theory has not done much to dispel that bias. During its early development, information theory did consider multiuser systems [1], =-=[2]-=- and much of the subsequent work on such systems tried to capture (and did) many of the fundamental differences between the classical, stand-alone, single-channel case and that of the shared channel i... |

11 |
Dynamic models of shortest path routing algorithms for communication networks with multiple destinations
- Bertsekas
- 1979
(Show Context)
Citation Context ...alized soon after the publication of [24] that Gallager 's algorithm is an independently derived solution to a special case of convex optimization problems eminently studied and analyzed by Bertsekas =-=[25]-=-. This realization led to the collaboration between these two authors that produced the classic text on networking [26] that summarizes the field in the most complete and scientifically sound fashion.... |

10 | Topological optimization of computer networks - Frank, Chou - 1972 |

10 | Communication networks: Message path delays - Rubin - 1974 |

9 |
The importance of long-range dependence of VBR traffic engineering: Myths and realities
- Ryu, Elwalid
- 1996
(Show Context)
Citation Context ... time that the system behaves in an unusual way to build up the queue length just before the queue length exceeds B. The quantitysC +B=t \Gamma fl=s t is the effective capacity of the link. Following =-=[48]-=-, we call t the critical time scale. In the first limiting regime, described above, t tends to infinity, so the effective bandwidth becomes ff j (1; s ). Use of the Gartner-Ellis theorem of large devi... |

7 |
Networks of Gaussian channels with applications to feedback systems
- Elias
- 1967
(Show Context)
Citation Context ...n-cut relationships is provided; thus, the notion of flow approximations, widely used in network studies, was again, first noted by Shannon (among others). This notion was pursued further by Elias in =-=[21]. TO APPEAR IN IEEE -=-TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 3 observation was that, nonetheless, i did carry information. It carried the "message-start" or "message-end" infor... |

5 |
A protocol for packet network communication
- Cerf, Kahn
- 1974
(Show Context)
Citation Context ...ume book [8] on queueing systems, popularized many of the innovative intricacies and challenges of communication networks. 1 A substantial volume of other work in the late sixties and early seventies =-=[9]-=--[15], mostly by computer scientists and engineers, and the global interest on the still embryonic, but rapidly growing, field led to the formulation of the seven-layer, Open System Interconnection (O... |

5 |
Notes on effective bandwidths ," Stochastic networks: Theory and Applications
- Kelly
- 1996
(Show Context)
Citation Context ... the Hausdorff distance between the sets A=C and A o =C tends TO APPEAR IN IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 7 to zero as C and fl tend to infinity with C=fl fixed =-=[43]-=-. This follows from Cram'er's theorem (see [44]), to the effect that Chernoff's bound gives the correct exponent. So far, only a bufferless link confronted with demand that is constant over all time h... |

4 |
A note on the maximal flow through a network
- Elias, Feinstein, et al.
- 1956
(Show Context)
Citation Context ...table point, however. If we do accept that it does, then, again, Shannon must be credited with the prophesy of almost all aspects of the field of communication. It is also interesting to note that in =-=[18]-=- (as well as in [19] and [20]), a first look at max-flow min-cut relationships is provided; thus, the notion of flow approximations, widely used in network studies, was again, first noted by Shannon (... |

4 |
On the Shannon capacity of discrete time queues
- Thomas
- 1997
(Show Context)
Citation Context ... if the exponential service time distribution is replaced by another with the same mean, then the capacity cannot decrease [33]. The Shannon capacity of a discrete-time queue is addressed in [34] and =-=[35]-=-. As an aside, we briefly note an application of [33] to the source coding problem of [22]. Given a ratesPoisson process of packet arrivals and a mean delay constraint D, the rate-distortion problem o... |

4 | A thermodynamic theory of broadband networks with application to dynamic routing
- Hui, Karasan
- 1995
(Show Context)
Citation Context ...de to self-similar datastream models and their use. The paper [50] presents significant bounds and analysis related to notions of equivalent bandwidth with a different terminology. Finally, the paper =-=[51]-=- connects the theory of effective bandwidths to thermodynamics and statistical mechanics. B. Network Engineering Through Traffic Constraints An alternative to treating datastreams with statistical met... |

3 |
Discrete-Event Systems: Modeling and Performance Analysis
- Cassandras
- 1993
(Show Context)
Citation Context ... OCTOBER 1998 4 works and control system methodology (another, not fully explored and exploited relationship) that has been identified in [28] as well as in several subsequent publications and forums =-=[29]-=-, was actually first pointed out, through [24], by information theorists. Information theorists played a part in the origins of the field of distributed network protocols. Early in the implementation ... |

2 | New optimization criteria for message switching networks - Meister, Muller, et al. - 1971 |

1 |
Modern statistical processing: a retrospective, " this issue
- Kailath, Poor
(Show Context)
Citation Context ...en in this case, the main thrust of the work continues to ignore the intrinsic role of delay and burstiness. Nonetheless, it has spawned the rapid development of the field of multiuser detection (see =-=[5]-=- in this issue and, for a more thorough account, [6]). In a sense, both multiuser information theory and multiuser detection theory represent major forays of information theory toward the field of net... |

1 |
Information theory - the first 25 years", Keynote Address
- Gallager
- 1973
(Show Context)
Citation Context .... B. Protocol Overhead The first to recognize the significance of networking to information theory, both in terms of the challenges as well as the opportunities it presented, was Gallager who in 1973 =-=[16]-=- offered a clear vision of the natural connection between the two areas 2 . And he was the first to point out the fundamental significance of source burstiness and its relationship to information rate... |

1 |
Data Networks, Prentice-Hall, First edition
- Bertsekas, Gallager
- 1987
(Show Context)
Citation Context ...case of convex optimization problems eminently studied and analyzed by Bertsekas [25]. This realization led to the collaboration between these two authors that produced the classic text on networking =-=[26]-=- that summarizes the field in the most complete and scientifically sound fashion. The algorithm originally proposed in [24] and modified accordingly in [25] is compatible with the class of distributed... |

1 |
The exponential distribution in Information Theory," Problemy Peredachi Informatsii
- Verd'u
- 1996
(Show Context)
Citation Context ...he mean number of arrivals per period. Note that the output would not determine the exact arrival times. A different ratedistortion function for Poisson processes was defined and identified by Verd'u =-=[32]-=-. Even if enough protocol information is provided to identify the packets at the destination within a specified delay, such delay may be unobtainable due to the possible queueing delay experienced by ... |

1 |
The information theoretic capacity of discrete-time queues
- Bedekar, Azizoglu
- 1998
(Show Context)
Citation Context ...addition, if the exponential service time distribution is replaced by another with the same mean, then the capacity cannot decrease [33]. The Shannon capacity of a discrete-time queue is addressed in =-=[34]-=- and [35]. As an aside, we briefly note an application of [33] to the source coding problem of [22]. Given a ratesPoisson process of packet arrivals and a mean delay constraint D, the rate-distortion ... |