We consider the problem of embedding one signal (e.g., a digital watermark), within another "host" signal to form a third, "composite" signal. The embedding is designed to achieve efficient tradeoffs among the three conflicting goals of maximizing information-embedding rate, minimizing distortion between the host signal and composite signal, and maximizing the robustness of the embedding. We introduce new classes of embedding methods, termed quantization index modulation (QIM) and distortion-compensated QIM (DC-QIM), and develop convenient realizations in the form of what we refer to as dither modulation. Using deterministic models to evaluate digital watermarking methods, we show that QIM is "provably good" against arbitrary bounded and fully informed attacks, which arise in several copyright applications, and in particular, it achieves provably better rate distortion--robustness tradeoffs than currently popular spread-spectrum and low-bit(s) modulation methods. Furthermore, we show that for some important classes of probabilistic models, DC-QIM is optimal (capacity-achieving) and regular QIM is near-optimal. These include both additive white Gaussian noise (AWGN) channels, which may be good models for hybrid transmission applications such as digital audio broadcasting, and mean-square-error-constrained attack channels that model private-key watermarking applications.

Abstract—An information-theoretic analysis of information hiding is presented in this paper, forming the theoretical basis for design of information-hiding systems. Information hiding is an emerging research area which encompasses applications such as copyright protection for digital media, watermarking, fingerprinting, steganography, and data embedding. In these applications, information is hidden within a host data set and is to be reliably communicated to a receiver. The host data set is intentionally corrupted, but in a covert way, designed to be imperceptible to a casual analysis. Next, an attacker may seek to destroy this hidden information, and for this purpose, introduce additional distortion to the data set. Side information (in the form of cryptographic keys and/or information about the host signal) may be available to the information hider and to the decoder. We formalize these notions and evaluate the hiding capacity, which upper-bounds the rates of reliable transmission and quantifies the fundamental tradeoff between three quantities: the achievable information-hiding rates and the allowed distortion levels for the information hider and the attacker. The hiding capacity is the value of a game between the information hider and the attacker. The optimal attack strategy is the solution of a particular rate-distortion problem, and the optimal hiding strategy is the solution to a channel-coding problem. The hiding capacity is derived by extending the Gel’fand–Pinsker theory of communication with side information at the encoder. The extensions include the presence of distortion constraints, side information at the decoder, and unknown communication channel. Explicit formulas for capacity are given in several cases, including Bernoulli and Gaussian problems, as well as the important special case of small distortions. In some cases, including the last two above, the hiding capacity is the same whether or not the decoder knows the host data set. It is shown that many existing information-hiding systems in the literature operate far below capacity. Index Terms—Channel capacity, cryptography, fingerprinting, game theory, information hiding, network information theory,

Watermarking models a copyright protection mechanism where an original source sequence or "covertext" is modified before distribution to the public in order to embed some extra information. The embedding should be transparent (i.e., the modified data sequence or "stegotext" should be similar to the covertext) and robust (i.e., the extra information should be recoverable even if the stegotext is modified further, possibly by a malicious "attacker"). We compute the coding capacity of the watermarking game for a Gaussian covertext and squared-error distortions. Both the public version of the game (covertext known to neither attacker nor decoder) and the private version of the game (covertext unknown to attacker but known to decoder) are treated. While the capacity of the former cannot, of course, exceed the capacity of the latter, we show that the two are, in fact, identical. These capacities depend critically on whether the distortion constraints are required to be met in expectation or with probability one. In the former case the coding capacity is zero, whereas in the latter it coincides with the value of related zero-sum dynamic mutual informations games of complete and perfect information. # Parts of this work were presented at the 2000 Conference on Information Sciences and Systems (CISS '00), Princeton University, Princeton, NJ, March 15--17, 2000, and at the 2000 IEEE International Symposium on Information Theory (ISIT '00), Sorrento, Italy, June 25--30, 2000.

We show that the duality between channel capacity and data compression is retained when state information is available to the sender, to the receiver, to both, or to neither. We present a unified theory for eight special cases of channel capacity and rate distortion with state information, which also extends existing results to arbitrary pairs of independent and identically distributed (i.i.d.) correlated state information @ I PA available at the sender and at the receiver, respectively. In particular, the resulting general formula for channel capacity a �— � @ A ‘ @ Y P A @ Y IA “ assumes the same form as the generalized Wyner–Ziv rate distortion function @ A a �� � @ A @ ” A ‘ @ Y I A @ Y PA“.

This tutorial paper reviews the theory and design of codes for hiding or embedding information in signals such as images, video, audio, graphics, and text. Such codes have also been called watermarking codes; they can be used in a variety of applications, including copyright protection for digital media, content authentication, media forensics, data binding, and covert communications. Some of these applications imply the presence of an adversary attempting to disrupt the transmission of information to the receiver; other applications involve a noisy, generally unknown, communication channel. Our focus is on the mathematical models, fundamental principles, and code design techniques that are applicable to data hiding. The approach draws from basic concepts in information theory, coding theory, game theory, and signal processing, and is illustrated with applications to the problem of hiding data in images. Keywords—Coding theory, data hiding, game theory, image processing, information theory, security, signal processing, watermarking. I.

Abstract — We develop coding strategies for estimation under communication constraints in tree-structured sensor networks. The strategies have a modular and decentralized architecture. This promotes the flexibility, robustness, and scalability that wireless sensor networks need to operate in uncertain, changing, and resource-constrained environments. The strategies are based on a generalization of Wyner-Ziv source coding with decoder side information. We develop solutions for general trees, and illustrate our results in serial (pipeline) and parallel (hub-and-spoke) networks. Additionally, the strategies can be applied to other network information theory problems. They have a successive coding structure that gives an inherently less complex way to attain a number of prior results, as well as some novel results, for the CEO problem, multiterminal source coding, and certain classes of relay channels. Index Terms — sensor networks, distributed estimation, data fusion, side information, Wyner-Ziv coding, rate distortion theory, CEO problems, multiterminal source coding, distributed detection, relay channels. I.

We show that the Lagrange dual problems of the channel capacity problem with input cost and the rate distortion problem are simple geometric programs. Upper bounds on channel capacity and lower bounds on rate distortion can be efficiently generated from their duals. For channel capacity, the geometric programming dual characterization is shown to be equivalent to the minmax Kullback–Leibler (KL) characterization in [10], [14]. For rate distortion, the geometric programming dual is extended to rate distortion with two-sided state information. A “duality by mapping ” is then given between the Lagrange dual problems of channel capacity with input cost and rate distortion, which resolves several apparent asymmetries between their primal problems in the familiar form of mutual information optimization problems. Both the primal and dual problems can be interpreted in a common framework of free energy optimization from statistical physics.

We consider a generalization of Costa's writing on dirty paper model in which a powerlimited communicator encounters two independent sources of additive noise, one of which is known non-causally to the encoder. We seek to characterize when the capacity for this channel would not change if the source of noise known to the encoder were also known to the decoder; we call this property private-public equivalence (PPE). Costa showed that this model has PPE if both sources of noise are IID Gaussian. We show that this model has PPE as long as the unknown noise is Gaussian (but not necessarily IID) for any distribution on the known noise.

We consider the problem of lossy joint source-channel coding in a communication system where the encoder has access to channel state information (CSI) and the decoder has access to side information that is correlated to the source. This configuration combines the Wyner-Ziv model of pure lossy source coding with side information at the decoder and the Shannon/Gel'fand-Pinsker model of pure channel coding with CSI at the encoder. We prove a separation theorem for this communication system, which asserts that there is no loss in asymptotic optimality in applying first, an optimal Wyner-Ziv source code and then, an optimal Gel'fand-Pinsker channel code. We then derive conditions for the optimality of a symbol-by-symbol (scalar) source-channel code, and demonstrate situations where these conditions are met. Finally, we discuss a few practical applications, including of overlaid communication where the model under discussion is useful.

We present simple, practical codes designed for the binary and Gaussian dirty-paper chan-nels. We show that the dirty paper decoding problem can be transformed into an equivalent multiple-access decoding problem, for which we apply superposition coding. Our concept is a generalization of the nested lattices approach of Zamir, Shamai and Erez. In a theoretical setting, our constructions are capable of achieving capacity using random component codes and maximum-likelihood decoding. We also present practical implementations of the con-structions, and simulation results for both dirty-paper channels. Our results for the Gaussian dirty-paper channel are on par with the best known results for nested-lattices. We discuss the binary dirty-tape channel, for which we present a simple, effective coding technique. Finally, we propose a framework for extending our approach to general Gel’fand-Pinsker channels. Index Terms- dirty paper, dirty tape, multiple-access channel, side information, superposition coding.