An information-theoretic model for image watermarking and data hiding is proposed in this paper. Recent theoretical results are used to characterize the fundamental capacity limits of image watermarking and data-hiding systems. Capacity is determined by the statistical model used for the host image, by the distortion constraints on the data hider and the attacker, and by the information available to the data hider, to the attacker, and to the decoder. We consider autoregressive, block-DCT and wavelet statistical models for images and compute datahiding capacity for compressed and uncompressed host-image sources. Closed-form expressions are obtained under sparse-model approximations. Models for geometric attacks and distortion measures that are invariant to such attacks are considered. Keywords --- image watermarking, data hiding, minimax techniques, autoregressive processes, discrete cosine transform, wavelets, image modeling, information theory. EDICS Category : 5---AUTH # Work presented in part at the IEEE International Conference on Image Processing, Vancouver, Canada, Oct. 2000. This research was supported by NSF grants MIP-97-07633 and CDA 96-24396. + Corresponding Author: Tel +1 217 244-8366, fax +1 217 244-8371, email: moulin@ifp.uiuc.edu. 1 1

We consider the estimation of the order, i.e., the number of hidden states, of a special class of discrete-time finite-alphabet hidden Markov sources. This class can be characterized in terms of equivalent renewal processes. No a priori bound is assumed on the maximum permissible order. An order estimator based on renewal types is constructed, and is shown to be strongly consistent by computing the precise asymptotics of the probability of estimation error. The probability of underestimation of the true order decays exponentially in the number of observations while the probability of overestimation goes to zero sufficiently fast. It is further shown that this estimator has the best possible error exponent in a large class of estimators. Our results are also valid for the general class of binary independent-renewal processes with finite mean renewal times.

In a world where different systems have to share the same spectrum, the received (interfering) power may be a more relevant constraint than the maximum transmit power. Motivated by such a spectrum-sharing approach, this paper investigates the behavior of capacity under received-power constraints, modeling for example the maximum interference that one system may inflict on another. The insight of the paper is that while in the point-to-point case, transmit and received-power constraints are largely equivalent, they can lead to quite different conclusions in network cases, including relay networks, multiple access channels with dependent sources and feedback, and collaborative communication scenarios. 1

Capacity formulas and random-coding exponents are derived for a generalized family of Gel’fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized Maximum a Posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning. Index terms: channel coding with side information, error exponents, arbitrarily varying channels,

Abstract—The multicast capacity is determined for acyclic networks that have deterministic links with broadcasting at the transmitters and no interference at the receivers. Such networks were studied by M. R. Aref, and are here called Aref networks. The multicast capacity is shown to have a max-flow, min-cut interpretation. This result complements existing theory for networks of directed channels, networks of undirected channels, and packet erasure networks. It is also shown that one cannot always separate channel and network coding in Aref networks. I.

Abstract—When information is to be transmitted over an unknown, possibly unreliable channel, an erasure option at the decoder is desirable. Using constant-composition random codes, we propose a generalization of Csiszár and Körner’s maximum mutual information (MMI) decoder with an erasure option for discrete memoryless channels. The new decoder is parameterized by a weighting function that is designed to optimize the fundamental tradeoff between undetected-error and erasure exponents for a compound class of channels. The class of weighting functions may be further enlarged to optimize a similar tradeoff for list decoders—in that case, undetected-error probability is replaced with average number of incorrect messages in the list. Explicit solutions are identified. The optimal exponents admit simple expressions in terms of the sphere-packing exponent, at all rates below capacity. For small erasure exponents, these expressions coincide with those derived by Forney (1968) for symmetric channels, using maximum a posteriori decoding. Thus, for those channels at least, ignorance of the channel law is inconsequential. Conditions for optimality of the Csiszár–Körner rule and of the simpler empirical-mutual-information thresholding rule are identified. The error exponents are evaluated numerically for the binary symmetric channel. Index Terms—Constant-composition codes, erasures, error exponents, list decoding, maximum mutual information (MMI) decoder, method of types, Neyman–Pearson hypothesis testing, random codes, sphere packing, universal decoding. I.

Probability-of-error exponents have recently been derived for watermarking systems based on spread-spectrum and quantization-index modulation methods. This paper takes this work one step further and presents minmax error exponents for any embedding scheme and any attack (subject to distortion constraints) at all rates below capacity. The decoders used are universal: they do not know the attack used. Randomized codes outperform deterministic codes, except in the case of memoryless attacks where the same performance is obtained using either kind of code.

The proliferation of multimedia and the advent of the Internet and other public networks have created many new applications of information hiding in multimedia security and forensics. This dissertation focuses on two of these application scenarios: steganography (and its counter problem, steganalysis), and fingerprinting. First, from a detection-theoretic perspective, we quantify the detectability of commonly used information-hiding techniques such as spread spectrum and distortion-compensated quantization index modulation, and also the detectability of block-based steganography. We devise a practical steganalysis method that exploits the peculiar block structure of block-DCT image steganography. To cope with the twin difficulties of unknown image statistics and unknown steganographic codes, we explore image steganalysis based on supervised learning and build an optimized classifier that outperforms previously proposed image steganalysis methods. Then, from an information-theoretic perspective, we derive the capacity and random-coding error exponent of perfectly secure steganography and public fingerprinting. For both games, a randomized stacked-binning scheme and a matched maximum penalized mutual information decoder are used to achieve capacity and to realize a random-coding error exponent that is strictly positive at all rates below capacity.

A strong converse for the Gel’fand-Pinsker channel is established in this paper. The method is then extended to a multiuser scenario. A strong converse is established for the multiple-access Gel’fand-Pinsker channel under the maximum error criterion, and the capacity region is determined. 1.

This paper studies fingerprinting games in which the number of colluders and the collusion channel are unknown. The fingerprints are embedded into host sequences (representing signals to be protected) and provide the receiver with the capability to trace back pirated copies to the colluders. The colluders and the fingerprint embedder are subject to signal fidelity constraints. Our problem setup unifies the signal-distortion and Boneh-Shaw formulations of fingerprinting. Several bounds on fingerprinting capacity have been presented in recent literature. This paper derives exact capacity formulas and presents a new randomized fingerprinting scheme with the following properties: (1) the receiver does not need to know the coalition size and collusion channel; (2) a tunable parameter ∆ trades off false-positive and false-negative error exponents; (3) the receiver provides a reliability metric for its decision; and (4) the scheme is capacity-achieving when the false-positive exponent ∆ tends to zero. A fundamental component of this scheme is the use of a “time-sharing ” randomized sequence. The decoder is a minimum penalized equivocation decoder, where the significance of each candidate coalition is assessed relative to a threshold, and the penalty is proportional to coalition size. A much simpler threshold decoder that satisfies properties (1)—(3) above but not (4) is also given. Index Terms. Fingerprinting, traitor tracing, watermarking, data hiding, randomized codes, universal codes, method of types, maximum mutual information decoder, minimum equivocation decoder, channel coding with side information, capacity, error exponents, multiple access channels, model order selection.