This paper addresses the problem of streaming packetized media over a lossy packet network to a wireless client, in a rate-distortion optimized way. We introduce an incremental redundancy error correction scheme that combats the effects of both packet loss and bit errors in an end-to-end fashion, without support from the underlying network or from an intermediate base station. The scheme is employed within an optimization framework that enables the sender to compute which packets it should send, out of all the packets it could send at a given transmission opportunity, in order to meet an average transmission rate constraint while minimizing the average end-to-end distortion. Experimental results show that our system is robust and maintains quality of service over a wide range of channel conditions. Up to 8 dB performance gains are registered over systems that are not rate-distortion optimized, at bit error rates as large as 10 .

Despite the well-known challenges of variations in throughput, delay, and packet loss over the Internet, video streaming has experienced phenomenal growth, owing to the extensive research in video coding and transmission. In this paper, we review several recent advances for channel-adaptive video streaming that, we believe, will benefit the design of video streaming systems in the future. Employed in different components of the system, these techniques have the common objective of providing efficient, robust, scalable and low-latency streaming video. First, by allowing the client to control the rate at which it consumes data, adaptive media playout can be used to reduce receiver buffering and therefore average latency, and provide limited rate scalability. Secondly, rate-distortion optimized packet scheduling, a transport technique, provides a flexible framework to determine the best packet to send given the channel behaviors, the packets' deadlines, their transmission histories, the distortion reduction associated with sending each packet, and the inter-packet dependencies. Thirdly, at the source encoder channel-adaptive packet dependency control can greatly improve the error-resilience of streaming video and reduce latency. Finally we address the specific additional challenges for wireless video streaming. We consider three architectures for wireless video and discuss the utility of the reviewed techniques for each architecture.

Multimedia applications involving the transmission of video over communication networks are rapidly increasing in popularity. Such applications can greatly benefit from adapting video coding parameters to network conditions as well as adapting network parameters to better support the application requirements. These two dimensions can both be viewed as allocating source and network resources to improve video quality. In this paper, we highlight recent advances in optimal resource allocation for real-time video communications over unreliable and resource constrained communication channels. More specifically, we focus on point-to-point coding and delivery schemes in which the sequences are encoded on the fly. We present a high-level framework for resource-distortion optimization. The framework can be used for jointly considering factors across network layers, including source coding, channel resource allocation, and error concealment. For example, resources can take the form of transmission energy in a wireless channel, and transmission cost in a DiffServ-based Internet channel. This framework can be used to optimally trade off resource consumption with end-to-end video quality in packet-based video transmission. After giving an overview of this framework, we review recent work in two areas—energy efficient wireless video transmission and resource allocation for Internet-based applications. Keywords—Cross-layer design, energy efficient, error resilience, distortion estimation, internet video, wireless video. I.

Differentiated Services (DiffServ) is one of the leading architectures for providing quality of service in the Internet. We propose a scheme for real-time video transmission over a DiffServ network that jointly considers video source coding, packet classification, and error concealment within a framework of cost-distortion optimization. The selection of encoding parameters and packet classification are both used to manage end-to-end delay variations and packet losses within the network. We present two dual formulations of the proposed scheme: the minimum distortion problem in which the objective is to minimize the end-to-end distortion subject to cost and delay constraints, and the minimum cost problem which minimizes the total cost subject to end-to-end distortion and delay constraints. A solution to these problems using Lagrangian relaxation and dynamic programming is given. Simulation results demonstrate the advantage of jointly adapting the source coding and packet classification in DiffServ networks.

Providing end-to-end quality of service (QoS) support is essential for video delivery over the next-generation wireless Internet. In this paper, we address several key elements in the end-to-end QoS support, including scalable video representation, network-aware end system, and network QoS provisioning. There are generally two approaches in QoS support: the network-centric and the end-system centric solutions. The fundamental problem in a network-centric solution is how to map QoS criterion at different layers respectively, and optimize total quality across these layers. In this paper, we first present the general framework of a cross-layer network-centric solution, and then describe the recent advances in network modeling, QoS mapping, and QoS adaptation. The key targets in end-system centric approach are network adaptation and media adaptation.In this paper, we present a general framework of the end-system centric solution and investigate the recent developments. Specifically, for network adaptation, we review the available bandwidth estimation and efficient video transport protocol; for media adaptation, we describe the advances in error control, power control, and corresponding bit allocation. Finally, we highlight several advanced research directions.

In this paper, we address the problem of multimedia streaming over DiffServ networks that offer various qualities of service (QoS) at different price points. In particular, we seek cost-distortion optimized transmission policies that minimize the distortion seen by the client under a cost constraint. Our results show that for ondemand streaming of stored media, DiffServ networks provide little or no gain over networks that offers only a single, cost-effective, QoS. However, for real-time conversational communication or multicast streaming, optimized transmission over DiffServ networks can perform better than optimized transmission over any single QoS by over 2 dB at the same cost.

Streaming video over IP is studied by integrating the error resilient scalable source codec with constant quality rate adaptation and prioritized network transmission. The scalable codec is based on the standardized ISO/IEC MPEG-4 fine granular scalable (FGS) coding scheme, which allows arbitrary truncation according to the given rate budget. To maximize the end-to-end visual quality, constant quality video rate control (CQVRC) is proposed to preserve smooth video quality when there is a significant amount of variation in the video source, the available bandwidth or both. The proposed CQVRC attempts to provide smooth video quality for both the base layer (BL) and the enhancement layer (EL). The video stream is then prioritized for priority dropping, where rate adaptation is dynamically performed to meet the time-varying available bandwidth. This kind of prioritized stream can benefit from QoS networks such as the IP differentiated service (DiffServ). All key components, including FGS encoding, CQVRC based rate adaptation and packetization, error resilient decoding, and differentiated forwarding, are seamlessly integrated into one system. By focusing on the end-to-end quality, we set both source and network parameters properly to achieve a superior performance of FGS video streaming. Keywords: MPEG-4 fine grain scalability (FGS), video streaming, constant quality, rate adaptation, differentiated service (DiffServ), unequal error protection (UEP), prioritized packetization, and priority networks. 1.

Internet transmission is characterized by variations in throughput, delay, and packet loss, which can severely affect the quality of multimedia presentations delivered over the network. Still, Internet video streaming has experienced phenomenal growth in the last few years, owing to the extensive research in video coding and transmission. In this paper, we review several recent advances for network-adaptive video streaming that, we believe, will benefit the design of video streaming systems in the future. Employed in different system components, these techniques have the common objective of providing efficient, robust, scalable and low-latency streaming video. They range from purely server or source encoder-based techniques, through transmission schemes that could be implemented either at the sender or at the receiver, to purely client-based techniques. We discuss each of them in detail, presenting also related work and experimental results. We end the paper with a summary of the reviewed techniques and a brief discussion of future research directions.

Abstract — In video communications over IP networks, quality of service (QoS) guarantees must be introduced to limit the effect of packet losses. In particular, end–to–end QoS can be improved if packets are protected according to the distortion that would be introduced at the receiver by their loss. In the traditional Analysis–by–Synthesis (AbS) approach, each packet is assumed lost, error concealment applied, the sequence decoded and the resulting overall distortion computed. This process produces reliable distortion estimates, but is computationally demanding. In this work we present a hybrid approach: the distortion introduced in the current frame is evaluated with the AbS method, while the distortion in future frames is estimated by means of a statistical error–propagation model. Results obtained on eight, widely different H.264 sequences show that the proposed model successfully estimates overall distortion with very low complexity. Network simulations also show that model–based packet classification, when used for video transmission over DiffServ networks, delivers PSNR results which are consistently within 0.1 dB compared to the AbS technique. I.

Video streaming is becoming an increasingly important part of the current Internet; however, before high-quality streaming becomes a reality, the best-effort model of the Internet may need to be adapted to provide certain scalable services specifically targeted at video flows. In this paper, we study one such mechanism and propose a new video streaming framework, which allows applications to mark their own packets with different priority and use multi-queue congestion control inside routers to automatically drop the less-important packets during buffer overflows. We describe priority AQM algorithms that provide optimal performance for video applications and study a variation of Kelly's congestion controls as part of this framework. Through simulation and analytical investigation, we find that our AQM based solution allows the application to maintain a much higher quality of video for the end-user compared to similar scenarios in a best-effort network. We call the combined framework PELS --- Partitioned Enhancement Layer Streaming.