Signal flow graphs with dataflow semantics have been used in signal processing system simulation, algorithm development, and real-time system design. Dataflow semantics implicitly expose function parallelism by imposing only a partial ordering constraint on the execution of functions. One particular form of dataflow called synchronous dataflow (SDF) has been quite popular in programming environments for digital signal processing (DSP) since it has strong formal properties and is ideally suited for expressing multirate DSP algorithms. However, SDF and other dataflow models use first-in first-out (FIFO) queues on the communication channels and are thus ideally suited only for one-dimensional (1-D) signal processing algorithms. While multidimensional systems can also be expressed by collapsing arrays into 1-D streams, such modeling is often awkward and can obscure potential data parallelism that might be present. SDF can be generalized...

The original vision of ubiquitous computing [14] is about enabling people to more easily accomplish tasks through the seamless interworking of the physical environment and a computing infrastructure. A major challenge to the practical realization of this vision involves the integration of commercial-o-the-shelf (COTS) hardware and software components: consider the awkwardness of such a mundane task as exporting a textual memo written on a Palm Pilot to a Microsoft Word document. It is not enough to overcome the protocol and data format mismatches that currently impede the interoperation of these entities: for the user experience to be truly seamless, we must provide a framework for the dynamic connection of such endpoints on demand, to support the ad-hoc interactions that are an integral part of ubiquitous computing. To this end, we oer a dynamic mediation framework called Paths. A Path consists of dynamically instantiated, automatically composable operators that brid...

The Chidi system is a PCI-bus media processor card which performs its processing tasks on a large fieldprogrammable gate array (Altera 10K100) in conjunction with a general purpose CPU (PowerPC 604e). Special address-generation and buffering logic (also implemented on FPGAs) allows the reconfigurable processor to share a local bus with the CPU, turning burst accesses to memory into continuous streams and converting between the memory's 64-bit words and the media data types. In this paper we present the design requirements for the Chidi system, describe the hardware architecture, and discuss the software model for its use in media processing. Keywords: video compression, field-programmable gate array, data-flow computing, digital signal processing 1. INTRODUCTION Now that field-programmable gate arrays (FPGAs) have reached gate densities at which they can perform useful computational tasks, particularly in certain mathematical and signal-processing domains, they are increasingly bein...

Abstract This thesis proposes a mechanism, streams, for overcoming many of the problems associated with the parallel processing of media. A programming model and runtime system using the stream mechanism, MagicEight, is also proposed. MagicEight supports medium to coarse grain parallelism, using a hybrid dataflow model of execution. Multidimensional streams of data elements provide a scalable means of both obtaining parallelism and ameliorating an indeterminate memory access latency. The machine architectures which MagicEight is intended to support range from a single general purpose processor to heterogenous multiprocessor systems of two to two hundred processors interconnected by communications channels of varying capabilities. In the multiprocessor case, some of the processors may be specialized-- capable of executing a restricted set of algorithms much more efficiently than a general purpose processor. 1 1 Introduction While the ability of personal computers to acquire, process, and present video and sound has now been established, the computational requirements of many media applications exceed that provided by a single general purpose processor. My thesis is that streams are a mechanism for enabling efficient dynamic parallelization of the computational tasks typically found in media processing. I am also proposing a programming model for media processing using this mechanism. The model is a variant of hybrid dataflow, utilizing multidimensional streams as both a basic data type and a mechanism for synchronization and obtaining parallelism. It supports machine architectures containing a heterogenous mix of processors. In order to provide higher compression, greater flexibility, and more semantic description of scene content, video1 is increasingly moving toward representations in which the data are segmented not into arbitrary fixed and regular patterns, but rather into objects or regions determined by scene-understanding algorithms [18][23][30][7][6]. These structured (or objectbased) representations are effectively sets of objects and "scripts " describing how to render output images from the objects. The media being presented is generated at the receiver, not merely decoded, allowing the presentation to adapt to receiver capabilities, viewing situation, and user preferences.

The generation of video and audio coding methods to follow the present, pioneering generation have not yet been standardized, but it is possible to predict many of their characteristics. I discuss these, with particular reference to their impact on the design of software and hardware systems for multimedia. Keywords: video compression, model-based coding, data-flow computing, digital signal processing 1. BACKGROUND Compressed digital video and audio have become common. The price of dedicated processing circuits has dropped to the point that they are cost-effective for consumer applications, while general-purpose processor performance is increasing to the point that standard personal computers can support digital multimedia applications with little or no additional hardware support. But digital media representations are a moving target, and now that we know appropriate hardware/software design methodologies and optimizations to achieve real-time performance for the current generation ...

This paper discusses the Chidi holographic video processing system (called Holo-Chidi) used for real-time computation of Computer Generated Holograms and the subsequent display of the holograms at video frame rates. Chidi is a reconfigurable multimedia processing system designed at the MIT Media Laboratory for real-time synthesis and analysis of multimedia data in general and digital video frames in particular. Holo-Chidi which is an adaptation of Chidi, comprises two main components: the sets of processor cards and the the display interface cards. Each processor card consists of a General Purpose Processor (GPP), a processor to PCI bridge, up to 128MByte DRAM, three SRAM-based Field Programmable Gate Arrays (FPGAs), and high bandwidth data transceivers, all resident on a standard PCI form-factor card. One of the FPGAs, called the RP (Reprogrammable Processor), is dynamically reconfigurable and enables Chidi to be used as a flexible specialized hardware for use in performing computatio...

We designed, implemented, and evaluated a signal-processing environment that runs on a general-purpose multiprocessor system, allowing easy prototyping of new algorithms and integration with applications. The environment allows the composition of modules implementing individual signal-processing algorithms into a functional application, automatically optimizing their performance. We decompose the problem into four independent components: signal processing, data management, scheduling, and control. This simplifies the programming interface and facilitates transparent parallel signalprocessing. For tested applications, our system both runs efficiently on single-processors systems and achieves near-linear speedups on symmetric-multiprocessor (SMP) systems. Thesis Supervisor: John V. Guttag Title: Professor, Computer Science and Engineering 4 5 Acknowledgements I would first like to thank to John Guttag for his guidance and support throughout my work on Pspectra. I would also like to...

The demands in terms of processing performance, communication bandwidth and real-time throughput of many multimedia applications are much higher than today's processing architectures can deliver. The PROPHID heterogeneous multiprocessor architecture template aims to bridge this gap. The template contains a general purpose processor connected to a central bus, as well as several high-performance application domain specific processors. A high-throughput communication network is used to meet the high bandwidth requirements between these processors. In this network multiple time-division-multiplexed data streams are transferred over several parallel physical channels. This paper presents a method for guaranteeing the throughput for hard-real-time streams in such a network. At compile time sufficient bandwidth is assigned to these streams. The assignment can be determined in polynomial time. Remaining bandwidth is assigned to soft-real-time streams at run time. We thus achieve efficient stream communication with guaranteed performance.