Citation Context

...ds its rules to the event handler (Section 3.3). Then each plan fragment is processed in turn, as a single, pipelined execution unit. The operator tree is executed using the top-down "iterator&qu=-=ot; model [11]-=-. (Note that our implementation of the double pipelined join is an iterator-based adaptation, as described in Section 4.2). Control flows from the root node and makes its way down the tree. At the lea...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

... The key contribution is that adaptivity is designed into its core to facilitate interleaving of planning and execution. Furthermore, Tukwilasprovides a platform for incorporating hybrid optimization =-=[19, p181]-=- and important query optimization techniques that have been developed previously in isolation (e.g., query scrambling [22], choose nodes [12], runtime re-optimization [15], optimization of fusion quer...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...pecially well suited for adaptive execution and for minimizing the time required to obtain the first answers to a query. Specifically, it employs an enhanced version of the double pipelined hash join =-=[24]-=- (a join implementation which executes in a symmetric, data-driven manner) and techniques for adapting its execution when there is insufficient memory. In addition, the Tukwila execution engine includ...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...d metadata, and even a plan that may be good on average should be abandoned if unexpected situations arise. While runtime adaptivity has been shown to speed up performance even in traditional systems =-=[15, 12]-=-, it becomes critical to performance in the data integration context (e.g., [22]). 1.2 Adaptive Features of Tukwila This paper describes the Tukwila 1 data integration system, designed with adaptivity...

Citation Context

...d metadata, and even a plan that may be good on average should be abandoned if unexpected situations arise. While runtime adaptivity has been shown to speed up performance even in traditional systems =-=[15, 12]-=-, it becomes critical to performance in the data integration context (e.g., [22]). 1.2 Adaptive Features of Tukwila This paper describes the Tukwila 1 data integration system, designed with adaptivity...

Citation Context

...esulting performance is worse than replanning from scratch. 7 Related work The INGRES query optimization algorithm originally interleaved steps of constructing a query execution plan and executing it =-=[26]-=-. However, their approach was largely eclipsed by less flexible System-R style optimizers. Only recently have Kabra and DeWitt demonstrated the utility of runtime re-optimization for conventional data...

Citation Context

...sty arrivals of data thereafter. Hence, even if the query optimizer is able to determine the best plan based on total work, the data arrival characteristics may cause it to be inefficient in practice =-=[22]-=-. ffl Overlap and redundancy among sources: as a result of the heterogeneity of the data sources, there is often significant overlap or redundancy among them. Hence, the query processor needs to be ab...

Citation Context

...nd is overlap information about pairs of data sources (that is, the probability that a data value d appears in source S 1 if d is known to appear in source S 2 ) for use by collector operators, as in =-=[8]-=-. In the extreme case, overlap information can indicate that two sites are mirrors of each other. Finally, the catalog may contain key statistics about the data, such as the cost of accessing each sou...

Citation Context

...ragments after one completes. Fragments unrelated in the partial order may execute in parallel. For example, we may execute one CPU-bound fragment in parallel with other network-bound fragments as in =-=[14]-=-. 3.1.1 Fragments and Operators A fragment consists of a fully pipelined tree of physical operators, and a set of local rules. Each node in the tree is a physical operator specifying: (1) the algebrai...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...ection, join), (2) the chosen physical implementation of the operator (e.g., hash join, double pipelined join), (3) the children of the node, (4) the memory allocated to the operator, as discussed in =-=[5, 18]-=-, and (5) an estimate of result cardinality. 3.1.2 Rules Rules are the key mechanism for implementing several kinds of adaptive behavior in Tukwila: ffl Re-optimization: At the end of a fragment, if t...

Citation Context

...hich to reliably compare candidate query execution plans, it may choose to send only a partial plan to the execution engine, and decide how to proceed after the partial plan has been completed, as in =-=[7]-=-. Alternatively, the optimizer may send a complete plan, but the execution engine may check for conditions that require incremental re-optimization. 1 Tukwila is a scenic city near Seattle in the Nort...

Citation Context

...tive operators: the dynamic collectorsand the double pipelined join operator. 4.1 Dynamic collectors A common task in data integration is to perform a union over a large number of overlapping sources =-=[27, 8]-=-. Common examples of such sources include those providing bibliographic references, movie reviews and product information. In some cases different sites are deliberately created as mirrors. For these ...

Citation Context

...ection, join), (2) the chosen physical implementation of the operator (e.g., hash join, double pipelined join), (3) the children of the node, (4) the memory allocated to the operator, as discussed in =-=[5, 18]-=-, and (5) an estimate of result cardinality. 3.1.2 Rules Rules are the key mechanism for implementing several kinds of adaptive behavior in Tukwila: ffl Re-optimization: At the end of a fragment, if t...

Citation Context

...nd manually combine the data from the different sources. The problem of data integration has received significant attention in the research community as evidenced by numerous research projects (e.g., =-=[10, 20, 25, 17, 9, 3, 6, 1, 25, 21, 4, 13]-=-) and the emergence of several commercial products (e.g., DataJoiner [23] and jango.excite.com). Three main challenges distinguish the design of a data integration system from that of a traditional da...

Citation Context

...n while [15] used the standard Paradise optimizer. Graefe and Ward's choose nodes allow the execution system to choose from a set of precompiled subplans based on runtime variables [12]. Oracle's Rdb =-=[2] used a &q-=-uot;dynamic optimization" strategy to deal with uncertainty by running alternative query plan subtrees in parallel competition. Although our query plans have a different structure, they can expre...