"... In PAGE 7: ... The behavior indicates that the ShopBots perform (almost) exclusively searches, and that usually they have long sessions. Table6 shows the averaged CBMG for the various crawlers. The rst observation is the much broader pool of states that are visited.... ..."

"... In PAGE 3: ...We performed a number of experiments, the results of which appear in Table1 . The first two rows show results for Version 1 with two sets of start points in the search engine hierarchies, general points relating to computer hardware retailers and more narrow points relating to notebooks and laptops.... ..."

"... In PAGE 21: ...Table 4: Sources of information used to initialize the crawler To configure the crawler, we required a target set of sites as well as some information to initialize the LVS table. Table4 lists the online sources we used to generate some of the basic LVS entries required by the crawler. These entries included partial lists of names of semiconductor manufacturing companies as well as list of sub-sectors (or areas) within the semiconductor industry.... In PAGE 21: ... These entries included partial lists of names of semiconductor manufacturing companies as well as list of sub-sectors (or areas) within the semiconductor industry. The first two sources listed in Table4 were (manually) used only once, to extract information for explicit initialization. The remaining two sources were wrapped by custom wrappers to interface with the LVS manager and automatically provide values at run-time.... ..."

"... In PAGE 28: ... 7.2 Performance comparison Table9 presents a performance comparison between the previously analyzed crawlers extracted from published results. This comparison must be taken with a grain of salt because the experiments were run using different setups and in different periods of time.... In PAGE 29: ... The speed of the connection to the Internet must be considered to analyze crawling performance. The 3rd, 4th and 5th columns of Table9 show that the most performant crawlers used the fastest connections to the Internet. So, this might be also a reason why VN presented a lower download rate than the most performant crawlers.... In PAGE 29: ... In the %dups. column of Table9 , we present the percentage of duplicates harvested by the crawlers. The results show that our efforts to minimize the download of duplicates, saving on bandwidth and storage space, yield good results in practice.... In PAGE 29: ... A crawler should also minimize the number of visits to URLs that do not reference a downloadable content. The downloads/URLs column of Table9 presents the ratio between the number of downloads and the URLs visited. VN was configured as a focused crawler of the Portuguese web and discarded contents considered irrelevant.... ..."

"... In PAGE 2: ... To construct the blogspace, we can repeat the basic algorithm for each distinct weblogs appear in the target end of the edges discovered so far. Table1 gives the crawling algorithm. In implementation, step 1 and step 2 are executed in pipeline style for efficiency and a variable depth is used to control the size of the blogspace.... ..."