DMCA
Efficient Update of Indexes for Dynamically Changing Web Documents (2007)
Cached
Download Links
| Citations: | 4 - 0 self |
BibTeX
@MISC{Lim07efficientupdate,
author = {Lipyeow Lim and Min Wang and Sriram Padmanabhan and Jeffrey Scott Vitter and Ramesh Agarwal},
title = { Efficient Update of Indexes for Dynamically Changing Web Documents},
year = {2007}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Recent work on incremental crawling has enabled the indexed document collection of a search engine to be more synchronized with the changing World Wide Web. However, this synchronized collection is not immediately searchable, because the keyword index is rebuilt from scratch less frequently than the collection can be refreshed. An inverted index is usually used to index documents crawled from the web. Complete index rebuild at high frequency is expensive. Previous work on incremental inverted index updates have been restricted to adding and removing documents. Updating the inverted index for previously indexed documents that have changed has not been addressed. In this paper, we propose an efficient method to update the inverted index for previously indexed documents whose contents have changed. Our method uses the idea of landmarks together with the diff algorithm to significantly reduce the number of postings in the inverted index that need to be
Keyphrases
inverted index s11280-006-0009-2 efficient update dynamically changing web document indexed document diff algorithm indexed document collection incremental crawling previous work world wide web index document high frequency incremental inverted index update efficient method abstract recent work search engine keyword index complete index rebuild
