Abstract

A robot is deployed by a Web search engine in order to maintain the currency of its data base of Web pages. This paper studies robot scheduling policies that minimize the fractions r i of time pages spend out-of-date, assuming independent Poisson page-change processes, and a general distribution for the page access time X . We show that, if X is decreased in the increasing-convex ordering sense, then r i is decreased for all i under any scheduling policy, and that, in order to minimize expected total obsolescence time of any page, the accesses to that page should be as evenly spaced in time as possible. We then investigate the problem of scheduling to minimize the cost function P c i r i ; where the c i are given weights proportional to the page-change rates ¯ i . We give a tight bound on the performance of such a policy and prove that the optimal frequency at which the robot should access page i is proportional to ln(h i ) \Gamma1 , where h i := Ee \Gamma¯ i X : Note that this...

Citations