UbiCrawler: a scalable fully distributed web crawler (2003)

by Paolo Boldi, Bruno Codenotti, Massimo Santini, Sebastiano Vigna
Venue:Software: Practice & Experience