Effect of Data Skewness in Parallel Mining of Association Rules (1998)
| Venue: | In Pacific-Asia Conference on Knowledge Discovery and Data Mining |
| Citations: | 19 - 1 self |
BibTeX
@INPROCEEDINGS{Cheung98effectof,
author = {David W. Cheung and Yongqiao Xiao},
title = {Effect of Data Skewness in Parallel Mining of Association Rules},
booktitle = {In Pacific-Asia Conference on Knowledge Discovery and Data Mining},
year = {1998},
pages = {48--60},
publisher = {SpringerVerlag}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
An efficient parallel algorithm FPM (Fast Parallel Mining) for mining association rules on a shared-nothing parallel system has been proposed. It adopts the count distribution approach and has incorporated two powerful candidate pruning techniques, i.e., distributed pruning and global pruning. It has a simple communication scheme which performs only one round of message exchange in each iteration. We found that the two pruning techniques are very sensitive to data skewness, which describes the degree of non-uniformity of the itemset distribution among the database partitions. Distributed pruning is very effective when data skewness is high. Global pruning is more effective than distributed pruning even for the mild data skewness case. We have implemented the algorithm on an IBM SP2 parallel machine. The performance studies confirm our observation on the relationship between the effectiveness of the two pruning techniques and data skewness. It has also shown that FPM outp...
