Web data extraction based on partial tree alignment

DMCA

Web data extraction based on partial tree alignment (2005)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Yanhong Zhai

Citations:

141 - 5 self

BibTeX

@INPROCEEDINGS{Zhai05webdata,
    author = {Yanhong Zhai},
    title = {Web data extraction based on partial tree alignment},
    booktitle = {},
    year = {2005},
    pages = {76--85}
}

OpenURL

Abstract

This paper studies the problem of extracting data from a Web page that contains several structured data records. The objective is to segment these data records, extract data items/fields from them and put the data in a database table. This problem has been studied by several researchers. However, existing methods still have some serious limitations. The first class of methods is based on machine learning, which requires human labeling of many examples from each Web site that one is interested in extracting data from. The process is time consuming due to the large number of sites and pages on the Web. The second class of algorithms is based on automatic pattern discovery. These methods are either inaccurate or make many assumptions. This paper proposes a new method to perform the task automatically. It consists of two steps, (1) identifying individual data records in a page, and (2) aligning and extracting data items from the identified data records. For step 1, we propose a method based on visual information to segment data records, which is more accurate than existing methods. For step 2, we propose a novel partial alignment technique based on tree matching. Partial alignment means that we align only those data fields in a pair of data records that can be aligned (or matched) with certainty, and make no commitment on the rest of the data fields. This approach enables very accurate alignment of multiple data records. Experimental results using a large number of Web pages from diverse domains show that the proposed two-step technique is able to segment data records, align and extract data from them very accurately.

Keyphrases

data record partial tree alignment web data extraction web page large number segment data record data field experimental result database table individual data record extract data new method serious limitation web site two-step technique first class many assumption second class diverse domain automatic pattern discovery visual information paper study several researcher tree matching novel partial alignment technique accurate alignment extract data item field multiple data record partial alignment machine learning data item many example

DMCA