On Intra-page and Inter-page Semantic Analysis of Web Pages

On Intra-page and Inter-page Semantic Analysis of Web Pages Jun Wang� Jicheng Wang, Gangshan Wu� Hiroshi Tsuda Fujitsu R&D Center Co., Ltd.� Institute of Multimedia Computing�Fujitsu Laboratories, Ltd. Room B1003, Eagle Run Plaza� Nanjing Univ.� 4-1-1 Kami-kodanaka, No.26 XiaoYun Road� Nanjing, CHINA. 210093� Nakahara-Kawasaki, Beijing, CHINA. 100016� {wjc,gswu}@graphics.nju.edu.cn� Kanagawa 211-8588, Japan jwang@frdc.fujitsu.com� htsuda@jp.fujitsu.com Abstract To make real Web information more machine processable, this paper presents a new approach to intra-page and inter-page semantic analysis of Web pages. Our approach consists of Web pages structure analysis and semantic clustering for intra-page semantic analysis, and machine learning based link semantic analysis for inter-page analysis. Based on the automatic repetitive patterns discovery in structure level and clustering in semantic level, we explore the intra-page semantic structure of Web pages and extend the processing unit from the whole page to a finer granularity, i.e., semantic information blocks within pages. After observing the various hyperlinks, we synthesize the Web inter-page semantic and define an information organizing oriented hyperlink semantic category. Considering the presentation of the hyperlink carrier and intra-page semantic structure, we propose corresponding feature selection and quantification methods, and then exploit the C4.5 decision-tree method to classify hyperlink semantic type and analyze the inter-page semantic structure. In our experiments, the results suggest that our approch is feasible for machine processing. I� Introduction In the recent years, the content and structure of Web pages become much more complex for business purposes with easy access and user-friendly. However, it brings a big challenge for automatic Web information processing system since most of the Web's content today is designed for humans to read, not for machines to manipulate meaningfully (Tim Berners-Lee 2001), and many problems occur in current Web machine processing due to lack of semantic analysis. According to these problems in practical application, two opposite kinds of semantic information of Web pages are analyzed in this paper. Web pages often represent a collection of different topics and functionalities that are loosely knit together to form a single entity. People can easily identify the information areas with different meaning and function in a Web page, but it's very difficult for automatic processing systems since HTML is designed for presentation instead of for content description. Now most of current Web IR (Information Retrieval) and DM (Data Mining) systems regard a Web page as an atomic unit and do not give due consideration to the intra-page semantic structure of a Web page, and in this case many problems occur. (Xiaoli Li 2002) illustrate the precision decline occurred when the query words scatter at different semantic information areas in a Web page. The use of templates has grown with the recent developments in Web site engineering, because the template is very valuable from a user's point of view since it provides context for browsing, however, (Ziv Bar-Yossef 2002 and Soumen Chakrabarti 2001) reveal that the template of Web pages skew ranking, IR and DM algorithms and consequently, reduce precision, and (Shian-Hua Lin 2002) describe the intra-page redundancy caused by the …

Content truncated to 500 words