An Infrastructure for Creating Web Automation Applications

An Infrastructure for Creating Web Automation Applications Wen Heng Yen1, Hao-Ren Ke2, and Wei-Pang Yang3 1Degree Program of Electrical Engineering and Computer Science, National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu, TAIWAN 30050, R.O.C. helio@lib.nctu.edu.tw 2University Library, National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu, TAIWAN 30050, R.O.C. claven@lib.nctu.edu.tw 3Dept. of Computer & Information Science, National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu, TAIWAN 30050, R.O.C. Dept. of Information Management, National Dong Hwa University, 1, Sec. 2, Da Hsueh Rd., Shou-Feng, Hualien, TAIWAN 97401, R.O.C. wpyang@cis.nctu.edu.tw Abstract. With the growth of the World Wide Web (WWW), many people nowadays spend a lot of time performing various tasks with browsers, most of these tasks repetitive and tedious. Many applications are created to reorganize and simplify the usage of Web resources, which are called Web automation applications. This paper proposes an infrastructure to create Web automation applications, the WIS (Web Integration Solution) system. WIS integrates a diversity of tools and technologies to provide a software environment for developing Web automation services. Developers can use this tool to create various Web automation applications. We show the versatility of WIS by implementing three exemplary services. The first is a metasearcher system that provides a single search interface for several indexing and abstract databases. The second is a cataloguing tool to simplify the task of retrieving bibliographic data from the Web; this tool is also integrated with a book-recommendation system for libraries. The third is a Web site checking system that periodically checks if some critical Web sites are working correctly, no matter how complex the check procedure is. Keywords: User Surrogates; Web Agent; Web Automation 1 Introduction The World Wide Web (WWW) provides a vast amount of information and plentiful services, and continues to grow at a staggering rate. Because of its explosive growth, people are spending more and more time on the Web, performing various tasks, many of these tasks repetitive and tedious. The following is some typical scenarios: � I daily check several sources, including online news, Usenet newsgroups, and bulletin board systems (BBS) for some specific topics. � Some WWW sites recommend several books to me, but I don't know where the best places to retrieve them are. � I want to buy a second-hand notebook that fits some requirements. There are several places where I could find them and I don't want to miss any good buy. From the above scenarios, it can be concluded that: (1) many tasks are repetitive; (2) a user may not know where to start surfing in the Internet; (3) many tasks involve several resources in different sites. What we need in these scenarios are agents, or Web automation applications, which act as our surrogates to perform these laborious tasks. A good surrogate should have the following benefits: (1) let nontechnical users be able to exploit the information available on the Web without being overwhelmed by technical detail; (2) free users from repetitive browsing tasks; (3) reformat and recombine the information from …

Content truncated to 500 words