Competitive Intelligence (CI) aims to monitor a firm's external environment for information relevant to its decision-making process. As an excellent information source, the Internet provides significant opportunities for CI professionals as well as the problem of information overload. Internet search engines have been widely used to facilitate information search on the Internet. However, many problems hinder their effective use in CI research. In this paper, we introduce the Competitive Intelligence Spider, or CI Spider, designed to address some of the problems associated with using Internet search engines in the context of competitive intelligence. CI Spider performs real-time collection of Web pages from sites specified by the user and applies indexing and categorization analysis on the documents collected, thus providing the user with an up-to-date, comprehensive view of the Web sites of user interest. In this paper, we report on the design of the CI Spider system and on a user study of CI Spider, which compares CI Spider with two other alternative focused information gathering methods: Lycos search constrained by Internet domain, and manual within-site browsing and searching. Our study indicates that CI Spider has better precision and recall rate than Lycos. CI Spider also outperforms both Lycos and within-site browsing and searching with respect to ease of use. We conclude that there exists strong evidence in support of the potentially significant value of applying the CI Spider approach in CI applications. D 2002 Elsevier Science B.V. All rights reserved.

Medical professionals and researchers need information from reputable sources to accomplish their work. Unfortunately, the Web has a large number of documents that are irrelevant to their work, even those documents that purport to be "medically-related." This paper describes an architecture designed to integrate advanced searching and indexing algorithms, an automatic thesaurus, or "concept space", and Kohonen-based Self-Organizing Map (SAM) technologies to provide searchers with fine-grained results. Initial results indicate that these systems provide complementary retrieval functionalities. HelpfulMed allows users to search not only Web pages and other online databases, but also allows them to build searches through the use of an automatic thesaurus and browse a graphical display of medical-related topics. Evaluation results

Abstract. Information extraction on the Web permits users to retrieve specific information related to the query especially on the name of a person or organization. As name is non-unique, the same name may be mapped to multiple entities. The aim of this paper is to describe an algorithm to cluster the Web pages returned by the search engine so that pages belonging to different entities are clustered into different group. The algorithm uses named entities as the features to divide the document set into direct or indirect pages. It then uses distinct direct pages as seeds of clusters to group indirect pages into different clusters. The algorithm has been found to be effective for Web-based applications. 1

Web 2.0 has great potential to serve as a public sphere (Habermas, 1974; Habermas, 1989) – a distributed arena of voices where all who want to do so can participate. A well-functioning public sphere is important for pluralistic decision-making at many levels, ranging from small organizations to society at large. In this paper, we analyze the capability of the blogosphere in its current form to support such a role. This analysis leads to the identification of the principal issues that prevent the blogosphere from realizing its full potential as a public sphere. Most significantly, we propose that the sheer volume of content overwhelms blog readers, forcing them to restrict themselves to only a small subset of valuable content. This ultimately reduces their level of informedness. Based on past research on managing discourse, we propose four design artifacts that would alleviate these issues: a communal repository, textual clustering, visual cues, and a participation facility for blog users. We present a prototype system, called FeedWiz, which implements several of these design artifacts. Based on this initial design, we formulate a research agenda for the creation of new tools that effectively harness the potential of the growing body of user-generated content in the blogosphere and beyond.