We survey the major techniques for information retrieval. In the rst part, weprovide an overview of the traditional ones (full text scanning, inversion, signature les and clustering). In the second part we discuss attempts to include semantic information (natural language processing, latent semantic indexing and neural networks).

Abstract ⎯ This paper describes the development and application of visualisation techniques for users to access and explore information in a digital library effectively and intuitively. Salient semantic structures and citation patterns are extracted from several collections of documents, including the ACM SIGCHI conference proceedings (1995 ⎯ 1997) and ACM Hypertext conference proceedings (1987 ⎯ 1998), using Latent Semantic Indexing and Pathfinder Network Scaling. The unique spatial metaphor leads to a natural combination of search and navigation within the same semantic space in a 3-dimensional virtual world. Author co-citation patterns are visualised through a number of author co-citation maps in attempts to reveal the structure of the field of hypertext, including an overall co-citation map of 367 authors and three periodical maps. These maps highlight predominant research areas in the field. This approach provides a means of transcending the boundaries of collections of documents and visualising more profound patterns in terms of semantic structures and co-citation networks. © 1999 Elsevier Science Ltd. All rights reserved. 1.

This paper presents a framework for knowledge discovery and concept exploration. In order to enhance the concept exploration capability of knowledge-based systems and to alleviate the limitations of the manual browsing approach, we have developed two spreading activation-based algo-rithms for concept exploration in large, heterogeneous net-works of concepts (e.g., multiple thesauri). One algorithm, which is based on the symbolic Al paradigm, performs a conventional branch-and-bound search on a semantic net representation to identify other highly relevant concepts (a serial, optimal search process). The second algorithm, which is based on the neural network approach, executes the Hopfield net parallel relaxation and convergence pro-cess to identify “convergent ” concepts for some initial queries (a parallel, heuristic search process). Both algo-rithms can be adopted for automatic, multiple-thesauri consultation. We tested these two algorithms on a large text-based knowledge network of about 13,000 nodes (terms) and 80,000 directed links in the area of computing technologies. This knowledge network was created from two external thesauri and one automatically generated thesaurus. We conducted experiments to compare the be-haviors and performances of the two algorithms with the hypertext-like browsing process. Our experiment revealed that manual browsing achieved higher-term recall but lower-term precision in comparison to the algorithmic sys-tems. However, it was also a much more laborious and cog-nitively demanding process. In document retrieval, there were no statistically significant differences in document re-call and precision between the algorithms and the manual browsing process. In light of the effort required by the man-ual browsing process, our proposed algorithmic approach presents a viable option for efficiently traversing large-scale, multiple thesauri (knowledge network). 1

Browsing accounts for much of people's interaction with digital libraries, but it is poorly supported by standard search engines. Conventional systems often operate at the wrong level, indexing words when people think in terms of topics, and returning documents when people want a broader view. As a result, users cannot easily determine what is in a collection, how well a particular topic is covered, or what kinds of queries will provide useful results. We have built

Tagging-based systems enable users to categorize web resources by means of tags (freely chosen keywords), in order to refinding these resources later. Tagging is implicitly also a social indexing process, since users share their tags and resources, constructing a social tag index, so-called folksonomy. At the same time of tagging-based system, has been popularised an interface model for visual information retrieval known as Tag-Cloud. In this model, the most frequently used tags are displayed in alphabetical order. This paper presents a novel approach to Tag-Cloud’s tags selection, and proposes the use of clustering algorithms for visual layout, with the aim of improve browsing experience. The results suggest that presented approach reduces the semantic density of tag set, and improves the visual consistency of Tag-Cloud layout.

We present entirely automatic methods for gathering documents for a hypertext, linking the set, and annotating those connections with a description of the type (i.e., nature) of the link. Document linking is based upon high-quality information retrieval techniques developed using the Smart system. We apply an approach inspired by relationship visualization techniques and by graph simplification, to show how to identify automatically tangential, revision, summary, expansion, comparison, contrast, equivalence, and aggregate links. KEYWORDS: Link generation, Link types, Information retrieval In a 1989 paper outlining some design issues for hypertexts, Robert Glushko's opinion of the future of automatically generating content links between documents was pessimistic: When we first began working in hypertext several years ago, we expected that it would soon be possible to extract these implicit links automatically with natural language processing or clever indexing techniques. . . , bu...

Queries to text collections are resolved by ranking the documents in the collection and returning the highest-scoring documents to the user. An alternative retrieval method is to rank passages, that is, short fragments of documents, a strategy that can improve effectiveness and identify relevant material in documents that are too large for users to consider as a whole. However, ranking of passages can considerably increase retrieval costs. In this paper we explore alternative query evaluation techniques, and develop new techniques for evaluating queries on passages. We show experimentally that, appropriately implemented, effective passage retrieval is practical in limited memory on a desktop machine. Compared to passage ranking with adaptations of current document ranking algorithms, our new "DO-TOS" passage ranking algorithm requires only a fraction of the resources, at the cost of a small loss of effectiveness.

this article we describe how developers can incorporate hypertext functionality into computational applications. What benefit do users gain from hypertext support in computational applications? Managing the myriad of interrelationships in a computational application's knowledge (data and calculated information) is difficult for a user. It can be improved by streamlining access and increasing user comprehension through interface enhancements (e.g., visualization). Augmenting an application with hypertext support results in new ways to view and manage the application's knowledge, by navigating among items of interest and annotating with comments and relationships (links). Unfortunately, many computational system developers view hypertext only in terms of accessing and managing documents (or smaller units of static information) . Such display-oriented behavior characterizes the majority of hypertext systems. SEPIA (see Thring et al. and Streitz in this issue), KMS [1], Aquanet and VIKI

Documents stored in a database system can have complex internal structure described by languages such as SGML. Howtotake advantage of this structure presents challenges for database system implementors. We classify the types of queries that need to be supported by SGML-conformant database systems. We then describe several data models that have been proposed for representing documents in a database system and discuss the support these models provide for SGML. Finally we consider query evaluation. 1

This paper describes a generic approach to structuring and visualising a hypertext-based information space on the WWW. This approach, called Generalised Similarity Analysis (GSA), provides a unifying framework for extracting structural patterns from a range of proximity data concerning three fundamental relationships in hypertext, namely, hypertext linkage, content similarity and browsing patterns. GSA emphasizes the integral role of users' interests in dynamically structuring the underlying information space. Pathfinder networks are used as a natural vehicle for structuring and visualising the rich structure of an information space by highlighting salient relationships in proximity data. In this paper, we use the GSA framework in the study of hypertext documents automatically retrieved over the Internet, including a number of departmental WWW sites and conference proceedings on the WWW. We show that GSA has several distinct features for structuring and visualising hyp...