Abstract: Partial coordination is a new method for cataloging documents for subject access. It is especially designed to enhance the precision of document searches in online environments. This paper reports a preliminary evaluation of partial coordination which shows promising results compared with full text retrieval. We also report the difficulties in empirically evaluating the effectiveness of automatic full-text retrieval in contrast to mixed methods such as partial coordination which combine human cataloging with computerized retrieval. Based on our study we propose research in this area will substantially benefit from a common framework for failure analysis and a common data set. This will allow information retrieval researchers adapting "library style " cataloging to large electronic document collections, as well as those developing automated or mixed methods, to directly compare their proposals for indexing and retrieval. This paper concludes by suggesting guidelines for constructing such a testbed. This work was supported by a grant from Disclosure Inc. Any opinions, findings, conclusions, recommendations, errors expressed in this material are those of the authors ' alone and do not reflect the views of Disclosure Inc. The authors also gratefully acknowledge the help of Kay Teel, Head Cataloger, Ingalls