From highly relevant to not relevant: examining different regions of relevance
User relevance judgments are central to both the systems and user-oriented approaches to information retrieval (IR) systems research and development. User-oriented relevance research has also operated on two largely unconnected tracks. First, a relevance level track that examines users' criteria for relevance judgments. Second, a regions of relevance track that examines the measurement of users' relevance judgments. Users judgments and criteria for highly relevant items have been central issues for much of the relevance research. Findings are presented from four separate studies of relevance judgments by 55 users, conducting their initial online search on a particular information problem. In three studies, the number of items judged "partially" relevant (on a scale of relevant, partially relevant or not relevant) was positively correlated with different aspects of changes in users', including: (1) information problem definition, (2) search intermediaries' perceptions that a user's question and information problem has changed during the mediated search interaction, (3) personal knowledge due to the search interaction, and (4) criteria for making relevance judgments. Users with high knowledge and topic levels were more likely to judge items as highly relevant. Differences between users' criteria for highly, partially and non-relevant items are also identified. Findings suggest the need to expand the framework for relevance research and further identify the characteristics of the middle region of relevance or partial relevance as: (1) partially relevant items may play an important role in the early stages of a user's information seeking process over time for a particular information problem and (2) a relationship may exist between partially relevant items retrieved and changes in users' information problems during an information seeking process. Results also suggest that partially relevant items may be useful at the early stages of users' information seeking processes. We propose a useful concept of relevance as a relationship and an effect on the movement of a user through the iterative stages of their information seeking process. Users' relevance judgments can also be plotted on a three-dimensional spatial model of relevance level, region and time. Implications for the development of IR systems, searching practice and relevance research are also discussed.
Impact and interest:
Citation counts are sourced monthly from and citation databases.
These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.
Citations counts from theindexing service can be viewed at the linked Google Scholar™ search.
Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.
|Item Type:||Journal Article|
|Keywords:||information retrieval, user, oriented, relevance, searching behaviour|
|Subjects:||Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > LIBRARY AND INFORMATION STUDIES (080700) > Information Retrieval and Web Search (080704)|
|Divisions:||Past > QUT Faculties & Divisions > Faculty of Science and Technology|
|Copyright Owner:||Copyright 1998 Elsevier|
|Copyright Statement:||Reproduced in accordance with the copyright policy of the publisher.|
|Deposited On:||31 Oct 2006|
|Last Modified:||09 Jun 2010 12:34|
Repository Staff Only: item control page