|
|
|
|
Methods for Evaluating Interactive Information Retrieval Systems with Users
Foundations and Trends® in
Information Retrieval
Volume 3 Issue 1—2
DOI: 10.1561/1500000012
Methods for Evaluating Interactive Information Retrieval Systems with Users
Diane Kelly
School of Information and Library Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, dianek@email.unc.edu
SUGGESTED CITATION:
Diane
Kelly
(2009)
"Methods for Evaluating Interactive Information Retrieval Systems with Users",
Foundations and Trends® in Information Retrieval: Vol. 3: No 1—2, pp 1-224.
http:/dx.doi.org/10.1561/1500000012
Abstract
This paper provides overview and instruction regarding the evaluation of interactive information retrieval systems with users.
The primary goal of this article is to catalog and compile material related to this topic into a single source. This article
(1) provides historical background on the development of user-centered approaches to the evaluation of interactive information
retrieval systems; (2) describes the major components of interactive information retrieval system evaluation; (3) describes
different experimental designs and sampling strategies; (4) presents core instruments and data collection techniques and measures;
(5) explains basic data analysis techniques; and (4) reviews and discusses previous studies. This article also discusses validity
and reliability issues with respect to both measures and methods, presents background information on research ethics and discusses
some ethical issues which are specific to studies of interactive information retrieval (IIR). Finally, this article concludes
with a discussion of outstanding challenges and future research directions.
|
|
|
|
|
|
|
|
|