What is the Difference? A Cognitive Dissimilarity Measure for Information Retrieval Result Sets

Keßler Carsten

Zusammenfassung

Result rankings from context-aware information retrieval are inherently dy- namic, as the same query can lead to signi cantly di erent outcomes in di erent con- texts. For example, the search term Digital Camera will lead to di erent { albeit poten- tially overlapping { results in the contexts Customer Reviews and Shops, respectively. The comparison of such result rankings can provide useful insights into the e ects of context changes on the information retrieval results. In particular, the impact of single aspects of the context in complex applications can be analyzed to identify the most (and least) in uential context parameters. While a multitude of methods exists for assessing the relevance of a result ranking with respect to a given query, the question how di erent two result rankings are from a user's point of view has not been tack- led so far. This paper introduces DIR, a cognitively plausible dissimilarity measure for information retrieval result sets that is based solely on the results and thus appli- cable independently of the retrieval method. Unlike statistical correlation measures, this dissimilarity measure re ects how human users quantify the changes in informa- tion retrieval result rankings. The DIR measure supports cognitive engineering tasks for information retrieval, such as work ow and interface design: Using the measure, developers can identify which aspects of context heavily in uence the outcome of the retrieval task and should therefore be in the focus of the user's interaction with the sys- tem. The cognitive plausibility of DIR has been evaluated in two human participants tests, which demonstrate a strong correlation with user judgments.