Comparing Click-through Data to Purchase Decisions for Retrieval Evaluation

Publication Type  Conference Paper
Author  Hofmann K., Huurnink B., Bron M., de Rijke M.
Year of Publication  2010
Conference Name  33rd Annual International ACM SIGIR Conference (SIGIR 2010)
Month Published  July
Publisher  ACM
Conference Location  Geneva
Abstract  

Traditional retrieval evaluation uses explicit relevance judgments which are expensive to collect. Relevance assessments inferred from implicit feedback such as click-through data can be collected inexpensively but may be less reliable. We compare assessments derived from click-through data to another source of implicit feedback that we assume to be highly indicative of relevance: purchase decisions. Evaluating retrieval runs based on a log of an audio-visual archive, we find a high agreement between system rankings based on clicks and purchase decisions.

Export  BibTex
Full paper  PDF (376.54 KB)
AttachmentSize
pp100-hofmann.pdf376.54 KB