RESIRE - Reproducibility and simulation of interactive retrieval experiments

Profile & Description

Although reproducibility of experiments is a topic with increasing importance in information retrieval (IR), there has been hardly any work on reproducing interactive IR experiments. The main idea of our proposed project is to describe actual users by quantitative modeling of their behavior and their content-related decisions during retrieval sessions. These model parameters can then form the basis for comparing two interactive IR experiments with regard to reproducibility, and can also be used for simulating interactive IR, e. g. for evaluating new systems or system variants without (or before) carrying out actual user experiments. As part of the proposed project, we will define a new framework model for reproducibility of interactive IR experiments, develop new metrics for reproducibility and session quality, design new simulation methods integrating behavior and content-related decisions, develop an open source framework for reproduction and simulation of interactive IR, and perform extensive experiments using existing IR log data as well as observation data from new user experiments.

Funding Agency
DFG - Deutsche Forschungsgemeinschaft
Partner Institution
University of Duisburg-Essen
People Involved
Prof. Dr. Philipp Schaer (Technische Hochschule Köln)
Dr. Timo Breuer (Technische Hochschule Köln)

ProjectRESIRE - Reproducibility and simulation of interactive retrieval experiments

2023 - 2025
Funded by



Simulating Users in Interactive Web Table Retrieval.
In: CIKM. ACM, 2023. to appear
Björn Engelmann, Timo Breuer and Philipp Schaer.
An in-depth investigation on the behavior of measures to quantify reproducibility.
Information Processing and Management, 60(3):103332, 2023.
Maria Maistro, Timo Breuer, Philipp Schaer and Nicola Ferro.
[doi] [pdf]  [BibTeX]