STELLA - Infrastructures for Living Labs

Profile & Description

The STELLA project aims to create an evaluation infrastructure that allows to evaluate search and recommendation services within productive web-based search systems with real users. STELLA provides an integrated e-Research environment that allows researchers in the field of information retrieval and recommendation services to conduct studies with real users in real environments. The experimental set-ups differ considerably from classical TREC studies, which can only be carried out offline, or also from user studies, which only allow laboratory experiments, and thus enable researchers to use an evaluation method that was previously reserved only for industrial research or the operators of large online platforms.

ProjectSTELLA - Infrastructures for Living Labs

Duration
2018 - 2021
Funded by
Further Reading: General Publications

If you are interested in background knowledge and would like to learn more about the STELLA - Infrastructures for Living Labs project and its progression feel free to study the following additional information.

General
Initial Press Release
Read the initial press release published by TH Köln covering the STELLA project.
Article on Hochschulnetzwerk NRW
The HN NRW introduces STELLA.
Publications

2019

Dockerizing Automatic Routing Runs for The Open-Source IR Replicability Challenge (OSIRRC 2019).
In: Proceedings of the The Open-Source IR Replicability Challenge (OSIRRC) @ SIGIR. 2019. 2019.
Timo Breuer and Philipp Schaer.
[BibTeX] 
Replicability and Reproducibility of Automatic Routing Runs.
In: Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, series CEUR Workshop Proceedings. CEUR-WS.org, 2019. (accepted)
Timo Breuer and Philipp Schaer.
[BibTeX] 
STELLA: Towards a Framework for the Reproducibility of Online Search Experiments.
In: Proceedings of the The Open-Source IR Replicability Challenge (OSIRRC) @ SIGIR. 2019.
Timo Breuer, Philipp Schaer, Narges Tavalkolpoursaleh, Johann Schaible, Benjamin Wolff and Bernd Müller.
[BibTeX]