List of Publications

2022

irmetadata: An Extensible Metadata Schema for IR Experiments.
In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3078-3089. ACM, 2022.
Timo Breuer, Jüri Keller and Philipp Schaer.
[doi] [pdf]  [BibTeX] 
Relevance assessments, bibliometrics, and altmetrics: a quantitative study on PubMed and arXiv.
Scientometrics, 127(5):2455-2478, 2022.
Timo Breuer, Philipp Schaer and Dirk Tunger.
[doi]  [BibTeX] 
Validating Simulations of User Query Variants.
In: M. Hagen, S. Verberne, C. Macdonald, C. Seifert, K. Balog, K. Nørvåg and V. Setty, editors, Advances in Information Retrieval - 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10-14, 2022, Proceedings, Part I, volume 13185, series Lecture Notes in Computer Science, pages 80-94. Springer, 2022.
Timo Breuer, Norbert Fuhr and Philipp Schaer.
[doi]  [BibTeX] 
Auditing Search Query Suggestion Bias Through Recursive Algorithm Interrogation.
In: 14th ACM Web Science Conference 2022, pages 219–227. ACM, 2022.
Fabian Haak and Philipp Schaer.
[doi]  [BibTeX] 
Evaluating Research Dataset Recommendations in a Living Lab.
In: A. Barrón-Cedeño, G. Da San Martino, M. Degli Esposti, F. Sebastiani, C. Macdonald, G. Pasi, A. Hanbury, M. Potthast, G. Faggioli and N. Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 135-148. Springer International Publishing, Cham, 2022.
Jüri Keller and Leon Paul Mondrian Munz.
[doi]  [abstract]  [BibTeX] 

2021

A Living Lab Architecture for Reproducible Shared Task Experimentation.
In: Information between Data and Knowledge, pages 348-362. Werner Hülsbusch, Glückstadt, 2021. Session 6: Emerging Technologies
Timo Breuer and Philipp Schaer.
[pdf]  [abstract]  [BibTeX] 
Evaluating Elements of Web-based Data Enrichment for Pseudo-Relevance Feedback Retrieval.
In: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Twelfth International Conference of the CLEF Association (CLEF 2021), series Lecture Notes in Computer Science. Springer Nature, 2021.
Timo Breuer, Melanie Pest and Philipp Schaer.
[BibTeX] 
reproeval: A Python Interface to Reproducibility Measures of System-Oriented IR Experiments.
In: D. Hiemstra, M. Moens, J. Mothe, R. Perego, M. Potthast and F. Sebastiani, editors, Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part II, volume 12657, series Lecture Notes in Computer Science, pages 481-486. Springer, 2021.
Timo Breuer, Nicola Ferro, Maria Maistro and Philipp Schaer.
[doi]  [BibTeX] 
IRCologne at TREC 2021 News Track - Relation-based re-ranking for background linking.
In: TREC. National Institute of Standards and Technology (NIST), 2021.
Björn Engelmann and Philipp Schaer.
[BibTeX] 
Design and Development of an Emoji Sentiment Lexicon.
In: Information between Data and Knowledge, pages 432-438. Werner Hülsbusch, Glückstadt, 2021. Gerhard Lustig Award Papers
Fabian Haak.
[pdf]  [abstract]  [BibTeX] 
Emojis in Lexicon-Based Sentiment Analysis: Creating Emoji Sentiment Lexicons from Unlabeled Corpora.
In: S. Thomas, F. Michael and O. Sandra, editors, Proceedings of the LWDA 2021 Workshops: FGWM, KDML, FGWI-BIA, and FGIR, pages 279-286. CEUR-WS, 2021.
Fabian Haak.
[doi] [pdf]  [BibTeX] 
IRCologne at GermEval 2021: Toxicity Classification.
In: Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments, pages 47-53. Association for Computational Linguistics, Duesseldorf, Germany, 2021.
Fabian Haak and Björn Engelmann.
[doi]  [abstract]  [BibTeX] 
Perception-Aware Bias Detection for Query Suggestions.
In: L. Boratto, S. Faralli, M. Marras and G. Stilo, editors, Advances in Bias and Fairness in Information Retrieval - Second International Workshop on Algorithmic Bias in Search and Recommendation, BIAS 2021, Lucca, Italy, April 1, 2021, Proceedings, volume 1418, series Communications in Computer and Information Science. Springer Nature, Switzerland, 2021.
Fabian Haak and Philipp Schaer.
[pdf]  [BibTeX] 
Living Lab Evaluation for Life and Social Sciences Search Platforms - LiLAS at CLEF 2021.
In: D. Hiemstra, M. Moens, J. Mothe, R. Perego, M. Potthast and F. Sebastiani, editors, Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part II, volume 12657, series Lecture Notes in Computer Science, pages 657-664. Springer, 2021.
Philipp Schaer, Johann Schaible and Leyla Jael Garca Castro.
[doi]  [BibTeX] 
Overview of LiLAS 2021 - Living Labs for Academic Search.
In: K. S. Candan, B. Ionescu, L. Goeuriot, B. Larsen, H. Müller, A. Joly, M. Maistro, F. Piroi, G. Faggioli and N. Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Twelfth International Conference of the CLEF Association (CLEF 2021), volume 12880, series Lecture Notes in Computer Science. 2021.
Philipp Schaer, Timo Breuer, Leyla Jael Castro, Benjamin Wolff, Johann Schaible and Narges Tavakolpoursaleh.
[BibTeX] 
Overview of LiLAS 2021 - Living Labs for Academic Search (Extended Overview).
In: G. Faggioli, N. Ferro, A. Joly, M. Maistro and F. Piroi, editors, Working Notes of CLEF 2021 - Conference and Labs of the Evaluation Forum, series CEUR Workshop Proceedings. 2021.
Philipp Schaer, Timo Breuer, Leyla Jael Castro, Benjamin Wolff, Johann Schaible and Narges Tavakolpoursaleh.
[BibTeX] 

2020

How to Measure the Reproducibility of System-oriented IR Experiments..
In: J. Huang, Y. Chang, X. Cheng, J. Kamps, V. Murdock, J.-R. Wen and Y. Liu, editors, SIGIR, pages 349-358. ACM, 2020.
Timo Breuer, Nicola Ferro, Norbert Fuhr, Maria Maistro, Tetsuya Sakai, Philipp Schaer and Ian Soboroff.
[doi] [pdf]  [BibTeX] 
Relations Between Relevance Assessments, Bibliometrics and Altmetrics.
In: Proceedings of the 10th International Workshop on Bibliometric-enhanced Information Retrieval co-located with 42nd European Conference on Information Retrieval, BIR@ECIR 2020, Lisbon, Portugal, April 14th, 2020 [online only], pages 101-112. 2020.
Timo Breuer, Philipp Schaer and Dirk Tunger.
[doi] [pdf]  [BibTeX] 
Reproducible Online Search Experiments.
In: Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II, pages 597-601. 2020.
Timo Breuer.
[doi] [pdf]  [BibTeX] 
Conference Indexing in Digital Libraries: A Ranking Model and Case Study on dblp.
In: Proceedings of the 10th International Workshop on Bibliometric-enhanced Information Retrieval co-located with 42nd European Conference on Information Retrieval, BIR@ECIR 2020, Lisbon, Portugal, April 14th, 2020 [online only], pages 30-41. 2020.
Christopher Michels, Mandy Neumann, Philipp Schaer and Ralf Schenkel.
[doi] [pdf]  [BibTeX] 
Editorial.
Datenbank-Spektrum, 20(1):1-3, 2020.
Philipp Schaer, Klaus Berberich and Theo Härder.
[doi]  [BibTeX] 
Licht und Schatten bei Online-Experimenten – Living Labs aus Sicht eines Forschenden.
In: C. Kaminsky, U. Seelmeyer, S. Siebert and P. Werner, editors, Digitale Technologien zwischen Lenkung und Selbstermächtigung. Beltz Juventa, Weinheim Basel, 2020.
Philipp Schaer.
[pdf]  [BibTeX] 
Living Labs for Academic Search at CLEF 2020.
In: J. M. Jose, E. Yilmaz, J. Magalhães, P. Castells, N. Ferro, Má. J. Silva and F. Martins, editors, Advances in Information Retrieval, pages 580-586. Springer International Publishing, Cham, 2020.
Philipp Schaer, Johann Schaible and Bernd Müller.
[pdf]  [abstract]  [BibTeX] 
Overview of LiLAS 2020 - Living Labs for Academic Search Workshop Lab (extended abstract)..
In: L. Cappellato, C. Eickhoff, N. Ferro and A. Névéol, editors, CLEF (Working Notes), volume 2696, series CEUR Workshop Proceedings. CEUR-WS.org, 2020.
Philipp Schaer, Johann Schaible and Leyla Jael García-Castro.
[doi]  [BibTeX] 
Overview of LiLAS 2020 - Living Labs for Academic Search..
In: A. Arampatzis, E. Kanoulas, T. Tsikrika, S. Vrochidis, H. Joho, C. Lioma, C. Eickhoff, A. Névéol, L. Cappellato and N. Ferro, editors, CLEF, volume 12260, series Lecture Notes in Computer Science, pages 364-371. Springer, 2020.
Philipp Schaer, Johann Schaible and Leyla Jael García Castro.
[doi]  [BibTeX] 
Evaluation Infrastructures for Academic Shared Tasks.
Datenbank-Spektrum, 20(1):29-36, 2020.
Johann Schaible, Timo Breuer, Narges Tavakolpoursaleh, Bernd Müller, Benjamin Wolff and Philipp Schaer.
[doi] [pdf]  [abstract]  [BibTeX] 

2019

Information Extraction for Semi-structured Email Corpora..
In: R. Jäschke and M. Weidlich, editors, LWDA, volume 2454, series CEUR Workshop Proceedings, pages 322-330. CEUR-WS.org, 2019.
Hendrik Adam and Philipp Schaer.
[pdf]  [BibTeX] 
An investigation of biases in web search engine query suggestions.
Online Information Review, 44(2):365-381, 2019.
Malte Bonart, Anastasiia Samokhina, Gernot Heisenberg and Philipp Schaer.
[doi] [pdf]  [abstract]  [BibTeX] 
Fair ranking in academic search - Notebook for the TREC 2019 Fair Ranking Track..
In: E. M. Voorhees and A. Ellis, editors, TREC, volume 1250, series NIST Special Publication. National Institute of Standards and Technology (NIST), 2019.
Malte Bonart.
[pdf]  [BibTeX] 
Dockerizing Automatic Routing Runs for The Open-Source IR Replicability Challenge (OSIRRC 2019)..
In: R. Clancy, N. Ferro, C. Hauff, J. Lin, T. Sakai and Z. Z. Wu, editors, OSIRRC@SIGIR, volume 2409, series CEUR Workshop Proceedings, pages 31-35. CEUR-WS.org, 2019.
Timo Breuer and Philipp Schaer.
[pdf]  [BibTeX] 
Replicability and Reproducibility of Automatic Routing Runs..
In: L. Cappellato, N. Ferro, D. E. Losada and H. Müller, editors, CLEF (Working Notes), volume 2380, series CEUR Workshop Proceedings. CEUR-WS.org, 2019.
Timo Breuer and Philipp Schaer.
[pdf]  [BibTeX] 
STELLA: Towards a Framework for the Reproducibility of Online Search Experiments..
In: R. Clancy, N. Ferro, C. Hauff, J. Lin, T. Sakai and Z. Z. Wu, editors, OSIRRC@SIGIR, volume 2409, series CEUR Workshop Proceedings, pages 8-11. CEUR-WS.org, 2019.
Timo Breuer, Philipp Schaer, Narges Tavakolpoursaleh, Johann Schaible, Benjamin Wolff and Bernd Müller.
[pdf]  [BibTeX] 
Data Librarian – ein neuer Studienschwerpunkt für wissenschaftliche Bibliotheken und Forschungseinrichtungen.
Bibliothek Forschung und Praxis, 43(2018/04):255–261, 2019.
Simone Fühles-Ubach, Philipp Schaer, Klaus Lepsky and Ragna Seidler-de Alwis.
[pdf]  [BibTeX] 
Computational Methods in Professional Communication..
In: ProComm, pages 275-285. IEEE, 2019.
André Calero Valdez, Lena Adam, Dennis Assenmacher, Laura Burbach, Malte Bonart, Lena Frischlich and Philipp Schaer.
[pdf]  [BibTeX] 

2018

Intertemporal Connections Between Query Suggestions and Search Engine Results for Politics Related Queries.
In: EuroCSS 2018 Dataset Challenge. Cologne, 2018.
Malte Bonart and Philipp Schaer.
[doi] [pdf]  [BibTeX] 
Introduction to OXPath.
2018. cite arxiv:1806.10899Comment: 63 pages.
Ruslan R. Fayzrakhmanov, Christopher Michels and Mandy Neumann.
[doi] [pdf]  [abstract]  [BibTeX] 
Towards an IR Test Collection for the German National Library..
In: R. Gemulla, S. P. Ponzetto, C. Bizer, M. Keuper and H. Stuckenschmidt, editors, LWDA, volume 2191, series CEUR Workshop Proceedings, pages 275-280. CEUR-WS.org, 2018.
Johanna Munkelt, Philipp Schaer and Klaus Lepsky.
[doi] [pdf]  [BibTeX] 
Prioritizing and Scheduling Conferences for Metadata Harvesting in dblp.
In: JCDL '18 Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries , pages 45-48. ACM, New York, NY, USA, 2018.
Mandy Neumann, Christopher Michels, Philipp Schaer and Schenkel Ralf.
[doi] [pdf]  [abstract]  [BibTeX] 

2017

Overview of TREC OpenSearch 2017..
In: E. M. Voorhees and A. Ellis, editors, TREC, volume Special Publication 500-324. National Institute of Standards and Technology (NIST), 2017.
Rolf Jagerman, Krisztian Balog, Philipp Schaer, Johann Schaible, Narges Tavakolpoursaleh and Maarten de Rijke.
[doi]  [BibTeX] 
Web-Scraping for Non-Programmers: Introducing OXPath for Digital Library Metadata Harvesting.
Code4Lib Journal, 38, 2017.
Mandy Neumann, Jan Steinberg and Philipp Schaer.
[doi]  [abstract]  [BibTeX] 
Enriching Existing Test Collections with OXPath.
In: G. J. F. Jones, S. Lawless, J. Gonzalo, L. Kelly, L. Goeuriot, T. Mandl, L. Cappellato and F. Nicola, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction 8th International Conference of the CLEF Association, CLEF 2017, Dublin, Ireland, September 11-14, 2017, Proceedings, volume 10456, series Lecture Notes in Computer Science. 2017.
Philipp Schaer and Mandy Neumann.
[doi] [pdf]  [abstract]  [BibTeX] 
Living Labs - An Ethical Challenge for Researchers and Platform Providers.
In: M. Zimmer and K. Kinder-Kurlanda, editors, Internet Research Ethics for the Social Age: New Challenges, Cases, and Contexts. Peter Lang, 2017.
Philipp Schaer.
[pdf]  [BibTeX] 

2016

Overview of the TREC 2016 Open Search track.
In: E. M. Voorhees and A. Ellis, editors, TREC, volume Special Publication 500-321. National Institute of Standards and Technology (NIST), 2016.
Krisztian Balog, Anne Schuth, Peter Dekker, Philipp Schaer, Po-Yu Chuang and Narges Tavakolpoursaleh.
[pdf]  [BibTeX] 
How Relevant is the Long Tail? - A Relevance Assessment Study on Million Short..
In: N. Fuhr, P. Quaresma, T. Gonçalves, B. Larsen, K. Balog, C. Macdonald, L. Cappellato and N. Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 7th International Conference of the CLEF Association, CLEF 2016, Évora, Portugal, September 5-8, 2016, Proceedings, volume 9822, series Lecture Notes in Computer Science, pages 227-233. Springer, 2016.
Philipp Schaer, Philipp Mayr, Sebastian Sünkler and Dirk Lewandowski.
[pdf]  [BibTeX] 
Ideas for a Standard LL4IR Extension - Living Labs from a System Operator's Perspective..
In: K. Balog, L. Cappellato, N. Ferro and C. Macdonald, editors, Working Notes of CLEF 2016 - Conference and Labs of the Evaluation forum, Évora, Portugal, 5-8 September, 2016, volume 1609, series CEUR Workshop Proceedings, pages 591-592. CEUR-WS.org, 2016.
Philipp Schaer and Narges Tavakolpoursaleh.
[pdf]  [BibTeX] 
Popularity Ranking for Scientific Literature Using the Characteristic Scores and Scale Method.
In: E. M. Voorhees and A. Ellis, editors, TREC, volume Special Publication 500-321. National Institute of Standards and Technology (NIST), 2016.
Philipp Schaer and Narges Tavakolpoursaleh.
[doi] [pdf]  [BibTeX]