Improving the Reproducibility of PAN’s Shared Tasks: Plagiarism Detection, Author Identification, and Author Profiling

Improving the Reproducibility of PAN’s Shared Tasks: Plagiarism Detection, Author Identification, and Author Profiling (2014)

by Martin Potthast, Tim Gollub, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, Benno Stein

Venue:

In CLEF’14: Proceedings of the 5th Int. Conference of the CLEF Initiative

Results 1 - 2 of 2

B.: Towards Data Submissions for Shared Tasks: First Experiences for the Task of Text Alignment

by Martin Potthast, Steve Göring, Paolo Rosso, Benno Stein - In: Working Notes Papers of the CLEF 2015 Evaluation Labs. CEUR Workshop Proceedings, CLEF and CEUR-WS.org (Sep 2015), http://www.clef-initiative.eu/publication/ working-notes

"... Abstract This paper reports on the organization of a new kind of shared task that outsources the creation of evaluation resources to its participants. We intro-duce the concept of data submissions for shared tasks, and we use our previous shared task on text alignment as a testbed. A total of eight ..."

Abstract - Cited by 3 (1 self) - Add to MetaCart

Abstract This paper reports on the organization of a new kind of shared task that outsources the creation of evaluation resources to its participants. We intro-duce the concept of data submissions for shared tasks, and we use our previous shared task on text alignment as a testbed. A total of eight evaluation datasets have been submitted by as many participating teams. To validate the submitted datasets, they have been manually peer-reviewed by the participants. Moreover, the submitted datasets have been fed to 31 text alignment approaches in order to learn about the datasets ’ difficulty. The text alignment implementations have been submitted to our shared task in previous years and since been kept operational on the evaluation-as-a-service platform TIRA. 1

(Show Context)

WORKSHOP REPORT Report on the Evaluation-as-a-Service (EaaS) Expert Workshop

by Frank Hopfgartner, Allan Hanbury, Noriko K, Simon Mercer, Jayashree Kalpathy-cramer, Martin Potthast, Tim Gollub, Anastasia Krithara, Jimmy Lin, Krisztian Balog, Ivan Eggel , 2015

"... In this report, we summarize the outcome of the “Evaluation-as-a-Service ” workshop that was held on the 5th and 6th March 2015 in Sierre, Switzerland. The objective of the meeting was to bring together initiatives that use cloud infrastructures, virtual machines, APIs (Application Programming Inter ..."

Abstract - Add to MetaCart

In this report, we summarize the outcome of the “Evaluation-as-a-Service ” workshop that was held on the 5th and 6th March 2015 in Sierre, Switzerland. The objective of the meeting was to bring together initiatives that use cloud infrastructures, virtual machines, APIs (Application Programming Interface) and related projects that provide evaluation of information retrieval or machine learning tools as a service. 1

(Show Context)