CLEF Lab – Call for Participation – Living Labs for IR Evaluation (LL4IR)
The Living Labs for Information Retrieval Evaluation (LL4IR) CLEF Lab
September, 2015, Toulouse, France
http://living-labs.net/clef-lab/
Register at: http://clef2015-labs-registration.dei.unipd.it/registrationForm.php
[Registration closes: 30 April, 2015]
===========================================================================================
This lab provides a benchmarking platform for researchers to evaluate their ranking systems in a live setting with real users in their natural task environments. The lab acts as a proxy between commercial organizations (live environments) and lab participants (experimental systems).
The lab focuses on three use-cases (using a shared API):
Task 1: product search (on the REGIO JÁTÉK e-commerce site)
Task 2: local domain search (on the University of Amsterdam’s website)
Task 3: web search (through Seznam, a major commercial web search engine)
——————————————-
What is in it for you as a lab participant?
——————————————-
– Access to privileged commercial data (click through, etc.).
– Access to new static TREC-style collections.
– Access to the living labs evaluation environments.
– Opportunity to test your IR systems on real live systems with real live users.
———————–
Current Important dates
———————–
31 Mar, 2015: Training period ends
1-15 Apr, 2015: Uploading test runs
15 Apr, 2015: Testing period begins
15 May, 2015: Testing period ends
——————-
Further Information
——————-
Further information is available on the lab website (http://living-labs.net/clef-lab/) or by emailing the lab organisers.
————–
Lab Organisers
————–
Krisztian Balog – University of Stavanger, Norway (krisztian.balog (at) uis.no)
Liadh Kelly – Trinity College Dublin, Ireland (liadh.kelly (at) tcd.ie)
Anne Schuth – University of Amsterdam, The Netherlands (anne.schuth (at) uva.nl)
Source: IRList