WebCLEF 2008


Like its predecessor, the 2007 edition of WebCLEF, WebCLEF 2008 will implement a multilingual "information synthesis" task, where, for a given topic, participating systems have to extract important snippets from web pages (fetched from the live web and provided by the task organizers). See the detailed description of the 2007 task.

Changes from 2007 concern the following:

  • The test collection, topics, assessments, and the source code of the best performing 2007 system and the evaluation scripts are available, so as to create a shared baseline for 2008 participants.
  • We will experiment with different evaluation metrics, over and above the ones used at WebCLEF 2007. In particular, lessons from DUC and from the TREC QA track will be taken on board.

WebCLEF 2008 has lots of similarities with (topic-oriented) multi-document summarization and with answering so-called "other questions". An important difference is that at WebCLEF, topics may come with extensive descriptions and with many thousands of documents from which important facts have to be mined. In addition, WebCLEF works with web documents, that may be very noisy.

Organizers

The WebCLEF 2008 is organized by the Information and Language Processing group at the University of Amsterdam. The advisory board of the task:

  • Julio Gonzalo, Universidad Nacional de Educación a Distancia
  • Valentin Jijkoun, University of Amsterdam
  • Jimmy Lin, University of Maryland
  • Constantin Orasan, University of Wolverhampton
  • Maarten de Rijke, University of Amsterdam

Contact

If you have any questions about the WebCLEF task please contact Valentin Jijkoun (jijkoun AT science.uva.nl). To stay up-to-date about WebCLEF developments, subscribe to the low-traffic WebCLEF mailing list by sending email with body
SUBSCRIBE webclef@science.uva.nl your.email.address
to majordomo AT science.uva.nl

News

June 6, 2008
Test topics and test collection released to the participants
June 4, 2008
Topic generation completed
May 13, 2008
Topic generation started
Apr 1, 2008
Final call for participation released
Feb 7, 2008
Second call for participation released
Jan 7, 2008
First call for participation for WebCLEF 2008 released.
Dec 13, 2007
All data and software for the WebCLEF 2007 is available. See also the detailed description of the task and participating systems in CLEF workshop working notes.