Test topics are available for both the main (REF) and the pilot (ELC) tasks from the active participants part of the TREC web site (you need to be a registered participant for TREC 2010 to be able to access it). Some additional notes on the ELC topics have been posted on the mailing list.
The submission deadline for both tasks is Sept 30. This is definitely a hard deadline for the REF task; as to the ELC task, it may be subject to extension.
The test topics are available for download at the TREC site. You need to be a registered participant to be able to access them.
Developing topics for this track has turned out to be a very hard task. Among the many reasons is that lots of entity homepages are not in the Category B subset of the collection.
Given the facts that this is a new task, a new collection, and we have a relatively small number of topics (20), evaluation will primarily focus on analysis of the results on a per-topic basis, rather than on average measures.
The guidelines and the timeline have been updated.
Submissions are due by Sept 21.
Tags: guidelines, timeline, topics
The timeline has been posted.
A number of training topics have been made available. There is a separate page to facilitate discussion.
The Entity track (among several other TREC tracks) will use the ClueWeb09 dataset, which has officially been released. The full collection consists of 1 billion pages, in 10 languages. For the first year of the Entity track we will use the smaller, “Category B” subset, which contains about 50 million English pages.
Tags: Data