Plans for Entity 2011
We plan to run two main tasks in 2011: related entity finding (REF) and entity list completion (ELC). In addition, there will be a Linked Open Data (LOD) variant of the REF task. Groups may choose to participate in any of these tasks.
Related entity finding (REF)
- Task: return a ranked list of entities of a specified type that engage in a given relationship with a given source entity.
- Collection: ClueWeb09 English
- Entity identification: Homepage
- Topics: 50 new topics for 2011
- Changes compared to 2010
- Only primary homepages are accepted, i.e., relevance is binary
- For each answer, a (single) supporting document is required; answers without supporting evidence will be judged incorrect
- Wikipedia pages are (still) not accepted as entity homepages, but they can be supporting documents
- Instead of the four high-level entity types (person, organization, location, product) a more fine-grained target type will be defined using the DBPedia Ontology; i.e., possible types are classes within this ontology
REF LOD variant
The task, collection, and topics are the very same as for the REF task, but there is one important difference:
- Entity identification: LOD URI (given a LOD crawl, which is the same as used for the ELC task)
Entity list completion (ELC)
The ELC task will be the same as in 2010, only with more topics and possibly on a larger Linked Open Data crawl.
- Task: same as REF, but topic definition includes a set of examples entities (identified by their LOD URI)
- Collection: Linked Open Data crawl (BTC-2009 or BTC-2010 or other)
- Entity identification: LOD URI
- Topics: based on REF 2010 topics (and possibly some more)
Questions, comments, feedback are welcome!
One Response to “Plans for Entity 2011”
Leave a Reply
You must be logged in to post a comment.
November 29th, 2010 at 2:50 pm
[...] TREC Entity Track: Plans for Entity 2011 [...]