[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[eu_members at aclweb dot org] SemEval-2010: 5th International Workshop on Semantic Evaluations -- Call for Task Proposals
SemEval-2010: 5th International Workshop on Semantic Evaluations
CALL FOR TASK PROPOSALS
The Senseval Committee invites proposals for tasks to be run as part
of SemEval-2010. Following the spirit of the past SemEval-2007,
the nature of the tasks in previous Senseval editions has evolved
to include semantic analysis tasks outside of word sense disambiguation.
Thus we welcome proposals for any tasks that can test an automatic system
for semantic analysis of text, be it application dependent or
independent. We especially encourage tasks for different languages,
cross-lingual tasks, and tasks that are relevant to particular NLP
applications such as machine translation, information retrieval and
It will help us if tasks are designed in such that they can be run and
scored automatically from a central website (as in Senseval-3 and SemEval-2007).
We'll be happy to help with task design, data formatting, and so on.
Information on the design of previous Senseval-1/2/3 tasks and SemEval-2007
can be found at:
The time period for SemEval-2010 has not yet been finalised, but it will be
held over a 1-2 month period in the first part of 2010.
Proposals for tasks will ideally contain:
- A description of the task (about 1 page)
- How the training/testing data will be built and/or procured
- The evaluation methodology to be used including clear evaluation
- The availability of the resources to the participants (copyright,
- The resources required to prepare the task (time, money, and human)
If you are not yet at a point to provide outlines of all of these,
that's fine, but please give some thought to each, and present an overview
of your first ideas. We will gladly give feedback.
Please submit proposals as soon as possible, preferably by electronic
mail in plain ASCII text to the SemEval-2010 chairs:
Katrin Erk, University of Texas at Austin (katrin.erk at mail.utexas.edu)
Carlo Strapparava, FBK-irst, Italy (strappa at fbk.eu)
Sep 21, 2008 Intention to submit (send a short-paragraph description)
Oct 19, 2008 Submission deadline for task proposals
Nov 16, 2008 Notification of acceptance
In SemEval-2010 we are going to give plenty of time both in tasks preparation phase,
especially regarding the tuning of data sets, and to task participants
to prepare for the task and to tune their systems. Along this rationale, we strongly
recommend that each task organizers come up with the presentation of trial data set,
well in advance (roughly, summer 2009). Please consider this point in designing the task schedule.