[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[eu_members at aclweb dot org] Maryland: Text Analysis Conference




Text Analysis Conference

Request for Track Proposals for TAC 2010

Deadline: September 25, 2009


The Text Analysis Conference (TAC) is a series of evaluation workshops
organized by the U.S. National Institute of Standards and Technology
(NIST) to encourage research in Natural Language Processing and
related applications, by providing a large test collection, common
evaluation procedures, and a forum for organizations to share their
results. TAC comprises sets of tasks known as "tracks," each of which
focuses on a particular subproblem of NLP. TAC tracks focus on
end-user tasks, but also include component evaluations situated within
the context of end-user tasks.

NIST is soliciting proposals for tracks to include in TAC 2010.  All
candidate tracks (whether new or existing) must submit a proposal by
September 25, 2009 in order to be considered for TAC 2010.  If you
would like to see a current TAC track continue in 2010 and have
suggestions for modifying the track, you are invited to contact the
track coordinator, listed at the TAC 2009 tracks page:

http://www.nist.gov/tac/2009/

Track proposals will be reviewed by the TAC Advisory Committee, and a
small number will be selected for discussion at the TAC 2009 workshop,
November 16-17, 2009, in Gaithersburg, MD, USA.  If your proposal is
selected for discussion, please be prepared to present the proposed
track at the TAC 2009 workshop. The final decision about which tracks
to include in TAC 2010, will be made after discussion at the TAC 2009
workshop.


Criteria for selection of tracks include:

   + a strong advocate who is willing to be the track coordinator
     (outside of NIST, track coordinator is a volunteer position);

   + a large enough core of interested researchers to make the
     track viable;

   + the availability of sufficient resources such as appropriate
     corpora and assessors with expertise in the area; and

   + the fit with other tracks.


Proposals must contain enough information for the Advisory Committee to
assess the criteria above, including:

   1. An explicit statement of the goals of the track (i.e., what is
expected to be learned and/or what infrastructure would be created if
the track were run).

   2. A description of the task(s) to be run in the track, and
possible methods of evaluation.

   3. What corpora would be necessary for the the track, and whether
there are any special constraints on the corpora.

   4. What manual assessment or annotation is required (if any).

      * Where the assessing/annotation would occur (NIST or elsewhere).

      * Whether there are any special qualifications or domain expertise
required of the assessors.

      * An estimate of the amount of time such assessing would require.

   5. Full contact details of the track proposer.


Please email track proposals as a postscript, PDF, or ASCII document to
tac-manager at nist dot gov by September 25, 2009.