[AISWorld] SemEval-2017 Task 11: End-User Development using Natural Language

[André Freitas]

* SemEval-2017 Task 11 *
= End-User Development using Natural Language =

website: http://alt.qcri.org/semeval2017/task11/

This task proposes a challenge to support the interaction of end-users with
applications, micro-services and software APIs using natural language. The
task aims at evolving the discussions around the application of natural
language processing techniques within the context of end-user natural
language programming.

Recent advances in Natural Language Processing bring the opportunity of
improving the interaction between users and software artefacts, supporting
end-users programming mediated by natural language. This ability to match
user’s action intents and information needs to formal actions within a set
of Application Programming Interfaces (APIs), using the semantics of
natural language as the mediation layer between both, can significantly
impact the accessibility of software development.

This SemEval task aims at developing the state-of-the-art discussions and
techniques concerning the semantic interpretation of natural language
commands and user action intents, bridging the semantic gap between users
and software artefacts.

*The Task:*

The test collection consists of a set of natural language commands. Each
natural language command is mapped to one or more actions within the APIs
Knowledge Base (KB). The interpretation of the natural language command and
its mapping to the API is also dependent on the user personal context (User
KB), which contains the user personal data (contacts, addresses, emails,
phones, ...).

*Important Dates:*

Sat 17 Sep 2016: Training data ready
Mon 09 Jan 2017: Evaluation starts
Mon 30 Jan 2017: Evaluation ends
Mon 06 Feb 2017: Results posted
Mon 27 Feb 2017: Paper submissions due
Mon 03 Apr 2017: Author notifications

Questions and comments:
https://groups.google.com/forum/#!forum/semeval-task11