[AISWorld] Sentiment Analysis: GermEval Task 2017 - Shared Task on Aspect-based Sentiment in Social Media Customer Feedback

[Chris Biemann]

(apologies for cross-posting)
==== Call for Participation ====

GermEval Task 2017 - Shared Task on Aspect-based Sentiment in Social 
Media Customer Feedback

Website: https://sites.google.com/view/germeval2017-absa/

We invite everyone from academia and industry to submit to the shared 
task on German Sentiment Analysis!


---- Introduction ----

In the connected, modern world, customer feedback is a valuable source 
for insights on the quality of products or services. This feedback 
allows other customers to benefit from the experiences of others and 
enables businesses to react on requests, complaints or recommendations. 
However, the more people use a product or service, the more feedback is 
generated, which results in the major challenge of analyzing huge 
amounts of feedback in an efficient, but still meaningful way.

We conduct a shared task on automatically analyzing customer reviews 
about “Deutsche Bahn” - the German public train operator with about two 
billion passengers each year. The task is associated with the GSCL 2017 
conference in Berlin, and will take place there as a half-day workshop 
on September 12, 2017.


---- Data ----

The data for the task has been annotated as part of a joint project 
between TU Darmstadt and Deutsche Bahn, see 
https://sites.google.com/view/germeval2017-absa/data for details.
All together it consists of 22,000 messages from various social media 
and web sources.
The data is annotated with relevance, document sentiment, aspect-based 
sentiment and opinion target expressions and is provided in both xml and 
tsv.


---- Task description ----

To exploit the richness of the data, we subdivided the task into four 
subtasks.
Participants can freely choose in what and how many sub-tasks they 
participate.

Subtask A) Relevance Classification
Determine whether a social media post contains feedback about the 
"Deutsche Bahn" or if the post is off-topic/contains no evaluation.
Example: “Ehrlich die männer in Der Bahn haben keine manieren?”
In the given post, the task is to identify that the post is "relevant".

Subtask B) Document-level Polarity
Identify, whether the customer evaluates the "Deutsche Bahn" or travel 
as positive, negative or neutral.
Example: "Ingo Lenßen Guten morgen Ingo...bei mir kein regen aber bahn 
fehr wieder nicht..."
In the given post, the task is to identify the posts' polarity : negative

Subtask C) Aspect-level Polarity
Identify all aspects which are positively and negatively evaluated 
within the review. In order to increase comparability, the aspects are 
previously divided into predefined categories. Consequently, the aim of 
the subtasks is to identify all contained categories and their 
associated polarity.
Example: “Alle so "Yeah, Streik beendet"" Bahn so "Okay, dafür werden 
dann natürlich die Tickets teurer" Alle so "Können wir wieder Streik 
haben?””
In the given post, the task is to identify the aspects and their 
polarity: <Ticketkauf#Haupt:negativ> <Allgemein#Haupt:positive>

Subtask D) Opinion Target Extraction
Identify the linguistic expression in the posts which are used to 
express the aspect-based sentiment (subtask C). The opinion target 
expression is defined by its starting and ending offsets.
Example: @m_wabersich IC 2151? Der fährt nicht. Ich habe Ihnen die 
Alternative bereits genannt. /je
In the given post the task ist to identify the target expression <fährt 
nicht> (from="26" to="37").

The organizers will provide a baseline system and a scoring method that 
all participants can use to lower the obstacles for participation.
Data, baseline system as well as the description of the tasks are 
distributed to the participants via the website 
https://sites.google.com/view/germeval2017-absa/.

Participating team/participants may submit several runs.

Submissions consist of one TSV or XML file per subtask providing 
predictions for the test data and a paper of 4 pages (excluding 
references) describing the chosen approach and analyzing the 
performance. Each participating team/participant should submit only one 
paper regardless of how many subtasks they participate in; per subtask, 
participants can add 1 page to the paper, up to a maximum of 7 pages. 
Papers should follow the GSCL 2017 style files. We expect authors to 
present summaries of their systems at the GSCL workshop and to 
participate in the discussions.


---- Important Dates ----

* March 2017 Release of Trial Data
* April 2017 Release of Training Data
* July 24, 2017 Release of Test Data
* August 14, 2017 Submission of System Runs
* August 21, 2017 Submission of System Description papers
* September 1, 2017 Feedback on System Description papers
* September 8, 2017 Final Submission of System Description papers
* September 12, 2017 Workshop co-located with GSCL 2017


---- Organizing committee ----

* Chris Biemann, Language Technology, Uni Hamburg, 
biemann[at]informatik.uni-hamburg.de
* Eugen Ruppert, Language Technology, Uni Hamburg, 
ruppert[at]informatik.uni-hamburg.de
* Michael Wojatzki, Language Technology Lab, Uni Duisburg-Essen, 
michael.wojatzki[at]uni-due.de
* Torsten Zesch, Language Technology Lab, Uni Duisburg-Essen, 
torsten.zesch[at]uni-due.de

To contact the organizing committee, please post to the GermEval 2017 
ABSA mailing list at 
https://groups.google.com/forum/#!forum/germeval2017-absa, or for 
private communication, send an e-mail to Michael Wojatzki.