Loading...

Baselines

Baselines used in the evaluation process

TASK 1 - ATE: Aspect Term Extraction

We decided to use a simple approach as a baseline for the ATE task. It uses the common names found in the sentence as "aspect terms." The algorithm is based on the result of Part of Speech Tagging obtained through the SpaCy tool on the Italian model 'it_core_news_sm'. The implementation of the baseline on the training set is available as a Python3 Notebook at the following address: http://www.di.uniba.it/~swap/ate_absita/baselines/Task1.ATE_baseline.ipynb

TASK 2 - ABSA: Aspect-based Sentiment Analysis

For the ABSA task we decided to assign at each aspect, found with the baseline strategy for Task 1, the most frequent polary class, i.e. positive. The implementation of the baseline on the training set is available as a Python3 Notebook at the following address: http://www.di.uniba.it/~swap/ate_absita/baselines/Task2.ABSA_baseline.ipynb

TASK 3 - SA: Sentiment Analysis

For the Sentiment Analysis task, since it is simpler than the previous ones, we wanted to use three baselines. The first predicts the most frequent value in the training set: 5. The second predicts for each sentence the average value of the scores on the training set 4.46299. The third one uses AlBERTo as an approach to develop a Numerical Regression task.

AlBERTo pre-trained model is available here: https://drive.google.com/file/d/1SGVrP59Uv7a77tqaVjsMzJ8M60neVwyd/view?usp=sharing

EVALUATION Script

To make the evaluation of the results produced during the development phase more simple, we have decided to release our evaluation script. It is written in Python3 and can be used via the command line:
#usage: ./evaluation_ate_absita.py RESULT_FILE GOLD_FILE