BLM-AgrI-Gen (Blackbird Language Matrices Subject-Verb agreement in Italian)

Get Data

Description

BLM-AgrI-Gen is a dataset of Italian data for learning the underlying rules of subject-verb agreement in sentences, developed in the Blackbird Language Matrices (BLM) framework (this dataset is a subset from the training data of https://www.idiap.ch/dataset/BLM-AgrI). In this task, an instance consists of a sequence of sentences with specific attributes. To predict the correct answer as the next element of the sequence, a model must correctly detect the set of underlying generative rules used to produce the dataset.

Blackbird Language Matrices (BLMs) are multiple-choice problems, where the input is a sequence of sentences built using specific generating rules, and the answer set consists of a correct answer that continues the input sequence, and several incorrect contrastive options, built by violating the underlying generating rules of the sentences. In a BLM matrix, all sentences share the targeted linguistic phenomenon (in this case subject-verb agreement), but differ in other aspects relevant for the phenomenon in question.

The BLM-AgrI-Gen is one of the six sub-tasks of the BLM-It challenge. All sub-tasks are instances of the general BLM task, but they differ along two dimensions: the linguistic problem they define (Agr, Caus, Od) and the lexical complexity of the data (II, III)1.

The data comes grouped by lexical variation (i.e. type II/III) and each subset is split into train/test. The statistics of the current iteration of the dataset are (train:test split information):

Reference

If you use this dataset,please cite the following publication:

Jiang, Chunyang & Samo, Giuseppe & Nastase, Vivi & Merlo, Paola. (2024). BLM-It — Blackbird Language Matrices for Italian: A CALAMITA Challenge. (TO APPEAR)