Aims and Scope

In the current Covid-19 crisis, as in many other emergency situations, the general public, as well as many other stakeholders, need to aggregate and summarize different sources of information into a single coherent synopsis or narrative, complementing different pieces of information, resolving possible inconsistencies, and preventing mis-information. This should happen across multiple languages, sources, and levels of linguistic knowledge that varies depending on social, cultural or educational factors.

Covid-19 MLIA Eval organizes a community evaluation effort aimed at accelerating the creation of resources and tools for improved MultiLingual Information Access (MLIA) in the current emergency situation with a reference to a general public use case:

Sofia has heard that a drug has been experimented in different countries and she would like to have a consolidated and trustworthy view of the main findings, whether the drug is effective or not, and whether there are any adverse effects.

Distillation for the general public also implies a level of specialist-non-specialist communication, when the aggregated sources contain both disseminative and specialised sources. Therefore, the general public would need to understand medical expertise by using their correspondent in the "popular" language or by using an appropriately calibrated language for the communication to be effective.

Community Evaluation Effort

Covid-19 MLIA Eval is an evaluation effort promoted by several communities which are closely working together.

EU logo
An effort supported by the European Commission
CLEF logo
An effort supported by the CLEF Initiative

We hope that other communities will join us in this shared effort. You are very welcome!

Just contact us  Email  !!!

Overall Approach

Covid-19 MLIA Eval adopts an incremental and iterative evaluation methodology to enable the release of intermediate (but functional) resources and to progressively (iteration-after-iteration) move towards finally consolidated tools and resources.

We plan for three rounds, tentatively one-month long each, organized as follows:

To facilitate the exchange of resources and components and to jointly work together for improving MLIA technologies for Covid-19, participants are provided with a dedicated git repository where to push and share the outcomes of your participation in the different rounds, i.e. runs, code, (language) resources, and a technical report. To facilitate sharing and re-use, all the contents of the repositories are realeased under the Creative Commons Attribution-ShareAlike 4.0 International License.

Covid-19 MLIA Eval Tasks

The following tasks are offered.

Information Extraction

Multilingual Semantic Search

Machine Translation

Coordinators

Overall

Khalid Choukri, Evaluations and Language resources Distribution Agency (ELDA), France
choukrielda.org

Nicola Ferro, University of Padua, Italy
ferrodei.unipd.it

Information Extraction

Thierry Declerck, DFKI, Germany
declerckdfki.de

Pierre Zweigenbaum, LMSI, France
pzlimsi.fr

Multilingual Semantic Search

Giorgio Maria Di Nunzio, University of Padua, Italy
dinunziodei.unipd.it

Maria Eskevich, CLARIN ERIC
mariaclarin.eu

Machine Translation

Manuel Herranz, Pangeanic, Spain
m.herranzpangeanic.es

Mercedes García-Martínez, Pangeanic, Spain
m.garciapangeanic.com