2022

Cross-lingual transfer of monolingual models

Evangelia Gogoulou, Ariel Ekgren, Tim Isbister, Magnus Sahlgren

Summary

Research on using existing language models for training in new languages to save computational resources and benefit low-resource languages. Method: replace source language vocabulary with target language vocabulary and continue pre-training on target language data.

Themes

llm

Keywords

cross-lingual transfer, BERT, low-resource languages, NLP, multilingual

Poster

Click image to open full size