Contextual Label Projection for Cross-Lingual Structured Prediction

Tanmay Parekh, I.-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, and Nanyun Peng, in NAACL, 2024.

Download the full text

Abstract

Label projection, which involves obtaining translated labels and texts jointly, is essential for leveraging machine translation to facilitate cross-lingual transfer in structured prediction tasks. Prior research exploring label projection often compromise translation accuracy by favoring simplified label translation or relying solely on word-level alignments. In this paper, we introduce a novel label projection approach, CLaP, which translates text to the target language and performs contextual translation on the labels using the translated text as the context, ensuring better accuracy for the translated labels. We leverage instruction-tuned language models with multilingual capabilities as our contextual translator, imposing the constraint of the presence of translated labels in the translated text via instructions. We benchmark CLaP with other label projection techniques on zero-shot cross-lingual transfer across 39 languages on two representative structured prediction tasks - event argument extraction (EAE) and named entity recognition (NER), showing over 2.4 F1 improvement for EAE and 1.4 F1 improvement for NER. We further explore the applicability of CLaP on ten extremely low-resource languages to showcase its potential for cross-lingual structured prediction.

Bib Entry

@inproceedings{parekh2024contextual,
  title = {Contextual Label Projection for Cross-Lingual Structured Prediction},
  author = {Parekh, Tanmay and Hsu, I-Hung and Huang, Kuan-Hao and Chang, Kai-Wei and Peng, Nanyun},
  booktitle = {NAACL},
  year = {2024}
}

Related Publications

LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs, ACL, 2026
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction, ACL, 2022
Evaluating the Values of Sources in Transfer Learning, NAACL, 2021
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training, EMNLP, 2021
Syntax-augmented Multilingual BERT for Cross-lingual Transfer, ACL, 2021
GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction, AAAI, 2021
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering, EMNLP-Finding, 2020
Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages, CoNLL, 2019
Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing, EMNLP, 2019
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing, NAACL, 2019