Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection

Dmitry Nikolaev; Sebastian Padó

doi:10.18653/v1/2022.sigtyp-1.2

Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection

Dmitry Nikolaev
, Sebastian Padó

University of Stuttgart

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

The capabilities and limitations of BERT and similar models are still unclear when it comes to learning syntactic abstractions, in particular across languages. In this paper, we use the task of subordinate-clause detection within and across languages to probe these properties. We show that this task is deceptively simple, with easy gains offset by a long tail of harder cases, and that BERT’s zero-shot performance is dominated by word-order effects, mirroring the SVO/VSO/SOV typology.

Original language	English
Title of host publication	Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP
Editors	Ekaterina Vylomova, Edoardo Ponti, Ryan Cotterell
Place of Publication	Seattle, Washington
Publisher	Association for Computational Linguistics
Pages	11–21
DOIs	https://doi.org/10.18653/v1/2022.sigtyp-1.2
Publication status	Published - Jul 2022
Externally published	Yes
Event	4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP - Seattle, United States Duration: 14 Jul 2022 → …

Workshop

Workshop	4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP
Country/Territory	United States
City	Seattle
Period	14/07/22 → …

Access to Document

10.18653/v1/2022.sigtyp-1.2

Cite this

Nikolaev, D., & Padó, S. (2022). Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection. In E. Vylomova, E. Ponti, & R. Cotterell (Eds.), Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP (pp. 11–21). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.sigtyp-1.2

Nikolaev, Dmitry ; Padó, Sebastian. / Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection. Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP. editor / Ekaterina Vylomova ; Edoardo Ponti ; Ryan Cotterell. Seattle, Washington : Association for Computational Linguistics, 2022. pp. 11–21

@inproceedings{d3fc9d752b8f4a5db6d19a5b25090dfa,

title = "Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection",

abstract = "The capabilities and limitations of BERT and similar models are still unclear when it comes to learning syntactic abstractions, in particular across languages. In this paper, we use the task of subordinate-clause detection within and across languages to probe these properties. We show that this task is deceptively simple, with easy gains offset by a long tail of harder cases, and that BERT{\textquoteright}s zero-shot performance is dominated by word-order effects, mirroring the SVO/VSO/SOV typology.",

author = "Dmitry Nikolaev and Sebastian Pad{\'o}",

year = "2022",

month = jul,

doi = "10.18653/v1/2022.sigtyp-1.2",

language = "English",

pages = "11–21",

editor = "Ekaterina Vylomova and Edoardo Ponti and Ryan Cotterell",

booktitle = "Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP",

publisher = "Association for Computational Linguistics",

address = "United States",

note = "4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP ; Conference date: 14-07-2022",

}

Nikolaev, D & Padó, S 2022, Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection. in E Vylomova, E Ponti & R Cotterell (eds), Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP. Association for Computational Linguistics, Seattle, Washington, pp. 11–21, 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP, Seattle, Washington, United States, 14/07/22. https://doi.org/10.18653/v1/2022.sigtyp-1.2

Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection. / Nikolaev, Dmitry; Padó, Sebastian.
Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP. ed. / Ekaterina Vylomova; Edoardo Ponti; Ryan Cotterell. Seattle, Washington: Association for Computational Linguistics, 2022. p. 11–21.

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection

AU - Nikolaev, Dmitry

AU - Padó, Sebastian

PY - 2022/7

Y1 - 2022/7

N2 - The capabilities and limitations of BERT and similar models are still unclear when it comes to learning syntactic abstractions, in particular across languages. In this paper, we use the task of subordinate-clause detection within and across languages to probe these properties. We show that this task is deceptively simple, with easy gains offset by a long tail of harder cases, and that BERT’s zero-shot performance is dominated by word-order effects, mirroring the SVO/VSO/SOV typology.

AB - The capabilities and limitations of BERT and similar models are still unclear when it comes to learning syntactic abstractions, in particular across languages. In this paper, we use the task of subordinate-clause detection within and across languages to probe these properties. We show that this task is deceptively simple, with easy gains offset by a long tail of harder cases, and that BERT’s zero-shot performance is dominated by word-order effects, mirroring the SVO/VSO/SOV typology.

U2 - 10.18653/v1/2022.sigtyp-1.2

DO - 10.18653/v1/2022.sigtyp-1.2

M3 - Conference contribution

SP - 11

EP - 21

BT - Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP

A2 - Vylomova, Ekaterina

A2 - Ponti, Edoardo

A2 - Cotterell, Ryan

PB - Association for Computational Linguistics

CY - Seattle, Washington

T2 - 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP

Y2 - 14 July 2022

ER -

Nikolaev D, Padó S. Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection. In Vylomova E, Ponti E, Cotterell R, editors, Proceedings of the 4th Workshop on Research in Computational Linguistic Typology and Multilingual NLP. Seattle, Washington: Association for Computational Linguistics. 2022. p. 11–21 doi: 10.18653/v1/2022.sigtyp-1.2

Word-order Typology in Multilingual BERT: A Case Study in Subordinate-Clause Detection

Abstract

Workshop

Access to Document

Fingerprint

Cite this