Abstract
State-of-the-art deep-learning-based approaches to Natural Language Processing (NLP) are credited with various capabilities that involve reasoning with natural language texts. In this paper we carry out a large-scale empirical study investigating the detection of formally valid inferences in controlled fragments of natural language for which the satisfiability problem becomes increasingly complex. We find that, while transformer-based language models perform surprisingly well in these scenarios, a deeper analysis reveals that they appear to overfit to superficial patterns in the data rather than acquiring the logical principles governing the reasoning in these fragments.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing |
Publisher | Association for Computational Linguistics |
Pages | 11184-11199 |
Number of pages | 16 |
Publication status | Published - 7 Dec 2022 |
Event | Conference on Empirical Methods in Natural Language Processing (2022) - Abu Dhabi Duration: 7 Dec 2022 → 11 Dec 2022 https://2022.emnlp.org/ |
Conference
Conference | Conference on Empirical Methods in Natural Language Processing (2022) |
---|---|
Abbreviated title | EMNLP 2022 |
Period | 7/12/22 → 11/12/22 |
Internet address |