Can Transformers Reason in Fragments of Natural Language?

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

State-of-the-art deep-learning-based approaches to Natural Language Processing (NLP) are credited with various capabilities that involve reasoning with natural language texts. In this paper we carry out a large-scale empirical study investigating the detection of formally valid inferences in controlled fragments of natural language for which the satisfiability problem becomes increasingly complex. We find that, while transformer-based language models perform surprisingly well in these scenarios, a deeper analysis reveals that they appear to overfit to superficial patterns in the data rather than acquiring the logical principles governing the reasoning in these fragments.
Original languageEnglish
Title of host publicationProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
PublisherAssociation for Computational Linguistics
Pages 11184-11199
Number of pages16
Publication statusPublished - 7 Dec 2022
EventConference on Empirical Methods in Natural Language Processing (2022) - Abu Dhabi
Duration: 7 Dec 202211 Dec 2022
https://2022.emnlp.org/

Conference

ConferenceConference on Empirical Methods in Natural Language Processing (2022)
Abbreviated titleEMNLP 2022
Period7/12/2211/12/22
Internet address

Fingerprint

Dive into the research topics of 'Can Transformers Reason in Fragments of Natural Language?'. Together they form a unique fingerprint.

Cite this