MWP-BERT: Numeracy-Augmented Pre-training for MathWord Problem Solving

Zhenwen Liang, Jipeng Zhang, Lei Wang, Wei Qin, Yunshi Lan, Jie Shao, Xiangliang Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

30 Scopus citations

Abstract

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving forMWPsolving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM.We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.
Original languageEnglish (US)
Title of host publicationFindings of the Association for Computational Linguistics: NAACL 2022 - Findings
PublisherAssociation for Computational Linguistics (ACL)
Pages997-1009
Number of pages13
ISBN (Print)9781955917766
StatePublished - Jan 1 2022
Externally publishedYes

Fingerprint

Dive into the research topics of 'MWP-BERT: Numeracy-Augmented Pre-training for MathWord Problem Solving'. Together they form a unique fingerprint.

Cite this