An Efficient Approach for Findings Document Similarity Using Optimized Word Mover’s Distance

Atanu Dey, Mamata Jenamani, Arijit De

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

40 Downloads (Pure)

Abstract

We introduce Optimized Word Mover’s Distance (OWMD), a similarity function that compares two sentences based on their word embeddings. The method determines the degree of semantic similarity between two sentences considering their interdependent representations. Within a sentence, all the words may not be relevant for determining contextual similarity at the aspect level with another sentence. To account for this fact, we designed OWMD in two ways: first, it decreases system’s complexity by selecting words from the sentence pair according to a predefined set of dependency parsing criteria; Second, it applies the word mover’s distance (WMD) method to previously chosen words. When comparing the dissimilarity of two text sentences, the WMD method is used because it represents the minimal “journey time” required for the embedded words of one sentence to reach the embedded words of another sentence. Finally, adding an exponent function to the inverse of the OWMD dissimilarity score yields the resulting similarity score, called Optimized Word Mover’s Similarity (OWMS). Using STSb-Multi-MT dataset, the OWMS measure decreases MSE, RMSE, and MAD error rates by , , and respectively than previous approaches. Again, OWMS reduces MSE, RMSE, and MAD error rates on Semantic Textual Similarity (STS) dataset by , , and respectively. For STSb-Multi-MT and STS datasets, the suggested strategy reduces run-time complexity by and , respectively, compared to the best of existing approaches.
Original languageEnglish
Title of host publicationPattern Recognition and Machine Intelligence. PReMI 2023. Lecture Notes in Computer Science
Pages3-11
Number of pages9
DOIs
Publication statusPublished - 4 Dec 2023

Fingerprint

Dive into the research topics of 'An Efficient Approach for Findings Document Similarity Using Optimized Word Mover’s Distance'. Together they form a unique fingerprint.

Cite this