Word Replaceability Through Word Vectors

AbstractThere have been many numerical methods developed recently that try to capture the semantic meaning of words through word vectors. In this study, we present a new way to learn word vectors using only word co-appearances and their average distances. However, instead of claiming semantic or syntactic word representation, we lower our assertions and claim only that we learn word vectors, which express word’s replaceability in sentences based on their Euclidean distances. Synonyms are a subgroup of words which can replace each other, and we will use them to show differences between training on words that appear close to each other in a local window and training that uses distances between words, which we use in this study. Using ConceptNet 5.5.0’s synonyms, we show that word vectors trained on word distances create higher contrast in distributions of word similarities than was done with Glove, where only word appearances close to each other were engaged. We introduce a measure, which looks at intersection of histograms of word distances for synonyms and non-synonyms.