Applied Bayesian inference for diachronic meaning change

Thesis

As a language evolves, the meanings or senses of many words in the language change. Examples include "gay", whose predominant sense has changed from bright or cheerful to homosexual; and "mouse", which has acquired a new sense of a computer pointing device in addition to the rodent sense. Modelling words with multiple senses, and learning their diachronic meaning changes from unlabelled text, is a fascinating challenge in statistical inference. One way to approach the problem is through a class of generative Bayesian models derived from the topic modelling literature. In this framework, the sense of a target word is represented as a distribution over context words, and sense prevalence is represented as a distribution over senses, both of which may change with time. This thesis works within this framework to posit new models, model-fitting procedures and inference methods for unsupervised learning of word senses and measurement of diachronic meaning change. Quantifying inferential uncertainty is a particular focus, since this aspect is important for modelling the small and sparse datasets used in our main application. Significant gains are achieved in terms of predictive accuracy, ground-truth recovery, sampling efficiency and scalability. An intuitive method for selecting the learning rate in a generalised Bayes' posterior is also explored. All results are demonstrated on real data from ancient Greek and English, as well as simulated examples.

Authors: Zafar, S

• DOI: 10.5287/ora-jbk2ap7zg
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: sense change models; Bayesian inference; diachronic lexical semantics; Markov Chain Monte Carlo (MCMC)
• Subjects: Statistics