Investigation of attention mechanisms of BERT and other language models – neural and otherwise – have found that these models are often, to one degree or another, implicitly learning details of syntactic structure and other linguistic features in the performance of various natural language processing tasks (Clark et al. 2019, Goldberg 2019, Shi et al. 2016, inter alia).
While this might be heartening news to linguists, it raises the questions of whether these frameworks might be partially reinventing the wheel with respect to the scientific knowledge of language, and also of what further linguistic structures and phenomena might be exploited to supplement the statistical models.
In this talk, I discuss various ways of leveraging linguistic structure to improve and expand NLP performance, particularly in a pipeline architecture, with a focus on experience data in the healthcare domain.
