Why do Transformers work so well?

In our invited talk at the CALCULUS Symposium at KU Leuven, we present our work on reinterpreting the latent representations of Transformers as nonparametric mixture distributions, and training a variational Bayesian version of Transformers on natural language.  We argue that the empirical success of Transformers is evidence that natural language understanding is nonparametric variational Bayesian inference of mixture distributions.