Header logo is

Kernel Topic Models


Conference Paper



Latent Dirichlet Allocation models discrete data as a mixture of discrete distributions, using Dirichlet beliefs over the mixture weights. We study a variation of this concept, in which the documents' mixture weight beliefs are replaced with squashed Gaussian distributions. This allows documents to be associated with elements of a Hilbert space, admitting kernel topic models (KTM), modelling temporal, spatial, hierarchical, social and other structure between documents. The main challenge is efficient approximate inference on the latent Gaussian. We present an approximate algorithm cast around a Laplace approximation in a transformed basis. The KTM can also be interpreted as a type of Gaussian process latent variable model, or as a topic model conditional on document features, uncovering links between earlier work in these areas.

Author(s): Hennig, P. and Stern, D. and Herbrich, R. and Graepel, T.
Book Title: Fifteenth International Conference on Artificial Intelligence and Statistics
Volume: 22
Pages: 511--519
Year: 2012
Day: 0

Series: JMLR Proceedings
Editors: Lawrence, N. D. and Girolami, M.
Publisher: JMLR.org

Department(s): Empirical Inference, Probabilistic Numerics
Bibtex Type: Conference Paper (inproceedings)

Event Name: AISTATS 2012
Event Place: La Palma, Canary Islands, Spain

Links: PDF


  title = {Kernel Topic Models},
  author = {Hennig, P. and Stern, D. and Herbrich, R. and Graepel, T.},
  booktitle = {Fifteenth International Conference on Artificial Intelligence and Statistics},
  volume = {22},
  pages = {511--519},
  series = {JMLR Proceedings},
  editors = {Lawrence, N. D. and Girolami, M.},
  publisher = {JMLR.org},
  year = {2012}