Header logo is

Sparse Gaussian Processes: inference, subspace identification and model selection


Conference Paper


Gaussian Process (GP) inference is a probabilistic kernel method where the GP is treated as a latent function. The inference is carried out using the Bayesian online learning and its extension to the more general iterative approach which we call TAP/EP learning. Sparsity is introduced in this context to make the TAP/EP method applicable to large datasets. We address the prohibitive scaling of the number of parameters by defining a subset of the training data that is used as the support the GP, thus the number of required parameters is independent of the training set, similar to the case of ``Support--‘‘ or ``Relevance--Vectors‘‘. An advantage of the full probabilistic treatment is that allows the computation of the marginal data likelihood or evidence, leading to hyper-parameter estimation within the GP inference. An EM algorithm to choose the hyper-parameters is proposed. The TAP/EP learning is the E-step and the M-step then updates the hyper-parameters. Due to the sparse E-step the resulting algorithm does not involve manipulation of large matrices. The presented algorithm is applicable to a wide variety of likelihood functions. We present results of applying the algorithm on classification and nonstandard regression problems for artificial and real datasets.

Author(s): Csato, L. and Opper, M.
Journal: Proceedings
Pages: 1-6
Year: 2003
Month: August
Day: 0
Editors: Van der Hof, , Wahlberg

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: 13th IFAC Symposium on System Identifiaction
Event Place: Rotterdam

Address: The Netherlands
Digital: 0
Institution: MPI for Biological Cybernetics, Tuebingen
Note: electronical version; Index ThA02-2
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF


  title = {Sparse Gaussian Processes: inference, subspace identification and model selection},
  author = {Csato, L. and Opper, M.},
  journal = {Proceedings},
  pages = {1-6},
  editors = {Van der Hof,  ,  Wahlberg},
  organization = {Max-Planck-Gesellschaft},
  institution = {MPI for Biological Cybernetics, Tuebingen},
  school = {Biologische Kybernetik},
  address = {The Netherlands},
  month = aug,
  year = {2003},
  note = {electronical version; Index ThA02-2},
  month_numeric = {8}