Introduction to Gaussian processes

  • Guyon I
  • Elisseeff A
  • Beck J
 et al. 
  • 1

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract

Feedforward neural networks such as multilayer perceptrons are popular tools for nonlinear regression and classification problems. From a Bayesian perspective, a choice of a neural network model can be viewed as defining a prior probability distribution over non-linear functions, and the neural network's learning process can be interpreted in terms of the posterior probability distribution over the unknown function. (Some learning algorithms search for the function with maximum posterior probability and other Monte Carlo methods draw samples from this posterior probability). In the limit of large but otherwise standard networks, Neal (1996) has shown that the prior distribution over non-linear functions implied by the Bayesian neural network falls in a class of probability distributions known as Gaussian processes. The hyperparameters of the neural network model determine the characteristic length scales of the Gaussian process. Neal's observation motivates the idea of discarding parameterized networks and working directly with Gaussian processes. Computations in which the parameters of the network are optimized are then replaced by simple matrix operations using the covariance matrix of the Gaussian process. In this chapter I will review work on this idea by Williams and Rasmussen (1996), Neal (1997), Barber and Williams (1997) and Gibbs and MacKay (1997), and will assess whether, for supervised regression and classification tasks, the feedforward network has been superceded.

Author-supplied keywords

  • 0545 Computational Geophysics: Modeling (1952
  • 0550 Computational Geophysics: Model verification
  • 0555 Computational Geophysics: Neural networks
  • 2447 Ionosphere: Modeling and forecasting
  • 425
  • A-PRIORI DISTINCTIONS
  • ALGORITHMS
  • Algorithms
  • Alpha-factor model
  • Anisotropy
  • Area metric
  • Artificial Intelligence
  • Augmented covariance matrix
  • BOOTSTRAP
  • Bandwidth selection
  • Bayes Theorem
  • Bayesia
  • Bayesian hierarchical model
  • Bayesian inference
  • Bayesian statistics
  • Biochemical oxygen demand
  • Boundary bias
  • CHOICE
  • Common-cause failure
  • Computational
  • Computational simulation
  • Conjugate prior
  • Data sharpening
  • Diffusion equation
  • Diffusive transport
  • Entropy
  • Environmental monitoring
  • Epistemic uncertainty
  • Error propagation
  • Galerkin projection
  • Gaussian process
  • Gaussian process regression
  • Gaussian processes
  • Heat kernel
  • Hypothesis testing
  • INFERENCE
  • Imprecise Dirichlet model
  • Information-Theoretic Learning with
  • Information-Theoretic Learning with Statistics
  • Inverse problems
  • Kernel discriminant analysis
  • Kernel partial least squares
  • LEARNING
  • LINEAR-MODEL
  • Langevin process
  • Learning/Statistics & Optimisation
  • Linear Models
  • Local model network
  • MEMS
  • MONTE-CARLO METHODS
  • Markov chain
  • Markov chain Monte Carlo
  • Maximum likelihood estimation
  • Model combination
  • Model fine-tuning
  • Model form uncertainty
  • Model prediction validation
  • Model probability
  • Model selection
  • Model validation
  • Modeling
  • Models
  • Monte Carlo
  • Network structure
  • Nonlinear system identification
  • Nonparametric
  • Nonparametric density estimation
  • Normal Distribution
  • Normal reference rules
  • Ocean modeling
  • Online learning algorithm
  • Optimal interpolation
  • Polynomial chaos
  • Predictive capability
  • Probabilistic forecasting
  • Radial basis function network
  • Regression Analysis
  • Reliability
  • SELECTION
  • Solids transport
  • Sparsification
  • Spatial deformation
  • Spectral methods
  • Statistical
  • Statistics
  • Structural models
  • Support vector classification
  • Support vector regression
  • Theory & Algorithms
  • Thermal challenge problem
  • Threshold velocity
  • UTILITY
  • Uncertainty quantification
  • Validation
  • Variable bandwidth
  • Verification
  • Water quality
  • Wind energy
  • analysis
  • and users of these
  • andrew would be delighted
  • blind system identification
  • calibration
  • computational experiment
  • e to other teachers
  • epistemic uncertainty
  • evidence theory
  • feel free
  • fu
  • gaussian process
  • hidden markov model
  • icle
  • if you found this
  • imprecise
  • imprecise probability
  • incremental learning
  • interval analysis
  • irradiation
  • kinect
  • long-span bridges
  • material useful in giving
  • model verification and validation
  • multi-level codes
  • multiple input motions
  • nonparametric Bayes
  • or to modify them
  • posture reconstruction
  • sensitivity
  • simulation
  • slides
  • source
  • spatial
  • time-frequency distribution
  • to fit
  • to use these slides
  • total electron content
  • uncertainty distribution
  • uncertainty quantification
  • verbatim
  • wave passage effects
  • your own lectures

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Isabelle Guyon

  • André Elisseeff

  • J L Beck

  • S K Au

  • Matthias C M Troffaes

  • Gero Walter

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free