Bayesian Inference

Describes likelihoods of random variable outcomes.

Bayesian parameter estimation using the mode of the posterior distribution.

Optimization problems where any local minimum is global.

Optimization with multiple local minima/saddle points; typical in neural networks.

A point where gradient is zero but is neither a max nor min; common in deep nets.

Measures how one probability distribution diverges from another.

Quantifies shared information between random variables.

Measures how much information an observable random variable carries about unknown parameters.

Estimating parameters by maximizing likelihood of observed data.