Maximum Likelihood Estimation

Updating beliefs about parameters using observed evidence and prior distributions.

Bayesian parameter estimation using the mode of the posterior distribution.

Optimization problems where any local minimum is global.

Optimization with multiple local minima/saddle points; typical in neural networks.

Measures divergence between true and predicted probability distributions.

Measures how one probability distribution diverges from another.

Quantifies shared information between random variables.

Measures how much information an observable random variable carries about unknown parameters.