Mutual Information

Measures how much information an observable random variable carries about unknown parameters.

Estimating parameters by maximizing likelihood of observed data.

Updating beliefs about parameters using observed evidence and prior distributions.

Bayesian parameter estimation using the mode of the posterior distribution.

Reduction in uncertainty achieved by observing a variable; used in decision trees and active learning.

A measure of randomness or uncertainty in a probability distribution.

Measures divergence between true and predicted probability distributions.

Measures how one probability distribution diverges from another.