Empirical Risk Minimization

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

Average of squared residuals; common regression objective.

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

Configuration choices not learned directly (or not typically learned) that govern training or architecture.

A scalar measure optimized during training, typically expected loss over data, sometimes with regularization terms.