RLHF

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.

Ensuring model behavior matches human goals, norms, and constraints, including reducing harmful or deceptive outputs.

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

Average of squared residuals; common regression objective.

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

Popular optimizer combining momentum and per-parameter adaptive step sizes via first/second moment estimates.