Alignment

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

A discipline ensuring AI systems are fair, safe, transparent, privacy-preserving, and accountable throughout lifecycle.

Automated detection/prevention of disallowed outputs (toxicity, self-harm, illegal instruction, etc.).

Systematic differences in model outcomes across groups; arises from data, labels, and deployment context.

Techniques to understand model decisions (global or local), important in high-stakes and regulated settings.

Studying internal mechanisms or input influence on outputs (e.g., saliency maps, SHAP, attention analysis).

Breaking documents into pieces for retrieval; chunk size/overlap strongly affect RAG quality.

Constraining outputs to retrieved or provided sources, often with citation, to improve factual reliability.

Fine-tuning on (prompt, response) pairs to align a model with instruction-following behaviors.

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.