Results for "training loss"

Randomly zeroing activations during training to reduce co-adaptation and overfitting.

Training objective where the model predicts the next token given previous tokens (causal modeling).

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

Ordering training samples from easier to harder to improve convergence or generalization.

Expanding training data via transformations (flips, noise, paraphrases) to improve robustness.

Training across many devices/silos without centralizing raw data; aggregates updates, not data.

Ability to replicate results given same code/data; harder in distributed training and nondeterministic ops.

Hardware resources used for training/inference; constrained by memory bandwidth, FLOPs, and parallelism.

Training a smaller “student” model to mimic a larger “teacher,” often improving efficiency while retaining performance.

Maliciously inserting or altering training data to implant backdoors or degrade performance.

Tendency to trust automated suggestions even when incorrect; mitigated by UI design, training, and checks.

Error due to sensitivity to fluctuations in the training dataset.

Adjusting learning rate over training to improve convergence.

Gradually increasing learning rate at training start to avoid divergence.

Prevents attention to future tokens during training/inference.

Recovering training data from gradients.

Inferring sensitive features of training data.

Models that learn to generate samples resembling training data.

Flat high-dimensional regions slowing training.

Model behaves well during training but not deployment.

Differences between training and inference conditions.

Artificial environment for training/testing agents.

Differences between training and deployed patient populations.

Combining simulation and real-world data.