Transfer Learning
IntermediateEnglish: Transfer Learning
Reusing knowledge from a source task/domain to improve learning on a target task/domain, typically via pretrained models.
Why It Matters
Transfer learning significantly reduces the time and resources needed to train models for new tasks, making it especially valuable in fields like healthcare, where labeled data can be hard to come by. Its ability to improve model performance with limited data has broad implications for advancing AI applications across various industries.
Definition
A machine learning technique that leverages knowledge gained from solving one problem (the source task) to improve the performance on a different but related problem (the target task). This is particularly effective when the target task has limited labeled data. The mathematical foundation often involves fine-tuning a pretrained model, typically a deep neural network, by adjusting its weights using a smaller dataset from the target domain. The process can be formalized as minimizing a loss function L(w, X_target, Y_target) where w are the model parameters, and (X_target, Y_target) are the input-output pairs from the target domain. Transfer learning is closely associated with domain adaptation techniques and has been successfully applied in areas such as natural language processing and computer vision, where large datasets are available for pretraining but limited data exists for specific applications.