Sequential Domain Adaptation on Heterogeneous Clinical Resources for Biomedical NER

Kurzfassung

In the field of Natural Language Processing (NLP), there is a high abundance of biomedical and clinical datasets. However, finding a specific dataset tailored to a clinical sub-domain for information extraction, model training, and evaluation remains a significant challenge. This scarcity of domain-specific data hinders the ability of models to learn underlying patterns effectively, limiting their performance and generalization. To address this issue, we propose leveraging sub-domains of clinical resources to enrich pre-trained models with task-specific knowledge through fine-tuning. In particular, our paper focuses on using continuous learning for transfering knowledge from pre-trained models across various sub-domains of biomedical resources. Using this approach, we develop an adaptable named-entity-recognition (NER) model which extracts and identifies biomedical entities across different resources. The focus of our research concerns with three core areas in biomedical NLP: scientific papers, clinical trials, and patient profiles. We use sequential fine-tuning with layer-freezing to mitigate catastrophic forgetting, ensuring that knowledge from previously learned sub-domains is retained while adapting to new ones. Additionally, we provide empirical validation on three diverse biomedical sub-domains, demonstrating the effectiveness of our approach.