关键词: 领域知识, 上下文学习, 高效参数微调, 大模型

Abstract: With the enhancement of large-scale language model capabilities, how to handle long text understanding has become a key issue facing us. At present, there are strategies such as reducing the length of input or expanding the attention window size of models to alleviate the difficulty of understanding long texts. However, in specific knowledge domains, it is difficult to achieve the expected results of understanding long texts without fine-tuning using domain-specific data, which can easily lead to problems such as illusionary answers and missing key information. In order to address these issues, a model fine-tuning strategy that enhances the capture of contextual location information has been proposed, which uses efficient parameter fine-tuning to more accurately capture professional terms and concepts in the home appliance field, aggregating important details from long texts. Finally, by validating on the instruction manual dataset in the home appliance field, the fine-tuning method was evaluated using retrieval-enhanced generation technology for its ability to understand and generate long text content. This strategy improved accuracy by about 5% compared to baseline testing.

Key words: Domain knowledge, Contextual learning, Efficient parameter fine-tuning, Large model