李正光;陈恒;林鸿飞;

• 1:大连外国语大学语言智能研究中心
• 2:大连理工大学计算机科学与技术学院

摘要(Abstract)：

与服药相关的社交文本中隐藏着更具时效和更广泛的药物不良反应信息，但是从相对短小、稀疏的社交短文本中提取药物不良反应非常困难。基于此，本文提出一种双向语言预训练模型和注意力机制相结合的神经网络识别方法。该方法利用双向字符级语言预训练模型提取特定字符级特征，而且在提取药物不良反应的同时，通过注意力机制捕获局部和全局语义上下文信息。此外，为了提高该方法的效率，将字符级特征与词级特征相结合，并采用词级预训练和字符级预训练模型代替协同训练。在PSB 2016社交媒体挖掘共享任务2中的实验结果表明，字符特征在形态学上有助于区分药物不良反应，而注意力机制通过捕获局部和全局语义信息提高了对药物不良反应的识别性能，宏平均F_1值为82.2%。

关键词(KeyWords)： 药物不良反应;社交媒体;双向语言模型;注意力机制;预训练模型

Abstract:

Keywords:

基金项目(Foundation): 国家自然科学基金(61806038);; 辽宁省高等学校创新人才项目(WR2019005);; 辽宁省教育厅科学研究经费项目(2020JYT03);; 教育部人文社科项目(18YJCZH2018);; 大连外国语大学科研基金(2021XJYB16,2021XJYB19)

作者(Authors): 李正光;陈恒;林鸿飞;

DOI: 10.16088/j.issn.1001-6600.2021091503

参考文献(References)：

• [1] SARKER A,BELOUSOV M,FRIEDRICHS J,et al.Data and systems for medication-related text classification and concept normalization from Twitter:insights from the Social Media Mining for Health(SMM4H)-2017 shared task[J].Journal of the American Medical Informatics Association,2018,25(10):1274-1283.DOI:10.1093/jamia/ocy114. [2] 朱晓旭，林鸿飞，曾泽渊.基于社交媒体的药物不良反应检测[J].山西大学学报(自然科学版),2020,43(1):14-21. [3] BENTON A,UNGAR L,HILL S,et al.Identifying potential adverse effects using the web:a new approach to medical hypothesis generation[J].Journal of Biomedical Informatics,2011,44(6):989-996. [4] 张亚飞，于琦，王于心，等.基于药物论坛中潜在不良反应与适应症的知识发现体系构建[J].中华医学图书情报杂志，2020,29(7):38-43. [5] ZHANG Y,CUI S,GAO H.Adverse drug reaction detection on social media with deep linguistic features[J].Journal of Biomedical Informatics,2020,106:103437. [6] 许力，李建华.基于BERT和BiLSTM-CRF的生物医学命名实体识别[J].计算机工程与科学，2021,43(10):1873-1879. [7] LUO L,YANG Z,YANG P,et al.An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition[J].Bioinformatics.2018,34(8):1381-1388. [8] ZHANG T,LIN H,REN Y,et al.Identifying adverse drug reaction entities from social media with adversarial transfer learning model[J].Neurocomputing,2021,45:254-262. [9] LI Z,YANG Z,WANG L,et al.Lexicon knowledge boosted interaction graph network for adverse drug reaction recognition from social media[J].IEEE Journal of Biomedical and Health Informatics,2021,25(7):2777-2786. [10] 佘朝阳，严馨，徐广义，等.基于数据增强和半监督学习的药物不良反应检测[J/OL].计算机工程[2021-10-15].https://doi.org/10.19678/j.issn.1000-3428.0062170. [11] SAHU S K,ANAND A.Recurrent neural network models for disease name recognition using domain invariant features[C]∥ Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2016:2216-2225.DOI:10.18653/v1/P16-1209. [12] JAGANNATHA A,YU H.Structured prediction models for RNN based sequence labeling in clinical text[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2016:856-865.DOI:10.18653/v1/D16-1082. [13] PANDEY C,IBRAHIM Z,WU H H,et al.Improving RNN with attention and embedding for adverse drug reactions[C]// Proceedings of the 2017 International Conference on Digital Health.New York,NY:Association for Computing Machinery,2017:67-71.DOI:10.1145/3079452.3079501. [14] PENG Y F,YAN S K,LU Z Y.Transfer learning in biomedical natural language processing:an evaluation of BERT and ELMo on ten benchmarking datasets[C]// Proceedings of the 18th BioNLP Workshop and Shared Task.Stroudsburg,PA:Association for Computational Linguistics,2019:58-65.DOI:10.18653/v1/W19-5006. [15] 申晨，林鸿飞.基于图嵌入的社交媒体药物不良反应事件检测方法[J].大连理工大学学报，2020,60(5):547-554. [16] 宋雅文，杨志豪，罗凌，等.基于字符卷积神经网络的生物医学变异实体识别方法[J].中文信息学报，2021,35(5):63-69. [17] SARABADANI S.Detection of adverse drug reaction mentions in tweets using ELMo[C]// Proceedings of the 4th Social Media Mining for Health Applications(#SMM4H) Workshop & Shared Task.Stroudsburg,PA:Association for Computational Linguistics,2019:120-122.DOI:10.18653/v1/W19-3221. [18] SRIVASTAVA R K,GREFF K,SCHMIDHUBER J.Highway networks[EB/OL].(2015-11-03)[2021-09-15].http:// arxiv.org/abs/1505.00387.DOI:10.48550/arXiv.1505.00387. [19] DEWI I N,蔡晓玲，刘晓锋，等.结合类别关键词与注意力机制的药物相互关系抽取模型[J].华南理工大学学报(自然科学版),2021,49(1):10-17. [20] YANG Z C,YANG D Y,DYER C,et al.Hierarchical attention networks for document classification[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg,PA:Association for Computational Linguistics,2016:1480-1489.DOI:10.18653/v1/N16-1174. [21] 魏巍，傅维刚.面向社交媒体的细粒度ADR本体的半自动构建方法研究[J].图书情报工作，2019,63(3):108-114. [22] COCOS A,FIKS A G,MASINO A J.Deep learning for pharmacovigilance:Recurrent neural network architectures for labeling adverse drug reactions in Twitter posts[J].Journal of the American Medical Informatics Association,2017,24(4):813-821.DOI:10.1093/jamia/ocw180. [23] NIKFARJAM A,SARKER A,O’CONNOR K,et al.Pharmacovigilance from social media:mining adverse drug reaction mentions using sequence labeling with word embedding cluster features[J].Journal of the American Medical Informatics Association,2015,22(3):671-681.DOI:10.1093/jamia/ocu041. [24] NIKFARJAM A,GONZALEZ G H.Pattern mining for extraction of mentions of adverse drug reactions from user comments[J].AMIA Annual Symposium Proceedings,2011,2011:1019-1026. [25] LAI S,LIU K,XU L,et al.How to generate a good word embedding?[J].IEEE Intelligent Systems,2016,31(6):5-14. [26] DUCHI J,HAZAN E,SINGER Y.Adaptive Subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12:2121-2159. [27] CHOWDHURY S,ZHANG C W,YU P S.Multi-Task Pharmacovigilance Mining from Social Media Posts[C]// Proceedings of the 2018 World Wide Web Conference.Geneva,Switzerland:International World Wide Web Conferences Steering Committee,2018:117-126.DOI:10.1145/3178876.3186053.
• (1)https://github.com/myleott/ark-twokenize-py (2)https://github.com/loretoparisi/word2vec-twitter