Skip to main content
Springer Nature Link
Log in

Improve relation extraction with dual attention-guided graph convolutional networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

To better learn the dependency relationship between nodes, we address the relationship extraction task by capturing rich contextual dependencies based on the attention mechanism, and using distributional reinforcement learning to generate optimal relation information representation. This method is called Dual Attention Graph Convolutional Network (DAGCN), to adaptively integrate local features with their global dependencies. Specifically, we append two types of attention modules on top of GCN, which model the semantic interdependencies in spatial and relational dimensions, respectively. The position attention module selectively aggregates the feature at each position by a weighted sum of the features at all positions of nodes internal features. Meanwhile, the relation attention module selectively emphasizes interdependent node relations by integrating associated features among all nodes. We sum the outputs of the two attention modules and use reinforcement learning to predict the classification of nodes relationship to further improve feature representation which contributes to more precise extraction results. The results on the TACRED and SemEval datasets show that the model can obtain more useful information for relational extraction tasks, and achieve better performances on various evaluation indexes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Yu M, Yin W, Hasan KS, dos Santos CN, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th annual meeting of the association for computational linguistics (ACL), pp 571–581

  2. Zhang Y, Zhong V, Chen D, Angeli G, Manning CD (2017) Position-aware attention and supervised data improve slot filling. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 35–45

  3. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of the 25th international conference on computational linguistics (COLING), pp 2335–2344

  4. Wang L, Cao Z, de Melo G, Liu Z (2016) Relation classification via multi-level attention cnns. In: Proceedings of the 54th annual meeting of the association for computational linguistics (ACL)

  5. Peng N, Poon H, Quirk C, Toutanova K, Yih W (2017) Cross-sentence n-ary relation extraction with graph lstms. Transact Assoc Computat Linguist 5:101–115

    Article  Google Scholar 

  6. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 conference on empirical methods in natural language processing (EMNLP), pp 2205–2215

  7. Das P, Das AK, Nayak J, Pelusi D, Ding W (2019) A graph based clustering approach for relation extraction from crime data. IEEE Access 7:101269–101282

    Article  Google Scholar 

  8. Priyanka D, Das AK, Nayak J, Pelusi D (2019) A framework for crime data analysis using relationship among named entities. Neural Comput Appl 32:7671–7689

    Google Scholar 

  9. Xu K, Feng Y, Huang S, Zhao D (2015) Semantic relation classification via convolutional neural networks with simple negative sampling. In: Proceedings of the 2015 conference on empirical methods in natural language processing (EMNLP), pp 536–540

  10. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. In: Proceedings of the 54th annual meeting of the association for computational linguistics (ACL)

  11. Guo Z, Zhang Y, Lu W (2019) Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics (ACL), pages 241–251

  12. Zhu H, Lin Y, Liu Z, Fu J, Chua TS, Sun M (2019) Graph neural networks with generated parameters for relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics (ACL), pp 1331–1339

  13. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the advances in neural information processing systems 30: annual conference on neural information processing Systems (NIPS), pp 5998–6008

  14. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representations (ICLR)

  15. Dong J, Sun L, Feng Y, Huang R (2007) Chinese automatic entity relation extraction. J Chin Inf Process 21:80–91

    Google Scholar 

  16. Zeng D, Liu K, Chen Y, Zhao J (2015) Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of the 2015 conference on empirical methods in natural language processing (EMNLP), pp 1753–1762

  17. Nguyen TH, Grishman R (2015) Relation extraction: perspective from convolutional neural networks. In: Proceedings of the 1st workshop on vector space modeling for natural language processing (NAACL), pp 39–48

  18. Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th annual meeting of the association for computational linguistics (ACL)

  19. Le P, Titov I (2018) Improving entity linking by modeling latent relations between mentions. In: Proceedings of the 56th annual meeting of the association for computational linguistics (ACL), pp 1595–1604

  20. Zeng W, Lin Y, Liu Z, Sun M (2017) Incorporating relation paths in neural relation extraction. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 1768–1777

  21. Marcheggiani D, Titov I (2017) Encoding sentences with graph convolutional networks for semantic role labeling. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 1506–1515

  22. Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Proceedings of the semantic Web—15th international conference (ESWC), pages 593–607

  23. De Cao N, Aziz W, Titov I (2019) Question answering by reasoning across documents with graph convolutional networks. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies (NAACL-HLT), pp 2306–2317

  24. Zhang H, Goodfellow IJ, Metaxas DN, Odena A (2019) Self-attention generative adversarial networks. In: Proceedings of the 36th international conference on machine learning (ICML), pp 7354–7363

  25. Fu TJ, Li PH, Ma WY (2019) Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics (ACL), pp 1409–1418

  26. Bellemare MG, Dabney W, Munos R (2017) A distributional perspective on reinforcement learning. In: Proceedings of the 34th international conference on machine learning (ICML), pp 449–458

  27. Dabney W, Kurth-Nelson Z, Uchida N (2020) A distributional code for value in dopamine-based reinforcement learning. Nature 577:671–675

    Article  Google Scholar 

  28. Santoro A, Raposo D, Barrett DGT, Malinowski M, Pascanu R, Battaglia PW, Lillicrap T (2017) A simple neural network module for relational reasoning. In: Proceedings of the advances in neural information processing systems 30: annual conference on neural information processing systems (NIPS), pp 4967–4976

  29. Lee K, He L, Lewis M, Zettlemoyer L (2017) End-to-end neural coreference resolution. In: Proceedings of the 2017 conference on empirical methods in natural language processing (EMNLP), pp 188–197

  30. Hendrickx I, Kim SN, Kozareva Z, Nakov P, Ó Séaghdha D, Padó S, Pennacchiotti M, Romano L, Szpakowicz S (2010) Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 5th international workshop on semantic evaluation (SEMEVAL), pp 33–38

  31. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  32. Xu Y, Mou L, Li G, Chen Y, Peng H, Jin Z (2015) Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 conference on empirical methods in natural language processing (EMNLP), pp 1785–1794

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 61966004, 61663004, 61967002, 61866004, 61762078), the Guangxi Natural Science Foundation (Nos. 2019GXNSFDA245018, 2018GXNSFDA281009, 2017GXNSFAA198365, 2016GXNSFAA380146), the Guangxi “Bagui Scholar” Teams for Innovation and Research Project, the Guangxi Talent Highland Project of Big Data Intelligence and Application, and Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing.

Author information

Authors and Affiliations

  1. Guangxi Key Lab of Multi-source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China

    Zhixin Li, Yaru Sun, Jianwei Zhu, Suqin Tang & Canlong Zhang

  2. College of Computer Science and Engineering, Northwest Normal University, Lanzhou, 730070, China

    Huifang Ma

Authors
  1. Zhixin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaru Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianwei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suqin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Canlong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huifang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhixin Li.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work. There is no professional or other personal interest of any nature or kind in any product, service, and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Sun, Y., Zhu, J. et al. Improve relation extraction with dual attention-guided graph convolutional networks. Neural Comput & Applic 33, 1773–1784 (2021). https://doi.org/10.1007/s00521-020-05087-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05087-z

Keywords

Search

Navigation