Skip to main content

Advertisement

Springer Nature Link
Log in

Learning on heterogeneous graph neural networks with consistency-based augmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Heterogeneous Graph Neural Networks(HGNNs), as an effective tool for mining heterogeneous graphs, have achieved remarkable performance on series of real-world applications. Yet, HGNNs are limited in their mining power as they require all nodes to have complete and reliable attributes. It is usually unrealistic since the attributes of many nodes in reality are inevitably missing or noisy. Existing methods usually take imputation schemes to complete missing attributes, in which topology information is ignored, leading to suboptimal performance. In this work, we study the consistency-based augmentation on heterogeneous graphs, completing the missing attributes and improving original attributes simultaneously, and propose a novel generic architecture−Learning on Heterogeneous Graph Neural Networks with Consistency-based Augmentation(CAHGNN), including random sampling, attribute augmentation and consistency training. In graph augmentation, to ensure attributes sensible and accurate, the attention mechanism is adopted to complete attributes under the guidance of the topological relationship between nodes. Extensive experiments on three benchmark datasets demonstrate the superior performance of CAHGNN over state-of-the-art baselines on semi-supervised node classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The authors confirm that the data supporting the findings of this study are available within the article.

Notes

  1. https://pan.baidu.com/s/1KW-elfyw9kCaK76FRTLRKg (password:data).

  2. https://dblp.uni-trier.de/

  3. https://www.imdb.com/

References

  1. Yang C, Xiao Y, Zhang Y, Sun Y, Han J (2022) Heterogeneous network representation learning: A unified framework with survey and benchmark. IEEE Trans Knowl Data Eng 34(10):4854–4873

    Article  Google Scholar 

  2. Shi C, Li Y, Zhang J, Sun Y, Philip SY (2016) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37

    Article  Google Scholar 

  3. Shen W, Han J, Wang J, Yuan X, Yang Z (2017) Shine+: A general framework for domain-specific entity linking with heterogeneous information networks. IEEE Trans Knowl Data Eng 30(2):353–366

    Article  Google Scholar 

  4. Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. In: The world wide web conference, pp 2022–2032

  5. Yun S, Jeong M, Kim R, Kang J, Kim HJ (2019) Graph transformer networks. Advances in neural information processing systems 32

  6. Fu X, Zhang J, Meng Z, King I (2020) Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. Proceedings of the web conference 2020:2331–2341

    Google Scholar 

  7. Zhang C, Song D, Huang C, Swami A, Chawla NV (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining, pp 793–803

  8. Fan S, Zhu J, Han X, Shi C, Hu L, Ma B, Li Y (2019) Metapath-guided heterogeneous graph neural network for intent recommendation. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining, pp 2478–2486

  9. Li A, Qin Z, Liu R, Yang Y, Li D (2019) Spam review detection with graph convolutional networks. In: Proceedings of the 28th ACM International conference on information and knowledge management, pp 2703–2711

  10. Yang T, Hu L, Shi C, Ji H, Li X, Nie L (2021) Hgat: Heterogeneous graph attention networks for semi-supervised short text classification. ACM Transactions on Information Systems (TOIS) 39(3):1–29

    Article  Google Scholar 

  11. Sun Y, Han J (2013) Mining heterogeneous information networks: a structural analysis approach. Acm Sigkdd Explorations Newsletter 14(2):20–28

    Article  Google Scholar 

  12. Jin D, Huo C, Liang C, Yang L (2021) Heterogeneous graph neural network via attribute completion. Proceedings of the web conference 2021:391–400

    Google Scholar 

  13. He D, Liang C, Huo C, Feng Z, Jin D, Yang L, Zhang W (2022) Analyzing heterogeneous networks with missing attributes by unsupervised contrastive learning. IEEE Transactions on Neural Networks and Learning Systems

  14. Xie Q, Dai Z, Hovy E, Luong T, Le Q (2020) Unsupervised data augmentation for consistency training. Adv Neural Inf Process Syst 33:6256–6268

    Google Scholar 

  15. Dong Y, Chawla NV, Swami A (2017) metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International conference on knowledge discovery and data mining, pp 135–144

  16. Fu T-y, Lee W-C, Lei Z (2017) Hin2vec: Explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 2017 ACM on Conference on information and knowledge management, pp 1797–1806

  17. Shang J, Qu M, Liu J, Kaplan LM, Han J, Peng J (2016) Meta-path guided embedding for similarity search in large-scale heterogeneous information networks. arXiv:1610.09769

  18. Wang X, Zhang Y, Shi C (2019) Hyperbolic heterogeneous information network embedding. Proceedings of the AAAI Conference on artificial intelligence 33:5337–5344

    Article  Google Scholar 

  19. Hu Z, Dong Y, Wang K, Sun Y (2020) Heterogeneous graph transformer. Proceedings of the web conference 2020:2704–2710

    Google Scholar 

  20. You J, Ma X, Ding Y, Kochenderfer MJ, Leskovec J (2020) Handling missing data with graph representation learning. Adv Neural Inf Process Syst 33:19075–19087

    Google Scholar 

  21. Chen X, Chen S, Yao J, Zheng H, Zhang Y, Tsang IW (2020) Learning on attribute-missing graphs. IEEE transactions on pattern analysis and machine intelligence

  22. Wang Y, Wang W, Liang Y, Cai Y, Liu J, Hooi B (2020) Nodeaug: Semi-supervised node classification with data augmentation. In: Proceedings of the 26th ACM SIGKDD International conference on knowledge discovery & data mining, pp 207–217

  23. Feng W, Zhang J, Dong Y, Han Y, Luan H, Xu Q, Yang Q, Kharlamov E, Tang J (2020) Graph random neural networks for semi-supervised learning on graphs. Adv Neural Inf Process Syst 33:22092–22103

    Google Scholar 

  24. Liu S, Ying R, Dong H, Li L, Xu T, Rong Y, Zhao P, Huang J, Wu D (2022) Local augmentation for graph neural networks. In: Proceedings of the 39th International conference on machine learning, vol 162, pp 14054–14072

  25. Zhao T, Liu Y, Neves L, Woodford O, Jiang M, Shah N (2021) Data augmentation for graph neural networks. Proceedings of the AAAI Conference on artificial intelligence 35:11015–11023

  26. Ding K, Xu Z, Tong H, Liu H (2022) Data augmentation for deep graph learning: A survey. In: Proceedings of the 28th ACM SIGKDD International conference on knowledge discovery & data mining, vol 24, pp 61–77

  27. Xu Z, Du B, Tong H (2022) Graph sanitation with application to node classification. Proc ACM Web conference 2022:1136–1147

    Google Scholar 

  28. Taguchi H, Liu X, Murata T (2021) Graph convolutional networks for graphs containing missing features. Future Gener Comput Syst 117:155–168

    Article  Google Scholar 

  29. Rossi E, Kenlay H, Gorinova MI, Chamberlain BP, Dong X, Bronstein MM (2022) On the unreasonable effectiveness of feature propagation in learning on graphs with missing node features. In: Proceedings of the 1st Learning on graphs conference, vol 198, pp 11–11116

  30. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: International conference on learning representations

  31. Lee J, Lee I, Kang J (2019) Self-attention graph pooling. In: International conference on machine learning, pp 3734–3743

  32. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

  33. McPherson M, Smith-Lovin L, Cook JM (2001) Birds of a feather: Homophily in social networks. Annual review of sociology, pp 415–444

  34. Pei H, Wei B, Chang KC, Lei Y, Yang B (2020) Geom-gcn: Geometric graph convolutional networks. In: International conference on learning representations

  35. Schlichtkrull M, Kipf TN, Bloem P, Berg Rvd, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, pp 593–607 . Springer

  36. Huang X, Song Q, Li Y, Hu X (2019) Graph recurrent networks with attributed random walks. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining, pp 732–740

  37. Mikolov T, Chen K, Corrado GS, Dean J (2013) Efficient estimation of word representations in vector space. In: International conference on learning representations

  38. Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA (2019) Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems 32

  39. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press

    MATH  Google Scholar 

  40. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations

  41. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: International conference on learning representations

  42. Suykens JA (2001) Support vector machines: a nonlinear modelling and control perspective. Eur J Control 7(2–3):311–327

    Article  MATH  Google Scholar 

  43. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. Journal of machine learning research 9(11)

Download references

Acknowledgements

This work is supported by the Fundamental Research Funds for the Central Universities, China under Grant 2021III030JC.

Funding

This work is supported by the Fundamental Research Funds for the Central Universities, China under Grant 2021III030JC.

Author information

Authors and Affiliations

  1. School of Science, Wuhan University of Technology, 122 Luoshi Road, Wuhan, 430070, Hubei, China

    Yixuan Liang & Yuan Wan

Authors
  1. Yixuan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Liang Yixuan. The first draft of the manuscript was written by Liang Yixuan, Wan Yuan commented on previous versions of the manuscript and critically revised the work. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yuan Wan.

Ethics declarations

Conflicts of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Y., Wan, Y. Learning on heterogeneous graph neural networks with consistency-based augmentation. Appl Intell 53, 27624–27636 (2023). https://doi.org/10.1007/s10489-023-04995-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04995-6

Keywords

Search

Navigation