GATv2-DDI: Structure Aware Graph Attention Networks for Drug–Drug Interaction Prediction, with a Leakage-Controlled EvaluationProtocol
DOI:
https://doi.org/10.5281/zenodo.22869994Keywords:
drug–drug interaction, graph attention network, GATv2, link prediction, molecular fingerprints, data leakage, pharmacovigilanceAbstract
Purpose of Study: This study develops GATv2-DDI, a structure-aware graph attention framework for drug–drug interaction prediction. It aims to combine molecular descriptors with graph-based representations while systematically evaluating the effects of transductive edge exposure and data leakage. The study also assesses whether graph attention provides meaningful predictive value beyond simpler feature-based learning approaches.
Methodology: A corpus of 1,090 approved small-molecule drugs was retrieved from ChEMBL and represented using 4,277-dimensional hybrid molecular descriptors. Structural-similarity proxy labels generated 1,098 balanced drug pairs. GATv2-DDI used a three-layer, eight-head GATv2 encoder with residual connections and an MLP decoder. Performance was evaluated using transductive and leakage-controlled protocols alongside logistic regression, random forest, and feature-only baselines.
Findings: The published GATv2-DDI checkpoint achieved an AUROC of 0.9678 and AUPRC of 0.955 under the transductive evaluation protocol. At the Youden optimal threshold, accuracy, F1, and MCC were 0.923, 0.923, and 0.845, respectively. However, a parameter-free edge-lookup rule achieved an AUROC of 1.0000, demonstrating that validation labels were exposed through message-passing edges. After validation edges were withheld, GATv2-DDI achieved an AUROC of 0.8598, compared with 0.9566 for the feature-only model. Logistic regression achieved 0.9998 AUROC, demonstrating that the similarity-derived labels were essentially linearly separable from molecular features.
Conclusion: GATv2-DDI provides a reproducible framework for graph-based drug-pair scoring and efficient prediction. However, its headline performance is substantially affected by transductive edge exposure and the structure-derived nature of the benchmark labels. Leakage-controlled evaluation and strong non-graph baselines are therefore essential. Clinically curated DDI datasets are needed to establish genuine clinical predictive utility.
Downloads
References
[1] D. M. Qato, J. Wilder, L. P. Schumm, V. Gillet, and G. C. Alexander, “Changes in prescription and over-the-counter medication and dietary supplement use among older adults in the United States, 2005 vs 2011,” JAMA Internal Medicine, vol. 176, no. 4, pp. 473–482, 2016, doi: 10.1001/jamainternmed.2015.8581.
[2] B. Guthrie, B. Makubate, V. Hernandez-Santiago, and T. Dreischulte, “The rising tide of polypharmacy and drug–drug interactions: Population database analysis 1995–2010,” BMC Medicine, vol. 13, p. 74, 2015, doi: 10.1186/s12916-015-0322-7.
[3] M. Pirmohamed et al., “Adverse drug reactions as cause of admission to hospital: Prospective analysis of 18 820 patients,” BMJ, vol. 329, no. 7456, pp. 15–19, 2004, doi: 10.1136/bmj.329.7456.15.
[4] L. Magro, U. Moretti, and R. Leone, “Epidemiology and characteristics of adverse drug reactions caused by drug–drug interactions,” Expert Opinion on Drug Safety, vol. 11, no. 1, pp. 83–94, 2012, doi: 10.1517/14740338.2012.631910.
[5] U. M. Zanger and M. Schwab, “Cytochrome P450 enzymes in drug metabolism: Regulation of gene expression, enzyme activities, and impact of genetic variation,” Pharmacology & Therapeutics, vol. 138, no. 1, pp. 103–141, 2013, doi: 10.1016/j.pharmthera.2012.12.007.
[6] S. Vilar, R. Harpaz, E. Uriarte, L. Santana, R. Rabadan, and C. Friedman, “Drug–drug interaction through molecular structure similarity analysis,” Journal of the American Medical Informatics Association, vol. 19, no. 6, pp. 1066–1074, 2012, doi: 10.1136/amiajnl-2012-000935.
[7] S. Vilar et al., “Similarity-based modeling in large-scale prediction of drug–drug interactions,” Nature Protocols, vol. 9, no. 9, pp. 2147–2163, 2014, doi: 10.1038/nprot.2014.151.
[8] A. Gottlieb, G. Y. Stein, Y. Oron, E. Ruppin, and R. Sharan, “INDI: A computational framework for inferring drug interactions and their associated recommendations,” Molecular Systems Biology, vol. 8, p. 592, 2012, doi: 10.1038/msb.2012.26.
[9] F. Cheng and Z. Zhao, “Machine learning-based prediction of drug–drug interactions by integrating drug phenotypic, therapeutic, chemical, and genomic properties,” Journal of the American Medical Informatics Association, vol. 21, no. e2, pp. e278–e286, 2014, doi: 10.1136/amiajnl-2013-002512.
[10] W. Zhang, Y. Chen, F. Liu, F. Luo, G. Tian, and X. Li, “Predicting potential drug–drug interactions by integrating chemical, biological, phenotypic and network data,” BMC Bioinformatics, vol. 18, p. 18, 2017, doi: 10.1186/s12859-016-1415-9.
[11] J. Y. Ryu, H. U. Kim, and S. Y. Lee, “Deep learning improves prediction of drug–drug and drug–food interactions,” Proceedings of the National Academy of Sciences, vol. 115, no. 18, pp. E4304–E4311, 2018, doi: 10.1073/pnas.1803294115.
[12] K. Huang, C. Xiao, T. N. Hoang, L. M. Glass, and J. Sun, “CASTER: Predicting drug interactions with chemical substructure representation,” in Proceedings of the AAAI conference on artificial intelligence, 2020, pp. 702–709. doi: 10.1609/aaai.v34i01.5412.
[13] A. K. Nyamabo, H. Yu, and J.-Y. Shi, “SSI–DDI: Substructure–substructure interactions for drug–drug interaction prediction,” Briefings in Bioinformatics, vol. 22, no. 6, p. bbab133, 2021, doi: 10.1093/bib/bbab133.
[14] Y. Deng, X. Xu, Y. Qiu, J. Xia, W. Zhang, and S. Liu, “A multimodal deep learning framework for predicting drug–drug interaction events,” Bioinformatics, vol. 36, no. 15, pp. 4316–4322, 2020, doi: 10.1093/bioinformatics/btaa501.
[15] M. Zitnik, M. Agrawal, and J. Leskovec, “Modeling polypharmacy side effects with graph convolutional networks,” Bioinformatics, vol. 34, no. 13, pp. i457–i466, 2018, doi: 10.1093/bioinformatics/bty294.
[16] X. Lin, Z. Quan, Z.-J. Wang, T. Ma, and X. Zeng, “KGNN: Knowledge graph neural network for drug–drug interaction prediction,” in Proceedings of the twenty-ninth international joint conference on artificial intelligence (IJCAI), 2020, pp. 2739–2745. doi: 10.24963/ijcai.2020/380.
[17] Y. Wang, Y. Min, X. Chen, and J. Wu, “Multi-view graph contrastive representation learning for drug–drug interaction prediction,” in Proceedings of the web conference 2021, 2021, pp. 2921–2933. doi: 10.1145/3442381.3449786.
[18] B. Rozemberczki, S. Bonner, A. Nikolov, M. Ughetto, S. Nilsson, and E. Papa, “A unified view of relational deep learning for drug pair scoring,” in Proceedings of the thirty-first international joint conference on artificial intelligence (IJCAI), survey track, 2022, pp. 5564–5571. doi: 10.24963/ijcai.2022/776.
[19] P. Veličković, G. Cangea, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in International conference on learning representations (ICLR), 2018.
[20] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in International conference on learning representations (ICLR), 2017.
[21] S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?” in International conference on learning representations (ICLR), 2022.
[22] S. Kaufman, S. Rosset, C. Perlich, and O. Stitelman, “Leakage in data mining: Formulation, detection, and avoidance,” ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 4, p. 15, 2012, doi: 10.1145/2382577.2382579.
[23] S. Kapoor and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,” Patterns, vol. 4, no. 9, p. 100804, 2023, doi: 10.1016/j.patter.2023.100804.
[24] P. Dewulf, M. Stock, and B. De Baets, “Cold-start problems in data-driven prediction of drug–drug interaction effects,” Pharmaceuticals, vol. 14, no. 5, p. 429, 2021, doi: 10.3390/ph14050429.
[25] D. Mendez et al., “ChEMBL: Towards direct deposition of bioassay data,” Nucleic Acids Research, vol. 47, no. D1, pp. D930–D940, 2019, doi: 10.1093/nar/gky1075.
[26] B. Zdrazil et al., “The ChEMBL database in 2023: A drug discovery platform spanning multiple bioactivity data types and time periods,” Nucleic Acids Research, vol. 52, no. D1, pp. D1180–D1192, 2024, doi: 10.1093/nar/gkad1004.
[27] G. Landrum et al., “RDKit: Open-source cheminformatics (release 2024.09.5).” https://www.rdkit.org, 2024. doi: 10.5281/zenodo.591637.
[28] D. S. Wishart et al., “DrugBank 5.0: A major update to the DrugBank database for 2018,” Nucleic Acids Research, vol. 46, no. D1, pp. D1074–D1082, 2018, doi: 10.1093/nar/gkx1037.
[29] N. P. Tatonetti, P. P. Ye, R. Daneshjou, and R. B. Altman, “Data-driven prediction of drug effects and interactions,” Science Translational Medicine, vol. 4, no. 125, p. 125ra31, 2012, doi: 10.1126/scitranslmed.3003377.
[30] D. Rogers and M. Hahn, “Extended-connectivity fingerprints,” Journal of Chemical Information and Modeling, vol. 50, no. 5, pp. 742–754, 2010, doi: 10.1021/ci100050t.
[31] D. Bajusz, A. Rácz, and K. Héberger, “Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations?” Journal of Cheminformatics, vol. 7, p. 20, 2015, doi: 10.1186/s13321-015-0069-3.
[32] J. L. Durant, B. A. Leland, D. R. Henry, and J. G. Nourse, “Reoptimization of MDL keys for use in drug discovery,” Journal of Chemical Information and Computer Sciences, vol. 42, no. 6, pp. 1273–1280, 2002, doi: 10.1021/ci010132r.
[33] S. A. Wildman and G. M. Crippen, “Prediction of physicochemical parameters by atomic contributions,” Journal of Chemical Information and Computer Sciences, vol. 39, no. 5, pp. 868–873, 1999, doi: 10.1021/ci990307l.
[34] P. Ertl, B. Rohde, and P. Selzer, “Fast calculation of molecular polar surface area as a sum of fragment-based contributions and its application to the prediction of drug transport properties,” Journal of Medicinal Chemistry, vol. 43, no. 20, pp. 3714–3717, 2000, doi: 10.1021/jm000942e.
[35] C. A. Lipinski, F. Lombardo, B. W. Dominy, and P. J. Feeney, “Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings,” Advanced Drug Delivery Reviews, vol. 46, no. 1–3, pp. 3–26, 2001, doi: 10.1016/S0169-409X(00)00129-0.
[36] G. Maggiora, M. Vogt, D. Stumpfe, and J. Bajorath, “Molecular similarity in medicinal chemistry,” Journal of Medicinal Chemistry, vol. 57, no. 8, pp. 3186–3204, 2014, doi: 10.1021/jm401411z.
[37] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929–1958, 2014.
[38] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016. https://doi.org/10.48550/arXiv.1607.06450
[39] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (ELUs),” in International conference on learning representations (ICLR), 2016. https://doi.org/10.48550/arXiv.1511.07289
[40] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp. 770–778. doi: 10.1109/CVPR.2016.90.
[41] Q. Li, Z. Han, and X.-M. Wu, “Deeper insights into graph convolutional networks for semi-supervised learning,” in Proceedings of the AAAI conference on artificial intelligence, 2018, pp. 3538–3545. doi: 10.1609/aaai.v32i1.11604.
[42] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International conference on learning representations (ICLR), 2019. https://doi.org/10.48550/arXiv.1711.05101
[43] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International conference on learning representations (ICLR), 2015. https://doi.org/10.48550/arXiv.1412.6980
[44] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in Proceedings of the 30th international conference on machine learning (ICML), in PMLR, no. 3, vol. 28. 2013, pp. 1310–1318. https://arxiv.org/pdf/1211.5063
[45] A. Paszke et al., “PyTorch: An imperative style, high-performance deep learning library,” in Advances in neural information processing systems, 2019, pp. 8024–8035. https://doi.org/10.48550/arXiv.1912.01703
[46] M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in ICLR workshop on representation learning on graphs and manifolds, 2019. https://doi.org/10.48550/arXiv.1903.02428
[47] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. https://doi.org/10.48550/arXiv.1201.0490
[48] J. A. Hanley and B. J. McNeil, “The meaning and use of the area under a receiver operating characteristic (ROC) curve,” Radiology, vol. 143, no. 1, pp. 29–36, 1982, doi: 10.1148/radiology.143.1.7063747.
[49] T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLoS ONE, vol. 10, no. 3, p. e0118432, 2015, doi: 10.1371/journal.pone.0118432.
[50] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, p. 6, 2020, doi: 10.1186/s12864-019-6413-7.
[51] W. J. Youden, “Index for rating diagnostic tests,” Cancer, vol. 3, no. 1, pp. 32–35, 1950, doi: 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.
[52] G. W. Brier, “Verification of forecasts expressed in terms of probability,” Monthly Weather Review, vol. 78, no. 1, pp. 1–3, 1950, doi: 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.
[53] B. Efron and R. J. Tibshirani, An introduction to the bootstrap. New York: Chapman & Hall/CRC, 1993. doi: 10.1201/9780429246593.
[54] M. Zhang and Y. Chen, “Link prediction based on graph neural networks,” in Advances in neural information processing systems, 2018, pp. 5165–5175. https://doi.org/10.48550/arXiv.1802.09691
[55] W. Hu et al., “Open graph benchmark: Datasets for machine learning on graphs,” in Advances in neural information processing systems, 2020, pp. 22118–22133. https://doi.org/10.48550/arXiv.2005.00687
[56] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.
[57] W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in neural information processing systems, 2017, pp. 1024–1034. https://doi.org/10.48550/arXiv.1706.02216
[58] J. Gilmer, K. T. Schütt, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in Proceedings of the 34th international conference on machine learning (ICML), in PMLR, vol. 70. 2017, pp. 1263–1272. https://arxiv.org/pdf/1704.01212
[59] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural network?” in International conference on learning representations (ICLR), 2019. https://doi.org/10.48550/arXiv.1810.00826
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Biomedical Informatics, Health Systems and Integrated Health Sciences
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
