Explainable ML for Clinical Decision Transparency
DOI:
https://doi.org/10.5281/zenodo.20744215Keywords:
Artificial Intelligence in Healthcare, Clinical Decision Support Systems, Explainable Artificial Intelligence (XAI), Model Transparency in Medicine, Interpretability of Machine Learning Models, Accountability in Clinical AI, Risk Prediction and Prognostic Modeling, Black-Box Model Limitations, Human-Centered AI Design, Trustworthy Medical AI, Ethical AI in Healthcare, Model Explainability Frameworks, Data-Driven Clinical Risk Assessment, AI Governance in Health SystemsAbstract
Artificial intelligence has the potential to augment clinical decision making. By learning patterns of risk and disease directly from empirical data, AI methods offer one solution to the difficulty health care professionals face in considering ever-increasing amounts of information. Clinicians making a medical decision for a patient want not only an accurate estimate of the risks associated with their patient's disease or treatment options but also an understanding of the reasoning behind these risks. This desire for explanation drives the growing interest in explainability in AI, particularly in AI for health care.
Explainable Artificial Intelligence (XAI) is defined as methods that generate new AI models for which the behaviour can be understood, directly or indirectly, by humans. The concept of human understanding encompasses three different levels – transparency, interpretability and accountability. The heart of the concern for transparency in AI is the incomprehensibility of the learned representations, the “black box” nature of the complex function learned from the training data.
References
[1] Adadi, A., & Berrada, M. (2020). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 8, 52138–52160.
[2] Ahmad, M. A., Eckert, C., & Teredesai, A. (2018). Interpretable machine learning in healthcare. Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, 559–560.
[3] Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., et al. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges. Information Fusion, 58, 82–115.
[4] Beam, A. L., & Kohane, I. S. (2018). Big data and machine learning in health care. JAMA, 319(13), 1317–1318.
[5] Bertsimas, D., & Kallus, N. (2020). From predictive to prescriptive analytics. Management Science, 66(3), 1025–1044.
[6] Biecek, P., & Burzykowski, T. (2021). Explanatory Model Analysis. CRC Press.
[7] Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019). Machine learning interpretability. Electronics, 8(8), 832.
[8] Chen, I. Y., Joshi, S., Ghassemi, M., & Ranganath, R. (2021). Probabilistic machine learning for healthcare. Annual Review of Biomedical Data Science, 4, 393–419.
[9] Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., et al. (2018). Opportunities and obstacles for deep learning in biology and medicine. Journal of the Royal Society Interface, 15(141), 20170387.
[10] Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint.
[11] Esteva, A., Robicquet, A., Ramsundar, B., et al. (2019). A guide to deep learning in healthcare. Nature Medicine, 25(1), 24–29.
[12] Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable AI in health care. The Lancet Digital Health, 3(11), e745–e750.
[13] Gilpin, L. H., Bau, D., Yuan, B. Z., et al. (2018). Explaining explanations: An overview of interpretability of machine learning. Proceedings of the IEEE, 106(8), 1178–1194.
[14] Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019). Causability and explainability of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(4), e1312.
[15] Holzinger, A., Carrington, A., & Müller, H. (2020). Measuring the quality of explanations. Artificial Intelligence and Law, 28, 193–198.
[16] Jiang, F., Jiang, Y., Zhi, H., et al. (2017). Artificial intelligence in healthcare. Stroke and Vascular Neurology, 2(4), 230–243.
[17] Johnson, A. E. W., Pollard, T. J., Shen, L., et al. (2016). MIMIC-III. Scientific Data, 3, 160035.
[18] Johnson, A. E. W., Stone, D. J., Celi, L. A., & Pollard, T. J. (2021). MIMIC-IV. Scientific Data, 8, 257.
[19] Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018). The artificial intelligence clinician. Nature Medicine, 24(11), 1716–1720.
[20] Kundu, S. (2021). AI in medicine must be explainable. Nature Medicine, 27, 1328.
[21] Lipton, Z. C. (2018). The mythos of model interpretability. Communications of the ACM, 61(10), 36–43.
[22] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
[23] Molnar, C. (2022). Interpretable machine learning (2nd ed.). Lulu.
[24] Montavon, G., Samek, W., & Müller, K. R. (2018). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1–15.
[25] Obermeyer, Z., & Emanuel, E. J. (2016). Predicting the future—Big data, machine learning, and clinical medicine. New England Journal of Medicine, 375(13), 1216–1219.
[26] Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm. Science, 366(6464), 447–453.
[27] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you? Proceedings of KDD, 1135–1144.
[28] Rudin, C. (2019). Stop explaining black box machine learning models. Nature Machine Intelligence, 1, 206–215.
[29] Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K. R. (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. Springer.
[30] Saria, S., & Subbaswamy, A. (2019). Tutorial: Safe and reliable machine learning. arXiv preprint.
[31] Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018). Deep EHR. IEEE Journal of Biomedical and Health Informatics, 22(5), 1589–1604.
[32] Shortliffe, E. H., & Cimino, J. J. (2021). Biomedical informatics (5th ed.). Springer.
[33] Sittig, D. F., & Singh, H. (2016). A socio-technical approach. Journal of the American Medical Informatics Association, 23(4), 641–647.
[34] Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want. npj Digital Medicine, 2, 1–7.
[35] Topol, E. (2019). Deep medicine. Basic Books.
[36] Van der Schaar, M., Alaa, A. M., Floto, A., et al. (2021). How machine learning can help healthcare systems. Machine Learning, 110(1), 1–20.
[37] Wiens, J., Saria, S., Sendak, M., et al. (2019). Do no harm. Nature Medicine, 25(9), 1337–1340.
[38] Wu, S., Roberts, K., Datta, S., et al. (2020). Deep learning in clinical NLP. Journal of the American Medical Informatics Association, 27(3), 457–470.
[39] Wehbe, R. M., et al. (2021). Deep learning in clinical NLP. Journal of the American Medical Informatics Association, 28(2), 1–15.
[40] Yang, G., Ye, Q., & Xia, J. (2020). Unbox the black-box for medical explainable AI. IEEE Journal of Biomedical and Health Informatics, 24(6), 1681–1690.
[41] Zhou, S., et al. (2023). A survey of explainable artificial intelligence in healthcare. Artificial Intelligence in Medicine, 138, 102473.
[42] Wang, F., & Preininger, A. (2019). AI in healthcare. Briefings in Bioinformatics, 20(6), 1986–2001.
[43] Choudhury, A., & Naumann, F. (2022). Interpretable ML in healthcare. IEEE Access, 10, 104541–104557.
[44] McCradden, M. D., Joshi, S., Anderson, J. A., et al. (2020). Patient safety and quality. npj Digital Medicine, 3, 1–5.
[45] Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020). Explainability for artificial intelligence in healthcare. BMC Medical Informatics and Decision Making, 20, 310.
[46] Björck, J., et al. (2021). Neural networks with monotonicity constraints. Proceedings of ICML.
[47] Caruana, R., et al. (2015). Intelligible models for healthcare. Proceedings of KDD, 1721–1730.
[48] Koh, P. W., & Liang, P. (2017). Influence functions. Proceedings of ICML, 1885–1894.
[49] Hooker, S. (2021). Moving beyond accuracy. Patterns, 2(11), 100344.
[50] Chen, J. H., & Asch, S. M. (2017). Machine learning and prediction in medicine. Annals of Internal Medicine, 167(3), 219–220.
[51] Krittanawong, C., et al. (2017). Machine learning in cardiovascular medicine. European Heart Journal, 38(25), 1857–1867.
[52] Rajkomar, A., et al. (2018). Scalable and accurate deep learning with EHRs. npj Digital Medicine, 1, 18.
[53] Beam, A. L., & Kohane, I. S. (2018). Translating AI into clinical care. NEJM, 379(23), 2165–2167.
[54] Sendak, M. P., et al. (2020). Real-world integration of predictive analytics. NEJM Catalyst, 1(3).
[55] Holzinger, A., et al. (2022). XAI in medicine: Why and how. Artificial Intelligence in Medicine, 126, 102164.
[56] Lakkaraju, H., et al. (2020). Interpretable decision sets. Journal of Machine Learning Research, 21(86), 1–38.
[57] Guidotti, R., et al. (2019). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), 1–42.
[58] Doshi-Velez, F., et al. (2020). Accountability of AI. Communications of the ACM, 63(11), 56–65.
[59] Raji, I. D., et al. (2020). Closing the AI accountability gap. Proceedings of FAT*, 33–44.
[60] Mitchell, M., et al. (2019). Model cards. Proceedings of FAT*, 220–229.
[61] Buolamwini, J., & Gebru, T. (2018). Gender shades. Proceedings of FAT*, 77–91.
[62] European Commission. (2021). Ethics guidelines for trustworthy AI.
[63] World Health Organization. (2021). Ethics and governance of artificial intelligence for health.
[64] National Academy of Medicine. (2022). Artificial intelligence in health care.
[65] OECD. (2022). Artificial intelligence in the health sector.
[66] U.S. Food and Drug Administration. (2021). Artificial intelligence/machine learning software as a medical device.
[67] Topol, E. (2023). The convergence of human and artificial intelligence. Nature Medicine, 29, 44–56.
[68] Rajpurkar, P., et al. (2017). CheXNet. arXiv preprint.
[69] Park, Y., et al. (2022). Interpretable deep learning for medical imaging. Medical Image Analysis, 75, 102288.
[70] Zhang, Q., et al. (2021). Interpreting deep learning models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10), 3378–3395.
[71] Tonekaboni, S., et al. (2021). Clinician-centered explainable AI. Nature Machine Intelligence, 3, 40–47.
[72] Yoon, J., et al. (2019). INVASE. Proceedings of ICLR.
[73] Louizos, C., et al. (2018). Causal effect inference. NeurIPS.
[74] Pearl, J. (2019). The book of why. Basic Books.
[75] Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference. MIT Press.
[76] Kearns, M., & Roth, A. (2019). The ethical algorithm. Oxford University Press.
[77] Floridi, L., et al. (2018). AI4People. Minds and Machines, 28, 689–707.
[78] Mittelstadt, B. D., et al. (2016). Ethics of algorithms. Big Data & Society, 3(2).
[79] Ribeiro, M. T., et al. (2018). Anchors. Proceedings of AAAI.
[80] Wang, C., et al. (2022). Explainable boosting machines for healthcare. Artificial Intelligence in Medicine, 126, 102187.
[81] Friedman, J. H. (2001). Greedy function approximation. Annals of Statistics, 29(5), 1189–1232.
[82] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
[83] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297.
[84] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
[85] Kingma, D. P., & Ba, J. (2015). Adam. Proceedings of ICLR.
[86] Vaswani, A., et al. (2017). Attention is all you need. NeurIPS.
[87] Devlin, J., et al. (2019). BERT. NAACL-HLT.
[88] Lundberg, S. M., et al. (2020). From local explanations to global understanding. Nature Machine Intelligence, 2, 252–259.
[89] Ji, Z., et al. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38.
[90] Zhou, Y., et al. (2023). Trustworthy AI in clinical decision support systems. Journal of Biomedical Informatics, 145, 104454.
Additional Files
Published
Issue
Section
License
The American Online Journal of Science and Engineering (AOJSE) is an open-access journal, and all published articles are made freely available to readers worldwide under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0). This license governs the use, sharing, adaptation, and distribution of all content published in AOJSE and is designed to promote the broadest possible dissemination and reuse of scholarly research.
Creative Commons Attribution 4.0 International License (CC BY 4.0)
Under the Creative Commons Attribution 4.0 International License, any user is free to share, copy, and redistribute the published material in any medium or format, and to adapt, remix, transform, and build upon the material for any purpose, including commercial use, provided that appropriate credit is given to the original authors and source. The license cannot be revoked as long as the terms are followed.
When using or redistributing content published in AOJSE, users must provide proper attribution by citing the original authors' names, the article title, the journal name (AOJSE), the volume and issue number, the year of publication, and the DOI or URL of the article. Users must also indicate if any changes were made to the original work. Attribution must be provided in a reasonable manner but must not suggest that the authors or AOJSE endorse the user or their use of the material.
Author Rights and Copyright Retention
AOJSE respects and upholds the intellectual property rights of all contributing authors. Authors who publish in AOJSE retain full copyright over their work. By submitting a manuscript to AOJSE, authors grant the journal a non-exclusive, worldwide, royalty-free license to publish, reproduce, distribute, publicly display, and archive the article in print and electronic formats. This license allows AOJSE to make the article freely accessible to readers globally while ensuring that the authors remain the rightful owners of their work.
Authors are permitted to deposit their published articles in institutional repositories, personal websites, academic networking platforms, and preprint servers, provided that the original publication in AOJSE is acknowledged and properly cited. Authors may also reuse their published content in subsequent works, presentations, teaching materials, and grant applications without requiring prior permission from the journal.
Third-Party Content
Authors are solely responsible for obtaining written permission to reproduce any third-party material, including figures, tables, images, data, or excerpts from other publications, included in their manuscript. Evidence of such permissions must be provided to the editorial office upon request. AOJSE does not assume any responsibility for copyright infringement arising from the unauthorized inclusion of third-party content in published articles.
Permitted Uses
Under the CC BY 4.0 license, the following uses of AOJSE published content are freely permitted without prior written permission from the journal or authors, provided that proper attribution is given. Users may read, download, print, and distribute articles for personal, educational, or research purposes. Researchers may reuse data, figures, and findings for meta-analyses, systematic reviews, and secondary research. Educators may incorporate published articles into course materials, syllabi, and academic presentations. Journalists, policymakers, and practitioners may reference and cite published findings for professional and public interest purposes.
Prohibited Uses
While the CC BY 4.0 license is broad and permissive, users may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Users may not misrepresent the original authorship of published content or imply endorsement by the original authors or AOJSE without explicit consent. Plagiarism, data fabrication, and any form of academic misconduct in the reuse of published material are strictly prohibited and are a violation of publication ethics standards.
Institutional and Repository Deposit Policy
AOJSE fully supports self-archiving and green open access. Authors are encouraged to deposit the final published version of their article, also known as the version of record, in institutional repositories, subject repositories, and open-access databases immediately upon publication. There is no embargo period. Authors should always link back to the original article on the AOJSE website and include the full citation and DOI when depositing their work in any repository.
Digital Preservation and Archiving
AOJSE is committed to the long-term digital preservation of all published content to ensure its permanent availability and accessibility. The journal utilizes the Open Journal Systems (OJS) platform, which supports internationally recognized digital archiving standards. AOJSE encourages participation in archiving networks such as LOCKSS (Lots of Copies Keep Stuff Safe) and CLOCKSS (Controlled LOCKSS) to safeguard published articles against data loss and ensure continued access for future generations of researchers and readers.
Disclaimer
The views, opinions, findings, and conclusions expressed in articles published in AOJSE are solely those of the authors and do not necessarily reflect the official position, policy, or views of the journal, its editorial board, or its affiliated organizations. AOJSE makes every effort to ensure the accuracy and integrity of published content but does not accept legal responsibility for any errors, omissions, or claims arising from the use of information published in the journal.
Contact for Licensing Inquiries
For any questions regarding the licensing terms, copyright, or permitted use of content published in AOJSE, please contact the editorial office through the journal website at https://aojse.org. The editorial team will be happy to assist authors, readers, and institutions with any licensing-related queries.
