Smart Decisions with Azure AI Pipelines
Keywords:
AI Decision Pipelines, Intelligent Decision Systems, Big Data Analytics, Decision Quality Metrics, Data Ingestion Systems, Data Governance in AI, Secure Data Pipelines, Compliance in AI Systems, Resilient Data Architectures, Azure Data Cloud, Cloud Data Engineering, Credit Scoring Models, Risk Assessment Analytics, Banking Decision Systems, Data Preprocessing Pipelines, Scalable Data Management, AI Pipeline Architecture, Decision Theory in AI, Enterprise Data Pipelines, Data-Driven Decision Making.Abstract
Artificial intelligence (AI) pipelines are central to building intelligent decision systems that leverage big data analytics to improve decision quality. A pipeline, in this context, is a computer-based implementation of an intelligent decision system that harnesses AI methods and big data analytics to enhance decision outcomes, with its results evaluated through the lens of decision theory. Decision quality itself is a multidimensional measure closely tied to the pipeline's processing flow, and it depends heavily on the integrity of the data feeding it — making data governance, security, compliance, and resilience essential design considerations.
Since decision quality is ultimately bounded by the quality of the data ingested and processed, automated data ingestion and management at scale are critical to realizing the full potential of these systems. This paper presents a case study using Microsoft Azure's Data Cloud capabilities to demonstrate large-scale data ingestion and management in practice, offering guidance for constructing AI pipelines on Azure. The framework is further validated through an application to credit scoring and risk assessment in bank-based decision-making, illustrating how these pipeline principles translate into a real-world financial use case.
References
1. Mäkinen, S. S., Skogström, H., Laaksonen, E., & Mikkonen, T. (2021). Who needs MLOps: What data scientists seek to accomplish and how can MLOps help? In 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN). IEEE.
2. Renggli, C., Rimanic, L., Gurel, N. M., Karlas, B., Wu, W., & Zhang, C. (2021). A data quality-driven view of MLOps. IEEE Data Engineering Bulletin, 44(1), 11–23.
3. Loganathan, R., & Kolla, S. H. (2024). Harnessing generative AI for adaptive risk policy generation in multi-regulatory data center environments. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 15(3), 505-515.
4. Granlund, T., Kopponen, A., Stirbu, V. A., Myllyaho, L., & Mikkonen, T. (2021). MLOps challenges in multi-organization setup: Experiences from two real-world cases. In 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN). IEEE.
5. Granlund, T., Stirbu, V. A., & Mikkonen, T. (2021). Towards regulatory-compliant MLOps: Oravizio’s journey from a machine learning experiment to a deployed certified medical product. SN Computer Science, 2, 342.
6. Mattaparthi, R. (2024). Transformer-Based Fault Diagnosis for Large-Scale Standby Power Generators: Partial Discharge Pattern Recognition at Hyperscale Data Center Installations. Journal of Computational Analysis and Applications (JoCAAA), 33(08), 8781-8799.
7. Spjuth, O., Frid, J., & Hellander, A. (2021). The machine learning life cycle and the cloud: Implications for drug discovery. Expert Opinion on Drug Discovery, 16(9), 1071–1079.
8. He, X., Zhao, K., & Chu, X. (2021). AutoML: A survey of the state-of-the-art. Knowledge-Based Systems, 212, 106622.
9. Mangalampalli, B. M. (2024). Transparent Intelligence Explainability Frameworks for AI-Driven Clinical Decision Support in Healthcare Business Intelligence. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(3), 10566-10579.
10. Baier, L., Schlör, T., Schoeffer, J., & Kühl, N. (2023). Detecting concept drift with neural network model uncertainty. In Proceedings of the 56th Hawaii International Conference on System Sciences (pp. 835–844).
11. Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022). MLOps: A taxonomy and a methodology. IEEE Access, 10, 63606–63626.
12. Mahadevan, S. (2024). Clouding the Future: Innovating Towards Net-Zero Emissions. International Journal of Computing and Engineering, 6(2), 17-23.
13. Symeonidis, G., Nerantzis, E., Kazakis, A., & Papakostas, G. A. (2022). MLOps—Definitions, tools and challenges. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) (pp. 0453–0460). IEEE.
14. Lima, A., Monteiro, L., & Furtado, A. P. (2022). MLOps: Practices, maturity models, roles, tools, and challenges—A systematic literature review. In Proceedings of the 24th International Conference on Enterprise Information Systems (ICEIS) (pp. 308–320). SciTePress.
15. Kolla, S. K., & Mangalampalli, B. M. (2024). Edge-Based Deep Learning Systems for Point-of-Care Diagnostic Intelligence. Journal of Neonatal Surgery, 13(1), 2387-2399.
16. Hewage, N., & Meedeniya, D. (2022). Machine learning operations: A survey on MLOps tool support. arXiv.
17. Martínez-Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A. M., & Wagner, S. (2022). Software engineering for AI-based systems: A survey. ACM Transactions on Software Engineering and Methodology, 31(2).
18. Kolla, S. H., & Peddi, R. K. (2024). Designing Governance-Aligned GenAI Pipelines Using Small Language Models for Enterprise Workflow Intelligence. International Journal of Science, Research and Technology, 7(6), 13256-13268.
19. Baum, K., Mantel, S., Schmidt, E., & Speith, T. (2022). From responsibility to reason-giving explainable artificial intelligence. Philosophy & Technology, 35, 12.
20. Schoeffer, J., Kühl, N., & Machowski, Y. (2022). “There is not enough information”: On the effects of explanations on perceptions of informational fairness and trustworthiness in automated decision-making. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency.
21. Jakubik, J., Vössing, M., Kühl, N., Walk, J., & Satzger, G. (2024). Data-centric artificial intelligence. Business & Information Systems Engineering, 66, 507–515.
22. Kreuzberger, D., Kühl, N., & Hirschl, S. (2023). Machine learning operations (MLOps): Overview, definition, and architecture. IEEE Access, 11, 31866–31879.
23. Reddy, V. A. R. (2024). Generative Intelligence for Healthcare Claims Processing and Personalized Benefits Management. International Journal of Advanced Engineering Science and Information Technology (IJAESIT), 7(3), 14099.
24. Steidl, M., Felderer, M., & Ramler, R. (2023). The pipeline for the continuous development of artificial intelligence models—Current state of research and practice. Journal of Systems and Software, 199, 111615.
25. Diaz-de-Arcaya, J., Torre-Bastida, A. I., Zárate, G., Miñón, R., & Almeida, A. (2024). A joint study of the challenges, opportunities, and roadmap of MLOps and AIOps: A systematic survey. ACM Computing Surveys, 56(4), Article 84.
26. Faubel, L., Schmid, K., & Eichelberger, H. (2023). MLOps challenges in Industry 4.0. SN Computer Science, 4, 828.
27. Bodor, A., Hnida, M., & Daoudi, N. (2023). MLOps: Overview of current state and future directions. In Innovations in Smart Cities Applications (Vol. 6, pp. 156–165).
28. Mangala, N. (2022). Real-Time Data Quality Monitoring and Gating Frameworks in Cloud-Based Data Pipelines. International Journal of Research and Applied Innovations, 5(6), 8197-8219.
29. Barbudo, R., Ventura, S., & Romero, J. R. (2023). Eight years of AutoML: Categorisation, review and trends. Knowledge and Information Systems, 65, 5097–5149.
30. Singh, R., Miller, T., Lyons, H., Sonenberg, L., Velloso, E., Vetere, F., Howe, P., & Dourish, P. (2023). Directive explanations for actionable explainability in machine learning applications. ACM Transactions on Interactive Intelligent Systems, 13(4), Article 23.
31. Wazir, S., Kashyap, G. S., & Saxena, P. (2023). MLOps: A review. arXiv.
32. Jakubik, J., Vössing, M., Kühl, N., Walk, J., & Satzger, G. (2022). Data-centric artificial intelligence. arXiv.
33. Esmaeilzadeh, A., Heidari, M., Abdolazimi, R., Hajibabaee, P., & Malekzadeh, M. (2022). Efficient large scale NLP feature engineering with Apache Spark. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC). IEEE.
34. Yandamuri, U. S. (2024). AI-Driven Decision Support Systems for Operational Optimization in Hospitality Technology. Metallurgical and Materials Engineering.
35. Thonglek, K., Takahashi, K., Ichikawa, K., Nakasan, C., Nakada, H., Takano, R., Leelaprute, P., & Iida, H. (2022). Automated quantization and retraining for neural network models without labeled data. IEEE Access, 10, 73818–73834.
36. Explainable AI for enhanced decision-making. (2024). Decision Support Systems, 184, 114276.
37. Explainable artificial intelligence and agile decision-making in supply chain cyber resilience. (2024). Decision Support Systems, 180, 114194.
38. Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction. (2024). Neurocomputing, 599, 128111.
39. Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022). MLOps: A taxonomy and a methodology. IEEE Access, 10, 63606–63626.
40. Granlund, T., Stirbu, V. A., & Mikkonen, T. (2021). Towards regulatory-compliant MLOps: Oravizio’s journey from a machine learning experiment to a deployed certified medical product. SN Computer Science, 2, 342.
41. Renggli, C., Rimanic, L., Gurel, N. M., Karlas, B., Wu, W., & Zhang, C. (2021). A data quality-driven view of MLOps. IEEE Data Engineering Bulletin, 44(1), 11–23.
42. Mäkinen, S. S., Skogström, H., Laaksonen, E., & Mikkonen, T. (2021). Who needs MLOps: What data scientists seek to accomplish and how can MLOps help? In 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN). IEEE.
43. Faubel, L., Schmid, K., & Eichelberger, H. (2023). MLOps challenges in Industry 4.0. SN Computer Science, 4, 828.
44. Jakubik, J., Vössing, M., Kühl, N., Walk, J., & Satzger, G. (2024). Data-centric artificial intelligence. Business & Information Systems Engineering, 66, 507–515.
Additional Files
Published
Issue
Section
License
Copyright (c) 2025 Ganesh Pambala (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The American Online Journal of Science and Engineering (AOJSE) is an open-access journal, and all published articles are made freely available to readers worldwide under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0). This license governs the use, sharing, adaptation, and distribution of all content published in AOJSE and is designed to promote the broadest possible dissemination and reuse of scholarly research.
Creative Commons Attribution 4.0 International License (CC BY 4.0)
Under the Creative Commons Attribution 4.0 International License, any user is free to share, copy, and redistribute the published material in any medium or format, and to adapt, remix, transform, and build upon the material for any purpose, including commercial use, provided that appropriate credit is given to the original authors and source. The license cannot be revoked as long as the terms are followed.
When using or redistributing content published in AOJSE, users must provide proper attribution by citing the original authors' names, the article title, the journal name (AOJSE), the volume and issue number, the year of publication, and the DOI or URL of the article. Users must also indicate if any changes were made to the original work. Attribution must be provided in a reasonable manner but must not suggest that the authors or AOJSE endorse the user or their use of the material.
Author Rights and Copyright Retention
AOJSE respects and upholds the intellectual property rights of all contributing authors. Authors who publish in AOJSE retain full copyright over their work. By submitting a manuscript to AOJSE, authors grant the journal a non-exclusive, worldwide, royalty-free license to publish, reproduce, distribute, publicly display, and archive the article in print and electronic formats. This license allows AOJSE to make the article freely accessible to readers globally while ensuring that the authors remain the rightful owners of their work.
Authors are permitted to deposit their published articles in institutional repositories, personal websites, academic networking platforms, and preprint servers, provided that the original publication in AOJSE is acknowledged and properly cited. Authors may also reuse their published content in subsequent works, presentations, teaching materials, and grant applications without requiring prior permission from the journal.
Third-Party Content
Authors are solely responsible for obtaining written permission to reproduce any third-party material, including figures, tables, images, data, or excerpts from other publications, included in their manuscript. Evidence of such permissions must be provided to the editorial office upon request. AOJSE does not assume any responsibility for copyright infringement arising from the unauthorized inclusion of third-party content in published articles.
Permitted Uses
Under the CC BY 4.0 license, the following uses of AOJSE published content are freely permitted without prior written permission from the journal or authors, provided that proper attribution is given. Users may read, download, print, and distribute articles for personal, educational, or research purposes. Researchers may reuse data, figures, and findings for meta-analyses, systematic reviews, and secondary research. Educators may incorporate published articles into course materials, syllabi, and academic presentations. Journalists, policymakers, and practitioners may reference and cite published findings for professional and public interest purposes.
Prohibited Uses
While the CC BY 4.0 license is broad and permissive, users may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Users may not misrepresent the original authorship of published content or imply endorsement by the original authors or AOJSE without explicit consent. Plagiarism, data fabrication, and any form of academic misconduct in the reuse of published material are strictly prohibited and are a violation of publication ethics standards.
Institutional and Repository Deposit Policy
AOJSE fully supports self-archiving and green open access. Authors are encouraged to deposit the final published version of their article, also known as the version of record, in institutional repositories, subject repositories, and open-access databases immediately upon publication. There is no embargo period. Authors should always link back to the original article on the AOJSE website and include the full citation and DOI when depositing their work in any repository.
Digital Preservation and Archiving
AOJSE is committed to the long-term digital preservation of all published content to ensure its permanent availability and accessibility. The journal utilizes the Open Journal Systems (OJS) platform, which supports internationally recognized digital archiving standards. AOJSE encourages participation in archiving networks such as LOCKSS (Lots of Copies Keep Stuff Safe) and CLOCKSS (Controlled LOCKSS) to safeguard published articles against data loss and ensure continued access for future generations of researchers and readers.
Disclaimer
The views, opinions, findings, and conclusions expressed in articles published in AOJSE are solely those of the authors and do not necessarily reflect the official position, policy, or views of the journal, its editorial board, or its affiliated organizations. AOJSE makes every effort to ensure the accuracy and integrity of published content but does not accept legal responsibility for any errors, omissions, or claims arising from the use of information published in the journal.
Contact for Licensing Inquiries
For any questions regarding the licensing terms, copyright, or permitted use of content published in AOJSE, please contact the editorial office through the journal website at https://aojse.org. The editorial team will be happy to assist authors, readers, and institutions with any licensing-related queries.
