MLOps Strategies for Scalable Cloud-Based Deployment
Keywords:
MLOps, Cloud-Native Machine Learning, Continuous Integration for ML, Continuous Delivery of Models, ML Reliability Engineering, Model Deployment Pipelines, Data Lineage and Reproducibility, ML Governance and Security, Service-Level Objectives (SLOs), Monitoring and Incident Response Automation, CI/CD for Machine Learning, Model Rollout Strategies, Containerization for ML (Docker), Orchestration for ML Workloads (Kubernetes), Model Serving Frameworks, High-Throughput Low-Latency Inference, Auto-Scaling ML Services, Model Versioning, Cloud Data Platform Architecture, Production ML Observability.Abstract
Machine learning operations (MLOps) comprises the practices, methods, and tooling that facilitate the deployment of reliable ML models in production environments. While many aspects of cloud data platforms are designed to enable reliability, only some managed ML services support the MLOps goals of continuous integration, continuous delivery, data lineage tracking, associated reproducibility, governance, and security. Furthermore, reliability encompasses not only the fulfillment of service-level objectives, but also systematic monitoring, alerting, and incident response automation. Architectural patterns are proposed to enable reliable deployment in cloud data platforms, focusing on the implementation of continuous integration and testing pipelines for ML models and the formulation of continuous delivery and rollout strategies. Continuous integration pipelines reduce the risk of regressions and ensure sufficient model performance at the time of deployment, while continuous delivery pipelines enable rapid updates to production models within acceptable risk profiles.
The landscape of publicly available MLOps frameworks, tools, and services is also examined, emphasizing the pros and cons of established and rising solutions in containerization, orchestration, model serving, and inference. Containerization and orchestration contributes to the building of reliable deployment pipelines in cloud data platforms, whether general-purpose tools (e.g. Docker and Kubernetes) or solutions tailored for ML workloads. Containerized serving frameworks designed for high-throughput, low-latency inference can benefit a wide range of business applications, while auto-scaling and model versioning capabilities enhance the ease of use of cloud-native ML services.
References
1. Garg, S., Pundir, P., Rathee, G., Gupta, P. K., Garg, S., & Ahlawat, S. (2021). On continuous integration/continuous delivery for automated deployment of machine learning models using MLOps. In 2021 IEEE Fourth International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) (pp. 25–28). IEEE.
2. Granlund, T., Kopponen, A., Stirbu, V. A., Myllyaho, L., & Mikkonen, T. (2021). MLOps challenges in multi-organization setup: Experiences from two real-world cases. In 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN) (pp. 7–13). IEEE.
3. John, M. M., Olsson, H. H., & Bosch, J. (2021). Towards MLOps: A framework and maturity model. In 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (pp. 334–341). IEEE.
4. Kolla, S. H. (2023). Large Language Model-Driven Enterprise Service Intelligence for Digital Workflow Transformation. International Journal of Research and Applied Innovations, 6(1), 8380-8391.
5. Raj, E., Buffoni, D., Westerlund, M., & Ahola, K. (2021). Edge MLOps: An automation framework for AIoT applications. In 2021 IEEE International Conference on Cloud Engineering (IC2E) (pp. 191–200). IEEE.
6. Renggli, C., Rimanic, L., Gurel, N. M., Karlas, B., Wu, W., & Zhang, C. (2021). A data quality-driven view of MLOps. IEEE Data Engineering Bulletin, 44(1), 11–23.
7. Kolla, S. K., & Reddy, V. A. R. (2023). Deep Learning Architectures For Multimodal Medical Data Integration. South Eastern European Journal of Public Health, 248–260.
8. Granlund, T., Stirbu, V. A., & Mikkonen, T. (2021). Towards regulatory-compliant MLOps: Oravizio’s journey from a machine learning experiment to a deployed certified medical product. SN Computer Science, 2, Article 342.
9. Mäkinen, S. S., Skogström, H., Laaksonen, E., & Mikkonen, T. (2021). Who needs MLOps: What data scientists seek to accomplish and how can MLOps help? In 2021 IEEE/ACM 1st Workshop on AI Engineering—Software Engineering for AI (WAIN). IEEE.
10. Xu, X., Mo, R., Yin, X., Khosravi, M. R., Aghaei, F., Chang, V., & Li, G. (2021). PDM: Privacy-aware deployment of machine-learning applications for industrial cyber-physical cloud systems. IEEE Transactions on Industrial Informatics, 17(8), 5819–5828.
11. Bandi, V. D. V. K. (2023, November). Production-Grade Machine Learning Pipelines For Healthcare Predictive Analytics. South Eastern European Journal of Public Health, 189–205.
12. Cai, H., Wang, C., & Zhou, X. (2021). Deployment and verification of machine learning tool-chain based on Kubernetes distributed clusters. CCF Transactions on High Performance Computing, 3(3).
13. Shah, S.-C. (2021). Design of a machine learning-based intelligent middleware platform for a heterogeneous private edge cloud system. Sensors, 21(22), Article 7701.
14. Singh, P. (2021). Machine learning deployment using Kubernetes. In Deploy machine learning models to production (pp. 127–146). Apress.
15. Testi, M., Ballabio, M., Frontoni, E., Iannello, G., Moccia, S., Soda, P., & Vessio, G. (2022). MLOps: A taxonomy and a methodology. IEEE Access, 10, 63606–63618.
16. Davuluri, P. S. L. N. (2023). Integrating artificial intelligence into event-driven financial crime compliance platforms. International Journal of Finance, 36(6), 707-736.
17. Symeonidis, G., Nerantzis, E., Kazakis, A., & Papakostas, G. A. (2022). MLOps—Definitions, tools and challenges. In 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) (pp. 0453–0460). IEEE.
18. Hewage, N., & Meedeniya, D. (2022). Machine learning operations: A survey on MLOps tool support. arXiv preprint arXiv:2202.10169.
19. Kreuzberger, D., Kühl, N., & Hirschl, S. (2022). Machine learning operations (MLOps): Overview, definition, and architecture. arXiv preprint arXiv:2205.02302.
20. Moreschini, S., Lomio, F., Hästbacka, D., & Taibi, D. (2022). MLOps for evolvable AI intensive software systems. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 1293–1294). IEEE.
21. Kolltveit, A. B., & Li, J. (2022). Operationalizing machine learning models—A systematic literature review. In 2022 IEEE/ACM 1st International Workshop on Software Engineering for Responsible Artificial Intelligence (SE4RAI) (pp. 1–8). IEEE.
22. Yandamuri, U. S. (2023). An Intelligent Analytics Framework Combining Big Data and Machine Learning for Business Forecasting. Zenodo.
23. Shankar, S., Garcia, R., Hellerstein, J. M., & Parameswaran, A. G. (2022). Operationalizing machine learning: An interview study. arXiv preprint arXiv:2209.09125.
24. Subramanya, R., Sierla, S., & Vyatkin, V. (2022). From DevOps to MLOps: Overview and application to electricity market forecasting. Applied Sciences, 12(19), Article 9851.
25. Leroux, S., Simoens, P., Lootus, M., Thakore, K., & Sharma, A. (2022). TinyMLOps: Operational challenges for widespread edge AI adoption. In 2022 IEEE 36th International Parallel and Distributed Processing Symposium Workshops (IPDPSW) (pp. 1003–1010). IEEE.
26. Pandey, A., Sonawane, M., & Mamtani, S. (2022). Deployment of ML models using Kubeflow on different cloud providers. arXiv preprint arXiv:2206.13655.
27. Chinthapatla, S. (2022). Edge AI with Kubernetes: Deploying machine learning models at scale. International Journal of Emerging Trends in Computer Science and Information Technology, 3(2), 32–41.
28. Böhm, S., & Wirtz, G. (2022). Cloud-edge orchestration for smart cities: A review of Kubernetes-based orchestration architectures. EAI Endorsed Transactions on Smart Cities, 6(18), Article e2.
29. Mangalampalli, B. M. Generative AI Applications In Healthcare Data Mart Design And Optimization.
30. Liang, B., Wu, D., Wu, P., & Su, Y. (2022). Efficient evolutionary optimization using predictive auto-scaling in containerized environment. Applied Soft Computing, 129, Article 109610.
31. Boosa, S., & Allam, K. (2022). End-to-end MLOps pipeline on Kubernetes for scalable healthcare applications. International Journal of Emerging Research in Engineering and Technology, 3(1), 74–85.
Additional Files
Published
Issue
Section
License
Copyright (c) 2024 Ethan Williams (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The American Online Journal of Science and Engineering (AOJSE) is an open-access journal, and all published articles are made freely available to readers worldwide under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0). This license governs the use, sharing, adaptation, and distribution of all content published in AOJSE and is designed to promote the broadest possible dissemination and reuse of scholarly research.
Creative Commons Attribution 4.0 International License (CC BY 4.0)
Under the Creative Commons Attribution 4.0 International License, any user is free to share, copy, and redistribute the published material in any medium or format, and to adapt, remix, transform, and build upon the material for any purpose, including commercial use, provided that appropriate credit is given to the original authors and source. The license cannot be revoked as long as the terms are followed.
When using or redistributing content published in AOJSE, users must provide proper attribution by citing the original authors' names, the article title, the journal name (AOJSE), the volume and issue number, the year of publication, and the DOI or URL of the article. Users must also indicate if any changes were made to the original work. Attribution must be provided in a reasonable manner but must not suggest that the authors or AOJSE endorse the user or their use of the material.
Author Rights and Copyright Retention
AOJSE respects and upholds the intellectual property rights of all contributing authors. Authors who publish in AOJSE retain full copyright over their work. By submitting a manuscript to AOJSE, authors grant the journal a non-exclusive, worldwide, royalty-free license to publish, reproduce, distribute, publicly display, and archive the article in print and electronic formats. This license allows AOJSE to make the article freely accessible to readers globally while ensuring that the authors remain the rightful owners of their work.
Authors are permitted to deposit their published articles in institutional repositories, personal websites, academic networking platforms, and preprint servers, provided that the original publication in AOJSE is acknowledged and properly cited. Authors may also reuse their published content in subsequent works, presentations, teaching materials, and grant applications without requiring prior permission from the journal.
Third-Party Content
Authors are solely responsible for obtaining written permission to reproduce any third-party material, including figures, tables, images, data, or excerpts from other publications, included in their manuscript. Evidence of such permissions must be provided to the editorial office upon request. AOJSE does not assume any responsibility for copyright infringement arising from the unauthorized inclusion of third-party content in published articles.
Permitted Uses
Under the CC BY 4.0 license, the following uses of AOJSE published content are freely permitted without prior written permission from the journal or authors, provided that proper attribution is given. Users may read, download, print, and distribute articles for personal, educational, or research purposes. Researchers may reuse data, figures, and findings for meta-analyses, systematic reviews, and secondary research. Educators may incorporate published articles into course materials, syllabi, and academic presentations. Journalists, policymakers, and practitioners may reference and cite published findings for professional and public interest purposes.
Prohibited Uses
While the CC BY 4.0 license is broad and permissive, users may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Users may not misrepresent the original authorship of published content or imply endorsement by the original authors or AOJSE without explicit consent. Plagiarism, data fabrication, and any form of academic misconduct in the reuse of published material are strictly prohibited and are a violation of publication ethics standards.
Institutional and Repository Deposit Policy
AOJSE fully supports self-archiving and green open access. Authors are encouraged to deposit the final published version of their article, also known as the version of record, in institutional repositories, subject repositories, and open-access databases immediately upon publication. There is no embargo period. Authors should always link back to the original article on the AOJSE website and include the full citation and DOI when depositing their work in any repository.
Digital Preservation and Archiving
AOJSE is committed to the long-term digital preservation of all published content to ensure its permanent availability and accessibility. The journal utilizes the Open Journal Systems (OJS) platform, which supports internationally recognized digital archiving standards. AOJSE encourages participation in archiving networks such as LOCKSS (Lots of Copies Keep Stuff Safe) and CLOCKSS (Controlled LOCKSS) to safeguard published articles against data loss and ensure continued access for future generations of researchers and readers.
Disclaimer
The views, opinions, findings, and conclusions expressed in articles published in AOJSE are solely those of the authors and do not necessarily reflect the official position, policy, or views of the journal, its editorial board, or its affiliated organizations. AOJSE makes every effort to ensure the accuracy and integrity of published content but does not accept legal responsibility for any errors, omissions, or claims arising from the use of information published in the journal.
Contact for Licensing Inquiries
For any questions regarding the licensing terms, copyright, or permitted use of content published in AOJSE, please contact the editorial office through the journal website at https://aojse.org. The editorial team will be happy to assist authors, readers, and institutions with any licensing-related queries.
