Scalable Validation and Continuous Verification of AI/ML Systems on AWS Using Python-Based Automation
Abstract
Data drift, model degradation, and inference inconsistency are the main issues related to the operational implementation of AI/ML systems in cloud-native settings. This paper suggests a Python-related code of scalable validation and the constant verification of AI/ML models on AWS. It incorporates automated validation of the dataset, performance benchmarking and drift detection in the containerized deployments of AWS EKS, Lambda and S3. Datapoints, e.g., precision, recall and distribution variance are introduced into workflow CI/CD pipelines as real-time tools. The empirical assessment exhibits better reproducibility, few post-deployment errors and better traceability. The methodology sustains the trustful, automatic and undisrupted AI/ML usage in series cloud environments that can be scaled.
Article Information
Journal |
International Journal of Advanced Engineering Science and Information Technology (IJAESIT) |
|---|---|
Volume (Issue) |
Vol. 3 No. 5 (2020): International Journal of Advanced Engineering Science and Information Technology (IJAESIT) |
DOI |
|
Pages |
5131-5138 |
Published |
October 5, 2020 |
| Copyright | |
Open Access |
This work is licensed under a Creative Commons Attribution 4.0 International License. |
How to Cite |
Lingaraj Kothokatta (2020). Scalable Validation and Continuous Verification of AI/ML Systems on AWS Using Python-Based Automation. International Journal of Advanced Engineering Science and Information Technology (IJAESIT) , Vol. 3 No. 5 (2020): International Journal of Advanced Engineering Science and Information Technology (IJAESIT) , pp. 5131-5138. https://doi.org/10.15662/IJAESIT.2020.0305002 |
References
[2] D. Baylor, E. Breck, H.-T. Cheng, N. Fiedel, C. Y. Foo, Z. Haque, S. Haykal, M. Ispir, V. Jain, L. Koc, C. Y. Koo, L. Lew, C. Mewald, A. N. Modi, N. Polyzotis, S. Ramesh, S. Roy, S. E. Whang, M. Wicke, J. Wilkiewicz, X. Zhang, and M. Zinkevich, "TFX: A TensorFlow-based production-scale machine learning platform," in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2017, pp. 1387–1395.
[3] S. Ghanta, K. Karra, and N. Maskey, "Machine learning health monitor," U.S. Patent 10 318 882, Jun. 11, 2019.
[4] S. Pulicharla, "Automating machine learning workflows with Kubeflow and TensorFlow Extended (TFX)," Medium, 2019.
[5] K. Pei, Y. Cao, J. Yang, and S. Jana, "Towards practical verification of machine learning: The case of computer vision systems," arXiv preprint arXiv:1712.01785, 2017.
[6] W. Hummer, V. Muthusamy, T. Rausch, P. Dube, K. El Maghraoui, A. Murthi, and P. Oum, "ModelOps: Cloud-based lifecycle management for reliable and trusted AI," in Proc. IEEE Int. Conf. Cloud Engineering (IC2E), 2019, pp. 113–120.
[7] A. A. Dakkak, C. Li, S. Xiong, I. Gelado, and W.-M. Hwu, "MLModelScope: Evaluate and introspect cognitive pipelines," arXiv preprint arXiv:1909.07290, 2019.
[8] E. Breck, S. Cai, E. Nielsen, M. Salib, and D. Sculley, "The ML test score: A rubric for ML production readiness and technical debt reduction," in Proc. IEEE Int. Conf. Big Data, 2017, pp. 1123–1132.
[9] C. Renggli, B. Karlaš, B. Ding, F. Liu, K. Schawinski, W. Wu, and C. Zhang, "A data quality-driven view of MLOps," arXiv preprint arXiv:1909.05372, 2019.
[10] S. Udeshi, P. Arora, and S. Chattopadhyay, "Automated directed fairness testing," in Proc. 33rd ACM/IEEE Int. Conf. Automated Software Engineering, 2018, pp. 98–108.