★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★Open research. Global perspectives.
editor@theamericanjournals.comPublisher Home ↗
Articles OPEN ACCESS

A Review Of Airflow-Based And Cloud-Native Approaches For Workflow Orchestration In Devops

Basheer Ahmedu Basha
Principal Build and Release Engineer IEEE Member,USA
DOI Crossref DOI 10.37547/tajiir/Volume08Issue10-06
tajiir 2026
VOL. 8 / NO. 10 OCTOBER
FULL TEXT PP. 55-64
DOWNLOAD PDF

View all articles in this issue ↗

VOLUME 8
ISSUE 10
YEAR 2026
PAGES 55-64
RESEARCH OVERVIEW

Abstract

Workflow orchestration has become an essential component of modern DevOps environments to automate, schedule, monitor, and coordinate complex software and data workflows. To create scalable and robust operations, use cloud-native techniques and CI/CD. Apache Airflow provides a DAG-based solution to task dependency, scheduling, monitoring and distributed process execution. This analysis examines cloud-native and Airflow-based workflow orchestration methods with a focus on their design, attributes, pros, cons, scalability, fault tolerance, resource consumption and complexity. To assess the advancement of workflow orchestration and the role Airflow plays in modern DevOps workflows, researchers examine past research. Researchers review past studies to gauge the evolution of workflow orchestration and the role that Airflow plays in modern DevOps pipelines. The review also looks into new strategies for cloud-native security, automated scaling, multi-cloud orchestration, machine learning-based failure prediction, and observability. The study indicates that cloud-native orchestration provides greater scalability, automation, resilience, and flexibility in infrastructure, while Airflow provides strong dependency management capability, flexibility in scheduling, monitoring, and extensibility. There are still challenges in real-time processing, complexity in operations, cost management, multi-cloud environments, and adaptive orchestration. The highlighted research gaps highlight the need for intelligent and comprehensive orchestration frameworks that integrate cloud-native technologies, predictive analytics, and workflow management. Overall, results indicate that the cloud-native orchestration framework along with the DAG-based workflow management can provide a scalable and flexible foundation for modern workflow automation in DevOps.

RESEARCH TOPICS

Keywords

Cloud Computing Apache Airflow Cloud-Native Orchestration Workflow DevOps

Downloads

Download data is not yet available.
SOURCES & SCHOLARSHIP

References

  1. K. Petrakis et al., “Enhancing DevOps Practices in the IoT–Edge–Cloud Continuum: Architecture, Integration, and Software Orchestration Demonstrated in the COGNIFOG Framework,” Software, vol. 4, no. 2, p. 10, Apr. 2025, doi: 10.3390/software4020010.
  2. Y. Wang, C. Lee, S. Ren, E. Kim, and S. Chung, “Enabling Role-Based Orchestration for Cloud Applications,” Appl. Sci., vol. 11, no. 14, p. 6656, Jul. 2021, doi: 10.3390/app11146656.
  3. I. Kazlaris, G. Papadopoulos, K. Diamantaras, M. Delianidi, E. Touliou, and A. Yenitzes, “An Airflow-Orchestrated AI Pipeline for Podcast Transcription, Topic Modeling, and Recommendation System,” Multimedia, vol. 2, no. 1, p. 1, Jan. 2026, doi: 10.3390/multimedia2010001.
  4. N. Karra, N. Karra, L. Echefu, and G. Bandapati, “DevOps-Enabled Cloud-Native Architectures for Scalable and Resilient IoT Applications: A Kubernetes-Based Approach,” in 2025 9th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), IEEE, Oct. 2025, pp. 48–53. doi: 10.1109/I-SMAC65734.2025.11597587.
  5. C. Lekkala, “Implementing Cloud-Native Technologies for Big Data Processing: A Case Study with Kubernetes and Airflow,” Int. J. Sci. Res., vol. 10, no. 5, pp. 1335–1340, May 2021, doi: 10.21275/SR24430152128.
  6. T. K. Chatterjee, “AI-Enabled Cloud Orchestration for Automated Workflows: A Paradigm Shift in Enterprise IT Operations,” Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., vol. 11, no. 2, pp. 3289–3296, Apr. 2025, doi: 10.32628/CSEIT25112809.
  7. Y. Muni, “A Review of Efficient Routing Architectures Using OSPF, BGP, and MPLS in Multi-Protocol Networks,” Eng. Technol. J., vol. 11, no. 01, Jan. 2026, doi: 10.47191/etj/v11i01.21.
  8. S. Vijayabaskar, P. R. Rao, P. Kanchi, S. Jain, and R. Agarwal, “Integrating Cloud-Native Solutions in Financial Services for Enhanced Operational Efficiency,” Univers. Res. Reports, vol. 10, no. 4, pp. 402–419, Oct. 2023, doi: 10.36676/urr.v10.i4.1355.
  9. J. Rahman and P. Lama, “Predicting the End-to-End Tail Latency of Containerized Microservices in the Cloud,” in 2019 IEEE International Conference on Cloud Engineering (IC2E), IEEE, Jun. 2019, pp. 200–210. doi: 10.1109/IC2E.2019.00034.
  10. A. P. Rajan, “A review on serverless architectures - function as a service (FaaS) in cloud computing,” TELKOMNIKA (Telecommunication Comput. Electron. Control., vol. 18, no. 1, p. 530, Feb. 2020, doi: 10.12928/telkomnika.v18i1.12169.
  11. I. Baldini et al., “Serverless Computing: Current Trends and Open Problems,” in Research Advances in Cloud Computing, Singapore: Springer Singapore, 2017, pp. 1–20. doi: 10.1007/978-981-10-5026-8_1.
  12. M. Waseem, P. Liang, and M. Shahin, “A Systematic Mapping Study on Microservices Architecture in DevOps,” J. Syst. Softw., vol. 170, p. 110798, Dec. 2020, doi: 10.1016/j.jss.2020.110798.
  13. D. S. Linthicum, “Cloud-Native Applications and Cloud Migration: The Good, the Bad, and the Points Between,” IEEE Cloud Comput., vol. 4, no. 5, pp. 12–14, Sep. 2017, doi: 10.1109/MCC.2017.4250932.
  14. M. R. C. Mukkolakkal, “1.5 Million Messages Per Second on 3 Machines : Benchmarking and Latency Optimization of Apache Pulsar at Enterprise Scale,” arXiv J., pp. 8–11, 2026, doi: 10.48550/arXiv.2603.29113.
  15. J. Yasmin, J. A. Wang, Y. Tian, and B. Adams, “An empirical study of developers’ challenges in implementing Workflows as Code: A case study on Apache Airflow,” J. Syst. Softw., vol. 219, p. 112248, Jan. 2025, doi: 10.1016/j.jss.2024.112248.
  16. S. K. Malaraju, G. C. Kakaraparthi, and R. C. Bondalapati, “Efficient Deployment of Scalable AI Models via Kubernetes Orchestration in Hybrid Cloud Setups,” in 2025 International Conference on Intelligent and Secure Engineering Solutions (CISES), IEEE, Aug. 2025, pp. 1450–1454. doi: 10.1109/CISES66934.2025.11265182.
  17. M. Avirneni and S. R. Sankiti, “Scalable Multi Cloud Data Engineering with Airflow Orchestration and Iceberg Driven Governance,” 2026. doi: 10.2139/ssrn.5926642.
  18. S. Sanghi, S. Sudhakaran, V. K. Koganti, and N. Ryali, “DevOps and Secure Cloud-Native Architectures for Finance,” in 2025 International Conference on Sustainability, Innovation & Technology (ICSIT), IEEE, Aug. 2025, pp. 1–5. doi: 10.1109/ICSIT65336.2025.11294986.
  19. S. S. Poojary, S. Kamble, and G. S. Mamatha, “A Scalable DevOps Framework Using Kubernetes-Orchestrated Microservices for Cloud-Native Infrastructure,” in 2025 9th International Conference on Computational System and Information Technology for Sustainable Solutions (CSITSS), IEEE, Nov. 2025, pp. 1–6. doi: 10.1109/CSITSS67709.2025.11294211.
  20. S. Lakkireddy, “HyScaleFlow: An ML-Driven DAG-Based Orchestration Framework for Real-Time Stream Processing in Hybrid Cloud Environments,” Informatica, vol. 49, no. 9, Oct. 2025, doi: 10.31449/inf.v49i9.9498.
  21. O. C. Oyeniran, O. T. Modupe, A. A. Otitoola, O. O. Abiona, A. O. Adewusi, and Oluwatayo Jacob Oladapo, “A comprehensive review of leveraging cloud-native technologies for scalability and resilience in software development,” Int. J. Sci. Res. Arch., vol. 11, no. 2, pp. 330–337, Mar. 2024, doi: 10.30574/ijsra.2024.11.2.0432.
  22. C. A. Supit, A. A. Pangeran, K. O. C. Laban, I. I. Gutandjala, and A. Ramadhan, “Incorporating Cloud Native Architecture and DevOps Culture to Improve Company Agility,” in 2023 3rd International Conference on Intelligent Cybernetics Technology and Applications, ICICyTA 2023, 2023. doi: 10.1109/ICICyTA60173.2023.10428951.
  23. A. Harika, P. Bhavani, P. Sriteja, S. Tajuddin, and S. S. Harsha, “Optimizing Scalability and Resilience: Strategies for Aligning DevOps and Cloud-Native Approaches,” in 3rd International Conference on Innovative Mechanisms for Industry Applications, ICIMIA 2023 - Proceedings, 2023. doi: 10.1109/ICIMIA60377.2023.10426288.
  24. J. K. Paruchuri, “Survey of Cloud-Native Workflow Orchestration with Apache Airflow,” Int. J. Comput. Tech., vol. 9, no. 3, pp. 1–15, 2022.