Real-Time Log Analytics in Distributed Systems: Minimal-Latency Detection of Critical Events for Cloud-Native Back-End Platforms
Ivan Akimov , Software Engineer Dubai, United Arab EmiratesAbstract
The paper examines real-time log analytics for distributed, cloud-native back-end systems, where operational decisions depend on the rapid recognition of critical runtime conditions. The relevance follows from the latency sensitivity of microservice-based finance and trading workloads, where propagation of failures, retries, and cascading timeouts rapidly degrades user-facing and internal processing. The novelty lies in an integrated analytical synthesis that ties stream-processing scalability evidence, tracing-tool capabilities, monitoring-tool taxonomies, instrumentation overhead studies, and modern log-anomaly detection research into one consistent engineering narrative. The study aims to develop a low-latency detection approach based on peer-reviewed findings. To achieve this goal, the work employs a systematic selection of recent literature, structured extraction of architectural patterns, and comparative reasoning across the ingestion, correlation, detection, and alerting stages. The analysis encompasses distributed stream processing benchmarks, near-real-time processing in practical architectures, runtime verification for streaming systems, and state-of-the-art log anomaly detection methods. The closing part derives design implications for practitioners building observability and incident-response pipelines.
Keywords
real-time log analytics, distributed systems, stream processing, observability, microservices, critical event detection, low latency, tracing, anomaly detection, cloud platforms
References
Aladib, L., Su, G., & Yang, J. (2025). Real-Time Monitoring for Distributed Stream Processing Systems Using Linear Temporal Logic. Electronics, 14(7), 1448. https://doi.org/10.3390/electronics14071448
Daksa, D., & Kemala, E. (2025). Comparative Analysis of Apache Flink and Apache Spark for Near Real-Time Fraud Detection in Stream Processing Architecture. Procedia Computer Science, 242, 4691–4698. https://doi.org/10.1016/j.procs.2024.11.247
Duan, X., Du, D., Liu, Z., Zhu, H., & Liang, C. (2024). LogEDL: Log Anomaly Detection by Evidential Deep Learning. Applied Sciences, 14(16), 7055. https://doi.org/10.3390/app14167055
Giamattei, L., Guerriero, A., Pietrantuono, R., Russo, S., Malavolta, I., Islam, T., … Simon Panojo, F. (2024). Monitoring tools for DevOps and microservices: A systematic grey literature review. Journal of Systems and Software, 208, 111906. https://doi.org/10.1016/j.jss.2023.111906
Hammad, M., Ahmad, A. & Andras, P. (2025). An empirical study on the performance overhead of code instrumentation in containerised microservices. Journal of Systems and Software, 230, 112573. https://doi.org/10.1016/j.jss.2025.112573.
Henning, S., & Hasselbring, W. (2024). Benchmarking scalability of stream processing frameworks deployed as microservices in the cloud. Journal of Systems and Software, 208, 111879. https://doi.org/10.1016/j.jss.2023.111879
Janes, A., Li, X., & Lenarduzzi, V. (2023). Open tracing tools: Overview and critical comparison. Journal of Systems and Software, 204, 111793. https://doi.org/10.1016/j.jss.2023.111793
Khan, Z. A., Shin, D., Bianculli, D., & Briand, L. C. (2024). The Impact of Log Parsing on Deep Learning-based Anomaly Detection in System Logs. Empirical Software Engineering, 29, 10. https://doi.org/10.1007/s10664-023-10440-5
Landauer, M., & Skopik, F. (2023). Deep learning for anomaly detection in log data: A survey. Internet of Things and Cyber-Physical Systems, 5, 100071. https://doi.org/10.1016/j.iotcps.2023.100071
Li, W., Wu, Y., Huang, W., Ou, W., Wang, H., Zhou, F., & Deng, L. (2025). System log anomaly detection based on contrastive learning and retrieval augmented. Scientific Reports, 15, 38370. https://doi.org/10.1038/s41598-025-22208-7
Download and View Statistics
Copyright License
Copyright (c) 2026 Ivan Akimov

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright of their manuscripts, and all Open Access articles are disseminated under the terms of the Creative Commons Attribution License 4.0 (CC-BY), which licenses unrestricted use, distribution, and reproduction in any medium, provided that the original work is appropriately cited. The use of general descriptive names, trade names, trademarks, and so forth in this publication, even if not specifically identified, does not imply that these names are not protected by the relevant laws and regulations.


Engineering and Technology
| Open Access |
DOI: