Anomaly detection model with deep learning methods from big data perspective in kubernetes architecture Kubernetes mimarisinde büyük veri perspektifinden derin öğrenme yöntemleriyle anomali tespit modeli
Journal of the Faculty of Engineering and Architecture of Gazi University, cilt.40, sa.3, ss.1647-1658, 2025 (SCI-Expanded, Scopus, TRDizin)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 40 Sayı: 3
- Basım Tarihi: 2025
- Doi Numarası: 10.17341/gazimmfd.1333162
- Dergi Adı: Journal of the Faculty of Engineering and Architecture of Gazi University
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, TR DİZİN (ULAKBİM)
- Sayfa Sayıları: ss.1647-1658
- Anahtar Kelimeler: Anomaly detection, Big data, Deep Learning, Kubernetes, Traffic data
- Ankara Yıldırım Beyazıt Üniversitesi Adresli: Hayır
Özet
Today, with the increasing number of sensors, data is generated at high speed, in various formats, and in large volumes. Analyzing this rapidly produced data from different sources is crucial. To address this challenge, big data systems with layered architecture offer viable solutions. Each layer contains different applications that interact with one another. This study introduces a model for combining sensor data using the Kubernetes architecture, incorporating anomaly detection through artificial intelligence methods, and processing stream data. The proposed model comprises three stages: data processing and merging, an artificial intelligence-based model, and visualization. To facilitate these stages, the orchestration process utilizing the Kubernetes architecture enables the creation of data merging, processing, and visualization processes by leveraging open-source applications. This approach allows for the development of a distributed, fault-tolerant, and efficient resource management system. In the anomaly detection stage, a comparative analysis is conducted using artificial intelligence algorithms, namely LSTM, GRU, and Conv1d. Additionally, the results are presented by comparing processor usage rates during normal system operation and during the training and stream data analysis phases. Consequently, we propose a Kubernetes-based anomaly detection model that encompasses the stages of data collection, data processing, and visualization. The proposed system model successfully combines streaming data and executes operations in a distributed and fault-tolerant manner during the analysis process.