Understanding Microservice Runtime Monitoring Data for Anomaly Detection with Structural Equation Modeling

Kurzfassung

Microservices reliability is critical, but runtime anomalies are increasingly common due to system complexity. Rule-based and AI-based anomaly detection methods assist practitioners in analyzing runtime monitoring data (logs, traces, metrics) to identify anomalies. However, these methods rely on high-quality datasets and deep domain knowledge to deliver accurate results. Thus, a significant challenge lies in the lack of consensus on which runtime monitoring parameters effectively represent the system and microservices, reliably indicate anomalies, or distinguish deviations that genuinely signal anomalies. A thorough understanding of the dataset, key monitoring parameters, and microservice dependencies is crucial to minimize bias and false positives, ultimately improving the effectiveness of anomaly detection methods. Thus, we investigate whether structural equation modeling can describe the system's or microservices' behavior via indicators extracted from runtime monitoring data and identify their causal relationships. We used EvoMaster to simulate user behavior in TrainTicket and extract runtime monitoring data to test our model. Our results show that the identified indicators effectively describe microservices' behavior, but network indicators alone are insufficient for describing the whole system's behavior. The model can also identify microservices that significantly influence the whole system's behavior.