Modeling And Simulating Stream Processing Platforms

Stream processing platforms allow processing and analyzing real-time data. Several tools have been developed for these platforms to guarantee that the applications running on them are scalable, fast, and fault-tolerant and that they can be deployed on many processors. However, determining the proper number of processors suitable to hold a given stream processing-based software application is challenging, especially if the application is intended to serve a large user community. In this paper, we propose to model and simulate stream processing platforms for performance evaluation purposes. In our case study, we simulated a commonly used application for the analysis of Twitter streams with Storm. We evaluate its performance under different workloads. Our simulator supports profiling to measure various aspects of the application's performance. Results show that the simulator can replicate the metrics reported by the application running on a real platform with minimal error.