Autonomous Resource Management in Distributed Stream Processing Systems

Resource management in Distributed Stream Processing Systems (DSPS) defines the way queries are deployed on in-network resources to deliver query results while fulfilling the Quality of Service (QoS) requirements of the end-users. Various resource management mechanisms have been proposed in DSPS; however, they become inefficient in challenging conditions imposed by the dynamic environment and heterogeneous resources. This is because they focus on pre-configuration of both single and static QoS requirements. In addition, they lack cooperation between heterogeneous resources which amplify the problem of coordination between resources. This could lead to severe performance degradation such as inconsistent and incorrect query results in comparison to homogeneous resources. To solve the above challenges, in this research work, we will propose mechanisms: (i) to forecast the performance of network and heterogeneous resources, (ii) to select an efficient resource management approach, and (iii) for cooperation between resources in a dynamic environment