User-centric workload analytics: Towards better cluster management

Abstract

Effective management of computing clusters and providing a high quality customer support is not a trivial task. Due to rise of community clusters there is an increase in the diversity of workloads and the user demographic. Owing to this and privacy concerns of the user, it is difficult to identify performance issues, reduce resource wastage and understand implicit user demands. In this thesis, we perform in-depth analysis of user behavior, performance issues, resource usage patterns and failures in the workloads collected from a university-wide community cluster and two clusters maintained by a government lab. We also introduce a set of novel analysis techniques that can be used to identify many hidden patterns and diagnose performance issues. Based on our analysis, we provide concrete suggestions for the cluster administrator and present case studies highlighting how such information can be used to proactively solve many user issues, ultimately leading to better quality of service