Automatic performance engineering workflows for high performance computing
Autori
Viac o knihe
During the typical performance engineering process, application developers must often complete multiple iterative tasks to get an in-depth runtime profile of their application in order to identify new optimization opportunities and implement an effective tuning strategy. However, the majority of today's mainstream tools do not support common workflows like analyzing the scalability and stability of parallel applications, studying their dynamic behavior, optimizing them for a particular architecture or even improving their power efficiency to help lower the operational costs of High-Performance Computing (HPC) centers. This, combined with the high complexity of today's computing systems, makes it extremely difficult for the developers to collect, maintain, and organize data from numerous performance engineering experiments and simultaneously track the evolution of their code throughout the overall software development and tuning cycle. However, problems such as long-running iterative processes, diverse data sources and totally different environments exist also in other research areas. Making relevant data-based decisions that lead to improving process efficiency has been the goal of business intelligence (BI) for many years now. A lot of research has been done in this field in order to accelerate and improve the overall decision-making process and create a smooth and stable working environment. Moreover, many tools have been developed to support the processing of huge data quantities and create model-driven, easier to use systems that greatly automate the overall analysis process. Consequently, this dissertation explores the concept of process automation in the field of parallel performance engineering and proposes a framework to support application developers in structuring and executing the overall tuning process and simultaneously tracking the evolution of the source code that is part of it. It adopts established standards and research ideas from the field of business intelligence and adapts them to the scientific domain of performance analysis and tuning of HPC applications.