Using balanced data placement to address I/O contention in production environments

• Verfasst von: Neuwirth, Sarah [VerfasserIn]
• Verfasst von: Brüning, Ulrich [VerfasserIn]
• Titel: Using balanced data placement to address I/O contention in production environments
• Verf.angabe: Sarah Neuwirth, Feiyi Wang, Sarp Oral, Sudharshan Vazhkudai, James H. Rogers and Ulrich Bruening
• Umfang: 9 S.
• Fussnoten: Gesehen am 23.05.2018
• Titel Quelle: Enthalten in: International Symposium on Computer Architecture and High Performance Computing (28. : 2016 : Los Angeles, CA): 2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)
• Jahr Quelle: 2016
• Band/Heft Quelle: (2016), S. 9-17
• ISBN Quelle: 978-1-5090-6108-2
• DOI: doi:10.1109/SBAC-PAD.2016.10
• Datenträger: Online-Ressource
• Sprache: eng
• K10plus-PPN: 1575446774

Abstract

Designed for capacity and capability, HPC I/O systems are inherently complex and shared among multiple, concurrent jobs competing for resources. Lack of centralized coordination and control often render the end-to-end I/O paths vulnerable to load imbalance and contention. With the emergence of data-intensive HPC applications, storage systems are further contended for performance and scalability. This paper proposes to unify two key approaches to tackle the imbalanced use of I/O resources and to achieve an end-to-end I/O performance improvement in the most transparent way. First, it utilizes a topology-aware, Balanced Placement I/O method (BPIO) for mitigating resource contention. Second, it takes advantage of the platform-neutral ADIOS middleware, which provides a flexible I/O mechanism for scientific applications. By integrating BPIO with ADIOS, referred to as Aequilibro, we obtain an end-to-end and per job I/O performance improvement for ADIOS-enabled HPC applications without requiring any code changes. Aequilibro can be applied to almost any HPC platform and is mostly suitable for systems that lack a centralized file system resource manager. We demonstrate the effectiveness of our integration on the Titan system at the Oak Ridge National Laboratory. Our experiments with a synthetic benchmark and real-world HPC workload show that, even in a noisy production environment, Aequilibro can improve large-scale application performance significantly.