Big Storage, Little Budget

Abstract

Kansas State University's HPC cluster was running out of storage space last year. Vendors of traditional HPC storage solutions were either too expensive to be feasible or too little capacity to be of long-term use. The system that ended up providing the best storage capacity for the available budget was Ceph, an open-source project that provides storage striped across many commodity servers. This session is a case study of the pros and cons of our implementation of a 1.5 PB Ceph-based storage cluster, discussing the history of network-based filesystems, including why our previous Gluster-based was no longer suitable. Questions and discussion are encouraged.