Block Krylov and Jacobi-Davidson methods on heterogenous systems

Abstract

Over the past five years we have developed two open source software packages called GHOST and PHIST (https://bitbucket.org/essex/[ghost|phist]. We discuss the software and performance engineering techniques used when designing these libraries and show some examples of use. GHOST provides optimized implementations of memory-bounded linear algebra operations on heterogenous CPU/GPU systems. PHIST provides the software infrastructure for implementing iterative sparse matrix algorithms in a portable and efficient way by introducing a kernel interface layer inspired by the message passing interface (MPI). Implementations of the interface are verified using an extensive test suite and performance models. Going beyond the isolated optimzation of linear algebra kernels, phist allows algorithm-level performance optimizations like kernel fusion and overlapping of communication and computation. To make phist algorithms easy to integrate into existing applications, we provide implementations of the kernel interface for various commonly used libraries such as Trilinos, PETSc and Eigen, and a Fortran+MPI reference implementation. Besides the standard C interface, Pyton, C++ and Fortran bindings are automatically generated for all functions. We show how the new libraries can be used to boost the performance of existing implementations of Block Krylov solvers in the Trilinos package Anasazi, and present results for our own implementation of the block Jacobi-Davidson QR method applied to model problems from quantum physics.