Performance of linear algebra building blocks for low-rank tensor algorithms on current multi-core CPUs and GPUs

Abstract

In this talk we discuss the node-level performance of basic operations needed in low-rank tensor algorithms. As starting point, we consider two specific problems: (1) compressing large dense data in the tensor-train (TT) format and (2) solving linear systems with an operator and right-hand side vector given in TT format. For both problems, we analyze and optimize suitable algorithms focussing on the required underlying operations such as tensor contractions and matrix decompositions. In particular, we obtain a significant speedup for orthogonalization and truncation steps by using a high-performance implementation of a "Q-less" tall-skinny QR decomposition. On multi-core CPUs, our implementation achieves a speedup of ~50x over reference implementations for the TT compression, and up to ~5x for solving linear systems. For GPUs, we give an overview of the performance of the most important operations and shortly discuss the implementation of a tall-skinny QR decomposition.