The Helmholtz Analytics Toolkit (HEAT): A scientific Big Data Library for HPC

Kurzfassung

We present the Helmholtz Analytics Toolkit (HeAT), a scientific big data analytics library for HPC systems. The large progress in big data analytics in general and machine learning/deep learning (ML/DL) in particular, has been considerably enforced by well-designed open source libraries like Hadoop, Spark, Storm, Disco, scikit-learn, H2O.ai, Mahout, TensorFlow, PaddlePaddle, PyTorch, Caffe, Keras, MXNet, CNTK, BigDL, Theano, Neon, Chainer, DyNet, Dask and Intel DAAL to mention only a few of them. Despite the large number of existing data analytics frameworks, a library taking the specific needs in scientific big data analytics under consideration is still missing. For instance, no pre-existing library operates on heterogeneous hardware like GPU/CPU systems while allowing transparent computation on distributed systems. Typical big data analytics frameworks like Spark are designed for distributed memory systems and consequently do not fully explore the shared memory architecture as well as the network technology of HPC systems. ML/DL frameworks like Theano or Chainer focus on single node computations or when providing mechanisms for distributed computation, as done by TensorFlow or PytTorch, they impose the details of the distributed computation to the programmer. Libraries designed for HPC like Dask and Intel DAAL do not provide any GPU support. The presented library - HeAT - is designed for the specific needs of big data analytics in the scientific context. It is based on a tensor data object on which operations can be performed like basic scalar functions, linear algebra algorithms, slicing or broadcasting operations necessary for most data analytics algorithms. The tensor data objects reside either on the CPU or on the GPU and, if needed, is distributed over different nodes. Operations on tensor objects are transparent to the user, i.e. they keep the same irrespective whether the tensor object resides on a single node or if it is distributed over several nodes allowing conveniently port algorithms from single nodes to multiple nodes or from CPUs to GPUs. HeAT builds in top of PyTorch which already provides many required features like automatic differentiation, CPU and GPU support, linear algebra operations and basic MPI functionality as well as an imperative programming paradigm allowing for fast prototyping essentially in scientific research. In addition to basic tensor operations, HeAT implements several common data analytics algorithms, e.g. kmeans, logistic regression and neural networks, optimized for large scale distributed systems. After motivating the framework and specifying its scope, we will describe its concept and its realization in detail. We demonstrate its usage by means of several typical examples from data analytics including benchmarks comparing HeAT to pre-existing frameworks. The presentation closes with a discussion on the downsides, further developments and future challenges of HeAT.