Are NVIDIA Tensor cores good for HPC?

The NVIDIA Volta GPU microarchitecture introduces a specialized unit, called Tensor Core that performs one matrix-multiply-and-accumulate on 4×4 matrices per clock cycle. The NVIDIA Tesla V100 accelerator, featuring the Volta microarchitecture, provides 640 Tensor Cores with a theoretical peak performance of 125 Tflops/s in mixed precision.

We investigate current approaches to program NVIDIA Tensor Cores, their performances and the precision loss due to computation in mixed precision. We found that NVIDIA Tensor Cores can deliver up to 83 Tflops/s in mixed precision on a Tesla V100 GPU, seven and three times the performance in single and half precision respectively.

While precision loss due to matrix multiplication with half precision input might be critical in many HPC applications, it can be considerably reduced at the cost of increased computation. Our results indicate that HPC applications using matrix multiplications can strongly benefit from using of NVIDIA Tensor Cores.

Ref: Stefano Markidis, Steven Wei-der Chien, Erwin Laure, Ivy Bo Peng, Jeffrey S. Vetter. 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)