In this talk I will review some of the numerical challenges that arise when training Gaussian processes. We will start from some basic definitions of these popular models within machine learning and then arise at two problems often encountered in scientific computing. Namely, the solution of a linear system and the evaluation of a matrix function. We focus on the particular case that the kernel matrix can be interpreted as a tensor for non-trivial rank and as such require numerical methods tailored for tensor operations.