How can Tensor Decompositions be used for Neural Network Compression? Collaboration by two ELLIS faculty presented at ICLR 2023

News - 15 February 2023 - ELLIS Unit Delft

This research is a collaboration by two ELLIS Delft researchers, Kim Batselier (DCSC, 3ME) and Julian Kooij (Cognitive Robotics, 3ME), and was funded by the 3ME Cohesion Grant.


At ICLR 2023 we will present our study on the applicability of tensor decompositions for neural network compression. Specifically, we investigate if the approximation error of a compressed weight tensor w.r.t. the original weight tensor is indicative of the compressed model’s performance, and if this can be used to select amongst hyperparameter choices for the decomposition. Our findings show that the approximation error is indeed a useful heuristic, suggesting that future network compression techniques could efficiently explore more varied types of tensor decomposition.

"How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?", J. Schuurmans, K. Batselier, J.F.P. Kooij, International Conference on Learning Representations (ICLR), 2023.