Idiaper wins EPFL's EEDE Thesis Award

The prize-winning work, entitled Stop Wasting My FLOPS: Improving the Efficiency of Deep Learning Models, was completed under the supervision of adjunct professor François Fleuret, former head of Idiap's Machine learning research group.

In his thesis, Katharopoulos proposes three methods for improving the efficiency of deep learning neural networks, which despite revolutionizing the field of machine learning, carry very costly computational and memory requirements. His efficiency recommendations focus in particular on an importance sampling algorithm to help improve the sample inefficiency of neural network training; a model for processing large input images with greatly reduced computational and memory requirements; and efficient approximations for the attention mechanism used in transformers which provide a better trade-off between performance and computation in comparison to original transformer architectures.

The annual EPFL EEDE Thesis Award honors an “outstanding and remarkable” PhD thesis work in the field of electrical engineering from an EEDE student.

More information

- Angelos Katharopoulos' thesis

- Idiap Machine learning research group

Written in collaboration with Celia Luterbacher (EPFL).