Pretrained LLMs have demonstrated impressive abilities, but it is hard to understand how they work or how well they will generalise to a new domain. Idiap researchers are developing a model of how information is represented inside LLMs. By identifying and removing unreliable information, this model can improve generalisation to new domains, without the need or any additional training.
Bayesian Language Understanding with Nonparametric Variational Transformers - Talk Jan 29 2024
As artificial intelligence technologies reach everyday new performances, their energy cost is also increasing significantly. Idiap researchers are proposing a novel approach to address this challenge during a period of rising energy costs.
Idiap Research Institute and the School of Engineering at EPFL invite applications for the directorship of Idiap. The successful candidate will also hold a faculty position as full professor at EPFL School of Engineering.
Demonstrating a significant improvements over the previous state-of-the-art results, James Henderson and Alireza Mohammadshahi of the Natural language understanding group propose a novel approach. Their work was presented during the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL) on April 21, 2021.
Perfect understanding of a text is not possible without context. Same goes for a computer. It cannot do a good translation without understanding what is really defined in a text. At Idiap, Lesly Miculicich conducted research to make this possible through her thesis which she successfully defended last February.
An article, co-authored by Prof. Andrei Popescu-Belis, has received the Best Paper Award at the International Joint Conference on Natural Language Processing (IJCNLP). The article presents the research of Dr Nikolaos Pappas - a postdoc at the Idiap Research Institute in Martigny - under the supervision of Prof. Popescu-Belis, former head of Idiap’s Natural Language Processing group.
James Henderson, who has joined Idiap in September 2017 and is now leading Idiap's Natural Language Understanding group, has recently been appointed as Action Editorfor the "Transactions of Association for Computational Linguistics (TACL)" journal.
The paper "Multilingual Hierarchical Attention Networks for Document Classification" by Nikolaos Pappas and Andrei Popescu-Belis has received the Best Paper Award at the 8th International Joint Conference on Natural Language Processing, held on November 27-December 1 in Tapei, Taiwan.
The paper "Multilingual Visual Sentiment Concept Matching" by Nikolaos Pappas, Mercan Topkara, Miriam Redi, Brendan Jou, Tao Chen, Hongyi Liu, Shih-Fu Chang has received the Best Multimodal Paper Award at the Annual ACM International Conference on Multimedia Retrieval (ICMR), held on June 6-9 in New York.
The Idiap NLP group has participated in the search and hyperlinking task at the MediaEval 2013 evaluation campaign. The NLP group was ranked first out of eleven participants on the hyperlinking task (finding video segments related to a given one) and third out of seven on the keyword video search task.
How do Large Language Models Represent Information?
