Partenariat humain-IA
Ce programme de recherche capitalise sur l'expertise bien établie de l'Idiap en matière d'interaction multimodale. Il tire parti de la capacité unique de l'institut à entreprendre des recherches multidisciplinaires approfondies sur la communication verbale et non verbale, le traitement du langage, les systèmes perceptifs et cognitifs et l'interaction homme-robot.
L'objectif de ce programme est d'étendre les capacités humaines sur plusieurs aspects : créativité, limites cognitives, collaboration, connaissance. Ce programme vise à améliorer la détection et la compréhension des activités humaines par les machines, à améliorer l'accès à l'information, par exemple grâce à des chatbots servant d'experts de domaine à la demande, à utiliser le retour humain d'information pour améliorer les systèmes d'apprentissage et à utiliser des robots pour aider les humains dans leurs tâches quotidiennes au travail et à la maison.
Domaines d'expertise
#Bioinformatics&HealthInformatics
#DataScience&SocialComputing
#HumanComputerInteraction
#Imaging&ComputerVision
#MachineLearning
#NaturalLanguageProcessing
#Robotics&AutonomousSystems
#Security&Privacy
#SignalProcessing
#Speech&AudioProcessing
Ce programme contribue aux ODD des Nations-Unies suivants
Personnes
ABBET, Philip
(Senior Research and Development Engineer)
- website
AKSTINAITE, Vita
(Postdoctoral Researcher)
- website
BEBER, Luca
(Research Intern)
- website
BEN MAHMOUD, Imen
(Research and Development Engineer)
- website
BILALOGLU, Cem
(PhD Student / Research Assistant)
- website
BORNET, Olivier
(Head of Research and Development Team)
- website
BOU SLEIMAN, Maroun
(Visitor)
- website
BURDISSO, Sergio (Gastón)
(R&D / Research Assistant)
CALINON, Sylvain
(Senior Research Scientist)
- website
CANÉVET, Olivier
(Senior Research and Development Engineer)
- website
CAROFILIS VASCO, Roberto Andrés
(Postdoctoral Researcher)
- website
CARRON, Daniel
(Senior Research and Development Engineer)
- website
CHEN, Haolin
(PhD Student / Research Assistant)
- website
CLIVAZ, Guillaume
(Senior Research and Development Engineer)
- website
COMAN, Andrei (Catalin)
(PhD Student / Research Assistant)
- website
DAYER, Yannick
(Senior Research and Development Engineer)
- website
DELMAS, Maxime
(Postdoctoral Researcher)
- website
DROZ, William
(Senior Research and Development Engineer)
- website
EL HAJAL, Karl
(PhD Student / Research Assistant)
- website
EL ZEIN, Dina
(PhD Student / Research Assistant)
- website
ESPINOSA MENA, Jose Rafael
(PhD Student / Research Assistant)
- website
EUTAMENE, Camil
(Research Intern)
FARKHONDEH, Arya
(Research Intern)
- website
FEHR, Fabio (James)
(PhD Student / Research Assistant)
- website
FREITAS, André
(Research Scientist)
- website
FUCHS, Michael
(Postdoctoral Researcher)
- website
GAIST, Samuel
(Senior Research and Development Engineer)
- website
GARNER, Philip
(Senior Research Scientist)
- website
GIRAUD, Raphaël
(Research Intern)
- website
GUPTA, Anshul
(PhD Student / Research Assistant)
- website
HE, Mutian
(PhD Student / Research Assistant)
- website
HENDERSON, James (Brinton)
(Senior Research Scientist)
- website
HERMANN, Enno
(Postdoctoral Researcher)
- website
HERMUS, James
(Postdoctoral Researcher)
- website
HOVSEPYAN, Sevada
(Research Associate)
- website
JIANG, Liangze
(PhD Student / Research Assistant)
- website
JUNG, Vincent
(PhD Student / Research Assistant)
- website
KAYAL, Salim
(Senior Research and Development Engineer)
- website
KHALIL, Driss
(Junior R&D / Research Assistant)
- website
KULKARNI, Ajinkya (Vijay)
(Postdoctoral Researcher)
- website
KUMAR, Shashi
(PhD Student / Research Assistant)
- website
LI, Yiming
(PhD Student / Research Assistant)
- website
LÖW, Tobias
(PhD Student / Research Assistant)
- website
MACEIRAS, Jérémy
(Senior Research and Development Engineer)
- website
MAGIMAI DOSS, Mathew
(Senior Research Scientist)
- website
MARCEL, Christine
(Senior Research and Development Engineer)
- website
MARIĆ, Ante
(PhD Student / Research Assistant)
- website
MAYORAZ, André
(Research and Development Engineer)
- website
MICHEL, Samuel
(Research and Development Engineer)
- website
MITRO, Ioanni
(Postdoctoral Researcher)
- website
MOTLICEK, Petr
(Senior Research Scientist)
- website
MURALIDHAR, Skanda
(Research Associate)
- website
NANCHEN, Alexandre
(Senior Research and Development Engineer)
- website
ODOBEZ, Jean-Marc
(Senior Research Scientist with Academic Title)
- website
PRASAD, Amrutha
(PhD Student / Research Assistant)
- website
PUROHIT, Tilak
(PhD Student / Research Assistant)
- website
RAJAPAKSHE, Shalutha
(PhD Student / Research Assistant)
- website
RANGAPPA, Pradeep
(Postdoctoral Researcher)
- website
RAZMJOO FARD, Amirreza
(PhD Student / Research Assistant)
- website
SANCHEZ LARA, Alejandra
(Research Intern)
- website
SANCHEZ-CORTES, Dairazalia
(Research Assistant)
- website
SARKAR, Eklavya
(PhD Student / Research Assistant)
- website
SCHONGER, Martin
(PhD Student / Research Assistant)
- website
SENFT, Emmanuel
(Research Scientist (CRG Head))
- website
SHAN, Jennifer
(Research Intern)
- website
SHIRAKAMI, Haruki
(PhD Student / Research Assistant)
- website
TAFASCA, Samy
(PhD Student / Research Assistant)
- website
TARIGOPULA, Neha
(PhD Student / Research Assistant)
- website
TENEY, Damien
(Research Scientist)
- website
THORBECKE (NIGMATULINA), Iuliia
(PhD Student / Research Assistant)
- website
TIMONINA-FARKAS, Anna
(Research Associate)
- website
TORNAY, Sandrine
(Postdoctoral Researcher)
- website
VALENTINO, Marco
(Postdoctoral Researcher)
- website
VAN DER PLAS, Lonneke
(External Research Fellow)
- website
VANNAY, Gabriel
(Apprentice)
- website
VÁSQUEZ RODRÍGUEZ, Laura
(Postdoctoral Researcher)
- website
VERZAT, Colombine
(Senior Research and Development Engineer)
- website
VILLAMIZAR, Michael (Alejandro)
(Research Associate)
- website
VILLATORO TELLO, Esaú
(Research Associate)
- website
VUILLECARD, Pierre
(PhD Student / Research Assistant)
- website
WANG, Anlan
(Research Intern)
WYSOCKA, Magdalena
(Research Intern)
- website
WYSOCKI, Oskar
(Postdoctoral Researcher)
- website
ZANGGER, Alicia
(Research and Development Engineer)
- website
ZHANG, Alice (Chi)
(Visitor)
- website
ZHANG, Yan
(PhD Student / Research Assistant)
- website
Publications choisies
Neural Network Adaptation and Data Augmentation for MultiSpeaker Direction-of-Arrival Estimation, W. He, P. Motlicek and J.-M. Odobez, IEEE/ACM Trans. on Audio, Speech and Language Processing, 29, pp. 1303-1317, 2021.
The first viable deep learning framework (task definition, network architecture, training paradigm) for solving fundamental auditory tasks such as sound source localization, speaker identification and speech/non-speech classification. The framework is suitable for highly noisy environments and overcomes limitations of previous methods, which heavily relied on idealized sound and environment models and are inadequate for everyday situations with multiple sound sources, background noise, short utterances, and lack of prior knowledge on the number of sound sources. The method learns sound source localization models with limited training resources leveraging simulated and weakly-labeled real audio data.
Active Learning by Feature Mixing, A. Parvaneh, E. Abbasnejad, D. Teney, G. R. Haffari, A. Van Den Hengel, & J. Q. Shi, In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 12227-12236, 2022.
A method to train deep learning models with humans in the loop. Current approaches to machine learning depend on large amounts of data that are costly or difficult to acquire. This paper presents an active learning approach where human experts interact with the learning algorithm to iteratively refine and resolve inconsistencies in a model by labeling a small set of training examples. This approach contributes to widening the accessibility of machine learning technologies to small organizations.
Learning Joint Space Reference Manifold for Reliable Physical Assistance, Razmjoo, A., Brecelj, T., Savevska, K., Ude, A., Petric, T. and Calinon, S., In Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), 2023.
Projets choisis
C-LING, 2022-2026, SNSF, Van der Plas: TOWARDS CREATIVE SYSTEMS WITH LINGUISTIC MODELLING
This project aims to investigate what aspects computational models need to perform creative cognitive tasks, from generating relatively simple novel concepts to more complex and structured ideas, across multiple domains and languages. More in particular, it aims to answer what types of structured and unstructured knowledge are needed and what models best integrate these types of knowledge.
NeuMath, 2022-2024, SNSF, Freitas: NEUMATH: NEURAL DISCOURSE INFERENCE OVER MATHEMATICAL TEXTS
NeuMath will develop models which can jointly represent and reason over two symbolic modalities (natural language and mathematical expressions) and will build the foundations to deliver embedding models which can interpret and support the generation of mathematical arguments (by leveraging available large-scale scientific corpora).
SMILE-II, 2021-2024, SNSF Sinergia, Magimai Doss: SMILE-II SCALABLE MULTIMODAL SIGN LANGUAGE TECHNOLOGY FOR SIGN LANGUAGE LEARNING AND ASSESSMENT PHASE-II
The proposed project SMILE-II aims to research and build advanced technology for sign language learning. More precisely, the proposed project builds on the groundwork laid down by the SNSF Sinergia project SMILE, which dealt with assessment of the manual activity of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) in isolated signs produced by early learners and L2 learners. SMILE-II will extend this technology to continuous sign language assessment including both manual and non-manual components of signs so that a DSGS learner’s sentence-level production can be assessed in an automatic manner.
Liste de tous les projets
C-LING, 2022-2026, SNSF, Van der Plas
Building computational models of human creative thinking to help with creative tasks
NeuMath, 2022-2024, SNSF, Freitas
Neuro-symbolic architectures for supporting mathematical discovery
NAST, 2020-2024, SNSF, Garner
Neural architectures for speech technology
SteADI, 2021-2025, SNSF, Garner
Storytelling algorithms for digital interviews
NKBP, 2020-2024, SNSF, Henderson
Deep learning models for continual extraction of knowledge from text
SINFONIA, 2023-2027, Innosuisse, Teney, Freitas
Generalization and domain adaptation of large language models
LUCIDELES, 2020-2023, SFOE, Kämpf
Research at the interface between humans and building control systems
CODIMAN, 2020-2024, National Research Programme "Digital Transformation", SNSF, Calinon
Cobotics, digital skills and the re-humanization of the workplace
SESTOSENSO, 2022-2025, Horizon Europe, Calinon
Physical cognition for intelligent control and safe human-robot interaction
SMILE-II, 2021-2024, SNSF Sinergia, Magimai Doss
Assistive technology for sign language learning and testing
Amazon research award, 2023, Teney
Addressing underspecification for improved fairness and robustness in conversational AI
MALORCA, 2016-2018, EU, Motlicek
Machine learning of speech recognition models for controller assistance
HAAWAII, 2020-2022, EU, Motlicek
Highly automated air-traffic controller workstations with artificial intelligence integration
ATCO2, 2019-2022, EU, Motlicek
Automatic collection and processing of voice data from air-traffic communications
EUROCONTROL, 2023-2024, France, Motlicek
Automatic speech recognition in air-traffic control simulation