Perception and Machine-Learning
Research on Perception and Machine-Learning
Robotic systems are generally based on the classic cycle: Perception → Reasoning+Decision/Planning → Action. The goal of the "Perception and Machine-Learning" research axis is to improve the ability to perceive and interpret in real time the environment of robots. Note that we consider robots in a broad meaning, including mobile robots, collaborative robots, an also intelligent vehicles. More specifically, the research and scientific contributions of this theme are covering the following topics:
- Real-time computer-vision
- Multi-sensor fusion
- Ego-localization and SLAM
- Machine-Learning (ML) in general and Deep-Learning (DL) in particular
- Pattern recognition (categorization or identification)
- Gesture or action recognition
- Time-series prediction
- Reinforcement-Learning (RL)
People
Permanent staff:
- Fabien MOUTARDE: Full Professor
- Bogdan STANCIULESCU: Associate Professor
- Cyril JOLY: Associate Professor
- Hsiu-Wen (Kelly) CHANG : Associate Professor
- Bruno SAUVALLE: Associate Professor
- Sascha HORNAUER: Associate Professor (from Nov.1st 2023)
PhD students:
- Simon DE MOREAU [March2023-...] : End-to-end holistic vehicle lighting, CIFRE PhD with Valeo lighting [PhD thesis director = Fabien Moutarde, PhD thesis co-supervisor = Bogdan Stanciulescu]
- Jérémy OCHIN [Feb.2023-...]: Algorithms of action recognition in sports with application to soccer, CIFRE PhD with Footovision [PhD thesis director = Sotiris Manitsaris, PhD thesis co-supervisor = Bogdan Stanciulescu]
- Angelika ANDO [Dec.2022-...] : Embedded 3D visualization using Neural Rendering, PhD funded by FORVIA [PhD thesis director = Fabien Moutarde, PhD thesis co-supervisor = Bogdan Stanciulescu]
- Amandine BRUNETTO [November2022-...] : Exploitation of sound for real-time reconstruction and understanding of 3D scenes, PhD thesis funded by Franco-American project (ANR-NSF) Omni-BatVision [PhD thesis director = Fabien Moutarde, PhD thesis co-supervisor = Sascha Hornauer]
- Iad ABDUL RAOUF [Jan.2022-...] : Magnetic field and vision-based loop closure, CEA-LIST PhD thesis [PhD thesis director = Alexis Paljic, PhD thesis co-supervisor = Cyril Joly]
- Raphaël CHEKROUN [Apr.2021-...] : Deep Reinforcement Learning for autonomous Driving (CIFRE Valeo Driving Assistance) [PhD thesis director = Fabien Moutarde, PhD thesis co-supervisor= Sascha Hornauer]
- Jesus BUJALANCE-MARTIN [Dec.2019-...] : Deep Reinforcement Learning for teaching tasks to manipulation robots [PhD thesis director = Fabien Moutarde]
Former PhD students:
- Thomas GILLES [March2020-Apr.2023]: Smart vehicule trajectory prediction in various autonomous driving scenarios (CIFRE PhD with Huawei-France)
- Arthur MOREAU [March2020-Apr.2023] : Learning-based algorithms for real-time visual localization of vehicles (CIFRE PhD with Huawei-France)
- Jade COULIN [Jan.2020-Mar.2023] : Magneto-Visual-Inertial SLAM for Indoor Localization (PhD funded by CEA-LIST)
- Joseph GESNOUIN [Nov.2019-Sept.2022] : Analysis of pedestrian postures and movement for predicting their intentions and trajectories (PhD funded by VeDeCom)
- Marin TOROMANOFF [Jan.2018-Mar.2021]: End-to-end Autonomous Driving using Deep Reinforcement Learning, CIFRE PhD with Valeo (Driving Assistance R&D) / Valeo.ai
- Guillaume DEVINEAU [Oct.2016-Sept.2020] : Deep-Learning for Multivariate Time Series: from Vehicle Control to Gesture Recognition and Generation
- Mathieu NOWAKOWSKI [Mar.2016-Mar.2019] : Localization of a humanoid robot in a non-constrained indoor environment, CIFRE PhD thesis with Aldebaran
- Li YU [Oct.2014-Avr2018] : Absolute Localization by Mono-camera for a Vehicle in Urban Area using Street View, PhD thesis funded by VeDeCom
- ...
PUBLICATIONS
2023 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
ImPosing: Implicit Pose Encoding for Efficient Visual LocalizationIEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023 ), Jan 2023, Waikoloa Village, United States Communication dans un congrès hal-03886744v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Autoencoder-Based Background Reconstruction and Foreground Segmentation With Background Noise EstimationWinter Conference on Applications of Computer Vision (WACV), 2023, Jan 2023, Waikoloa, United States Communication dans un congrès hal-03931728v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Unsupervised Multi-Object Segmentation Using Attention and Soft-ArgmaxWinter Conference on Applications of Computer Vision (WACV), 2023, Jan 2023, Waikoloa, United States Communication dans un congrès hal-03931734v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Real-time traffic impedance and priority based cooperative path planning mechanism for SOC-ITS: Efficiency and equilibriumSimulation Modelling Practice and Theory, 2023, 122, pp.102683. ⟨10.1016/j.simpat.2022.102683⟩ Article dans une revue hal-03884116v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
2022 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Pre-trained Image Encoder for Data-Efficient Reinforcement Learning and Sim-to-Real transfer on Robotic-Manipulation tasksWorkshop on Pre-training Roboto Learning of the 2022 conference on Robot Learning (CoRL'2022), Dec 2022, Auckland (virtual), New Zealand Communication dans un congrès hal-03927719v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Online Magnetometer Calibration in Indoor Environments for Magnetic field-based SLAM2022 IEEE 12th International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sep 2022, Beijing, China. pp.1-8, ⟨10.1109/IPIN54987.2022.9917514⟩ Communication dans un congrès hal-03917848v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Asymmetrical Bi-RNN for Pedestrian Trajectory EncodingReconnaissance des Formes, Image, Apprentissage et Perception (RFIAP), Jul 2022, Vannes, France Communication dans un congrès hal-03682456v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Assessing Cross-dataset Generalization of Pedestrian Crossing Predictors33rd IEEE Intelligent Vehicles Symposium, Jun 2022, Aachen, Germany Communication dans un congrès hal-03682452v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Uncertainty estimation for Cross-dataset performance in Trajectory predictionICRA 2022 Fresh Perspectives on the Future of Autonomous Driving Workshop, May 2022, Philadeplhie, United States Communication dans un congrès hal-03918989v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
GOHOME: Graph-Oriented Heatmap Output for future Motion EstimationIEEE International Conference on Robotics and Automation (ICRA), May 2022, Philadelphie, United States Communication dans un congrès hal-03683555v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
THOMAS: Trajectory Heatmap Output with learned Multi-Agent SamplingInternational Conference on Learning Representations, Apr 2022, Virtuel, France Communication dans un congrès hal-03683506v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Tightly-Coupled Magneto-Visual-Inertial Fusion for Long Term Localization in Indoor EnvironmentIEEE Robotics and Automation Letters, 2022, 7 (2), pp.952-959. ⟨10.1109/LRA.2021.3136241⟩ Article dans une revue hal-03940611v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
CoordiNet: uncertainty-aware pose regressor for reliable vehicle localizationWinter Conference on Applications of Computer Vision 2022, Jan 2022, Waikoloa Village, United States Communication dans un congrès hal-03521883v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Fast and Accurate Background Reconstruction Using Background BootstrappingJournal of Imaging, 2022, 8 (1), pp.9. ⟨10.3390/jimaging8010009⟩ Article dans une revue hal-03895240v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
2021 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing predictionIEEE International Conference on Automatic Face and Gesture Recognition, Dec 2021, Jodhpur (virtual event), India Communication dans un congrès hal-03441855v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
LEARNING FROM DEMONSTRATIONS WITH SACR2: SOFT ACTOR-CRITIC WITH REWARD RELABELING'Deep Reinforcement Learning' workshop of the 35th Conference on Neural Information Processing Systems (NeurIPS'2021), Dec 2021, Virtual, United States Communication dans un congrès hal-03519790v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
GRIAD: General Reinforced Imitation for Autonomous Driving. Method ranked #1 and #4 in CARLA Challenge 2021NeurIPS 2021 Workshop on Machine Learning for Autonomous Driving, Dec 2021, Virtual, Canada Communication dans un congrès hal-03683499v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous DrivingNeurIPS 2021, Machine Learning for Autonomous Driving Workshop, Dec 2021, Virtual, United States Communication dans un congrès hal-03442328v3 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
LENS: Localization enhanced by NeRF synthesis5th Annual Conference on Robot Learning, Nov 2021, London, United Kingdom Communication dans un congrès hal-03520671v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Asymmetrical Bi-RNNs, 3rd place solution at the ICCV Trajnet++ ChallengeICCV 2021 Multi-Agent Interaction and Relational Reasoning Workshop, Oct 2021, Montreal, Canada Communication dans un congrès hal-03381740v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
HOME: Heatmap Output for future Motion Estimation2021 IEEE International Intelligent Transportation Systems Conference (ITSC'2021), Sep 2021, Indianapolis, United States Communication dans un congrès hal-03356794v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Asymmetrical Bi-RNNs (U-RNNs), 2nd place solution at the Trajnet++ Challenge for pedestrian trajectory forecastingWorkshop on Long-term Human Motion Prediction, 2021 IEEE International Conference on Robotics and Automation (ICRA), May 2021, Xi'an, China Communication dans un congrès hal-03244860v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
2020 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Generative Model for Skeletal Human Movements Based on Conditional DC-GAN Applied to Pseudo-ImagesAlgorithms, 2020, 13 (12), pp.319. ⟨10.3390/a13120319⟩ Article dans une revue hal-03039879v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning NetworkAlgorithms, 2020, 13 (12), pp.331. ⟨10.3390/a13120331⟩ Article dans une revue hal-03059635v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Generalized inverses of increasing functions and Lebesgue decompositionMarkov Processes And Related Fields, 2020 Article dans une revue hal-03113748v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit AffordancesIEEE conference on Computer Vision and Pattern Recognition (CVPR'2020), Jun 2020, Seattle, United States Communication dans un congrès hal-02513566v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Vision and Wi-Fi fusion in probabilistic appearance-based localizationThe International Journal of Robotics Research, 2020, pp.027836492091048. ⟨10.1177/0278364920910485⟩ Article dans une revue hal-02888255v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Lane-level localization system using surround-view cameras adaptive to different driving conditionsInternational Journal of Advanced Robotic Systems, 2020, 17 (2), pp.172988142092163. ⟨10.1177/1729881420921630⟩ Article dans une revue hal-03113787v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
2019 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Is Deep Reinforcement Learning Really Superhuman on Atari?Deep Reinforcement Learning Workshop of 39th Conference on Neural Information Processing Systems (Neurips'2019), Dec 2019, Vancouver, Canada Communication dans un congrès hal-02368263v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
DISPOSITIF DE DETECTION DE GESTUELLES DE DOIGT(S) ET DE MAIN D’UN CONDUCTEUR DE VEHICULE POUR CONTROLER DES FONCTIONSFrance, Patent n° : FR3080928. 2019 Brevet hal-02379202v1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Current research projects
- Academic supervision of industrial PhD thesis (CIFRE) of Valeo lighting (Simon DE MOREAU, End-to-end holistic vehicle lighting)
- Academic supervision of an industrial PhD thesis (CIFRE) of Footovision [Jérémy OCHIN on "Algorithms of action recognition in sports with application to soccer"]
- PhD thesis direct contract with FORVIA [Angelika ANDO, Embedded 3D visualization using Neural Rendering]
- Franco-American public project (ANR-NSF) Omni-BatVision
- Academic supervision of a CEA-LIST PhD [Iad ABDUL RAUF on "Magnetic field and vision based loop closure "]
- Franco-German public project (ANR-DLR) HAIBrid
- Academic supervision of industrial PhD thesis (CIFRE) of Valeo Driving Assistance [Raphaël CHEKROUN, "Deep Reinforcement Learning for autonomous Driving"]