Point Cloud and 3D Modeling (NPM3D)

Research TOPIC

People

Publications

Projects

We created NPM3D Benchmark Suite on 3D Point Clouds: https://npm3d.fr/

Research TOPIC NPM3D

NPM3D ("Nuage de Points et Modélisation 3D") is a research topic in Centre of robotics in 3D Deep Learning on Point Clouds (from LiDAR and RGB-D sensors) for autonomous vehicles, city mapping, archeology...

Main subjects of the NMP3D research topic are:

  • LiDAR SLAM
  • Point cloud registration
  • Scene understanging in LiDAR data: semantic segmentation, object detection
  • Point cloud rendering

We have multiple 3D sensors (Kinect 2, scanner FARO Focus X130, Velodyne VLP16, HDL32) and two mobile mapping platforms:

  • L3D2: vehicle with GPS/IMU localization system, Velodyne HDL32 and Ladybug5
  • Drone with Velodyne VLP16 and Camera FLIR Blackfly

People

Permanent staff:

  • Jean-Emmanuel Deschaud: Associate Professor, Mines Paris - PSL, jean-emmanuel.deschaud@mines-paristech.fr

External Collaborator:

  • François Goulette: Full Professor, Deputy Director U2IS, ENSTA Paris, francois.goulette@ensta-paris.fr

PhD students:

  • Samir Abou-Haidar (2022-): Deep neural networks on 3D point cloud adapted to embedded architectures. In collaboration with Cyril Joly and CEA LIAE lab (Alexandre Chariot and Mehdi Darouich)
  • Hugo Blanc (2022-): Physico-realistic differentiable rendering of point clouds for interactive visualization of real environments, and applications to cultural heritage. In collaboration with Alexis Paljic.
  • Fabio Elnecave Xavier (2022-): Odometry and 3D Reconstruction of the Close Environment of a Leg Exoskeleton. In collaboration with François Goulette and Wandercraft (Guillaume Burger and Marine Petriaux)
  • Louis Soum-Fontez (2021-): Unsupervised real-time object detection from LiDAR data. In collaboration with François Goulette.
  • Jules Sanchez (2020-): Real-time semantic segmentation and object detection of lidar data for the autonomous vehicle. In collaboration with François Goulette.
  • Pierre Dellenbach (2020-): Self-supervised Deep SLAM on camera and LiDAR data. In collaboration with François Goulette and Kitware (Raphaël Cazorla).

Former PhD students:

  • Jean-Pierre Richa (2019-2022): Urban Scene Modeling From 3D Point Clouds and Massive LiDAR Simulation for Autonomous Vehicles (working now at Outsight as R&D engineer)
  • Sofiane Horache (2019-2022): Pattern comparison on 3D point clouds and application on Celtic coins and objects (working now at TheraPanacea as R&D engineer)
  • David Duque (2018-2021): 3D urban scene understanding by analysis of LiDAR, color and hyperspectral data (working now in the start-up Thecrossproduct as R&D engineer)
  • Hugues Thomas (2016-2019): Learning new representations for 3D point cloud semantization (now Post-Doc at University of Toronto)
  • Xavier Roynard (2015-2019): On-the-fly semantization of 3D point clouds acquired by embedded systems (working now at SAFRAN as R&D engineer)
  • Hassan Bouchiba (2014-2018): Contributions in point-based processing for rendering and simulation in fluid mechanics (now co-founder of the start-up Exwayz)
  • Houssem Nouira (2013-2016): Point cloud refinement with self-calibration of a mobile multi-beam lidar (working now at MENSI/TRIMBLE as R&D engineer)

PUBLICATIONS

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/cticp-150x150.jpg

CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closure
P. Dellenbach, J.-E. Deschaud, B. Jacquet, F. Goulette
ICRA, 2022
Paper    Github

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/pariscarla3d-150x150.jpg

Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping
J.-E. Deschaud, D. Duque, J. P. Richa, S. Velasco-Forero, B. Marcotegui, F. Goulette
MDPI Remote Sensing, 2021
Paper    Dataset

https://www.caor.minesparis.psl.eu/wp-content/uploads/2022/11/riedones3d-150x150.jpg

Riedones3D: a Celtic Coin Dataset for Registration and Fine-grained Clustering
S. Horache, J.-E. Deschaud, F. Goulette, K. Gruel, T. Lejars, O. Masson
Eurographics Workshop on Graphics and Cultural Heritage, 2021
Paper    Dataset

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/mssvconv-150x150.png

3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning
S. Horache, J.-E. Deschaud, F. Goulette
3DV, 2021
Paper    GitHub

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/pylidar-150x150.png

What’s in My LiDAR Odometry Toolbox?
P. Dellenbach, J.-E. Deschaud, B. Jacquet, F. Goulette
IROS, 2021
Paper    GitLab

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/power_jaccard-150x150.png

On power Jaccard losses for semantic segmentation
D. Duque-Arias, S. Velasco-Forero, J.-E. Deschaud, F. Goulette, A. Serna, E. Decencière, B. Marcotegui
VISAPP, 2021
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/coins-1-150x150.png

Automatic clustering of celtic coins based on 3D Point Cloud Pattern Analysis
S. Horache, F. Goulette, J.-E. Deschaud, T. Lejars, K. Gruel
ISPRS Congress, 2020
Paper    Video

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/network-150x150.png

SHREC 2020 Track: 3D Point Cloud Semantic Segmentation for Street Scenes
T. Ku, R. C. Veltkamp, B. Boom, D. Duque-Arias, S. Velasco-Forero, J.-E. Deschaud, F. Goulette, B. Marcotegui, S. Ortega, A. Trujillo, J. Pablo Suárez, J. Miguel Santana, C. Ramírez, K. Akadas, S. Gangisetty
Computer & Graphics, 2020
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/pc_bunny_flow_3d-150x150.jpg

Computational Fluid Dynamics on 3D Point Set Surfaces
H. Bouchiba, S. Santoso, J.-E Deschaud, L. Rocha-Da-Silva, F. Goulette, T. Coupez
Journal of Computational Physics, 2020
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/kpconv-150x150.png

KPConv: Flexible and Deformable Convolution for Point Clouds
H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, L. J. Guibas
ICCV, 2019
Paper    GitHub TensorFlow    GitHub PyTorch

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/color_lines-150x150.png

A Graph-based Color Lines model for image analysis
D. Duque-Arias, S. Velasco-Forero, F. Goulette, J.-E. Deschaud, B. Marcotegui
ICIAP, 2019
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/ijrr-150x150.png

Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification
X. Roynard, J.-E. Deschaud, F. Goulette
IJRR, 2018
Paper    Dataset

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/IROS2018-150x150.png

Classification of Point Cloud for Road Scene Understanding with Multiscale Voxel Deep Network
X. Roynard, J.-E. Deschaud, F. Goulette
Workshop IROS, 2018
Paper    GitHub

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/3DV2018-150x150.png

Semantic Classification of 3D Point Clouds with Multiscale Spherical Neighborhoods
H. Thomas, J.-E. Deschaud, B. Marcotegui, F. Goulette, Y. Le Gall
3DV, 2018
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/Eurographics2018-150x150.png

Raw point cloud deferred shading through screen space pyramidal operators
H. Bouchiba, J.-E. Deschaud, F. Goulette
Short Paper EUROGRAPHICS, 2018
Paper    Video

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/ICRA2018-150x150.png

IMLS-SLAM: scan-to-model matching based on 3D data
J.-E. Deschaud
ICRA, 2018
Paper

https://www.caor.minesparis.psl.eu/wp-content/uploads/2021/03/Record2017-150x150.png

Point Cloud Refinement with Self-calibration of a mobile multi-beam Lidar Sensor
H. Nouira, J.-E. Deschaud, F. Goulette
The Photogrammetric Record, 2017
Paper

Current research projects

  • CIFRE with Wandercraft (2022-2025): Wandercraft
  • 5GMed (H2020, 2021-2024): Obstacle Detection on LiDAR data through 5G for automotive and railway cases Project Website
  • CIFRE with Kitware (2021-2023): Kitware
  • Research project with ANSYS (2020-2022): ANSYS