Adversarial Domain Adaptation for Sensor-Robust Panoptic Segmentation of 3D Scans
Collaboration with Laboratory: STRUDEL team, LaSTIG laboratory (IGN/Univ. Gustave Eiffel)
Location: IGN: Saint Mandé, France (3 min outside Paris), SAMP: Station F, Paris
Advisors: Loic Landrieu, PhD and Shivani Shah, PhD
Remuneration: 1200 euros gross / month
Starting Date: May 2021, 5 months duration
Key words: Domain Adaptation, 3D Data, Panotic Segmentation, Deep Learning
Development Environment: Linux, Python, PyTorch.
This internship is proposed in collaboration with the STRUDEL laboratory at IGN. STRUDEL Team is a machine-learning research team with IGN, the French Mapping Agency. It focuses on solving large-scale computer vision and remote sensing challenges by developing state-of-the-art methods. In particular, it focuses on scalable 3D deep learning and open-source frameworks.
Bolstered by the rapid progress of 3D sensor technology, private and public actors have seen a stark increase in both the quantity and quality of available 3D data. Alongside this accessibility, recent methodological advancement in terms of deep learning applied to 3D data have considerably improved the capacity for automated analysis of 3D scans. However, training high performance deep learning methods requires large quantities of annotated data. Furthermore, this approach tends to be very sensitive to the data distribution used in the training phase.
Companies such as SAMP have access to a large amount of data to train their models. However, the scans come from many different sites that may differ in their nature (Nuclear Plants, Oil and Gas plants, Chemical Plants, Manufacturing Plants) as well as the characteristics of the sensors. This makes it hard to leverage the quantity of available data to train models across several datasets.
The objective of this internship is to implement an approach allowing to train a network to analyse 3D data of different industrial sites and acquired with a variety of sensors. To this end, an existing panoptic segmentation network will be modified in order to handle the difference in data distribution. The intern will investigate adversarial domain adaptation techniques such as DAN : by making the learned features indistinguishable across all sites, a single model can leverage the entirety of SAMP database.
The tasks of this internship are as follows:
- Understand and familiarize with the models developed within SAMP for panoptic semantic segmentation, as well as the characteristics of the available datasets.
- Implement a domain adaptation training routine across different dataset in order to make features Site-independent.
- Train a model across all available datasets.
- Validate the approach on available public datasets.
- Validate approach on SAMP’s datasets.
Provided satisfying results, this work will lead to the writing of a conference paper with the student.
- Student in Master 2 in computer science, applied mathematics or other relevant courses
- Familiarity with machine learning and computer vision concepts
- Experienced with Python and familiar with PyTorch
- Curiosity, rigor
- (Optional) Experienced with 3D neural networks and versioning interfaces (github)
- (Optional) Good level of written English.