Chopin, Jeremy and Fasquel, Jean-Baptiste and Mouchère, Harold and Dahyot, Rozenn and Bloch, Isabelle (2023) Model-based inexact graph matching on top of DNNs for semantic scene understanding. Computer Vision and Image Understanding, 235. p. 103744. ISSN 1077-3142

Download (17MB)

Abstract

Deep learning based pipelines for semantic segmentation often ignore structural information available on annotated images used for training. We propose a novel post-processing module enforcing structural knowledge about the objects of interest to improve segmentation results provided by deep neural networks (DNNs). This module corresponds to a “many-to-one-or-none” inexact graph matching approach, and is formulated as a quadratic assignment problem. Our approach is compared to a DNN-based segmentation on two public datasets, one for face segmentation from 2D RGB images (FASSEG), and the other for brain segmentation from 3D MRIs (IBSR). Evaluations are performed using two types of structural information: distances and directional relations that are user defined, this choice being a hyper-parameter of our proposed generic framework. On FASSEG data, results show that our module improves accuracy of the DNN by about 6.3% i.e. the Hausdorff distance (HD) decreases from 22.11 to 20.71 on average. With IBSR data, the improvement is of 51% better accuracy with HD decreasing from 11.01 to 5.4. Finally, our approach is shown to be resilient to small training datasets that often limit the performance of deep learning methods: the improvement increases as the size of the training dataset decreases.

Item Type:	Article
Keywords:	Graph matching; Deep learning; Image segmentation; Volume segmentation; Quadratic assignment problem;
Academic Unit:	Faculty of Science and Engineering > Computer Science Faculty of Science and Engineering > Research Institutes > Hamilton Institute
Item ID:	18940
Identification Number:	https://doi.org/10.1016/j.cviu.2023.103744
Depositing User:	Rozenn Dahyot
Date Deposited:	26 Sep 2024 14:36
Journal or Publication Title:	Computer Vision and Image Understanding
Publisher:	Elsevier
Refereed:	Yes
URI:
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads