Rémi Giraud

SPM: SuperPatchMatch algorithm for superpixel matching

Back to research page

Highlights



  • New SuperPatch structure containing superpixel neighborhood information
  • New SuperPatchMatch (SPM) algorithm for fast and accurate superpixel matching using Superpatches
  • Application to exemplar-based segmentation and labeling on natural and medical images
  • SPM outperforms state-of-the-art methods based on deep learning

Method


SuperPatch definition

  • New structure of superpixel neighborhood considering geometric information
  • Possibility to use neighboring superpixels as for regular patches





Comparison between two superpatches Ai and Bj

  • Registration of the spatial barycenter of superpatch Bj on the one of Ai
  • Superpixels of Ai are compared to the spatially closest ones in Bj



SuperPatchMatch algorithm

  • Adaptation of the PatchMatch [3] algorithm to these superpatches
  • Fast and robust superpixel matching algorithm using neighboring information with superpatches


Robustness of the superpatch
  • The same image is segmented by two methods and the superpixels are matched using superpixels or the proposed superpatches
  • Superpatches provide much more accurate matching since they consider the neighborhood information
  • They are also robust to deformations such as shearing (bottom)


The displacement between the matched superpixels is illustrated with the standard optical flow representation


Validation on face labeling
  • Matching with SuperPatchMatch and transfer of ground truth labels on the LFW dataset [4]




  • Comparison of results obtained with SPM using superpixels and the proposed superpatches



  • Comparison to state-of-the-art methods on labeling accuracy. SPM outperforms methods that need learning steps






Main (to cite)

  1. R. Giraud, V.-T. Ta, A. Bugeau, et al. : SuperPatchMatch: an Algorithm for Robust Correspondences using Superpixel Patches
    IEEE Trans. on Image Processing (TIP), 2017   https://hal.archives-ouvertes.fr/hal-01432116/file/Giraud_TIP_2017_SPM.pdf   BibTex  
Others

  1. R. Giraud, V.-T. Ta, A. Bugeau, et al. : SuperPatchMatch : Un algorithme de correspondances robustes de patchs de superpixels
    Reconnaissance des Formes, Image, Apprentissage et Perception (RFIAP), 2018   https://hal.archives-ouvertes.fr/hal-01702277v1   BibTex





See project page

Main (to cite)

  1. R. Giraud, V.-T. Ta and N. Papadakis: Superpixel-based Color Transfer
    IEEE International Conference on Image Processing (ICIP), 2017   https://hal.archives-ouvertes.fr/hal-01519644/file/Giraud_SCT_ICIP17.pdf   BibTex    
Others

  1. R. Giraud, V.-T. Ta and N. Papadakis: Transfert de couleurs basé superpixels
    Groupe d'Études du Traitement du Signal et des Images (GRETSI), 2017   https://hal.archives-ouvertes.fr/hal-01542596/file/Giraud_SCT_GRETSI_2017.pdf   BibTex    



  1. C. Barnes, E. Shechtman, A. Finkelstein, et al.: PatchMatch: A randomized correspondence algorithm for structural image editing
    ACM Transactions of Graphics, 2009
  2. G. Huang, et al.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments
    Tech. Rep. 07-49, Univ. of Massachusetts, 2007
  3. A. Kae, K. Sohn, H. Lee, and E. Learned-Miller: Augmenting CRFs with Boltzmann machine shape priors for image labeling
    IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013
  4. S. Liu, J. Yang, C. Huang, and M. Yang: Multi-objective convolutional learning for face labeling
    IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015


Back to top