Rémi Giraud

DSP: Dual Superpixel Descriptors for Accurate Superpatch Matching

Back to research page

Highlights



  • New Dual SuperPatch (DSP) containing intra-region (R) and interface (I) superpixel features
  • New multi-scale approach for the matching of DSP at multiple scales
  • Application to matching and supervised labeling on standard image dataset





Method


Dual superpixel descriptor
  • Intra-region (R) descriptor extraction with an offset to the superpixel borders
  • Interface (I) descriptor extracted at triple superpixel intersections to capture structure information
  • Dual SuperPixel (DSP) containing I and R descriptor sets in a superpixel neighborhood (superpatch)
  • Generalized framework able to compare both intra-region and structure information between two superpatches Si and Sj:
        D(Si,Sj) = α.d(Ri,Rj) + (1-α).d(Ii,Ij)




Multi-scale framework
  • Automatic rescale to compare DSP at extracted from different radius sizes
  • Ability to capture patterns at different scales using rescale (green lines):




Fast DSP matching using approximate nearest neighbor algorithm
  • Dual SuperPatchMatch (DSPM) algorithm using SPM with the proposed DSP descriptors
  • DSPM is a partly random matching algorithm relying on the propagation of good matches based on the adjacent superpixels






Robustness of the DSP
  • The same image is segmented by two methods and the superpixels are matched using standard superpatch (full region information) and the proposed DSP, containing both cropped intra-region and superpixel interface information
  • DSP (α=0.5) provide much more accurate matching since they explicitly consider structure information in a dedicated interface descriptor


The displacement between the matched superpixels is illustrated with the standard optical flow representation


Quantitative results
  • Validation on exemplar-based superpixel labeling experiment similar to the one used in SPM on the LFW dataset [2]
  • Comparison of DSPM to state-of-the-art methods on labeling accuracy:

       




  • C implementation (with optional Matlab/C-Mex wrapper) of the DSPM method

    To come



Main (to cite)

  1. R. Giraud, M. Boyer and M. Clément: Multi-Scale Superpatch Matching using Dual Superpixel Descriptors
    Pattern Recognition Letters, 2020


  1. G. Huang, et al.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments
    Tech. Rep. 07-49, Univ. of Massachusetts, 2007
  2. A. Kae, K. Sohn, H. Lee, and E. Learned-Miller: Augmenting CRFs with Boltzmann machine shape priors for image labeling
    IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013
  3. S. Liu, J. Yang, C. Huang, and M. Yang: Multi-objective convolutional learning for face labeling
    IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015
  4. R. Giraud, V.-T. Ta, A. Bugeau, et al.: SuperPatchMatch: an Algorithm for Robust Correspondences using Superpixel Patches
    IEEE Transactions on Image Processing (TIP), 2017


Back to top