Rémi Giraud

Texture-Aware Superpixel Clustering

Back to research page

Highlights



  • New superpixel approaches able to segment texture (TASP)
  • New pixel to superpixel texture homogeneity term using patch-based nearest neighbor searches
  • Automatic adaptation to the image type (synthetic texture, natural color) without parameter tuning
  • Limitation of complexity with the NNSC method

       


Texture-Aware SuperPixel (TASP) method using K-means clustering


  • The proposed TASP approach [1] is based on the K-means-based clustering framework of the SLIC framework [4]
  • Initial grid partition into S×S superpixels, and clustering of pixels within 2S×2S boxes for each superpixel Sk
  • Color and spatial distances are considered to gather spatially close and homogeneous pixels so
    the clustering distance between a pixel p and a superpixel Sk is:     D(p,Sk) = dcolor(p,Sk) + dspatial(p,Sk)

       



Automatic adaptation of superpixel regularity
  • TASP adapts the regularity constraint according to the superpixel content to get relevant results on homogeneous and textured regions
  • The regularity parameter mk for a superpixel Sk is set such that:
        m -> mk=exp(σ(Fp in Sk)/beta)         (1)




Pixel to Superpixel Texture Homogeneity Term
  • Patch-based nearest neighbor (NN) matching within the superpixel
  • Fast selection of N similar patches of size nxn with PatchMatch [12]
  • New pixel to superpixel texture homogeneity term:
        dtexture(p,Sk)=1/N Σ
    pk∈Sk     1/n ||FP(p) - FP(pk)||2             (2)





  • Complementary term to ensure texture unicity by encouraging the selection of patches P(pk) close to the superpixel barycenter:
        dunicity(p,Sk)= 2/N Σ
    pk∈Sk     (1-exp(-||Xpk-XSk||2/(SxS)))             (3)

Final TASP clustering distance
  • TASP clustering distance is defined as:   D(p,Sk) =

    dcolor(p,Sk) + dspatial(p,Sk)mk + dtexture(p,Sk) + dunicity(p,Sk)mk

Important Complexity
  • High complexity due to the computation of nearest neighbor matching in each overlapping superpixel window at each iteration
  • CTASP=Ο((hxw)x4xIterKmeansxIterNN)
  • Time = 45s for images of 321x481 pixels



  • The proposed NNSC method [2] reduces the complexity of TASP using the same clustering distance
  • Direct pixel label update using local NN search:

       



Reduced Complexity
  • Reduced complexity due to the direct pixel-wise processing and use of nearest neighbor matching to update the label map
  • CNNSC=Ο((hxw)xIterNN)     Time = 1.5s for images of 321x481 pixels



Qualitative results
  • TASP and NNSC provide visually satisfying results with regular superpixels on texture and color images using the same parameters




Quantitative results

       




  • C implementation (with optional Matlab/C-Mex wrapper) of the TASP method

    Download here



Main (to cite)

  1. R. Giraud, V.-T. Ta, N. Papadakis and Yannick Berthoumieu: Texture-Aware Superpixel Segmentation
    IEEE International Conference on Image Processing (ICIP), 2019  
Others

  1. R. Giraud, and Yannick Berthoumieu: Texture Superpixel Clustering from Patch-based Nearest Neighbor Matching
    European Signal Processing Conference (EUSIPCO), 2019  
  2. R. Giraud, V.-T. Ta, N. Papadakis and Yannick Berthoumieu: Superpixels adaptés localement aux textures
    Groupe d'Études du Traitement du Signal et des Images (GRETSI), 2019  


  1. M. Y. Liu, O. Tuzel, S. Ramalingam and R. Chellappa: Entropy rate superpixel segmentation
    IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2011
  2. M. Van den Bergh, X. Boix, G. Roig, B. de Capitani and L. Van Gool: SEEDS: Superpixels extracted via energy-driven sampling
    European Conference on Computer Vision (ECCV), 2012
  3. R. Achanta, A. Shaji, K. Smith, A. Lucchi, et al.: SLIC superpixels compared to state-of-the-art superpixel methods
    IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2012
  4. P. Buyssens, I. Gardin, S. Ruan and A. Elmoataz: Eikonal-based region growing for efficient clustering
    Image and Vision Computing, 2014
  5. J. Yao, M. Boben, S. Fidler and R. Urtasun: Real-time coarse-to-fine topologically preserving segmentation
    IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2015
  6. J. Chen, Z. Li and B. Huang: Linear spectral clustering superpixel
    IEEE Trans. on Image Processing, 2017
  7. R. Achanta and S. Süsstrunk: Superpixels and polygons using simple non-iterative clustering
    IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2017
  8. R. Giraud, V.-T. Ta and N. Papadakis: Robust Superpixels using Color and Contour Features along Linear Path
    Computer Vision and Image Understanding, 2018


Back to top