Brian Jackson

Associate Professor of Physics at Boise State University

  • About
  • Research
    • CV
    • Joining the Boise State Planetary Science Research Group
    • Research Experiences for Undergrads
    • Ultra-short-period planet database
    • The Short Period Planets Group — S(u)PerP(i)G
    • Google Scholar Page
    • Code
  • Teaching
  • Press
  • Extracurricular Activities
    • Field Trips
      • Bruneau Sand Dunes – 2022 Jun 29
  • Public Outreach
    • Boise State’s Astronomical Observatory
    • Central Idaho Dark Sky Reserve STEM Network

Shi & Malik (2000) — “Normalized Cuts and Image Segmentation”

Posted by admin on July 16, 2014
Posted in: Data Science.

As part of my new push into data science, I read Shi & Malik’s paper on the perceptual grouping problem in which they develop a new algorithm for dividing an image up into coherent regions.

A portion of Figure 8 from Shi & Malik (2000).

A portion of Figure 8 from Shi & Malik (2000).

What does it mean to divide an image into coherent regions? Say, for example, you had a satellite image of a large storm system, such as the figure at left from Shi and Malik’s paper. Panel (a) shows the original image, while (b) and (c) show two portions of the image grouped together by Shi and Malik’s algorithm. The result makes intuitive sense (at least, to me): (b) is a large part of the storm, while (c) is the ground underneath.

Basically, Shi and Malik’s algorithm treats the entire image as a weighted graph, with portions of the image treated as nodes. Weights on the edges connecting the nodes are larger for portions that are closer to each other and for portions that are similar (where similarity may depend on the pixel brightnesses, textures, colors, etc.).

The algorithm decides which portions belong together as one segment using the weights for all the nodes connected in that segment. Shi and Malik developed a clever way to turn this process into an eigenvalue/vector problem, thereby dramatically facilitating the calculation. Their technique amounts to treating the pixels as individual masses connected by springs, with spring constants given by the edge weights, and then finding the normal modes of oscillation for the system: pixels that are strongly coupled are grouped together as one segment.

Conveniently, the algorithm is implemented in the scikit-learn python module. Using their example code, I was able to reproduce the segmentation of the Lena image easily (shown below), so I thought to try it on some VIMS observations of Titan. (Here’s the original Titan image. I took a small portion of it.)

Unfortunately, the result was not promising: the algorithm did NOT break up the image into the segments I expected. Instead, the segments seem pretty random. It also took about 40 minutes to work, even though the Titan image is smaller than the Lena image (100 x 100 pixels vs. 128 x 128).

(Left) Result from the spectral clustering example code on the scikit-learn page. (Right) My own attempt at segmenting a VIMS image of Titan.

(Left) Result from the spectral clustering example code on the scikit-learn page. (Right) My own attempt at segmenting a VIMS image of Titan.

Next things to try: it seems the algorithm needs me to tell it how many regions to use — I used the number given in the original example, 11. Maybe I should try a smaller number. There are also a few options as to how the graph weight are calculated in the original Shi and Malik algorithm (not sure if the scikit-learn module has that capability).

Posts navigation

← Roger Fu — “Exploring Solar System Formation with Meteorite Paleomagnetism”
Jura+ (2014) — “A Pilot Search for Evidence of Extrasolar Earth-analog Plate Tectonics” →
  • Twitter: decaelus

    Brian Jackson
    • Come join us at @BoiseState for an exoplanets and outreach postdoc - https://t.co/LgQkDfkVJd. Apps due by end of Ap… https://t.co/37IiiVvl6D 08:41:55 2023 Mar 8
    • Records show powerful, wealthy funders outside Idaho back school choice campaign https://t.co/tzrZgyYk4U via @idahocapitalsun 11:03:33 2023 Feb 3
    • Learn how to save the dark (https://t.co/CqYzD5o2Xh at @BroncoPhysics First Friday Astronomy event this Fri (Feb 3… https://t.co/8B3Set3tW0 11:49:28 2023 Jan 30
    @decaelus
  • Recent Posts

    • LPSC 2023
    • Taking the Universe’s Measure
    • Research Group Meeting – 2023 Mar 2
    • Resources for the Jupiter-Venus Conjunction
    • PHYS305 – 2023 Mar 1
  • Archives

    • March 2023
    • February 2023
    • January 2023
    • December 2022
    • November 2022
    • October 2022
    • September 2022
    • August 2022
    • July 2022
    • June 2022
    • May 2022
    • April 2022
    • March 2022
    • February 2022
    • January 2022
    • December 2021
    • November 2021
    • October 2021
    • September 2021
    • August 2021
    • July 2021
    • June 2021
    • May 2021
    • April 2021
    • March 2021
    • February 2021
    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
    • October 2014
    • September 2014
    • August 2014
    • July 2014
    • June 2014
    • May 2014
    • April 2014
    • March 2014
    • February 2014
    • January 2014
    • December 2013
    • November 2013
    • October 2013
    • September 2013
    • August 2013
    • July 2013
Proudly powered by WordPress Theme: Parament by Automattic.