The goal of this homework is to use thresholding for image segmentation and to get familiar with different thresholding algorithms.

Thresholding is the most common method to separate image components (features) from the background. Often, the threshold value is selected manually. In special cases, algorithms exist to automatically determine an optimal threshold. Your task is to apply thresholding methods to separate objects from their background. This goal can be achieved by generating binary masks, i.e. images where pixel values of 1 (or 255) correspond to the selected feature, and 0 corresponds to background.

Click on the above image to download a photo of differently colored gummibears. Focus on the red and green bears and ignore the yellow ones for now. Note that this is a RGB image, and the different color channels carry different information. A good start would therefore be the separation of the channels.

You are free to use any image analysis software of your choice. Most advanced image analysis programs (ImageJ, IMAL) can perform all functions that you need in this homework. For Crystal Image, some instructions follow.

In Crystal Image, you can separate color channels by editing the data type from RGB into a stack. The first slice is red, followed be green and blue. By convention, we use black as background, so you will have to invert the image and work with the negative. This function can be found under Process -> Invert. You may save individual slices from your stack in the regular save function by using the from/to sliders.

Note that the illumination is somewhat inhomogeneous; background correction is advisable. Appropriate methods are the pre-packaged background removal (see hints below) and unsharp masking. Also note that some blurring may be useful when preparing segmentation masks.

The images above show the red channel after background correction with a FFT highpass (left) and fitted biparabolic plane subtraction (right).

TASK 1:
Generate a histogram for the red and green channels and identify the peaks. Export the histograms to create a graph. Label the peaks (background; red, green gummibears etc). Indicate your (manual) threshold selection for both the red and green gummibears.

Next, choose ONE channel and use binary thresholding to create a mask. Measure the size of the segmented gummibears (for example by cluster labeling or using the area measurement in Measure -> Statistics)). Vary your threshold selection and determine its influence on the size. Show this relationship by creating a graph: size versus threshold.

TASK 2:
Use the iterative threshold function and Otsu's threshold to perform automated thresholding. Discuss: (1) How well does the automated threshold values match your manual selection? (2) How well do they perform with the red and green channels, and - if there is a problem - why?

TASK 3:
Perform the actual segmentation by creating two masks, one mask for the green gummibears and one for the red ones. The masks should completely cover the respective gummibears, and should contain no holes, fuzz, or noise pixels. Important: Manual interaction is not allowed in Task 3! In other words, any operation must be applied on the entire image. Selection of a region of interest (ROI), editing of individual pixels, or photoshop-like manipulation are not acceptable. On the other hand, you MAY use manual thresholding (that is an operation that acts on the entire image) for the primary segmentation in Task 3. To obtain the full score, you need to document ALL image processing steps.

BONUS TASK:
Segment the yellow gummibears. Pure thresholding will not work, because the intensity values are too close to the green gummibears, so you need to find an alternative way. To obtain the full score for this task, you need to document the individual steps that lead to your solution!

Some hints and notes:

• A key operation is edit->Data Type to split the image into R, G, and B channels that are stored as an image stack
• The background removal operation in Crystal Image (and many other functions) assume a dark background. You need to make sure that this condition is met (e.g., Process -> Invert).
• You have a choice of various background removal functions. The homomorphic FFT highpass, background removal by fitting a biparabolic plane, or background removal by unsharp masking (i.e., moving average subtraction). Experiment with them to see which one gives best results.
• Jagged parts of the mask can be corrected by using morphological closing; small speckles can be corrected by morphological opening.
• The gummibears have a distinct size. "Dirty" spots of the mask that defy morphological operators can be eliminated by a feature size filter: Cluster labeling (Process menu). Use a threshold of 1 and specify the minimum size of allowable clusters. After the process, binarize by thresholding. Another interesting function is Fill holes to stuff interior holes.
• Cluster labeling is also the optimum method to determine feature sizes.
• You may also use hysteresis thresholding; however, region growing is not allowed as this requires manual interaction (seed points). If you use hysteresis thresholding, explain how you obtained the threshold values from an automated method (e.g., Otsu's).
• If you wish, you may try an alternative approach by transforming the original RGB image into a different color space (Edit -> Data values -> RGB to HSV)
• The use of manual interaction (editing values, manual thresholding, selecting seed points, selecting regions of interest) is NOT allowed. All operations have to be performed on the entire image.
• Masks are images containing only ones and zeroes. Many programs (image processing as well as word processing etc) expect the full range form 0 to 255, where 0 is black and 255 is white. Thus a 0/1 image will be black mixed with almost-black. This won't print. You may convert this by changing the data type to IEEE float and then back to 8-bit uchar.
• THIS IS A LONG HOMEWORK - START WORKING ON IT EARLY

In the end, you should obtain masks such as displayed below (left), and maybe even a composite image (right).

Grading

This homework will give you a maximum of 35 points towards your total score as follows:

• 7 points for the image preparation: RGB-to-grayscale conversion, background flattening, filtering as needed.
• 10 points for Task 1, histograms and manual threshold selection, discussion of the influence of the threshold on size
• 6 points for Task 2, automated thresholds and performance discussion
• 12 points for Task 3, segmentation and creation of masks
• 10 bonus points for Task 4, the separation of the yellow gummibears and the creation of a composite image similar to the one shown above.

Due date

The homework is due on 12/01/11 in class. Since this is the day of class, no extension is possible.