Course Home
Administration
Syllabus
References
Lecture Notes
Readings
Assignments
Projects
Resources


Comments? Idea?























































































































































































































































































































































Grape Detection in Vineyards

Final project by

Rotem Rachmani & Erez Gavish

rotemra@bgu.ac.il    erezg@bgu.ac.il


Introduction

The vineyard environment is one rich with variety of elements making the task of grape detection, as most segmentation tasks, a very difficult one. Besides familiar difficulties with detection of objects in images (fuzziness, low-light, etc.). In most typical vineyard images, grapes, the objects to be detected, are least dominant. Other surrounding elements such as the soil or the grass-like vegetation, the shoots secured to trellis wires and the vine alternately covering them are much more dominant then grapes, making the grapes difficult to detect. In addition, the size of grapes may differ between images, which makes the task even harder.

Approach and Method

An important feature of most grape clusters in the images, is that most of the time they form groups of maximum regions that usually seem to be uniformly distant. Providing this, we expect to find that the regional maxima of the grapes have many more 'neighbors' then other regional maxima. Our attempt at overcoming the described difficulties mainly relies on exploiting regional maxima characteristic of grape clusters, utilizing their shape properties and their distance in relation to each other.

First goal:finding maximum regions of the intensity image

Regional maxima are connected components of pixels in the intensity image, with a constant intensity value higher than that of their boundary pixels. To find maximum regions, we first suppress all maxima whose height is less than H and then use the extended-maxima transform, which is the regional maxima of the H-maxima transform. The following image (on the right) shows the regional maxima (formed on a grape cluster cut from the intensity image) as red spots. From here on we will refer to the mapped regional maxima as "red spots".

regional Maxima

The image below demonstrates the problem that arises after applying the transformation The image below demonstrates the problem which arises after applying the transformation to the whole intensity image: there are many 'red spots' representing regional maxima, formed on objects other than grapes. Is there any way of telling the difference between 'red spots' on top of grapes and other 'red spots'?

the problem with regional Maxima

Second goal:removing irrelevant red spots

To highlight the regional maxima of the grapes, we use their shape properties. At first, we notice that many of the 'red spots' are significantly 'too large' or 'too small' to be representing regional maxima typical of grapes. We remove 'red spots' whom size exceed a pre-determined threshold value (T in range 5..300). Another way of removing irrelevant regional maxima involves using the eccentricity of the best fit ellipse of representative 'red spots'. We notice that 'red spots' on top of grapes are usually rounder, which implies that the eccentriciy of their best fit ellipse should have a low value. For each 'red spot' we compute the eccentricity of its best fit ellipse, and remove those whose eccentricity value are 'too high'. That is, those who are less round. The removal of 'red-spots' by thresholding according to area size (left image) and eccentricity value (right image) is demonstrated below.

the problem with regional maxima transform

Additional techniques to further remove irrelevant red spots

In order to remove further irrelevant 'red spots', we categorize them as one of the following: Irrelevant 'red spots' representing regional maxima formed on: (a) foliage areas. (b) 'grass-like' areas.(c) significantly bright areas.To remove 'red-spots' falling into one of these categories, we use different techniques to remove each area type:

Removing foliage areas

To remove most of the (a) type areas, we exploit the fact that the leaves in the image are usually characterized by smoothly varying intensities while grape clusters present a much greater variance of intensity. Therefore, it is expected that when applying edge detection to the intensity image, edge concentration is higher on grape clusters than it is on foliage areas. We define the continuous area covering the grape clusters in the intensity image by applying morphological operations (dilation, closing and opening) on the edges information obtained by use of Canny's edge detector. We then extract the complement of that area in order to get a binary image representing the areas which do not include grape clusters, but mostly leaves. The image below on the left shows the result of applying edge detection to the intensity image, superimposed on the intensity image. The image below on the right shows the complement of the binary image resulted from applying morphological operations on the edges image, superimposed on the original image.

remove foliage areas

Removing 'grass-like' areas

In order to remove type (b) areas, we use a different technique, involving the use of the top-hat transformation. An important property of the top-hat transformation is that it highlights elements of the intensity image which are brighter than their surroundings. Since grass-like areas in the image are mostly characterized by an abundance of such elements, use of the top-hat transformation extracts them. The left image below shows the top-hat transformation of an intensity image. The image on the right shows the conversion of the top-hat transformation to a binary image (using thresholding by intensity), superimposed on the original image.

remove grass-like areas

Removing type (c) areas is easily done by conversion of the intensity image to a binary image, using thresholding by intensity. The image below on the right shows the bright areas to be removed, superimposed on the original image (shown on the left).

remove grass-like areas

Third goal: using the remaining 'red spots' in order to detect grape clusters

We decide that two 'red spots' are neighbors if the distance between their centroid pixels is smaller than a predetermined value. In the image below and to the left, the neighbors of each of the remaining 'red spots' are visualized by drawing lines between centroids of neighboring 'red spots'. Next, we compute the relative number of neighbors (RNN) for each 'red spot': the ratio between its neighbors and the maximal number of neighbors per 'red spot'. We use the RNN of each 'red spot' as an intensity value for its centroid pixel in a grayscale image. The right image below demonstrates the results: the dots are dilated centroids of the remaining 'red spots'. The brighter a dot is, the more neighbors the 'red spot' it represents has.

remove grass-like areas

The distance between 'Red spots' representing regional maxima of grapes is almost identical for each pair and these 'red spots' are organized in relatively crowded clusters. At this phase, having removed 'red spots' by their shape properties and filtered out irrelevant areas from the image (using the techniques described earlier), we expect that 'red spots' representing regional maxima of grapes would have more neighboring 'red spots' then non-grape 'red spots'. This means that the intensity value of their cendtroid pixel would be higher. We then use a threshold value (T=0.5) to extract only the dots referring to centroids of 'red spots' with a high RNN value. Finally, we apply morphological operations to fill the area between these dots and extract the grape clusters to whom the remaining 'red spots' correlate.

Results

Following are images of our results in applying our method (ordered left to right): First image: The binary image representing the segmentation superimposed on the original image. Second image: The segment removed in the first image, showing how some segments which passed all filtration techniques were not in fact grape clusters. Third image: This image is a simulation of the desired segmentation results. The blank white areas are the segments which were not detected by our method.

remove grass-like areas

This second set of resulting images showcases our method's shortcomings. In some cases (if further segmentation methods are not applied - i.e. K-means clustering, use of color properties etc.) irrelevant objects are mistakenly identified as grape clusters. This result is due to regional maxima clusters of similar properties not being filtered out by complementary methods.

remove grass-like areas

Conclusions

Analyzing the results of applying the method on various images, we can see most non-grape-cluster areas were identified as such. Notwithstanding, we also see that the process is not perfect as it produces false positives: in some of the images there are areas which were detected as grape clusters but in actuality portrayed shaded grass. The method of focusing on the regional maxima of the brighter areas does not differentiate the regional maxima of grape clusters from other kinds of regional maxima clusters and as a result we had quite a few clusters of 'red spots' falsely identified as grape clusters. By checking which images produced the best results we can define the optimal conditions for applying our method.

The best results are achieved when analyzing a. well lit images with b. minimum clustered regional maxima beside the grape clusters and c. grape clusters of similar proportions.

Additional Information

References