Predicting Yield of Fruit and Flowers using Digital Image Analysis

Indian Journal of Science and Technology, Vol 8(32), DOI: 10.17485/ijst/2015/v8i32/93730, November 2015 ISSN (Print) : 0974-6846 ISSN (Online) : 0974-5645 Predicting Yield of Fruit and Flowers using Digital Image Analysis Humaira Nisar*, Hoo Zhou Yang and Yeap Kim Ho Department of Electronic Engineering, Faculty of Engineering and Green Technology, Universiti Tunku Abdul Rahman, Jalan Universiti, Bandar Barat, 31900, Kampar, Malaysia; humaira@utar.edu.my, hoozy0153@hotmail.com, yeapkh@utar.edu.my Abstract Background/Objectives: The objective of this research is to predict the yield of fruit and flowers to help farmers to plan the sales, the shipment and operations related to the harvest. Methods/Statistical Analysis: The proposed algorithm involves noise removal, image segmentation, size thresholding and shape analysis; for automated counting of the regions of interest, and finally yield prediction. We have used different channels of two color spaces RGB and YCbCr for our study. 28 images of Dragon fruit and 26 images of Daisy flower are used for simulations. Findings: The percentage error in automated counting for RGB model (R-G channel) is 8.75% for Dragon fruit and 11.30% for Daisy flower while for YCbCr model (Cr channel) percentage error is 8.07% for Dragon fruit and 5.54% for Daisy flower. Based on our analysis we may conclude that Cr channel of YCbCr color model gives better results. Regression analysis gives R2 equal to 0.9517 and 0.9751 for Dragon fruit and Daisy flower respectively between the manual and automated counting. The average percentage error in yield prediction for Dragon fruit is 1.40% and Daisy flower is 5.52%. Application/Improvement: Based on our findings we can conclude that image processing based automated system for agricultural yield prediction can help to estimate the agricultural harvest. Keywords: Automated Counting, Agriculture, Dragon Fruit, Daisy Flower, Yield Prediction 1. Introduction In agriculture the counting of the number of fruits and lowers play an important role to estimate the amount of harvest. he manual counting of fruit and lowers in a farm is a very tiresome job, it needs plenty of time to complete the task, involves high cost and has low accuracy. Image processing techniques can help to accurately count the harvest of the ield/orchard. hus, automated fruit and lower counting is introduced in the agriculture ield by using digital image analysis to count the total number of fruit/lowers and hence predict or estimate the yield of the produce. Digital Image analysis is commonly used in many applications for automating the process. For example biomedical imaging1–3, satellite imaging4, agriculture5, biotechnology, industrial automation, soil sciences etc. Manual counting of products in a farm may lead to bad estimation due to the inaccuracy associated with * Author for correspondence manual counting. If overestimated, it will cause the farm to lose money on the shipping part since ordered more placements and the harvested product is less in amount. On the other hand, if the products are underestimated; the farm will sufer from insuicient picker and packer staf to handle the harvested product. hus, the pre-order shipment will need to add-on extra weight to ship the harvest to desired destinations. Hence, the automated fruit and lower counting technique will be a very helpful system for the agricultural community. To date we have not seen any automated method for counting and yield estimation for the dragon fruit and daisy lower prior to harvest. he irst step in automated fruit counting is input image acquisition. he light source, the picture background and the distance between camera and object requires to be controlled to improve the quality of the digital image for accurate image segmentation of object of interest6. here are several techniques and algorithms to perform the segmentation in the digital images such as Predicting Yield of Fruit and Flowers using Digital Image Analysis thresholding, clustering, edge detection, histogram-based methods etc. hresholding techniques are widely used for image segmentation because of their simplicity and to separate the selected image into numerous areas based on the gray levels of the image. Manual threshold selection is done by trial and error by using the histogram of the selected image7. Clustering techniques separate diferent areas based on the similarity without prior information. FCM (Fuzzy C-Means) clustering method is suitable for more than one cluster while the crisp method is suitable to classify only one cluster. FCM is sensitive to the variation in illumination. Color is a fundamental feature in a digital image. An algorithm can be based on color to diferentiate the objects of interest in an image8,9. he RGB color value can be used to segment fruit from the input image using an appropriate threshold based on the color intensity of the object of interest. For example, apples in an image can be segmented from background with the color channel diferences of R-B in the RGB color space10. he fruit recognition methods also use initial points of interest and Bag-of-Words (BoW) model. he points of interest include color transformations and color classiier. he RGB color intensity was transformed to diferentiate the fruit and other plant parts, the transformations done were G-B, G-R and G/(R+G+B). his color transformation is more sensitive compared to original R, G and B values than changing the illumination conditions11. he rest of the paper is organized as follows. In Section 2, the research methodology will be discussed. It will be followed by experimental results in Section 3. Finally paper will be concluded in Section 4 followed by references. 2. Methodology In this research we have proposed a method for automated fruit and lower counting and yield prediction which may be very helpful for agricultural automation. he algorithms have been used in MATLAB and Image Processing toolbox is used. 28 images of dragon fruit and 26 images of daisy lower are used in the analysis. he images are obtained from the web. he automated counting of an object of interest can be divided into ive steps; which are image acquisition, image noise removal, image segmentation, object recognition, automated object counting and lastly the yield prediction. Figure 1 gives an overview of the steps involved in the proposed algorithm. We will discuss all step in detail in the following subsections. 2 Vol 8 (32) | November 2015 | www.indjst.org Figure 1. Block diagram of the proposed algorithm. 2.1 Image Noise Removal A simple smoothing ilter is used for reducing noise and blurring efect from the images. he smoothing ilter is also applied to remove small details and bridging small gaps in contour ater segmentation. he smoothing ilter replaces the value of every pixel in the image by the average of the intensity levels in the neighborhood deined by the ilter mask. he random noise typically consists of sharp transitions in the intensity levels, thus the smoothing ilter is able to perform the noise reduction efectively. Ater trial and error, Gaussian ilter of 3x3 size is used as shown in Figure 2. Figure 2. he comparison between input image and Gaussian iltered image. 2.2 Image Segmentation and Post Processing Image segmentation is the process of partitioning an image into numerous segments to ind the object of interest. In this paper, the thresholding method is used to segment the image. We have used two color spaces to study the efect of diferent color spaces (channels) on segmentation. hese are RGB and YCbCr. RGB consists of Red (R) , Green (G) and Blue (B) color channels, whereas YCbCr consists of Luminance (Y), Chrominance Blue (Cb) and Chrominance Red (Cr). he YCbCr color space is chosen for the image segmentation as the Cr is strong in places of occurrence of reddish color. hus, the dragon fruits are very bright in the images in Cr color space and may be able to be segmented out easily. While using RGB color space, the images under Red channel show many bright objects that includes sky, tree stem along with dragon fruit has Indian Journal of Science and Technology Humaira Nisar, Hoo Zhou Yang and Yeap Kim Ho the same intensity. hus, when the image obtained from the Red plane is threshold into binary image, all bright objects in the image will be separated using a threshold of 0.7. he red, green and blue colors have high correlation in RGB color space. So instead of using R channel alone, R-G channel is used to eliminate other bright objects that are not needed. his is shown in Figure 3. Figure 4 shows the red chrominance (Cr) image converted into binary image with a threshold of 0.6, resulting in the pixels of fruit in white color and the pixels of the background is black color. he segmented region will have some holes or noise in the binary image, morphological image processing function “imill” is used to ill the holes in the binary segmented image as shown in the Figure 5. Ater that image erosion is applied to the binary image to separate the connected fruit regions as shown in Figure 6. he image erosion uses the diamond structuring element of size 20. Lastly, the morphological opening function is applied on the binary segmented image followed by the morphological closing to remove noise from the segmented image. Figure 5. Holes illed up in the segmented region. Figure 6. Erosion of segmented region. 2.3 Object Recognition using Morphological Shape Analysis Figure 3. Color channel comparison (Red channel, R-G channel and Cr channel). Morphological image processing gives an idea of the shape or morphology of objects in an image. Shape analysis gives the shape of the segmented region of interest; and then classiies the objects into diferent classes based on the shape of interest. In this paper the roundness analysis is used to improve the accuracy of object detection. he area and perimeter of each segmented region will be estimated. he boundary coordinate will be used to estimate the area and perimeter of the segmented region using morphological functions with the extracted boundary. he roundness value of the segmented region is obtained by applying the formula. (1) Figure 4. Convert red chrominance image into binary image by thresholding using 0.6 value. Vol 8 (32) | November 2015 | www.indjst.org Indian Journal of Science and Technology 3 Predicting Yield of Fruit and Flowers using Digital Image Analysis Where, R = Roundness of the segmented region. A = Area of the segmented region. P = Perimeter of the segmented region. he roundness value is equal to 1 for a circle and its value less than 1 for any other shape. From experimental analysis, it has been found that the roundness value between 0.45 and 0.60 refers to occluded and/or overlapped objects. Segmented regions having the roundness values between 0.45 to 0.60 will be counted as two fruits in our algorithm, as shown in Figure 7. Figure 7. objects. he roundness value obtained for identiied 2.4 Automated Counting he automated counting uses the size threshold for the segmented region. If the area of segmented region is greater than a threshold, then the white area will be counted as 1 dragon fruit. Next, the shape analysis algorithm will count the segmented regions with roundness value between 0.45 and 0.60 as one more dragon fruit. he total fruit count is the sum of the fruit count in size threshold and shape analysis. 2.5 Yield Prediction Dragon fruit is red in color, round in shape and is about 10-15 cm in size. It weights between 300 to 500 grams. Dragon fruit can be harvested all year round and the peak seasons are between April and September. he harvesting time may vary from place to place. Once the crop matures the average production in one hectare is more than 10 tons per hectare. Weight of one Dragon fruit can be taken as around 400 gms as reference. here are about 1500 dragon fruit plants in one hectare. he size of fruit depends on several factors such as weather, suicient water and farm management12. he spacing of daisy plant between the rows should be 30-40 cm and 25-30 cm within the row accommodating 8-10 plants/m2 and harvests 3 times per year. Average yield of cut lowers under open conditions are around 130-160 lowers/m2/year of which only 15-20 % of I grade quality is obtained13. he reference yield takes the average of the lowers under open conditions which is 145 lowers/m2/ year. he complete lowchart of the proposed method is shown in Figure 8. Figure 8. he low chart of the proposed algorithm. 4 Vol 8 (32) | November 2015 | www.indjst.org Indian Journal of Science and Technology Humaira Nisar, Hoo Zhou Yang and Yeap Kim Ho 3. Experimental Results Table 3. Percentage error in yield prediction A total of 28 images for dragon fruit and 26 daisy images are used in our experiments. We have used R minus G (RG) channel of RGB and Cr channel of YCbCr color space for segmentation. Based on the segmentation results, the YCbCr color space is better than the RGB color space since in YCbCr color space Cr component segments the object better than the R-G plane that has more noise. In RGB color space the chrominance and luminance components are mixed that is why RGB is not a very good choice for color analysis and color based segmentation algorithm. Table 1 and Table 2 shows the results for dragon fruit segmentation and daisy segmentation. Figure 9 and 10 shows the segmented dragon fruit and daisy images respectively. Figure 11 and 12 shows the regression analysis for no of counted fruit and lowers respectively for manual and automated counting. Table 3 provides yield prediction results. For the yield prediction of Dragon fruit, number of trees is taken into account. For example, for 10 trees average fruit on each tree is 16.9, and hence the weight of dragon fruit will be 6.76 kg which gives an estimated yield of 10140 kg/hectare. For Daisay, the predicted yield is based on total lowers in the images and the area in m2. he area estimated based on the lowers size in the image and is compared to the actual image size. he average predicted yield is 137.12 lowers/m2/year compared to the reference yield which is 145 lowers/ m2/year. he predicted yield has the percentage error of 5.52% as shown in the Table 4. Dragon Fruit (Kg/hectare) Daisy (lowers/ m2 /year Reference Predicted Yield Yield 10000 10140 145 % Error 137 1.40 5.52 Figure 9. Input, segmented binary image and shape analysis result for dragon fruit. Figure 10. Input and the segmented daisy image. Table 1. Comparison of results obtained from diferent color spaces and shape analysis for segmentation of Dragon fruit Av. error (%) R2 in Regression analysis RGB (R-G) YCbCr (Cr) 13.57 0.97 11.76 0.98 RGB+ Shape analysis 8.75 0.94 YCbCr+ Shape analysis 8.078 0.952 Figure 11. Regression analysis between the no. of fruits counted by automated and manual counting. Table 2. Comparison of results for segmentation for Daisy lower Av. error(%) R2 in Regression analysis RGB(R-B) 11.30 0.93 Vol 8 (32) | November 2015 | www.indjst.org YCbCr(Cr) 5.54 0.98 Indian Journal of Science and Technology 5 Predicting Yield of Fruit and Flowers using Digital Image Analysis 5. References Figure 12. Regression analysis between the no. of lowers counted by automated and manual counting. 4. Conclusion and Recommendations In this paper we have successfully proposed digital image processing and analysis techniques for automation of agricultural products and prediction of yields. he implemented image processing techniques include color, size and shape features. he proposed algorithm is able to segment fruit and lower (dragon fruit and daisy) and quantify total number of fruit and lower with an average error of 8.0779% and 5.5434% with the R2 value of 0.9517 and 0.9751 respectively. he predicted yield for dragon fruit is 10140 kg/hectare, while the daisy lower is 137 lowers/m2/year with the percentage error of 1.40% and 5.52% respectively. his is a simple and cheap method that may help farmers to predict the yield of the farm and able to arrange the transportation and sale of the harvested products with ease and proper planning. In order to further improve the accuracy of segmentation and the predicted yield result. Some extra features and improvement can be added into the algorithm developed such as unsupervised learning and surface texture feature to increase the region of interest segmentation accuracy. A robot mounted with camera can be designed to move around the farm and automatically predict the yield of farm. 6 Vol 8 (32) | November 2015 | www.indjst.org 1. Verma A, Khanna G. A survey on digital image processing techniques for tumor detection. Indian Journal of Science and Technology. 2016 Apr; 22:9(14). 2. Kumar SL, Swathy M, Sathish S, Sivaraman J, Rajasekar M. Identiication of lung cancer cell using watershed segmentation on CT images. Indian Journal of Science and Technology. 2016 Jan 31; 9(1). 3. Tae-Eun K. A study of medical image analysis system. Indian Journal of Science and Technology. 2015 Oct 9; 8(25). 4. Ganesan P, Rajini V, Sathish BS, Kalist V, Basha SK. Satellite image segmentation based on YCbCr color space. Indian Journal of Science and Technology. 2015 Jan 16; 8(1):35–41. 5. Chitra HS, Suguna S, Sujatha SN. A survey on image analysis techniques in agricultural product. Indian Journal of Science and Technology. 2016 Mar 30; 9(12). 6. Dorj UO, Lee KK, Lee M. A computer vision algorithm for tangerine yield estimation. International Journal of Bio-Science and Bio-Technology. 2013; 5(5):101–10. 7. Sarkate RS, Kalyankar NV, Khanale PB. Application of computer vision and color image segmentation for yield prediction precision. 2013 International Conference on Information Systems and Computer Networks (ISCON); Mathura, India. 2013 Mar 9-10. 8. Hung C, Underwood J, Nieto J, Sukkarieh S. A feature learning based approach for automated fruit yield estimation. IEEE Intelligent Robots and Systems (IROS); Tokyo, Japan. 2013 Nov 3-7. 9. Nisar H, Ch’ng YK, Chew TY, Tang JJ, Yap VV, Yeap KH. A color space study for skin lesion segmentation. 2013 IEEE International Conference on Circuits and Systems (ICCAS); 2013 Sep 18. p. 172–6. 10. Anisha S., Divya G., Shanu S. A Survey of Computer Vision Methods for Counting Fruits and Yield Prediction. International Journal of Computer Science Engineering (IJCSE) 2013, 2 (6), pp.346-350 11. Song Y, Glasbey CA, Horgan GW, Polder G, Dieleman JA, van der Heijden GWAM. Automatic fruit recognition and counting 1 from multiple images. Biosystems Engineering. 2014; 118:203–15. ISSN: 1537-5110. 12. Zainudin M. Dragon fruits cultivation in Malaysia. Available from: http://www.scribd.com/doc/91726161/DragonFruits-Cutivation-in-Malaysia#scribd 13. Sandesh K. Agriculture in India; 2010. Available from: http://www.krishisandesh.com/lowers/gerbera Indian Journal of Science and Technology