3688

Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/361083688

DETECTION OF A FRUIT USING COMPUTER VISION

Conference Paper · May 2022

CITATIONS READS

0 127

2 authors, including:

Camilo Pardo-Beainy
Saint Thomas University
38 PUBLICATIONS 41 CITATIONS

SEE PROFILE

All content following this page was uploaded by Camilo Pardo-Beainy on 04 June 2022.

The user has requested enhancement of the downloaded file.


CIETA

DETECTION OF A FRUIT USING COMPUTER VISION

DETECCIÓN DE FRUTA USANDO VISIÓN POR COMPUTADOR

Shantanu Joshi*, Camilo Pardo-Beainy**

*Manipal Institute of Technology


Electrical and Electronics Department,
Karnataka, India.
E-mail: shantanu.joshi@learner.manipal.edu

**Universidad Santo Tomás


Electronic Engineering Faculty,
Tunja, Boyacá Colombia.
Tel: 57-8-7440404, Ext. 5612
E-mail: camilo.pardo@usantoto.edu.co

Abstract: Computer Vision technology can be very efficient in tracking and analyzing
agricultural produce, enabling the collection of useful data and implementation of future
strategies, while being extremely cost effective compared to sensor technologies used in
the past. It can be useful in rural areas as a part of precision agriculture. A simple system
consisting of only a webcam can be used for detection, tracking, and measurement of a
fruit by utilizing simple computer vision algorithms and image processing techniques, as
is indicated in this paper. Tomato has been selected as the trial fruit for this project, which
is widely cultivated in India. In order to emphasize the simplicity and cost effectiveness
of such a system, acquisition has been made with an inbuilt webcam on a computer.
Common algorithms and techniques have been used in the pre-processing and
segmentation stages, followed by measurements of a few sample tomatoes. The paper
also contains an overview of the processes involved. The results obtained through the
system have been compared with physical measurements and the percentage difference
between the two observations has been tabulated, to show the reliability of even such an
inexpensive model.

Keywords: Image Processing, Computer Vision, Fruit detection, Precision agriculture.

Resumen: La tecnología de visión por computador puede ser muy eficaz en el


seguimiento y el análisis de los productos agrícolas, permitiendo la recogida de datos
útiles y la aplicación de futuras estrategias, al tiempo que resulta extremadamente rentable
en comparación con las tecnologías de sensores utilizadas en el pasado. Puede ser útil en
las zonas rurales como parte de la agricultura de precisión. Un sistema simple que
consiste en sólo una cámara web puede ser utilizado para la detección, el seguimiento y la
medición de una fruta mediante la utilización de algoritmos simples de visión por
ordenador y técnicas de procesamiento de imágenes, como se indica en este documento.
Se ha seleccionado el tomate como fruta de prueba para este proyecto, que se cultiva
ampliamente en la India. Para destacar la simplicidad y rentabilidad de este sistema, la
adquisición se ha realizado con una cámara web incorporada en un ordenador. Se han
utilizado algoritmos y técnicas comunes en las etapas de preprocesamiento y
segmentación, seguidas de mediciones de unos pocos tomates de muestra. El documento
también contiene una descripción general de los procesos implicados. Los resultados
obtenidos mediante el sistema se han comparado con las mediciones físicas y se ha
tabulado el porcentaje de diferencia entre ambas observaciones, para mostrar la fiabilidad
incluso de un modelo tan económico.

Palabras clave: Procesamiento de imágenes, visión por computador, detección de frutas,


agricultura de precisión.

1
Universidad de Pamplona
CIETA

1. INTRODUCTION growth with aid of artificial intelligence and deep


learning advancements in recent years (Kakani et
This project is aimed at detecting and measuring al., 2020; Pardo-Beainy et al., 2020).
a fruit using the OpenCV library for Computer
Vision. In this paper, tomatoes have been used to Computer vision technologies that use digital
test the system and to compare the results of the images to interpret and understand the world, are
measurements with the actual physical capable of providing accurate, site-specific
measurements. Different image processing information about crops and their environments
techniques have been implemented to obtain (Lu & Young, 2020).
clear and reliable output data to carry out the
final measurements of the fruit. 2.2 Camera Types

In India, the agricultural sector is the largest The discipline of Computer Vision uses many
source of livelihood. The country accounts for different types of cameras based on the required
10.9% and 8.6% of the world fruit and vegetable applications. Some of the cameras used
production, respectively. However, it is the commonly are linear cameras, 3-D cameras,
majority of the rural households that depend on high-speed cameras, infrared and thermal
agriculture with around 82% farmers being small cameras, etc.
and marginal (India at a Glance | FAO in
India | Food and Agriculture Organization of the For the implementation of this project however, a
United Nations, n.d.). Due to limited resources, it simple webcam has been used, as it serves the
is the cost-effective methods of precision
purpose of capturing objects clearly from a close
agriculture that can greatly help such farmers.
range.
Computer vision has paved way for finding such
solutions as presented in (Tripathi & Maktedar,
2020) (Kakani et al., 2020). 2.3 Methodology

There are several major tomato producing states The following methodological design was
in India. While these states produce a high yield followed in this project.
of tomatoes, these tomatoes are often prone to
several diseases such as Early Blight which can 2.3.1 Acquisition
result in lesions that cover a significant part of The integrated webcam cam (Realtek) along with
the fruit. While grading, such rotten and a stable and constant light source for luminosity
damaged fruits should be removed, and only the is required for the acquisition of image frames.
healthy and clean fruits should be selected. Size
of the tomato has to be considered as well. Size Luminosity plays a very important role in any
grading is done by retailers for local markets. computer vision system and can greatly affect the
Bureau of Indian Standards has specified four results, especially since the colour space used in
grades as Super A, Super, Fancy and this project is HSV, and the thresholding values
Commercial for tomato crop (National are kept fixed.
Horticulture Board, n.d.). Thus, it is important to
select tomatoes in the right color range and size. 2.3.2 Pre-processing
Pre-processing includes performing certain
The system, once developed further, can help operations on the input data without which noise
local farmers with limited resources estimate the and other effects degrade its quality and do not
quality of the produce based on the color, count,
provide correct information for further
and size of the fruit, which can save time and be
processing (Arakeri & Lakshmana, 2016).
also used for real-time data collection.
Changing colour spaces, applying filters, scaling,
2. MATERIALS AND METHODS translation, etc. are some techniques applied at
this stage, shown in Fig.1.
2.1 Computer Vision

Computer Vision is the field of computer science


that focuses on enabling computers to interpret
and process digital images and videos and gain a
high-level understanding of them (Tian et al.,
2020). It can be used for the automation of a Fig. 1. Pre-processing operations.
diverse range of tasks and has seen a significant

2
Universidad de Pamplona
CIETA

The elements used for the pre-processing and


segmentation steps were Ubuntu-64 bits, AMD
Ryzen 7 4700U with Radeon Graphics 2.00
GHz, and an 8 GB RAM.

Color space is essentially a space of colors, or a


system representing an array of pixel colors.
Color spaces such as RGB, LAB, YCrCb, and
HSV are some of the common color spaces used.
The selection of a color space is determined
based on the desired selection criteria of the sites
of interest in a given image.

Fig. 3. BGR histograms before and after


applying a mask.

2.3.3 Segmentation
Image segmentation, with its operations shown
Fig. 2. RGB (left) and HSV (right) colorspaces. in Fig.4, is typically used to detect contours and
(Ireri et al., 2019) areas of the image, among other features.

In this project, the use of HSV color space,


shown in Fig.2, has been made. RGB color space
is sensitive to ambient lighting conditions (Ireri
et al., 2019). Hue, Saturation, and Value can be
thought of as relating to the dominant
wavelength of color, the purity or shade of color,
and the intensity of color respectively. This color Fig. 4. Segmentation stage operations.
space has one separate channel to describe color
only, as opposed to an additive color space and is The use of morphological transformations has
also device independent. Further, a drastic been made for closing the small holes inside the
change for red color can be observed in both dim foreground objects.
and bright conditions, making it the ideal color
space to use in our case of analyzing tomatoes. A certain threshold is applied on the pixel
intensity of an image, after it has been converted
Following the change of color space, two masks to grayscale, to separate objects from their
are applied which cover the lower and upper background (Nyalala et al., 2021). Due to the
range of red tones that can be found in tomatoes absence of a completely controlled lighting
in ideal conditions. The use of AND logical environment, these steps are necessary to be
operation has been made which interprets any included, in order to fix the imperfections in the
values not in the mask range as zero and displays images. If the value of the pixels exceeds the
them as black pixels (OpenCV: Image threshold, they are coded as white, otherwise
Thresholding, n.d.). they are coded black. This is known as binary
thresholding (OpenCV: Image Thresholding,
To approximate the lower and upper range of the n.d.).
masks, and to confirm that only the red pixels are
being selected, BGR image histograms, shown in In order to find the desired contours, a
Fig.3, can be used (Tripathi & Maktedar, 2020). conditional statement is used which selects the
Next, the image is converted to gray scale before contours larger than 2500 pixels in area. This
further processing. This reduces the image value will be dependent on the distance of the
dimensionality. stationary camera and the objects. Once the
objects, in this case the tomatoes, are detected, a
frame will be captured into an image, using
which the objects will be measured.

2
Universidad de Pamplona
CIETA

2.3.4 Measurement dimensions of the other objects are calculated


The frame captured in the last step will go using the method in (Measuring Size of Objects
through further processing with the application in an Image with OpenCV - PyImageSearch,
of a gaussian blur, canny edge detection, erosion n.d.). The size is then displayed beside the lines
and dilation. joining the midpoints of the rectangle. The flow
of the program is shown in Fig.5.
Blurring an image using a Gaussian function
(OpenCV: Smoothing Images, n.d.) is called
Gaussian Blurring. The Gaussian filter is a subset
of linear smoothing filters. The kernel of this
filter has isotropic or circular symmetrical
weights drawn from a normal distribution (Ismail
& Malik, 2021). It is a very commonly used
effect to reduce image noise and reduce detail.

Canny Edge Detection is an edge detection


algorithm used to detect a wide range of edges
and also to extract useful structural information
from an image (OpenCV: Canny Edge Detection,
n.d.).
Fig. 5. Flowchart of the program.
Erosion is useful for removing small white
noises, detach two connected objects, etc. It Once the program is executed on Jupyter
erodes away the boundaries of objects present in Notebook, the following results are obtained.
the foreground. The pixels near the boundary of
the object will be discarded as per the size of 3. RESULTS
kernel selected. So, the thickness or size of the
foreground object decreases, or put simply, the Relevant information such as the number of
white region decreases in the image. objects detected and the coordinates of their
centroids are displayed along with the objects
Dilation is the opposite of erosion. It increases themselves being encircled, as shown in Fig.6.
the white region in the image or we can say that
the size of foreground object increases. For noise
removal, erosion is followed by dilation because
first erosion removes white noises and shrinks
the object. After noise is gone, we can dilate the
object and increase its area. It is also used to join
broken parts of an object (OpenCV: Fig. 6. Steps leading to the detection of tomatoes
Morphological Transformations, n.d.). which are encircled. Their total number and
centroid positions are displayed.
While measuring the objects, it is required to
have a fixed object which can be taken as The measurements obtained, as shown in Fig.7,
reference. This will serve as a calibration step. It through the program (the average of the two
is also required to sort the contours from left to lengths between the midpoints of the rectangle)
right, keeping the reference object as the leftmost are compared with the actual measurements
one and assigning the vertices of the contours in taken for the three test tomatoes and the results
top-left, top-right, bottom-right, bottom-left are tabulated in table 1.
arrangement so that the object can be measured
kept in any orientation (Ordering Coordinates Table 1. Comparison of Measurements.
Clockwise with Python and OpenCV -
PyImageSearch, n.d.). Program Physical
Percentage
# Object Measurement Measurement
Difference
(cm) (cm)
Following this, bounding rectangles are drawn
Tomato
on the contours of a certain minimum area and 1 5.08 5.0 1.60%
A
the Euclidean distance between the opposite Tomato
midpoints of the rectangle is calculated. A value 2 5.33 5.1 4.51%
B
is kept fixed for the reference object. In this case, Tomato
3 6.99 6.6 5.91%
the value was taken as 1.2 inches, and the C

3
Universidad de Pamplona
CIETA

REFERENCES

Arakeri, M. P., & Lakshmana. (2016). Computer


Vision Based Fruit Grading System for
Quality Evaluation of Tomato in
Agriculture industry. Procedia Computer
Science, 79, 426–433.
https://doi.org/10.1016/j.procs.2016.03.055
Illustration of the HSV Color Space B. Color
Feature Extraction Color... | Download
Scientific Diagram. (n.d.). Retrieved July
8, 2021, from
https://www.researchgate.net/figure/llustrat
ion-of-the-HSV-Color-Space-B-Color-
Feature-Extraction-Color-feature-is-
extracted_fig1_321126312
India at a glance | FAO in India | Food and
Agriculture Organization of the United
Nations. (n.d.). Retrieved July 8, 2021,
from http://www.fao.org/india/fao-in-
Fig. 7. Measuring objects with reference to a
india/india-at-a-glance/en/
fixed object (leftmost).
Ireri, D., Belal, E., Okinda, C., Makange, N., &
4. CONCLUSIONS Ji, C. (2019). A computer vision system for
defect discrimination and grading in
It is important to maintain constant luminosity tomatoes using machine learning and
while making the observations as it is a factor image processing. Artificial Intelligence in
which can greatly affect the results. Also, it is Agriculture, 28–37.
important to place the camera perpendicular to https://doi.org/10.1016/j.aiia.2019.06.001
the plane where the objects rest, to avoid errors Ismail, N., & Malik, O. A. (2021). Real-time
in measurements with respect to the reference visual inspection system for grading fruits
object. Once the reference object has been fixed, using computer vision and deep learning
the distance between the plane and the camera techniques. Information Processing in
should not change. Agriculture,1–14.
https://doi.org/10.1016/j.inpa.2021.01.005
To make sure the measurements can be taken in
any orientation of the contours, it is important to Kakani, V., Nguyen, V. H., Kumar, B. P., Kim,
arrange the vertices of the contours in order, and H., & Pasupuleti, V. R. (2020). A critical
also to sort the contours themseleves from left to review on computer vision and artificial
right. intelligence in food industry. Journal of
Agriculture and Food Research,
In the measurement algorithm, pixels per metric 2(November 2019), 100033.
concept is used. Pixels per metric = 150px / https://doi.org/10.1016/j.jafr.2020.100033
0.955in = 157px (Measuring Size of Objects in Lu, Y., & Young, S. (2020). A survey of public
an Image with OpenCV - PyImageSearch, n.d.). datasets for computer vision tasks in
Thus, implying there are approximately 157 precision agriculture. Computers and
pixels per every 0.955 inches in our image. Electronics in Agriculture, 178(May),
Using this ratio, we can compute the size of 105760.
objects in an image. The results obtained and https://doi.org/10.1016/j.compag.2020.105
tabulated show that the system for measurement 760
is reliable given the simplicity of the setup. Measuring size of objects in an image with
OpenCV - PyImageSearch. (n.d.).
As a continuation of this project, advanced image Retrieved July 6, 2021, from
processing techniques, along with the use of https://www.pyimagesearch.com/2016/03/
machine learning algorithms, it can be proposed 28/measuring-size-of-objects-in-an-image-
to determine the quality of a batch of harvested with-opencv/
tomatoes based on the number count, the colour, National Horticulture Board. (n.d.). Retrieved
size, and shape. The results could be compared July 8, 2021, from
with a dataset of an ideal harvest and an http://nhb.gov.in/PDFViwer.aspx?enc=3Z
estimation could be made regarding the quality OO8K5CzcdC/Yq6HcdIxFBfvWYIQVxm
of harvest.

4
Universidad de Pamplona
CIETA

3j7FuQFh+g8=
Nyalala, I., Okinda, C., Kunjie, C., Korohou, T.,
Nyalala, L., & Chao, Q. (2021). Weight
and volume estimation of poultry and
products based on computer vision
systems: a review. Poultry Science, 100(5),
101072.
https://doi.org/10.1016/j.psj.2021.101072
OpenCV: Canny Edge Detection. (n.d.).
Retrieved July 6, 2021, from
https://docs.opencv.org/3.4/da/d22/tutorial
_py_canny.html
OpenCV: Image Thresholding. (n.d.). Retrieved
July 6, 2021, from
https://docs.opencv.org/3.4/d7/d4d/tutorial
_py_thresholding.html
OpenCV: Morphological Transformations. (n.d.).
Retrieved July 6, 2021, from
https://docs.opencv.org/3.4/d9/d61/tutorial
_py_morphological_ops.html
OpenCV: Smoothing Images. (n.d.). Retrieved
July 6, 2021, from
https://docs.opencv.org/3.4/d4/d13/tutorial
_py_filtering.html
Ordering coordinates clockwise with Python and
OpenCV - PyImageSearch. (n.d.).
Retrieved July 9, 2021, from
https://www.pyimagesearch.com/2016/03/
21/ordering-coordinates-clockwise-with-
python-and-opencv/
Pardo-Beainy, C., Gutiérrez-Cáceres, E., Pardo,
D., Medina, M., & Jiménez, F. (2020).
Sistema de Interacción con Kinect
Aplicado a Manipulación de Procesos.
Revista Colombiana de Tecnologias de
Avanzada (RCTA), 3(Especial), 11–16.
https://doi.org/10.24054/16927257.VESPE
CIAL.NESPECIAL.2020.849
Tian, H., Wang, T., Liu, Y., Qiao, X., & Li, Y.
(2020). Computer vision technology in
agricultural automation —A review.
Information Processing in Agriculture,
7(1), 1–19.
https://doi.org/10.1016/j.inpa.2019.09.006
Tripathi, M. K., & Maktedar, D. D. (2020). A
role of computer vision in fruits and
vegetables among various horticulture
products of agriculture fields: A survey.
Information Processing in Agriculture,
7(2), 183–203.
https://doi.org/10.1016/j.inpa.2019.07.003

5
Universidad de Pamplona

View publication stats

You might also like