Brosnan y Sun PDF
Brosnan y Sun PDF
Brosnan y Sun PDF
www.elsevier.com/locate/jfoodeng
Review
FRCFT Group, Department of Agricultural and Food Engineering, University College Dublin, National University of Ireland,
Earlsfort Terrace, Dublin 2, Ireland
Received 29 April 2002; accepted 6 May 2003
Abstract
With increased expectations for food products of high quality and safety standards, the need for accurate, fast and objective
quality determination of these characteristics in food products continues to grow. Computer vision provides one alternative for an
automated, non-destructive and cost-eective technique to accomplish these requirements. This inspection approach based on image
analysis and processing has found a variety of dierent applications in the food industry. Considerable research has highlighted its
potential for the inspection and grading of fruits and vegetables. Computer vision has been successfully adopted for the quality
analysis of meat and sh, pizza, cheese, and bread. Likewise grain quality and characteristics have been examined by this technique.
This paper presents the signicant elements of a computer vision system and emphasises the important aspects of the image processing technique coupled with a review of the most recent developments throughout the food industry.
2003 Elsevier Ltd. All rights reserved.
Keywords: Machine vision; Computer vision; Image processing; Image analysis; Fruit; Vegetables; Grain; Meats; Online inspection
Contents
1.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.
3.
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1. Bakery products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2. Meat and sh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3. Vegetables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4. Fruit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5. Prepared consumer foods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6. Grain . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.............................
3.7. Food container inspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8. Other applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.
Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6
7
7
8
9
11
11
12
13
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
0260-8774/$ - see front matter 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S0260-8774(03)00183-3
1. Introduction
The increased awareness and sophistication of consumers have created the expectation for improved
quality in consumer food products. This in turn has
increased the need for enhanced quality monitoring.
Quality itself is dened as the sum of all those attributes
which can lead to the production of products acceptable
to the consumer when they are combined. Quality has
been the subject of a large number of studies (Shewfelt &
Bruckner, 2000). The basis of quality assessment is often
subjective with attributes such as appearance, smell,
texture, and avour, frequently examined by human
inspectors. Consequently Francis (1980) found that
human perception could be easily fooled. Together with
the high labour costs, inconsistency and variability associated with human inspection accentuates the need for
objective measurements systems. Recently automatic
inspection systems, mainly based on cameracomputer
technology have been investigated for the sensory
analysis of agricultural and food products. This system
known as computer vision has proven to be successful
for objective measurement of various agricultural (He,
Yang, Xue, & Geng, 1998; Li & Wang, 1999) and food
products (Sun, 2000; Wang & Sun, 2001).
Computer vision includes the capturing, processing
and analysing images, facilitating the objective and nondestructive assessment of visual quality characteristics in
food products (Timmermans, 1998). The potential of
computer vision in the food industry has long been
recognised (Tillett, 1990) and the food industry is now
ranked among the top 10 industries using this technology (Gunasekaran, 1996). Recent advances in hardware
and software have aided in this expansion by providing
low cost powerful solutions, leading to more studies on
the development of computer vision systems in the food
industry (Locht, Thomsen, & Mikkelsen, 1997; Sun,
2000). As a result automated visual inspection is under
going substantial growth in the food industry because of
Table 1
Benets and drawbacks of machine vision
Reference
Advantages
Generation of precise descriptive data
Quick and objective
Reducing tedious human involvement
Consistent, ecient and cost eective
Automating many labour intensive process
Easy and quick, consistent
Non-destructive and undisturbing
Sapirstein (1995)
Li, Tan, and Martz (1997)
Disadvantages
Object identication being considerably more dicult in unstructured scenes
Articial lighting needed for dim or dark conditions
Table 2
Applications using X-ray imaging in machine vision
Frame grabber
Camera
Lights
Computer
Sample
Application
Accuracy (%)
Reference
Detection of bones in
sh and chicken
Internal defects of sweet
onions
99
Jamieson (2002)
90
98
92
Pinhole damage in
almonds
81
Representation
Pre-processing
Image
Acquisition
Problem
Domain
high level
processing
Description
Segmentation
Low level
processing
Intermediate
level
processing
Recognition
Knowledge Base
(a)
Interpretation
(b)
(c)
Fig. 3. Typical segmentation techniques: (a) thresholding, (b) edgebased segmentation and (c) region-based segmentation (Sun, 2000).
Results
3. Applications
Computer vision systems are being used increasingly
in the food industry for quality assurance purposes. The
system oers the potential to automate manual grading
practices thus standardising techniques and eliminating
as a means of predicting beef tenderness. Colour, marbling and textural features were extracted from beef
images and analysed using statistical regression and
neural networks. Their ndings indicated that textural
features were a good indicator of tenderness. Image
analysis was also used for the classication of muscle
type, breed and age of bovine meat (Basset, Buquet,
Abouelkaram, Delachartre, & Culioli, 2000). The meat
slices analysed came from 26 animals with the focus of
the analysis based on the connective tissue (directly related to meat tenderness) which contains fat and collagen variables with muscle type, breed, age, and clearly is
visible on photographic images. The study led to a
representation of each meat sample with a 58 features
vector and indicated the potential of image analysis for
meat sample recognition.
Machine vision has also been used in the analysis of
pork loin chop images (Lu et al., 2000; Tan, Morgan,
Ludas, Forrest, & Gerrard, 2000). Colour image features were extracted from segmented images (Lu et al.,
2000). Both statistical and neural network models were
employed to predict colour scores by using the image
features as inputs and then compared to the sensory
scores of a trained panel. The neural network model was
determined as the most accurate with 93% of the 44
samples examined producing a prediction error of less
than 0.6. Fig. 4 shows an original and segmented pork
loin chop image used in this study. In a similar study
over 200 pork lion chops were evaluated using colour
machine vision (CMV) (Tan et al., 2000). Agreement
between the vision system and the panellists was as high
as 90%. The online performance of CMV was also determined in this study with repeatedly classifying 37
samples at a speed of 1 sample per second.
A technique for the spectral image characterisation of
poultry carcasses for separating tumourous, bruised and
skin torn carcasses from normal carcasses was investigated by Park, Chen, Nguyen, and Hwang (1996).
Carcasses were scanned by an intensied multi-spectral
camera with various wavelength lters (542847 nm)
with the results indicating that the optical wavelengths
of 542 and 700 nm were the most useful for the desired
classication. For separating tumourous carcasses from
normal ones, the neural network performed with 91%
accuracy. Co-occurrence matrix texture features of
multi-spectral images were used to identify unwholesome poultry carcasses (Park & Chen, 2001). Both
quadratic and linear discriminant models had an accuracy of 97% and 95%, respectively. Defects were also
detected using the chromatic content of chicken images
(Barni, Cappellini, & Mecocci, 1997). Possible defect
areas were rst extracted by means of morphological
image reconstruction and then classied according to a
predened list of defects. Soborski (1995) investigated
the online inspection of shape and size of chicken pieces.
Fig. 5 shows a schematic of the system used.
Fig. 4. Pork loin images: (a) original image and (b) segmented muscle image (Lu et al., 2000).
Operator interface/
System Manager
Vision processor
Camera
Digital I/O
Vision / Reject
logic controller
Reject
solenoid
Existing line
controller
PIR sensor
Fig. 5. Flow diagram of a machine vision system for shape/size inspection of chicken pieces (Soborski, 1995).
energies allowing the defect to be revealed. The developed system has a throughput of 10,000 llets per hour
and can correctly identify remaining bones with an accuracy of 99%. Tao, Chen, Jing, and Walker (2001) also
investigated the use of X-ray imaging and adaptive
thresholding for the internal inspection of deboned
poultry with ndings indicating that the developed
technique can implement thickness invariant image
segmentation.
3.3. Vegetables
The necessity to be responsive to market needs places
a greater emphasis on quality assessment resulting in the
greater need for improved and more accurate grading
and sorting practices. Computer vision has shown to be
a viable means of meeting these increased requirements
for the vegetable industry (Shearer & Payne, 1990).
Shape, size, colour, blemishes and diseases are important aspects which need to be considered when
10
from good ones for the 600 samples tested. Three major
apple surface features were considered in a study by
Yang (1993) using machine vision. Extracted features
were used as inputs to neural network which gave an
average classication accuracy of 96.6% for the separation of defective samples. Further studies by Yang
(1994, 1996) reported the use of a ooding algorithm for
defect detection and stem and calyx identication. Hue
itself, and hue, saturation and intensity have also been
used as a basis for the classication of Golden Delicious
apples in studies by Heinemann, Varghese, Morrow,
Sommer, and Crassweller (1995) and by Tao, Morrow,
Heinemann, and Sommer (1995b). Both systems
achieved over 90% accuracy for the inspection of the
apples by representing features as histograms and by
applying multi-variate discriminant analysis technique
for classication.
Steinmetz et al. (1999) combined two non-destructive
sensors to predict the sugar content of apples. A spectrophotometer and computer vision system implemented online resulted in an accuracy of 78% for the
prediction of sugar content with a processing time of
3.5 s per fruit.
Analysis of the shape prole of apples using a Fourier
expansion procedure was performed by Paulus and
Schrevens (1999). These results were then compared to
Fourier coecients of proles from an existing shape
descriptor list. The rst and second components were
found to explain 92% of the total variance in the fruit.
Fourier and Fourier inverse transform were used to
describe Huanghua pear shape (Ying, Jing, Ma, Zhao,
& Jiang, 1999). The rst 16 harmonics of the Fourier
descriptor represented the primary shape of the pea. The
shape identication accuracy was 90% by applying the
Fourier descriptor in combination with the articial
neural network. Dewulf, Jancsok, Nicolai, De Roeck,
and Briassoulis (1999) used image-processing techniques
to obtain the geometrical model of a Conference pear in
a study using nite element modal analysis to determine
its rmness.
In the research by Singh and Delwiche (1994) a
monochromatic camera with a near infrared band pass
lter was used to capture images of stone fruit for detecting and identifying major defects. Correlation coefcients between machine predicted and manually
measured defects areas were 0.75 and 0.72 for bruise and
scar, respectively. The detection of split pits in peaches
was investigated using X-ray technology in association
with computer vision (Han et al., 1992). A detection
algorithm was developed based on a threshold equation
and tested on 198 peaches with 98% identied correctly.
Kim and Schatzki (2000) investigated the use of 2-D
X-ray imaging to detect internal water core damage in
apples. A total of eight features were extracted from Xray scanned apple images and classied using a neural
network to categorise the apple samples. Apples were
11
functional properties continues to grow. Present inspections are tedious and subjective based on empirical
and sensory assessments hence the use of computer
vision has been implemented for these tasks. Trials by
Wang and Sun (2001, 2002a, 2002b, 2002c) investigated
meltability and browning properties of cheddar and
Mozzarella cheeses under dierent cooking condition
and dierent sizes of samples using machine vision.
Cheese shred dimensions were determined from skeletonised images using syntactic networks in a study by Ni
and Gunasekaran (1995). This technique was successful
in recognising individual shred when two were touching
or overlapping and results compared well with manual
measurements.
Topping type, percentage and distribution are the
factors which inuence the attractive appearance and
separate many dierent varieties of pizza available
commercially. At present inspection of these quality
attributes is performed manually, however Sun (2000)
investigated the use of computer vision for the analysis
of these features. A new region based segmentation
technique was developed and an accuracy of 90% was
found when topping exposure and topping evenness
were examined. Fig. 7 shows an example of comparison
between an original pizza image and the corresponding
segmented image. In other trials computer vision and
fuzzy logic were used for the classication of pizza bases,
sauce spread distribution and topping evenness for determining acceptable and defective quality samples.
Yin and Panigrahi (1997) also used computer vision
to evaluate the internal texture of French fries. A set of
150 sample images were categorised into normal and
hollow classes by three algorithms. The co-occurrence
matrix gave the best results with 100% accuracy.
3.6. Grain
Cereal quality requirements dier with respect to the
end users such as the preparation of varieties of bread,
cakes, cookies and pasta products. The current visual
classication procedure is demanding, even for trained
inspectors because of the wide variation in visual
Fig. 7. Pizza images: (a) original image and (b) segmented image.
12
13
Table 3
Throughput of selected online applications of computer vision
Area of use
Speed/processing time
Accuracy (%)
Reference
1 sample/s
0.21 m/s conveyor
10,000/ h
2.2 s/sample
10/s
3000/min
3.5 s/fruit
66 nuts/s
60,000/h
90
95
99
94
78
81
4. Conclusions
This review presents the recent developments and applications of image analysis in the food industry, the basic
concepts and technologies associated with computer vision. Image processing is recognised as being the core of
computer vision with the development of more ecient
algorithms assisting in the greater implementation of this
technique. The automated, objective, rapid and hygienic
inspection of diverse raw and processed foods can be
achieved by the use of computer vision systems.
Computer vision has the potential to become a vital
component of automated food processing operations as
increased computer capabilities and greater processing
speed of algorithms are continually developing to meet
the necessary online speeds. The exibility and non-destructive nature of this technique also help to maintain its
attractiveness for application in the food industry. Thus
continued development of computer vision techniques
such as X-ray, 3-D and colour vision will ensue higher
implementation and uptake of this technology to meet
the ever expanding requirements of the food industry.
References
Abdullah, Z. M., Aziz, A. S., & Dos-Mohamed, A. M. (2000). Quality
inspection of bakery products using a colour-based machine vision
system. Journal of Food Quality, 23(1), 3950.
Anon (1995). Focus on container inspection. International Bottler and
Packaging, 69(1), 2231.
Bachelor, B. G. (1985). Lighting and viewing techniques in automated
visual inspection. Bedford, UK: IFS Publication Ltd.
Ballard, D. A., & Brown, C. M. (1982). Computer vision. Englewood
Clis, NJ, USA: Prentice-Hall.
14
Hayashi, S., Kanuma, T., Ganno K., & Sakaue, O. (1998). Cabbage
head recognition and size estimation for development of a selective
harvester. In 1998 ASAE Annual International Meeting, Paper No.
983042. St. Joseph, Michigan, USA: ASAE.
He, D. J., Yang, Q., Xue, S. P., & Geng, N. (1998). Computer vision
for colour sorting of fresh fruits. Transactions of the Chinese
Society of Agricultural Engineering, 14(3), 202205.
Heinemann, P. H., Hughes, R., Morrow, C. T., Sommer, H. J.,
Beelman, R. B., & Wuest, P. J. (1994). Grading of mushrooms
using a machine vision system. Transactions of the ASAE, 37(5),
16711677.
Heinemann, P. H., Varghese, Z. A., Morrow, C. T., Sommer, H. J.,
III, & Crassweller, R. M. (1995). Machine vision inspection of
Golden Delicious apples. Transactions of the ASAE, 11(6), 901
906.
Howarth, M. S., & Searcy, S. W. (1992). Inspection of fresh carrots by
machine vision. In Food Processing Automation II Proceedings of
the 1992 Conference. St. Joseph, Michigan, USA: ASAE.
Jamieson, V. (2002). Physics raises food standards. Physics World, 1,
2122.
Kim, S., & Schatzki, T. (2001). Detection of pinholes in almonds
through X-ray imaging. Transactions of the ASAE, 44(4), 997
1003.
Kim, S., & Schatzki, T. F. (2000). Apple watercore sorting using X-ray
imagery: I. Algorithm development. Transactions of the ASAE,
43(6), 16951702.
Kondo, N., Ahmada, U., Montaa, M., & Muraseb, H. (2000).
Machine vision based quality evaluation of Iyokan orange fruit
using neural networks. Computers and Electronics in Agriculture,
29(12), 135147.
Krutz, G. W., Gibson, H. G., Cassens, D. L., & Zhang, M. (2000).
Colour vision in forest and wood engineering. Landwards, 55, 29.
Laykin, S., Edan, Y., Alchanatis, V., Regev, R., Gross, F., Grinshpun,
J., Bar, E., Fallik, E., & Alkalai, S. (1999). Development of a
quality sorting machine vision and impact. In 1999 ASAE Annual
International Meeting, Paper No. 993144. St. Joseph, Michigan,
USA: ASAE.
Leemans, V., Magein, H., & Destain, M. F. (1998). Defects segmentation on Golden Delicious apples by using colour machine vision.
Computers and Electronics in Agriculture, 20, 117130.
Li, J., Tan, J., & Martz, F. A. (1997). Predicting beef tenderness
from image texture features. In 1997 ASAE Annual International Meeting, Paper No. 973124. St. Joseph, Michigan, USA:
ASAE.
Li, Q. Z., & Wang, M. H. (1999). Development and prospect of real
time fruit grading technique based on computer vision. Transactions of the Chinese Society of Agricultural Machinery, 30(6), 17.
Liu, J., & Paulsen, M. R. (1997). Corn whiteness measurement and
classication using machine vision. In 1997 ASAE Annual International Meeting, Paper No. 973045. St. Joseph, Michigan, USA:
ASAE.
Locht, P., Thomsen, K., & Mikkelsen, P. (1997). Full color image
analysis as a tool for quality control and process development in
the food industry. In 1997 ASAE Annual International Meeting,
Paper No. 973006. St. Joseph, Michigan, USA: ASAE.
Lu, J., Tan, J., Shatadal, P., & Gerrard, D. E. (2000). Evaluation of
pork color by using computer vision. Meat Science, 56, 5760.
Majumdar, S., & Jayas, D. S. (2000a). Classication of cereal grains
using machine vision: I. Morphology models. Transactions of the
ASAE, 43(6), 16691675.
Majumdar, S., & Jayas, D. S. (2000b). Classication of cereal grains
using machine vision: II. Colour models. Transactions of the ASAE,
43(6), 16771680.
Majumdar, S., & Jayas, D. S. (2000c). Classication of cereal grains
using machine vision: III. Texture models. Transactions of the
ASAE, 43(6), 16811687.
15
Tao, Y., Heinemann, P. H., Varghese, Z., Morrow, C. T., & Sommer,
H. J., III (1995a). Machine vision for colour inspection of potatoes
and apples. Transactions of the ASAE, 38(5), 15551561.
Tao, Y., Morrow, C. T., Heinemann, P. H., & Sommer, H. J. (1995b).
Fourier based separation techniques for shape grading of potatoes
using machine vision. Transactions of the ASAE, 38(3), 949957.
Tao, Y., & Wen, Z. (1999). An adaptive spherical image transform for
high-speed fruit defect detection. Transaction of the ASAE, 42(1),
241246.
Tarbell, K. A., & Reid, J. F. (1991). A computer vision system for
characterising corn growth and development. Transactions of the
ASAE, 34(5), 22452249.
Timmermans, A. J. M. (1998). Computer vision system for online
sorting of pot plants based on learning techniques. Acta Horticulturae, 421, 9198.
Tillett, R. D. (1990). Image analysis for agricultural processes Divsion
Note DN 1585. Silsoe Research Institute.
Tollner, E. W., Shahin, M. A., Maw, B. W., Gitaitis, R. D., &
Summer, D. R. (1999). Classication of onions based on internal
defects using imaging processing and neural network techniques. In
1999 ASAE Annual International Meeting, Paper No. 993165. St.
Joseph, Michigan, USA: ASAE.
Uthu, H. (2000). Application of the feature selection method to
discriminate digitised wheat varieties. Journal of Food Engineering,
46(3), 211216.
Van Loon, P. C. C. (1996). Het bepalen van het ontwikkelingsstadium
bij dechampignon met computer beeldanalyse. Champignoncultuur,
40(9), 347353.
Vzhany
o, T., & Felfoldi, J. (2000). Enhancing colour dierences in
images of diseased mushrooms. Computers and Electronics in
Agriculture, 26, 187198.
Vzhany
o, T., Tillett, R. D., (1998). Analysis of mushroom spectral
characteristics. Proceedings of AgEng98 International Conference
on Agricultural Engineering, Paper No. 98-F08019981-7, Oslo, 24
27 August 1998.
Wallin, P., & Haycock, P. (1998). Foreign body prevention, detection
and control. London: Blackie Academic & Professional.
Wan, Y. N., Lin, C. M., & Chiou, J. F. (2000). Adaptive classication
method for an automatic grain quality inspection system using
machine vision and neural network. In 2000 ASAE Annual
International Meeting, Paper No. 003094. St. Joseph, Michigan,
USA: ASAE.
Wang, H.-H., & Sun, D.-W. (2001). Evaluation of the functional
properties of cheddar cheese using a computer vision method.
Journal of Food Engineering, 49(1), 4751.
Wang, H.-H., & Sun, D.-W. (2002a). Correlation between cheese
meltability determined with a computer vision method and with
Arnott and Schreiber. Journal of Food Science, 67(2), 745749.
Wang, H.-H., & Sun, D.-W. (2002b). Assessment of cheese browning
aected by baking conditions using computer vision. Journal of
Food Engineering, 56(4), 339345.
Wang, H.-H., & Sun, D.-W. (2002c). Melting characteristics of cheese:
analysis of the eects of cheese dimensions using image processing
techniques. Journal of Food Engineering, 52(3), 279284.
Whelan, P. (1991). Quality vision. Technology Ireland, 23(2), 23.
Wills, R. T., Graham, D., Mc Glasson, W. B., & Joyce, D. (1998).
Postharvest: an introduction to the physiology and handling of
fruits, vegetables and ornamentals (4th ed.). Sydney: UNSW Press
Ltd.
Wooten, J. R., White, J. G., Thomasson, J. A., & Thompson, P. G.
(2000). Yield and quality monitor for sweet potato with machine
vision. In 2000 ASAE Annual International Meeting, Paper No.
001123. St. Joseph, Michigan, USA: ASAE.
Yang, Q. (1993). Classication of apple surface features using machine
and neural networks. Computers and Electronics in Agriculture, 9,
112.
16