See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/225328355

Color image based sorter for separating red and white wheat

Article in Sensing and Instrumentation for Food Quality and Safety · December 2008
DOI: 10.1007/s11694-008-9062-0

CITATIONS READS

38 4,350

3 authors:

T.C. Pearson D.L. Brabec

Amgen United States Department of Agriculture
92 PUBLICATIONS 2,277 CITATIONS 38 PUBLICATIONS 657 CITATIONS

SEE PROFILE SEE PROFILE

Scott Haley
Colorado State University
197 PUBLICATIONS 6,076 CITATIONS

SEE PROFILE

All content following this page was uploaded by Scott Haley on 17 March 2015.

The user has requested enhancement of the downloaded file.

Sens. & Instrumen. Food Qual. (2008) 2:280–288
DOI 10.1007/s11694-008-9062-0

ORIGINAL PAPER

Color image based sorter for separating red and white wheat
Tom Pearson Æ Dan Brabec Æ Scott Haley

Received: 1 July 2008 / Accepted: 12 September 2008 / Published online: 3 October 2008
! Agricultural Research Service 2008

Abstract A simple imaging system was developed to Keywords Image histogram ! White wheat ! Red wheat !
inspect and sort wheat samples and other grains at mod- Fusarium ! Scab
erate feed-rates (30 kernels/s or 3.5 kg wheat/h). A single
camera captured color images of three sides of each kernel
by using mirrors, and the images were processed using a Introduction
personal computer (PC). Real time image acquisition and
processing was enabled on an ordinary PC under Windows The use of imaging for high-speed sorting of agricultural
XP operating system using the IEEE 1394 data transfer products has been limited, in part, by the high cost of
protocol, DirectX application software, and dual-core imaging and processing hardware for real-time applica-
computer processor. Image acquisition and transfer to the tions. Traditionally, images are captured using a ‘‘frame
PC required approximately 17 ms per kernel, and an grabber’’ card hosted inside a PC. The frame grabber may
additional 1.5 ms was required for image processing. After have some real-time digital signal processing capabilities,
classification, the computer could output a signal from the but all outputs and decision making must be processed by
parallel port to activate an air valve to divert (sort) kernels the PC. Thus, for high-speed image-based sorting appli-
into a secondary container. Hard red and hard white wheat cations, the PC operating system must have the capacity to
kernels were used in this study to test and demonstrate operate in real time, using critical timings as short as two
sorter capability. Simple image statistics and histograms milliseconds. This adds to the cost and complexity of the
were used as features. Discriminant analysis was per- overall system. A high-speed image-based sorting device
formed with one, two, or three features to demonstrate for almonds using a PC running under the DOS operating
classification improvements with increased numbers of system was developed [10] to save the cost of a real time
features. The sorter was able to separate hard red kernels operating system and computer. Using a frame grabber and
from hard white kernels with 95 to 99% accuracy, two linescan cameras, the system was able to inspect
depending on the wheat varieties, feed-rate, and number of almonds at a rate of 40 nuts/s. However, the cost of the
classification features. The system is an economical and parts alone for this system was over $10,000 and the
useful instrument for sorting wheat and other grains with images were mono-chromatic.
high accuracy. Most commercial high-speed sorting machines used for
agricultural products either have no spatial resolution or do
not fully utilize the spatial resolution produced by their
sensors. These devices perform limited signal processing in
T. Pearson (&) ! D. Brabec
USDA-ARS-Grain Marketing and Production Research Center, hardware for robustness and high speed, rather than on a
1515 College Ave., Manhattan, KS 66502, USA micro-processor or PC. For those sorters with some spatial
e-mail: thomas.pearson@ars.usda.gov resolution (*0.2 mm/pixel), the only image processing
performed is thresholding and pixel counting. Conse-
S. Haley
Soil and Crop Sciences Department, Colorado State University, quently, for many products, certain defects are difficult to
Fort Collins, CO 80523, USA detect and remove using currently available commercial

123
Color image based sorter 281

sorters. Shriveled and Fusarium head blight (scab) dam- provide enhanced video and audio handling capability for
aged wheat kernels are a case in point. The efficacy of computer games with advanced high-resolution graphics.
using a limited spatial resolution (*0.5 mm) commercial However, capabilities for cameras connected by IEEE 1394
dual-band (one near infrared (NIR), one visible) sorter for cables are also available, and can be exploited in real-time
removal of scab-damaged kernels has been studied [4]. imaging applications on Microsoft Windows operating
Only 50% of the scab-damaged kernels were removed, systems, including Windows XP and Vista.
while about 5% of the undamaged kernels were also There have been many developments relevant to the
rejected. Preliminary studies have shown that the use of inspection of agricultural products using imaging, such as
simple histograms of moderate resolution images enables inspection of apples [2], rice [5], and wheat [14]. However,
scab-damaged kernels to be distinguished from sound few of these developments are able to be implemented as
kernels with over 90% accuracy [9]. high-speed sorting applications at an economically feasible
Commercial color sorters can have throughputs as high as cost. Few developments where sorter throughput and cost
1200 kg/h for wheat. This is accomplished by running as are considered have been reported in the literature. A
many as 80 channels in parallel within one machine. Thus, vision-based sorting machine system that identified stain-
each channel has a throughput rate of approximately 15 kg/ ing patterns on pistachio nuts, where these indicated
h, or about 126 kernels/s for wheat. While sorter throughput aflatoxin contamination has been developed [8]. This sys-
of this magnitude is needed to meet processing plant tem performed some image processing and maintained a
requirements, this is often more than is needed for smaller sorting rate of 40 nuts per second per channel. In this
scale operations such as seed cleaning and selection for system, the spatial resolution was very low since the
plant breeders. With wheat breeding, color sorting would be maximum data transfer rate between camera and processor
useful in early generations, when segregating breeding used was low (250 kHz over an 8-bit bus). This allowed the
populations, or in later breeding generations, where breed- processing to be performed on a low cost digital signal
ing lines contain mixed color classes due to natural processor, without the need for a PC. However, the low
heterogeneity or admixture. In the case of early generation spatial resolution of the device reduced its applicability to
populations, wheat breeders would typically have only small other sorting problems. More recently, an image-based
quantities of grain available for sorting, usually less than sorter based on transmitted light in order to detect almonds
5 kg. In this situation, lower throughput with associated with shell fragments embedded into the kernel was devel-
lower machine cost and higher accuracy is more desirable. oped [10]. Another low-cost machine vision system was
Charge coupled devices (CCD) have traditionally been developed to inspect meat for foreign objects using color
the most common image sensors used. While these devices [11]; however, this system does not perform sorting. A
produce high quality images at very high resolution, they color imaging system was developed for inspection and
require significant support electronics [16]. However, this sorting of grains at rates of over 30 kernels/s [13]. How-
need for elaborate support electronics for image sensors ever, this system only images one side of the kernel.
has recently been alleviated by the now widespread pro- Reflectance color and monochromatic transmittance
duction of complementary metal–oxide–semiconductor imaging algorithms were developed to aid inspection of
(CMOS) image sensors [12]. Production of CMOS devices small grains using a commercial imaging device [15]. This
is less expensive than that required for charge coupled device was able to inspect single kernels at rates
devices (CCD), and the required support electronics are approaching 20 kernels/s, and could perform moderate
integrated onto the sensor chip. Additionally, support image processing at this inspection rate. However, this
electronics, such as external triggering inputs and analog- machine has a high cost and does not perform sorting. A
to-digital converters, have been added to low-cost cameras dual-wavelength machine vision system for inspection of
that transfer images from camera to PC through IEEE 1394 chicken carcasses was developed [3]. This device utilizes a
or Universal Serial Bus (USB) ports. Of these two transfer high degree of image processing through neural networks
protocols, IEEE 1394 is preferred for high-speed, real-time, and can inspect chickens at rates over two per second. This
applications as it supports true direct memory transfers device uses traditional CCD cameras, frame grabber
without the need for the computer processors or operating boards, and a PC for processing images. Finally, robust,
system to manage the data transfer [1]. In other words, the low-cost color machine vision systems for industrial
camera is able to load the images into PC memory on its inspection (e.g. Banner Engineering Corporation) are now
own, without additional processing by the PC. becoming available to distinguish objects with large color
Finally, direct memory transfers and image manipula- differences. These devices require a large color difference
tion are enabled in real time on Microsoft Windows to be distinguished and do not have the throughput rate
operating systems by an application programming interface (over 20 objects/s) to be practical for inspection of most
called DirectX [6]. This interface was initially developed to agricultural products, such as grains.

123
282 T. Pearson et al.

The objectives of this research were to develop a labo- hardware, external from the computer. As such, the kernel
ratory-scale sorting device which improved the uniformity position is very consistent at the moment of image acqui-
of breeder’s wheat samples. Secondly, this research studied sition. The position of the leading edge of the kernel varies
the classification performance of this sorting device when only 1 mm from image to image.
using a single feature (as used by commercial color sorters) Images from the backside of each kernel (opposite of the
versus multiple image features. While the feed-rate of a camera) are reflected back to the camera with two 45"
single channel imaging system cannot match those of mirrors (RA EN AL 20 9 28 mm, Edmund Optics, Bar-
commercial color sorters, the system could find uses where rington, NJ). This allows capture of the entire perimeter of
accuracy is more of a factor than throughput. each kernel with one image and one camera, simplifying
the system design and lowering the overall cost.
The kernels were illuminated with two small 35 W
Materials and methods reflector lamps (12V35 W/N/FG––MR8, Ushio, Japan).
The shutter speed of the camera was set at a constant
Image and sorting system hardware 1/6875 s and a gain of 500 (the range for this camera was
1–1,000). White balance was set using a flat piece of white
Figure 1 depicts a machine vision system linked to a per- Teflon placed over the mirrors. Before performing any
sonal computer. The camera (DFK21BF04, The Imaging sorting, the camera captured a blank image with no kernel
Souce, Sommerstrabe, Germany) is a color camera utilizing as well as a gray image using a standard 18% gray reflector
a Red-Green-Blue (RGB) Bayer pattern filter over a total of card (421869, X-rite Inc. Grand Rapids, MI).
640 9 480 pixels. The camera transfers images to the The feeding system was two-staged, comprising a seed
computer via an IEEE 1394 cable in direct memory access metering device and a vibratory feeder. An auger style
(DMA) mode using the DirectX application programming metering device (3010FC/B, Seedburo Equipment Co,
interface (DirectX version 9.0, Microsoft Corp. Redmond, Chicago, IL) metered wheat onto the feeder’s v-trough. The
WA). Images are transferred in raw format without color vibratory feeder (F-TO-C, FMS-Syntron, Homer City, PA)
interpolation, in order to reduce the amount of data trans- advanced kernels onto a slide which also had a v-shape. The
mitted from the camera by one-third. The camera uses a slide was 15 cm long and fabricated out of aluminum bar
25 mm C-mount lens (M2514-MP, Computar, Japan). stock. Kernels accelerated and separated on the slide so that
A photo-electric switch (D21DAB6FP05, Banner Engi- only one kernel would appear in each image.
neering, Minneapolis, MN) was used to externally trigger The feed-rates were varied by changing the motor on the
the camera, allowing image acquisition to be performed in metering device. The 15 RPM (Bodine KYC-24T4), 22

Fig. 1 Schematic of automated

imaging device

123
Color image based sorter 283

RPM (Bodine KCI-24T4), and 30 RPM (Bodine KYC- approximately 128 for the red, green and blue color
24T3) motors delivered 30, 60, and 90 g/min which cor- channels. Due to the inconsistent light coverage from the
responded to 15, 30, and 45 kernels/s, respectively. lamps and the wheat kernel’s geometry, the edges of each
After image capture, analysis, and classification, if a image were darker than the center. To equalize the lighting
kernel was classified as white, an air valve (35A–AAA– distribution and help make a calibration repeatable over
DDBA–1BA, Mac Valves, Inc., Wixom, Mich.) was acti- time, the intensity of every kernel pixel from each corre-
vated to divert these kernels from the red kernels into a sponding red, green, and blue image was divided by the
separate hopper. This valve was chosen for its fast response intensity of the corresponding gray card image pixel, then
time and longevity. While the manufacturer does not multiplied by 128.
specify a life expectancy for this valve, these valves have Three intensity histograms were extracted as features for
been used over several years on other sorting machines in classification. Each histogram represented the scaled red,
processing plants without failing. A wide air nozzle green, or blue pixel intensities from the wheat kernel
(31875K41, McMaster-Carr Co. Elmhurst, IL) was used to image. The bin type histograms were converted to cumu-
ensure proper rejection. The air valve was activated by lative histograms and then to percent kernel pixels by
outputting a signal from the computer’s parallel port which scaling with the total number of pixels within the kernel.
was used to trigger a timer circuit to control the air blast This normalized the histograms for varying kernel sizes.
duration. This external circuit allowed the computer to be Each histogram contained 64 bin levels representing
available to process another kernel while the air valve was intensity levels from low to high. The three histograms
opened. It was found, by trial and error, that an air blast produced ‘3 9 64’ or 192 potential image features. In
duration of 5 ms was sufficient to remove 99% of the addition, the mean and standard deviation of the red, green,
kernels when fed at a rate of 15 kernels/s. and blue intensities were computed, resulting in six more
features. Thus, from the three histograms, means, and
Image processing and analysis standard deviation pairs, a total of 198 features were
computed for each kernel image.
Each image had 480 rows and 640 columns of pixels.
Rather than color interpolating the images, the red, green, Wheat sorter software and training
and blue pixels from the Bayer filter pattern (alternate lines
had blue-green and red-green pixels) were separated to Prior to sorting wheat samples, the image sorter software
form three images of 240 9 320 pixels in size. A kernel’s was trained using data from 200 typical red kernels and 200
length would span 100–150 pixels. The leading edge of the typical white kernels. Linear discriminant analysis was used
kernel would vary by approximately 30 pixels from the top as the classification method using a small subset from the
of the image. Since the kernel was consistently in the pool of image features. The data was randomly divided into
center of the image frame, the first 20 rows and last 20 equal training and validation sets. The training set was used
rows of each 240 9 320 image were not processed to to compute the group means and covariances of the feature
reduce image processing time. subsets. The validation set was used to test the classifica-
The first image processing step was to subtract the blank tions. An exhaustive search was used to test all possible
image, acquired prior to sorting, from the new image one-, two-, and three-feature combinations. The best dis-
containing the kernel. Only the red image pixels were criminating single-, two-, and three-feature combinations
subtracted in this step as the red pixels were less prone to were selected to be used for sorting. The whole training
noise since the sensor sensitivity is greater at light corre- process took less than 5 min to perform per sample pair.
sponding to red wavelengths. When the intensity between
the red pixels of the blank image and the red pixels from Wheat sorter tests
the new image was greater than 10, these pixels were
considered to be part of the kernel. If the difference was The wheat sorter has two distinct functions. The camera
less than or equal to 10, the red pixels and adjacent blue collects images and discriminates with some level of
and green pixels were set to zero and considered back- classification efficiency. Secondly, the system removes
ground. This enabled nearly flawless segmentation of the moving kernels with a blast of air with some level of
kernel from the background. The threshold settings were ejection efficiency. The overall efficiency of the sorting
determined by trial and error before experiments began but system is a combination of these two efficiencies. For
were kept the same for every subsequent experiment. instance, if the images are discriminated with 95% accu-
The pixel intensities of the red, green, and blue images racy and the ejectors can remove 95% of the kernels
were then scaled using the image of the 18% gray card. The passing by, then the overall efficiency of the system is
average intensity values for the gray card were 95% 9 95%, or 90%.

123
284 T. Pearson et al.

The image classification efficiency was affected by the The sorter software was trained and tested separately
number of features used in the classification (1, 2, or 3) and with each reference wheat pair. Then, a sample was mixed
the contrast between the red and white samples. The from the reference wheat which contained 95 g of hard red
ejection efficiency was affected by the sample feed-rate, wheat and 5 g of hard white wheat. The sorter was set to
mixture level of red and white kernels, and computer reject the white kernels with the air valve since they were
process timing. the smaller portion. Sorting tests were performed with the
The sorter was tested with two reference red/white best three feature combinations for each reference wheat
wheat pairs. One pair had good contrast between the red pair.
and white kernels and was easy to distinguish by the human The sorter was tested at rates of 1, 15, 30, and 45 ker-
eye. The second pair had less contrast and thus was more nels/s to determine if higher throughput rates caused more
difficult to distinguish. This was due to some of the red errors due to kernels touching each other as they are
wheat kernels being lightened from weathering, in addition imaged and rejected. Each sample was run through the
to the white kernels having a more beige, rather than white, machine 10 times at each of the four sorter throughput
appearance. The white kernels from the difficult sample rates. The feed rate was adjusted by changing the motors
were marked with an invisible ink (U.V. Security Marker on the auger metering feeder, as discussed earlier. The
Ink 57-52-E1, Blacklight World, Cub Run, KY) that fluo- average and standard deviation of the sorting accuracies
resces under UV light but is otherwise invisible. After were recorded. Additionally, data were collected to quan-
sorter testing with these kernels, the samples were tify the time intervals between kernels for the different feed
inspected under UV light to determine sorter accuracy. rates of 15, 30, and 45 kernels/s.
Images of kernels from the two samples are shown in System sorting and lamp stability were tested over five
Fig. 2. consecutive days. Before testing the sorter each day, a

Fig. 2 Images of red (left) and

white (right) kernels from the
easy sample (top row) and
difficult sample (bottom row).
Note that the contrast between
the red and white kernels is
greater for the easy sample than
the difficult sample. Also note
that the many red kernels from
the difficult sample have a
yellow hue, these are called
‘‘yellow berries’’

123
Color image based sorter 285

new image of the 18% gray card and a blank image were obtained with the computer using the AMD Athlon 64 X2
acquired for kernel segmentation and pixel intensity 6000? dual-core processor. The timings from the com-
scaling. The sorter was tested at the 30 kernels/s rate puter using the Intel Core Duo 6600 processor were
only. Two mixed wheat samples were run through the slightly slower than, but just as consistent as the AMD
machine ten times each day and the average and standard dual-core processor. The PC with the Intel processor may
deviation of the sorting accuracies were recorded each have been slightly slower due to the slower memory speed
day. It is important to note that the sorter was trained only in this computer. After receipt of a trigger signal, the
once before all testing began. The lamps on the sorter camera requires 16.7 ms to acquire and transfer an image
remained on for the entire week so there would not be into the computer. The average time of 17.2 ms for a PC to
any error due to lamp warm up. One hundred kernels recognize that a new image has been transferred indicates
from the two red and white samples were used for this that the PC required 0.5 ms to recognize it after the image
test. was completely loaded. The timings from the computer
To test the image collection and processing times, having the single-core AMD Athlon 3000? processor
software was written to output a pulse from the parallel were, on average, 3.7 ms slower. More importantly, how-
port upon receipt of a new image and after all image pro- ever, these timings were much more variable, with a
cessing was completed. The times between pulses was then standard deviation of 3.5 ms. Assuming a normal distri-
measured on a digital oscilloscope (196C, Fluke Inc., bution, the PC with the AMD Athlon 64 X2 6000? dual-
Everett, WA). Three different computers were tested, all core processor would be able to process over 95% of the
using the Windows XP operating system. One computer kernels within a 2.4 ms window, while the single-core
contained a single-core Advanced Micro Devices (AMD) AMD Athlon 3000? processor requires a window of
Athlon 3000? processor with 2 GB of 400 MHz DDR 14.0 ms. Given that the kernels are traveling at approxi-
SDRAM (XC Cube EX761, AOpen Inc., Milpitas, CA). mately 2 m/s, this means that the kernel position
The second computer contained a dual-core AMD X2 uncertainty from the single-core processor is 28 mm,
Athlon 6000? processor with 2 GB of 800 MHz DDR2 compared to only 4.8 mm for the dual-core processor.
SDRAM (Pundit P1-AH2, ASUS Computer International, Having a lower time uncertainty in this sorting system
Freemont, CA). The third computer used an Intel Core Duo allows the air valve blast duration to be shorter and reduces
6600 processor with 2 GB of 667 MHz DDR2 SDRAM the likelihood of rejecting two kernels with one air blast.
(Optiplex 745, Dell Computer, Round Rock, TX). The For this reason, in all further tests, the computer with the
image transfer and processing times from 30 kernels were dual-core AMD Athlon 64 X2 6000? processor was used.
recorded for each computer. Use of this computer allowed the air valve blast duration to
be set at 5 ms, which enabled the sorting throughput to be
relatively high. Windows XP and Vista operating systems
Results have traditionally not been considered to be capable of real
time operatations. The combination of image triggering in
Image transfer and processing timings hardware, DMA image transfer to a computer via IEEE
1394 protocol, DirectX API and dual core processors
As shown in Table 1, the dual-core processors have faster appear to give adequate real time capability for this
image processing times and, more importantly, much more application. Since this software and hardware has lower
consistent times than the single-core processor. The fastest cost and is much easier to program than traditional real
average time to recognize a new image and process it was time hardware and software, this finding should stimulate
18.3 ms with a standard deviation of 0.6 ms (17.2 ms to more real time applications with the configuration used in
recognize the image and 1.1 ms to process it). This was this study.

Table 1 Average time to recognize a new image after kernel breaks the trigger, and average image processing times in milliseconds for the three
computers tested
ASUS––AMD Athalon Aopen––AMD Athalon Dell Optiplex 745––Intel Core Duo
64 X2 6000? (single core) 3000? 6600 2.4Ghz
Time to recognize Image processing Time to recognize Image processing Time to recognize Image processing
new image time new image time new image time

Average 17.2 1.1 20.3 1.7 17.3 1.6

Std dev 0.5 0.1 3.4 0.1 0.5 0.1

123
286 T. Pearson et al.

Feature selection and classification less than 72, and the standard deviation of the red pixel
intensities. The red kernels had higher counts of low
Validation set classification accuracies for both the ‘‘easy’’ intensity blue pixels, higher counts of red pixels with low
and ‘‘difficult’’ samples are shown in Table 2. For the easy intensities, and higher standard deviations of red pixel
sample, the best feature subset for distinguishing hard red intensities. While the red kernels in this sample also had
wheat from hard white wheat kernels was comprised of higher percentages of high-intensity red pixels than white
three features: the mean of the green pixels; the percentage kernels, similar to the easy sample, the classification in this
of blue pixels with an intensity less than 112; and the per- case is based more on the darker portions of the red ker-
centage of red pixels having an intensity less than 176. nels. Many of the red kernels in this sample contained
White wheat kernels tended to have higher values for the some darker areas as well as very light areas from weath-
mean green pixels, indicating an overall brighter appear- ering. This causes the standard deviation of the pixel
ance. Although the white kernels were brighter overall, they intensities of the red kernels to be high, whereas the white
had a higher percentage of red pixels with a low-to-mod- kernels tend to have more consistent intensity levels across
erate intensity (176) than the red kernels did. Conversely, their entire surface.
the red kernels had higher percentages of blue pixels with Discrete histogram values and average pixel values
low-to-moderate intensities (less than 112). Since red colors could be approximated by single sensors coupled with the
absorb blue light, it is not surprising that a higher percent- appropriate color filter, such as those used in commercial
age of red kernels had lower blue pixel intensities than the color sorters; however, the variance of pixel intensities
white kernels. Validation set accuracy indicated that that requires an image and simple image processing to compute.
accuracy for red wheat should be 98%, while the accuracy Even though this is a simple feature to extract from an
for white wheat should be 99%, for an average accuracy of image, it improves classification accuracy of kernels in the
98.5%. When only one or two features are selected, the ‘‘difficult’’ sample. When the pixel standard deviations are
average accuracy reduces to 94% for two features and 88% removed from the pool of features, the best three-feature
for one feature. The accuracy for one feature is an indication classification accuracy obtained was 93% for hard red
of what could be accomplished with a mono-chromatic wheat and 90% for hard white wheat for an average
sorting device such as those available from Satake USA, accuracy of 91.5%. Conversely, the average accuracy when
Inc, Sortex, etc. The effectiveness of a commercial mono- selection of the standard deviation of the red pixels was
chromatic color sorting machine for separating hard white allowed was 97% (Table 2).
wheat from hard red wheat has been studied [7]. The result
of this study was that, depending on the variety and amount Sorter testing
of weathering the red kernels incurred, the sorting machine
needed to reject 15 to 20% of all material in order to remove The accuracy of a sorting system is dependent on robust
80 to 90% of the red wheat from the white. The 88% image processing and classification as well as efficient
classification accuracy, when using only one image histo- mechanical feeding and sorting. When the feed rate
gram feature for kernels that have high contrast between increases, errors can occur due to kernels being too close to
them, is consistent with the results from [7]. one another. The camera requires approximately 17 ms to
While the ‘‘difficult’’ sample had similar classification capture and transfer an image to a PC. If kernels are sep-
accuracies as the ‘‘easy’’ sample, the features selected were arated by less than this time period, these kernels are not
different. The best average classification accuracy (97%) imaged correctly, or are missed entirely. Additionally,
for this set also required three features. These features kernels that are too close to another kernel can be diverted
were: the percentage of red pixels with an intensity less together with a single air blast. The air valve needs to be
than 64, the percentage of blue pixels having an intensity open for a minimum of 5 ms to account for variations of
the potential kernel position with respect to the air blast.
Testing of the sorter at feed-rates of 1, 15, 30, and
Table 2 Classification results from linear discriminant analysis on 45 kernels/s shows how accuracy decreased as throughput
the training sets of the ‘‘easy’’ and ‘‘difficult’’ samples
increased. Table 2 displays sorting results versus feed-rate.
Number of Accuracy for the Accuracy for the The accuracies for throughputs of 1 and 15 kernels/s are
features selected ‘‘easy’’ sample ‘‘difficult’’ sample
almost identical, and close to the accuracies achieved when
Red White Red White training. This indicates that kernels fed at a rate of
15 kernels/s are well-separated so that only one kernel is in
1 86 91 88 78
the field of view at a time. The time gaps at 15 kernels/s
2 92 96 92 85
averaged 67 ms, and the minimum time gap was 30 ms
3 98 99 97 97
(from a 100 kernel sample). When the throughput was

123
Color image based sorter 287

increased to 30 kernels/s, the accuracy dropped by an sorter can perform for at least a five day period with one
average of 1.2%. This slight drop in accuracy may be calibration. However, a new image of the 18% gray card
tolerable, because the feed-rate increased and the sample and new blank image are acquired each day to scale the
processing time was cut in half. The accuracies at a image. Even though the discriminant function can be re-
throughput of 45 kernels/s dropped an average of 12.0%. trained within 5 min, it is important for the sorter per-
The increased feed-rates affected accuracy for white formance to be repeatable over a reasonably long time
wheat more than red wheat since the higher feed rate span (Table 3).
caused some kernels to be ignored. The kernel separation
times at the 45 kernels/s feed-rate ranged from 12 ms to
44 ms. The average time gap was 22 ms. Most of the Conclusion
kernels had separation times of 17 ms to 28 ms. However,
15% were separated by less than 17 ms, which is the The image sorting system has low cost and a moderately
minimum time necessary for proper imaging. In contrast, high throughput of 30 kernels/s. Real-time sorting is
when kernels were fed at a rate of 30 kernels/s, only 2% enabled by the use of IEEE 1394 direct memory access
were separated by less than 17 ms. image transfer, DirectX, and dual-core processors available
The image processing for this application took less than in low cost PCs. The sorter appears to be nearly as accurate
1.5 ms while the image transfer time required nearly at separating wheat samples with low contrast differences
17 ms. If the computer had access to the image data before as it is with samples having high contrast. With image
the entire image was transferred, this could possibly allow processing, three features were extracted and showed to
enough time for more elaborate image processing to be more accurately discriminate red and white wheat samples
performed without sacrificing any throughput. This will be than a single feature, as is performed with commercial
the basis of future study and development. color sorters. In particular, standard deviation features
Two wheat samples were repeatedly sorted over a five provided unique and helpful information with kernels of
day period. As can be seen from Fig. 3, there is no sig- more variable color.
nificant trend over that period. This indicates that the Sorting at throughputs of 15 kernels/s and below
showed that the actual sorting accuracy is very near the
96.5
validation set accuracy. Sorting at throughputs of 30 ker-
easy sample nels/s showed reduced sorting accuracies by an average of
difficult sample
96.0 only 1.2%. Feed-rates of 45 kernels/s reduced accuracies
average sorting accuracy (%)

by an average of 12%.
95.5 When operating the sorter at a throughput of 30 kernels/s,
the accuracy for the two samples studied was, on average,
95.0 95.5% and 94.7% for the ‘‘easy’’ and ‘‘difficult’’ samples,
respectively. These sorting accuracies are over 10% greater
94.5
than what can be expected from commercial color sorters.
Given the low cost, accuracy, and moderate throughputs of
94.0
the image sorting system, this device should prove to be
practical where accuracy is more important than through-
93.5
0 1 2 3 4 5 6 put, such as with wheat breeding applications where it may
day be desirable to sort red and white wheat from mixed
Fig. 3 Average sorting accuracies for the easy and difficult sample
samples. Other grain applications will be studied in the
over a five day period and feed rate of 30 kernels/s. Note that there is future, including the sorting of scab damaged wheat and
no significant trend over the five day period fungal damaged popcorn.

Table 3 Average and standard deviations from sorting tests at various throughput rates for the two samples
Throughput rate (kernels/s) Easy sample Difficult sample
Accuracy for white wheat Accuracy for red wheat Accuracy for white wheat Accuracy for red wheat

1 98.1 (0.1) 95.9 (0.2) 96.0 (0.2) 95.1 (0.2)

15 98.0 (0.1) 95.8 (0.2) 95.9 (0.3) 94.9 (0.3)
30 96.1 (0.2) 94.9 (0.5) 95.5 (0.3) 93.9 (0.5)
45 90.1 (1.6) 78.2 (1.9) 89.9 (1.8) 76.0 (2.2)

123
288 T. Pearson et al.

References 9. P.A. Kumar, S. Bal, Automatic unhulled rice grain crack detec-
tion by x-ray imaging. Trans. ASABE 50(5), 1907–1911 (2007)
1. T.C. Pearson, R. Young, Automated sorting of almonds with 10. N. Wang, N. Zhang, F.E. Dowell, T.C. Pearson, Determination of
embedded shell by laser transmittance imaging. Appl. Eng. Agric. durum wheat vitreousness using transmissive and reflective
18(5), 637–641 (2001) images. Trans. ASAE 48(1), 219–222 (2004)
2. S.R. Delwiche, T.C. Pearson, D.L. Brabec, High-speed optical 11. T.C. Pearson, Machine vision system for automated detection of
sorting of soft wheat for reduction of deoxynivalenol. Plant Dis. stained pistachio nuts. Lebensmittel-Wissenschaft Technologie
89(11), 1214–1219 (2005). doi:10.1094/PD-89-1214 29(3), 203–209 (1996). doi:10.1006/fstl.1996.0030
3. T.C. Pearson, in Low-cost bi-chromatic Image Sorting Device for 12. J. Stewart, in Computer Vision to Detect Foreign Objects (AS-
Grains (ASABE, St. Joseph, Mich, 2006), ASABE Paper No. ABE, St. Joseph, Mich, 2005), ASABE Paper No. 056015
063085 13. Y.-N. Wan, Kernel handling performance of an automatic grain
4. T. Yamada, CCD Image Sensors. in Image Sensors and Signal quality inspection system. Trans. ASAE 45(2), 369–377 (2002)
Processing for Digital Still Cameras (Taylor and Francis Group, 14. F. Xie, T.C. Pearson, F.E. Dowell, N. Zhang, Detecting vitreous
Boca Raton, Florida, 2006), pp. 95–142 wheat kernels using reflectance and transmittance image analysis.
5. I. Takayanagi, CMOS Image Sensors. in Image Sensors and Cereal. Chem. 81(4), 490–498 (2004). doi:10.1094/CCHEM.
Signal Processing For Digital Still Cameras (Taylor and Francis 2004.81.4.490
Group, Boca Raton, Florida, 2006), pp. 143–178 15. Y.R. Chen, K. Chao, W.R. Hruschka, On-line automated
6. A. Baker, J.J. Lozano, The Windows 2000 device driver book: a Inspection of Poultry Carcasses by Machine Vision. in Pro-
guide for programmers, 2nd edn. (Prentice Hall PTR, Upper ceedings of the World Congress of Computers in Agriculture and
Saddle river, NJ, 2001) Natural Resources. ASAE Publication Number 701P0301 (AS-
7. M. Linetsky, Programming Microsoft Direct Show (Wodware ABE, St. Joseph, Mich, 2002), pp. 78–85
publishing, Inc., Plano, TX, 2002) 16. M.C. Pasikatan, F.E. Dowell, Evaluation of a high-speed color
8. B.S. Bennedsen, D.L. Peterson, Identification of apple stem and sorter for segregation of red and white wheat. Appl. Eng. Agric.
calyx using unsupervised feature extraction. Trans. ASAE 47(3), 19(1), 71–76 (2003)
889–894 (2004)

123

View publication stats