Skip to main content

Lewis Griffin

University College London, Computer Science, Faculty Member

Followers

240

Following

52

Co-author

1

Public Views

Computational Vision Scientist
Phone: 07976 893011
Address: Computer Science
Malet Place Engineering Building
University College London
London WC1E 6BT

less

Harvard University

İbn Haldun Üniversitesi

School of the Art Institute of Chicago

The University of New South Wales

University College Cork

Kansas State University

Armando Marques-Guedes

UNL - New University of Lisbon

Macquarie University

Vivek Nityananda

Newcastle University

Oxford Brookes University

InterestsView All (19)

Uploads

Papers by Lewis Griffin

Transferring x-ray based automated threat detection between scanners with different energies and resolution

Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies

A significant obstacle to developing high performance Deep Learning algorithms for Automated Thre... more A significant obstacle to developing high performance Deep Learning algorithms for Automated Threat Detection (ATD) in security X-ray imagery, is the difficulty of obtaining large training datasets. In our previous work, we circumvented this problem for ATD in cargo containers, using Threat Image Projection and data augmentation. In this work, we investigate whether data scarcity for other modalities, such as parcels and baggage, can be ameliorated by transforming data from one domain so that it approximates the appearance of another. We present an ontology of ATD datasets to assess where transfer learning may be applied. We define frameworks for transfer at the training and testing stages, and compare the results for both methods against ATD where a common data source is used for training and testing. Our results show very poor transfer, which we attribute to the difficulty of accurately matching the blur and contrast characteristics of different scanners.

Automated detection of smuggled high-risk security threats using Deep Learning

7th International Conference on Imaging for Crime Detection and Prevention (ICDP 2016), 2016

The security infrastructure is ill-equipped to detect and deter the smuggling of non-explosive de... more The security infrastructure is ill-equipped to detect and deter the smuggling of non-explosive devices that enable terror attacks such as those recently perpetrated in western Europe. The detection of so-called "Small Metallic Threats" (SMTs) in cargo containers currently relies on statistical risk analysis, intelligence reports, and visual inspection of X-ray images by security officers. The latter is very slow and unreliable due to the difficulty of the task: objects potentially spanning less than 50 pixels have to be detected in images containing more than 2 million pixels against very complex and cluttered backgrounds. In this contribution, we demonstrate for the first time the use of Convolutional Neural Networks (CNNs), a type of Deep Learning, to automate the detection of SMTs in fullsize X-ray images of cargo containers. Novel approaches for dataset augmentation allowed to train CNNs from-scratch despite the scarcity of data available. We report fewer than 6% false alarms when detecting 90% SMTs synthetically concealed in streamof-commerce images, which corresponds to an improvement of over an order of magnitude over conventional approaches such as Bag-of-Words (BoWs). The proposed scheme offers potentially superhuman performance for a fraction of the time it would take for a security officers to carry out visual inspection (processing time is approximately 3.5s per container image).

Reconciling the statistics of spectral reflectance and colour

PLOS ONE, 2019

The spectral reflectance function of a surface specifies the fraction of the illumination reflect... more The spectral reflectance function of a surface specifies the fraction of the illumination reflected by it at each wavelength. Jointly with the illumination spectral density, this function determines the apparent colour of the surface. Models for the distribution of spectral reflectance functions in the natural environment are considered. The realism of the models is assessed in terms of the individual reflectance functions they generate, and in terms of the overall distribution of colours which they give rise to. Both realism assessments are made in comparison to empirical datasets. Previously described models (PCA-and fourier-based) of reflectance function statistics are evaluated, as are improved versions; and also a novel model, which synthesizes reflectance functions as a sum of sigmoid functions. Key model features for realism are identified. The new sigmoid-sum model is shown to be the most realistic, generating reflectance functions that are hard to distinguish from real ones, and accounting for the majority of colours found in natural images with the exception of an abundance of vegetation green and sky blue.

Categorical colour geometry

PLOS ONE, 2019

Ordinary language users group colours into categories that they refer to by a name e.g. pale gree... more Ordinary language users group colours into categories that they refer to by a name e.g. pale green. Data on the colour categories of English speakers was collected using online crowd sourcing-1,000 subjects produced 20,000 unconstrained names for 600 colour stimuli. From this data, using the framework of Information Geometry, a Riemannian metric was computed throughout the RGB cube. This is the first colour metric to have been computed from colour categorization data. In this categorical metric the distance between two close colours is determined by the difference in the distribution of names that the subject population applied to them. This contrasts with previous colour metrics which have been driven by stimulus discriminability, or acceptability of a colour match. The categorical metric is analysed and shown to be clearly different from discriminability-based metrics. Natural units of categorical length, area and volume are derived. These allow a count to be made of the number of categorically-distinct regions of categorically-similar colours that fit within colour space. Our analysis estimates that 27 such regions fit within the RGB cube, which agrees well with a previous estimate of 30 colours that can be identified by name by untrained subjects.

The Atlas Structure of Images

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

Many operations of vision require image regions to be isolated and interrelated. This is challeng... more Many operations of vision require image regions to be isolated and interrelated. This is challenging when they are different in detail and extent. Practical methods of Computer Vision approach this through the tools of downsampling, pyramids, cropping and patches. In this paper we develop an ideal geometric structure for this, compatible with the existing scale space model of image measurement. Its elements are apertures which view the image like fuzzy-edged portholes of frosted glass. We establish containment and cause/effect relations between apertures, and show that these link them into cross-scale atlases. Atlases formed of Gaussian apertures are shown to be a continuous version of the image pyramid used in Computer Vision, and allow various types of image description to naturally be expressed within their framework. We show that views through Gaussian apertures are approximately equivalent to the jets of derivative of Gaussian filter responses that form part of standard Scale Space theory. This supports a view of the simple cells of mammalian V1 as implementing a system of local views of the retinal image of varying extent and resolution. As a worked example we develop a keypoint descriptor scheme that outperforms previous schemes that do not make use of learning.

A spatial frequency spectral peakedness model predicts discrimination performance of regularity in dot patterns

Vision research, Aug 1, 2018

Subjective assessments of spatial regularity are common in everyday life and also in science, for... more Subjective assessments of spatial regularity are common in everyday life and also in science, for example in developmental biology. It has recently been shown that regularity is an adaptable visual dimension. It was proposed that regularity is coded via the peakedness of the distribution of neural responses across receptive field size. Here, we test this proposal for jittered square lattices of dots. We examine whether discriminability correlates with a simple peakedness measure across different presentation conditions (dot number, size, and average spacing). Using a filter-rectify-filter model, we determined responses across scale. Consistently, two peaks are present: a lower frequency peak corresponding to the dot spacing of the regular pattern and a higher frequency peak corresponding to the pattern element (dot). We define the "peakedness" of a particular presentation condition as the relative heights of these two peaks for a perfectly regular pattern constructed using...

Intensity-Based Object Extraction from 3D Medical Images Including a Correction of Partial Volume Errors

Procedings of the British Machine Vision Conference 1994, 1994

Feature extraction by applying a threshold window to image intensity values is a simple and commo... more Feature extraction by applying a threshold window to image intensity values is a simple and common image processing technique. We consider the case of 3D images where intensity based feature extraction is used to determine the volume of objects of interest. We show that the accuracy of volume determination is limited by partial volume (PV) effects. We outline a new method for correcting for PV effects based on object geometry and object intensity. Although this PV correction has been developed with respect to a specific application in magnetic resonance imaging, it is applicable to volume extraction by thresholding in any image and can possibly be extended to other intensity-based extraction techniques.

A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery

Anomaly Detection and Imaging with X-Rays (ADIX) II, 2017

Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called S... more Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called Small Metallic Threats (SMTs) hidden amongst legitimate goods inside a cargo container. We trained CNNs from scratch on data produced by a Threat Image Projection (TIP) framework that generates images with realistic variation to robustify performance. We achieved 90% detection of containers that contained a single SMT, while raising 6% false positives on benign containers. The best CNN architecture used the raw high energy image (single-energy) and its logarithm as input channels. Use of the logarithm improved performance, thus echoing studies on human operator performance. However, it is an unexpected result with CNNs. In this work, we (i) investigate methods to exploit material information captured in dual-energy images, and (ii) introduce a new CNN training scheme that generates 'spot-the-difference' benign and threat pairs on-the-fly. To the best of our knowledge, this is the first time that CNNs have been applied directly to raw dual-energy X-ray imagery, in any field. To exploit dual-energy, we experiment with adapting several physics-derived approaches to material discrimination from the cargo literature, and introduce three novel variants. We hypothesise that CNNs can implicitly learn about the material characteristics of objects from the raw dual-energy images, and use this to suppress false positives. The best performing method is able to detect 95% of containers containing a single SMT, while raising 0.4% false positives on benign containers. This is a step change improvement in performance over our prior work.

Segmentation of phase contrast microscopy images based on multi-scale local Basic Image Features histograms

Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2015

Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in... more Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in all fields of biology and biomedicine. Key decisions for experimental protocols are often taken by an operator based on typically qualitative observations. However, automated processing and analysis of PCM images remain challenging due to the low contrast between foreground objects (cells) and background as well as various imaging artefacts. We propose a trainable pixel-wise segmentation approach whereby image structures and symmetries are encoded in the form of multi-scale Basic Image Features local histograms, and classification of them is learned by random decision trees. This approach was validated for segmentation of cell versus background, and discrimination between two different cell types. Performance close to that of state-of-the-art specialised algorithms was achieved despite the general nature of the method. The low processing time (, 4 s per 1280 £ 960 pixel images) is suitable for batch processing of experimental data as well as for interactive segmentation applications.

Measuring and correcting wobble in large-scale transmission radiography

Journal of X-Ray Science and Technology, 2017

BACKGROUND: Large-scale transmission radiography scanners are used to image vehicles and cargo co... more BACKGROUND: Large-scale transmission radiography scanners are used to image vehicles and cargo containers. Acquired images are inspected for threats by a human operator or a computer algorithm. To make accurate detections, it is important that image values are precise. However, due to the scale (∼ 5 m tall) of such systems, they can be mechanically unstable, causing the imaging array to wobble during a scan. This leads to an effective loss of precision in the captured image. OBJECTIVE: We consider the measurement of wobble and amelioration of the consequent loss of image precision. METHODS: Following our previous work, we use Beam Position Detectors (BPDs) to measure the cross-sectional profile of the X-ray beam, allowing for estimation, and thus correction, of wobble. We propose: (i) a model of image formation with a wobbling detector array; (ii) a method of wobble correction derived from this model; (iii) methods for calibrating sensor sensitivities and relative offsets; (iv) a Random Regression Forest based method for instantaneous estimation of detector wobble; and (v) using these estimates to apply corrections to captured images of difficult scenes. RESULTS: We show that these methods are able to correct for 87% of image error due wobble, and when applied to difficult images, a significant visible improvement in the intensity-windowed image quality is observed. CONCLUSIONS: The method improves the precision of wobble affected images, which should help improve detection of threats and the identification of different materials in the image.

Detection of concealed cars in complex cargo X-ray imagery using Deep Learning

Journal of X-Ray Science and Technology, 2017

Non-intrusive inspection systems based on X-ray radiography techniques are routinely used at tran... more Non-intrusive inspection systems based on X-ray radiography techniques are routinely used at transport hubs to ensure the conformity of cargo content with the supplied shipping manifest. As trade volumes increase and regulations become more stringent, manual inspection by trained operators is less and less viable due to low throughput. Machine vision techniques can assist operators in their task by automating parts of the inspection workflow. Since cars are routinely involved in trafficking, export fraud, and tax evasion schemes, they represent an attractive target for automated detection and flagging for subsequent inspection by operators. In this contribution, we describe a method for the detection of cars in X-ray cargo images based on trained-from-scratch Convolutional Neural Networks. By introducing an oversampling scheme that suitably addresses the low number of car images available for training, we achieved 100% car image classification rate for a false positive rate of 1-in-454. Cars that were partially or completely obscured by other goods, a modus operandi frequently adopted by criminals, were correctly detected. We believe that this level of performance suggests that the method is suitable for deployment in the field. It is expected that the generic object detection workflow described can be extended to other object classes given the availability of suitable training data.

Automated X-ray image analysis for cargo security: Critical review and future promise

Journal of X-Ray Science and Technology, 2017

We review the relatively immature field of automated image analysis for X-ray cargo imagery. Ther... more We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs-and securityrelated threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo.

Tackling the x-ray cargo inspection challenge using machine learning

Anomaly Detection and Imaging with X-Rays (ADIX), 2016

The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate ex... more The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate exploding commerce volumes and increasingly stringent regulations. There is a pressing need to develop methods to automate parts of the inspection workflow, enabling expert operators to focus on a manageable number of high-risk images. To tackle this challenge, we developed a modular framework for automated X-ray cargo image inspection. Employing state-of-the-art machine learning approaches, including deep learning, we demonstrate high performance for empty container verification and specific threat detection. This work constitutes a significant step towards the partial automation of X-ray cargo image inspection.

The Use of English Colour Terms in Big Data

This study explores the use of English colour names in large datasets from informal Twitter messa... more This study explores the use of English colour names in large datasets from informal Twitter messages and the well-structured corpus of Google Books. Because colour names in text have no directly associated chromatic stimuli, the corresponding colour categories of colour words was assessed from responses in an online colour naming experiment. A comparison of the frequency in the three datasets revealed that the mapping of colour names to perceptually uniform colour spaces does not reflect natural language colour distributions.

Multi-scale Hierarchical Segmentation

Procedings of the British Machine Vision Conference 1993, 1993

In the absence of a priori information on scales of interest vision systems should initially proc... more In the absence of a priori information on scales of interest vision systems should initially process in a scale invariant manner. The fact that any signal can only be sampled discretely further constrains the initial processing. The paper argues that a representation satisfying these requirements is an hierarchical segmentation of scalespace. An algorithm is presented to compute such representations. The algorithm has been designed so that its operation is scale invariant in the following sense: the addition of finer scale information only ever adds to the computed representation and never changes what was discoverable from coarser scales. It is noted that such a scheme has benefits even when the scales of interest are known.

Hierarchical Segmentation Satisfying Constraints

Procedings of the British Machine Vision Conference 1994, 1994

A new hierarchical segmentation algorithm is described. Its computational complexity and memory r... more A new hierarchical segmentation algorithm is described. Its computational complexity and memory requirements are detailed, showing it to be practicably applicable to images of useful size. A simple modification of the algorithm adapts it to produce hierarchical segmentations that satisfy a constraint set. Results are given showing that this adapted algorithm can be used as the basis of a semi-automatic object definition tool or as the interface between a low-level image description module and a high-level module coding for knowledge and expectation.

Multiscale Histogram of Oriented Gradient Descriptors for Robust Character Recognition

2011 International Conference on Document Analysis and Recognition, 2011

Characters extracted from images or graphics pose a challenge for traditional character recogniti... more Characters extracted from images or graphics pose a challenge for traditional character recognition techniques. The high degree of intraclass variation along with the presence of clutter makes accurate recognition difficult, yet the semantic information conveyed by sections of text within images or graphics makes their recognition an important problem. Previous work has shown that, on the two most commonly used datasets of such characters, Histogram of Oriented Gradient (HOG) descriptors have outperformed other methods. In this work we consider two extensions of the HOG descriptor to include features at multiple scales, and evaluate their performance using characters taken from images and graphics. We demonstrate that, by combining pairs of oriented gradients at different scales, it's possible to achieve an increase in performance of 12.4% and 5.6% on the two datasets.

An absolute interval scale of order for point patterns

Journal of the Royal Society, Interface / the Royal Society, Jan 6, 2014

Human observers readily make judgements about the degree of order in planar arrangements of point... more Human observers readily make judgements about the degree of order in planar arrangements of points (point patterns). Here, based on pairwise ranking of 20 point patterns by degree of order, we have been able to show that judgements of order are highly consistent across individuals and the dimension of order has an interval scale structure spanning roughly 10 just-notable-differences (jnd) between disorder and order. We describe a geometric algorithm that estimates order to an accuracy of half a jnd by quantifying the variability of the size and shape of spaces between points. The algorithm is 70% more accurate than the best available measures. By anchoring the output of the algorithm so that Poisson point processes score on average 0, perfect lattices score 10 and unit steps correspond closely to jnds, we construct an absolute interval scale of order. We demonstrate its utility in biology by using this scale to quantify order during the development of the pattern of bristles on the ...

Trainable Segmentation of Phase Contrast Microscopy Images Based on Local Basic Image Features Histograms

Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in... more Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in all fields of biology and biomedicine. Key decisions for experimental protocols are often taken by an operator based on typically qualitative observations. However, automated processing and analysis of PCM images remain challenging due to the low contrast between foreground objects (cells) and background as well as various imaging artefacts. We propose a trainable pixel-wise segmentation approach whereby image structures and symmetries are encoded in the form of multi-scale Basic Image Features local histograms and classification of them is learned by random decision trees. This approach was validated for segmentation of cell versus background, and discrimination between two different cell types. Performance close to that of state-of-the-art specialised algorithms was achieved despite the general nature of the method. The low processing time (<4s per 1280 × 960 pixel images) is suitable for batch processing of experimental data as well as for interactive segmentation applications.

Perceptual Similarity: A Texture Challenge

Procedings of the British Machine Vision Conference 2011, 2011

Transferring x-ray based automated threat detection between scanners with different energies and resolution

Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies

A significant obstacle to developing high performance Deep Learning algorithms for Automated Thre... more A significant obstacle to developing high performance Deep Learning algorithms for Automated Threat Detection (ATD) in security X-ray imagery, is the difficulty of obtaining large training datasets. In our previous work, we circumvented this problem for ATD in cargo containers, using Threat Image Projection and data augmentation. In this work, we investigate whether data scarcity for other modalities, such as parcels and baggage, can be ameliorated by transforming data from one domain so that it approximates the appearance of another. We present an ontology of ATD datasets to assess where transfer learning may be applied. We define frameworks for transfer at the training and testing stages, and compare the results for both methods against ATD where a common data source is used for training and testing. Our results show very poor transfer, which we attribute to the difficulty of accurately matching the blur and contrast characteristics of different scanners.

Automated detection of smuggled high-risk security threats using Deep Learning

7th International Conference on Imaging for Crime Detection and Prevention (ICDP 2016), 2016

The security infrastructure is ill-equipped to detect and deter the smuggling of non-explosive de... more The security infrastructure is ill-equipped to detect and deter the smuggling of non-explosive devices that enable terror attacks such as those recently perpetrated in western Europe. The detection of so-called "Small Metallic Threats" (SMTs) in cargo containers currently relies on statistical risk analysis, intelligence reports, and visual inspection of X-ray images by security officers. The latter is very slow and unreliable due to the difficulty of the task: objects potentially spanning less than 50 pixels have to be detected in images containing more than 2 million pixels against very complex and cluttered backgrounds. In this contribution, we demonstrate for the first time the use of Convolutional Neural Networks (CNNs), a type of Deep Learning, to automate the detection of SMTs in fullsize X-ray images of cargo containers. Novel approaches for dataset augmentation allowed to train CNNs from-scratch despite the scarcity of data available. We report fewer than 6% false alarms when detecting 90% SMTs synthetically concealed in streamof-commerce images, which corresponds to an improvement of over an order of magnitude over conventional approaches such as Bag-of-Words (BoWs). The proposed scheme offers potentially superhuman performance for a fraction of the time it would take for a security officers to carry out visual inspection (processing time is approximately 3.5s per container image).

Reconciling the statistics of spectral reflectance and colour

PLOS ONE, 2019

The spectral reflectance function of a surface specifies the fraction of the illumination reflect... more The spectral reflectance function of a surface specifies the fraction of the illumination reflected by it at each wavelength. Jointly with the illumination spectral density, this function determines the apparent colour of the surface. Models for the distribution of spectral reflectance functions in the natural environment are considered. The realism of the models is assessed in terms of the individual reflectance functions they generate, and in terms of the overall distribution of colours which they give rise to. Both realism assessments are made in comparison to empirical datasets. Previously described models (PCA-and fourier-based) of reflectance function statistics are evaluated, as are improved versions; and also a novel model, which synthesizes reflectance functions as a sum of sigmoid functions. Key model features for realism are identified. The new sigmoid-sum model is shown to be the most realistic, generating reflectance functions that are hard to distinguish from real ones, and accounting for the majority of colours found in natural images with the exception of an abundance of vegetation green and sky blue.

Categorical colour geometry

PLOS ONE, 2019

Ordinary language users group colours into categories that they refer to by a name e.g. pale gree... more Ordinary language users group colours into categories that they refer to by a name e.g. pale green. Data on the colour categories of English speakers was collected using online crowd sourcing-1,000 subjects produced 20,000 unconstrained names for 600 colour stimuli. From this data, using the framework of Information Geometry, a Riemannian metric was computed throughout the RGB cube. This is the first colour metric to have been computed from colour categorization data. In this categorical metric the distance between two close colours is determined by the difference in the distribution of names that the subject population applied to them. This contrasts with previous colour metrics which have been driven by stimulus discriminability, or acceptability of a colour match. The categorical metric is analysed and shown to be clearly different from discriminability-based metrics. Natural units of categorical length, area and volume are derived. These allow a count to be made of the number of categorically-distinct regions of categorically-similar colours that fit within colour space. Our analysis estimates that 27 such regions fit within the RGB cube, which agrees well with a previous estimate of 30 colours that can be identified by name by untrained subjects.

The Atlas Structure of Images

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

Many operations of vision require image regions to be isolated and interrelated. This is challeng... more Many operations of vision require image regions to be isolated and interrelated. This is challenging when they are different in detail and extent. Practical methods of Computer Vision approach this through the tools of downsampling, pyramids, cropping and patches. In this paper we develop an ideal geometric structure for this, compatible with the existing scale space model of image measurement. Its elements are apertures which view the image like fuzzy-edged portholes of frosted glass. We establish containment and cause/effect relations between apertures, and show that these link them into cross-scale atlases. Atlases formed of Gaussian apertures are shown to be a continuous version of the image pyramid used in Computer Vision, and allow various types of image description to naturally be expressed within their framework. We show that views through Gaussian apertures are approximately equivalent to the jets of derivative of Gaussian filter responses that form part of standard Scale Space theory. This supports a view of the simple cells of mammalian V1 as implementing a system of local views of the retinal image of varying extent and resolution. As a worked example we develop a keypoint descriptor scheme that outperforms previous schemes that do not make use of learning.

A spatial frequency spectral peakedness model predicts discrimination performance of regularity in dot patterns

Vision research, Aug 1, 2018

Subjective assessments of spatial regularity are common in everyday life and also in science, for... more Subjective assessments of spatial regularity are common in everyday life and also in science, for example in developmental biology. It has recently been shown that regularity is an adaptable visual dimension. It was proposed that regularity is coded via the peakedness of the distribution of neural responses across receptive field size. Here, we test this proposal for jittered square lattices of dots. We examine whether discriminability correlates with a simple peakedness measure across different presentation conditions (dot number, size, and average spacing). Using a filter-rectify-filter model, we determined responses across scale. Consistently, two peaks are present: a lower frequency peak corresponding to the dot spacing of the regular pattern and a higher frequency peak corresponding to the pattern element (dot). We define the "peakedness" of a particular presentation condition as the relative heights of these two peaks for a perfectly regular pattern constructed using...

Intensity-Based Object Extraction from 3D Medical Images Including a Correction of Partial Volume Errors

Procedings of the British Machine Vision Conference 1994, 1994

Feature extraction by applying a threshold window to image intensity values is a simple and commo... more Feature extraction by applying a threshold window to image intensity values is a simple and common image processing technique. We consider the case of 3D images where intensity based feature extraction is used to determine the volume of objects of interest. We show that the accuracy of volume determination is limited by partial volume (PV) effects. We outline a new method for correcting for PV effects based on object geometry and object intensity. Although this PV correction has been developed with respect to a specific application in magnetic resonance imaging, it is applicable to volume extraction by thresholding in any image and can possibly be extended to other intensity-based extraction techniques.

A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery

Anomaly Detection and Imaging with X-Rays (ADIX) II, 2017

Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called S... more Previously, we investigated the use of Convolutional Neural Networks (CNNs) to detect so-called Small Metallic Threats (SMTs) hidden amongst legitimate goods inside a cargo container. We trained CNNs from scratch on data produced by a Threat Image Projection (TIP) framework that generates images with realistic variation to robustify performance. We achieved 90% detection of containers that contained a single SMT, while raising 6% false positives on benign containers. The best CNN architecture used the raw high energy image (single-energy) and its logarithm as input channels. Use of the logarithm improved performance, thus echoing studies on human operator performance. However, it is an unexpected result with CNNs. In this work, we (i) investigate methods to exploit material information captured in dual-energy images, and (ii) introduce a new CNN training scheme that generates 'spot-the-difference' benign and threat pairs on-the-fly. To the best of our knowledge, this is the first time that CNNs have been applied directly to raw dual-energy X-ray imagery, in any field. To exploit dual-energy, we experiment with adapting several physics-derived approaches to material discrimination from the cargo literature, and introduce three novel variants. We hypothesise that CNNs can implicitly learn about the material characteristics of objects from the raw dual-energy images, and use this to suppress false positives. The best performing method is able to detect 95% of containers containing a single SMT, while raising 0.4% false positives on benign containers. This is a step change improvement in performance over our prior work.

Segmentation of phase contrast microscopy images based on multi-scale local Basic Image Features histograms

Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 2015

Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in... more Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in all fields of biology and biomedicine. Key decisions for experimental protocols are often taken by an operator based on typically qualitative observations. However, automated processing and analysis of PCM images remain challenging due to the low contrast between foreground objects (cells) and background as well as various imaging artefacts. We propose a trainable pixel-wise segmentation approach whereby image structures and symmetries are encoded in the form of multi-scale Basic Image Features local histograms, and classification of them is learned by random decision trees. This approach was validated for segmentation of cell versus background, and discrimination between two different cell types. Performance close to that of state-of-the-art specialised algorithms was achieved despite the general nature of the method. The low processing time (, 4 s per 1280 £ 960 pixel images) is suitable for batch processing of experimental data as well as for interactive segmentation applications.

Measuring and correcting wobble in large-scale transmission radiography

Journal of X-Ray Science and Technology, 2017

BACKGROUND: Large-scale transmission radiography scanners are used to image vehicles and cargo co... more BACKGROUND: Large-scale transmission radiography scanners are used to image vehicles and cargo containers. Acquired images are inspected for threats by a human operator or a computer algorithm. To make accurate detections, it is important that image values are precise. However, due to the scale (∼ 5 m tall) of such systems, they can be mechanically unstable, causing the imaging array to wobble during a scan. This leads to an effective loss of precision in the captured image. OBJECTIVE: We consider the measurement of wobble and amelioration of the consequent loss of image precision. METHODS: Following our previous work, we use Beam Position Detectors (BPDs) to measure the cross-sectional profile of the X-ray beam, allowing for estimation, and thus correction, of wobble. We propose: (i) a model of image formation with a wobbling detector array; (ii) a method of wobble correction derived from this model; (iii) methods for calibrating sensor sensitivities and relative offsets; (iv) a Random Regression Forest based method for instantaneous estimation of detector wobble; and (v) using these estimates to apply corrections to captured images of difficult scenes. RESULTS: We show that these methods are able to correct for 87% of image error due wobble, and when applied to difficult images, a significant visible improvement in the intensity-windowed image quality is observed. CONCLUSIONS: The method improves the precision of wobble affected images, which should help improve detection of threats and the identification of different materials in the image.

Detection of concealed cars in complex cargo X-ray imagery using Deep Learning

Journal of X-Ray Science and Technology, 2017

Non-intrusive inspection systems based on X-ray radiography techniques are routinely used at tran... more Non-intrusive inspection systems based on X-ray radiography techniques are routinely used at transport hubs to ensure the conformity of cargo content with the supplied shipping manifest. As trade volumes increase and regulations become more stringent, manual inspection by trained operators is less and less viable due to low throughput. Machine vision techniques can assist operators in their task by automating parts of the inspection workflow. Since cars are routinely involved in trafficking, export fraud, and tax evasion schemes, they represent an attractive target for automated detection and flagging for subsequent inspection by operators. In this contribution, we describe a method for the detection of cars in X-ray cargo images based on trained-from-scratch Convolutional Neural Networks. By introducing an oversampling scheme that suitably addresses the low number of car images available for training, we achieved 100% car image classification rate for a false positive rate of 1-in-454. Cars that were partially or completely obscured by other goods, a modus operandi frequently adopted by criminals, were correctly detected. We believe that this level of performance suggests that the method is suitable for deployment in the field. It is expected that the generic object detection workflow described can be extended to other object classes given the availability of suitable training data.

Automated X-ray image analysis for cargo security: Critical review and future promise

Journal of X-Ray Science and Technology, 2017

We review the relatively immature field of automated image analysis for X-ray cargo imagery. Ther... more We review the relatively immature field of automated image analysis for X-ray cargo imagery. There is increasing demand for automated analysis methods that can assist in the inspection and selection of containers, due to the ever-growing volumes of traded cargo and the increasing concerns that customs-and securityrelated threats are being smuggled across borders by organised crime and terrorist networks. We split the field into the classical pipeline of image preprocessing and image understanding. Preprocessing includes: image manipulation; quality improvement; Threat Image Projection (TIP); and material discrimination and segmentation. Image understanding includes: Automated Threat Detection (ATD); and Automated Contents Verification (ACV). We identify several gaps in the literature that need to be addressed and propose ideas for future research. Where the current literature is sparse we borrow from the single-view, multi-view, and CT X-ray baggage domains, which have some characteristics in common with X-ray cargo.

Tackling the x-ray cargo inspection challenge using machine learning

Anomaly Detection and Imaging with X-Rays (ADIX), 2016

The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate ex... more The current infrastructure for non-intrusive inspection of cargo containers cannot accommodate exploding commerce volumes and increasingly stringent regulations. There is a pressing need to develop methods to automate parts of the inspection workflow, enabling expert operators to focus on a manageable number of high-risk images. To tackle this challenge, we developed a modular framework for automated X-ray cargo image inspection. Employing state-of-the-art machine learning approaches, including deep learning, we demonstrate high performance for empty container verification and specific threat detection. This work constitutes a significant step towards the partial automation of X-ray cargo image inspection.

The Use of English Colour Terms in Big Data

This study explores the use of English colour names in large datasets from informal Twitter messa... more This study explores the use of English colour names in large datasets from informal Twitter messages and the well-structured corpus of Google Books. Because colour names in text have no directly associated chromatic stimuli, the corresponding colour categories of colour words was assessed from responses in an online colour naming experiment. A comparison of the frequency in the three datasets revealed that the mapping of colour names to perceptually uniform colour spaces does not reflect natural language colour distributions.

Multi-scale Hierarchical Segmentation

Procedings of the British Machine Vision Conference 1993, 1993

In the absence of a priori information on scales of interest vision systems should initially proc... more In the absence of a priori information on scales of interest vision systems should initially process in a scale invariant manner. The fact that any signal can only be sampled discretely further constrains the initial processing. The paper argues that a representation satisfying these requirements is an hierarchical segmentation of scalespace. An algorithm is presented to compute such representations. The algorithm has been designed so that its operation is scale invariant in the following sense: the addition of finer scale information only ever adds to the computed representation and never changes what was discoverable from coarser scales. It is noted that such a scheme has benefits even when the scales of interest are known.

Hierarchical Segmentation Satisfying Constraints

Procedings of the British Machine Vision Conference 1994, 1994

A new hierarchical segmentation algorithm is described. Its computational complexity and memory r... more A new hierarchical segmentation algorithm is described. Its computational complexity and memory requirements are detailed, showing it to be practicably applicable to images of useful size. A simple modification of the algorithm adapts it to produce hierarchical segmentations that satisfy a constraint set. Results are given showing that this adapted algorithm can be used as the basis of a semi-automatic object definition tool or as the interface between a low-level image description module and a high-level module coding for knowledge and expectation.

Multiscale Histogram of Oriented Gradient Descriptors for Robust Character Recognition

2011 International Conference on Document Analysis and Recognition, 2011

Characters extracted from images or graphics pose a challenge for traditional character recogniti... more Characters extracted from images or graphics pose a challenge for traditional character recognition techniques. The high degree of intraclass variation along with the presence of clutter makes accurate recognition difficult, yet the semantic information conveyed by sections of text within images or graphics makes their recognition an important problem. Previous work has shown that, on the two most commonly used datasets of such characters, Histogram of Oriented Gradient (HOG) descriptors have outperformed other methods. In this work we consider two extensions of the HOG descriptor to include features at multiple scales, and evaluate their performance using characters taken from images and graphics. We demonstrate that, by combining pairs of oriented gradients at different scales, it's possible to achieve an increase in performance of 12.4% and 5.6% on the two datasets.

An absolute interval scale of order for point patterns

Journal of the Royal Society, Interface / the Royal Society, Jan 6, 2014

Human observers readily make judgements about the degree of order in planar arrangements of point... more Human observers readily make judgements about the degree of order in planar arrangements of points (point patterns). Here, based on pairwise ranking of 20 point patterns by degree of order, we have been able to show that judgements of order are highly consistent across individuals and the dimension of order has an interval scale structure spanning roughly 10 just-notable-differences (jnd) between disorder and order. We describe a geometric algorithm that estimates order to an accuracy of half a jnd by quantifying the variability of the size and shape of spaces between points. The algorithm is 70% more accurate than the best available measures. By anchoring the output of the algorithm so that Poisson point processes score on average 0, perfect lattices score 10 and unit steps correspond closely to jnds, we construct an absolute interval scale of order. We demonstrate its utility in biology by using this scale to quantify order during the development of the pattern of bristles on the ...

Trainable Segmentation of Phase Contrast Microscopy Images Based on Local Basic Image Features Histograms

Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in... more Phase contrast microscopy (PCM) is routinely used for the inspection of adherent cell cultures in all fields of biology and biomedicine. Key decisions for experimental protocols are often taken by an operator based on typically qualitative observations. However, automated processing and analysis of PCM images remain challenging due to the low contrast between foreground objects (cells) and background as well as various imaging artefacts. We propose a trainable pixel-wise segmentation approach whereby image structures and symmetries are encoded in the form of multi-scale Basic Image Features local histograms and classification of them is learned by random decision trees. This approach was validated for segmentation of cell versus background, and discrimination between two different cell types. Performance close to that of state-of-the-art specialised algorithms was achieved despite the general nature of the method. The low processing time (<4s per 1280 × 960 pixel images) is suitable for batch processing of experimental data as well as for interactive segmentation applications.

Perceptual Similarity: A Texture Challenge

Procedings of the British Machine Vision Conference 2011, 2011