Automatic identification of "who" is present in individual digital images within a photo manageme... more Automatic identification of "who" is present in individual digital images within a photo management system using only content-based analysis is an extremely difficult problem. We present a system which enables identification of person reoccurrences within a personal photo management application by combining image content-based analysis tools with context data from image capture. This combined system employs automatic face detection and body-patch matching techniques, which collectively facilitate identifying person re-occurrences within images grouped into events based on context data. We introduce a face detection approach combining a histogrambased skin detection model and a modified BDF face detection method to detect multiple frontal faces in colour images. Corresponding body patches are then automatically segmented relative to the size, location and orientation of the detected faces in the image. We investigate the suitability of using different colour descriptors, including MPEG-7 colour descriptors, Color Coherent Vectors (CCV) and Color Correlograms for effective body-patch matching. The system has been successfully integrated into the MediAssist platform, a prototype web-based system for personal photo management, and runs on over 13000 personal photos.
A novel solution to the challenge of automatic image annotation is described. Given an image with... more A novel solution to the challenge of automatic image annotation is described. Given an image with GPS data of its location of capture, our system returns a semantically-rich annotation comprising tags which both identify the landmark in the image, and provide an interesting fact about it, e.g. "A view of the Eiffel Tower, which was built in 1889 for an international exhibition in Paris". This exploits visual and textual web mining in combination with content-based image analysis and natural language processing. In the first stage, an input image is matched to a set of community contributed images (with keyword tags) on the basis of its GPS information and image classification techniques. The depicted landmark is inferred from the keyword tags for the matched set. The system then takes advantage of the information written about landmarks available on the web at large to extract a fact about the landmark in the image. We report component evaluation results from an implementation of our solution on a mobile device. Image localisation and matching offers 93.6% classification accuracy; the selection of appropriate tags for use in annotation performs well (F1M of 0.59), and it subsequently automatically identifies a correct toponym for use in captioning and fact extraction in 69.0% of the tested cases; finally the fact extraction returns an interesting caption in 78% of cases.
In this paper, we describe an approach to camera metadata estimation using regression based on In... more In this paper, we describe an approach to camera metadata estimation using regression based on Independent Component Analysis (ICA). Semantic scene classification of images using camera metadata related to capture conditions has had some success in the past. However, different makes and models of camera capture different types of metadata and this severely hampers the application of this kind of approach in real systems that consist of photos captured by many different users. We propose to address this issue by using regression to predict the missing metadata from observed data, thereby providing more complete (and hence more useful) metadata for the entire image corpus. The proposed approach uses an ICA based approach to regression.
Automatic identification of "who" is present in individual digital images within a photo manageme... more Automatic identification of "who" is present in individual digital images within a photo management system using only content-based analysis is an extremely difficult problem. We present a system which enables identification of person reoccurrences within a personal photo management application by combining image content-based analysis tools with context data from image capture. This combined system employs automatic face detection and body-patch matching techniques, which collectively facilitate identifying person re-occurrences within images grouped into events based on context data. We introduce a face detection approach combining a histogrambased skin detection model and a modified BDF face detection method to detect multiple frontal faces in colour images. Corresponding body patches are then automatically segmented relative to the size, location and orientation of the detected faces in the image. We investigate the suitability of using different colour descriptors, including MPEG-7 colour descriptors, Color Coherent Vectors (CCV) and Color Correlograms for effective body-patch matching. The system has been successfully integrated into the MediAssist platform, a prototype web-based system for personal photo management, and runs on over 13000 personal photos.
A novel solution to the challenge of automatic image annotation is described. Given an image with... more A novel solution to the challenge of automatic image annotation is described. Given an image with GPS data of its location of capture, our system returns a semantically-rich annotation comprising tags which both identify the landmark in the image, and provide an interesting fact about it, e.g. "A view of the Eiffel Tower, which was built in 1889 for an international exhibition in Paris". This exploits visual and textual web mining in combination with content-based image analysis and natural language processing. In the first stage, an input image is matched to a set of community contributed images (with keyword tags) on the basis of its GPS information and image classification techniques. The depicted landmark is inferred from the keyword tags for the matched set. The system then takes advantage of the information written about landmarks available on the web at large to extract a fact about the landmark in the image. We report component evaluation results from an implementation of our solution on a mobile device. Image localisation and matching offers 93.6% classification accuracy; the selection of appropriate tags for use in annotation performs well (F1M of 0.59), and it subsequently automatically identifies a correct toponym for use in captioning and fact extraction in 69.0% of the tested cases; finally the fact extraction returns an interesting caption in 78% of cases.
In this paper, we describe an approach to camera metadata estimation using regression based on In... more In this paper, we describe an approach to camera metadata estimation using regression based on Independent Component Analysis (ICA). Semantic scene classification of images using camera metadata related to capture conditions has had some success in the past. However, different makes and models of camera capture different types of metadata and this severely hampers the application of this kind of approach in real systems that consist of photos captured by many different users. We propose to address this issue by using regression to predict the missing metadata from observed data, thereby providing more complete (and hence more useful) metadata for the entire image corpus. The proposed approach uses an ICA based approach to regression.
Uploads
Papers by N. O'Connor