Active Learning techniques are used to tackle learning problems where obtaining training labels i... more Active Learning techniques are used to tackle learning problems where obtaining training labels is costly. In this work we use Meta-Active Learning to learn to select a subset of samples from a pool of unsupervised input for further annotation. This scenario is called Static Pool-based Meta-Active Learning. We propose to extend existing approaches by performing the selection in a manner that, unlike previous works, can handle the selection of each sample based on the whole selected subset.
3D video coding includes the use of multiple color views and depth maps associated to each view. ... more 3D video coding includes the use of multiple color views and depth maps associated to each view. An adequate coding of depth maps should be adapted to the characteristics of depth maps: smooth regions and sharp edges. In this paper a segmentation-based technique is proposed for improving the depth map compression while preserving the main discontinuities that exploits the color-depth similarity of 3D video. An initial coarse depth map segmentation is used to locate the main discontinuities in depth. The resulting partition is improved by fusing a color partition. We assume that the color image is first encoded and available when the associated depth map is encoded, therefore the color partition can be segmented in the decoder without introducing any extra cost. A new segmentation criterion inspired by super-pixels techniques is proposed to obtain the color partition. Initial experimental results show similar compression efficiency to hevc with a big potential for further improvements.
The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus dom... more The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji) to reconstruct the 3D model of 11 trees by using structure-from-motion (SfM) photogrammetry. The data provided in this article is related to the research article entitled “Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry” [1]. The Fuji-SfM dataset includes: (1) a set of 288 colour images and the corresponding annotations (apples segmentation masks) for training instance segmentation neural networks such as Mask-RCNN; (2) a set of 582 images defining a motion sequence of the scene which was used to generate the 3D model of 11 Fuji apple trees containing 1455 apples by using SfM; (3) the 3D point cloud of the scanned scene with the corresponding apple positions ground truth in global coordinates. With that, this is the first dataset for fruit detection containing images acquired in a motion sequence to build the 3D model of the scanned trees with SfM and including the corresponding 2D and 3D apple location annotations. This data allows the development, training, and test of fruit detection algorithms either based on RGB images, on coloured point clouds or on the combination of both types of data.
IEEE Transactions on Circuits and Systems for Video Technology, 1997
This paper presents a generic video coding algorithm allowing the content-based manipulation of o... more This paper presents a generic video coding algorithm allowing the content-based manipulation of objects. This manipulation is possible thanks to the definition of a spatiotemporal segmentation of the sequences. The coding strategy relies on a joint optimization in the rate-distortion sense of the partition definition and of the coding techniques to be used within each region. This optimization creates the link between the analysis and synthesis parts of the coder. The analysis defines the time evolution of the partition, as well as the elimination or the appearance of regions that are homogeneous either spatially or in motion. The coding of the texture as well as of the partition relies on region-based motion compensation techniques. The algorithm offers a good compromise between the ability to track and manipulate objects and the coding efficiency.
A crucial aspect for selecting optimal areas for commercial advertising is the probability with w... more A crucial aspect for selecting optimal areas for commercial advertising is the probability with which that publicity will be seen. This paper presents a method based on topview camera measurement, where the probability of viewing is estimated according to the trajectories and movements of the head of the passerby individuals in the area of interest. Using a camera with a depth sensor, the head of the people in the range of view can be detected and modelled. That method allows to determine the orientation of the head which is used to estimate the direction of vision. A tracking by detection algorithm is used to compute the trajectory of each user. The attention given at each advertising spot is estimated based on the trajectories and head orientations of the individuals in the area of interest.
Journal of Real-time Image Processing, Mar 15, 2017
With the development of novel parallel architectures for image processing, the implementation of ... more With the development of novel parallel architectures for image processing, the implementation of well-known image operators needs to be reformulated to take advantage of the so-called massive parallelism. In this work, we propose a general algorithm that implements a large class of nonlinear filters, called stack filters, with a 2D-array processor. The proposed method consists of decomposing an image into bitplanes with the bitwise decomposition, and then process every bitplane hierarchically. The filtered image is reconstructed by simply stacking the filtered bitplanes according to their order of significance. Owing to its hierarchical structure, our algorithm allows us to trade-off between image quality and processing time, and to significantly reduce the computation time of low-entropy images. Also, experimental tests show that the processing time of our method is substantially lower than that of classical methods when using large structuring elements. All these features are of interest to a variety of real-time applications based on morphological operations such as video segmentation and video enhancement.
In 3D video, view synthesis is used to process new virtual views between encoded camera views. Er... more In 3D video, view synthesis is used to process new virtual views between encoded camera views. Errors in the coding of the depth maps introduce geometry inconsistencies in synthesized views. In this paper, a 3D plane representation of the scene is presented which improve the performance of current standard video codecs in the view synthesis domain. Depth maps are segmented into regions without sharp edges and represented with a plane in the 3D world scene coordinates. This 3D representation allows an efficient representation while preserving the 3D characteristics of the scene. Experimental results are provided obtaining gains from 10 to 40 % in bitrate compared to HEVC.
The quantification of the attention received by advertisements is of paramount importance to dete... more The quantification of the attention received by advertisements is of paramount importance to determine its effectiveness. In this work, a simple and effective objective method for the assessment of the attention given to advertisements is provided. The proposed method is based on computing the oriented trajectory of the different test persons along with their head pose. This way, it is possible to determine if a given person is looking towards the advertisement. While other similar methods use more complex setups, requiring a camera at each advertisement location, our method needs only a single (or a few) ceiling camera. Even though the apparent simplicity, the method can compute attention measures at each point of the scene.
A novel approach to implement gray-scale morphological operations is presented in this work. This... more A novel approach to implement gray-scale morphological operations is presented in this work. This new technique is based on the bitwise decomposition of the gray-scale image, yielding bitplanes disposed according to their bit of significance. It is of particular interest for implementations on focal plane processors. Our approach relies on the binary search method to obtain either the maximum or
This article contains data related to the research article entitle "Multi-modal Deep Learning for... more This article contains data related to the research article entitle "Multi-modal Deep Learning for Fruit Detection Using RGB-D Cameras and their Radiometric Capabilities" [1]. The development of reliable fruit detection and localization systems is essential for future sustainable agronomic management of high-value crops. RGB-D sensors have shown potential for fruit detection and localization since they provide 3D information with color data. However, the lack of substantial datasets is a barrier for exploiting the use of these sensors. This article presents the KFuji RGB-DS database which is composed by 967 multi-modal images of Fuji apples on trees captured using Microsoft Kinect v2 (Microsoft, Redmond, WA, USA). Each image contains information from 3 different modalities: color (RGB), depth (D) and range corrected IR intensity (S). Ground truth fruit locations were manually annotated, labeling a total of 12,839 apples in all the dataset. The current dataset is publicly available at http://www.grap.udl.cat/publicacions/datasets.html.
This thesis work is mainly motivated to solve the stereo correspondence problem. As solving the c... more This thesis work is mainly motivated to solve the stereo correspondence problem. As solving the correspondence problem could result a precise disparity map allowing the interpolation of intermediate views in 3DTV, the disparity map estimation receives nowadays constant research interests by lots researchers worldwide. According to the state-of-art, the colour segmentation-based stereo matching algorithm achieves best performance among all algorithms. This thesis work dedicates to use image’s colour segmentation information for the region’s disparity modeling by polynomial function, and refine the final disparity map via a region merging process. The parameters of local matching algorithm have been studied and we address the attention at disparity polynomial model fitting, disparity map refinement by region merging process in this thesis. viii ACKNOWLEDGEMENTS
Journal of Visual Communication and Image Representation, 2012
In this paper a new coding method for multiview depth video is presented. Considering the smooth ... more In this paper a new coding method for multiview depth video is presented. Considering the smooth structure and sharp edges of depth maps, a segmentation based approach is proposed. This allows further preserving the depth contours thus introducing fewer artifacts in the depth perception of the video. To reduce the cost associated with partition coding, an estimation of the depth partition is built using the decoded color view segmentation. This estimation is refined by sending some complementary information about the relevant differences between color and depth partitions. For coding the depth content of each region, a decomposition into orthogonal basis is used in this paper although similar decompositions may be also employed. Experimental results show that the proposed segmentation based depth coding method outperforms H.264/AVC and H.264/MVC by more than 2dB at similar bitrates.
The AI4Agriculture Grape Dataset is a collection of 250 images taken in a vineyard in Ribera de D... more The AI4Agriculture Grape Dataset is a collection of 250 images taken in a vineyard in Ribera de Duero. The grapefruits are from the Tempranillo variety. The images are annotated using bounding boxes. The goal of this dataset is to provide images and the associated annotations to train and validate object detection models. These models can be used in viticulture applications, such as yield estimation, fruit counting, and field robotics. The images were taken in twelve different parcels. Six areas were defined in each parcel. Each area contains several lines of plants (grapevines). For each line of plants, 4 picture samples have been taken (two on each side). The samples are from non-contiguous grapevines, so the pictures never overlap. The pictures were taken by a Xiaomi Redmi 8 (with resolution 4032x3024), Xiaomi Mi A3 (with resolution 3000x4000) and a Xiaomi Redmi 9 (with resolution 3264x2448). To avoid showing grapes from other rows, the pictures were taken at approximately half t...
Active Learning techniques are used to tackle learning problems where obtaining training labels i... more Active Learning techniques are used to tackle learning problems where obtaining training labels is costly. In this work we use Meta-Active Learning to learn to select a subset of samples from a pool of unsupervised input for further annotation. This scenario is called Static Pool-based Meta-Active Learning. We propose to extend existing approaches by performing the selection in a manner that, unlike previous works, can handle the selection of each sample based on the whole selected subset.
3D video coding includes the use of multiple color views and depth maps associated to each view. ... more 3D video coding includes the use of multiple color views and depth maps associated to each view. An adequate coding of depth maps should be adapted to the characteristics of depth maps: smooth regions and sharp edges. In this paper a segmentation-based technique is proposed for improving the depth map compression while preserving the main discontinuities that exploits the color-depth similarity of 3D video. An initial coarse depth map segmentation is used to locate the main discontinuities in depth. The resulting partition is improved by fusing a color partition. We assume that the color image is first encoded and available when the associated depth map is encoded, therefore the color partition can be segmented in the decoder without introducing any extra cost. A new segmentation criterion inspired by super-pixels techniques is proposed to obtain the color partition. Initial experimental results show similar compression efficiency to hevc with a big potential for further improvements.
The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus dom... more The present dataset contains colour images acquired in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji) to reconstruct the 3D model of 11 trees by using structure-from-motion (SfM) photogrammetry. The data provided in this article is related to the research article entitled “Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry” [1]. The Fuji-SfM dataset includes: (1) a set of 288 colour images and the corresponding annotations (apples segmentation masks) for training instance segmentation neural networks such as Mask-RCNN; (2) a set of 582 images defining a motion sequence of the scene which was used to generate the 3D model of 11 Fuji apple trees containing 1455 apples by using SfM; (3) the 3D point cloud of the scanned scene with the corresponding apple positions ground truth in global coordinates. With that, this is the first dataset for fruit detection containing images acquired in a motion sequence to build the 3D model of the scanned trees with SfM and including the corresponding 2D and 3D apple location annotations. This data allows the development, training, and test of fruit detection algorithms either based on RGB images, on coloured point clouds or on the combination of both types of data.
IEEE Transactions on Circuits and Systems for Video Technology, 1997
This paper presents a generic video coding algorithm allowing the content-based manipulation of o... more This paper presents a generic video coding algorithm allowing the content-based manipulation of objects. This manipulation is possible thanks to the definition of a spatiotemporal segmentation of the sequences. The coding strategy relies on a joint optimization in the rate-distortion sense of the partition definition and of the coding techniques to be used within each region. This optimization creates the link between the analysis and synthesis parts of the coder. The analysis defines the time evolution of the partition, as well as the elimination or the appearance of regions that are homogeneous either spatially or in motion. The coding of the texture as well as of the partition relies on region-based motion compensation techniques. The algorithm offers a good compromise between the ability to track and manipulate objects and the coding efficiency.
A crucial aspect for selecting optimal areas for commercial advertising is the probability with w... more A crucial aspect for selecting optimal areas for commercial advertising is the probability with which that publicity will be seen. This paper presents a method based on topview camera measurement, where the probability of viewing is estimated according to the trajectories and movements of the head of the passerby individuals in the area of interest. Using a camera with a depth sensor, the head of the people in the range of view can be detected and modelled. That method allows to determine the orientation of the head which is used to estimate the direction of vision. A tracking by detection algorithm is used to compute the trajectory of each user. The attention given at each advertising spot is estimated based on the trajectories and head orientations of the individuals in the area of interest.
Journal of Real-time Image Processing, Mar 15, 2017
With the development of novel parallel architectures for image processing, the implementation of ... more With the development of novel parallel architectures for image processing, the implementation of well-known image operators needs to be reformulated to take advantage of the so-called massive parallelism. In this work, we propose a general algorithm that implements a large class of nonlinear filters, called stack filters, with a 2D-array processor. The proposed method consists of decomposing an image into bitplanes with the bitwise decomposition, and then process every bitplane hierarchically. The filtered image is reconstructed by simply stacking the filtered bitplanes according to their order of significance. Owing to its hierarchical structure, our algorithm allows us to trade-off between image quality and processing time, and to significantly reduce the computation time of low-entropy images. Also, experimental tests show that the processing time of our method is substantially lower than that of classical methods when using large structuring elements. All these features are of interest to a variety of real-time applications based on morphological operations such as video segmentation and video enhancement.
In 3D video, view synthesis is used to process new virtual views between encoded camera views. Er... more In 3D video, view synthesis is used to process new virtual views between encoded camera views. Errors in the coding of the depth maps introduce geometry inconsistencies in synthesized views. In this paper, a 3D plane representation of the scene is presented which improve the performance of current standard video codecs in the view synthesis domain. Depth maps are segmented into regions without sharp edges and represented with a plane in the 3D world scene coordinates. This 3D representation allows an efficient representation while preserving the 3D characteristics of the scene. Experimental results are provided obtaining gains from 10 to 40 % in bitrate compared to HEVC.
The quantification of the attention received by advertisements is of paramount importance to dete... more The quantification of the attention received by advertisements is of paramount importance to determine its effectiveness. In this work, a simple and effective objective method for the assessment of the attention given to advertisements is provided. The proposed method is based on computing the oriented trajectory of the different test persons along with their head pose. This way, it is possible to determine if a given person is looking towards the advertisement. While other similar methods use more complex setups, requiring a camera at each advertisement location, our method needs only a single (or a few) ceiling camera. Even though the apparent simplicity, the method can compute attention measures at each point of the scene.
A novel approach to implement gray-scale morphological operations is presented in this work. This... more A novel approach to implement gray-scale morphological operations is presented in this work. This new technique is based on the bitwise decomposition of the gray-scale image, yielding bitplanes disposed according to their bit of significance. It is of particular interest for implementations on focal plane processors. Our approach relies on the binary search method to obtain either the maximum or
This article contains data related to the research article entitle "Multi-modal Deep Learning for... more This article contains data related to the research article entitle "Multi-modal Deep Learning for Fruit Detection Using RGB-D Cameras and their Radiometric Capabilities" [1]. The development of reliable fruit detection and localization systems is essential for future sustainable agronomic management of high-value crops. RGB-D sensors have shown potential for fruit detection and localization since they provide 3D information with color data. However, the lack of substantial datasets is a barrier for exploiting the use of these sensors. This article presents the KFuji RGB-DS database which is composed by 967 multi-modal images of Fuji apples on trees captured using Microsoft Kinect v2 (Microsoft, Redmond, WA, USA). Each image contains information from 3 different modalities: color (RGB), depth (D) and range corrected IR intensity (S). Ground truth fruit locations were manually annotated, labeling a total of 12,839 apples in all the dataset. The current dataset is publicly available at http://www.grap.udl.cat/publicacions/datasets.html.
This thesis work is mainly motivated to solve the stereo correspondence problem. As solving the c... more This thesis work is mainly motivated to solve the stereo correspondence problem. As solving the correspondence problem could result a precise disparity map allowing the interpolation of intermediate views in 3DTV, the disparity map estimation receives nowadays constant research interests by lots researchers worldwide. According to the state-of-art, the colour segmentation-based stereo matching algorithm achieves best performance among all algorithms. This thesis work dedicates to use image’s colour segmentation information for the region’s disparity modeling by polynomial function, and refine the final disparity map via a region merging process. The parameters of local matching algorithm have been studied and we address the attention at disparity polynomial model fitting, disparity map refinement by region merging process in this thesis. viii ACKNOWLEDGEMENTS
Journal of Visual Communication and Image Representation, 2012
In this paper a new coding method for multiview depth video is presented. Considering the smooth ... more In this paper a new coding method for multiview depth video is presented. Considering the smooth structure and sharp edges of depth maps, a segmentation based approach is proposed. This allows further preserving the depth contours thus introducing fewer artifacts in the depth perception of the video. To reduce the cost associated with partition coding, an estimation of the depth partition is built using the decoded color view segmentation. This estimation is refined by sending some complementary information about the relevant differences between color and depth partitions. For coding the depth content of each region, a decomposition into orthogonal basis is used in this paper although similar decompositions may be also employed. Experimental results show that the proposed segmentation based depth coding method outperforms H.264/AVC and H.264/MVC by more than 2dB at similar bitrates.
The AI4Agriculture Grape Dataset is a collection of 250 images taken in a vineyard in Ribera de D... more The AI4Agriculture Grape Dataset is a collection of 250 images taken in a vineyard in Ribera de Duero. The grapefruits are from the Tempranillo variety. The images are annotated using bounding boxes. The goal of this dataset is to provide images and the associated annotations to train and validate object detection models. These models can be used in viticulture applications, such as yield estimation, fruit counting, and field robotics. The images were taken in twelve different parcels. Six areas were defined in each parcel. Each area contains several lines of plants (grapevines). For each line of plants, 4 picture samples have been taken (two on each side). The samples are from non-contiguous grapevines, so the pictures never overlap. The pictures were taken by a Xiaomi Redmi 8 (with resolution 4032x3024), Xiaomi Mi A3 (with resolution 3000x4000) and a Xiaomi Redmi 9 (with resolution 3264x2448). To avoid showing grapes from other rows, the pictures were taken at approximately half t...
Uploads
Papers by R. Morros