ABSTRACT The multi-view three-dimensional (3D) visualization by means of a D display requires rep... more ABSTRACT The multi-view three-dimensional (3D) visualization by means of a D display requires reproduction of scene light fields. The complete light field of a scene can be reproduced from the images of a scene ideally taken from infinite viewpoints. However, capturing the images of a scene from infinite viewpoints is not feasible for practical applications. Therefore, in this work, we propose a sparse camera image capture system and an image based virtual image generation method for D imaging applications. We show a resulting virtual image produced by the proposed algorithm for generating in-between view of two real images captured with our multi-camera image capture system.
We present a general concept of the proposed 3D imaging system called 3D-geometric camera (3D-gCa... more We present a general concept of the proposed 3D imaging system called 3D-geometric camera (3D-gCam) to pick up pixel-wise 3D surface profile information along with color information. The 3D-gCam system includes two isotropic light sources placed at different geometric locations, an optical alignment system for aligning the light rays projected onto the scene, and a high precision camera. To determine the pixel-wise distance information, the system captures two images of the same scene during the stropping of the each light source. Then, the intensity of each pixel location in these two images along with the displacement between the light sources are utilized for calculating the distance information of object points corresponding to pixel locations in the image to generate a dense 3D point cloud. The approach is suitable for capturing of 3D and color information in high definition image format synchronously.
A general image analysis and segmentation method using fuzzy set classification and learning is d... more A general image analysis and segmentation method using fuzzy set classification and learning is described. The method uses a learned fuzzy representation of pixel region characteristics, based upon the conjunction and disjunction of extracted and derived fuzzy color and texture features. Both positive and negative exemplars of some visually apparent characteristic which forms the basis of the inspection, input by a human operator, are used together with a clustering algorithm to construct positive similarity membership functions and negative similarity membership functions. Using these composite fuzzified images, P and N, are produced using fuzzy union. Classification is accomplished via image defuzzification, whereby linguistic meaning is assigned to each pixel in the fuzzy set using a fuzzy inference operation. The technique permits: (1) strict color and texture discrimination, (2) machine learning of color and texture characteristics of regions, (3) and judicious labeling of each pixel based upon leaned fuzzy representation and fuzzy classification. This approach appears ideal for applications involving visual inspection and allows the development of image-based inspection systems which may be trained and used by relatively unskilled workers. We show three different examples involving the visual inspection of mixed waste drums, lumber and woven fabric.
In this work, we present a novel concept to sense 3D surface profile of scenes along with their c... more In this work, we present a novel concept to sense 3D surface profile of scenes along with their color images. The method for achieving this functionality includes two isotropic light sources placed at different geometric locations, an optic apparatus for aligning the light rays projected onto the scene, and a high precision camera. To determine the pixel-wise distance information, the system captures two sequential images during the stropping of each light source in sequence. Then, the intensity of each pixel location in these two images are utilized for calculating the distance information of object points corresponding to pixel locations in the image to generate a 3D surface profile of a scene. The approach is suitable for both capturing color information and sensing 3D distance information in high definition format synchronously.
We have already developed glasses-free three-dimensional (3-D) displays using multi-projectors an... more We have already developed glasses-free three-dimensional (3-D) displays using multi-projectors and a special diffuser screen that results in a highly realistic communication system. The system can display 70-200 inch large-sized 3-D images with full high-definition video image quality. The displayed 3-D images were, however, only computergenerated graphics or still images of actual objects. In this work, we studied a 3-D video capturing method for our multiprojection 3-D display. We analyzed the optimal arrangement of cameras for the display, and the image quality as influenced by calibration error. In the experiments, we developed a prototype multi-camera system using 30 highdefinition video cameras. The captured images were corrected via image processing optimized for the display. We successfully captured and displayed, for the first time, 3-D video of actual moving objects in our glasses-free 3-D video system.
In this paper we present a stereovision based model free 3D head pose (orientation and position) ... more In this paper we present a stereovision based model free 3D head pose (orientation and position) estimation system suitable for human-machine interface applications. The system works by obtaining a 'face plane' from the 3D reconstructed face data, which is then used for head pose estimation. The key novelty in this work is the utilization of the face plane together with the eye locations on the reconstructed face data to obtain a robust head pose estimate. This approach leads to a model and initialization free head pose estimation system; therefore it is suitable for natural human-machine interfaces. In order to quantitatively asses the accuracy of the system for such applications, several evaluation experiments were conducted using a commercial motion capture system. The evaluation results indicate that this system can be used in human-computer and human-robot applications.
... Sabri Gurbuz a,b , Shunsuke Yoshida a,b and Naomi Inoue a,b ... (7) Now, we want to remove th... more ... Sabri Gurbuz a,b , Shunsuke Yoshida a,b and Naomi Inoue a,b ... (7) Now, we want to remove the above noise term by defining a set of new coefficients as ak = avg( Ak−scene) + avg( Ak−noise). Then, we get a = [a1,a2,...,aN ], (8) b = [b1,b2,...,bN ]. (9) ...
In this paper, we describe a new algorithm to enhance the calibrated stereo camera based depth ma... more In this paper, we describe a new algorithm to enhance the calibrated stereo camera based depth map (disparity) estimation to view the 3D scene content on autostereoscopic displays. There are various similarity measure methods for calculating the disparity between a block in the first image and the corresponding block in the second image that is most similar to the block in the first image. This process is also called template matching. In the case of rectified stereo images, the template matching algorithm compares the template block from the first image against all blocks along the epipolar line in the second image assuming that all the corresponding pixels of the template block displaced by the same amount. The proposed algorithm employs the same principle, but differs with its similarity measure algorithm which is noise robust as well as intensity-wise shift and scale invariant. A comparative evaluation between the normalized cross-correlation based similarity measure and the proposed algorithm are given in the paper.
In multi-view imaging, a number of cameras capture the same scene from different viewpoints for r... more In multi-view imaging, a number of cameras capture the same scene from different viewpoints for regeneration of the light rays which may be emitted by a special display for 3D visualization. However, there are many conditions such as spectral characteristics of the optics, lighting variations, and sensor characteristics that cause color variations among captured multi-view images; therefore, these variations cause difficulties in accurately reproducing color consistency in multi-view 3D imaging systems. As a result, color variations between the camera views may deteriorate viewer's 3D visualization experiences. The process employed to compensate for the color variations of intercamera images is called multi-camera color calibration. In this paper, a color calibration method with respect to a reference camera for multi-camera systems is devised by utilizing a set of robustly detected stereo correspondences between camera pairs. The method results a 3x4 coefficient matrix multiplier that can be used for color correction operations rapidly. Objective evaluation of the experimental results shows that the proposed method can minimize the color differences between the captured multi-view images to an acceptable level relative to subjective evaluation results.
AbstractIn this paper, we describe a novel real-time stereo facial feature tracking algorithm in... more AbstractIn this paper, we describe a novel real-time stereo facial feature tracking algorithm in presence of geometry and illu-mination changes. The algorithm is applied to real-time human-humanoid interactions. First, facial feature tracking algorithm locates the facial ...
Journal of International Technology and Information Management, 2008
A human-centered computing platform that improves teamwork and transforms the "humancomputer inte... more A human-centered computing platform that improves teamwork and transforms the "humancomputer interaction experience" for distributed teams is presented. This Ubiquitous Collaboration, or uC ("you see"), platform's objective is to transform distributed teamwork (i.e., work occurring when teams of workers and learners are geographically dispersed and often interacting at different times). It achieves this goal through a multimodal team interaction interface realized through a reconfigurable open architecture. The approach taken is to integrate: (1) an intuitive speech-and video-centric multi-modal interface to augment more conventional methods (e.g., mouse, stylus and touch), (2) an open and reconfigurable architecture supporting information gathering, and (3) a machine intelligent approach to analysis and management of heterogeneous live and stored sensor data to support collaboration. The system will transform how teams of people interact with computers by drawing on both the virtual and physical environment.
Page 1. Biologically Valid Jaw Movements for Talking Humanoid Robots SABRI GURBUZ ATR Humon Infom... more Page 1. Biologically Valid Jaw Movements for Talking Humanoid Robots SABRI GURBUZ ATR Humon Infomotion Science Labs, Kyoto 619-0488, Japan sabrig@atr.jp ... We want to extract and store the motor tapes by analyzing a human speaker's 782 Page 3. ...
ABSTRACT The multi-view three-dimensional (3D) visualization by means of a D display requires rep... more ABSTRACT The multi-view three-dimensional (3D) visualization by means of a D display requires reproduction of scene light fields. The complete light field of a scene can be reproduced from the images of a scene ideally taken from infinite viewpoints. However, capturing the images of a scene from infinite viewpoints is not feasible for practical applications. Therefore, in this work, we propose a sparse camera image capture system and an image based virtual image generation method for D imaging applications. We show a resulting virtual image produced by the proposed algorithm for generating in-between view of two real images captured with our multi-camera image capture system.
We present a general concept of the proposed 3D imaging system called 3D-geometric camera (3D-gCa... more We present a general concept of the proposed 3D imaging system called 3D-geometric camera (3D-gCam) to pick up pixel-wise 3D surface profile information along with color information. The 3D-gCam system includes two isotropic light sources placed at different geometric locations, an optical alignment system for aligning the light rays projected onto the scene, and a high precision camera. To determine the pixel-wise distance information, the system captures two images of the same scene during the stropping of the each light source. Then, the intensity of each pixel location in these two images along with the displacement between the light sources are utilized for calculating the distance information of object points corresponding to pixel locations in the image to generate a dense 3D point cloud. The approach is suitable for capturing of 3D and color information in high definition image format synchronously.
A general image analysis and segmentation method using fuzzy set classification and learning is d... more A general image analysis and segmentation method using fuzzy set classification and learning is described. The method uses a learned fuzzy representation of pixel region characteristics, based upon the conjunction and disjunction of extracted and derived fuzzy color and texture features. Both positive and negative exemplars of some visually apparent characteristic which forms the basis of the inspection, input by a human operator, are used together with a clustering algorithm to construct positive similarity membership functions and negative similarity membership functions. Using these composite fuzzified images, P and N, are produced using fuzzy union. Classification is accomplished via image defuzzification, whereby linguistic meaning is assigned to each pixel in the fuzzy set using a fuzzy inference operation. The technique permits: (1) strict color and texture discrimination, (2) machine learning of color and texture characteristics of regions, (3) and judicious labeling of each pixel based upon leaned fuzzy representation and fuzzy classification. This approach appears ideal for applications involving visual inspection and allows the development of image-based inspection systems which may be trained and used by relatively unskilled workers. We show three different examples involving the visual inspection of mixed waste drums, lumber and woven fabric.
In this work, we present a novel concept to sense 3D surface profile of scenes along with their c... more In this work, we present a novel concept to sense 3D surface profile of scenes along with their color images. The method for achieving this functionality includes two isotropic light sources placed at different geometric locations, an optic apparatus for aligning the light rays projected onto the scene, and a high precision camera. To determine the pixel-wise distance information, the system captures two sequential images during the stropping of each light source in sequence. Then, the intensity of each pixel location in these two images are utilized for calculating the distance information of object points corresponding to pixel locations in the image to generate a 3D surface profile of a scene. The approach is suitable for both capturing color information and sensing 3D distance information in high definition format synchronously.
We have already developed glasses-free three-dimensional (3-D) displays using multi-projectors an... more We have already developed glasses-free three-dimensional (3-D) displays using multi-projectors and a special diffuser screen that results in a highly realistic communication system. The system can display 70-200 inch large-sized 3-D images with full high-definition video image quality. The displayed 3-D images were, however, only computergenerated graphics or still images of actual objects. In this work, we studied a 3-D video capturing method for our multiprojection 3-D display. We analyzed the optimal arrangement of cameras for the display, and the image quality as influenced by calibration error. In the experiments, we developed a prototype multi-camera system using 30 highdefinition video cameras. The captured images were corrected via image processing optimized for the display. We successfully captured and displayed, for the first time, 3-D video of actual moving objects in our glasses-free 3-D video system.
In this paper we present a stereovision based model free 3D head pose (orientation and position) ... more In this paper we present a stereovision based model free 3D head pose (orientation and position) estimation system suitable for human-machine interface applications. The system works by obtaining a 'face plane' from the 3D reconstructed face data, which is then used for head pose estimation. The key novelty in this work is the utilization of the face plane together with the eye locations on the reconstructed face data to obtain a robust head pose estimate. This approach leads to a model and initialization free head pose estimation system; therefore it is suitable for natural human-machine interfaces. In order to quantitatively asses the accuracy of the system for such applications, several evaluation experiments were conducted using a commercial motion capture system. The evaluation results indicate that this system can be used in human-computer and human-robot applications.
... Sabri Gurbuz a,b , Shunsuke Yoshida a,b and Naomi Inoue a,b ... (7) Now, we want to remove th... more ... Sabri Gurbuz a,b , Shunsuke Yoshida a,b and Naomi Inoue a,b ... (7) Now, we want to remove the above noise term by defining a set of new coefficients as ak = avg( Ak−scene) + avg( Ak−noise). Then, we get a = [a1,a2,...,aN ], (8) b = [b1,b2,...,bN ]. (9) ...
In this paper, we describe a new algorithm to enhance the calibrated stereo camera based depth ma... more In this paper, we describe a new algorithm to enhance the calibrated stereo camera based depth map (disparity) estimation to view the 3D scene content on autostereoscopic displays. There are various similarity measure methods for calculating the disparity between a block in the first image and the corresponding block in the second image that is most similar to the block in the first image. This process is also called template matching. In the case of rectified stereo images, the template matching algorithm compares the template block from the first image against all blocks along the epipolar line in the second image assuming that all the corresponding pixels of the template block displaced by the same amount. The proposed algorithm employs the same principle, but differs with its similarity measure algorithm which is noise robust as well as intensity-wise shift and scale invariant. A comparative evaluation between the normalized cross-correlation based similarity measure and the proposed algorithm are given in the paper.
In multi-view imaging, a number of cameras capture the same scene from different viewpoints for r... more In multi-view imaging, a number of cameras capture the same scene from different viewpoints for regeneration of the light rays which may be emitted by a special display for 3D visualization. However, there are many conditions such as spectral characteristics of the optics, lighting variations, and sensor characteristics that cause color variations among captured multi-view images; therefore, these variations cause difficulties in accurately reproducing color consistency in multi-view 3D imaging systems. As a result, color variations between the camera views may deteriorate viewer's 3D visualization experiences. The process employed to compensate for the color variations of intercamera images is called multi-camera color calibration. In this paper, a color calibration method with respect to a reference camera for multi-camera systems is devised by utilizing a set of robustly detected stereo correspondences between camera pairs. The method results a 3x4 coefficient matrix multiplier that can be used for color correction operations rapidly. Objective evaluation of the experimental results shows that the proposed method can minimize the color differences between the captured multi-view images to an acceptable level relative to subjective evaluation results.
AbstractIn this paper, we describe a novel real-time stereo facial feature tracking algorithm in... more AbstractIn this paper, we describe a novel real-time stereo facial feature tracking algorithm in presence of geometry and illu-mination changes. The algorithm is applied to real-time human-humanoid interactions. First, facial feature tracking algorithm locates the facial ...
Journal of International Technology and Information Management, 2008
A human-centered computing platform that improves teamwork and transforms the "humancomputer inte... more A human-centered computing platform that improves teamwork and transforms the "humancomputer interaction experience" for distributed teams is presented. This Ubiquitous Collaboration, or uC ("you see"), platform's objective is to transform distributed teamwork (i.e., work occurring when teams of workers and learners are geographically dispersed and often interacting at different times). It achieves this goal through a multimodal team interaction interface realized through a reconfigurable open architecture. The approach taken is to integrate: (1) an intuitive speech-and video-centric multi-modal interface to augment more conventional methods (e.g., mouse, stylus and touch), (2) an open and reconfigurable architecture supporting information gathering, and (3) a machine intelligent approach to analysis and management of heterogeneous live and stored sensor data to support collaboration. The system will transform how teams of people interact with computers by drawing on both the virtual and physical environment.
Page 1. Biologically Valid Jaw Movements for Talking Humanoid Robots SABRI GURBUZ ATR Humon Infom... more Page 1. Biologically Valid Jaw Movements for Talking Humanoid Robots SABRI GURBUZ ATR Humon Infomotion Science Labs, Kyoto 619-0488, Japan sabrig@atr.jp ... We want to extract and store the motor tapes by analyzing a human speaker's 782 Page 3. ...
Uploads
Papers by Sabri Gurbuz