An Introduction to the Log-Polar Mapping

An Introduction to the Log-Polar Mapping zyx Jorge M. Dias Helder Araujo zyxwvuts zyxwvuts Department of Electrical Eng. University of Coimbra Coimbra, PORTUGAL 3030 Department of Electrical Eng. University of Coimbra Coimbra, PORTUGAL 3030 Abstract One interesting feature of the human visual system is the topological transformation of the retinal image into its cortical projection. The excitation of the cortex can be approximated by a log-polar mapping of the eye’s retinal image. In this tutorial paper we describe the log-polar mapping and its main properties. 1 Introduction One interesting feature of the human visual system is the topological transformation ([Schwartz 84, Sandini et al. SO]) of the retinal image into its cortical projection. In our own human vision system, as well as in those of other animals, it has been found that the excitation of the cortex can be approximated by a log-polar mapping of the eye’s retinal image. In other words, the real world projected in the retinas of our eyes, is reconfigured onto the cortex by a process similar to log-polar mapping before it is examined by our brain [Schwartz 841. In the human visual system, the cortical mapping is performed through a space-variant sampling strategy, with the sampling period increasing almost linearly with the distance from the fovea. Within the fovea the sampling period becomes almost constant.This retinocortical mapping can be described through a transformation from the retinal plane ( p , 8 ) onto the cortical plane (log(p),6 ) as shown in figure 1. This transformation presents some interesting properties as scale and rotation invariance about the origin in Cartesian plane which are represented by shifts parallel to real and imaginary axis, respectively. This transformation is applied just on the non-foveal part of a retinal image. If ( x , y ) are Cartesian coordinates and ( p , B ) are the polar coordinates, by denoting z = x j y = pej’ a point in the complex plane, the complex logarithmic (or log-polar) mapping is Figure 1: Log-polar transformation. Any point (xi,yi) in the image plane (left) can be expressed in terms of ( p , 0) in the cortical plane (right) by (lnb(p),0 ) . zyxwvu zyxwvuts zyxwvutsr zy zyxwvutsrqp zyxwvu + 0 2 k n . However we constrain angle 8 to the range of [0,27r).This logarithmic mapping is a known conformal mapping preserving the angle of intersection of two curves. 1.1 Log-Polar Mapping and its Properties Log-polar mapping can be performed from regular image sensors by using a space-variant sampling structure similar to the structure proposed in [Massone et al. 851. This mapping is characterized by a linear relationship between the sampling period and the eccentricity p , defined as the distance from the image center. The figure 2 gives one example of these type of sampling structures. The spatial variant geometry of the sampling points is obtained through a regular tesselation and a sampling grid formed by concentric circles with Nangsamples over each circle. The number of samples for each circle is always constant and they differ by the arc 27r + w = ln(z). for every z Nang between samples. For a given N a n g ,the radius p,. of the circle i is expressed by and an equation of the type (1) # 0 where Real(w) = l n ( p ) and I m ( w ) = 139 0-8186-8058-X/97 $10.00 0 1997 IEEE zyxwv zyxwvutsrqpo zyxwvutsrq zyxwvutsrqp zyxwvuts zyxwvutsr zyx Figure 3: Example of images sampled by regular sampling structure and remapped by using a space-variant structure. The original image has 256 x 256 samples and the cortical plane has 71 x 60 samples. Figure 2: Graphical representation of the sampling structure. In this example the number of angular samples Nang = 60.In this scheme the sampling point is shifted half sample period between consecutive circles. with i = O..NciTcand pfovea> 0 and representing the minimum radius of the sampling circles. The transforrnation for discrete entries of cortical plane is performed by using the following expressions Figure 4: Graphical representation of a more simple sampling structure. The figure represents a structure with N a n g= 60 angular samples. (4) with A = N o n g For the example illustrated in figure 2 the base b is ' sampling circles and the expression tion is given by (5) For different types of sampling strategies (different bases b ) the concentric circles are always sampled with the period ori 27T = -( j Nang for the equa- odd(i) + 7) (9) where j = 0.. Naris. Results from this kind of transformation are ilustrated in figure 3.The intensity value in the cortical plane is obtained by the mean of the intensity values inside the circle centered at the sampling point (prir6 r i ) . That is the case of the images in figure 3. This space-variant sampling structure can be modified for a more simplified sampling structure as it is illustrated in Figures 4 and 5. This simplified structure does not use as many samples as the structure described before and it is useful t o speedup the algorithms based on this type of data sampling. This structure is similar to the structure described above. The spatial variant geometry of the sampling points is also obtained through a tessellation and a sampling grid formed by concentric circles with Naris samples over each circle. The number of samples for each circle is also constant and for a given N a n g ,the radius basis b is expressed by For the case where the base is expressed by (5) each sample covers a patch of the image corresponding to a circle with radius given by The value for p f o v e a could be chosen equal to the minimum sampling period t o cover all the image center without generate oversampling in the retinal plane. If we want to obey to this constraint then zyx In this sampling structure the angular sampling is shifted by half sampling period between successive 140 zyx zyxwv zyxwvutsrq zyxwvutsrq zyxwvu zyxwvutsr 1 I b = (1 b- + *) I zyxwvutsrq zyxwvutsr Reg. Samp. Log-Polar 262144 samples 6540 samples 262144 samples 1860 samples 9 Nang+TT N..---T Table 1: Different sampling schemes also require different storage in memory. Figure 5: Example of images remapped in logpolar using the simplified version of sampling. The original images have 256 x 256 samples and the cortical plane only have 20 x 60 samples. " tl + Nang r Nang - r with i = 0..N,irc and p f o v e a the minimum radius of the sampling circles. The radius pT of the circle is expressed by b= Figure 6: The log-polar mapping applied to regular patterns. (a) Applied to concentric circles in the zmage plane are mapped in vertical lines in the cortzcal plane. (b) Applied to radial lines in the zmage plane are mapped in horizontal lines in the cortzcal plane. The concentric circles are sampled with the period described by the expression (2) and each sample covers a patch of the image corresponding to a circle with radius given by results in similarly regular patterns in the other domain. From the figure 6(a) the concentric circles in the image plane become vertical lines in the cortacal plane. A single circle maps to a single vertical line since the constant radius T at all angles 8 of the circle gives a constant pc coordinate for all Bc coordinates. Similarly an image of radial lines which have constant angle but variable radius, result in a map of horizontal lines. These mapping characteristics are fundamental for some properties such as rotation and scaling invariance. Rotation and scaling result in shifts along the 8, and pc axis, respectively. For rotation invariance notice that all possible angular orientations of a point at given radius will map t o the same vertical line. Thus, if an object is rotated around the origin, between successive images, this will result in only a vertical displacement of the mapped image. This same result is valid for radial lines. As a radial line rotates about the origin, its entire horizontal line mapping moves only vertically. Scaling invariance is another characteristic of this log-polar mapping. From the figure 6(b) we seen that as point moves out from the origin along a radial line, its mapping stays on the same horizontal line mov- The value pfoveu could be chosen equal t o t,he minimum sampling period to cover all the image center without generating oversampling in the retinal plane. This constraint is expressed by Examples of images sampled with this structure are shown in Figure 5. The intensity value at each point of the cortical p l a n e are obtained by the mean of the intensity values inside the circle centered at the sampling point ( p r i , Ori). This image presents some gaps between the circles but a better result is obtained if the area around the sampling point is filled in. This simplified version of space variant structure needs less storage then the first sampling structure, as we can verify in the Table 1 1.1.1 Log-Polar Properties The log-polar mapping has number of important properties that make it useful as a sampling structure. The mapping of two regular patterns as shown in figure 6 141 zy t zyxw e lateral motion f m w d motion iateni motion Figure 8: The optical flow vectors for different types of translational motion. For lateral motion the optical flow vectors generate in the cortical plane, stream lines of vectors with the same orientation. For forward motion these lines are equal in all the plane. Figure 7: The effect of rotation and scaling with logpolar mapping. The original image in the left is rotated and scaled. The effect in the cortical plane is an image with similar shape with the edges at different position but equivalent t o a circular shift in the cortical plane. expressions we obtain ing from the left t o the right. The mappings of the concentric circles remain vertical lines and only move horizontally as the circles change in size. The images of figure 7 illustrate these two properties. The original image is rotated and scaled and the images correspondent to the cortical planes before and after the transformation are similar. The edges have similar shape at position equivalent t o a circular shift in the cortical plane. These properties were fundamental for the development of algorithms for pattern recognition [Reitboeck et al. 841, [Massone et al. 851. Another property is related with projection of the images when the sensor translates. The images of Figure 8 show the mapping of the optic flow vectors for different types of translational motion of the sensor. Notice that when the sensor translates in same direction as the optical axis the optical flow generated appears as vectors diverging from the image center. The effect in the cortical plane is a set of lines with vectors with the same orientation, as illustrated in Figure 8. Defining zyxw P < = lnb Pfovea zyxwvutsrq zyxwvu zy and y = 0, the relationship between the time derivatives of and p is given by < From ( 1 5 ) and using (17) we obtain the relationship between the motion field in Cartesian coordinates ( 5 ,y) and log-polar coordinates (<, +) as zyxwvu zyxwvu zyxwvutsrqp zyxw [ ] d cosy m [ -m sin" 7 *] [ ]. cosy Inb (18) The relative motion of the observer with respect to the scene gives rise to motion of the brightness patterns in the image plane. The instantaneous changes of the brightness pattern in the image plane are analyzed t o derive the optical flow field, a two-dimensional vector field ( U , U ) reflecting the image displacements. The optical flow value of each pixel is computed locally - that is, only information from a small spatiotemporal neighborhood is used t o estimate it. In general, it is not possible t o compute the true velocity of an image point just by observing a small neighborhood. Suppose that we are watching a feature ( a piece of contour or a line) at two instants of time and through a small aperture smaller than the feature see figure 9. 1.2 Normal Optical Flow on Log-Polar The space variant resolution and sampling exhibits interesting properties for the optical flow. In this point we study some of these properties of the optical flow. The relate the optical flow field in log-polar coordinates with the 2D velocity field in Cartesian coordinates let us write e, = zyxwvutsr where b, x, stand for the derivatives with respect to time. Substituting the partial derivatives by their 142 zyxwv zyxwv zyxwvutsrqponm zyxwvutsrqp zyxwvutsrqponmlkjih zyxwvutsrqponmlkjih Final edge pasition (e, onto image point y) at time t and onto the image point (6 S(”y Sy) at time (t d t ) we obtain the optical flow constraint equation [Horn et al. $11, + \ \ + + which relates the flow ( U , U) to the partial derivatives ( I t , I T ,I t ) of the image I . From this constraint alone, without making any additional assumptions, we can only compute the normal flow u n l equivalent to projection of optical flow on the gradient direction: Candidate motions of a p i n 1 in Ule edge Figure 9: A line feature or contour observed through a small aperture at time t moves t o a new position at timet +&. In the absence of knowledge of camera motion, when we are looking at a viewpoint-independent edge in an image through an aperture, all we can say about the evolving image position of an indistinguishable point along the edges is that this position continues to lie somewhere along the evolving image of the edge. un = -It- 1 IlAIll. Let I ( 6 ,y,t ) denote the image intensity, and consider the optical flow field v = (U,.) and the motion field v, = ( u m , v m ) at the point ( t , y ) , where the normalized local intensity gradient is n = ( I t ,1 7 ) / d m The . normal motion field at point (<,y)is by definition Watching through this small aperture, it is impossible to determine where each point of the feature has moved to. The only information directly available from local measurements is the componcnt of the velocity which is perpendicular t o the feature, the normal flow. The component of the optical flow parallel t o the feature can not be determined. This ambiguity, is known as the aperture problem and exists independently of the technique employed for local estimation of flow. However in cases where the aperture is located around an endpoint of a feature, the true velocit,y can be computed, because t,he exa.ct, locatlion of the endpoint at two instants of time can be computed. Thus, the aperture problem exists in regions that have strongly oriented intensity gradients, and may not exist at locations of higher-order intensity variations, such as corners. Any optical flow procedure involves two computational steps. In the first, assuming the local conservation of some form of information, only local velocity is computed. In a second step, in order to compute the other component of the optical flow vectors, additional assumptions have to be made. The approach introduced by [Horn et al. 811 is based on the assumption that for a given scene point the intensity I at the corresponding image point remains constant over a short time instant. This corresponds to a brightness constancy assumption, $ = 0, that gives a relationship that can be used t o estimate the flow parameters directly from the spatial and temporal grey-level gradients. If a scene point projects If we approximate the differential by its total derivative we get a relation between the equatioris (20) arid (21) which shows that the two fields are close to equal when the local image intensity gradient A I is high. This confirms that the normal flow is a good measurement of the normal motion field in locations where the intensity gradient exhibits high magnitude [Fermuller et al. 951. References zyxwv [Aloimonos et al. 881 Y. Aloimonos, Y. Weiss, A. Bandopadhay, ”Active Vision”, International Journal of Computer Vision, vo1.7, 1988, pp. 333356. [Bailey et. a1 901 J. G. Bailey and R. A. Messner, “Docking target design and spacecraft tracking system stability” , in SPIE - Intelligent Robots and Computer Vision VIII: Algorithms and Techniques, Vo1.1192, pp. 820-831, 1990. [Bajcsy 881 R. Bajcsy, ”Active Perception”, Proceedings of the IEEE, vo1.76, August 1988, pp. 9961005. 143 zyxwvutsrqpo zyxwvuts zyxwvutsrq zyxwvuts Mount”, Technical Report TR 1/95, LIRA Lab, University of Genova, Italy, February 1995. [Bederson et al. 921 B. B. Bederson, R. S. Wallace, and E. L. Schwartz, “A miniaturized active vision system”, in Proc. 11th IAPR International Conference on Pattern Recognition Vol. IV, Conference D: Architectures for Vision and Pattern Recognition, pp. 58-61, 1992. [Reitboeck et al. 841 H. Reitboeck and J. Altmann, ” A model for size- and rotation- invariant pattern processing in the visual system” , Biological Cybernetics, Vol. 51, 1984, pp. 113-121. [Bederson et al. 941 B. B. Bederson, R. S. Wallace, and E. L. Schwartz, “A Miniaturized SpaceVariant Active Vision System: Cortex-I”, in Artificial Neural Networks for Speech and Vision, Richard J. Mammone, ed., pp. 429-456, Chapman and Hall, 1993. [Sandini et al. 801 G. Sandini, and V. Tagliasco, ” A anthropomorphic retina-like structure for scene analysis” , Computer, Graphics and Image Processing, Vol. 14, 1980, pp. 365-372. [Hacisalihzade et al. 921 S. Hacisalihzade, L. Stark, J. Allen, ”Visual Perception and Sequences of Eye Movement Fixations: A Stochastic Modeling Approach”, IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-22, No. 3, May/June 1992, pp. 474-48 1. [Bederson et al. 951 B. B. Bederson, R. S. Wallace, and E. L . Schwartz, “A miniaturized space-variant active vision system: Cortex-I” , Machine Vision and Applications, vo1.8, pp.101-109, 1995. [Carpenter 881 H. Carpenter, Movements of the Eyes, London, Pion Limited, 2nd edition. [Schwartz 841 E. Schwartz, ”Anatomical and Physiological Correlates of Visual Computation from Striate to Infero-Temporal Cortex” , IEEE Trans. on Systems, Man, and Cybernetics, Vol. SMC-14, No. 2, March/April 1984, pp. 257-271. [Coombs 92b] D. Coombs and C. Brown, ”Real-time smooth pursuit tracking for a moving binocular robot”, in Proc. 1992 IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, pp. 232 8 , Champaign, USA. [Spiegel et al. 891 J . V. der Spiegel, G. Kreider, C. Claeys, I. Debusschere, G. Sandini, P. Belutti, and G. Soconi, “A foveated retina-like sensor using CCD technology”, in Analog VLSI Implementations of Neural Systems, Carver Mead, ed., Boston, Kluwer, 1989. [Fermuller et al. 951 C. Fermuller, Y. Aloimonos, ”Qualitative Egomotion” , International Journal of Computer Vision, vo1.15, 1995, pp. 7-29. zyxwvutsr [Dias et al. 951 J . Dias, C. Paredes, I. Fonseca, J . Batista, H. Aracjo, A. de Almeida, ”Simulating Pursuit with Machines - Experiments with Robots and Artificial Vision”, in 1995 IEEE Int. Conf. on Robotics and Automation, Nagoya, Japan, May 1992. [Wallace et al. 941 R. S. Wallace, P.-W. Ong, B. B. Bederson, E. L. Schwartz “Space Variant Image Processing”, Int. Journal of Computer Vision, Vol. 13:1, pp.71-90, 1994. [Horn et al. 811 B. Horn and B. Schunk, ”Determining optical flow”, Artificial Intelligence, Vol. 17, pp. 185-204, 1981. [Weiman 881 C. F. Weiman, “Exponential sensor array geometry and simulation” , in Proc. SPIE Conf. on Pattern Recognition and Signal Processing, Vol. 938, pp. 129-137, Orlando, 1988. [Manzotti et al. 941 R. Manzotti, R. Tiso, E. Grosso, G. Sandini, ”Primary Ocular Movements Revisited” , Technical Report 7/94, LIRA Lab, University of Genova, Italy, November 1994. [Weiman 901 C. F. Weiman, “Video compression via log-polar mapping”, in Proc. SPIE Conf. on Real Time Image Processing I I , Vol. 1295, pp. 266-277, Orlando, 1990. [Massone et al. 851 L. Massone, G. Sandini, V. Tagliasco, ”Form-Invariant Topological Mapping Strategy for 2D Shape Recognition”, Computer Vision, Graphics, and Image Processing, Vol. 30, 1985, pp. 169-189. [Weiman 941 C. F. Weiman, Log-Polar Binocular Vision System-Final Report, SBIR Phase I1 Contract #NAS 9-18637, T.R.C., Danbury, December 1994. zy [Panerai et al. 951 F. Panerai, C. Capurro, G. Sandini, ”Space Variant Vision for an Active Camera 144