surface painting

Painting Detail Nathan A. Carr John C. Hart University of Illinois, Urbana-Champaign Static Conformal Atlas Dynamic MMA Figure 1: A static conformal atlas (left) limits the precision of surface painting, whereas a multiresolution meshed atlas (right) dynamically resizes its charts during the painting process to better sample the texture signal, especially in areas of fine detail. Abstract Surface painting is a technique that allows a user to paint a texture directly onto a surface, usually with a texture atlas: a 1:1 mapping between the surface and its texture image. Many good automatic texture atlas generation methods exist that evenly distribute texture samples across a surface based on its area and/or curvature, and some are even sensitive to the frequency spectrum of the input texture. However, during the surface painting process, the texture can change non-uniformly and unpredictably and even the best atlases are static and can thus fail to reproduce sections of finely painted detail such as surface illustration. We present a new texture atlas algorithm that distributes initial texture samples evenly according to surface area and texture frequency, and, more importantly, maintains this distribution as the texture signal changes during the surface painting process. The running time is further accelerated with a novel GPU implementation of the surface painting process. The redistribution of samples is transparent to the user, resulting in a surface painting system of seemingly unlimited resolution. The atlas construction is local, making it fast enough to run interactively on models containing over 100K faces. Keywords: Mesh parametrization, face clustering, texture atlas, 3D painting. 1 Introduction In computer graphics, a texture atlas, provides a convenient mapping from a surface embedded in 3-D into a planar texture domain. Many methods now exist for automatically generating a good quality texture atlas for a surface in 3-D, e.g. [Maillot et al. 1993; Lévy and Mallet 1998; Lévy et al. 2002; Sheffer and Hart 2002]. These techniques evenly distribute texture samples across the surface based on the area or curvature of a known, fixed surface, or the frequency spectrum of a fixed, known texture. But surface area and curvature are dynamic during the modeling process, and the texture frequency spectrum changes during the surface painting process. This paper focuses on the problem of constructing and maintaining a texture atlas that efficiently and effectively distributes its texture samples to accurately represent a dynamic signal on a dynamic surface. The resolution of a surface painting is often heterogeneous, with fine details in a few localized regions of the object surface. A good static texture parameterization that distributes its samples uniformly across the surface can wash out fine details as demonstrated in Figure 1. A texture atlas that distributes texture samples based on the spectrum of the texture, e.g. using the signal-stretch metric [Sander et al. 2001], could accurately represent the color signal after application of all of the paint strokes. But during the 3-D painting process, the surface signal changes with each new stroke. One could recompute the signal-stretch metric atlas per stroke, but its computation would create unreasonable delays during the painting process. Octree textures forego surface parameterization altogether, storing color in an adaptive volumetric representation [Benson and Davis 2002; Debry et al. 2002]. Octree textures can adapt their resolution to the fine detail of surface painting and are fast enough to adapt per-stroke. However, octree textures are not well-suited to capitalize on the acceleration and anti-aliasing benefits of modern graphics hardware, and they require the surface geometry to remain static. We have developed a method for texture re-parameterization that accurately represents heterogeneous detail and is fast enough to interactively update the parameterization as the surface signal changes. Our algorithm is based on the multi-resolution meshed texture atlas [Carr and Hart 2002], reviewed in Sec. 3. This atlas uses a quaternary tree to identify a mesh cluster hierarchy with a quadtree partitioning of the texture, such that each cluster at a given level in the hierarchy receives the same number of texture samples. One can thus control the distribution of texture samples by the relative sizes of clusters that share the same corresponding level in the hierarchy. The main contribution of this paper, as detailed in Sec. 4, is a new method for quickly adapting the multi-resolution meshed texture atlas to accurately sample a recently changed surface signal. A change in the surface signal can cause the cluster hierarchy to fall out of balance according to a given texture sample distribution metric. We rebalance the hierarchy, reassigning more samples to surface regions of greater signal importance. Additional contributions include the application of the new rapid texture re-parameterization to surface painting, and a novel implementation of the surface painting process on the GPU, described in Sec. 5. Sec. 6 supports our claims with examples, results and direct comparison with other parameterizations and texture atlases. Sec. 7 summarizes the method and its limitations, and discusses directions for further investigation. 2 tributes texture samples with each new stroke, and does not suffer from changing the pose of the object. Morover, the charts of a multiresolution meshed atlas do not overlap and their atlas covers the entire texture space leading to a more efficient utilization of available texture samples to reconstruct the painted signal. 3 The Multiresolution Meshed Atlas Following Carr and Hart [2002], we construct a multiresolution meshed atlas (MMA) to evenly distribute texture samples according to a given metric across the surface triangle mesh M of a 3-D object. (a) Previous Work (b) Several recent texture atlas generation techniques have focused specifically on distributing texture samples to reduce texture aliasing. Sander et al. [2001] developed an atlas for texturing progressive meshes that inhibited texture aliasing across different mesh resolutions. Carr and Hart [2002] used an area-balanced hierarchical face clustering to create an atlas to support real-time procedural solid texturing. Both techniques strived to distribute samples evenly across the surface, and both supported MIP mapping. Our atlases are generated using a variation of [Carr and Hart 2002] because it uses more of the available texture samples. Sander et al. [2002] developed a signal-stretch metric that combines both surface area and surface signal bandwidth. Their metric guided texture atlas construction to better sample high frequency texture areas, but runs too slowly to support interactive rates. Balmelli et al. [2002] developed a method that reduces texture space by optimizing texture coordinate locations using wavelet analysis on the texture map. Sec. 5.2 shows how the GPU can quickly perform an edge-detecting high-pass filter of a texture by computing its gradient magnitude. Our work focuses on the redistribution of texture samples to recover dynamic surface signals. Sloan et al. [1998] also noticed the weakness of a static assignment of texture coordinates, and formalized the notion of an importance map to guide the reassignment of texture coordinates based on the surface signal. Alliez et al. [2002] interactively reparameterized a surface by halftoning the importance map to discretize it into a new set of surface samples evenly distributed according to the importance map. Their method was designed for semi-regular meshing and displacement mapping. Moreover, their halftoning scheme constructs the opposite of an atlas; it distributes mesh vertices in the texture map whereas an atlas distributes texture samples on the mesh. Our goals for a paint system are most closely aligned with those of Igarashi and Cosgrove [2001]. They stored the paint strokes image that occurred for each object pose as a separate charts packed into a texture atlas. The method clipped each chart to only include the region with paint strokes which allowed it to reproduce fine detail. However, the accumulation of sometimes overlapping charts with each new pose led to the wasteful allocation of multiple texture samples for the same surface location. Our approach redis- (c) Figure 2: A multiresolution meshed atlas of a cow. Each node in the tree (a) corresponds to a cluster of triangles in the model (b) and a square region in the texture domain (c). Each node’s cluster/region is the union of its children’s clusters/regions. The MMA is based on a hierarchical face clustering where each node in the tree represents a simply connected cluster of mesh triangles. A node’s children represent the disjoint partitioning of a cluster into simply connected subsets. Leaf nodes in this hierarchy correspond to simple clusters of a small number of triangles. Construction. The MMA constructs a hierarchical face clustering where each non-leaf node has four children. This quaternary tree structure allows the cluster hierarchy to be mapped into a texture image quadtree. The entire mesh, represented by the root of the cluster hierarchy, maps to the entire texture image. Each of the root’s four children represent four clusters that each map to the four quadrants of the texture image. This subdivision continues down to the leaf clusters, which map to small subquadrants of the texture image using, for example Floater’s [1997] shape preserving convex mapping method. Coverage. The goal of a texture atlas is to reproduce a texture signal across a surface as accurately as possible. An important but often overlooked principle of texture atlas construction is coverage: to use as many of the available texture samples as possible. By design, the MMA utilizes all of the texture domain samples. MIP-Mapping. The MMA supports MIP-mapped texture antialiasing. At a given level of the MIP-map, four subquadrants will merge into a single quadrant. This quadrant corresponds to a cluster, and its four subquadrants correspond to a connected collection of four subclusters. The layout of these subquadrants will likely not match the layout of their corresponding clusters in the surface mesh, but this does not affect MIP-mapping because the subquadrants do not interact until their contents are averaged into a single color for the quadrant. Packing. The MMA avoids the constraints of mesh topology when constructing an atlas, freely packing square texture regions into the quadtree hierarchy and disregarding the configuration of neighboring clusters. The only constraint is that clusters should be connected, though clusters appear best when they are disk-shaped with non-jagged boundaries. This disregard for shared boundaries would ordinarily lead to seam artifacts in the textured mesh, but the MMA avoids such artifacts by surrounding leaf-node subquadrants with a half-pixel gutter ensuring texture samples are not interpolated from neighboring subquadrants. Thus the MMA works equally well on manifold and non-manifold meshes of arbitrary genus and orientability. Texture Sample Distribution Metric. We denote the texture sample distribution metric as the value δn assigned to every node n in our binary tree. This metric is proportional to the number of texture samples its corresponding cluster should receive. Each of the clusters at the same level of the hierarchy will receive the same number of texture samples, so the tree should be balanced such that the metrics of nodes at the same level are approximately equal. The texture sample distribution metric of a parent node is the sum of the metrics of its children. If the texture sample distribution metric is surface area, then clusters at the same level will be the same size. The relative sizes of same-level clusters can vary under different metrics. Under a signal sensitive metric, e.g. signal-stretch [Sander et al. 2002], clusters with higher-frequency texturing are smaller than clusters with lower-frequency texturing at the same level of the hierarchy, thereby distributing more texture samples to triangles with higherfrequency texturing. The children of a non-leaf node evenly divide the node’s texture samples between the node’s child clusters. Given a texture sample distribution metric, the cluster corresponding to a non-leaf node must be subdivided into two clusters of approximately equal metric. Optimal metric balancing of a hierarchy is NP-hard. Techniques for balanced graph partitioning with minimal cuts such as Metis [Karypis and Kumar 1998] provide good approximations to a balanced hierarchy in feasible times (several minutes for meshes with 10K triangles) but are too slow for interactive applications. Our goal is to find a method for maintaining a balanced hierarchy for a dynamic surface metric at interactive rates. 4 Rapid Reparameterization This section describes the main contribution of this paper, a rapid method for maintaining the hierarchy behind a multi-resolution meshed atlas. It first reviews the steps for constructing the MMA datastructure, then addresses the details for its rapid maintainance during the painting process. chart’s u,v coordinates within the texture domain. During surface signal changes, each chart’s importance is updated and the MMA tree is re-balanced. This effectively redistribes samples across the surface of the mesh. At run-time when the signal over the surface changes, not all of these steps need be applied. For example, the simplest scheme is to perform only chart re-balancing following each change in surface signal. Better quality is achievable by re-parameterizing each patch, or even re-clustering. This provides a performance versus quality trade-off. Our current method adaptively re-computes both parameterization and balancing during a single change. 4.1 Chart Formation We cluster mesh faces into charts using the algorithm of Sander et al. [2003]. This algorithm adapts Lloyd’s [1982] vector quantization algorithm to face clustering by replacing the Euclidean distance metric with the shortest path between points computed using Dikstra’s shortest path algorithm. The cost function between adjacent faces F and F 0 (connected vertices in the dual graph) is given by: cost(F, F 0 ) = (λ − (Nc − NF 0 ))||PF − PF 0 ||2 (1) where PF and PF 0 are the location of the faces centers, Nc is the average normal of the chart, and NF 0 is the normal for face F 0 . We chose λ = 2.0 to encourage the growth of compact patches that parameterize better into squares. Figure 3 shows results from the chart formation process. Corner triangles (triangles that are connected to a cluster by a single edge) greatly impact sample distribution when parameterizing onto a square domain [Gu et al. 2002]. To avoid these corner triangles from being degenerate in the parameter domain they are mapped to one of the four corners of the rectangular parameter space. A cluster containing more than four corner triangles cannot be mapped without forming a degeneracy. To avoid this problem, we start by eliminating as many corner triangles as possible. Any corner triangle that shares two edges with the same cluster may be eliminated by reassigning the triangle to that cluster. Corner triangles that are adjacent to three different clusters are greedily assigned to the cluster with the fewest number of corner triangles. If both clusters have the same number of corner triangles, the corner triangle is assigned to the cluster that maximizes the distance between any two corner triangles within a cluster. This keeps the corner triangles assignments spread out within a cluster lead to a better parameterization. Figure 3, shows an example of the quality clustering that may be achieved using Sander’s algorithm followed by corner triangle reduction. Chart Formation (4.1). This step breaks the mesh into a number of charts. Each chart is a collection of simply connected triangles that form a surface that is parameterizable into a plane with no overlap (homeomorphic to a disc). Our current implementation treats the chart formation step as an off-line pre-process only utilizing the geometric information of the mesh when forming clusters. Chart Parameterization (4.2). This step maps each chart into the square u, v ∈ [0, 1]2 domain. A surface signal based metric is used to guide the parameterization process, providing a better distribution samples within a chart. When the surface signal over a chart changes, the chart is re-parameterized to take into account the new surface information. Chart Balancing (4.3). A scalar sample distribution metric δ (based on signal frequency) is maintained for every chart. The collection of charts are placed into a balanced MMA tree hierarchy, where the charts form the leaves of the tree. The location of a chart in the MMA tree hierarchy determines a scale and offset for the Charted Bunny (a) (b) Figure 3: Chart clustering before (a) and after (b) triangle corner reduction. 4.2 Chart Parameterization Charts may be statically parameterized into the square u, v ∈ [0, 1]2 domain using any number of methods [Sander et al. 2001; Floater 2003; Desbrun et al. 2002]. Methods that solve a linear system to determine u, v parameter locations, provide a fast means for flattening a mesh. Some linear methods guarantee the absence of fold-over [Floater 2003]. In practice we have found that linear parameterizations are not sufficient to adequately control distortion and sample distribution. As a result we turned to the non-linear geometric L2 stretch metric proposed by Sander et al.[2001]. A Signal Biased Shape Metric. The L2 metric used is purely geometry based. Let Pt = (p1 , p2 , p3 ) and Ut = (u1 , u2 , u3 ) denote the 3D and 2D coordinates of triangle t respectively, and let A(Pt ) denote the area of the triangle defined by coordinate points Pt . We wish to bias this metric to account for the varying signal importance across the surface of our mesh. This can be done by applying a unique transformation St to each triangles 3D coordinates upon evaluation of the error metric: (L2 (M ))2 = P t∈M A(St (Pt ))(L2 (St (Pt ), Ut ))2 P t∈M A(St (Pt )) (2) In our implementation, we use the scalar per-triangle sample distribution metric δt to form the uniform scaling matrix St . Application of non-uniform scaling could be applied to better represent triangles with strongly anisotropic surface signal frequencies. Solving the System. The L2 stretch metric leads to a non-linear minimization problem. Solving such systems may be expensive and costly. We start the optimization process by providing an initial starting guess using the linear solve weighting scheme proposed by Tutte [Floater 1997]. We chose this weighting scheme for two reasons: first, it guarantees no fold-overs occur and secondly, it leads to a sparse symmetric system that can be rapidly solved using the pre-conditioned conjugate gradient method. Rather than performing random line searches on local neighborhoods by optimizing a single vertex at a time as done in [Sander et al. 2001], we attempted a global approach to the optimization problem. To simplify the computation of the analytic gradient, we minimize the square of the L2 metric. Let Uˆn denote a vector containing the locations of the freely moving u, v coordinates of our mesh. This includes all of the u, v vertex location on the interior of the parameterization, in addition to the freely moving u or v coordinates from vertices on the boundary. We compute the analytic gradient (denoted ∇((L2 (M ))2 )) of the error metric (L2 (M ))2 with respect to the freely moving u, v coordinates Uˆn . We use the conjugate gradient method adapted for non-linear optimization as detailed by Fletcher and Reeves [1987] to search for a minimum. Chart Re-parametrization. When the signal changes, each impacted chart may be re-parameterized to move more samples to locations where they will better reproduce the new signal. We use the existing parametrization as a starting point for the optimization process, which converges rapidly especially when the change in surface signal is small. 4.3 Chart Balancing Each parameterized chart is placed into a balanced MMA hierarchy similar to that presented in section 3. In this case the leaves of the MMA tree are charts rather than triangles. Rather than working with a quaternary tree structure as suggested in 3, we work with a binary tree for simplicity. Odd levels of the binary tree correspond to mip-map subquadrants. In the case a leaf terminates at an even level of the tree hierarchy, the chart is mapped into a rectangular region corresponding to half of a mip-map subquadrant. Two primary goals are present in the construction of the MMA tree. First the sample metric should be evenly balanced at every node of the tree. Secondly, for correct mip-mapping results, clusters contained within a subtree should be spatially proximate. A subtree, however, need not be comprised of clusters that are simply connected. The location of a chart within the MMA tree determines a unique rectangular region of the texture atlas for the chart. Each chart’s texture coordinates are scaled and translated (from the [0, 1]2 domain) during run-time to their appropriate location in the texture atlas. We form the MMA hierarchy by combining two heuristic approaches: a top down divisive approach, and a bottom up agglomerative approach. We let the sample distribution metric δ of a cluster be the area weighted mean of the δ values of its triangles. The average reflects the effectiveness of the parametrization redistributing samples within a chart. We start forming our tree in a divisive manner using recursive coordinate bisection [Simon 1991]. The center of mass for each cluster is computed, yielding a single 3D coordinate value per-cluster. The points are used to compute a 3 tensor matrix whose smallest eigenvector determines an axis where the point have greatest variance. The axis is used to define the orientation of a partitioning plane. The plane is swept along this axis until the sum of the chart sample metrics δn on either side of the plane are nearly equal. Using this plane, the points (clusters) are divided into two spatially coherent sets with nearly equal metric values. This process may be recursively applied to each sub-group to construct the full MMA tree. To find the axis we form the symmetric 3 × 3 inertial tensor matrix I from the cluster centers. The splitting axis is given by the smallest eigenvector of the inertial tensor matrix. Given a collection of n clusters with coordinate values xp , yp , zp ∈ R3 , the inertial tensor matrix I is given by   Ixx Ixy Ixz I =  Iyx Iyy Iyz  (3) Izx Izy Izz where Ixx = n(var(yi ) + var(zi )), Iyy = n(var(xi ) + var(zi )), Izz = n(var(xi ) + var(yi )), Ixy = Iyx = −n cov(xi , yi ), Iyz = Izy = −n cov(yi , zi ), Ixz = Izx = −n cov(xi , zi ), where var(), cov() indicate the sample variance and covariance of the n cluster centers. The inertial tensor is construced in linear time, but since its size is always 3 × 3, its eigenvalues are found in constant time. The recursive coordinate bisection approach produces good results in forming balanced partitions at the top of the MMA tree. Near the leaves of the final tree, the algorithm fails to find a good partitioning due to the small set size. To prevent poor balancing near the leaves of the tree, we detect when the recursive bisection algorithm divides a set into subsets that fall below a threshold size of 16 clusters. A bottom-up approach is then used for forming that subtree. The agglomerative algorithm joins together clusters/sub-trees that are both of equal sample metric importance and spatially proximate to form larger subtrees. All of the clusters in a sub-tree are placed in a priority queue based on minimum δ. At each iteration, a cluster c0 is removed from the top of the queue. A search is performed on the elements in the queue to find the best cluster c1 with which to pair c0 with. We choose the cluster whose center is closest c0 , and whose sample metric δc1 is within 30% of δc0 . If no matching cluster can be found, we pair c0 with the next cluster on the top of the queue yielding the best balance. We remove c1 from the queue, and group c0 and c1 into a subtree/super-cluster represented by a parent node p. The node p is inserted into the queue for the next iteration. The algorithm continues until the group of clusters has been combined into one subtree. level 1 level 2 level 3 Figure 4: Tree formed on the bunny model using recursive bisection with inertial partitioning at the top levels of the tree, and agglomerative approach for constructing lower subtrees. Figure 4 shows the top three levels of the MMA tree formed by this process. Levels 1 and 2 clearly show the location of the splitting planes used to bisect regions. At level 3 the agglomeration strategy was executed on some of subtrees as evidenced by the loss of some proximity between neighboring clusters. 5 Surface Painting We used the rapid atlas rebalancing technique described in Section 4 to manage the resolution of a texture atlas used by a 3-D surface painting system. Ideally, a paint package should hide the underlying parametrization from the user [Hanrahan and Haeberli 1990], but realistically the parameterization is important because it affects the resolution of the paint strokes that are textured onto the surface. At the start of the painting process, we initialize our mesh parametrization to distribute texture samples according to surface area, since no additional information is available. During the painting process, we periodically (e.g. after each stroke) analyze the spatial frequency of the current texture, and reorganize the texture atlas to provide more samples to higher frequency regions of the texture, all at interactive rates. We also found many aspects of the managed resolution paint system to be amenable to GPU acceleration. An additional contribution of this paper is that it shows how to implement many of the surface painting tasks (e.g. stroke rendering and texture signal analysis) directly on the GPU using multipass processing, as diagrammed in Fig. 5. 5.1 Stroke Rendering Each collection of paint strokes applied in the same object pose are rendered directly into our texture map using GPU hardware. We accomplish this by first rendering the strokes into an empty texture called the stroke buffer. The alpha channel of the stroke buffer indicates the presence of stroke information. A depth-buffer of the object in its given pose is also rendered into a depth texture which is used later to prevent paint from being applied to non-visible portions of the model. We then resample the strokes from screen coordinates into the texture atlas. This resampling occurs by rendering the surface geometry with a simple vertex shader that first transforms the modelspace position into viewing coordinates, then swaps the vertex’s position (in viewing coordinates) with its texture coordinates. Rasterization of the triangles under these new coordinates resamples the stroke buffer into the texture atlas. The OpenGL shadow buffer extension compares each fragment’s depth with a texture lookup into the depth map to prevent surfaces that were not visible in the pose from receiving paint. The alpha channel of the stroke buffer ensures that only the strokes overwrite the existing texture atlas image contents. This process is repeated for all poses where strokes occurred, in order of stroke occurrence, until all strokes have been rasterized into the texture map. At some point, the number of poses may become quite large, slowing the stroke rendering phase below acceptable interactive update rates. To avoid this problem we allow the strokes to be flattened into a base texture atlas so the individual stroke buffers and pose information can be discarded. The resolution of these strokes, once flattened, is thereby fixed and can no longer be improved by the dynamic resolution management algorithm. They do, however, impact the future frequency analysis and texture atlas layouts. 5.2 Texture Frequency Analysis We next analyze the area-balanced texture atlas to find regions in need of additional samples. These undersampled regions occur in the atlas wherever the texture color gradient becomes too large. We use a four-tap gradient magnitude filter to find undersampled regions, implemented as a GPU pixel shader to support interactive processing speeds. The output texture of this filter is half the horizontal and vertical resolution of the input texture. We execute the pixel shader by rendering a single quadrilateral extending across the pixels of the filter output. This quadrilateral is textured with the input, such that each pixel of the output falls at the center of its 2 × 2 support of pixels in the input. The pixel shader, for each fragment, fetches four samples (taps) from the input texture. Each of these four taps falls directly between two adjacent input texture samples, and so its bilinearly interpolated value will be the mean of these two samples. The pixel shader arithmetic uses these four taps to construct a central difference estimate of the horizontal and vertical derivatives, and outputs, as the fragment luminance, the magnitude of the gradient as the sum of the absolute values of the two central differences. When applied to our texture, this yields an importance map, encoding the areas of our texture with high frequencies. The support of the gradient magnitude filter is only 2 × 2 and aligned with the MIP-map such that it does not interfere with (nor is it sensitive to) the structure of the parameterization. The atlas rebalancing algorithm, described earlier in Section 4, is implemented on the CPU, but requires the importance map generated by the GPU. The lower bandwidth available for the “readback” of data from the GPU is a bottleneck of current graphics architectures. Though we only perform one readback per texture analysis, it can be quite costly. We can reduce this cost by downsampling the importance map on the GPU (via the automatic mip-map generation available in modern graphics hardware) to a size (256 × 256) that can be read back more efficiently. 5.3 Tree Rebalancing and Reparametrization The intensities in our importance map encode areas that are undersampled. As described in Section 3, each node in the face cluster tree corresponds to a rectangular region of the texture domain, in this case the importance map. The distribution metrics stored in the cluster hierarchy nodes are initialized to a uniform area distribution. Since the importance map is non-negative, we use non-zero importance values to increase the distribution metrics of corresponding nodes in the cluster hierarchy. Sampling is performed on the importance map to assign every triangle a sample metric δ. We choose δ for a triangle to be the maximum pixel intensity over its covered area in the importance map. Chart sample metrics are are then computed as described in Sec. 3. Charts that contain triangles with updated metric values are re-parameterized. Finally, the MMA tree is reconstructed to ideally balance the placement of samples. Area-Balanced MMA Stroke Buffer Model Render Stroke Buffer into Atlas Importance Map Gradient Filter GPU MIP Re-Render Stroke Buffer into New Atlas MMA Tree Re-Balance & Re-Parameterize New TexCoords Signal-Balanced MMA Depth Buffer Read Pixels CPU Painted Image Render Model with MMA Texture Figure 5: The GPU-accelerated surface painting process. The strokes are rendered into the MMA by swapping vertex positions and texture coordinates, and using the stroke buffer as texture (Sec. 5.1). An edge detector pixel shader converts the MMA into an importance map (Sec. 5.2). The CPU reads back the MMA and a downsampled importance map, rebalances and reparameterizes the MMA to assign more texture samples to strokes (Sec. 5.3), and loads the new texture map to render the model with painted detail. 5.4 Resampling a Base Texture We can use the GPU to resample a base texture to coincide with the new parametrization. This resampling is performed by first loading the base texture, and rendering the surface geometry vertices using the new atlas coordinates as position, and its previous basetexture parameterization as texture coordinates. This rendering redistributes the base texture image into a multiresolution mesh atlas image that is consistent with the new parameterization. If the base texture is already a multiresolution meshed atlas, then we can take advantage of its MIP-map. We can use MIP-mapping to inhibit minification aliases that may occur when resampling a previous atlas image into a new atlas image, particularly in regions that will now receive fewer texture samples. Resampling is performed from the original base texture, to avoid the accumulation of error that can result from successive resampling. 6 method still preserved the salient features of the surface information and required only a fraction of the computation time. We tested our painting system on a wide variety of models ranging in size from 6k to 100k faces. All tests were performed on a 2.08GHz Athlon, with a QuadroFX 2000 graphics card. (a) Authalic (b) Conformal (c) Mean Value (d) L2 Stretch Results Figure 6 compares the quality of parametrization achievable by our method with that of other popular schemes. These existing methods do not adapt to surface signal information so we examined sample distribution based on geometric properties of the mesh. We use the relative scale metric as define in Carr [2002], which relates the ratio of normalized triangle surface area in 3D to its area in texture space. Methods (e.g. (a)-(c)) lead to large over-sampled and undersampled regions of texture space. The non-linear L2 metric with a single chart (d) leads to much more equitable sample distribution than the linear methods. Further improvement is made by breaking the mesh into multiple charts (e), however, the mesh is undersampled everywhere since many texels are left unused. The MMA map (f) further reduced the relative scale error by utilizing all available samples. The maximum relative scale error for any triangle in the mesh is slightly greater than that of (e) since charts were forced into rectangular domains, rather than taking natural boundaries(f). Figure 7 shows our dynamic parametrization scheme in comparison with the signal stretch metric detailed in [Sander et al. 2002]. We found that the signal stretch metric provides a better signal reconstruction for this particular model and resolution, however, our (f ) MMA L2 oversampled (e) L2 Multi-Chart undersampled 0.5 1.0 2.0 Figure 6: Plots showing the area distortion for a number of parametrization methods. Single-chart linear solves (a)–(d) [Desbrun et al. 2002; Floater 2003], non-linear geometric stretch metric with single chart and multiple charts (d)-(e) [Sander et al. 2001], and Dynamic MMA mapping (f). and texture atlas re-sampling. We suspect better texture sample distribution metrics exist, including ones that can recognize the principal components of surface signals to anisotropically distribute texture samples. It may also be worth looking at the acceleration of existing metrics that currently evaluate too slowly for interactive use. We are also examining adaptive approaches for chart formation that will allow re-parametrization of meshes undergoing free-form deformations. Such adaptive chart generation methods could also benefit from the knowledge of surface signal anisotropy, forming charts that better represent surface signal when parameterized. (a) Signal Stretch 128x128 (b) MMA 128x128 (c) MMA 256x256 Figure 7: Signal Stretch versus Dynamic MMA mapping. model bunny parasaur head faces 69K 7.6K 7.2K param (sec.) 9.2 1.04 1.03 re-param (sec.) 0.072 - 1.10 0.03- 0.186 0.034 - 0.183 Table 1: Algorithm performance times for various models. Param time includes clustering, chart parameterization, and MMA tree construction. Re-param time includes analyzing the importance map, chart parameterization update, and MMA tree re-construction. One of the freely chosen parameters of our system is the number of charts per mesh. With too few charts it is difficult to construct a well balanced tree hierarchy. Choosing to break the model into many charts results in poorly parameterized charts. This is due to fact that charts with few triangles tend not to parameterize into square/rectangular regions without high distortion and scaling error. For this reason we found our dynamic parametrization method worked better on larger polygon count models. In practice we achieved good results by breaking our models into clusters containing an average of 64 faces. Table 1 shows some of the performance results from our dynamic parameterization algorithm. The re-parameterization times greatly varied in our painting system based on the number patches affected by a given stroke. The upper ranges in the table reflect reparametrization times with a surface signal that affects all charts. For typical strokes that cover only a fraction of the model we were able to re-parameterize our models at a rate of 5-60Hz. We found the overall performance of our GPU paint system a bit disappointing. Readback of the importance map (at 5122 ) from the GPU accounted for as much a 90% of the total processing time required to re-parameterize and re-render all strokes into a new atlas. Reading back a down-sampled 2562 version of the importance map did run faster, but was often too low of resolution to capture finer gradient details for optimizing the atlas. We hope to see improvement in this with the release of faster graphics card buses that offer symmetric transfer speeds such as PCI Express. Regardless of current bus transfer speeds, the performance of our GPU painting system should scale since only a single image must be readback during texture atlas optimization. 7 Figure 8: Painted 100k face Buddha model (right) and its signal balance dynamic MMA map (left). We like to think of the texture atlas as a mechanism for harnessing the power of the GPU for surface, shape and mesh processing, as demonstrated by the catalog of geometry maps used to hold surface retesselation variables [Alliez et al. 2002] or geometry images used to process geometry as images [Gu et al. 2002]. Like these, the multires meshed atlas works on models of arbitrary genus (e.g. Fig. 8) but also has the advantage that the model need not be manifold (e.g. Fig. 9), and can even work on polygon soup (e.g. Fig. 10) depending on the clustering method used. The multiresolution capabilities of the MMA allow the GPU to focus its processing power on specific areas of a surface, and the dynamic MMA allows these areas of focus to change over time. We believe the techniques in this paper are likely useful in areas beyond parameterization. For example Garland et al. [2001] Conclusion We have demonstrated the effectiveness of a dynamic reparametrization algorithm at the task of supporting fine resolution strokes in a 3-D surface paint system. The previously published multiresolution meshed atlas was extended to perform at interactive rates and to be sensitive to surface signal information. We also utilized the GPU to accelerate common surface paint operations, including stroke rasterization, texture signal analysis Figure 9: The Utah teapot is the non-manifold union of manifoldwith-boundary pieces (e.g. spout, handle), which the multires meshed atlas manages cleanly without any additional special processing. D ESBRUN , M., M EYER , M., AND A LLIEZ , P. 2002. Intrinsic parameterizations of surface meshes. Computer Graphics Forum 21, 209–218. F LETCHER , R. 1987. Practical Methods of Optimization. John Wiley & Sons, New York. F LOATER , M. S. 1997. Parametrization and smooth approximation of surface triangulations. Computer Aided Geometric Design 14, 4, 231–250. F LOATER , M. S. 2003. Mean value coordinates. Comput. Aided Geom. Des. 20, 1, 19–27. G ARLAND , M., W ILLMOTT, A., AND H ECKBERT, P. S. 2001. Hierarchical face clustering on polygonal surfaces. In Proceedings of the 2001 Symposium on Interactive 3D Graphics, 49–58. G U , X., G ORTLER , S. J., AND H OPPE , H. 2002. Geometry images. Proc. SIGGRAPH 2002, ACM Trans. on Graphics 21, 3, 355–361. Figure 10: Painting detail on a heart model polygonized from voxel data. provides algorithms for fast face cluster maintenance with application to radiosity. A static MMA was used to organize and evaluate precomputed scattering factors for real-time subsurface scattering [Carr et al. 2003]. The dynamic MMA could similarly organize surface samples for real-time processing of dynamic lighting on a moving surface. Acknowledgements This research was supported in part by Nvidia and the NSF under grant #ACI-0113968. Pedro Sander graciously provided the Parasaur data that allowed us to directly compare the dynamic MMA to signal stretch. Thanks to Zoe Wood for the clean genus6 Buddha and Jerome Maillot for the Corinne Chaix head model. Thanks also to Jesse Hall, Patrick Lacz, and Mark Flider for their help in preparing the paper, images and video. References A LLIEZ , P., M EYER , M., AND D ESBRUN , M. 2002. Interactive geometry remeshing. Proc. SIGGRAPH 2002, ACM Trans. on Graphics 21, 3, 347–354. BALMELLI , L., TAUBIN , G., AND B ERNARDINI , F. 2002. Spaceoptimized texture maps. In Eurographics, 411–420. H ANRAHAN , P., AND H AEBERLI , P. E. 1990. Direct WYSIWYG painting and texturing on 3D shapes. In Proc. SIGGRAPH 1990, vol. 24:4, 215–223. I GARASHI , T., AND C OSGROVE , D. 2001. Adaptive unwrapping for interactive texture painting. Symposium on Interactive 3D Graphics 2001, 209–216. K ARYPIS , G., AND K UMAR , V. 1998. Multi-level algorithms for multi-constraint graph partitioning. Supercomputing 1998. L ÉVY, B., AND M ALLET, J.-L. 1998. Non-distorted texture mapping for sheared triangulated meshes. Proc. SIGGRAPH 98, 343–352. L ÉVY, B., P ETITJEAN , S., R AY, N., AND M AILLOT, J. 2002. Least squares conformal maps for automatic texture atlas generation. Proc. SIGGRAPH 2002, ACM Trans. on Graphics 21, 3, 362–371. L LOYD , S. P. 1982. Least squares quantization in PCM. IEEE Trans. Inform. Theory 28, 129–137. M AILLOT, J., YAHIA , H., AND V ERROUST, A. 1993. Interactive texture mapping. In Proc. SIGGRAPH 93, 27–34. S ANDER , P. V., S NYDER , J., G ORTLER , S., AND H OPPE , H. 2001. Texture mapping progressive meshes. SIGGRAPH 2001, 409–416. S ANDER , P. V., G ORTLER , S. J., S NYDER , J., AND H OPPE , H. 2002. Signal-specialized parametrization. Proc. Eurographics Rendering Workshop (July), 87–98. B ENSON , D., AND DAVIS , J. 2002. Octree textures. Proc. SIGGRAPH 2002, ACM Trans. on Graphics 21, 3, 785–790. S ANDER , P. V., W OOD , Z. J., G ORTLER , S. J., S NYDER , J., AND H OPPE , H. 2003. Multi-chart geometry images. In Proceedings of the Eurographics/ACM SIGGRAPH symposium on Geometry processing, Eurographics Association, 146–155. C ARR , N. A., AND H ART, J. C. 2002. Meshed atlases for realtime procedural solid texturing. ACM Trans. on Graphics 21, 2, 106–131. S HEFFER , A., AND H ART, J. C. 2002. Seamster: Inconspicuous low-distortion texture seam layout. Proc. Visualization 2002, 291–298. C ARR , N. A., H ALL , J. D., AND H ART, J. C. 2003. GPU algorithms for radiosity and subsurface scattering. Proc. Graphics Hardware 2003 (July), 51–59. S IMON , H. D. 1991. Partitioning of unstructured problems for parallel processing. Computing Systems in Engineering 2, 135– 148. D EBRY, D., G IBBS , J., P ETTY, D. D., AND ROBBINS , N. 2002. Painting and rendering textures on unparameterized models. Proc. SIGGRAPH 2002, ACM Trans. on Graphics 21, 3, 763– 768. S LOAN , P.-P. J., W EINSTEIN , D. M., AND B REDERSON , J. D. 1998. Importance driven texture coordinate optimization. Computer Graphics Forum 17, 3, 97–104.