Abstract
Stroke-based rendering is one of the major approaches for creating synthetic paintings, but with only a minor attention so far to stereo painting synthesis. In this article, a fully automatic stereoscopic oil-painting synthesis algorithm is proposed, which takes a photograph and a depth map as input, and generates a pair of oil-painting style, stereo-viewable paintings. Common drawbacks of existing stroke-based rendering results are impressions of repetition and flatness due to the regularity of the used 2D stroke patterns. To reduce these impressions, the proposed approach introduces the concepts of a defocused image, a complexity map, a point map, and a direction map. Those maps serve as important references for decision making and thus, are the foundation for the entire painting simulation process. The key feature of making the developed stroke-based algorithm different from others is that it generates a unique 3D brushstroke according to the characteristics of a local image region. This has greatly reduced the undesirable machine-like appearance in the resulting image. Moreover, a comfortable stereo-viewing experience is assured by the proposed stereo painting and hole-filling strategies. Experimental results show that the proposed algorithm is applicable to a wide variety of image subjects and different depth distributions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The argument here is not to disapprove noticeable stroke effects, because, after all, the purpose of painterly rendering algorithms is to generate stroke effects to simulate paintings. This article is trying to point out that when similar strokes are repeatedly used for an entire painting, the overall result of the painting might lead to a mechanized impression.
The size of the directional templates was set to 9 × 9 because the width of the enhanced Canny edges is three pixels.
References
AKVIS ArtWork. [Software]. Available from akvis.com/en/artwork/
Bae S, Durand F (2007) Defocus magnification. Comput Graph Forum 26(3):571–579
Benedetti L, Winnemöller H, Corsini M, Scopigno R (2014) Painting with Bob: assisted creativity for novices. In: Annual ACM symposium on user interface software technology, pp 419–428
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698
Chang W-H, Cheng M-C, Kuo C-M, Huang G-D (2015) Feature-oriented artistic styles transfer based on effective texture synthesis. J. Inf Hiding Multimed Signal Process 6(1):29–46
Chen X, Jin X, Zhao Q, Wu H (2012) Artistic illumination transfer for portraits. Comput Graph Forum 31(4):1425–1434
Deep Art Mobile App. [Software]. Available from deeparteffects.com/
DEPTHY. [Online software]. Available at depthy.me
FotoSketcher. [Software]. Available from fotosketcher.com/
Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style.: computing research repository (CoRR). arXiv:1508.06576
Gooch B, Coombe G, Shirley P (2002) Artistic vision: painterly rendering using computer vision techniques. In: International symposium on non-photorealistic animation rendering, pp 83–90
Guo C-E, Zhu S-C, Wu YN (2007) Primal sketch: integrating structure and texture. Comput Vis Image Underst 106(1):5–19
Hertzmann A (1998) Painterly rendering with curved brush strokes of multiple sizes. In: International conference on computer graphics interactive techniques, pp 453–460
Howard I, Rogers B (1995) Binocular vision and stereopsis Oxford psychology series, vol 29. Oxford University Press, New York
Kang HW, Chui CK, Chakraborty UK (2006) A unified scheme for adaptive stroke-based rendering. Vis Comput 22(9):814–824
Kim Y, Winnemöller H, Lee S (2014) WYSIWYG stereo painting with usability enhancements. IEEE Trans Vis Comput Graph 20(7):957–969
Lee KJ, Kim DH, Yun ID, Lee SU (2007) Three-dimensional oil painting reconstruction with stroke-based rendering. Vis Comput 23(9):873–880
Lee H, Seo S, Ryoo S, Yoon K (2011) Directional texture transfer with edge enhancement. Comput Graph 35(1):81–91
Litwinowicz P (1997) Processing images and video for an impressionist effect. In: International conference on computer graphics interactive techniques, pp 407–414
Liu D (2016) Computer vision meets arts: stereo computational photography. Ph.D. dissertation, Dept. Comp. Sci., The Univ. of Auckland, New Zealand
Meier B J (1996) Painterly rendering for animation. In: International conference on computer graphics interactive techniques, pp 477–484
Northam L, Asente P, Kaplan CS (2012) Consistent stylization and painterly rendering of stereoscopic 3D images. In: Proceedings of the symposium on non-photorealistic animation rendering, pp 47–56
Shiraishi M, Yamaguchi Y (2000) An algorithm for automatic painterly rendering based on local source image approximation. In: International symposium on non-photorealistic animation rendering, pp 53–58
Stavrakis E (2008) Stereoscopic non-photorealistic rendering . Ph.D. dissertation, Vienna Univ. of Tech., Austria
Wang B, Wang W, Yang H, Sun J (2004) Efficient example-based painting and synthesis of 2D directional texture. IEEE Trans Vis Comput Graph 10(3):266–277
Ware C, Gobrecht C, Paton M (1998) Dynamic adjustment of stereo display parameters. IEEE Trans Syst Man Cybern 28:56–65
Zeng K, Zhao M, Xiong C, Zhu S-C (2009) From image parsing to painterly rendering. ACM Trans Graph 29(1):2:1–2:11
Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
This study was funded by the Ministry of Science and Technology, Taiwan (MOST 104-2221-E-197-020-MY2). Fay Huang is a member of Chinese Image Processing and Pattern Recognition Society (IPPR, Taiwan). She worked as a postdoctoral fellow at Institute of Information Science, Academic Sinica, Taiwan, from 2003 to 2004. She was also a consultant of Smart System Institute, Institute for Information Industry, Taiwan, in 2017. Bo-Ru Huang declares that he has no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Funded by the Ministry of Science and Technology, Taiwan (MOST 104-2221-E-197-020-MY2).
Appendix
Appendix
The proportion of the size of a real oil painting and the width of brushstrokes applied in this painting is the key for painting simulation applications. Let ℓ denote the length in mm (i.e., width or height, whichever is longer) of a real oil painting, excluding pictures in printing forms. Although there is no rule about what size of brushes are recommended be used to paint a canvas of length ℓ. The following statistics and relations can be summarized by observations.
The width of brushstrokes in real oil paintings is generally from 3 to 20 mm excluding some extreme cases or styles. The average width of brushstrokes is close to 10 mm for most of the paintings sized from 600 to 900 mm in length. Let b denote the average width in mm of brushstrokes applied on a painting. As the value of ℓ decreases, the value of b also decreases towards zero. As the value of ℓ increases, the value of b also increases towards an upper bound, roughly equals to 15 mm. This is because in real-world situation the paint brushes come with standard sizes. Relation between the length ℓ and the width b can be approximated by a log curve as follows
Rights and permissions
About this article
Cite this article
Huang, F., Huang, BR. Stereoscopic oil paintings from RGBD images. Multimed Tools Appl 78, 18249–18270 (2019). https://doi.org/10.1007/s11042-019-7167-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-7167-6