Abstract
Photos play a critical role in online shopping. To examine their impact on consumers, most previous studies rely on human assessments to develop measures for photos. Such an approach limits the number of dimensions and samples that can be investigated in one study. This study exploits image-processing techniques to tackle this challenge. We develop a framework and differentiate two types of computer-generated measures, aggregative and decompositive measures, which may be used in different ways in empirical research. We review the major image-processing technologies that have potential to be used in consumer behavior research. To showcase the feasibility of the framework, we conduct an example study on product photos’ impact on consumer click-through. Moreover, we conduct a simulation to investigate the robustness of the framework under the attack of image-processing algorithm errors. We find that image-processing techniques with 90~95% accuracy will be sufficient for empirical research.









Similar content being viewed by others
Notes
Image-processing and the related computer vision area are one major field in computer science related to artificial intelligence, machine learning, and pattern recognition. In this paper, we use image-processing to represent the family of computer science techniques to understand and derive meanings from images.
Recently, computers can achieve higher performance than humans on certain tasks, such as face recognition. But humans perform better on most tasks.
The image-processing field has grown dramatically in the past two decades. High-quality papers in the field are often published in the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), International Conference on Computer Vision (ICCV), and European Conference on Computer Vision (ECCV). Advanced algorithms and novel topics can also be found at the Neural Information Processing Systems (NIPS) and ACM SIGKDD conferences on knowledge discovery and data mining (KDD). Their top journal is IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).
We use a code name to keep the company’s identity confidential.
References
Alexander, C. (2005). The nature of order: An essay on the art of building and the nature of the universe: A vision of a living world. Book three. Berkeley: Center for Environmental Structure.
Andriluka, M., Roth, S., & Schiele, B. (2009). Pictorial structures revisited: People detection and articulated pose estimation. In 2009 IEEE conference on computer vision and pattern recognition, Miami, FL, USA, June 20-25 2009 (pp. 1014-1021). https://doi.org/10.1109/CVPR.2009.5206754.
Archak, N., Ghose, A., & Ipeirotis, P. G. (2011). Deriving the pricing power of product features by mining consumer reviews. Management Science, 57(8), 1485–1509.
Bauerly, M., & Liu, Y. L. (2008). Effects of symmetry and number of compositional elements on interface and design aesthetics. International Journal of Human Computer Interaction, 24(3), 275–287. https://doi.org/10.1080/10447310801920508.
Berg, H., Söderlund, M., & Lindström, A. (2015). Spreading joy: Examining the effects of smiling models on consumer joy and attitudes. Journal of Consumer Marketing, 32(6), 459–469.
Berry, S. T. (1994). Estimating discrete-choice models of product differentiation. The Rand Journal of Economics, 25(2), 242–262.
Bertamini, M., Bennett, K. M., & Bode, C. (2011). The anterior bias in visual art: The case of images of animals. Laterality: Asymmetries of Body, Brain and Cognition, 16(6), 673–689.
Bhattacharya, S., Sukthankar, R., & Shah, M. (2010). A framework for photo-quality assessment and enhancement based on visual aesthetics. In Proceedings of the 18th ACM international conference on Multimedia, (pp. 271–280). https://doi.org/10.1145/1873951.1873990.
Bland, E. M., Black, G. S., & Lawrimore, K. (2007). Risk-reducing and risk-enhancing factors impacting online auction outcomes: Empirical evidence from ebay auctions. Journal of Electronic Commerce Research, 8(4), 236–243.
Bodor, R., Jackson, B., & Papanikolopoulos, N. (2003). Vision-based human tracking and activity recognition. Paper presented at the 11th IEEE Mediterranean conference on control and automation, Rhodes, Greece, June 18-20.
Boratto, L., Carta, S., Kaltenbrunner, A., & Manca, M. (2018). Guest editorial: Behavioral-data mining in information systems and the big data era. Information Systems Frontiers, 20(6), 1153–1156. https://doi.org/10.1007/s10796-018-9884-1.
Boutell, M. R., Luo, J., Shen, X., & Brown, C. M. (2004). Learning multi-label scene classification. Pattern Recognition, 37(9), 1757–1771.
Buunk, A. P., & Dijkstra, P. (2011). Does attractiveness sell? Women's attitude toward a product as a function of model attractiveness, gender priming, and social comparison orientation. Psychology & Marketing, 28(9), 958–973.
Chang, W.-L., & Chen, Y.-P. (2019). Way too sentimental? A credible model for online reviews. Information Systems Frontiers, 21(2), 453–468. https://doi.org/10.1007/s10796-017-9757-z.
Chau, P. Y. K., Au, G., & Tam, K. Y. (2000). Impact of information presentation modes on online shopping: An empirical evaluation of a broadband interactive shopping service. Journal of Organizational Computing and Electronic Commerce, 10(1), 1–21.
Cheung, M. Y. M., Hong, W. Y., & Thong, J. Y. L. (2017). Effects of animation on attentional resources of online consumers. Journal of the Association for Information Systems, 18(8), 605–632.
Chiang, I. R., & Nunez, M. A. (2007). Improving web-catalog design for easy product search. INFORMS Journal on Computing, 19(4), 510–519. https://doi.org/10.1287/ijoc.1060.0184.
Creusen, M. E., Veryzer, R. W., & Schoormans, J. P. (2010). Product value importance and consumer preference for visual complexity and symmetry. European Journal of Marketing, 44(9/10), 1437–1452.
Cyr, D., Head, M., Larios, H., & Pan, B. (2009). Exploring human images in website design: A multi-method approach. MIS Quarterly, 33(3), 539–566.
Cyr, D., Head, M., & Larios, H. (2010). Colour appeal in website design within and across cultures: A multi-method evaluation. International Journal of Human-Computer Studies, 68(1), 1–21.
Das, S. R., & Chen, M. Y. (2007). Yahoo! For amazon: Sentiment extraction from small talk on the web. Management Science, 53(9), 1375–1388.
Dhar, S., Ordonez, V., & Berg, T. L. (2011). High level describable attributes for predicting aesthetics and interestingness. In 2011 IEEE computer society conference on computer vision and pattern recognition, Colorado Springs, CO, USA, June 20-25 (pp. 1657-1664). https://doi.org/10.1109/CVPR.2011.5995467.
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1627–1645.
Freimuth, M., & Wapner, S. (1979). Influence of lateral organization on the evaluation of paintings. British Journal of Psychology, 70(May), 211–218.
Friedenberg, J. (2012). Aesthetic judgment of triangular shape: Compactness and not the golden ratio determines perceived attractiveness. i-Perception, 3(3), 163–175.
Gardner, J. S., Fowlkes, C., Nothelfer, C., & Palmer, S. E. (2008). Exploring aesthetic principles of spatial composition through stock photography. Journal of Vision, 8(6), 337–337.
Ghose, A., & Ipeirotis, P. G. (2011). Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering, 23(10), 1498.
Godes, D., & Mayzlin, D. (2004). Using online conversations to study word-of-mouth communication. Marketing Science, 23(4), 545–560.
Golder, S. A., & Macy, M. W. (2011). Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science, 333(6051), 1878–1881. https://doi.org/10.1126/science.1202775.
Goswami, A., Chittar, N., & Sung, C. H. (2011). A study on the impact of product images on user clicks for online shopping. In Proceedings of the 20th International Conference Companion on World Wide Web, Hyderabad, India, March 28–April 01 (pp. 45–46). https://doi.org/10.1145/1963192.1963216.
Ha, Y., Kwon, W.-S., & Lennon, S. J. (2007). Online visual merchandising (vmd) of apparel web sites. Journal of Fashion Marketing and Management, 11(4), 477–493.
Hagtvedt, H., & Patrick, V. M. (2008). Art infusion: The influence of visual art on the perception and evaluation of consumer products. Journal of Marketing Research, 45(3), 379–389.
Harris, L. C., & Goode, M. M. (2010). Online servicescapes, trust, and purchase intentions. Journal of Services Marketing, 24(3), 230–243.
Hemphill, M. (1996). A note on adults' color–emotion associations. The Journal of Genetic Psychology, 157(3), 275–280.
Ismagilova, E., Slade, E. L., Rana, N. P., & Dwivedi, Y. K. (2019). The effect of electronic word of mouth communications on intention to buy: A meta-analysis. Information Systems Frontiers, 1–24. https://doi.org/10.1007/s10796-019-09924-y.
Isola, P., Xiao, J., Torralba, A., & Oliva, A. (2011). What makes an image memorable? In 2011 IEEE conference on computer vision and pattern recognition, Colorado Springs, CO, USA, June 20-25 (pp. 145-152). https://doi.org/10.1109/CVPR.2011.5995721.
Jiang, Y.-G., Ngo, C.-W., & Yang, J. (2007). Towards optimal bag-of-features for object categorization and semantic video retrieval. In Proceedings of the 6th ACM International Conference on Image and Video Retrieval, Amsterdam, Netherlands, July 09–11 (pp. 494–501). https://doi.org/10.1145/1282280.1282352.
Joan, S. P. F., & Valli, S. (2017). An enhanced text detection technique for the visually impaired to read text. Information Systems Frontiers, 19(5), 1039–1056. https://doi.org/10.1007/s10796-016-9699-x.
Joblove, G. H., & Greenberg, D. (1978). Color spaces for computer graphics. Computers and Graphics, 12(3), 20–25.
Joseph, W. B. (1982). The credibility of physically attractive communicators: A review. Journal of Advertising, 11(3), 15–24.
Juneja, M., Vedaldi, A., Jawahar, C., & Zisserman, A. (2013). Blocks that shout: Distinctive parts for scene classification. Paper presented at the 2013 IEEE conference on computer vision and pattern recognition, Portland, OR, USA, June 23-28.
Kim, H., & Lennon, S. J. (2010). E-atmosphere, emotional, cognitive, and behavioral responses. Journal of Fashion Marketing and Management, 14(3), 412–428. https://doi.org/10.1108/13612021011061861.
Kim, J.-H., Kim, M., & Lennon, S. J. (2009). Effects of web site atmospherics on consumer responses: Music and product presentation. Direct Marketing: An International Journal, 3(1), 4–19.
Korkmaz, O. (2009). Primary perceptual field in visual materials. The Social Sciences, 4(5), 525–533.
Kuan, K. K. Y., Zhong, Y. Q., & Chau, P. Y. K. (2014). Informational and normative social influence in group-buying: Evidence from self-reported and eeg data. Journal of Management Information Systems, 30(4), 151–178.
Kumar, G., & Bhatia, P. K. (2014). A detailed review of feature extraction in image processing systems In Fourth International Conference on Advanced Computing & Communication Technologies, Rohtak (pp. 5–12). https://doi.org/10.1109/ACCT.2014.74.
Lee, Y. J., Ghosh, J., & Grauman, K. (2012). Discovering important people and objects for egocentric video summarization. In 2012 IEEE conference on computer vision and pattern recognition, Providence, RI, USA, June 16-21 (pp. 1346-1353). https://doi.org/10.1109/CVPR.2012.6247820.
Lewis, G. (2011). Asymmetric information, adverse selection and online disclosure: The case of ebay motors. The American Economic Review, 101(4), 1535–1546.
Leyvand, T., Cohen-Or, D., Dror, G., & Lischinski, D. (2008). Data-driven enhancement of facial attractiveness. ACM SIGGRAPH 2008 papers on - SIGGRAPH '08, 27(3), 1.
Li, H., Doermann, D., & Kia, O. (2000). Automatic text detection and tracking in digital video. IEEE Transactions on Image Processing, 9(1), 147–156.
Liang, T.-P., Li, X., Yang, C.-T., & Wang, M. (2015). What in consumer reviews affects the sales of mobile apps: A multifacet sentiment analysis approach. International Journal of Electronic Commerce, 20(2), 236–260.
Liao, J. X., Yang, D., Li, T. H., Wang, J. Y., Qi, Q., & Zhu, X. M. (2014). A scalable approach for content based image retrieval in cloud datacenter. Information Systems Frontiers, 16(1), 129–141. https://doi.org/10.1007/s10796-013-9467-0.
Liu, L., Chen, R., Wolf, L., & Cohen-Or, D. (2010). Optimizing photo composition. Computer Graphics Forum, 29(2), 469–478.
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., & Shum, H. Y. (2011). Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(2), 353–367.
Liu, Y., Li, H., & Hu, F. (2013). Website attributes in urging online impulse purchase: An empirical investigation on consumer perceptions. Decision Support Systems, 55(3), 829–837. https://doi.org/10.1016/j.dss.2013.04.001.
Loiacono, E. T., Watson, R. T., & Goodhue, D. L. (2007). Webqual: An instrument for consumer evaluation of web sites. International Journal of Electronic Commerce, 11(3), 51–87.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.
Loy, G., & Eklundh, J.-O. (2006). Detecting symmetry and symmetric constellations of features. In 2006 9th European conference on computer vision, Graz, Austria, May 7-13 (pp. 508-521). https://doi.org/10.1007/11744047_39.
Makinen, E., & Raisamo, R. (2008). Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 541–547.
Marchesotti, L., Perronnin, F., Larlus, D., & Csurka, G. (2011). Assessing the aesthetic quality of photographs using generic image descriptors. In 2011 IEEE international conference on computer vision, Barcelona, Spain, November 6-13 (pp. 1784-1791). https://doi.org/10.1109/ICCV.2011.6126444.
Matas, J., Chum, O., Urban, M., & Pajdla, T. (2004). Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 22(10), 761–767.
McManus, I. C. (2005). Symmetry and asymmetry in aesthetics and the arts. European Review, 13(S2), 157–180.
Moreno-Noguer, F., Sanfeliu, A., & Samaras, D. (2005). Integration of conditionally dependent object features for robust figure/background segmentation. Paper presented at the 2005 IEEE international conference on computer vision, Beijing, China, October 15-21.
Moshagen, M., & Thielsch, M. T. (2010). Facets of visual aesthetics. International Journal of Human-Computer Studies, 68(10), 689–709.
Motiwalla, L., Deokar, A. V., Sarnikar, S., & Dimoka, A. (2019). Leveraging data analytics for behavioral research. Information Systems Frontiers, 21(4), 735–742. https://doi.org/10.1007/s10796-019-09928-8.
Nair, H., Chintagunta, P., & Dubé, J.-P. (2004). Empirical analysis of indirect network effects in the market for personal digital assistants. Quantitative Marketing and Economics, 2(1), 23–58.
Nilsson, M., Nordberg, J., & Claesson, I. (2007). Face detection using local smqt features and split up snow classifier. Paper presented at the the 42nd IEEE international conference on acoustics, speech and signal processing, New Orleans, USA, march 5-9.
Norman, D. (2002). Emotion & design: Attractive things work better. Interactions, 9(4), 36–42. https://doi.org/10.1145/543434.543435.
Palmer, J. W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151–167.
Palmer, S. E., Gardner, J. S., & Wickens, T. D. (2008). Aesthetic issues in spatial composition: Effects of position and direction on framing single objects. Spatial Vision, 21(3), 421–449.
Parboteeah, D. V., Valacich, J. S., & Wells, J. D. (2009). The influence of website characteristics on a consumer's urge to buy impulsively. Information Systems Research, 20(1), 60–78.
Park, J., Lennon, S. J., & Stoel, L. (2005). On-line product presentation: Effects on mood, perceived risk, and purchase intention. Psychology & Marketing, 22(9), 695–719.
Park, J., Stoel, L., & Lennon, S. J. (2008). Cognitive, affective and conative responses to visual simulation: The effects of rotation in online product presentation. Journal of Consumer Behaviour, 7(1), 72–87.
Popovič, A., Hackney, R., Tassabehji, R., & Castelli, M. (2018). The impact of big data analytics on firms’ high value business performance. Information Systems Frontiers, 20(2), 209–222. https://doi.org/10.1007/s10796-016-9720-4.
Samal, A., & Iyengar, P. A. (1992). Automatic recognition and analysis of human faces and facial expressions: A survey. Pattern Recognition, 25(1), 65–77.
Sammartino, J., & Palmer, S. E. (2012). Aesthetic issues in spatial composition: Effects of vertical position and perspective on framing single objects. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 865–879.
Shen, L. X., Wang, H., Xu, L. D., Ma, X., Chaudhry, S., & He, W. (2016). Identity management based on pca and svm. Information Systems Frontiers, 18(4), 711–716. https://doi.org/10.1007/s10796-015-9551-8.
Shim, S., Eastlick, M. A., & Lotz, S. (2000). Assessing the impact of internet shopping on store shopping among mall shoppers and internet users. Journal of Shopping Center Research, 7(2), 7–43.
Shin, S. H., Yang, S. B., Nam, K., & Koo, C. (2017). Conceptual foundations of a landmark personality scale based on a destination personality scale: Text mining of online reviews. Information Systems Frontiers, 19(4), 743–752. https://doi.org/10.1007/s10796-016-9725-z.
Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. Hoboken: John Wiley & Sons, Ltd..
Song, S. S., & Kim, M. (2012). Does more mean better? An examination of visual product presentation in e-retailing. Journal of Electronic Commerce Research, 13(4), 345.
Steinbrück, U., Schaumburg, H., Duda, S., & Krüger, T. (2002). A picture says more than a thousand words: Photographs as trust builders in e-commerce websites. Paper presented at the CHI'02 extended abstracts on human factors in computing systems, Minneapolis, Minnesota, USA, April 20-25.
Sun, C.-Y., & Lee, A. J. T. (2017). Tour recommendations by mining photo sharing social media. Decision Support Systems, 101, 28–39. https://doi.org/10.1016/j.dss.2017.05.013.
Svobodova, K., Sklenicka, P., Molnarova, K., & Vojar, J. (2014). Does the composition of landscape photographs affect visual preferences? The rule of the golden section and the position of the horizon. Journal of Environmental Psychology, 38, 143–152.
Then, N. K., & DeLong, M. R. (1999). Apparel shopping on the web. Journal of Family and Consumer Sciences, 91(3), 65.
Tuch, A. N., Bargas-Avila, J. A., & Opwis, K. (2010). Symmetry and aesthetics in website design: It’s a man’s business. Computers in Human Behavior, 26(6), 1831–1837.
Verhagen, T., & Van Dolen, W. (2009). Online purchase intentions: A multi-channel store image perspective. Information & Management, 46(2), 77–82.
Vieira, V. A. (2010). Visual aesthetics in store environment and its moderating role on consumer intention. Journal of Consumer Behaviour, 9(5), 364–380. https://doi.org/10.1002/cb.324.
Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154.
Wang, M. Y., & Li, X. (2017). Effects of the aesthetic design of icons on app downloads: Evidence from an android market. Electronic Commerce Research, 17(1), 83–102. https://doi.org/10.1007/s10660-016-9245-4.
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2003). Multiscale structural similarity for image quality assessment. Paper presented at the the thirty-seventh Asilomar conference on signals, systems and computers, Pacific grove, California, USA, November 9-12.
Wang, X., Jia, J., Yin, J., & Cai, L. (2013). Interpretable aesthetic features for affective image classification. Paper presented at the 2013 20th IEEE international conference on image processing, Melbourne, Australia, September 15-18.
Wu, K., Vassileva, J., Zhao, Y., Noorian, Z., Waldner, W., & Adaji, I. (2016). Complexity or simplicity? Designing product pictures for advertising in online marketplaces. Journal of Retailing and Consumer Services, 28, 17–27.
Xiao, L., & Ding, M. (2014). Just the faces: Exploring the effects of facial features in print advertising. Marketing Science, 33(3), 338.
Yamaguchi, K., Kiapour, M. H., Ortiz, L. E., & Berg, T. L. (2012). Parsing clothing in fashion photographs. In 2012 IEEE conference on computer vision and pattern recognition, Providence, RI, USA, June 16-21 (pp. 3570-3577). https://doi.org/10.1109/CVPR.2012.6248101.
Yan, X. B., Wang, J., & Chau, M. (2015). Customer revisit intention to restaurants: Evidence from online reviews. Information Systems Frontiers, 17(3), 645–657. https://doi.org/10.1007/s10796-013-9446-5.
Yoo, J., & Kim, M. (2012). Online product presentation: The effect of product coordination and a model's face. Journal of Research in Interactive Marketing, 6(1), 59–72.
You, Q., Luo, J., Jin, H., & Yang, J. (2016). Building a large scale dataset for image emotion recognition: The fine print and the benchmark. Paper presented at the the thirtieth AAAI conference on artificial intelligence, Phoenix, Arizona, USA, February 12-17.
Zhang, D. S., Islam, M. M., & Lu, G. J. (2012a). A review on automatic image annotation techniques. Pattern Recognition, 45(1), 346–362. https://doi.org/10.1016/j.patcog.2011.05.013.
Zhang, Z., Li, X., & Chen, Y. (2012b). Deciphering word-of-mouth in social media: Text-based metrics of consumer reviews. ACM Transactions on Management Information Systems, 3(1), 5.
Zhao, S., Gao, Y., Jiang, X., Yao, H., Chua, T.-S., & Sun, X. (2014). Exploring principles-of-art features for image emotion recognition. Paper presented at the proceedings of the 22nd ACM international conference on multimedia, Orlando, FL, USA, November 03-07.
Zhou, J., & Hoang, J. (2005). Real time robust human detection and tracking system. Paper presented at the 2005 IEEE computer society conference on computer vision and pattern recognition-workshops, San Diego, CA, USA, June 20-25.
Zhu, F., & Zhang, X. (2010). Impact of online consumer reviews on sales: The moderating role of product and consumer characteristics. Journal of Marketing, 74(2), 133–148.
Acknowledgments
This research is partially supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (CityU 11503115), and the Natural Science Foundation of China (71572169).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, M., Li, X. & Chau, P.Y.K. Leveraging Image-Processing Techniques for Empirical Research: Feasibility and Reliability in Online Shopping Context. Inf Syst Front 23, 607–626 (2021). https://doi.org/10.1007/s10796-020-09981-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-020-09981-8