Spectacularization and Immersivity

DIGITAL CULTURAL HERITAGE Spectacularization and Immersivity Teacher: Giulio Lughi un programma di partner tecnico-scientifici • • • CNR Consiglio Nazionale delle Ricerche, ISPC Istituto di Scienze del Patrimonio Culturale IIT Istituto Italiano di Tecnologia, Center for Cultural Heritage Technology Ca’ Foscari (CCHT@Ca’Foscari) INFT Istituto Nazionale di Fisica Nucleare, CHNet Cultural Heritage Network Spectacularization and immersivity A short definition • Spectacularization: the tendency to enhance works of art and cultural activities through emphasized forms of visualization, closely connected with the consumer economy in mass society • Immersivity: when the viewer has the sensation of being inside a visual work of art, or inside a cultural artifact or situation. It is not the primary natural sensation, as when the viewer is admiring a natural landscape, but a sensation mediated by the most innovative digital technologies Spectacularization and immersivity An experimental project Bagan - Embracing the Future to Preserve the Past was created with 3D scanning, photogrammetry, and 360 video. You can travel through the landscape and walk inside decorated temples at risk of collapse, while learning the ancient stories they tell. Data, maps, storytelling. Spectacularization in sociology Edgar Morin, L'Esprit du temps. Essai sur la culture de masse, 1962. "Loisir" is a condition of life and a mentality that arise directly from the modes of production and consumption of mass society, and are implemented in the institutionalization of free time, in the exaltation of the spectacular components of entertainment : «The loisir is not only the receptacle in which the essential contents of life enter and in which the aspiration to individual happiness becomes a requirement, but it is in itself cultural ethics; it is not only the framework of private values, but it is also a realization in itself." (pp. 71-72). Spectacularization in sociology Guy Debord: La société du spectacle (1967) Debord highlights the elements of fascination that the spectacle has on contemporary society, and its character of ambivalence between reality and illusion. Spectacularization in sociology • MacDonaldization, (Ritzer, 1997): the spread of cultural homologation and economic globalization • Commodification, Rifkin (2000): the commercialization of arts and cultural resources, transformed into individual entertainment for a fee • Disneyization, Bryman (2004): the spectacularization of physical spaces (the "theme parks") as well as the immersive participation of the spectator • "Shop-windowization", Codeluppi (2007): the relationship between the economic production and the visibility of goods Spectacularization in art • Aesthetic perception, (Simmel, 1903): man's ability to make sense of his complex experience of the world • Aesthetization, (Maffesoli, 1996): the appreciation of the work of art is no longer just a personal matter, but gives way to a kind of "mass subjectivity" • Artist capitalism, Lipovetsky and Serroy (2013): the need to spectacularize artistic creation and aesthetic fruition to overcome its strictly utilitarian dimension • Artistization, Perniola (2015): the appreciation of the work of art does not depend on its intrinsic characteristics, but on a system of economic and social relations and connections Spectacularization, interpretation and AI During the exhibition "The last Caravaggio" (Milan 2018), the psychophysiological responses of the spectators in front of the works of art were detected with the aid of three wearable digital devices: Eye tracking, Brain Computer Interface, Stress Bracelet. Spectacularization, interpretation and AI "We examined the entire collection of Rembrandt’s work, studying the contents of his paintings pixel by pixel. To get this data, we analyzed a broad range of materials like high resolution 3D scans and digital files, which were upscaled by deep learning algorithms to maximize resolution and quality. This extensive database was then used as the foundation for creating The Next Rembrandt." Visual Culture The age of images Visual Culture deals with images and their growing importance in the transmission of culture: after many centuries in which cultural transmission has been achieved only through writing, today it is necessary to consider the impact of images from a psychological, social, political and cultural point of view.. Image studies have a long historical tradition, particularly in the 1960s, but only today are Visual Culture Studies a well-defined academic sector (in Germany Bildwissenshaft) Technological evolution ICT Information and Communication Technologies Processing speed Bandwidth Mass storage 1940 numbers 1970 alphabetic characters 1990 still images / audio 2000 audiovisual (Big) Data and Databases The new cultural organization Thanks to the developments of digital technology, the deep organization of culture passes from the linear dimension, typical of books and written documents, to the reticular dimension typical of the database. British Museum This project is a partnership between Visual timeline British Museum and Google Cultural Institute: an interactive experience through time, continents and cultures, featuring some of the most fascinating objects in human history. You can jump back in time to explore objects from across diverse cultures, click to connect objects across time and space, and discover hidden links between then and now. Maps Geographical data The complete mapping of the entire earth's surface (not only Google, but also Open Street Maps) allows us to develop products and applications that change our vision of the territory and establish new relationships with the elements of culture. Maps American Literature’s Most Epic Road Trips This map is the result of an effort to catalog the country as it has been described in the American road-tripping literature. It includes every place-name reference in 12 books about cross-country travel, from Mark Twain(1872) to Cheryl Strayed (2012), and maps the authors’ routes on top of one another. You can track an individual writer’s descriptions of the landscape as they traveled across it, or you can zoom in to see how different authors have written about the same place at different times. Visual mapping An experimental socio-cultural exploration Embodiment In the philosophical tradition, embodiment is a concept that concerns the relationship between mind and body in the study of cognitive processes and simulation, and considers bodily factors as constitutive of cognition itself. In digital culture, embodiment defines the complex relationship established between people and the real world through different digital interface devices Mobile/locative paradigm A new context for human activities With the diffusion of mobile communication devices, this paradigm develops ever more. Locative media are communication systems that use specific location based technologies in order to give life to significant spatial and temporal relationships between people, groups and institutions: recovering strong connections with local realities, creating shared representations of the surrounding territory, becoming a link between physical reality and the internet. Storytelling In cultural processes, Digital Media extend the notion of storytelling from a simple author–text–user relationship, mediated by the screen interface, into a complex narrative project that positions the body and its media competences within a precise physical space characterized by geographical, historical and cultural coordinates. The digital storyteller has now the ability to elaborate complex narrative projects in order to position and cognitively orient the user in a universe where digital technology is controlling bodies, objects and physical spaces. How many storytelling? Storytelling is a key concept in Digital Media: • Visual Storytelling • Educational Storytelling • Journalism Storytelling • Advertising Storytelling • Corporate Storytelling • Data Storytelling • … Mobile/locative storytelling Ingress Ingress is a multiplayer storygame based on the interaction of GPS technology, player's physical location, information databases, geographical maps and Augmented Reality: the fiction scenes that appear on the smartphone of the user are located exactly in the physical space where the body of the player is. Total work of art das Gesamtkunstwerk To indicate the ability of digital media to pervade cultural products in various ways, we can use a concept that runs through our entire cultural history, from classical antiquity, to Richard Wagner, to the historical avant-gardes of the twentieth century: the concept of Total Work, with which we indicates an artifact capable of involving all our senses, sight, hearing, touch, smell, even taste, and the real position of the body in physical space. Technological re-enchantment In social science, dis-enchantment is the cultural rationalization and devaluation of religion apparent in modern society: the term was proposed by Max Weber (1919) to describe the character of a modernized, bureaucratic, secularized society. Now, thanks to digital tools and their pervasive ability to represent the complex variety of the world, we can see technology as a kind of re-enchantment of the world. Technological re-enchantment > digital creativity From the concept of technological re-enchantment derives the concept of digital creativity: today, digital tools greatly expand the creative potential, and above all they can mix and contaminate the expressive languages of the single arts that in the past were clearly separated (literature, theater, painting, sculpture, music) Remediation "The content of a medium is always another medium" Picking up from McLuhan ("The content of a medium is always another medium"), media theorist David Bolter describes how media forms interact with each other through remediation, examining how emerging kinds of media, such as websites, blogs, wiki pages, and digital video, both delimit the ways people can use them, and provide new avenues for the production of social relations and meanings. All media, from literature to new digital media, is unable to make a radical break away from what has come before, and continues to function in a constant dialectic with earlier media forms. In this sense, digital media have the ability to absorb and mix all previous media, written texts, images, music, videos, adding the ability to develop new forms typical of digital. Deep remixability Lev Manovich, 2013 "Collected in a common software environment, the languages of film, animation, computer animation, special effects, graphic design, and print have come to form a new metalanguage." Author's reconfiguration New chances for cultural products • hypertext: the reticular, non-linear, text organization • multimedia: the coexistence of different expressive languages • program languages: the possibility to have the texts changed automatically • interactivity: give the user the opportunity to respond • immersivity: give the user the impression of being inside a work of art Immersivity Virtual Reality A wide galaxy Augmented Reality Extended Reality Hybrid Reality 360° Photo/Video Mixed Reality Enhanced Reality 3D Graphics Videomapping Immersive Experience… "heavy" / "light" Immersivity: from Virtual Reality to Art Experience body engaged by visors, gloves, sensors body free from technological devices "heavy" / "light" Movie culture: from Kinetoskope to Cinema Thomas Alva Edison Auguste and Louis Lumière "heavy" / "light" Digital media: from stand alone to mobile/locative the user sitting at the desk the user who moves in the environment "heavy" / "light" Videogames: from arcades to pervasive gaming the user in a game room Ingress / Pokémon Go , "heavy" / "light" Contemporary art: from contemplation to participation the user in the museum the user in a site specific installation Virtual reality Klimt Magic Garden KLIMT‘S MAGIC GARDEN is a virtual reality experiment inspired by Klimt’s designs for the mosaic frieze in the dining room of the Stoclet House in Brussels. Using highresolution digital photography, the virtual reality artist and filmmaker Frederick Baker has created a fantastic virtual world in which you can set out on an interactive cinematographic journey. VR failure "Future of Storytelling", march 2021 "When Facebook released its first Oculus Rift VR headset in early 2016, there was a lot of excitement in Hollywood about a new era of immersive filmmaking that would finally allow audiences to step into the worlds they had long admired from afar. Five years later, the path to that future seems as windy as ever. In recent years, Viacom, Google, Hulu, Samsung, and Intel have all either shuttered or downsized their VR operations, and a number of promising indie VR studios have disappeared." VR failure Arcagni, D'Aloia, VR Storytelling: Potentials and Limitations of Virtual Reality Narratives, 2022 "… we are today faced with an indisputable fact: despite the predictions made some years ago, from both the production and technological sectors, VR apparatuses have not broken into the market; there has been no diffusion of VR arcades, let alone a market that would produce and distribute content. VR as an entertainment medium is not experiencing success (not even in the more vibrant field of gaming)." Techno-glasses failure Despite the failure of Google Glasses, attempts to offer similar products continue: Snapchat, Rayban / Facebook, and Apple which has been postponing the launch of its viewers for years. Augmented Reality Los Angeles County Museum of Art Compared to Virtual Reality, which engages the body with invasive tools, Augmented Reality is much "lighter" and therefore similar to the daily experience we have of the surrounding reality. Furthermore, Virtual Reality closes perception in an isolated and autonomous world, while Augmented Reality allows a perceptual and cognitive expansion, putting into action a multilayer dialectic between physical reality and the contents displayed by the smartphone. Augmented Reality Acute Art On the Acute Art platform, great contemporary artists such as Christo & JeanneClaude, Marina Abramovic, Jeff Koons, Anish Kapoor, Olafur Eliasson explore the potential of Augmented Reality. Matterport Matterport is a very functional system for Syntetic image creating digital twins of buildings and environments, especially for commercial use applied to shops or real estate properties, but recently also to museums and art exhibitions. Matterport has a very simple scanning system with a fluid rendering, similar but much more defined than Google Street View. For the viewer, the effect is sometimes similar to being in a video game rather than in a cultural setting. The Hermitage Museum One of the largest museums in the world, fully mapped with 360 ° photos ("panoramas") that allow the virtual visit of the rooms and of the buildings, accompanied by detailed photos and information on the works of art A virtual visit The Garden of Fluid Sculptures The Garden of Fluid Sculptures is a sitespecific installation by the famous contemporary artist Giuseppe Penone, consisting of fourteen sculptures located in a large historical park. From videomapping to Immersive Art Experiences The Immersive Art Experiences were born as an evolution of videomapping, the nocturnal projection on the facades of historic buildings of dynamic luminous images that play with the structural elements of the buildings (windows, pediments, moldings, etc.). Immersive Art Experiences are the transfer of videomapping in closed environments, organized in large rooms (often post-industrial) where walls, ceilings and floors become screens for large-scale projections of images and videos with surround music Team Lab Spectacularization of original creative works teamLab is a Japanese project that offers a global visual experience: it is based in Tokyo, where it founded the first digital museum, but has exported its activities to Paris, Turin, Helsinki, New York and in various parts of the world, both with temporary exhibitions both with permanent venues; his works are exhibited in almost all major contemporary art museums, and are offered on the market by specialized galleries. Atelier des Lumières Spectacularization of artists of the past Experiences were initially shown in buildings adapted for projection; instead, in 2018 the Atelier des Lumières was inaugurated in Paris, located in a disused industrial building: a fixed location where different experiences can be projected, a sort of multitasking and multidimensional post-cinema Audience attendance Economic impact of Immersive Art Experiences: • Grande Experience produced 215 exhibitions in 32 languages in 165 cities, for a total of over 20 million spectators; • Exibition Hub produced 90 exhibitions in 56 cities for over 10 million spectators; • Culturespaces has 14 offices, 5 million visitors, 400 collaborators; • teamLab had more spectators than the Van Gogh Museum Government funding A French initiative At the beginning of 2021, the Réunion des Musées Nationaux et du Grand Palais, a French public company with the task of promoting cultural activities, announced its entry into the sector of production and dissemination of immersive exhibitions with the launch of the Grand Palais Immersif project. The Réunion des Musées Nationaux et du Grand Palais currently manages thirtyfour of the most important French museums, including the Louvre, and therefore its initiative strongly states the cultural, touristic and commercial importance of Immersive Art Experiences Thanks! Contacts: giulio.lughi@gmail.com / www.giuliolughi.it un programma di partner tecnico-scientifici • • • CNR Consiglio Nazionale delle Ricerche, ISPC Istituto di Scienze del Patrimonio Culturale IIT Istituto Italiano di Tecnologia, Center for Cultural Heritage Technology Ca’ Foscari (CCHT@Ca’Foscari) INFT Istituto Nazionale di Fisica Nucleare, CHNet Cultural Heritage Network