VISUAL THEORIES

VCD 151 · Fall 2023-24

Over the centuries, psychologists, philosophers, and professionals have proposed many
theories trying to explain how we see and how we learn from images. The four theories we
discuss have direct connection to mass communications. They can be divided into two
fundamental groups: sensory and perceptual.

Those who advocate the sensory theories (gestalt and constructivism) maintain that direct or
mediated images are composed of light objects that attract or repel us. They are more
concerned with what the brain sees—the visual cues of color, form, depth, and
movement—but not so much of how the mind considers them. The perceptual theories
(semiotics and cognitive) are concerned mainly with the meaning that humans associate with
images—what the mind sees.

The two sets of theories can be summed by the difference between what something looks
like and what it actually is. To understand any of these approaches to visual communication,
you must first know the difference between visual sensation and visual perception.
A sensation is a stimulus from the outside world that activates nerve cells within your
sense organs. Wood burning in a fireplace activates the cells in your ears because you
can hear the logs cracking and hissing, in your nose because you can smell the rich
aroma of the wood, in your hands and face because you can feel the warmth of the fire,
in your mouth if you pop a hot toasted marshmallow into it, and in your eyes as you
watch the hypnotizing glow of the yellow flames. Sensations are lower-or-physical
responses to stimuli and alone convey no meaning. Nerve cells in your ears, nose,
hands, mouth, and eyes do not have the capacity to make intelligent thoughts. They
are simply conveyors of information to the brain. Our minds make meaning of all the
sensory input. Conclusions based on those data are almost instantaneous. Our minds
interpret the noises, smells, temperatures, tastes, and sights as a fire. Visual
perception concentrates on the conclusions that are made from information gathered
by our eyes.
Sitting before such a fire, your brain registers its heat, colors, and sounds while your mind notices the artificial logs and natural gas
jets, but if you are cold, the brain wins.
SENSORY THEORIES OF VISUAL COMMUNICATION
Researchers and theorists who concentrate on sensory theories of visual communication are
mainly concerned with how the brain notices or fails to see the visual cues of color, form, depth,
and movement. A useful motion picture that helps you imagine what it’s like to be a brain cell is the
documentary Koyaanisqatsi (1982) directed by Godfrey Reggio
(https://www.youtube.com/watch?v=i4MXPIpj5sA). With the thousands of slow- and fast-motion
images and quick and frenetic cuts between scenes of everything from crowded freeways to
women who work on a snack cake assembly line, there are too many images to process and
remember. Scenes in the film become a blur composed not of content but of the four visual cues.
The film shows what being a brain cell in your visual cortex at the back of your brain is like—the
cell notes the stimulation and passes it on to your mind without considering it. As a general rule,
sensory theories are not concerned with the literal meaning of what is possible to be seen. They
help us understand how we can be attracted and distracted from visual messages.
Gestalt The gestalt theory of visual perception emerged from a simple observation. German
psychologist Max Wertheimer received his inspiration during a train trip in the summer of
1910. As he looked out the windows as the train moved through the sunny German
countryside, he suddenly realized that he could see the outside scene even though the
opaque wall of the train and the window frame partially blocked his view. He left the train in
Frankfurt, went to a toy store, and bought a popular children’s toy of the day—a stroboscope,
similar to what we call a flipbook today. The flipbook is a simple form of cartoon animation.
On the first page of the book, a drawing—say, of a cartoon character in a running
position—is displayed on the right-hand side of the page. On each subsequent page, the
drawing of the figure is slightly different depending on the actions intended by the artist. To
see the effect of the moving character, a viewer uses a thumb to flip the pages rapidly. A
modern example of this animation technique can be seen in a music video from the Dutch
pop group Kraak & Smaak for their 2007 hit, “Squeeze Me”.
Wertheimer’s observations during the train trip and using the flipbook led to more research at
the University of Frankfurt. Wertheimer concluded that the eye merely takes in all the visual
stimuli, whereas the brain arranges the sensations into a coherent image. Without a brain
that links individual sensory elements, the phenomenon of movement would not take place.
His ideas led to the famous statement:

The whole is different from the sum of its parts.

In other words, perception is a result of a combination of sensations and not of individual

sensory elements. The word gestalt comes from the German noun that means form or
shape. Gestalt psychologists further refined the initial work by Wertheimer to conclude that
visual perception is a result of organizing sensory elements or forms into various groups.
Discrete elements within a scene are combined and understood by the brain through a
series of four fundamental principles of grouping that are often called laws: similarity,
proximity, continuation, and common fate.
Similarity This gestalt law states that objects that look similar will be automatically grouped
together by the brain. Simple experiments with the basic shapes of filled-in circles and
square forms made that clear to gestalt researchers. Whether for print or screen media,
words are easily separated from images. However, when a page is composed of nothing but
similarly sized words or pictures, the viewing of it can be tedious. Visual interest comes from
dissimilarity, not similarity.
Three boys dressed in similar Mardi Gras costumes pose for a picture while they wait to be photographed with the queen of a ball in New
Orleans. One of the reasons the brain links the boys as a single unit is because of their similarity. Regardless, look at their expressions
closely—they have quite different personalities—innocent, cynical, and friendly.
Proximity The brain more closely associates objects close to each other than it does an
object that is farther apart. Likewise, two friends standing near each other will be viewed as
being more closely related than a third person standing 20 yards from the couple. Proximity
is also a factor with the visual cue of depth. The illusion is enhanced if an object is perceived
as being close to the viewer while another seems farther away. If two objects appear to be
on the same horizontal plane or are the same size, their proximity is equal and the sensation
of depth is reduced.
A poster produced by the Federal Art Project in the 1930s is used to promote education and civic activity. It also is an example of proximity—the
brain naturally divides the two sets of penguins because of their closeness to each other.
Continuation The brain does not prefer sudden or unusual changes in the movement of a
line. In other words, the brain seeks as much as possible a smooth continuation of a
perceived movement. The line can be a drawing, or it can be several objects placed together
along an imaginary line. Objects viewed as belonging to a continuous line will be mentally
separated from other objects that are not a part of that line. Continuation also refers to
objects that are partially blocked by a foreground object with a viewer’s mind continuing the
line in order to achieve a kind of graphic closure.
A billboard for a Target department store in Adelaide, Australia, demonstrates continuation as you continue the circles that make up the store’s
logo in your mind. This image also demonstrates that non-American advertisers can get away with more than their U.S. counterparts.
Common Fate Finally, another principle of gestalt psychology is common fate. A viewer
mentally groups five arrows or five raised hands pointing to the sky because they all point in
the same direction. An arrow or a hand pointed in the opposite direction will create tension,
because the viewer will not see it as part of the upwardly directed whole. Again, a visual
communicator can use this principle to direct a viewer’s eyes toward or away from a graphic
element in a picture or design. The placement of a warning label in a cigarette ad is made
through research by tobacco industry graphic designers to be the least viewed part of a
page.

Danish gestalt psychologist Edgar Rubin developed the principle of camouflage when he
made patterns with little or no separation between the foreground and the background.
Understanding and manipulating this trait of visual perception led directly to military
applications of merging the colors of uniforms and equipment with those of surrounding
backgrounds in order to hide them. This principle also influenced the work of artists M. C.
Escher and Paul Klee, both of whom were inspired by the writings and findings of several
gestalt psychologists.
A Work Progress Administration poster from the 1930s promotes the early treatment of syphilis with the use of the gestalt principle of common
fate. Happily dancing men free of the disease move up while those who waited for treatment sadly trudge downward.
The Danish psychologist Edgar Rubin used the gestalt principles to draw conclusions about how the mind tells the difference between a foreground and a
background (also known as figure and ground). Noting how we see differences led to the idea of creating similarities and camouflage clothing for military uses. PFC
Joel Graham applies camouflage paint as his unit prepares to board a ship that will take him to the verdant countryside of Puerto Rico for a military exercise.
Visual communicators learned at least two important lessons from the gestalt theory—studying
individual elements of a picture helps you to better understand its whole meaning, and the theory
helps you create more noticeable print and screen media designs.

When analyzing a visual message, tiny details within a frame should be studied first to discover
how they create a different and often surprising whole. For example, a photographic craze in the
1990s used a computer to mesh the exposures and compositions of hundreds of similarly themed
single images into a gestalt whole. As a student at the Massachusetts Institute of Technology,
Robert Silvers invented a photographic technique called “photomosaic” that can arrange hundreds
of single photographs by their exposures into one picture. Through his company Runaway Tech-
nologies, he creates examples such as a picture of the Earth comprising single images from the
ground, Leonardo da Vinci’s “Mona Lisa” that is a composite of paintings from the artist’s
notebooks and the Renaissance period, and a close-up portrait of Nazi Holocaust victim Anne
Frank as a combination of pictures from that era. Silvers has also produced images for use in
advertisements for Audi, Coca Cola, Master-Card, and other corporations.
The photographic work of Robert Silvers is a demonstration of the gestalt phrase, “The whole is different from the sum of its parts.” If you have no idea who this
person is, looking closely at the individual images that compose the whole picture reveals clues to the subject of this portrait. Holocaust identification photographs,
German officials, and prison camp views are placed within the work depending on their exposures. Once the whole image is revealed, it is obvious the portrait is of
a smiling and vibrant Annelies “Anne” Frank, who hid with her family from the Germans within a house in Amsterdam until they were discovered in 1944. Seven
months later she died of typhus at the Bergen-Belsen concen- tration camp at the age of 15. In 1952 the English translation of her The Diary of a Young Girl was
published to international critical acclaim.
In addition, gestalt helps to focus on a tiny, cropped version of a photograph so that additional
insights can be learned when attention is turned to the entire image. Diane Arbus was an inspired
portrait photographer who helped invent a genre of photography known as the “snapshot
aesthetic.” Many times she took pictures of unusual-looking persons—circus performers, nudists,
identical twins, a giant, and so on—and photographed them in such a way as to make the portrait
look like an ordinary picture you would find in a family’s photo album. If you look at just the face of
Arbus’ portrait titled “A Woman with her Baby Monkey, N.J., 1971,” you won’t see any signs of the
animal that sits on her lap dressed like a baby. What you might notice with the close-up view,
however, is the broken slat in the blinds and reflective wood paneling behind her, the cut and style
of her hair, and her plain clothes that all indicate economic status. Studying her face, you will see
her slight, closed-mouth smile is belied by the sadness evident in her eyes. Finally, the turn of her
head and the distracting shadow in the background caused by a flash are indications of the
snapshot aesthetic style. After that concentrated and focused view, seeing the entire photograph
confirms and also expands the initial observations. Poignancy is added to the photograph when it
is also considered that the portrait was made the year Arbus killed herself.
Diane Arbus’ “A Woman with her Baby Monkey, N.J., 1971” is a study in how the gestalt visual communication theory can teach how to better study
images—concentrate on small details and put them together into a whole.
Gestalt research radically changed the design philosophy of graphic artists. The strength of gestalt
is its attention to the individual forms that make up a picture’s content. Any analysis of an image
should start by concentrating on those forms that naturally appear in any picture. Recall that color,
form, depth, and movement all are basic characteristics of an image that the brain notices. Gestalt
teaches a visual communicator to combine those basic elements into a meaningful whole. The
approach also teaches the graphic artist to focus attention on certain elements by playing against
the gestalt principles. For example, a company’s logo (or trademark) will be noticed in an
advertisement if it has a dissimilar shape, size, or location in relation to the other elements in the
layout.

Gestalt also helped alter the front-page layouts of newspapers. Before the theory was advanced,
newspapers were a mind-numbing collection of gray words on a page separated by six to eight
long, vertical columns that ran the length of the page. The main story’s small headline, followed by
smaller subheads, and then even smaller body copy, started at the top-left of the page and
continued on without breaks or pictures.
These designs, intended for highly literate readers, were dull and unattractive. In gestalt terms, the page
was an example of similarity, the columns showed equal proximity, continuation was imposed as the
reader’s eyes finished the bottom of one column to advance to the top of the next, and all the columns
were meant to be read in the same direction, indicating a common fate. Today, a newspaper’s front page
is a varied, sometimes desperately cacophonous collection of different sized headlines, columns, and
pictures. In gestalt terms, the page is now dissimilar, because separate stories are easily differentiated,
proximity is noted when two photographs for the same story are printed closely together, continuation is
indicated by both a vertical and a horizontal flow of the eyes depending on the layout, and the relative
importance of each story is shown by its position and size on the page. Modern newspaper design is a
lesson in the application of the gestalt theory. Most experts agree, however, that gestalt design won’t be
enough to save the delivery of news on paper.

The work of gestalt theorists clearly shows that the brain is a powerful organ that classifies visual material
in discrete groups. What we see when looking at a picture is modified by what we are directed to see or
miss by photographers, filmmakers, and graphic designers.
This first edition of the Daily Globe (St. Paul, Minnesota), published on January 1, 1880, illustrates how the four gestalt laws of similarity, proximity, continuation, and
common fate do not necessarily create a graphic design that holds much visual interest for a reader.
Modern newspaper design creates visual interest by displaying contrasts in headline, picture, and story styles, sizes, and locations, as shown by the front pages
taped to a wall as editors for the German publication Bild work on the next issue in 1977.
Constructivism In 1970, Julian Hochberg, a professor of psychology at Columbia
University, found that the eyes of his experimental subjects were constantly in motion as they
scanned an image. These quick fixations all combined within the viewer’s short-term memory
to help build a mental picture of a scene. For Hochberg, a viewer constructs a scene with
short-lived eye fixations that the mind combines into a whole picture. If memorable, the
scene will be added to a person’s long-term memory. The gestalt approach described a
viewer as being passive. In contrast, constructivism emphasizes the viewer’s eye
movements in an active state of perception.

Hochberg had his subjects use eye-tracking machines in his visual perception experiments.
These devices can chart the way a viewer looks at an image. Eye-tracking machines simply
made obvious the eyes’ frenetic journey across a direct or mediated image.

Two graphic designers and researchers helped make Hochberg’s theory practical for visual
communicators by showing how viewers notice elements on a page or screen.
In order to conduct research on graphic design attributes for print and online publications, subjects were asked to wear headsets that recorded eye movements and
time spent viewing various elements. This elaborate set-up was used by eye-tracking researchers in the 1960s and perhaps later to gain information from terrorists.
In 1990, Dr. Mario García of the Poynter Institute and García Media and Dr. Pegie Stark
Adam of Poynter used an eye-track testing machine to record on videotape the eye
movements of participants as they read different versions of a newspaper. Participants wore
special glasses that “contained two small cameras—one that recorded eye movement and
another that recorded where the reader looked.” The time spent on each element of a page
could also be recorded. As expected, researchers found that readers noticed the largest
picture on a page first and a headline before a story. Captions under photographs were the
third most viewed element on a page. Clearly, the results indicated how important graphic
designs are in capturing a reader’s attention.

Subsequent eye-track studies looked at the reading habits of online readers. One study
found that web users noticed text on a screen first, unlike their print counterparts who
noticed images first. For García it was clear that online “is more like reading a book, where
one concentrates on the text and prefers photos to appear separately.”
In 2007 a more elaborate study tried to find differences in readers as they navigated the
various elements that make up print and online pages. Studying more than 350 elements
that could be found on pages—headlines, stories, pictures, briefs, advertisements, podcasts,
blogs, teasers, and so on—the study was an exhaustive use of the eye-track procedure. The
research discovered several differences between online and print readers. Online users read
more of a story than readers in broadsheet and tabloid formats. Online users scanned
through various stories whereas print readers tended to start at the beginning and read to
the end. More attention by readers of all graphic formats was given if stories also contained
informational graphics, sidebars, and lists. Bigger headlines and photographs got a lot of
attention in print, but online readers noted navigational elements and links. Photojournalistic
images of real people were preferred over studio set-up shots. Both groups liked color over
black and white.
Ironically, the news organizations that participated in the study in order to find out how to
keep the readers they have and attract new ones included the Star Tribune of Minneapolis
and the Philadelphia Daily News, whose parent companies filed for bankruptcy in 2009, and
the Rocky Mountain News, which quit publishing altogether the same year. This downward
trend for newspapers is why analysts with the Future Exploration Network predict that
“newspapers in the U.S. will become insignificant by 2017 and the rest of the world by 2040”
while academics at USC’s Annenberg School for Communication and Journalism in the
Center of the Digital Future predict that most U.S. newspapers will be gone in five years.

Although limited by their emphasis on “what the brain sees” and not what the objects seen
mean, sensory theories can be thought to be limited in their application for visual
communicators. Nevertheless, the gestalt and constructivism theories both have important
uses for print and/or screen media graphic designers. No work is useful if readers, viewers,
and users do not notice it.
PERCEPTUAL THEORIES OF VISUAL
COMMUNICATION
The semiotics and cognitive theories of visual perception can be considered to be content driven.
Although recognizing that vision cannot happen without light illuminating, structuring, and sometimes
creating perceptions, these two theories stress that humans are unique in the animal kingdom because
we assign complex meaning to the objects we see.

Semiotics The flag that is raised high above a baseball stadium and is watched reverently during the
singing of the national anthem by those in the stands and on the field is a sign. The right hand placed over
the approximate location of the heart during the singing of the anthem is a sign. The words printed in a
program about the players on the field are signs. The close-up photograph of a player holding a bat
awaiting a pitch are signs. The refs’ black outfits and the managers’ hand signals are signs. The
illuminated numbers on the scoreboard are signs. Even cleat marks in the dirt are signs. The “high-five”
slap with a friend after a team’s home run is a sign. The simple silhouette illustration of a woman on a
restroom door is a sign. The green traffic light as you make your way home from the game is a sign.
Fans of the Los Angeles Angels wear a fake Mohawk and standard red baseball caps along with a jersey to support their team.
A sign is simply anything that stands for something else. After reading the preceding list of signs
you might well ask: What is not a sign? That’s a good question, because almost any action, object,
or image will mean something to someone somewhere. Any word or physical presentation, from a
yelled comment to an orange jacket, is a sign if it has meaning beyond the object itself.
Consequently, the meaning behind any sign must be learned. In other words, for something to be a
communicated sign, the viewer must understand its meaning. But if you don’t understand the
meaning behind the orange color of a jacket, it isn’t a sign for you. It’s just a jacket.

Semiotics (called semiology in Europe) is the study or science of signs. The field is the culmination
of Aldous Huxley’s mantra: The more you know, the more you see. Images will be much more
interesting and memorable if signs that are understood by many are used in a picture. The study of
semiotics is vital because signs permeate every message, whether verbal or visual. The academic
study of semiotics attempts to identify and explain the signs used by every society in the world.
Although semiotics has gained popularity relatively recently, it is an old concept. In 397 ce,
Augustine of Hippo, a Roman philosopher, linguist, and bishop of the Roman Catholic
Church, first proposed the study of signs. He recognized that nature is filled with universally
understood entities that afforded communication on many nonverbal levels. For Augustine,
signs were the link between nature and humans—between the outer and inner worlds. More
importantly, signs from nature also linked individuals to form cultural meaning that could be
transferred to future generations. The word semiotics comes from the language of his
country: Semeion is the Greek word for sign.

Contemporary semiotics emerged through the work of two theorists just before World War I.
Swiss linguist Ferdinand de Saussure developed a general theory of signs while a professor
at the University of Geneva.
We know about his work in semiotics because of the notes written by his students during his
lectures, which were later published. A lesson: Listen closely to your instructors and take
good notes. At about the same time, American philosopher Charles Sanders Peirce
(pronounced “purse”) published his own ideas about the effect of signs on society. De
Saussure and Peirce inspired others to concentrate in this field of study. The Americans
Arthur Asa Berger, Charles Morris, and Thomas Sebeok, the Italian Umberto Eco, the French
Roland Barthes, and many others have contributed greatly to the study of semiotics. Eco’s
novels The Name of the Rose, Foucault’s Pendulum, and The Island of the Day Before are
fascinating and amusing explorations of symbolic meaning.

De Saussure and Peirce weren’t particularly interested in the visual aspects of signs. They
were traditional linguists who studied the way words were used to communicate meaning
through narrative structures. However, over the years semiotics has evolved into a theory of
perception that involves the use of images in unexpected ways.
For example, Sebeok, a professor emeritus at Indiana University who died in 2002, identified some of the
topics that semiotics researchers have studied. Besides the obvious subject of visual signs and symbols
used in graphic designs, they include the semiotics of the theater, where performance elements are
analyzed; the semiotics of puppetry, in which the colors, costumes, gestures, and staging of the
characters are studied; the semiotics of television and commercials; the semiotics of tourism; the
semiotics of the signs used in Boy Scout uniforms and rituals; the semiotics of notational systems used in
dance, music, logic, mathematics, and chemistry; and urban semiotics, in which the growth and physical
attributes of cities are seen as social symbols. The field has become so popular that journals, international
conferences, and academic departments at universities are devoted to semiotics.

Peirce’s contribution to semiotics was in the formulation of three different types of signs: iconic, indexical,
and symbolic. All signs must be learned, but the speed of comprehension of each type of sign varies.
Thinking about iconic, indexical, and symbolic signs is a way to really look and study a visual message in
a much more thorough and critical manner. Once this process is done, you soon realize that even the
simplest image has complex cultural meaning. However, it is important to realize that the three categories
of signs are not mutually exclusive. The written and visual examples given are meant to focus your
attention on one particular type of sign.
The Hof’s Hut restaurant napkin is a study of many different types of semiotic signs—iconic, the illustration of a pot pie; indexical, the “heat” lines emanating from
the crust; and symbolic, the words and colors.
Iconic Signs Icon, from the Greek word eikenai, means “to be like” or “to seem.” Iconic signs are
the easiest to interpret because they most closely resemble the thing they are meant to represent.
Examples of icons are the accurate cave paintings of animals by prehistoric humans, the simple
drawings above restroom doors that communicate the gender allowed inside, the trash can, printer,
and home images on the desktops of many computers, street signs that indicate dangerous road
conditions, and—the most common of all—photographs and motion pictures that are meant to be
representations of what they depict.

Almost any documentary photographer’s images would be good examples of iconic signs. The
portraits of German August Sander work well. Looking at his many images you cannot doubt that at
some time the persons pictured resembled their portraits.

For the restaurant’s napkin, the tilted, brown illustration of a chicken pot pie in the top-left corner is
an iconic sign—the drawing is meant to represent a pie similar to one you can order at the
restaurant.
Indexical Signs Indexical signs have a logical, common sense connection to the thing or idea they represent
rather than a direct resemblance to the object. Consequently, their interpretation takes a little longer than that of
icons. We learn indexical signs through everyday life experiences. Peirce used a sundial as an illustration of an
indexical sign. The sun’s shadow implied the movement of time. Other indexical signs can be a footprint on the
beach or on the surface of the moon, smoke spewing out of a high smokestack or an automobile exhaust pipe, or
the high temperature reading of a sick patient. Footprints stand for the person who impressed them. Smoke
represents the pollution generated by the furnace or engine. Fever indicates that the patient has an infection. A
famous portrait of trumpeter Louis Armstrong taken in 1965 by John Loengard of Life magazine is a tightly
cropped image that shows lines on two fingers and his lips, which are indexical signs of his age. The portrait
captures him rubbing petroleum jelly to his lips to soften them before a concert.

Back to the napkin, we have learned through our experiences that the six lines trailing from the pot pie are meant
to represent heat. Someone’s food is fresh and right out of the oven. However, to someone else, those lines might
indicate that the pie is falling, a common visual device used in printed cartoons. More bizarre, a creative person
might think that the lines mean that the pie is a marionette, a type of puppet controlled by strings, or even
stranger, a pie full of rats, with the brown lines representing the tails showing through the crust. All of these
interpretations come from the indexical sign on the napkin of lines exuding from the pie.
This 19th century drawing presents good examples of indexical signs. There are two possible sources for the smoke—a steam engine on the horse-drawn fire
engine and a fire inside the house. But is there a blaze on the fire engine or is the man at the window simply smoking a (rather large) cigar? Experience helps us
decipher indexical signs. But as with all images, there are other signs to analyze. The photographic quality of the image makes it an iconic sign, and the buildings,
clothing, and horses are symbolic signs of an earlier age.
Symbolic Signs The third type of sign is the most abstract. Symbols have no logical or
representational connection between them and the things they represent. Symbols, more
than the other types of signs, have to be taught. For that reason, social and cultural
considerations influence them greatly. Words, numbers, colors, gestures, flags, costumes,
most company logos, music, and religious images all are considered symbols. Because
symbols often have deep roots in the culture of a particular group, with their meanings being
passed from one generation to the next, symbolic signs mean more than iconic or indexical
signs. The burning of a country’s national flag as a protest gesture is a powerful symbol of
defiance and anger. It isn’t simply an act to create heat through the burning of a piece of
fabric.
A black cloth over the head of a person symbolizes death in many cultures. In reality, this man is simply avoiding the sun or the photographer on the boardwalk of
Atlantic City.
New Orleans photographer and poet Clarence John Laughlin made pictures of objects that
for him had complex symbolic meanings. One image is of a statue by the side of a grave he
named “Figure from the Underworld, 1951.” His typically lucid and elaborate caption directs
the viewer to notice all three types of signs. Iconic: “A horrible little stucco figure, probably
turned out by the thousands in a mold, and found in a Louisiana country garden.” Indexical:
“The fierce suns and heavy rains of Louisiana have eaten it as though by acid.” Symbolic:
“Leaving it as though with its brain exposed, and with a sweet smile turned sickly and
defeated. It rises as if from some nether plane—the dark and ragged American world of the
1930s.” His image becomes a powerful metaphor that makes a comment about our present
economic times.

Besides iconic, indexical, and symbolic signs, Roland Barthes described another way to
think of individual elements within an image. He developed the concept of a chain of
associations that make up a picture’s narrative. To understand this concept, we must first
discuss how we communicate through words.
New Orleans photographer Clarence John Laughlin’s “Figure from the Underworld, 1951” is a visual metaphor for the degradation of society and a clear example of
an indexical sign in semiotics.
In verbal language, the narrative or story we are telling/reading/hearing is linear. One word follows
the next in a specific rule-based order known as its syntax, or grammar. These rules of syntax have
been established and agreed upon over centuries for a language and its people. Pictures, on the
other hand, are presentational. All the elements of an image, whether still or moving, are presented
all at once with a viewer free to look at them in any order. Signs within an image are presented in
various ways for a variety of media, many times depending on the style of the image maker. But
since most of us think of images through thoughts composed of words within our minds, we usually
link individual elements within a picture into a narrative whole. For Barthes, each element is a link
that forms a chain of associations, or meaning. The common term for Barthes’s chain of
associations is codes. A code is an amalgamation of hundreds of ideas and/or elements into one,
convenient concept. The next time you are stopped at a street corner and see a stop sign, think of
all the underlying statutes and laws that regulate the sign itself—its color, location, shape, size,
height, and so on—and its meaning—to stop, of course, but also where, for how long, and in which
order if other cars are present. Asa Berger elaborated on Barthes and suggested four types of
codes: metonymic, analogical, displaced, and condensed.
As one of the few traffic signs that originated in the United States, the first stop sign was erected in Michigan, the home of the American automobile industry, in 1915
and showed the word “STOP” in black letters against a yellow octagon background. A common stop sign stands for a complicated set of legal specifications and
codes. In 1954 the design was changed to a standard 30 inches across each side of its octagonal area with a three-quarter-inch white bor-
der. The white uppercase sans serif letters forming the word “STOP” are ten inches high. The height of a sign from the base must be at least five feet. The color red
was chosen because the same color is used for traffic lights. The sign is also retroreflective. It reflects headlights back with a minimum of scattering. But the traffic
sign also stands for a complex set of legal codes—for example, if you arrive first at an intersection you can proceed first, but if two or more drivers arrive at the
junction at the same time, the one on the left must yield to the one on the right. This combination of a series of underlying codes is what makes visual messages
interesting.
Metonymic Code A collection of signs that cause the viewer to make assumptions about what is
seen is a metonymic code. In that way, this type of code is closely associated with indexical signs.
You assume something about what you see. Most advertisers, whether working in commercial,
non-profit, or political venues, want the viewer to make assumptions about a particular product or
service. A viewer of a studio set-up portrait of a smiling family—father, mother, daughter, and
dog—playfully wrapped in bed linens used in an advertisement shown in a magazine makes a
number of assumptions about the picture—that this is a real family, that they are actually happy,
and that their choice of cotton comforters has brought them to this blissful state. Furthermore, it is
hoped that when you see this pleasant quartet of good-looking models, you will think that if you
had the same products on your bed you would be just as satisfied with your life.
http://goo.gl/NB2Huu
Analogical Code This type of code is a group of signs that cause the viewer to make mental
comparisons. Examples, often called figures of speech, might compare an old tree to a
human face, a live mouse to a computer device, and lined yellow paper to a lemon peel. A
large piece of equipment, such as a blast furnace in a steel mill, might have shapes and
patterns that when seen at a particular angle and under specific lighting conditions
resembles the face of a robot. It is unlikely that the architect of the factory positioned parts of
the furnace to resemble the eyes, nose, and mouth of a face, but it is likely that an observant
photographer would take a picture in such a way to show those features because it adds
interest to the picture. During the 2009 U.S. Open tennis tournament, American Express
introduced a commercial for its charge card that showed everyday objects, singularly and in
combination that resembled human faces when sad or happy. Produced by WPP’s Ogilvy &
Mather advertising agency, it puts the analogical code to creative commercial use.
In the summer of 1976, the Viking Orbiter 1 took photographs of the Cydonia region of Mars in the planet’s northern hemisphere to find possible landing sites for its
sister spaceship, the Viking Lander 2. One of the images revealed what a NASA public relations person described as a “huge rock formation . . . which resembles a
human head . . . formed by shadows giving the illusion of eyes, nose and mouth.” The so-called “Face on Mars” became an instant popular culture phenomenon,
There was no face, but the analogical code lives on.
Displaced Code Whenever there is a transfer of meaning from one set of signs to another, a
displaced code is used. In the classic movie Dr. Strangelove or: How I Learned to Stop Worrying
and Love the Bomb (1964) directed by Stanley Kubrick, rifles, missiles, airplanes, and other phallic
shapes were photographed purposely to communicate the idea of sexual tension among certain
military characters. The film’s climax shows the character Major T.J. “King” Kong played by Slim
Pickens gleefully riding the bomb to his and the world’s doom.

Images of penises are not acceptable pictures for most members of society and so are displaced
by their phallic equivalents. Liquor, lipstick, and cigarette advertisers also commonly use phallic
imagery in the form of their products’ shapes in the hope that potential customers will link the use
of their products with possible sexual conquest. Ads from Skyy vodka, Tom Ford eyewear, and
Sisley clothing regularly employ such symbolism. To attract attention and to link sex with their
products, “shock ads” from these companies have used the shape of a tie, the placement of a
bottle, a man’s middle finger in the mouth of a woman, and a female model holding a snake and
attempting to lick its head.
Stanley Kubrick’s Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb (1964) is a classic study of the displaced code. The nuclear warhead
prop that actor Slim Pickens rides at the end of the movie is the ultimate phallic symbol.
Condensed Code In many respects, this type of code is the most interesting. Condensed codes are
several signs that combine to form a new, composite message. Televised music videos and the
advertisements inspired by them have unique and often unexpected meanings. The signs of musicians,
dancers, music, quick-editing techniques, graphics, colors, and so on all form a complex message. Within
the culture a message is intended for, a condensed code has relevant meaning. For those outside that
culture being represented, the images can be confusing, random, and without purpose. But the way
individuals combine signs and form their own meaningful messages often cannot be controlled by the
creators of the signs. The photographic work of American Jerry Uelsmann combines elements from
several images to make intriguing composite pictures.

Semiotics teaches the importance of symbolism in the act of visual perception and communication. A
viewer who knows the meaning behind the signs used in a complex picture will gain insights from it,
making the image more memorable. The motion pictures of American director David Lynch (Blue Velvet,
1986; Lost Highway, 1997; Mulholland Dr., 2001) are often examples of complex semiotic signs. For
example, the opening scene from Twin Peaks: Fire Walk with Me (1992) has a strange woman wearing a
red dress and a blue rose who communicates through gestures important information to the characters
played by Chris Isaak and Kiefer Sutherland.
In his 1980 photograph “Untitled,” Jerry Uelsmann creates a cloud-filled sky contained within a box is suspended above the edge of an ocean with crashing waves.
Without much help from the title, any meaning from this set of elements must come from the viewer.
The 1991 music video from the rock group REM for their song “Losing My Religion” is a rich
and potentially confusing collection of signs. Its overall meaning is aided by knowing the
history of the band and the biography of its members, the lyrics of the song, the fact that the
director, Tarsem Singh is from India, the “Myth of Icarus,” the history of fascism, and the
paintings of Italian Michelangelo Merisi da Caravaggio, particularly his 1601-02 oil, “The
Incredulity of Saint Thomas”.

The problem in using complex signs as a part of an image is that they may be
misunderstood, ignored, or interpreted the wrong way. Nevertheless, the challenge for visual
communicators, expressed in the study of semiotics, is that signs can enhance the visual
experience and educate, entertain, and persuade a viewer.
Cognitive According to the cognitive theory, what is going on in a viewer’s mind is just as
important as the images that can be seen. Mental activities focus attention on a visual
element, but they can also distract a viewer. The cultural anthropologist Carolyn Bloomer
identified several mental activities that affect visual perception: memory, projection,
expectation, selectivity, habituation, salience, dissonance, culture, and words.

Memory Arguably the most important mental activity involved in accurate visual perception,
memory is our link with all the images we have ever seen. A historic photograph of President
Lincoln’s funeral in Washington, D.C. in 1865 or a crowd shot from the 1969 Woodstock,
New York music concert might be of interest to you as they trigger memories and
associations because of a funeral you have attended, a visit to America’s capitol, something
you read about the 16th president, someone you know who was at the historic concert, or a
music festival you attended. Someone else might have no associations with either picture
and quickly turn the page.
“The Incredulity of St. Thomas,” 1601–1602, by Caravaggio. The Italian painter Michelangelo Merisi da Caravaggio popularized a technique called selective
illumination that became a signature of the Baroque school of art that he initiated. His use of spot lighting and dark shadows added dramatic interest to his works.
He also preferred to use ordinary people he met on the streets as models in his paintings to give them a realistic quality, unlike the idealized religious works of the
day. “The Doubting of St. Thomas,” hanging in the former summer palace of Frederick the Great, captures the moment told in the Bible when St. Thomas
investigates the wounds of Jesus and no longer doubts that he has returned to life after his crucifixion.
President Lincoln’s funeral procession in New York City, with 11,000 members of the military and about 75,000 civilians marching along, was as elaborate in real life
as it is shown here in this lithograph based on a photograph attributed to Matthew Brady and published in Harper’s Weekly on May 13, 1865. If you are a history buff
interested in Lincoln, or have been to funeral or parade recently, this image may be of interest to you because it might trigger memories.
Projection Creative individuals see recognizable forms in Cheerios floating in a bowl of milk
in the morning or in rock patterns while sitting near a pool in Palm Springs, California. Others
make sense out of clouds or trees, or find comfort in the messages learned from tarot cards,
astrological forecasts, and the I-Ching. One reason the common inkblot test developed in
1921 by the Swiss Freudian psychologist Hermann Rorschach is used is that individuals
often reveal personality traits by deriving meaning from the oddly formed shapes. A person’s
mental state of mind is thus “projected” onto an inanimate object or generalized statement.
Someone may spend hours marveling at the humanlike face formed by the curves and
shadows in a tree trunk while another will walk past. The difference between the two may be
in the mental processes that affect what they see.
Projection is in the mind of the beholder. Whether you see a seemingly random and meaningless pattern created by a plant climbing the wall
of a building in Santa Monica, California or a representation of the Statue of Liberty depends on your mental state.
Expectation Having preconceived expectations about how a scene should appear often
leads to false or missed visual perceptions. Italian artist Guido Daniele paints images of
zebras, elephants, eagles, and snakes on human hands. The intriguing visual result is that a
viewer often forgets that the animal paintings use a hand as the substrate for the work.
Daniele lives and works in Milan, Italy where he produces photo-realistic paintings for private
collectors and advertising agencies. http://www.guidodaniele.com

A clever magazine advertisement and commercial for Johnnie Walker’s Blue Label whisky is
at first an artistic collection of wavy lines—until a full- length image of a man standing before
a woman sitting in a chair holding a bottle suddenly is noticed under the catch phrase, “For
those who know what to look for”. Researchers Robert Becklen and Daniel Cervone devised
an experiment in 1983 in which subjects were asked to count the number of times a
three-person team wearing black T-shirts passed a basketball. In a recent recreation, Daniel
J. Simons substitute an umbrella with someone in an ape costume walking casually through
the room.
The irony of Johnnie Walker’s “For those who know what to look for” campaign is that most viewers will miss the hidden visual message in the advertisement
because they don’t know what to look for.
Selectivity Aldous Huxley discussed this cognitive element when he wrote of combining
selecting with sensing and perceiving. Most of what people see within a complicated visual
experience is not part of conscious processing. For example, rarely do people think about
their own breathing unless made aware of it. Most of visual perception is an unconscious,
automatic act by which large numbers of images enter and leave the mind without being
processed. We usually focus only on significant details within a scene. If you are trying to
locate a friend sitting in packed bleachers during a baseball game, all the other unknown
faces in the crowd will have little significance. When you see your friend, your mind suddenly
locks on that person as if with the help of a spotlight in a darkened room. Concentrating on
people observing a funeral procession in a documentary photograph taken during the violent
era of 1981 in Belfast, Northern Ireland, you might miss the covered face of an Irish
Republican Army (IRA) soldier.
Expectation is a mental condition that can lead to heightened observation if a scene matches your mental imaginings or to poor visual perception if your
pre-conceived idea of what you will see is not matched by reality. A casual viewer would most likely overlook the masked Irish Republican Army (IRA) soldier at the
lower right of the frame during a funeral for a hunger striker in Belfast, Northern Ireland, in 1981.
Habituation To protect itself from overstimulation and unnecessary images that might fatigue and
confuse, the mind tends to ignore visual stimuli that are a part of a person’s every day, habitual activities.
When you walk or drive to school or work the same way every day, your brain will ignore the sights along
your route. Many people like to travel to new areas because the images experienced in an unfamiliar
place often are striking and interesting. One way to prevent your mind from thinking habitually is to search
for new ways to think about familiar objects or events in your daily life so that you save money on a flight
to Amsterdam. Practicing creative thought readies your mind to think actively about new images when you
see them. Walker Evans, one of the most famous photographers in the medium’s history, produced a
series of pictures of everyday tools—a pair of pliers, a wrench, and so on—using high-quality studio
lighting and camera techniques to celebrate their often overlooked designs. Accompanying his 1955
portfolio in Fortune magazine titled “Beauties of the Common Tool,” Walker wrote, “Almost all the basic
small tools stand, aesthetically speaking, for elegance, candor, and purity”. Likewise, another master of
photography, Edward Weston, photographed ordinary objects such as a seashell, a bell pepper, and a
toilet seat for the same reason. Looking at their images, you can’t help but find the sublime in their
banality.
Two pages from Fortune magazine with photographs by Walker Evans. Former Farm Security Administration (FSA) photographer Evans made a series of pictures
of everyday objects under studio lighting that demonstrate the cognitive element of habituation. Without being able to study Evan’s photographs, you might miss the
metal texture of the trowel and the crescent wrench, the contrast of curved versus straight lines, and the shadows that
define their thickness.
Salience A stimulus will be noticed more if it has meaning for an individual. If you recently met someone
you like whose favorite food is from India, whenever you smell curry, hear other people talking about the
country, or watch Slumdog Millionaire (2008), you will be reminded of that person. If you are hungry you
will notice the smells of cooking food emanating from an open window. A trained biologist will see more in
a slide under a microscope than the average person will; both individuals see all there is to see under the
microscope, but what the biologist sees is consciously processed in the mind.
The cognitive element of salience refers directly to Aldous Huxley’s famous phrase, “The more you know, the more you see.” If you are a medical research, you will
more likely find meaning from this electron microscope view of the HIV virus (small green spheres) attacking a white blood cell. As a general rule, the more salience
you bring to any visual message the more interesting it will be to you.
Dissonance For many, trying to read while a television or stereo is loudly playing in the same room is
difficult because the mind tends to concentrate on only one activity at a time. A book is set aside the
moment a television program or the lyrics of a song become interesting. Television programs that combine
written and spoken words, multiple images, and music run the risk of creating visual messages that the
viewer cannot understand because of all the competing formats. A classic example of dissonance came
from the cable network CNN when it introduced in August 2001 its new version of the 1982 staple
“Headline News.” Television critics across the country voiced their negative opinion about the format
because of all the competing bits of information—an anchor-person talking on camera or as a voice-over,
still and/or moving images, graphics with headlines, stock details, weather reports, news “crawls” that
updated news events, and advertising logos. Because of the negative feedback, CNN toned down most of
its visual display, but kept the crawl along the bottom of the screen.

Dissonance can also happen if a room is too warm or too cold, if there is a personal matter that you
cannot stop thinking about, or if there are too many road sign advertisements competing for your attention
on a highway. Too many distractions and you will find it difficult to concentrate on a single visual message
“Little Chapel,” Las Vegas, 1999, by Gerry Davey. Sandwiched between the Yucca Motel with its “UEEN” beds and the Oasis Motel with its adult movies, fantasy
rooms, and Jacuzzis, you might miss the fact that if you get married in the Little Chapel of the Flowers you can get your wedding webcast FREE. Dissonance is a
result of so many elements within a visual array that important details are missed.
Culture As a manifestation of the way people act, talk, dress, eat, drink, behave socially, and practice
their religious beliefs, cultural influences have a tremendous impact on visual perception. Religious icons,
state and country flags, T-shirt designs, and hairstyles all have individual and cultural meanings. If you are
aware of the signs that are a part of a particular culture you also will comprehend some of the underlying
reasons behind their use. Culture isn’t simply the concept of a country’s borders or the idea of high-class
or upper-class “culture.” It spans ethnicity, economic situations, places of work, gender, age, sexual
orientation, physical disability, geographic location, and so on. Franc Boas, a leader in the field of
anthropology in his book Anthropology and Modern Life explained that culture is “the community of
emotional life that rises from our everyday habits.” For Boas culture is more important than race. Culture
determines the importance of the signs that affect the people who live with and among us.
One of the most important determinants of what you notice and what you miss visually is your cultural identity. For this group of persons waiting for an Easter
parade in the French Quarter of New Orleans, what each person notices may be a factor of race, age, gender, weight, and alcohol intake.
Words Although we see with our eyes, most of us think with words. Consequently, words, like memory
and culture, profoundly affect our understanding and subsequent long-term recall of an image. One of the
strongest forms of communication is when words and images are combined in equally respectful ways.
That is why magazines, newspapers, and websites regularly have captions for each photograph and news
anchors and broadcast journalists use voice-overs to explain what is being shown.

Semiotics and cognitive approaches to visual communication state that the human mind is an infinitely
complex living organism that science may never fully understand. But meaningful connections between
what people see and how they use those images arise when mental processing is viewed as a human
rather than an automatic, mechanical process.

The sensory theories of gestalt and constructivism and the perceptual theories of semiotics and cognitive
teach visual communicators to look closely at their world, create designs that attract attention, be mindful
of the varied messages that come from images, and understand the possible mental enhancers and
distractions to anything that might be attempted graphically.
A photograph of a haphazard stack of snow skis of various lengths and conditions has little meaning without the addition of the sign posted on a building at the
Snakedance Condominiums in the Taos Ski Valley. Skiers are free to take whatever they want in order to enjoy the thrill of the slopes. However, with no evidence of
snow anywhere in the picture (taken in June), perhaps it is not the best time to enjoy that outdoor activity.