The Science of Subtle Signals
The Science of Subtle Signals
The Science of Subtle Signals
1
features business models
2
Signals by Mark
Buchanan
Previous pages: Mark Buchanan
Professor Alex Pentland (mark.buchanan@wanadoo.fr)
(seated) and researchers at is the author of The Social
the MIT Media Lab Human Atom: Why the Rich Get
Dynamics Group (from left to Richer, Cheaters Get Caught,
right): Benjamin Waber, and Your Neighbor Usually
Agnes Chang, Taemie Kim, Looks Like You (Bloomsbury
and Koji Ara. USA, 2007). Formerly an editor
with Nature and New Scientist,
he was a guest columnist
for the New York Times in
2007 and holds a Ph.D. in
physics from the University of
Virginia. His Weblog is http://
thesocialatom.blogspot.com/.
In 2006, when Vertex Data Science — a US$724 sounds perky and inviting. If operators do it right,
million private company based near Liverpool, England, they’re almost certain to be successful.” Armed with this
features business models
and one of the world’s largest providers of call center understanding, a company like Vertex can train its oper-
outsourcing — wanted to improve the performance ators to converse more effectively, and can seek new
of its telephone sales operators, the managers went look- hires who exhibit these speech patterns. If a call starts
ing for an unusual kind of self-understanding. They going badly, a supervisor can detect the signs quickly
enlisted the aid of Alex Pentland and his colleagues from enough to switch it to another operator. Early experi-
the Human Dynamics Group at the Massachusetts ments have suggested that these insights can improve a
Institute of Technology’s Media Lab, the elite research company’s telephone sales performance by 20 percent or
institute for digital technology founded by technology more. And the same is true of other forms of corporate
pioneers Nicholas Negroponte and Jerome Wiesner. communication. “In pitching business plans, for
The researchers at the Human Dynamics Group were instance,” Pentland points out, “consistency of tone and
best known for their experiments in human–machine pace is key to getting your plan rated highly.”
interplay and wearable computing: using portable This story is a straightforward tale of managerial
devices built into eyeglasses and clothing to track move- intervention and success. But it also throws down a
ment and other human activity. They traveled to Vertex’s profound challenge to the prevailing views of organiza-
operations offices in Inverness, Scotland, to set up elec- tional effectiveness. Most explanations of human behav-
3
tronic devices that analyzed the speech patterns of the ior in the business world presume that people — be
operators on the call center floor. The devices captured they employees, consumers, or executives — are influ-
neither the specific words that the operators used nor enced most by meaning and reasoning. It’s what gets
the logic of their conversations, but only the physical said that matters, not how it is said. But the performance
voice signal: the measured variations in tone and pitch. of these telephone operators and a growing volume of
Even so, Pentland and his researchers predicted accu- other evidence suggest that this view is seriously flawed.
rately, after only a few seconds of listening, the ultimate In a wide variety of facets of everyday business, the
success or failure of almost every call. keys to sustained success may actually lie in understand-
Photographs by Peter Gregoire
Successful operators, it turned out, speak little and ing the kinds of signals that are ordinarily overlooked:
listen much. When they do speak, their voices fluctuate tone of voice, body language, the ways people congre-
strongly in amplitude and pitch, suggesting interest gate (or don’t), the time spent on tasks, the rhythms of
and responsiveness to the customer’s needs. Operators workplace activity, and the patterns of social networks.
who speak with little variation come across as too deter- Those on Pentland’s team — and their counterparts at
mined and authoritative, but by speaking invitingly, other research institutions, such as Xerox’s Palo Alto
being responsive but not pushy, a skilled operator can let Research Center (PARC) and Intel Research in Seattle
callers find their own way to a sale. “Like a mother — are designing new ways to track and make sense of
speaking singsong to a baby,” says Pentland, “variation such indicators. The resulting new science of subtle sig-
What if sensors could transform
organizational research much
as microscopes transformed medicine
in the 18th and 19th centuries?
nals may lead not just to more profitable sales pitches, about a dozen researchers who have developed a range of
but also to a richer, deeper understanding of the practice small, wearable electronic devices that can easily and
corporate behavior thoroughly, researchers like Pentland negotiations. One student played a middle manager tak-
and his team have to monitor and analyze people’s behav- ing a job in a new division, and the other the vice pres-
ior in unprecedented detail, putting potentially sensitive ident of that division. They were asked to negotiate the
data on subtle personal cues and social habits into perma- manager’s salary package, with real monetary rewards at
nent computer storage. It’s not yet clear how researchers stake for the participants. The negotiations often lasted
an hour or more. Yet in just five minutes, an electronic College, researchers asked experimental subjects to say
sensor could predict with 87 percent accuracy which which of three lines on a paper matched the length of
person would come out on top, merely by cueing in to another line, using lengths so different that the correct
bodily movements and manner of speech, ignoring answer was obvious. If they heard a number of people
words and strategy. As the sensor data revealed, success- give the same wrong answer, many people followed
ful middle managers tended to be strong on “mirroring” along with the crowd, completely ignoring the clear
behavior — unconscious mimicking of the gestures and input of their senses. Recent experiments conducted
movements of their conversational partners. This with magnetic resonance imaging (MRI) machines by
demonstrated empathy and understanding. In contrast, Gregory Berns of Emory University suggest that peer
the most successful vice presidents tended to talk more pressure can alter how people actually see the lines.
and control the pace of the conversation, a social behav- In other words, people in group situations don’t
ior that the researchers referred to as “engagement.” For consciously weigh the options and then deliberately (or
both participants, a consistent emphatic tone, conveying timidly) choose to conform. Instead, the conforming
confidence, was also critical. happens automatically and unconsciously. Those dy-
This type of research confirms in the business set- namics happen so often, and so consistently, that they
ting what some psychologists have suspected for years inevitably play a role in the ways people make decisions
— that human behavior can often be predicted with re- in the business world. But this unconscious behavioral
leading directors were all extremely charismatic and build up accurate pictures of the networks of friends or
certain of themselves; everyone else went along with co-workers to which the students belonged and identify
features business models
whatever they said, almost without thinking. “This ex- their most important social links.
perience really affected me,” says Pentland. He began The ultimate aim of this kind of work — as anoth-
studying the scientific literature on nonlinguistic human er collaborator, Mark Mortensen of the Sloan School,
communication, a body of research that is extensive, but points out — is to go far beyond the capabilities of tra-
mostly qualitative. And then he focused on building ditional social network analysis, which mostly relies on
devices to measure that communication. “You need human recall, with all its attendant weaknesses. “A lot of
instruments,” he says, “because as people we can’t really workplace communication takes place through sponta-
observe others objectively.” neous interactions, the watercooler kind of stuff,” he
The instruments he and his colleagues developed says. “You can sit people down with paper and ask them
exploit modern telecommunications technology in a who they interacted with, how, when, and so on over the
number of ways. They can program personal digital last three months, but the results are always biased.”
assistants (PDAs) and specially configured “smart” cell Those limitations, says Mortensen, make current
phones to keep track of their owners’ proximity to oth- analyses of team behavior inadequate. “People decide to
ers, using the unique identifiers built into cell phone and work on something, in virtual teams or whatever, and
Bluetooth transmitters to identify each individual’s loca- afterward, if they didn’t kill anyone, they write a book
7
tion. Other electronic badges complement these locators about it. But what they say is purely anecdotal and
with more precise position measurements — based on there’s no science in it. We want to begin building a real
global positioning system (GPS) data, they are accurate science that is quantitative.”
within two meters — as well as capturing audio signals
and measuring upper-body movements with an elec- Self-Awareness, Stress, and Groupthink
tronic accelerometer. It may seem overly reductionist to try to understand
Gathering this data is just the first step. Pentland people through signals from gadgets hooked to their
and computer scientist Nathan Eagle have developed a belts. But a number of major organizations have already
method they call “reality mining” for analyzing and lined up to try it out. In collaboration with Thomas
drawing meaning from the data. In one study, about Malone of the MIT Center for Collective Intelligence,
100 students carried reprogrammed Nokia cell phones Pentland’s research team has begun to use sensors to
around with them for nine months; researchers then observe creative group behavior at a major German
strategy + business issue 48
analyzed the voluminous data set for patterns in the bank. (One preliminary finding: People who maintain
behaviors of both individuals and groups. They found lots of e-mail and face-to-face contact report high job
they could make accurate predictions on where any per- satisfaction and personal productivity; those who social-
son was likely to be seen at a certain time of day, and ize less, even with the intention of getting more work
whom they’d probably be talking with. They could also done, express overall less satisfaction.) Soon, working
with Eric Brynjolfsson at the MIT Center for Digital communication channel, they would largely ignore the
Business, they’ll be helping network hardware company reactions that matter most.
Cisco Systems improve one of its emergency call centers. The sensors could be used in other applications as
The Human Dynamics Group from the Media Lab is well. For example, many studies have shown that work-
also running projects with automobile engineering place burnout is a serious issue that costs companies mil-
teams at Nissan, with several universities, and with a lions each year. But because people tend to hide stress,
Boston hospital. (Indeed, sensor technology may well it can be very difficult, if not impossible, to detect.
have medical applications, as two clinical trials showed Sensors may change all that. In a trial study, Pentland
that a significant lack of social signaling activity, readily and student Michael Sung fixed physiological sensors on
detected by the sensors, correlated strongly with well- students playing poker for real monetary stakes, and
known signs of clinical depression.) monitored bodily movements, skin conductance, and
In the business setting, a company might use this heart rate. They found that they could identify mo-
type of computer-augmented self-awareness to train its ments of especially high stress (as later reported by the
negotiators. Or its sales force. By identifying social sig- participants) with 80 percent accuracy. They could also
have been aware of — that almost all communication ing camera are routinely posted on the Internet as a part
between the two groups passed through one particularly of the search engine’s mapping service, the perceived
skilled and approachable person, who was consequently danger lies in the breadth of observation. People can
overwhelmed and often behind schedule. After identify- never be quite sure what activity will be gathered and
ing this hidden problem, executives introduced other inadvertently exposed in the random tracking of an
go-betweens to share the load and improve the depart- unsupervised set of sensors. Knowing this, people might
ments’ coordination. censor themselves more, thereby cutting back the very
type of informal and free-form creativity that most busi-
Sensors and Sensibility nesses need more of.
Sensors, working all the time or close to it, could gather Pentland, Mortensen, and the other researchers
far more accurate data about information flows happen- insist that the ethical challenges the use of sensors raises
ing on a minute-by-minute basis. In one study, after must be taken seriously if the technology is really to be
constructing a social diagram of a company, the Human beneficial. “Companies shouldn’t just look at this as
Dynamics Group researchers could actually see polariza- another way to spy on employees,” says Pentland. Using
tion taking place — as if the company had been put sensors for monitoring and control would be a surefire
9
under a microscope. “You’ll see two people going at it in recipe for resentment and loss of morale, he says.
a meeting,” says Pentland, “and then polarization grow- Several ideas might help companies prevent
ing around them,” reflected in the way people respond problems. Pentland suggests, for example, that the tech-
to the two main figures, and gather around them in dis- nology ought to be used on a voluntary basis, with indi-
tinct factions. Analysis of the sensory data in this case viduals adopting it because they learn the benefits that it
showed two people, in particular, trying to lead — both brings for both themselves and the company. An organ-
very active, with voice and body language conveying ization could store information on individuals’ own per-
determination and authority. Neither individual showed sonal computers, rather than in a central location. It
the kind of mimicry or voice variation that would con- might also give people the opportunity, at the end of
vey empathy; in other words, neither backed down. each day, to review the data that’s been recorded about
Soon others began to be recruited into the two opposing their activities. They could have the option of deleting
teams. If a manager saw this type of pattern in real time, anything they’d prefer to keep private. The devices
strategy + business issue 48
he or she could tune in to the emerging problem and try might be fitted with an additional button that would
to defuse it — addressing the root of the tension and erase, say, the last 10 minutes of data, or data collection
helping the two sides get through it. might be strictly limited to teams, time frames, and
Another valuable asset of sensors is their ability to workplace settings where there has been explicit agree-
track patterns over time. They show not only who inter- ment in advance to allow the analysis. Although all these
possibilities reduce the amount and quality of data that problem exists. It will monitor team dynamics through
would be gathered, some steps along such lines will be time, catching patterns of stress or stagnation, and inter-
crucial for giving people confidence that their privacy is vening to keep people working together creatively. And
being protected. it will uncover social patterns that today we cannot even
If privacy issues can be resolved, a new world of recognize or talk about, but which can explain, more
organizational understanding may be at hand. History definitively than ever before, the shining success of one
teaches us that data, when it becomes available, leads to company and the dismal failure of another. +
powerful transformations of human understanding and Reprint No. 07307
capability. For example, the great scientific break-
through of Johannes Kepler, working out the laws of
planetary motion, was made possible by the painstaking
Resources