Information Technology and Economic Performance: A Critical Review of the Empirical

Evidence
JASON DEDRICK, VIJAY GURBAXANI, AND KENNETH L. KRAEMER
University of California, Irvine
For many years, there has been considerable debate about whether the IT revolution was paying off in higher productivity.
Studies in the 1980s found no connection between IT investment and productivity in the U.S. economy, a situation referred to as
the productivity paradox. Since then, a decade of studies at the firm and country level has consistently shown that the impact of
IT investment on labor productivity and economic growth is significant and positive. This article critically reviews the published
research, more than 50 articles, on computers and productivity. It develops a general framework for classifying the research,
which facilitates identifying what we know, how well we know it, and what we do not know. The framework enables us to
systematically organize, synthesize, and evaluate the empirical evidence and to identify both limitations in existing research and
data and substantive areas for future research.
The review concludes that the productivity paradox as first formulated has been effectively refuted. At both the firm and the
country level, greater investment in IT is associated with greater productivity growth. At the firm level, the review further
concludes that the wide range of performance of IT investments among different organizations can be explained by
complementary investments in organizational capital such as decentralized decision-making systems, job training, and business
process redesign. IT is not simply a tool for automating existing processes, but is more importantly an enabler of organizational
changes that can lead to additional productivity gains.
In mid-2000, IT capital investment began to fall sharply due to slowing economic growth, the collapse of many Internet-
related firms, and reductions in IT spending by other firms facing fewer competitive pressures from Internet firms. This reduction
in IT investment has had devastating effects on the IT-producing sector, and may lead to slower economic and productivity
growth in the U.S. economy. While the turmoil in the technology sector has been unsettling to investors and executives alike, this
review shows that it should not overshadow the fundamental changes that have occurred as a result of firms’ investments in IT.
Notwithstanding the demise of many Internet-related companies, the returns to IT investment are real, and innovative companies
continue to lead the way.
Categories and Subject Descriptors: K.4.1 [Computers and Society]: Public Policy Issues; K.4.2 [Computers and Society]: Social
Issues; K.4.3 [Computers and Society]: Organizational Impacts; K.6.0 [Management of Computing and Information Systems]:
General—Economics
General Terms: Economics, Performance, Management
Additional Key Words and Phrases: Information technology, productivity, economic performance, firm, industry, country,
management practices
This review has been supported by grants from the U.S. National Science Foundation (Computer, Information and Engineering
Directorate, Information and Intelligent Systems Division, Program on Digital Society and Technologies), and IBM Global
Services. Authors’ addresses: Center for Research on Information Technology and Organizations, Graduate School of
Management, University of California, Irvine, 3200 Berkeley Place, Irvine, CA 92697; email:
{jdedrick,vgurbaxa,kkraemer@uci.edu}. Permission to make digital/hard copy of part or all of this work for personal or
classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the
copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of ACM, Inc.
To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.
c 2003 ACM 0360-0300/03/0300-0001 $5.00
ACM Computing Surveys, Vol. 35, No. 1, March 2003, pp. 1–28.
2 Dedrick et al.
1. INTRODUCTION
“You can see the computer age everywhere but in the productivity statistics.”
[Solow 1987].
“Despite differences in methodology and data sources, a consensus is building that the remark- able behavior of IT prices
provides the key to the surge in economic growth.”
[Jorgenson 2001].
1.1. Background
There has been a long-running debate in the business press and the information systems and economics literature
over whether the information technology (IT) revolution is paying off in higher produc- tivity. The first studies,
conducted in the 1980s, found no connection between IT investment and productivity at the level of firms,
industries, or the economy as a whole [Loveman 1994; Roach 1987, 1989, 1991; Strassmann 1990]. Skeptics
pointed out that heavy IT investment had occurred concurrently with the productivity slow- down that began in 1973
in the U.S.
This so-called productivity paradox stimulated economists, management scientists, and information systems
researchers to conduct more rigorous scientific analyses of the relationship be- tween IT and productivity
[Brynjolfsson 1993, 1996; Bresnahan 1999; Brynjolfsson and Hitt 1995, 1996, 1998; Oliner and Sichel 2000;
Jorgenson 2001; Jorgenson and Stiroh 2000; Bosworth and Triplett 2000; Council of Economic Advisers (CEA)
2001]. These studies, which used larger datasets and more refined research methods, revealed positive and
significant impacts from IT investments at the firm and country level. Moreover, some of these studies showed that
the economic boom and surge in productivity of the late 1990s was largely due to heavy investment in IT and the
growth of the Internet.
The debate over IT and productivity then shifted to whether the IT-led economy would lead to permanent
improvement in the prospects for economic growth, or whether it was a temporary phenomenon, with much of the
acceleration in produc- tivity driven by the business cycle and con-
centrated in just a few sectors of the econ- omy, a point of view espoused by Gordon [2000].
Given the continuing debate about whether IT investments pay off, this research review critically evaluates the
large body of evidence-based research on the subject. The purpose is to critically survey the published research on
IT and economic performance to determine what we know and what we do not know about the returns to IT
investments. The goal is to help direct future research into poten- tially productive channels so that it can contribute
to knowledge about whether or not, as well as how, IT investments can be effectively introduced and managed for
greater payoffs.
Specifically, the aim of this article is to (1) organize and integrate the research on returns to IT investment in a
way that adds understanding to work in the area, (2) provide an unbiased and objective view of the documented
returns (or lack of returns) to IT investment at three levels of analysis—firm, industry, and country, (3) identify the
factors that contribute to payoffs from these investments, (4) eval- uate issues in current research, and (5) identify
opportunities for future re- search. It is intended that this review will help readers to understand this important area
of research, stimulate experts to deal with unresolved issues in ongoing and future research, and assist senior execu-
tives in future decision-making about IT investments in their organizations.
We begin with a discussion of the scope of the literature reviewed and the organi- zation of this article.
1.2. Scope of Literature Review
This review examines more than 50 em- pirical studies based on economic anal- ysis that have appeared between
1985 and 2002.1 Early studies were based on
1 There is other writing on the subject that is not part of this review. For example, Laudon and Marr [1994] brought political
perspectives to bear on understand- ing the productivity paradox when they argued that productivity returns from IT investment
might not
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
Information Technology and Economic Performance 3
Fig. 1. IT and economic performance—framework for literature review.
small samples and limited data, whereas recent studies have been able to take ad- vantage of both better data and
larger samples, including time-series data. The review concentrates mainly on studies whose results have been
published in peer-reviewed scholarly journals such as American Economic Review, Communica- tions of the ACM,
Information Systems Research, Journal of Economic Perspec- tives, Journal of Management Informa- tion Systems,
Management Science, MIS Quarterly, Organization Science, Quar- terly Journal of Economics, The Infor- mation
Society, The Brookings Papers, and World Development. These journals are known for having high standards for
review and acceptance and there- fore are most appropriate for a critical review that seeks to achieve an objec- tive
and balanced perspective on the re- search. The review also includes a few other works because of their significance
or because they help to round out the re- search in an area. Finally, Brynjolfsson [1993] and Brynjolfsson and Yang
[1996] have provided excellent reviews of earlier research.
even be an objective in the macroculture of some or- ganizations. Similarly David [1990] and King [1996] brought historical
perspectives to the IT and produc- tivity debate.
1.3. Organization of this Review
The review is organized as follows. Section 2 introduces a conceptual frame- work for thinking about IT and
economic performance. Sections 3 through 5 assess the literature at three levels of analysis: firm, industry, and
country. Section 6 presents both substantive and method- ological issues that need to be addressed in future research,
and Section 7 presents specific, high-priority recommendations for future research based upon key deficits in the
current body of knowledge. Finally, Section8 discusses limitations of the survey and highlights some major
conclusions.
2. CONCEPTUAL FRAMEWORK
In order to organize the prior research and to identify gaps for future efforts, we developed a conceptual framework
(see Figure 1) that allows us to map and as- sess the research findings. The framework helps to define the key
variables and re- lationships addressed in the different re- search studies reviewed herein. Moving from left to right
in Figure 1, the frame- work identifies the various inputs (labor and capital) to the production process and
complementary factors of production that influence the production process, and en- ables an assessment of the
contribution of
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
4 Dedrick et al.
those inputs to outputs (value added, gross domestic product) and to various outcomes (economic growth, labor
productivity, prof- itability, and consumer surplus). It further distinguishes between firm, industry, and country
levels of analysis. We begin our presentation of the framework by defining the key terms in the title of this review—
IT investment and economic performance.
2.1. Definition of IT Investment
Traditional economic studies of productiv- ity focused on capital such as plants and equipment using aggregate
measures of capital that include all its component cat- egories. In studies of IT and productiv- ity, it becomes
necessary to disaggregate capital into the component categories of investment—IT and the traditional forms of
capital, labeled non-IT. IT investment, broadly defined, includes investments in both computers and
telecommunications, and in related hardware, software, and services. However, as operationally de- fined in nearly
all of the research included in this review, IT investment is limited mainly to computer hardware. In most studies,
investment is defined as an an- nualized value of the stock of computer in- vestments including the depreciated
value of previous investments that are still in service, or as annual spending.
2.2. Definition of Economic Performance
Economic performance can be interpreted in a variety of ways at each level of anal- ysis. At the country level, where
much of the debate has occurred, it usually refers to economic growth, labor productivity growth, and consumer
welfare (Figure 1). Economic growth is the rate of change in real output, or GDP, and is measured at the country
level. Labor productivity growth, or growth in output per worker, is a measure of the efficient use of resources to
create value. It “allows the economy to provide lower-cost goods and services rela- tive to the income of domestic
consumers, and to compete for customers in interna- tional markets” [McKinsey Global Insti- tute 2001, p. 1].
Corresponding measures
focusing on the output of an industry sec- tor are utilized at the industry level.
Clearly, labor productivity growth is also an indicator of the economic perfor- mance of firms. A firm that is more
pro- ductive than its competitors will gener- ally enjoy higher profitability, which is of course, also an important
measure of eco- nomic performance for firms. A more pro- ductive firm will either produce the same output with
fewer inputs and thus experi- ence a cost advantage, or produce higher- quality output with the same inputs, en-
abling a price premium. However, as will be discussed later in the review in re- gard to firm-level research,
competition induces other firms to catch up in produc- tivity. Sustaining higher profits through productivity gains
requires a firm to main- tain productivity levels higher than its competitors. Therefore, over time, prof- its might be
competed away with the re- sult that consumers benefit. This bene- fit is measured as consumer surplus and refers to
the aggregate value realized by consumers from their purchase of a good less the price paid.
2.3. Modeling the Production Process
In order to better understand the IT and productivity debate, it is useful to be- gin with a discussion of the production
process by which inputs are transformed into outputs in firms and economies, and the specific role of IT as a factor
of pro- duction. Economists use two related ap- proaches to modeling the production pro- cess by which inputs are
transformed into outputs. One approach to understand- ing the output of an economic system is production
economics, which uses spe- cific functional forms, called production functions, to model the production pro- cess
[Bresnahan 1999; Brynjolfsson and Hitt 2000]. This approach uses economet- ric techniques to relate the output of a
firm, industry, or economy to the inputs based on estimation models derived from the production function. Inputs
typically accounted for in this approach include labor and capital, including both IT and non-IT capital. Most of the
studies at the
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
Information Technology and Economic Performance 5
firm level use this approach. The primary approach used to model the production process inherent in an economy (or
in- dustry) is growth accounting [Oliner and Sichel 2000; Jorgenson and Stiroh 1999, 2000; Council of Economic
Advisors 2001]. This method also assumes specific proper- ties of the production process and, based on these
assumptions, allocates shares of output to the various inputs to production. Output growth in firms, industries, and
the economy may arise from increases in input levels, improvement in the quality of inputs, and growth in the
productiv- ity of inputs. The focus of the literature reviewed here has been on understand- ing how increased levels
of investment in IT impacts labor productivity. First, labor productivity can increase when workers are provided
with more capital, a phe- nomenon called capital deepening. Sec- ond, technical progress in the production process
or in the quality of output can in- crease the level of output without addi- tional investment in input, a phenomenon
labeled multifactor productivity (MFP). An increase in MFP means that for a fixed level and quality of inputs, a
firm, indus- try or economy is achieving higher levels of output. This form of productivity improve- ment is of great
importance because it re- flects structural gains that are permanent. The framework also posits that there are
complementary factors that influ- ence the payoff from IT investments (Figure 1). At the firm level, these include
organization and management practices [Brynjolfsson 1996; Bresnahan et al. 2002; Brynjolfsson et al. 2000]; at the
industry level, they include industry organization [Melville 2001]; and at the national level, they might include eco-
nomic structure, government policy, and investment in human capital [Dewan and Kraemer 2000]. The following
discussions elaborate the framework for each level of analysis. Indeed, as we shall see below, it is these
complementary factors that have been suggested as helping to explain some of the extraordinary estimates of returns
to IT investments.
The analysis that results from these ap- proaches indicates the relative contribu-
tion of labor and capital to output, and the relative contribution of the different drivers of labor productivity growth,
as illustrated in Figure 1 above. As will be seen later, the assumptions that various researchers make in doing the
analysis can have substantial impacts on the re- sults and implications.
2.4. Role of IT in the Production Process
Much of the debate among economists has addressed these economic performance is- sues in the aggregate, at the
country and industry levels. Yet, the decision makers who choose to invest in IT are managers who deploy IT for
use in their organiza- tions and who use investment criteria that are related to the outcomes at the level of the firm.
While labor productivity is cer- tainly one often-used criterion, managers also use measures such as profitability,
market share, margins, and product vari- ety and quality as justifications for invest- ment in IT systems.
In order to understand the overall im- pact of IT at the firm level, it is use- ful to begin by thinking about the qual-
itative impacts of introducing IT into a firm’s production processes. Past re- search has distinguished between using
IT to automate processes, to provide bet- ter information, and to transform entire processes [Zuboff 1988]. The
impact of automation is primarily the direct sub- stitution of capital for labor, consistent with capital deepening. For
example, a cashier at a retail chain store using a computer-based information system such as a scanner can process a
transaction in less time. The impact of improved in- formation is that it allows workers and managers to make
decisions more effec- tively. For example, information provided by the store-based system allows the man- agers to
better manage inventory. Trans- formation impacts occur when a firm re- designs a process to achieve significantly
higher levels of productivity. In our ex- ample, the firm may redesign its supply chain using a supply chain
management system, of which the store system is a key element.
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
6 Dedrick et al.
 One key difference between IT capital and other forms of capital is the dual roles that IT can play in a firm. First,
like other types of capital, IT can be used directly as a production technology to improve la- bor productivity, as in
the case of a bank’s transaction processing system. However, research suggests IT has its greatest im- pact in its
second role as a technology for coordination [Bresnahan 1997; Gurbaxani and Whang 1991; Malone et al. 1989]. In
this literature, IT is viewed as an espe- cially potent technology that has a signif- icant impact on the costs of
coordinating economic activity both within and between organizations. Research in this arena sug- gests that the
unique value of IT is that it enables fundamental changes in busi- ness processes and organizational struc- tures that
can enhance MFP.
2.5. The Aggregate Impact of IT
While IT is deployed at the level of the firm, the analysis of the impact of IT at the level of an industry or the
economy allows researchers to answer a related but different set of questions that are of critical importance. At the
level of the economy, it furthers an understanding of the role of IT and the IT sector in foster- ing economic growth
and, ultimately, the wealth of a country. It also facilitates a discussion of whether steady-state growth rates are
higher in an IT-intensive econ- omy. It enables the documentation and understanding of industry differences, and an
examination of the role of industry characteristics such as structure and regulation in moderating the returns to IT
investment.
While IT can increase productivity via capital deepening and via MFP growth, the results might be substantially
differ- ent in different industry sectors. In par- ticular, an important distinction has to be made between the IT-
producing and IT-using sectors. The IT-producing sectors are those which manufacture semicon- ductor, computer,
or telecommunications hardware or provide software and services that enable these technologies to be used
effectively in organizations. The IT-using
sectors are all the other sectors of the economy that apply IT as part of their operations in order to achieve greater
effi- ciency and effectiveness. They include sec- tors such as manufacturing (durable2 and nondurable), wholesale
and retail trade, finance, insurance and real estate, busi- ness and professional services, and so on. As we shall see
below, there is no ques- tion that there have been very rapid im- provements in productivity and in MFP in the IT-
producing industries, particularly in computer hardware and components as a result of research and development on
product and process technologies.
A critical question is whether there have been similar gains in productivity and MFP outside the IT-producing
industry and, if so, whether those gains can be at- tributed to investment in IT capital. Has the use of IT allowed
industries to achieve superior production methods than were previously unavailable? Put differently, are there
spillovers from IT-producing in- dustries to IT-using industries?
2.6. Implications for this Review
As the foregoing suggests, a comprehen- sive review of the payoffs from IT invest- ment must examine the returns
to this in- vestment at the disaggregate level of the firm as well as the aggregate level of in- dustries and the
economy since the na- ture of the payoffs at these levels may be quite different. If IT investments are in- creasing
productivity at the firm level, it might be the case that in the aggregate they will also increase the productivity of
entire industries and countries. How- ever, it is possible that gains will show up at one level and not the other,
depending on whether individual firms capture the returns on their investment, or whether some or all of the gains
are competed away and flow to consumers, creating social benefits but not providing a measurable return to the
firms making the invest- ment. Furthermore, in addition to firm- specific factors, industry characteristics
2 Ordinarily, durable goods manufacturing includes the IT-producing sector.
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
Information Technology and Economic Performance 7
also affect the payoffs that firms within an industry receive from their IT invest- ments and how these payoffs are
shared between firms. It is therefore important to examine the results at both disaggregate and aggregate levels.
In order to understand whether IT in- vestment results in greater productivity, we next look systematically at the
re- search on the returns from IT investments for each level of analysis—firm, industry and country—in that order.
We examine the research as a basis for understand- ing the nature, extent, and limitations of payoffs from IT
investments. We review the evidence provided by numerous sys- tematic, empirical studies. We summarize some of
the major studies in the Appendix and Tables I and II.
3. FIRM-LEVEL RESEARCH
While the productivity paradox as orig- inally framed focused on aggregate country-level productivity statistics, ac-
tual IT investments are made by organiza- tions, mostly firms, that are interested in their own return on investment,
not that of the country as a whole. Knowing that IT investment improves aggregate produc- tivity does not imply
that individual firms enjoy similar benefits. In fact, there may be significant social benefits from IT in- vestments
that increase consumer welfare but are not captured by the firms making those investments. Therefore, it is of great
concern to business and technology executives whether their IT investments are paying off at the level of the firm.
3.1. IT and Firm-Level Productivity
Motivated by the productivity paradox, many firm-level studies were launched in the 1980s and 1990s. Early studies
were unable to show that IT investments led to payoffs, in most cases because of inad- equate data on IT
investments and small sample sizes [Brynjolfsson and Hitt 1996, 2000; Brynjolfsson and Yang 1996]. Most
discouraging were several studies of ser- vice firms, such as banks and insurance firms, which showed weak or
nonexis-
tent links between IT and productivity, but where output measurement is notori- ously difficult [Franke 1987;
Strassmann 1990; Alpar and Kim 1991; Harris and Katz 1991]. Some studies of manufactur- ing firms did show
positive returns on IT investment, partly because it is easier to measure the output of manufacturing and adjust for
improvements in quality [Weill 1992; Barua et al. 1995]. These studies began to highlight the importance of the
accurate measurement of outputs, partic- ularly in the technology-intensive service industries where the largest
investments in IT capital were being made.
Starting around 1993, more rigorous studies with larger samples were being re- ported by researchers
[Brynjolfsson 1993, 1996; Bresnahan 1999; Brynjolfsson and Hitt 1995, 1996, 1998; Lichtenberg 1995]. These
studies involved large U.S. corpo- rations, using data on IT capital invest- ment from market research firms and
from surveys of chief information officers and other executives, coupled with financial data from reliable sources.
The research used econometric techniques based in pro- duction economics that relate firm output (measured as
value added by a firm) to a set of inputs including labor hours, non- IT capital stock, and IT capital stock, and
estimated the marginal product or output elasticity of IT capital.3
Each of these studies found that IT in- vestments contribute to firm productivity, and show higher gross marginal
returns than non-IT investments. The fact that these researchers found a strong relation- ship between IT capital and
productivity that was not evident in earlier studies may partly reflect the fact that the data was more recent, that
levels of IT investment had increased, making it easier to distin- guish its contribution, and that over time firms were
learning to apply IT capital more productively. They may also simply reflect better data sets and analytical tools that
make it possible to isolate and mea- sure the true impacts of IT investment.
3 The output elasticity of IT is the increase in value added associated with a 1% increase in IT investment.
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
8 Dedrick et al.
 More recently, Brynjolfsson and Hitt [2000] have found that payoffs to IT in- vestment occur not just in labor
productiv- ity increases but also in MFP growth, and that the impact on MFP growth is maxi- mized after a lag of 4
to 7 years. Gilchrist et al. [2001], using the same dataset, focus on the manufacturing companies in the sample and
show that IT has a substan- tial and contemporaneous impact on labor productivity growth and on MFP growth in
the durable goods sector, which exceeds the impact that would be predicted by its factor share. They find that, in the
non- durable goods sector, the returns to labor productivity accrue primarily via capital deepening, and are
consistent with IT fac- tor share. Moreover, these returns are cor- related with decentralized computing ar-
chitectures, suggesting that the diffusion and networking of computing throughout the organization contributes
substantially to the payoff.
In addition to these U.S. studies, a few other studies have been conducted on firms in other countries. Greenan et
al. [2001] analyzed data on French firms’ IT investment and productivity and came to results consistent with the
find- ings of Brynjolfsson and Hitt [1996] and Lichtenberg [1995] for U.S.-based firms. By contrast, Lal [2001] did
not find a rela- tionship between IT investment and pro- ductivity in Indian garment makers. This is consistent with
the cross-country stud- ies, which are discussed later [Dewan and Kraemer 2000; Pohjola 2001], that have found a
strong relationship between IT and productivity in developed countries, but not in developing countries. With low
unit costs of labor and higher capital costs, it is not surprising that there are fewer op- portunities for capital-labor
substitution in developing countries. Also, Lal’s sample included many small and medium-sized firms, a group not
included in most U.S. studies.
Most of the studies found that IT in- vestments were associated with higher marginal product than other capital
in- vestments. These are translated into “excess returns” by some authors, who pointed out that, in theory, all invest-
ments should pay the same risk-adjusted return at the margin. These returns do need to be adjusted to account for the
high rate of obsolescence of IT capital, so that the net returns are much lower. However, Brynjolfsson and Hitt
[1996] and Lichtenberg [1995] found that after sub- tracting standard estimates of the cost as- sociated with the
obsolescence of IT cap- ital of up to 42% per year from the gross returns, the net returns from IT were still higher
than those of non-IT investments. These results of firm-level studies have sometimes been taken to imply that firms
are systematically underinvesting in IT, given the high marginal returns to such investments.
Some answers have been proposed to this question in the literature and others will be suggested here, but we
would warn that claims of systematic underinvest- ment in IT should be viewed cautiously. First, as Brynjolfsson
and Hitt [2000] pointed out, the true cost of such invest- ments may be underestimated. All studies include the direct
investment in computer hardware; others attempt to include labor, software, and services, but it is difficult to
estimate these with a high degree of preci- sion. Importantly, they do not include the costs of complementary
investments such as training and process reengineering that can be much larger than the actual di- rect investment in
IT. If these costs are in- cluded on the investment side of the equa- tion, the returns might look more modest.
Moreover, taking into account the large standard deviations in the payoffs docu- mented by many studies, it is
possible that the net returns to IT investments are con- sistent with non-IT investments.
Given these caveats, it is still possible that IT investment does show higher than normal returns. There are several
reasons why this could be so. IT investment might be riskier than other investment. Firms invest when the net return
is sufficient to cover the risk-adjusted cost of capital. This would argue that returns need to be higher to compensate
for the additional risk. Most studies do not assess the impact of the risk of these investments. Moreover, there might
be adjustment costs. It is difficult
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
Information Technology and Economic Performance 9
and costly for firms to introduce new IT innovations. With decreasing prices for IT, the optimal level of IT
investment and cap- ital stock increases in steady state. How- ever, firms face real costs and delays due to the
duration of software development, re- tirement of older systems, and changes in practices that suggest that firms
might not achieve these optimal levels in the short run. It is therefore difficult to conclude that the “excess returns”
found in firm- level studies imply that firms are system- atically underinvesting in IT, or that man- agers are acting
irrationally.
This recent research also highlights other interesting questions that remain unresolved regarding the payoffs from
in- vestments in IT capital. First, it is not well understood why firms in different industries accrue different payoffs.
For example, it would be valuable to iden- tify the specific characteristics of durable goods manufacturing firms that
enable them to achieve higher returns relative to nondurable goods manufacturing firms. Second, a better
understanding of the tim- ing of the payoff from investments in IT capital is also needed. Clearly, a firm’s many
individual investments in specific systems will have different periods over which the payoffs will be realized. Some
systems will realize immediate payoffs, while others will realize payoffs after a lag. The duration over which the
payoffs will be realized will also vary. Some will have short-term impacts and others will have longer-term impacts.
This understanding will go a long way toward resolving the de- bate on whether the impacts of these in- vestments
are contemporaneous with the investiments or occur in the future.
3.2. Variance Among Individual Firms
The preponderance of evidence points to positive and significant returns to IT in- vestment among firms. Clearly,
higher lev- els of IT investment are associated with higher levels of productivity across a large sample of companies,
and this has been true since the mid-1980s at least. How- ever, looking at a scatter plot of IT invest- ment and
productivity, as Brynjolfsson
and Hitt [1995, 1996] have presented in several of their papers, one is struck by how widely scattered the actual data
points are around the trend line. This leads to the next major finding in the firm- level data.
The productivity impacts of IT in- vestments vary widely among different companies.4 In other words, some
firms use IT much more productively than others. Brynjolfsson and Hitt [1995] es- timated that these “firm effects”
may account for as much as half of the produc- tivity benefits attributed to IT investment in their earlier work, but
stated that the elasticity of IT remains positive and sig- nificant even after firm effects are taken into account. Still,
this raises the question of what causes these firm effects.
Two factors stand out. First, there are idiosyncratic firm characteristics such as market position, rigidities in cost
struc- tures (e.g., labor contracts), brand recog- nition, or the vision and leadership abili- ties of key executives,
which affect a firm’s strategic options and therefore its poten- tial to derive benefits from IT investment. These can
change over time, but are not easily manipulated by management in the short run.
Second, there are specific features of organizational structure, strategy, and management practices that can be
com- pared systematically across companies. The management of a firm, through restructuring, new management
control systems, the redesign of processes, or by upgrading employee training, can directly influence these features.
3.3. Impact of Business Practices on Value
of IT Investments
Management practices and complemen- tary investments explain part of the varia- tion in IT payoffs. Loveman’s
[1994] early analysis of manufacturing firms, which found evidence of net marginal benefit for non-IT investments
but not for IT investments, highlighted complementary
4 The variance of returns to IT capital is larger than the variance of returns to non-IT capital.
ACM Computing Surveys, Vol. 35, No. 1, March 2003.
10 Dedrick et al.
organizational factors as a possible expla- nation for the research results. He argued that the evidence could be
interpreted as management failure to effectively inte- grate IT with the firm’s business strategy, human resource
management strategy, and efficient resource allocation. That is, management did not implement the orga- nizational
changes that should accompany IT investment in order to create value.
Subsequent studies at the firm level explicitly show that the value of IT in- vestments is substantially impacted by
the structure and business practices of the firms making the investment. For instance, Weill [1992] showed that the
quality of a firm’s management and its commitment to IT enhances the contri- bution of IT investments to firm per-
formance. Francalanci and Galal [1998] showed that firms with a higher propor- tion of information workers gain
more from their IT investments than those with a lower proportion. Tallon et al. [2000] found that aligning IT with
business strategy increased the payoffs from IT in- vestments. In addition, firms with higher levels of investment
gained greater pay- offs from alignment. Devaraj and Kohli [2000] found that business process reengi- neering
enhanced the payoffs in firms that also made greater IT investments. Ramirez et al. [2001] found that organi- zations
which invested more in IT and im- plemented management practices such as employee involvement and total quality
management received higher IT returns.
Black and Lynch [1997] studied the im- pacts of workplace practices, IT capital, and human capital development
on pro- ductivity. They found that what affected productivity was less the presence or ab- sence of a particular
management prac- tice, such as total quality management, than the way in which the practice was implemented.
Particularly important was employee involvement—for instance, the proportion of workers involved in regular
decision-making in a plant.
In addition to these studies of individ- ual management practices, Brynjolfsson and Hitt [2000] and Bresnahan et
al. [2002] showed that firms with a cluster
of management practices, including de- centralized decision-making (which they called organizational capital) along
with high levels of IT investments, outper- form all others. Interestingly, firms with traditional centralized
organizations and high IT investments actually do worse than similar organizations that invest less in IT.
While the evidence shows the benefits of certain classes of management practices, these can be difficult to
translate into specific actions for individual companies. It is logical that executives and man- agers can improve the
performance of their IT investments by combining these investments with proven complementary managerial
practices. However, the re- search evidence is limited as to specific links between management practices and
productivity. In particular, understanding the relationship between firm-specific fac- tors and management practices
is critical and by definition cannot be addressed in large-sample studies. For instance, the fact that decentralized
firms earn higher returns to their IT investments than centralized firms on average is not sufficient to advise a
particular firm to switch from a centralized structure to a decentralized one. Given the firm’s id- iosyncratic
characteristics, a centralized structure might be more appropriate.
3.4. IT and Firm Financial Performance
There is mixed evidence at the firm level as to the impacts of IT capital on financial performance measures such as
profitability or market value, partly because the linkage is less direct. While IT investments can directly affect a
firm’s output and many operational indicators (e.g., inventory turnover, plant produc- tivity, product quality), a
firm’s financial performance is determined by a wider range of strategic and competitive factors that go beyond
productivity.
Several studies show a relationship between IT investment and intermedi- ate measures of operational
performance. Barua et al. [1995] found that IT in- vestment affects intermediate measures
ACM Computing Surveys, Vol. 35, No. 1, March 2003.