Empirical Analysis of Programming Language Adoption

Leo A. Meyerovich Ariel S. Rabkin

UC Berkeley ⇤ Princeton University
lmeyerov@eecs.berkeley.edu asrabkin@cs.princeton.edu

Abstract 1. Introduction
Some programming languages become widely popular while Some programming languages succeed and others fail. Un-
others fail to grow beyond their niche or disappear alto- derstanding this process is a foundational step towards influ-
gether. This paper uses survey methodology to identify encing it, assisting language designers and advocates in fos-
the factors that lead to language adoption. We analyze tering adoption. Likewise, understanding adoption will aid
large datasets, including over 200,000 SourceForge projects, developers in determining when and whether to bet on a new,
590,000 projects tracked by Ohloh, and multiple surveys of experimental language. To date, the language adoption pro-
1,000-13,000 programmers. cess has not been quantitatively studied at a large scale. This
We report several prominent findings. First, language paper addresses that gap. We use a combination of survey
adoption follows a power law; a small number of languages research and software repository mining to investigate the
account for most language use, but the programming mar- factors that influence developer language choices.
ket supports many languages with niche user bases. Second, Since little is quantified about the programming language
intrinsic features have only secondary importance in adop- adoption process, we focus on broad research questions:
tion. Open source libraries, existing code, and experience What statistical properties describe language popu-
strongly influence developers when selecting a language for larity? We begin (Section 3) with an empirical analysis
a project. Language features such as performance, reliability, of language use across many open source projects. Such
and simple semantics do not. Third, developers will steadily a macro-scale analysis reveals what trajectories languages
learn and forget languages. The overall number of languages tend to follow. Our analysis includes the overall distribu-
developers are familiar with is independent of age. Finally, tion of language use, and how it varies based on the kind
when considering intrinsic aspects of languages, develop- of project and developer experience.
ers prioritize expressivity over correctness. They perceive We found that popularity follows a power law, which
static types as primarily helping with the latter, hence partly means that most usage is concentrated in a small number
explaining the popularity of dynamic languages. of languages, but many unpopular languages will still find
Categories and Subject Descriptors D.3.0 [Programming a user base. The popular languages are used across a vari-
Languages]: general ety of application domains while less popular ones tend to
be used for niche domains. Even in niche domains, popular
General Terms Languages, Human Factors languages are still more typically used.
Keywords programming language adoption; survey re- Which factors most influence developer decision-
search making for language selection? Section 4 examines the
subjective motivations of developers when picking lan-
⇤ Research supported by Microsoft (Award #024263) and Intel (Award guages for specific projects. Knowing what matters to de-
#024894) funding and by matching funding by U.C. Discovery (Award velopers helps language designers and advocates address
#DIG07-10227). Additional support comes from Par Lab affiliates National
Instruments, Nokia, NVIDIA, Oracle, and Samsung.
their perceived needs.
Through multiple surveys, we saw that developers value
open source libraries as the dominant factor in choosing pro-
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
gramming languages. Social factors not tied to intrinsic lan-
for profit or commercial advantage and that copies bear this notice and the full citation guage features, such as existing personal or team experience,
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
also rate highly.
republish, to post on servers or to redistribute to lists, requires prior specific permission How do developers acquire languages? Knowledge
and/or a fee. Request permissions from permissions@acm.org.
OOPSLA ’13, October 29–31, 2013, Indianapolis, Indiana, USA. about the learning process is important because developers
Copyright is held by the owner/author(s). Publication rights licensed to ACM. are much more likely to use a language they already know.
ACM 978-1-4503-2374-1/13/10. . . $15.00.
http://dx.doi.org/10.1145/2509136.2509515

1
In Section 5, we examine how age and education shape lan- behavior is important in its own right, and as our results will
guage learning. show, commercial developers prioritize open source when
We found that developers rapidly and frequently learn making their own adoption decisions.
languages. Factors such as age play a smaller role than sug- 2. Online Poll: Hammer. “The Hammer Principle” is a
gested by media. In contrast, which languages developers website that invites readers to compare various items, such
learn is influenced by their education, and in particular, cur- as programming languages, based on a multidimensional se-
riculum design. ries of metrics [13]. Respondents pick a set of languages
What language features do developers value? Whereas that they are comfortable with out of a pool of 51 lan-
Section 4 looks at how developers pick languages for spe- guages. They picked 7 languages on average. Respondents
cific projects, Section 6 examines their feelings about in- are then shown a randomly sorted series of statements, such
trinsic features of languages, such as type systems. The re- as “When I write code in this language I can be very sure
sults can help designers craft better languages and advocate it is correct.” For each statement, the respondent orders the
them, and even influence curriculum design due by exposing languages that they selected based on how well they match
knowledge gaps. the statement. The survey includes 111 statements, and re-
We found that developers generally value expressiveness spondents sorted languages for an average of 10 statements
and speed of development over language-enforced correct- each before tiring. The survey period was 2010-2012.
ness. They see more value in unit tests than types, both for The raw and anonymized data was provided by the site’s
debugging and overall. How features are presented strongly maintainer, David MacIver. For each statement, we used a
influences developer feelings about them: developers rank variant of the Glicko-2 ranking algorithm [8] to convert the
class interfaces more highly than static types. sparse data of inconsistent pair-wise comparisons into a total
Before addressing the above research questions, we de- ranking of languages. A prior publication [14] describes this
scribe our data sets and methodology. We finish the paper by analysis in further detail.
discussing threats to the validity of our results (Section 7), The intuition is that we treat each statement as a tour-
related work (Section 8), and our conclusions (Section 9). nament between languages. Glicko converts the sparse pair-
wise comparison into a total order. The Glicko family of al-
2. Methodology and Data gorithms is used in chess tournaments and online game rank-
ings for producing a complete ranking of players without ev-
This paper is based on several different surveys and data ery pairwise comparison: beating a highly ranked language
sources: we used software repositories, surveys conducted contributes more than beating a low-ranked language. Each
by others, and a survey conducted by us. These were used in player (i.e., language) has an absolute rank and a statistical
a sequence; the results of one analysis informed the design of confidence for it.
the next. We began with pre-existing data from the Source- 3. Course Surveys: MOOC. We gained access to a
Forge repository and The Hammer Principle, a long-running survey of 1,185 students in a massive online open course
online survey about programming languages. These led us (MOOC) on software-as-a-service (SaaS). The survey was
to preliminary hypotheses, which we investigated with sur- administered at the beginning of the course, so student be-
veys. Finally, we cross-validated our results with the Ohloh liefs were not be altered by the instructors, though the re-
project database. sults do reflect sample bias towards programmers with an
interest in SaaS development. Most respondents were not
2.1 Data
traditional undergraduate students. Their median age was 30
We now describe our data sets in more detail. In chronolog- and a majority (62%) described themselves as professional
ical order, we used: programmers.
1. Open Source Repository Metadata: SourceForge. The survey was primarily conducted for pedagogical pur-
We wrote a crawler to download descriptions of 213,471 poses, not research. However, we advised the instructors on
projects from SourceForge [2], an online repository for question wording, and were given access to the raw col-
open source software. Of most relevance to our analysis, lected data. Respondents were asked if they consented to
the downloaded metadata for each projects documents the research use of their responses; 1,142 said yes (96.5% of
project’s languages used, primary project category (e.g., ac- all responses). We only analyze responses from adults who
counting), date of creation, and the project’s owners. The agreed to research use.
languages are drawn from a list of 100 curated by Source- Respondents were randomly divided into four subsam-
Forge; the categories, from a set of 223. ples. Some questions were asked to every respondent (MOOC
The years examined are 2000-2010. This data set is a rea- all) while others were only asked to one subsample. We di-
sonable proxy for open source behavior because, for most of vided the questions to avoid fatigue in respondents from
the analyzed period, SourceForge was the dominant software overly long surveys while still achieving enough responses
repository. For example, its modern competitor GitHub was for statistically significant analysis of many questions. Only
not even created until mid 2008. Open source community

2
 Name Responses Age Degree Pro. relations).1 The accessed SourceForge webpages and Ohloh
Quartiles API are publicly accessible at time of writing.
Not every data source is applicable to every question we
MOOC b 166 25 - 30 - 39 51% 60%
ask. Table 3 summarizes how the data was used in various
MOOC d 415 25 - 30 - 38 51% 58%
sections.
MOOC a–d 1,142 25 - 30 - 38 53% 62%
Slashdot 1,679 30 - 37 - 46 55% 92% 2.2 Respondent Demographics
SourceForce 266,452 people and 217,368 projects.
We tracked demographic information on the MOOC and
Ohloh 590,000 projects.
Slashdot surveys. For both surveys, the respondent popu-
Hammer 13,271
lations are primarily professional developers. A majority in
Table 1: Overview of data sets and populations. Degree = each have computer science degrees. The MOOC population
percentage with at least a bachelor’s in CS or related field. skews younger than Slashdot and towards fewer profession-
Pro = Professional programmers. als (Table 1), though the majority are still adult profession-
als with degrees. Comparing the results from the MOOC and
Slashdot surveys helps us tease out population-specific and
wording-specific effects.
Programmers are diverse. Professional developers are es-
Name Date Top 6 Languages
timated to be a minority of all individuals who write code
MOOC 2012 Java, SQL, C, C++, at work or as hobbyists [23]. Our research therefore does
JavaScript, PHP not exhaust the universe of programmers. However, study-
Slashdot 2012 C, Java, C++, Python, ing professional developers is of particular interest for un-
SQL, JavaScript derstanding adoption, both in terms of societal impact and
SourceForge 2000-2010 Java, C++, PHP, C, Python, understanding the relevance of best-effort practices such as
C# technical education. We therefore emphasize the results for
Ohloh 2000-2013 XML, HTML, CSS, professionals.
JavaScript, Java, Shell We qualitatively validate our sample against that of pre-
vious work. Table 2 compares the six most popular lan-
Hammer 2010-2012 Shell, C, Java, JavaScript,
guages of our surveys against the TIOBE index, which mea-
Python, Perl
sures the volume of web search results for programming lan-
TIOBE Index Feb. 2013 Java, C, Obj.-C, C++, C#, guages [3]. The tables suggest that our survey samples are
PHP broadly in alignment with one another and with the TIOBE
survey. Differences appear attributable to details in ques-
Table 2: Most popular languages in surveyed populations.
tion wording. For example, programmers might use CSS and
There is substantial, but not total, overlap.
XML regularly, but not think of it as a programming lan-
guage unless prompted to include it. They might use SQL,
but not consider themselves experts. Finally, they might use
a shell scripting language, but not consider it important. We
two of the subsamples (MOOC b and MOOC d) are relevant
performed similar grounding comparisons against other re-
to the questions addressed in this paper.
sults throughout our work.
4. Online Survey: Slashdot. We created interactive visu-
alizations of the Hammer data set and put them on a public 2.3 Methodology
website. These attracted a substantial amount of web traf-
To maximize the quality of our surveys, we applied standard
fic. Viewers were invited to answer a short survey. Most of
methodology for iterative gathering of large-scale and cross-
the readers arrived via links from popular websites such as
sectional data.
Slashdot and Wired; we refer to this as the “Slashdot” sur-
Throughout the survey design process, we drew upon ex-
vey. Over 97% of the responses were collected during a two-
ternal sources. At the start, we examined adoption studies in
week span in the summer of 2012.
various social sciences, such as the diffusion of innovation
5. Ohloh We cross-validated some results using Ohloh [1].
model by Rogers [22]. We also performed a literature sur-
Ohloh is a website, bought by SourceForge, that tracks
vey on beliefs of prominent language designers [15]. Finally,
over 590,000 open source projects hosted on SourceForge,
we engaged in a series of open-ended discussions with pro-
GitHub, and elsewhere. We used it for queries such as how
grammers and language designers. The interviewees were
many repositories contain a Java file.
1 Currently hosted at http://www.eecs.berkeley.edu/~lmeyerov/
Raw anonymized data from the MOOC and Slashdot sur- projects/socioplt/data/all.tar.gz and http://www.eecs.
veys are available online, as are the visualization and data berkeley.edu/~lmeyerov/projects/socioplt/viz/index.html,
exploration tools for the Hammer data (and underlying cor- respectively

3
 Section Question SourceForge Ohloh Hammer MOOC Slashdot
§3 What is the macro-behavior of lan- X + X+ §4.1
guages and communities?
§4 What factors influence language + §3.2 and 3.3 + §6.2 X
choice for projects
§5 How do programmers learn and lose X X
languages
§6 What do programmers believe is im- X X + §4
portant in a language?

Table 3: Summary of where each data source is used. X= data source used in that section, + indicates that the result is
cross-validated against a related result elsewhere in the paper.

primarily visitors to UC Berkeley, attendees at programming challenges of survey research on programmers appeared at
language and software engineering workshops and confer- PLATEAU 2012 [14].
ences, Bay Area startup employees, and were academic, in-
dustrial, and government programmers and language design- 3. Popularity and Niches
ers. These discussions covered several pre scripted questions We first examine the macro-level question of how adoption
and were otherwise driven by the participant. The interviews of popular languages differs from unpopular ones. We divide
helped frame hypotheses as well as helped inform us on how the analysis into three sub-questions: What is the overall
to phrase subsequent survey questions neutrally and broadly. distribution of popularity? What is the relationship between
Overall, we received advice and input from about 30 people. languages and application domains? How do developers
To improve the survey instruments themselves, we ap- move between languages?
plied several techniques.
3.1 Usage Falls Off Quickly, then Plateaus
• Piloting questions. In developing survey questions, we
The distribution of usage across different languages indi-
first prepared a preliminary questionnaire and tested it
cates the risk/reward trade-off for creating a language. If the
on several undergraduate and graduate students. We then
median language has significant usage, that makes creating
held a discussion with about 10 graduate students in
a language a more promising endeavor than if the median
programming languages about hypotheses. We then re-
language accounts for a negligible fraction of usage. (Even
peatedly revised the questions by asking undergraduates,
in the latter case, there may still be utility in building a lan-
graduate students, and visiting researchers and profes-
guage, but the creator has fewer grounds to expect usage.)
sionals about how they understood each question.
Figure 1a shows that a small number of languages ac-
• Free response. Each survey asked respondents if any count for most usage on SourceForge and Ohloh reposito-
questions were confusing and, after some individual ries and in the Slashdot survey results. On SourceForge, the
questions, whether they had more to add. We do not re- top six languages account for 75% of the projects; the top
port on questions that respondents flagged as confusing. 20 languages for 95%. General-purpose languages comprise
To preserve anonymity, we do not release the answers to most of the top 20 languages in all three data sets.
free response questions. Some domain-specific languages also rank highly. Con-
• Demographic questions. We applied two techniques to sidering only a project’s primary language, SQL is the only
detect and compensate for sample bias. First, we included domain-specific language in the top 20 languages. But if we
a variety of demographic questions, such as for age and include all languages for a project, SQL, HTML, and CSS
education. Second, we compare results from several dif- are in the top 20. We cross-validated with Ohloh’s analysis
ferent surveys and different populations. of overall language use in actual repositories. Ohloh reports
that XML, HTML, and CSS are the top three languages in
Our cross-sectional survey methods have limitations. For terms of any use. SQL and Make are also in the top 20. That
example, while we asked several questions about respon- a small set of general-purpose languages dominate language
dent’s early experiences with programming languages, a lon- use is not surprising. However, none of the language design-
gitudinal study might assist future work that explores spe- ers we interviewed suggested that a language such as CSS,
cific hypotheses. Likewise, we focus on correlations. To sup- a constraint-based language for web page layout, would be
port future examination of causation, we solicited informa- more popular than all of the general-purpose languages such
tion on potentially confounding factors such as developer de- as C and Java. The prominence of domain-specific languages
mographics. Further discussion of our methodology and the among other popular languages is surprising.

4
 Language Rank vs. % of Projects (Zipf 's Law)
100.0000%
y = 0.08e-0.09x
10.0000% R² = 0.98

1.0000% y = 0.52x-1.52
Proportion of R² = 0.95
Projects for Language 0.1000%
(Log-scale)
0.0100% y = 0.04e-0.09x
R² = 0.95
0.0010%

0.0001%
1 10 100
Language Rank by Decreasing Popularity (Log-scale)
Slashdot Sourceforge Ohloh
Power (Slashdot) Expon. (Sourceforge) Expon. (Ohloh)
(a) Probability Mass Function. Each point represents a language.
Language Popularity CDF (# Languages with >= # Projects)
100%
y = -0.09ln(x) + 1.18
R² = 0.95
y = -0.101ln(x) + 1.0392 y = 0.90x-0.63
10% R² = 0.97
CDF of Languages R² = 0.93
with >= # of Projects
(Log-scale)
1%

0%
1,000,000 100,000 10,000 1,000 100 10 1
# of Individual Projects (Log-scale Binning)
Slashdot Sourceforge Ohloh
Power (Slashdot) Log. (Sourceforge) Log. (Ohloh)
(b) Cumulative Distribution Function. Each point represents the set of languages with the number of projects listed on the x axis.

Figure 1: Language popularity. Slashdot survey data follows a heavy-tailed power law while curated SourceForge and Ohloh
data better follow an exponential curve.

We turn now from the most popular to the least-popular The different data sets do not include the same languages,
languages, the tail of the popularity distribution. Our ini- however: they diverge in tail languages. Over half of the
tial analysis of SourceForge shows an exponential decline languages reported in Slashdot are not tracked by Ohloh.
in language popularity, as does cross-validation with Ohloh For example, Slashdot respondents report using SAS. Al-
(Figure 1a). In contrast, popularity in the distribution’s tail though manual verification shows SAS being used in cov-
plateaus in the Slashdot survey of developers. The average ered projects, Ohloh does not count it. We conclude that
difference in rank for languages in common between the programming language popularity has a heavy tail but that
Slashdot and SourceForge is only 6.9, suggesting that the experimental artifacts can conceal this in many data sources.
two rankings are reporting on similar underlying usage. The The heavy tail covers many unpopular languages. For ex-
disparity in tail behavior stems from the fact that the Source- ample, the least popular languages (those with only a single
Forge and Ohloh language counts are based on a curated project) in Slashdot cover 6% of the projects (Figure 1b).
list of languages. In contrast, the Slashdot results include all Three such languages cover 0.2% of the projects. Lacking
responses and thus tail behavior is not filtered out.2 Even the heavy tail, SourceForge and Ohloh show an equivalent
though the Slashdot survey measures orders of magnitudes percentage of less than 0.002% for three of the least-popular
fewer projects, it yields a similar number of languages to languages. The unpopular languages can be quite domain-
SourceForge and Ohloh. specific in practice. For example, one respondent reported
developing in the Linden Scripting Language, used for the
2 We manually inspected the Slashdot results and merged synonyms.

5
 1" Java&
ActionScript
Tcl
Overall&Popularity&

ASP.NET
0.1" Objective C
PL/SQL
C#& Ruby
0.01" PL/SQL& Assembly& JSP
Assembly
Fortran& Visual Basic .NET
0.001" Prolog& Visual Basic
VBScript& y"="0.02x(2" Delphi/Kylix
R²"="0.63" Unix Shell
0.0001" Perl
JavaScript
0" 1" 2" 3" 4" 5" C#
Dispersion&of&popularity&across&project&categories&& Python
C
(coefficient&of&variaEon&=&σ&/&μ)&& PHP
C++
Java

Figure 2: Dispersion of language popularity across Project Categories (223)

project categories. The less popular the language, the

more variable its popularity is across application domains. 0 0.1 0.2 0.3 0.4 0.5 0.6
(SourceForge data set). Percent of projects in a category

Figure 3: Fraction of projects in each language for differ-

Second Life virtual world system. The heavy tail in our data ent project categories. Y axis is the top 20 (95%) languages
shows that the market for languages supports creating many and X axis is project categories with at least 100 projects
unpopular languages, not just extending the few relatively written in any language. Dark red cells indicate a high prob-
popular ones such as Java and Haskell. ability of using a particular language within a given project
category. (SourceForge).
3.2 Unpopular Languages are Niche Languages
Our next question is how the popularity of a language relates
Top 6 Languages
to its domain-specificity. To evaluate this, we use the project
Java C++ PHP C PYTHON C#
category labels provided by the SourceForge metadata. For
each language, we compared its overall popularity (as a frac- µ 0.205 0.167 0.119 0.140 0.063 0.062
0.094 0.101 0.117 0.100 0.028 0.029
tion of all projects) to its popularity in each category (e.g.,
x̄ 0.006 0.007 0.008 0.007 0.002 0.002
accounting). We formalize this comparison as the coefficient
of variation (standard deviation divided by mean). Figure 2 Top 25-30 Languages
plots the coefficient of variation of language usage against ASP BASIC Obj Pascal Matlab Fortran
the percentage of projects using them (i.e., popularity). Ta-
µ 0.003 0.003 0.002 0.003 0.002
ble 4 shows several languages in detail. 0.004 0.004 0.003 0.008 0.009
We find that popular languages receive broad-based sup- x̄ 0.001 0.001 0.000 0.000 0.000
port while unpopular languages tend to be used in a few par-
ticular domains. For example, Java is used by 20% of the Table 4: Mean, variance, and standard error of language
projects. For 70% of the project categories, Java is used by popularity in different project categories. Only project
10–30% of the projects within that category; for those cate- categories with at least 100 projects are considered. We show
gories, Java’s popularity is within 50% of its overall popular- the top six (75% total usage) popular languages and also
ity. In contrast, a relatively unpopular language like Prolog languages 25-30 (1.2% total usage). Order of categories is
is only used in a handful of categories. For 70% of cate- arbitrary. (SourceForge).
gories, Prolog’s popularity within a category will differ from
its overall popularity by +/- 800% rather than Java’s +/-50%.
A few outliers stand out in our analysis. Assembly (µ = VBScript is an outlier of the opposite sort: our model pre-
0.011, = 0.03) and Fortran (µ = 0.002, = 0.03) vary dicts more variability across categories than actually occurs.
in popularity across categories 3-6X more than our model We hypothesize that, since VBScript is a scripting language
predicts based on their overall popularity. In effect, these packaged with and made for Windows, it attracted a varied
are domain-specific languages for low-level or numeric pro- base of developers that were performing a wide range of
gramming, respectively. That makes them unusual cases of tasks. Despite occasional outliers such as Fortran and VB-
domain-specific languages that are popular overall. A pos- Script, Figure 2 shows a clear curve that relates overall pop-
sible explanation is that both used to be viewed as general- ularity to the variation in popularity across niches.
purpose languages; they have held onto niches, rather than Even though unpopular languages have their usage con-
colonized them for the first time. centrated in a few niches, they are rarely the most popu-

6
 !"#$%&'"($)#
!'*
only a small fraction of Java developers also use C/C++
!'*+,-.
!//01234 (7%), and C/C++ developers are five times as likely as Java

!"#$%"$&'()',-&01(%2',-(.&/+
5!'67
7 developers to use Perl [12]. We hypothesized that we would
78
799
:03);$<=43$>
see a similar clustering pattern in our data.
?%(#(@&
A(%%B4 We again used SourceForge data to answer this question.
C@B@
C@B@'"($)# For each developer that contributed to multiple projects, we
C'*
D$/)
DE@
examined how the choice of language for a project influ-
F!.D!5
G2H0"#I*@/"@3 enced the language choice for the project with the chrono-
G2H0"#$B0I7
*@/"@3 logically next creation date.
*0(3
*J* We present the results in Figure 4. Each row depicts the
*D<'KD
*(%3%L
*4#;%&
probability that a developer’s next SourceForge project will
ME24
'";010 use the language on the bottom, given that the developer
."3
N&$>I';033 was previously in a project that used the language labeled
O5'"($)#
O$/E@3I5@/$"
O$/E@3I5@/$"I+,-.
on the right. The bright diagonal line shows that developers
P'D often keep using the same languages. If we select a first
P'D
F!.D!5
!"#$%&'"($)#
!'*
!'*+,-.
!//01234
5!'67
7
78
799

G2H0"#I*@/"@3
G2H0"#$B0I7
*@/"@3
*0(3
*J*
*D<'KD
:03);$<=43$>
?%(#(@&
A(%%B4
C@B@
C@B@'"($)#
C'*
D$/)
DE@

*(%3%L
*4#;%&
ME24
'";010
."3
N&$>I';033
O5'"($)#
O$/E@3I5@/$"
O$/E@3I5@/$"I+,-.

language uniformly at random, developers will keep to that

language 18% of the time. More often – 52% of the time
– they will switch to one of the top six languages overall.
!"#$%"$&'()'#&*+',-(.&/+
These overall-popular languages correspond to the vertical
bands in Figure 4.
Q Q+R Q+S Q+T Q+U Q+V Q+W A given prior language only occasionally correlated with
*(%2@2$3$#4I%XI)$"Y$&LI@I3@&LE@L0IL$B0&I#;0I)(0B$%E/I%&0 the choice of a specific different language for the next
project. Most notably, developers have high propensities
Figure 4: Probability of picking a language given the lan- to switch between Windows scripting and application lan-
guage of the previous project. Y axis shows the language guages, such as VBScript and C#. These languages also
of the previous project and X axis shows the language of the correlate with Microsoft web-development languages such
next: each point is the probability p(L0 = x | L = y). Lan- as ASP. Such correlations are also visible in the results of
guages with less than 100 projects are elided. (SourceForge) Karus and Gall [12], who found groupings such as WSDL
and XML being used in conjunction with Java.
Notably, we do not see significant exploration within
lar languages in those niches: the consistently popular lan- linguistic families. There is a relatively low probability of
guages tend to win out. Figure 3 illustrates this with a heat- switching between Scheme and LISP, or between Ruby,
map showing the relative popularity of languages across Python, and Perl. We conclude that developer movement
niches. The languages are sorted by popularity, with the bot- between languages is driven more by external factors such
tom rows containing the most popular languages. as the developer’s background or technical ecosystem than
There are cases where a language that is less popular by similarity of the underlying languages. This implies that
overall will beat out more-popular languages in specific language advocates should focus on a domain and try to
niches. For example, C++ is more popular than C in general convince programmers in that domain, instead of trying to
but not for compilers in particular. Figure 3 shows that PHP, convince programmers who use languages with semantic
C, and C++ (the top three languages) vary in relative popu- similarities to the new language.
larity across domains. This effect is largely limited to the top One limitation of our result is that project participation
few languages, however. Beyond that, a niche-specific lan- is an imperfect proxy for language use. Some cases of lan-
guage’s comparative advantage is dominated by the overall guage reuse may be due to developers that only saw a lan-
popularity of the top overall languages. guage being used on one project and then used it themselves
on the next. Likewise, we only sample SourceForge projects:
3.3 Developer Migration
the effects we report may be stronger in general because
Developers work on many projects over the course of their they may carry through intermediate projects not reported
careers. We do not expect one language selection decision on SourceForge.
to be independent of the next, so we examine how devel- Overall, we found a simple and representative model de-
opers move from language to language. More precisely, we scribes language selection: language popularity and prior
ask how using a language for one project influences a devel- use predicts over 75% of the language selection decisions
oper’s selection for the next. in SourceForge. Despite our model’s simplicity, it is effec-
Karus and Gall have analyzed the commit logs from 22 tive. More complicated refinements to ours would only need
open source projects, and report that developers tend to stick to address 25% of the unexplained projects.
to “clusters” of languages — for example, they report that

7
 Open source libs. (M)
Extending existing code (E)
Already used in group (E)
Personal familiarity (E)
Team familiarity (E)
Performance (M)
Portability/platform (E)
Development speed (M)
Tools (M)
Safety/correctness (I)
Potential team famil. (E)
Particular language feature (I) Overall
Commercial libs. (M) 1-100 employees
Simplicity (I) 101+ employees

0 10 20 30 40 50 60 70 80
Percent of respondents describing aspect as
medium or strong importance

Figure 5: Importance of different factors when picking a language. Self-reported for every respondent’s last project. Bars
show standard error. E = Extrinsic factor, I = Intrinsic, M = Mixed. Shows results broken down by company size for respondents
describing a work project and who indicated company size. (Slashdot, n = 1679)

4. Decision Making resented. Pretesting helped form the list, and free response
Above, we analyzed the overall “macro”-level process of comments from final respondents suggest that no significant
language adoption. One of our observations was that unpop- categories are missing.
ular languages usually have their popularity concentrated in A wide gap separates the most and least influential fac-
a few niches. We now offer a “micro” view: we focus on tors. The most influential factor, the availability of open
how individual developers make decisions. The central re- source libraries, was “strong” or “medium” for over 60% of
search question being investigated is which factors influence respondents. For the least influential factor, simplicity, only
developer selection of languages? We break this into two 25% said the same.
parts: how do developers weigh features of languages when We wanted to see how these results depend on a devel-
they are making choices, and how do demographics affect oper’s work environment. To do this, we broke out those re-
the languages that developers pick? This will help explain spondents who picked a work project and who indicated an
the observations in the previous section. organization size. We divided respondents into those work-
ing at companies with fewer than 100 employees and those
4.1 Weighing Language Features with more than 100. (This threshold is closest to splitting
the data set evenly, facilitating comparison.) The results for
In the Slashdot survey, we asked respondents to rate the these subpopulations are also shown in Figure 5.
influence of particular factors in picking the language for Some of the factors, such as the language’s features or
their most recent project. Ratings used a four-point scale simplicity, depend on the language design, not on its user
from “none” to “strong”. By asking about their most recent base. Others, such as whether a developer already knows the
project, we encouraged reflection on a historical event and language or the ease of hiring developers who know it, are
deemphasized ideal preferences that might not be acted upon extrinsic and depend on the social context of the language.
in practice. Respondents assigned individual priorities to 14 Some factors include a mix of extrinsic and intrinsic aspects.
factors (Figure 5). We selected the 14 categories such that For example, the presence of libraries or good development
the dominant choice for picking a language would be rep-

8
tools is a combination of social and technical factors. Figure 24%

Probability of use on last project

5 is labeled with our judgement about whether a factor was 20%
intrinsic, extrinsic, or mixed.
16%
We emphasize four results from the data:
12%
1. Open source libraries. Open source libraries are the
8%
most influential factor for language choice overall and the
4%
most influential factor for commercial projects at small
companies. They are an important factor, but not the most 0%
objectivec perl python
important factor, at large companies.
any 1-19 employees 20+ employees age <= 25 age > 25
2. Social factors outweigh intrinsics. Existing code or ex-
pertise with the language are four of the top five factors
Figure 6: Demographic influences on selecting partic-
for adoption. In contrast, intrinsic factors, such as a lan-
ular languages. Self-reported for every respondent’s last
guage’s simplicity or safety, rank low. Implementation at-
project. Bars show standard error. (Slashdot, nany = 1679,
tributes, like performance and tool quality, have both in-
n1-19 employees = 290, n20+ employees = 790, nage25 = 154,
trinsic and extrinsic components. (Some languages lend
nage>25 = 1382).
themselves more easily than others to a high-performance
implementation.) These mixed attributes vary in impor-
tance. whether a particular language was selected. For example,
employees at small companies are 1.4-3.0 times more likely
3. Domain specialization. Libraries, developer experience,
to use Objective-C than the typical respondent. Likewise,
and legacy code are all important in language selection.
when looking at age, developers under 25 rarely use Perl but
These factors are often associated with particular appli-
disproportionately select Python.
cation domains. Thus, the developer emphasis on these
Compared to the breakdowns of Figure 5, we see that
attributes helps explain the result in Section 3.2 that less-
language selection is especially sensitive to demographic ef-
popular languages are more niche-specific.
fects. Notice also that different languages are sensitive to dif-
4. Company size matters. Employees at larger compa- ferent demographic variables: Perl and Python are age sen-
nies place significantly more value on legacy code and sitive, but appear insensitive to company size; Objective-C
knowledge than do employees at small companies. Cor- is sensitive to company size, not to age. Our observations
rectness becomes more influential for large companies, suggest that it is unsafe to generalize about how any particu-
while simplicity, platform constraints, and development lar demographic variable will correlate with language usage.
speed matter less. Compared to developers overall, those Different languages will have different social dynamics.
at larger organizations weigh commercial libraries more
and open source libraries less (although open source is 5. Language Acquisition
still weighted more highly).
Here, we switch focus from the decision to use a language at
These results help inform language designers seeking a particular instant to the process of learning languages. We
adoption where to focus their efforts. Developing high- examine three related questions: How long does it take de-
value open source libraries is likely to have a large influ- velopers to learn languages? When in their careers do they
ence to language adoption, particularly for individuals and learn? How does education affect learning? The previous
small companies. Simplicity will not attract many program- section demonstrated that developers prefer to use languages
mers. Smaller companies are less constrained by legacy code they already know. Understanding how quickly developers
and experience. We suspect that they are therefore likely learn and what induces them to learn helps explain this adop-
more willing to adopt new languages that are not backward- tion factor.
compatible. In contrast, a backward-compatible change to a
5.1 Learning Speed
language might be more valuable to a large company.
For the language used on their most recent project, the Slash-
4.2 Demographic Influences on Language Selection dot survey asked respondents to estimate how long it took to
We now look at how developer demographics affect the lan- learn to use the language well. To “know a language well”
guages that developers select. Understanding this extrinsic is an intentionally imprecise and subjective standard. We
factor clarifies the generality of our results and its role for showed above that when developers pick languages for a
future empirical analysis or targeting of developers. project, they are heavily influenced by the languages they
We observed significant correlations when distinguish- believe they know: developers will be using their own sub-
ing the different demographics that selected a particular lan- jective standard.
guage. Figure 6 shows that, for languages selected by Slash- Figure 7 shows the results for all languages for which
dot respondents, age and company size strongly influence we had at least 50 responses. The question was framed

9
 Probability*of*learning*the*primary*language*
during*a*project*(by*demographic)*

Probability*of*learning*during*a*project*
0%# 5%# 10%# 15%# 20%# 25%# 30%# 35%#
overall#
<#20#employees#
20+#employees#
age:#21622#
age:#23624#
age:#25630#
age:#31640#
age:#41+#

Figure 8: Probability of learning the primary language

Figure 7: Median reported speed of language acquisition. during a project. Shading denotes demographics and bars
Bars are standard error. (Slashdot, n = 1679) are standard error. (Slashdot, n = 1536)

as multiple choice; the y-axis labels of Figure 7 were the

available options. higher increase in learning rate. The difference quickly ta-
The median learning times for the most challenging lan- pers off, with developers aged 25-30 behaving similarly to
guage and the most approachable differ by a factor of ten. those aged 31-40 olds (20% vs. 19%). Relative to age, orga-
Programmers report C++ as the slowest to learn while the nization size has minimal influence.
fastest are PHP, Python, and Ruby. Java and C#, which are Our analysis shows that developers in our sample steadily
semantically similar, are between these extremes and have learn languages throughout their career. As a result, they are
similar learning times. not limited by the languages that were popular when they
The relative time to learn PHP is noteworthy. PHP is noto- were young or in school. This means that the time scale of
rious for its ad-hoc design while Python is well-regarded for language adoption is not driven by the career timelines of
its simplicity. Despite their differences, both languages have developers.
comparable reported learning times of just a few months.
We infer that developers determine that they can “use the 5.3 Languages Over Time
language well” even if they have not mastered every nuance. The next sub-question we examine is how a developer’s
Developers may only need to learn a subset sufficient for age influences the particular languages that the developer
completing routine work. knows. Some professional recruiters claim that there are
More fundamentally, the relative ease of learning PHP large and significant differences in the languages that older
suggests that, while complexity is a barrier to adoption [22], and younger developers use [11]. Our results refute this.
what makes a language complicated for developers need not Figure 10 shows the median age for programmers who
be what makes it complicated for a designer or researcher claim to know each of the indicated languages, along with
(e.g., convoluted formal semantics). For the same reason, the 25th and 75th percentiles of age. The distributions are all
what makes a language simple in a formal sense need not very close. The highest median age we observed was 38.9
make it simple for developers and otherwise adoptable. (BASIC and Perl); the lowest was 37.3 (Ruby). This is sur-
prising: we would have expected changes in education over
5.2 On-site Language Learning time to result in substantial variation in median programmer
We now examine the relationship between developer demo- age. Instead, the 95% confidence interval for every language
graphics and the languages that they learn. Knowing which includes the overall response mean age (38). The deviations
demographics are likely to learn helps language proponents are not statistically significant. We observed similar age in-
target their outreach at those potential early adopters. variance patterns in the results of the MOOC survey. There
The Slashdot survey asked respondents whether, during as well, developers of different ages know a similar number
their last project, they learned the primary language for it. of languages and a similar mix of languages.
Figure 8 shows the results broken down by age and orga- This shows that differences in learning by age, described
nization size. We found that younger developers are more above, get washed out over the course of a career. In our sur-
prone to learning new languages. Overall, 21% of respon- veys, programmers of any age are equally likely to remem-
dents learned a language for their most recent project. That ber or newly learn older languages like Pascal. Young pro-
rate increases to 29% for 21-to-22-year olds. While a jump grammers catch up to old ones, and older ones keep learning.
for younger programmers is not surprising due to inexperi- Popular languages do not thrive or wither based on the age
ence or perhaps eagerness, we had expected to see a much breakdown of their adherents. Programmers keep learning,

10
 Mean # Langs. known
8

2
know slightly
know well
0
20 30 40 50 60
Age

Figure 9: Number of languages by age. Developers of Figure 11: Number of languages by age. As with Figure 9;
different ages seem to know a similar number of languages. there is no clear trend with age. (MOOC, n = 1142)
Lightly shaded rectangles show 25th and 75th percentiles,
error bars show standard error of mean. (Slashdot, n = 1679)
The MOOC survey asked developers to list the languages
they know well and the number they know slightly. The re-
sults are shown in Figure 11. Compared to the Slashdot sur-
50 vey, the average developer in the MOOC survey knows fewer
languages. However, the results are qualitatively similar in
Age of users

45
that there is no age trend. This shows that our result is robust
40 to both varying the detailed wording of the question and to a
35
change in the underlying population.
There is a tension between the flat lines on Figures 9
30
and 11 and the fact that developers are steadily learning.
25 Since developers of different ages are similarly likely to have
va

pt
l

rl

by
sic

learned a language for their last project, we might expect the

sca

Pe
C+

PH
ri
Ja

Ru
Ba

Sc
Pa

number of languages they know to rise over time. Instead,

va
Ja

older and younger developers report a similar degree of mul-

tilingualism. It follows that developers are losing languages
Figure 10: Mean, standard error, and 25th and 75th per- as well as gaining them. They are forgetting – or at least,
centiles for ages of programmers who claim to know each forgetting to mention – some languages.
language. Languages are sorted by creation date. Distribu- This result shows that adult developers effectively have a
tions are nearly the same for every language. (Slashdot, n = limited capacity for languages. They typically maintain skill
1679) in a limited number of languages, and will forget as many
languages as they learn. This implies that immediate devel-
oper familiarity is a limited resource for which languages
and learn often enough and quickly enough that their age must compete. Such competition is the basis for ecological
does not predict which languages they know. theories of adoption [15].
We now look at the overall number of languages a devel-
oper knows. The Slashdot survey asked developers to esti- 5.4 Effects of Education
mate the number of languages they have learned, and also to We also looked to see how a respondent’s computer science
list the languages that they know well. These different ques- education affected their subsequent programming language
tions capture different levels of knowledge and familiarity knowledge. The Slashdot survey asked respondents to mark
and we include both in our results. Figure 9 plots the mean which families of languages they learned while in school
number of languages known by developers against their age, (e.g., assembly, functional languages, dynamic languages).
along with the inter-quartile range. Both lines are remark- Table 5 shows our results.
ably flat. The mean respondent to our survey claims to have The vast majority of respondents know a compiled non-
learned ten languages, and lists six that they “know well.” functional language, such as C or Java, regardless of major or
Likewise, the upper and lower quartile range does not show curriculum. Likewise, dynamic languages are widely known.
any clear trend over time; the gap between the 25th and 75th Less-popular language families (assembly, functional, and
percentile of number of languages does not change over time mathematical languages) are more sensitive to prior educa-
in a systematic way. tion. Promisingly, developers who learned a functional or

11
 Language Examples Overall For CS Non- Correlation of If taught If not Correlation of
Family majors majors CS major vs. in school taught learned in
knowing school vs
knowing
Functional Lisp, Scheme, 22% 24% 19% 0.053 40% 15% 0.262
Haskell, ML (0.004 - 0.101) (0.217 - 0.307)
Dynamic Perl, Python, 79% 78% 79% -0.008 84% 77% 0.069
Ruby (-0.056 - 0.040) (0.021 - 0.117)
Assembly 14% 14% 14% 0.004 20% 10% 0.138
(-0.044 - 0.052) (0.091 - 0.185)
Imperative/OO C/C++,Java/C# 94% 97% 90% 0.134 95% 87% 0.133
(0.087 - 0.181) (0.085 - 0.180)
Math R, Matlab, Math- 11% 10% 11% -0.020 31% 7% 0.268
ematica, SAS (-0.068 - 0.028) (0.223 - 0.312)

Table 5: Probability of knowing at least one language in the indicated family, overall and grouped by major and specific
educational experience. Also shows correlation coefficients between knowing a language in that family and (a) having a
CS degree, and (b) having learned a language in that family in school. Whether developers learn a language in that family
in school has much more influence than being a CS major. Shown below each correlation is the 95th percentile confidence
interval. (Slashdot, n = 1679)

math-oriented language in school are more than twice as Beliefs help shape developer and manager decisions to
likely to know one later than those who did not. learn and advocate languages, and thereby affect the so-
Educational intervention has limits, however. For math- cial dynamics of adoption. Understanding what developers
ematical, functional and assembly languages, the large ma- value, and how developers perceive languages, can help de-
jority of developers that learned a language in that family no signers both to build better-liked languages and also to ad-
longer know any similar language. Consequently, the corre- vocate more effectively on behalf of their languages.
lation between education and later knowledge is relatively
modest. 6.1 Perceived Value of Features
Notably, having been a computer science major does not
The MOOC-d survey asked developers to rate the impor-
lead to the same linguistic versatility of students who learned
tance of different types of language features on a scale from
different language families as part of the course curriculum:
unimportant to crucial. The results are shown in Figure 12.
we saw no measurable correlation between being a CS ma-
As can be seen, libraries are the top-rated feature. Such per-
jor and knowing particular programming paradigms. Our re-
ceived importance of libraries cross-validates our finding
sults suggest that an undergraduate curriculum that does not
that libraries matter in practice (Section 4).
introduce students to a variety of languages is unlikely to re-
We included several options on the MOOC-d survey
sult in more versatile programmers later in their careers. This
that represent related concepts. For example, higher-order
demonstrates a weakness of curriculums that only focus on
functions are a strictly more powerful language primitive
languages such as Java and Python.
than inheritance. Interfaces are often used for type systems.
One limitation of this finding is that it measures only
Threads can be used to implement task parallelism.
correlation, and there could be causation in both directions.
While these features are similar in many ways from a se-
Developers who expect to use distinctive families such as
mantics point of view, they had sharply different priorities
assembly languages might choose to study them in school.
with developers. For example, 72% of developers consid-
Demonstrating how much causation flows in each direction
ered inheritance important or crucial; only 45% felt that way
is beyond the scope of this paper.
about higher-order functions. Interfaces likewise were con-
sidered far more important than static types. This is surpris-
ing, given that interfaces are little-used in languages without
6. Beliefs about Languages static types.
This section turns from programmer actions to programmer Performance was ranked the second most important fac-
beliefs. This section looks at what attributes of languages tor. This is significant in two ways. First, particular features
do developers like or dislike? The question is in contrast used for low-level programming, such as threads, macros,
to that of Section 4, which examines how developers pick static types, and templates all rate much lower. There is a gap
languages within the context of a specific project. between the importance of performance and the language

12
 0%# 20%# 40%# 60%# 80%# 100%#
elsewhere; confusion in this case was expected as it is in-
tended to reflect not understanding the underlying language
libraries# semantics.)
performance# We conclude that a large fraction of developers are disin-
object#inheritance# clined to reason about the semantic power of language fea-
classes#w.#interfaces# tures. Features that appear similar to language designers do
excep<ons# not appear so to users. For example, implementation inheri-
documenta<on#
tance can be implemented atop higher order functions: meth-
ods can be replaced by open functions and inheritance can be
error#messages#
modeled with constructor delegation.
threads#
The optional free text responses were interesting. They
garbage#collec<on# included “didn’t know this was possible. Sounds like it could
simplicity# be very useful” and “Objects require imperative program-
HoFs# ming style while functional programming is a lot less ver-
func<onal#purity# bose and allows for recursive definition.” These responses
task#parallelism# reaffirm our observation that many developers do not per-
determinism# ceive languages in the same terms as language designers.
Taken together, we see that developers’ impression of fea-
sta<c#types#
ture importance is not closely tied to the underlying seman-
generics#
tic power of the language construct in question. We infer that
eval#
developer preferences here are shaped by factors extrinsic to
generators# the language, such as experience and (mis-)education.
templates#
6.2 What Programmers Enjoy
annota<ons#
macros# We now turn to the closely related question of what pro-
grammers enjoy in a language. This is an open-ended ques-
con<nua<ons#
tion, and our data is preliminary. The Hammer data set, with
Most#Important# Medium#Importance#
its large pool of languages and statements, is the data set
available to us that is best suited to the inquiry. The features
of a language that developers enjoy are conceptually related
Figure 12: Feature preferences (MOOC-d data set, n = to the features developers value, and therefore this question
415). serves as cross-validation for the results of Figure 12.
The Hammer survey asked developers to rank 51 lan-
guages against 111 different statements, including “I enjoy
using this language.” As described in Section 2, we use the
0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Glicko-2 algorithm [8] to totally order all the languages for
I#don't#understand# Totally#unrelated# A#li=le# each such statement [14]. From these rankings, we corre-
Somewhat# Very#similar# MathemaDcally#equivalent#
lated different statements. Enjoyment is the closest proxy in
the data set for what we mean by “liking a language”, so we
Figure 13: Higher-order functions and objects. Most de-
looked for other attributes of languages that correlate with
velopers do not see a close connection between the two.
enjoyment.
(MOOC-d data set, n = 415).
The statement with the highest correlation with enjoy-
ing a language is “This language is expressive” (correla-
features used to achieve it today. It is unclear if the gap is tion 0.76). Other highly correlated statements include “I
inherent or if language designers should look for ways to fill find code written in this language very elegant” (correla-
it. Second, given the mid-tier importance of performance in tion 0.73), and “I rarely have difficulty abstracting patterns
picking a language for an actual project (Figure 5), we also I find in my code” (correlation 0.66). While open source
see a gap between perception and practice. libraries significantly influence developer actions, “Third-
The survey asked programmers “How similar are higher- party libraries are readily available, well-documented, and of
order functions to objects?” to gauge their beliefs about ex- high quality” only weakly correlates (correlation 0.10) with
pressive or practical differences. Responses are shown in enjoyment.
Figure 13. A third of respondents found the question con- Attributes other than “enjoyment” are also relevant to our
fusing. Another 20% consider them either totally or largely inquiry about developer preferences. We examined correla-
unrelated. Only 20% reported “very similar” or “mathemat- tions with the statement “This language has unusual features
ically equivalent.” (We do not report on confusing questions that I often miss when using other languages.” The results

13
 Statement Corr. Question Agreement
This language is expressive 0.87 Unit testing will reveal bugs that static 31% (+/- 9)
This language excels at symbolic manipulation 0.77 types miss
This language encourages writing reusable code. 0.62 Static types will reveal bugs that unit 7% (+/- 5)
The semantics of this language are much different 0.56 testing misses
than other languages I know. Unit testing will reveal more bugs that I 19% (+/- 8)
... care about than static types
This language has a strong static type system 0.29 Static types will reveal many bugs that 6% (+/- 5)
... I simultaneously care about and are
Libraries in this language tend to be well docu- 0.00 missed by unit testing
mented. I see the value of static types 36% (+/- 10)
The resources for learning this language are of -0.02 I see the value of unit testing 62% (+/- 10)
high quality I enjoy using static types 18% (+/- 8)
This language is large -0.03 I enjoy using unit testing 33% (+/- 9)
I find it easy to write efficient code in this lan- -0.06 Most of the value of unit testing is in 33% (+/- 9)
guage finding bugs
... Most of the value of static types is in 8% (+/- 6)
There are many good tools for this language -0.14 finding bugs
... I have used statically typed languages 39% (+/- 10)
This lang. has a niche outside of which I would -0.42 for large or many projects
not use it I have used unit testing for large or 34% (+/- 10)
This is a low level language -0.53 many projects
Using types helps improve readability 45% (+/- 10)
Table 6: Feature desires. Correlations with the statement Using types helps improve safety 44% (+/- 10)
“This language has unusual features that I often miss when Using types helps improve program 19% (+/- 8)
using other languages.” (Hammer) modularity
Using types is generally important, de- 19% (+/- 8)
spite the costs
are shown in Table 6. Perceived expressivity strongly corre-
Using types is rarely important 8% (+/- 6)
lates with this statement (correlation 0.87); developers value
features that ease development. In contrast, there is no sig- Table 7: Beliefs about types and testing. Shows fraction of
nificant correlation between having unusual-but-desired fea- responses agreeing with each statement, and 95th percentile
tures and the ease of writing efficient code. Expressivity and confidence bounds. Results from self-identified professional
performance are perceived to be unrelated across actual lan- developers in MOOC-b sample. (n = 96)
guages.

6.3 Types vs. Testing value” of static types. In contrast, 62% – nearly twice as
We close by examining the perceived tradeoffs around static many – see the value of unit testing. Developers are nearly
types, particularly as contrasted with unit testing. Static twice as likely to “enjoy using” unit tests (33%) as compared
types are a controversial choice in language design, and we with static types (18%).
hope our results help show designers how developers react. Contrary to our initial suspicions, developers describe
The MOOB-b survey included a set of statements about neither types nor testing to be primarily about finding bugs.
types and testing; respondents were asked to mark if they Only 8% of professional programmers think finding bugs
agreed or disagreed with each. The questions were binary; is the chief benefit of static types, while 33% say the same
yes and no were the only options. We present only the re- about unit tests. Respondents instead find static types to be
sults from the self-identified professional developers in the important in two areas: readability (45% agreement) and
sample. This filters out responses from students, whose ex- safety (44% agreement). Bug finding ranked third and mod-
perience may have been derived from small classroom as- ularity a distant fourth.
signments, rather than realistic industrial development. This We suspect the survey population was biased in favor of
leaves us with 96 responses — small compared to the other dynamic languages because the course associated with it
populations in this paper, but large enough for meaningful focused on software-as-a-service and therefore many web
statistical analysis. technologies written in dynamic languages. Even so, this
Table 7 displays the results. Even despite the large mar- result shows that there is a population of developers who are
gins of error, the results are striking. Only 36% “see the broadly skeptical of the benefits of static types.

14
 We cross-validated this result using the Hammer data. We we examine correlations. Empirical analysis of causality is
looked for statements in the Hammer dataset that correlated an important growing area [9] and would help elucidate the
with “This language has a strong static type system.” Static adoption process.
types correlate strongly with statements about correctness
such as “If my code in this language successfully compiles 7.2 Reliability
there is a good chance my code is correct.” (correlation
0.85). However, languages with static types are much less Section 3 uses data from the SourceForge repository, which
closely correlated with languages developers claim to enjoy hosts open-source software. It is possible that proprietary
(correlation 0.38) and with expressivity (correlation 0.31). code bases are statistically different: for instance, obscure
Irrespective of the objective value of static typing, we see languages might hang on longer inside corporate IT depart-
evidence that many developers do not value it highly. ments. However, open source development is a major part of
The MOOC population is by no means representative of all development activities and its effects will constrain cor-
all programmers. The underlying course was taught in Ruby porate development. As we showed, open source libraries
and targeting web application development. However, we have a major influence on language selection even within
believe the results are valid for dynamic language program- corporate development.
mers. Even on this restricted population, there are useful Our survey samples, while large, are self-selected. In par-
take-aways from these results. Our data suggests that devel- ticular, our Slashdot survey will be biased towards highly
opers value the readability provided by types more than they engaged programmers who read technology blogs and are
do many other benefits. The data also suggests that develop- interested in programming languages. The Hammer Princi-
ers do not find that types improve modularity — it contra- ple results are likely from a similar population. Our MOOC
dicts claims often made on behalf of types. These observa- data will be biased towards programmers who wish to learn
tions suggest points where future language design research more and improve their skills. Because the MOOC course
could better meet perceived needs. was focused on web-based applications and was taught in
Ruby, we expect that the population will be biased towards
dynamic-language users. This is a particular concern for the
7. Threats to Validity Hammer data, where we lack any sort of demographic infor-
This section discusses the limitations of our work. We begin mation about respondents.
with threats to validity (whether our results are accurate All three surveys are biased towards Americans and
on the populations we sample) and then consider reliability English-speakers. While these are important constituencies,
(whether our results would apply to other populations.) they are not the full universe of programmers. More work is
needed to check whether these results hold in other popula-
7.1 Validity tions. We highlighted points of agreement between our data
Respondents to the Slashdot survey had the opportunity to sources. The overlap in results across our surveys suggest
explore the Hammer data visualizations before answering our work does generalize to some extent.
the survey. This may have biased them. Because the survey While many of our results are cross-validated with sev-
questions are not closely related to the visualized data, we eral data sets, some are not: different surveys asked different
expect that the priming effect and consequent bias will not questions and different data sources include different meta-
be large. data. The SourceForge data is our only source in Sections
Not everybody who starts a survey will complete it. We 3.2 and 3.3. The observations in Section 4 about the impor-
lack information about respondents who declined to submit. tance of legacy knowledge and code are based solely on the
They may therefore differ in demographics, background, or Slashdot data set, as are the results about language learning
motivation from those whose responses we have. time in Section 5.1. In Section 6, the MOOC-d survey is our
The methodology used to extract data from the Hammer only source for the observation that developers have diver-
Project is novel and we do not have a rigorous analysis of the gent opinions about semantically-similar features.
statistical margin of error. The underlying Glicko algorithm Going forwards, we believe it would be valuable to study
is well-documented and widely used, however. professionals for whom programming is a significant but
Likewise, extracting data from SourgeForge required not primary job responsibility. For example, engineers and
making assumptions and judgement calls. Projects can scientists often do not come from computing fields but are
change categories, languages, and authors over time. still important classes of programmers.
Our work is largely cross-sectional, looking at one mo-
ment in time. We do answer some questions by examining
longitudinal data from the 10 years of SourceForge data and 8. Related Work
careful phrasing of some of our survey questions. Program- Relatively few studies empirically analyze language adop-
ming languages are reaching the point where we can and tion. Fewer focus on developer decisions, explore cross-
should examine questions that span decades. Furthermore, language phenomena, or use large data sets.

15
 Most similar to our work is that of Chen et al. [5]. They versity found no significant bias in the languages learned
gathered or estimated data about 17 different languages in by undergraduates there, however [19]. Not presented, our
1993, 1998, and 2003 and then performed regression. In con- data shows that language selection is gender-neutral on a
trast, we examine developer actions and decision making. broader sample than the above work. However, we reported
We increase the scale and fidelity and change the intent to even stronger biases relating to age and organization size.
identifying and quantifying influential factors. The large scale of our survey has enabled regressing along
Our analysis of SourceForge is a variant of software en- many dimensions such as these.
gineering literature in mining software repositories. For ex- Adoption decisions for domains beyond programming
ample, Parnin et al. [18] found that only 14% of develop- languages is studied by social psychologists, management
ers are responsible for incorporating generic classes into ex- science researchers, and other social scientists. Several
isting Java programs; most developers did not adopt this leading models of adoption arose over the years, such
new language feature but a few became enthusiastic ad- as the Technology Acceptance Model (TAM) [7] and the
vocates. Surveying programmers about Java (generics) and Unified Theory of Acceptance and Use of Technology
C++ (templates), we found differences in developer percep- (UTAUT) [25]. These causal, quantitative models relate fac-
tion of the same phenomena. This may entail that adoption tors such as perceived ease of use to ultimate adoption deci-
should be studied across languages and, for individual de- sions. Within a particular population, they predict much of
velopers, across projects. the variance in an individual’s desire to adopt a new technol-
Others also mine repositories to understand feature and ogy [24]. Models that have been tuned for software develop-
the API adoption within an individual language or project. ment have been able to explain 63% of the variance in devel-
Okur and Dig [17] show that, given a large library of paral- oper intention to use object-oriented design techniques [10].
lel constructs, 90% of usage is accounted for by 10% of API Whereas that work aims to understand the general factors
methods. Only a limited portion of functionality has been behind technology adoption, we seek those that are specific
adopted. Likewise, Vitek et al. characterize the use of lazi- to programming languages.
ness in the R language [16] and dynamic code evaluation in Generalizing the notion of adoption even further, Rogers’
JavaScript [20]. We examine different questions. seminal Diffusion of Innovation process is perhaps the most
Perhaps the most relevant repository mining research is extensively studied model of adoption [22]. A 2000 study
that of Karus and Gall who investigate the propensity of shows that this model accurately described the process by
open source developers to use multiple languages [12]. They which COBOL programmers at a large financial-services
found significant overlap, particularly between closely re- company learned the C language [4]. A subsequent single-
lated languages such as XML Schema and XSL. Their find- organization study looks at the decision of whether to adopt
ing does not conflict with our result that transitions between a formal development methodology [21]. In both cases, the
languages are mostly related to popular and past experience researchers ignored the intrinsic technical attributes of the
(Figure 4). Consider the likely case that the programmer will language or methodology in question, and considered social
switch from editing a WSDL file to editing a Java file: these factors exclusively: we examined both. Furthermore, both
two languages are often part of the same ecosystem. The cases use sample sizes much smaller than ours: 71 in the
probability of then writing PHP is closer to the overall pop- first case, and 128 in the latter. We provide a broader view
ularity of PHP; PHP is outside of the ecosystem. by examining more factors and over wider scenarios.
Many of the questions we answer fundamentally differ Finally, various histories of programming languages pro-
from those in the above repository mining studies [12, 16– vide insight into the adoption of specific languages. For ex-
18, 20]. Mining exposes the “ground truth” of development ample, SIGPLAN sponsors a series of conferences on the
practices by focusing on artifacts. We use surveys to enable history of programming languages (HOPL). The bulk of the
more direct inquiries to humans about their decision making papers are retrospectives by language designers on a par-
process. Decision making has unclear physical artifacts and ticular language or related sequence of languages. In con-
is subject to perception: it is unclear how to understand de- trast, our work seeks to make comparisons across languages
cision making based on just typical repository information. and communities. Furthermore, HOPL retrospectives tend to
Furthermore, we use surveys to reveal extrinsic data that is include deep but anecdotal analysis by language designers,
not in typical software repositories, such as demographics. while we perform quantitative data analysis.
Small-scale surveys have been used to answer some lan-
guage usage questions. Datero and Galup ran a web survey
9. Conclusions
to examine differences in language knowledge by gender [6].
They found modest differences. For example, within a pool This paper has looked at programming language adoption
of professionals, male developers were more likely to know through the lenses of four separate research questions: large-
most languages, and COBOL was the only language with a scale statistics, programmer decisions, language learning,
pronounced female lean. A study at a single American uni- and general beliefs about languages. These separate lines
of inquiry support each other and paint a unified picture

16
of adoption, with lessons for language designers, advocates, similar languages features. This suggests that experience and
educators, and employers. training shape developer language perception, underscoring
In Section 3, we asked what statistical patterns language the results in Section 5 about the importance of education.
adoption obeyed. We demonstrated three claims: First, pop- We find that developers consider ease and flexibility to be
ularity falls off steeply and then plateaus according to a more important than correctness. Developers show signif-
power law. Second, the less popular a language, the more its icant unease and unenthusiasm for static typing. This sug-
popularity varies from niche to niche – popular languages gests that today’s type systems may err too much on the side
are consistently popular across domains of use, less-popular of catching bad programs rather than enabling flexible devel-
languages tend to have specific domains. Last, developers opment styles. Developers emphasize the benefit of types in
switch between languages based primarily on the domain understanding programs, suggesting one benefit researchers
and use of a language, not based on its linguistic features can build upon.
(syntax or semantics.) Our results also help inform the broader computer science
Section 4 used survey data to show what factors influ- community. Since language selection is tied to libraries,
ence developers when picking projects. We found that exist- legacy, and familiarity, there are history-effects and therefore
ing code, existing expertise, and open source libraries are potentially multiple stable equilibria. This suggests that if
the dominant drivers of adoption. This dovetails with the today’s popular languages can be replaced or improved, the
previous finding: libraries and code are niche-specific, and changes will be durable.
therefore this developer motivation helps explain the statisti- Beyond the immediate results of this study, our experi-
cal findings above. The fact that developers typically do not ence has implications for future empirical research of pro-
consider particular language features important in choosing gramming languages, which is less actively practiced than in
languages is likewise consonant with our statistical finding software engineering. We found survey methods to be a pow-
that developers do not tend to switch between semantically erful tool for exploring hypotheses about language adoption.
related languages. We suspect these methods will be increasingly valuable go-
One consequence of these findings, taken together, is ing forward, especially given the popularity of the Internet
that language designers and advocates should emphasize and online courses. Likewise, we hope our data, and the
libraries. It is easy to find anecdotal examples of libraries methodology subtleties we encountered in gathering it, will
that had major influence on language adoption within niches, support future analysis efforts. Of particular note are our
such as numpy for numerical programming in Python and large set of responses, tracking of respondent demograph-
Rails for web applications in Ruby. ics, and solicitation of data about concrete languages and
Section 5 asked what causes developers to learn lan- projects.
guages. We find that professional developers learn and forget We have examined basic questions about language adop-
languages throughout their careers, and that as a result, age tion. Going forward, there are many more questions about
has little to do with language choice. Some languages are the sociotechnical nature of programming languages [15].
easier to learn than others, and the self-reported ease with
which developers learn does not seem closely related to the Acknowledgments
underlying simplicity of a language’s formal semantics. Past
education has moderate influence. Having been exposed to a We thank David MacIver for providing the Hammer data,
language paradigm in school makes developers more likely David Patterson and Armando Fox for the MOOC data,
to learn or remember similar languages later in their career. and SourceForge for theirs. Matt Torok, Philip Guo, and
In both Sections 4 and 5, we found that developers demo- Jean Yang helped publicize our Slashdot survey and pro-
graphics strongly differ in their language preferences. We vided valuable advice. Adrienne Porter Felt and Julie Wu
make two observations relating to this occurrence. First, em- provided helpful methodological guidance. Anonymous re-
pirical analysis of programs written in the same language viewers significantly helped revise earlier drafts. Some of
may need to check for sample bias due to developer demo- our anonymous reviews were among the most careful and
graphics. Second, ecological models may apply to language constructive ones that we have ever received.
adoption as we hypothesized in earlier work [15]. Ecologi-
cal models would predict that languages spread along demo- References
graphic boundaries because the languages compete for them. [1] Ohloh, the open source network. http://ohloh.net.
Our data reveals that spread patterns exist, and that develop-
[2] Sourceforge. http://sourceforge.net.
ers do indeed maintain a limited working set of languages.
Finally, we looked at developer feelings about languages [3] Tiobe index. http://www.tiobe.com/index.php/
when not tied to particular projects. Libraries still rate content/paperinfo/tpci/index.html.
highly, supporting the results of Section 4. We also found [4] R. Agarwal and J. Prasad. A Field Study of the Adoption of
that developers have divergent feelings about semantically Software Process Innovations by Information Systems Profes-
sionals. IEEE Trans. Engr. Management, 47(3), 2000.

17
 [5] Y. Chen, R. Dios, A. Mili, L. Wu, and K. Wang. An empir- [16] F. Morandat, B. Hill, L. Osvald, and J. Vitek. Evaluating the
ical study of programming language trends. IEEE Software, design of the R language. In European Conference on Object-
22:72–78, May 2005. Oriented Programming (ECOOP), 2012.
[6] R. Dattero and S. D. Galup. Programming languages and [17] S. Okur and D. Dig. How do developers use parallel libraries?
gender. Communications of the ACM, 47(1):99–102, 2004. In Foundations of Software Engineering (FSE), 2012.
[7] F. D. Davis, R. P. Bagozzi, and P. R. Warshaw. User accep- [18] C. Parnin, C. Bird, and E. Murphy-Hill. Java generics adop-
tance of computer technology: a comparison of two theoreti- tion: how new features are introduced, championed, or ig-
cal models. Management science, 35(8):982–1003, 1989. nored. In Proceedings of the 8th Working Conference on Min-
[8] M. E. Glickman. Parameter estimation in large dynamic ing Software Repositories (MSR), 2011.
paired comparison experiments. Journal of the Royal Sta- [19] D. Patitucci. Gender and programming language preferences
tistical Society: Series C (Applied Statistics), 48(3):377–394, of computer programming students at moraine valley commu-
1999. nity college. Master of Science, Old Dominion University,
[9] S. Hanenberg. Faith, hope, and love: an essay on software sci- 2005.
ence’s neglect of human factors. In Proceedings of the ACM [20] G. Richards, C. Hammer, B. Burg, and J. Vitek. The eval that
International Conference on Object-Oriented Programming men do: A large-scale study of the use of eval in JavaScript
Systems, Languages, and Applications (OOPSLA), 2010. applications. In European Conference on Object-Oriented
[10] B. C. Hardgrave and R. A. Johnson. Toward an information Programming (ECOOP), 2011.
systems development acceptance model: the case of object- [21] C. K. Riemenschneider, B. C. Hardgrave, and F. D. Davis. Ex-
oriented systems development. IEEE Trans. Engr. Manage- plaining software developer acceptance of methodologies: A
ment, 50(3), 2003. comparison of five theoretical models. IEEE Trans. Software
[11] Q. Hardy. Technology workers are young (really young). Eng., 28, 2002.
http://bits.blogs.nytimes.com/2013/07/05/ [22] E. Rogers. Diffusion of innovations. Free Press., New York,
technology-workers-are-young-really-young/, NY, 1995.
2013. [23] C. Scaffidi, M. Shaw, and B. Myers. Estimating the numbers
[12] S. Karus and H. Gall. A study of language usage evolution of end users and end user programmers. In IEEE Symposium
in open source software. In Proceedings of the 8th Working on Visual Languages and Human-Centric Computing, pages
Conference on Mining Software Repositories (MSR), 2011. 207–214, 2005.
[13] D. R. MacIver. The hammer principle. http:// [24] S. Sutton. Predicting and explaining intentions and behavior:
hammerprinciple.com/therighttool, 2010. How well are we doing? Journal of Applied Social Psychol-
[14] L. A. Meyerovich and A. Rabkin. How not to survey devel- ogy, 28(15):1317–1338, 2006.
opers and repositories: experiences analyzing language adop- [25] V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis.
tion. In Workshop on Evaluation and usability of program- User acceptance of information technology: Toward a unified
ming languages and tools (PLATEAU), 2012. view. MIS quarterly, pages 425–478, 2003.
[15] L. A. Meyerovich and A. Rabkin. Socio-PLT: Principles for
programming language adoption. In Onward!, 2012.

18