THE TOOLBOX REVISITED

Paths to Degree Completion From High School

Through College

U.S. Department of Education

1
THE TOOLBOX REVISITED

Paths to Degree Completion from High School Through College

Clifford Adelman
Senior Research Analyst
Policy, Research, and Evaluation Staff
Office of Vocational and Adult Education
U.S. Department of Education

2
U.S. Department of Education
Margaret Spellings
Secretary

Office of Vocational and Adult Education

Beto Gonzalez
Acting Assistant Secretary

February 2006

The views expressed herein are those of the author and do not necessarily represent the positions
or policies of the U.S. Department of Education. No official endorsement by the U.S.
Department of Education of any product, commodity, service, or enterprise mentioned in this
publication is intended or should be inferred. This document is in the public domain.
Authorization to reproduce it in whole or in part is granted. While permission to reprint this
publication is not necessary, the citation should be:

Adelman, C. The Toolbox Revisited: Paths to Degree Completion From High School Through
College. Washington, D.C.: U.S. Department of Education, 2006.

To order copies of this report,

write to: ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box
1398, Jessup, MD 20794-1398;

or fax your request to: (301) 470-1244;

or e-mail your request to: edpubs@inet.ed.gov;

or call in your request toll-free: 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet
available in your area, call 1-800-872-5327 (1-800-USA-LEARN). Those who use a
telecommunications device for the deaf (TDD) or a teletypewriter (TTY) should call 1-877-576-
7734;

or order online at: www.ed.gov/pubs/edpubs.html.

This report is also available on the Department’s Web site at:

www.ed.gov/rschstat/research/pubs/toolboxrevisit/index.html

On request, this publication is available in alternate formats, such as Braille, large print, or
computer diskette. For more information, please contact the Department’s Alternative
Format Center at (202) 260-0852 or (202) 260-0818.Organization of This Data Essay

The following is a brief outline of The Toolbox Revisited so the reader knows what to expect.

3
 Part I: Background. The introduction presents the basic question, the data sets invoked,
the purposes and statistics of the investigation, and the demography of the subject universe.

Part II: Variables Explored and Used in This Analysis. This short section of the study
lists all the independent variables that were considered and provides brief definitions and basic
statistical characteristics. A summary figure (pp. 20–21) indicates which of these met the criteria
for inclusion in the logistic narrative of Parts III and IV. A more elaborate glossary
(pp.179–193) provides details on the construction of these variables, allied data and commentary,
and will be of particular interest to researchers.

Part III: What Is and What Happens Before Matriculation. Here we begin the
chronological narrative, using both descriptive and multivariate data, of what ultimately made a
difference in bachelor’s degree attainment by December 2000 for 1992 12th-graders who
attended a four-year college at any time. Part III begins with background demographic
characteristics, then adds the critical components of high school academic history.

Part IV: Matriculation and Beyond. This section continues the cumulative steps of the
logistic narrative, starting with the characteristics of entry to the postsecondary world, and
continuing with first calendar year performance, financing considerations, attendance patterns,
and extended performance (that is, taking students’ entire undergraduate careers into account). It
includes a special consideration for the second calendar year of enrollment and concludes its
logistic narrative with attention to two very powerful variables: continuous enrollment and the
ratio of course withdrawals and repeats to the number of all courses attempted.

Part V: Closing the Gap. Having demonstrated how the universe of independent
variables is related to degree completion, The Toolbox Revisited then asks two questions: (1) To
what extent do the major national data sets agree on "graduation rates"? And (2) Given what we
have learned about what makes a difference in degree completion, what variables provide the
most promising guidance for closing the gap in graduation rates, by race/ethnicity and
socioeconomic status, for students who attend a four-year college at any time?

Part VI: The Missing Element of This Story. A key missing part of the story that is a
by-product of the limited features of the NELS:88/2000 is addressed in this section: the content
standards of high school and postsecondary course work. Other brief "excursions"—timing and
reasons for permanent ("status") drop out from college, and time-to-degree—are placed in
Appendices H and K respectively.

Part VII: Messages. Finally, The Toolbox Revisited offers some messages—to students
and to those who engage in public discourse about the issues we have covered—and highlights
the major conclusions of the study.

Appendices: With one exception, appendices are presented in the order in which they are
cited in the text. The exception is the last appendix, Appendix L, that contains a variety of
reference tables on miscellaneous topics raised in the text.

4
[THIS PAGE INTENTIONALLY LEFT BLANK]

5
 Part I
Background

This study explores the academic resources and momentum students build through their high
school and college careers, and analyzes the relationships between those factors and degree
completion rates. It departs from most previous research on attainment by focusing on the
details of students’ high school and college curricula and academic performance that are
available from transcript records. Its principal data are drawn from the National Education
Longitudinal Study of 1988 (hereafter referred to as the NELS:88/2000 or just NELS). This
longitudinal study followed a national sample of over 12,000 students (representing a weighted
2.9 million students) from the time they were in the eighth grade in 1988 to roughly age 26 or 27
in December 2000.

In round numbers, of the high school graduates in this cohort, 83 percent engaged in some form
of postsecondary education by age 26, and 68 percent attended a four-year college at some time.
Of the group who attended four-year colleges at some time (which includes a substantial
proportion of students who began postsecondary study in community colleges), 66 percent
earned a bachelor’s degree.

While the 66 percent completion rate sounds impressive for a mass system of higher education, it
masks an unhappy differential by race/ethnicity, and more so by socioeconomic status. As we
strive to improve high school graduation rates, to invite greater numbers of high school graduates
into the postsecondary system, simply to maintain—let alone improve—our completion rates
will take a great deal of effort. We need constructive guidance and benchmarks.

This study was designed as a follow-up and replication of a previous attempt to provide that
guidance—Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor’s
Degree Attainment (hereafter referred to as the original Tool Box).1 Since its publication in 1999,
the original Tool Box has become one of the most frequently cited works in public discussions
about—and initiatives to improve—the preparation of students for higher education. Its most
visible uses have included the presentations of the Texas “Master Scholars” program (2002), the
revisions in entrance requirements for the University of North Carolina system (2000), standards
goals set by the Illinois Board of Higher Education (2001), and suggestions of the Education
Commission of the States for state high school graduation policies (2001). It was summarized in
major litigation addressed to inequities in opportunity-to-learn in high schools (e.g., Daniel v.
California 1999), and in the research literature, its analyses of both precollegiate preparation and
post-matriculation attendance patterns and performance have also been marked (e.g., Horn and
Kojaku, 2001; Zucker & Dawson, 2001; Cabrera and La Nasa, 2001) and improved on
(DesJardins, McCall, Ahlburg and Moye, 2002; Cabrera, Burkum, and LaNasa 2005).

The analyses in the original Tool Box were based on the High School & Beyond/Sophomore

1
Out of print but available at http://www.ed.gov/students/prep/college/thinkcollege/early/aboutus/edlite-
resources.html.

6
cohort longitudinal study (hereafter occasionally referred to as the HS&B/So), the second of the
U.S. Department of Education’s national grade-cohort longitudinal studies,2 which followed a
national sample of 10th-graders from 1980 to1992 in surveys and to September 1993 on
postsecondary transcripts. As with all of the grade cohort longitudinal studies carried out by the
National Center for Education Statistics,3 test scores are included in the database, along with
surveys of parents, school teachers, and school administrators. The modal high school graduation
year for the HS&B/So was 1982, and student age at the conclusion of the study was 29 or 30.

The HS&B/So data, while compelling, are now somewhat dated. There were considerable
changes in both the demography and postsecondary entrance behavior of high school seniors in
the comparatively short span of the decade between 1982 and 1992. Appendix A highlights
contrasts in selected background characteristics of all 12th-graders in the two longitudinal studies
cohorts. Some of these changes have been frequently observed (higher proportion of minority
students, westward and southward movements of populations, higher proportion of high school
seniors entering higher education and planning to earn a bachelor’s degree), and some rarely
observed (higher proportion who grew up in mixed-ethnicity neighborhoods, and a higher
proportion with parents who indicated some postsecondary education). The question naturally
arises as to whether the hypotheses and analyses based on the history of the High School &
Beyond/Sophomore cohort would hold up in the story of the NELS:88/2000. The NELS history
offers a more contemporary account,4 one that, in terms of secondary school records, may reflect
the high school curriculum reforms of the mid- and later-1980s5 that followed the discussion of
the report of the National Commission on Excellence in Education, A Nation at Risk (1983).
The Toolbox Revisited follows the same path of analysis using the NELS:88/2000 as did its
predecessor with the HS&B/So. In that sense, it is a replication. It is a modified replication,
however, because it introduces new constructs based on critiques of the original Tool Box and
because it offers a more refined chronology of steps from high school to the end of a student’s
undergraduate education.

A substantial amount of the analyses and endorsements that followed the original Tool Box
revived the flagging "seamless" K–16 themes of the 1980s post-Nation at Risk reform effort. In
an October 2000 policy brief that visited these issues, the Education Commission of the States
reminded people of what most of the 1980s reform reports did not address: the fact that K-12 and
postsecondary systems are governed in very different ways, even in the public sector of
postsecondary, and that there is a consequent "disconnect" of substantial dimensions (Education

Commission of the States 2000). The metaphor of bridges (Venezia, Kirst and Antonio 2003)
and the rhetoric of disconnect (e.g., Conley 2003) created a more sophisticated focus of analysis
2
The first was the National Longitudinal Study of the High School Class of 1972 (1972–86).
3
For a description of the grade-cohort studies, see Appendix B.
4
A fourth-grade-cohort longitudinal study is currently in progress. Called the ELS2002, it began with a national
sample of 10th-graders in 2002 and is expected to run at least through 2012. We obviously will have to wait some
years before we see its full results.
5
For an account of changes in high school graduation requirements in the years immediately following issuance of A
Nation at Risk, see Stedman and Jordan (1986).

7
than the reform reports of the 1980s embraced. The major work that sought to build new bridges
and connect the disconnects underscored the obligation of the system not merely to assure
college "access," but degree completion, with curriculum playing the key role. While "degree
completion," in public discourse, refers to the bachelor’s degree, the same principles apply to
associate degrees granted principally by community colleges as well (Adelman 2005a).

What Did the Original Tool Box Say?—And Based on What Kind of Evidence?

In a nutshell, here are the major conclusions of the High School & Beyond/Sophomore-based
Answers in theTool Box analysis:

1) Of three traditional measures of precollegiate educational history—curriculum configuration,

academic performance (on a scale that combines class rank and GPA), and assessed general
learned abilities (a senior year mini-SAT)—the intensity and quality of one’s secondary school
curriculum was the strongest influence not merely on college entrance, but more importantly, on
bachelor’s degree completion for students who attended a four-year college at any time. The
highest level of mathematics the student reached in high school played a significant role in the
strength of the curriculum configuration. One of the major contributions of Tool Box not only to
the literature, but to practice as well, was to change what was understood by the “entry
characteristics” of students by digging out and demonstrating the power of the academic
intensity of secondary school curriculum over combinations of standardized test scores and
grades.

2) By moving into the top two quintiles of the curriculum measure and completing a high school
mathematics course beyond Algebra 2, African-American students who started out in a four-year
college would increase their bachelor’s degree attainment rate from 45 percent to 73 percent;
Latino students who did the same would increase their bachelor’s degree attainment rate from 61
percent to 79 percent.6 These increases were significantly greater than those for white and Asian
students under this scenario, and, more importantly, were considerably greater than the effect of
moving into the top two quintiles of either test scores or class rank/GPA. In other words,
curriculum counts, particularly for minority students.

3) The three traditional precollegiate measures can be combined in a composite measure of

“Academic Resources.” This measure had continuing, statistically significant relationships with
bachelor’s degree completion in both linear and (more appropriately and convincingly) logistic
regression sequences involving four post-matriculation steps (blocks of variables for financial
aid, attendance patterns, first-year performance, and extended postsecondary performance).

4) Post-matriculation behaviors and attendance patterns that were strongly and positively
associated with bachelor’s degree attainment were continuous enrollment, transfer from a
community college to a four-year institution after more than 10 credits earned at the community
6
For African-American students in the HS&B/So who reached the 12th grade and entered postsecondary education,
52.3 percent started in four-year colleges, a percentage that rose to 54.2 in the NELS:88/2000. The comparable
percentages for Latino students, who are more likely to start in community colleges, were 38.9 percent for the
HS&B/So and 38.6 percent for the NELS:88/2000. For white students, the proportions starting in four-year colleges
were 54.3 percent for the HS&B/So and 57.4 percent in the NELS:88/2000.

8
college, and the trend in students’ grades.

5) Post-matriculation behaviors and attendance patterns that had a strong negative influence on
bachelor’s degree attainment were the ratio of courses from which the student withdrew or
repeated to all courses attempted, and earning less than 20 credits in the first calendar year of
postsecondary attendance.

6) Socioeconomic status had a modest and diminishing association with bachelor’s degree
attainment. Minority status had a modest negative association until performance (first-year
performance and continuing performance) was taken into account, at which point it had no
effect. Gender had no effect at any stage of the model. The only demographic variable to have a
strong (and in this case, negative) association with degree completion was becoming a parent by
age 20.

The overall message was about academic momentum and what adds to that momentum at each
stage of a student’s history from secondary school onward. The evidence was archival, was
treated in the tradition of quantitative history (Elder, Pavalko, and Clipp 1993; Clubb, Austin and
Kirk 1989; Haskins and Jeffrey 1990), and like this essay, does not claim causality.

Why use these data sets?

There are three types of national data sets available to construct longitudinal analyses such as the
original Tool Box and this replication, but only one type of data set—the NCES transcript-based
grade-cohort study—is truly suited to the task. The other two are (1) the Cooperative
Institutional Research Project (CIRP) occasional longitudinal follow-ups to its annual survey of
entering college freshmen, and (2) the NCES Beginning Postsecondary Students studies (BPS).
Each of these has its virtues, and will be revisited in Part V of this study when we compare what
three different longitudinal studies of the 1990s say about degree-completion rates. The CIRP
produces an enormous amount of information on student attitudes, values, and college
experiences, and does so with large samples of (principally) entering four-year college students.
Its occasional longitudinal follow-ups involve sufficient history (e.g., six or nine years) to track
not only long-term undergraduate completion rates, but also postbaccalaureate education (Astin
1993). The BPS longitudinal studies are shorter (five or six years), not dependent on
institutional decisions to participate (as is the CIRP), inclusive of students of all ages at entry,
and, as befits their principal population sample (a subset of the triennial National Postsecondary
Student Aid Study), contain very strong and reliable financial aid data. The BPS study of
1995/96–2001 will be used at a number of points in this study to expand the range of our
observations.

However, in both cases, all features of precollegiate history must be rendered as exogenous
variables. The high school histories provided by students in CIRP are retrospective, and the
precollegiate histories in the Beginning Postsecondary Students studies derive from a

combination of retrospective offerings by students and accounts on the student information

questionnaires for those who took either the Scholastic Assessment Test (SAT) or American
College Test (ACT) within the two years prior to the BPS start date (nearly half the students in

9
the BPS studies did not take either exam). As Kahn and Nauta (2001) demonstrated, there is an
inevitable loss of accuracy in the process of these retrospections.

On the other hand, the very nature of a longitudinal study population assembled in the 10th grade
(High School & Beyond/Sophomore cohort) or eighth grade (NELS:88/2000) renders
precollegiate history endogenous to analytic models. The transcript-base overrides student
accounts and provides far more detail than either paper-and-pencil surveys (used by CIRP) or
computer-assisted telephone interviews (used by the BPS studies) can provide.

Every data set sacrifices something. No data set is constructed with the questions a particular
researcher may have a decade later, so variables are derived and secondary. The High School &
Beyond/Sophomore study and the NELS:88/2000 are wanting in many ways where the other
studies are strong: financial aid (BPS), and changes in values and opinions (CIRP). But as
stories about the core activities we call education, they are unsurpassed by the others.

Purposes and Statistics of This Monograph

The primary purpose of this monograph is to trace the elements of academic momentum as they
played out in the secondary school and college history of the High School Class of 1992 (through
December 2000) compared with the parallel history of the High School Class of 1982 that was
the foundation for the original Tool Box study. In the process, the analysis is enriched by
including variables we learned to construct or modify on the basis of commentaries and critiques
of Tool Box, and by a more accurate chronological order in presentation of those variables. The
portrait of academic momentum that emerges is a framework within which more sophisticated
analyses can be pursued, and within which ameliorative policies (the tools) can be advocated.
This study does not pretend to answer complex questions about indirect effects of home, peer,
school, and postsecondary institution interactions, but rather trusts future research to deal with
those issues.

A secondary task is to demonstrate the construction of a replication when the two data sources, a
decade apart, are presumably parallel, but turn out to be something less (for a key example, see
Appendix C). The principal encouragement for the replication is that the bachelor’s degree
attainment rate for students who attended a four-year college at any time remained the same
despite differences in the length of cohort history: 65.6 percent for the HS&B/So over 11 years,
and 66.5 percent for the NELS:88/2000 cohort over 8.5 years (Adelman 2004a, table 2.2, p. 21).
If the HS&Beyond/So history were truncated at 8.5 years, the bachelor’s degree attainment rate
would have been 59.7 percent. From this perspective there was a marked improvement in degree
completion for traditional age college students over the two decades in question. It is natural to
ask how this happened, and if the answers to that question provide any guidance for the future.

In terms of statistical technique, both the original Tool Box and The Toolbox Revisited use simple
logistic regression, not structural equations or other path models that are common to causal
inquiries or searches for indirect effects, e.g., of discrete aspects of school or college

10
environments (Dey and Astin 1993). A logistic regression is focused on an event that either
happens or it doesn’t. The dependent variable is dichotomous: yes or no. The independent
variables are judged within each model by the degree to which they contribute to what happened
in relation to or controlling for all other independent variables in the model (Hair, Anderson,
Tatham, and Black 1995).

There are a number of ways of expressing this “degree.” One is by an “odds ratio,” which,
expressed in a simple way, is a ratio of the odds that X will happen given a unit of change in the
independent variable to the odds of X not happening, and ultimately shows the strength of
association between the independent and dependent variables—with the closer the odds ratio
to 1, the less the strength of the association.1 This was the measure used in the original Tool Box.
Another way of expressing the value of the contribution of an independent variable is by a
“Delta-p” statistic that says every unit change in the independent variable changes the
probability that X will happen by Y percent given the values of the other variables in the model
(Peterson 1985; Cabrera 1994). The narrative of The Toolbox Revisited relies on Delta-p,2 and
the logistic model tables provide Delta-p statistics only for those parameter estimates that are
statistically significant since there is no way to determine the statistical significance of the Delta-
p itself (Cabrera 1994).3

But in this paper there is a major methodological departure from the original Tool Box study:
there are seven (and not five) steps in the model employed, all driven by the empirical history of
the NELS:88/2000 students. Following St. John, Paulsen, and Starkey (1996), the blocks of
variables in each step were entered "in a sequence that parallels the order in which students pass
through well-established stages of persistence behavior" (p. 194) on their way toward bachelor’s
degree completion (or not). Each of the seven steps, too, is cumulative. That is, variables in one
step that meet the statistical criteria for remaining in the model are carried forward to the next
step. This extended accounting, which we will call a "logistic narrative," allows "a meaningful
examination of the direct effects of variables on persistence, as well as their interactions with the
variables entered in successive steps" (St. John, Paulsen, and Starkey 1996, p. 194).

The reader can already tell that there is a great deal of technical material in this presentation, but
it is presented in the spirit of the U.S. Department of Education’s goal of building a culture of
evidence. The author trusts that reports such as The Toolbox Revisited will contribute to the
attainment of that goal. Some of the technical material is placed in appendices so that
researchers have access to documentation, and all readers have access to reference material.

Who Are We Talking About? And Who Are We NOT Talking About?

1
Odds ratios, as Peng, So, Stage, and St. John (2002) remind us, are not odds. Their interpretation is not as
transparent as the original Tool Box assumed them to be. And while an R2 statistic is presented in the logistic tables
of this study, it cannot be read the way one would interpret an R2 in a linear regression, i.e., it does not indicate the
percent of variance in the dependent variable that is explained by the independent variables (Long, 1997). For that
reason, it is called a "pseudo R2" (Cabrera 1994) and is one of a number of measures of goodness-of-fit. As blocks of
variables are added to the model in the stepwise manner followed here, the pseudo R2 should increase.
2
For the computation of Delta-p, I am using a shortcut recommended by Paul Allison of the University of
Pennsylvania: bp(1-p), where b is the logistic coefficient and p is the probability for the dependent variable in the
model. This heuristic produces slightly higher values than the formula advanced by Petersen (1985).
3
For all technical issues, please see Appendix D.

11
As was the case for the original Tool Box, the basic universe of analysis in The Toolbox Revisited
does not consist of everybody who started out in the cohort. We have to be very clear about this.
The basic question of both studies is:

What demographic, high school performance, postsecondary entry, and

postsecondary history (attendance patterns, academic performance) factors are
convincingly associated with bachelor’s degree attainment for 12th-graders who
subsequently attended a four-year college at any time in their undergraduate careers?

This is not a question about completing high school, completing high school on time, or
completing high school on time with a standard diploma (as opposed to a GED or certificate of
attendance).4

This is not a question about entering the postsecondary system. We are not talking about
"access." Nor is it a question about "persistence" to the second term or the second year
following entry to postsecondary education. It certainly is not about "retention" in the same
institution to the second term or the second year.

This is a question about completion of academic credentials—the culmination of opportunity,

advisement, choice, effort, and commitment.

The question is carefully worded. The universe does not include people who never reached the
12th grade. In fact, as a consequence of the sampling design and weighting of the two
longitudinal studies, both essays include only those students who were in the 12th grade in the
same year,5 along with those who graduated from high school. The definition of the universe
also requires that we have demographic information, high school transcripts,6 test scores, and
postsecondary transcripts for everybody whose careers are subject to analysis. Not everybody
can present all this information, and, with rare exceptions (involving some aspects of high school
records, and as explained in Appendix C) both the original Tool Box and The Toolbox Revisited
decline to impute any of this information.

There are two other notable features of the basic question. First, both essays argue that,
regardless of what students say on a survey about their education expectations, it is what they
actually do that counts. So the only students we can talk about honestly with respect to
bachelor’s degree attainment are those who set foot in a bachelor’s degree-granting institution at
some time. This criterion obviously includes students who started their postsecondary careers in
community colleges and other types of sub-baccalaureate institutions. Second, the term used to

4
For details on the high school graduation status of those who were eighth-graders in 1988, see Appendix L, table
L1.
5
For details on those in the NELS:88/2000 cohort who, though in the 12th grade in 1992 along with others, had
previously been held back at least one year in the course of their schooling, see Appendix L, table L2.
6
Only 21.3 percent (s.e. = 1.17) of the NELS:88/2000 who were in the 12th grade in 1992 were missing high school
transcripts, compared with 73.7 percent (s.e. = 2.58) of those who had either dropped out of high school or were not
in-grade. Nearly 100 percent of the former group graduated from high school with a standard diploma.

12
 describe the relationship between the dozens of independent variables describing demography,
high school performance, postsecondary entry, and postsecondary history and the dependent
variable of bachelor’s degree completion is "convincingly associated." Neither essay claims
"cause" or "prediction."

To illustrate how these conditions restrict the universe, let us start with 2.93 million 1988 eighth-
graders in the NELS:88/2000 cohort. Table 1 lays out the stages of contraction of the basic
population until we reach the subject universe of this study.
Table 1. From macro to micro: Contraction of the universe of 1988 eight-graders to
the universe subject to analysis in The Tool Box Revisited

Descending
Description of Universe Percent weighted Na

A. Initial universe of 1988 8th graders 100.0 2.93M

B. Of (A), those who were in the 12th grade in 83.6 (0.98) 2.45M
1992

C. Of (B), those who continued to postsecondary 81.7 (1.28) 2.0M

education at any time through December 2000

D. Of (C), those who presented complete high 80.5 (1.01) 1.61M

school transcripts, test scores,b complete
postsecondary transcripts records, and
socioeconomic status information

E. Of (D), those who attended a four-year college 73.5 (1.00) 1.19M

at any time.

Net percentage of 1988 8th grades in the universe 41 1.19M

Net percentage of 1992 12th graders in the 51 1.19M
universe
a
Like other NCES longitudinal sudies, the NELS:88/2000 chort is stratified sample, in which each student is
assigned a weight to represent other similar students in the chort (see Curtin, Ingels, Wu, and Heuer 2002)

b
See definition of SRTSQUIN in Glossary.
NOTES: Standard errors are in parentheses.
SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES
2003-402 and Supplement).

13
Looking more closely at our universe in terms of the types of schools attended, we find that
20.2 percent (s.e. = 1.13) started in community colleges, and 41.0 percent (s.e. = 1.29) earned
credits from community colleges, whether or not they started at community colleges. While we
will describe the attendance patterns of the NELS:88/2000 cohort in more detail later, the point is
that the universe under analysis is not limited to those who attended only four-year institutions,
and that the community college plays a significant role in the careers of the group of students
under consideration.
To repeat: Our subject universe consists of half the 1992 12th-grade students in the
NELS:88/2000 cohort. We are not talking about:
• students who did not graduate from high school or those who graduated
with something other than a standard diploma (G.E.D. or Certificate of
Attendance);
• students for whom we do not have full high school records (transcripts,
grades, test scores);
• students for whom socioeconomic status could not be determined;
• students who did not enter any postsecondary institution by December
2000, when they were 26 or 27 years old; and
• students who entered the postsecondary system, but never set foot in a
bachelor’s degree-granting institution.

That is the other half. Some 22.2 per cent (s.e. = 1.05) of 1992 12th-graders who attended a
four-year college at any time are in this “other half” because they are missing key information
that excludes them from the universe under analysis. Readers interested in the demographic
differences between included and excluded students due to missing data elements are referred to
Appendix E.

Comparative demography of the subject universe

Given the strong boundaries drawn around the subject universe, how does its demography
compare to less restrictive definitions? Table 2 sets out the major demographic categories, and
compares the target subject universe with (a) all 1988 eighth-graders who participated in the last
survey of the NELS:88/2000 in 2000, (b) all participants in the 1992 survey, whether or not they
were in school or in the 12th grade,7 and (c) all 1992 12th-graders who attended any
postsecondary institution by December 2000.

As readers work from left to right across the columns of table 2, they will notice that gender
balances are even until the moment of entry to postsecondary education, at which point women
pull ahead of men. The same type of observation applies to the race/ethnicity distribution: At the
point of entry to postsecondary education, the proportion of African-Americans, Latinos, and
American Indians declines, while that of whites and Asians rises. In the universe for this study,

7
The reader should note that, unlike the High School & Beyond/Sophomore cohort study, the NELS:88/2000 sample
was "refreshed" in 1992 to be representative of 12th-graders in that year. The "refreshed" students are included in
our analyses.

14
Table 2. For each of four definitions of the universe of students in the NELS:88/2000, percent
distribution by gender, race/ethnicity, socioeconomic status quintile, and second
language background
All 1992 12th- 1992 12th-graders who
graders who attended a four-year
1988 All 1992 entered college at any time, and
Demographic eighth - survey postsecondary met other criteria to be
variable graders participants education subjects of this studya
Gender

Men 49.7 (1.01) 49.9 (0.83) 46.5 (0.93) 48.8 (1.27)

Women 50.3 (1.01) 50.1 (0.83) 53.5 (0.93) 51.2 (1.27)

Race/ethnicity

White 71.7 (1.50) 71.5 (1.30) 74.9 (1.29) 78.2 (1.31)

African-American 12.9 (1.26) 12.7 (0.94) 10.3 (0.90) 9.4 (1.03)
Latino 10.5 (0.87) 10.4 (0.84) 9.1 (0.88) 7.0 (0.72)
Asian 3.5 (0.32) 3.7 (0.31) 4.8 (0.43) 4.7 (0.42)
American Indian 1.4 (0.43) 1.7 (0.55) 0.7 (0.23) 0.6 (0.18)

Second language
background

Non-native 8.6 (0.68) 10.1 (0.83) 10.0 (0.90) 7.4 (0.67)

speaker of
English
Native speaker 3.3 (0.33) 2.7 (0.21) 2.4 (0.23) 2.2 (0.28)
of English from
a second
language
household

Socioeconomic
status quintile

Highest quintile 21.3 (0.92) 21.1 (0.88) 29.1 (1.08) 38.5 (1.52)
2nd quintile 20.8 (0.79) 21.0 (0.69) 25.3 (0.88) 26.4 (1.24)
3rd quintile 20.7 (1.10) 19.8 (0.68) 20.2 (0.73) 17.7 (0.85)
4th quintile 19.6 (0.83) 19.2 (0.66) 15.4 (0.61) 11.7 (0.59)
Lowest quintile 17.6 (0.93) 18.9 (0.85) 10.0 (0.73) 6.8 (0.50)
a
1992 12th-graders with known socioeconomic status and high school records (transcripts and test scores), who graduated
from high school by December 1996, and attended a four-year college at any time.
NOTES: Standard errors are in parentheses. Columns for gender, race/ethnicity, and socioeconomic status quintile may
not add to 100.0 percent due to rounding.
SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES 2003-402 and
Supplement).

15
the proportion of whites rises even further, while that of Asians, American Indians, and African-
Americans8 holds steady, and that of Latinos declines. There are no statistically significant
differences in the proportion of students who were nonnative speakers of English9 until the final
boundaries are drawn for the universe for this study.

The most dramatic and expected changes in the distributions of table 2 are by socioeconomic
status quintile, with quantum leaps in the proportion of students from the top two quintiles as
soon as one crosses the postsecondary line, and parallel declines in the proportion of students
from the bottom two quintiles. Only the third quintile (the 41st–60th percentile) remains stable
until the final contraction of the cohort for the universe used in this study.

What do these demographic changes within the universes of students who might be the subjects
for a study mean? The Toolbox Revisited—and Answers in the Tool Box before it—is a study
about students who graduate from high school and go on to college. Given historical data, it is
not surprising that this group will evidence a higher SES profile and a lower percentage of under-
represented minorities than others in the same grade cohort. Since the 1980s, women have been
in the majority in this group.10 Within our subject universe, demography is not necessarily
destiny, as the original Tool Box demonstrated. For the comparison group—the other half that
either does not graduate from high school, graduates but doesn’t continue its education, or
graduates and continues but never at a four-year college—The Toolbox Revisited says: If we want
more people to wind up with at least a bachelor’s degree in life, here are some guidelines based
on the experience of people who tried.

Summary of Part I

What do Answers in the Tool Box and The Toolbox Revisited do, and what do they not do? Both
studies use the histories of 12th-graders who subsequently attended a four-year college at any
time (entered in any term and not just the fall term, entered as part-time as well as full-time
students, entered in community colleges as well as four-year institutions) to indicate what factors
in those histories are associated with completing a bachelor’s degree—not in four years, not in
six years, but whenever the longitudinal tracking ended (8.5 years from high school graduation
for the class of 1992; 11 years for the class of 1982). They do not ascribe cause or pretend to
predict. They recognize that what is associated with degree completion in one generation may
not be associated with it in the next, or that the strength of association may change. Conditions
and populations change, after all. Rigid prediction is a risky call, and besides, that’s not what the
data and statistical standards allow one to do.

8
The difference in the African-American proportion of the two universes is not statistically significant.

9
Nonnative speakers were defined as those who said their first language was not English and, in grade 10 and/or 12
(the 1992 survey) indicated that they spoke to their mothers most or all of the time in their first language.
10
Though to the extent to which the group includes students who commenced their postsecondary studies in
community colleges—and it does—that majority will be a little less.

16
[THIS PAGE INTENTIONALLY LEFT BLANK]

17
 With two major exceptions—high school academic curriculum intensity and
the composite high school performance variable we call "Academic
Resources, both of which are covered in Part III—the construction and
allied information for most of the variables explored in The Toolbox
Revisited, are detailed in the Glossary (starting on p. 179). Whenever the
reader asks for details on "How was that defined?" or “How did they get
that?” the Glossary is the place to go.

Part II
Variables Explored and Used in This Analysis

The purpose of this section is to provide the reader with an advance reference list of the variables
considered and used in this analysis. The reader should keep in mind that “variables” are
representations of realities (e.g., first-year college grades) or constructs (e.g., transfer). We use
them as a shorthand.

The variables listed and described are presented in the order of the steps of the logistic narrative
for which they were considered. The collection represents neither a "fishing expedition" nor a
"mind dump," as a majority of the variables considered were determined by the attempt to
replicate the original Tool Box study with a more recent parallel cohort. A quick summary of their
properties can be found in figure1.

Demographic background variables defined

1) NNSE—Nonnative speaker of English. A dichotomous variable marking those students

whose first language was not English and who, in grade 12, reported that they conversed in that
language with their mothers most or all of the time.

2) IMMIG—Parent immigrant status. A dichotomous variable indicating whether the student’s

parents were immigrants to the United States within the previous 10 years.

3) BROSIS—Number of siblings. A dichotomous variable marking students with three or more

siblings, versus those with one, two, or none.

4) FIRSTGEN—First generation postsecondary student. A dichotomous variable indicating

students whose parents had never attended a postsecondary institution.

5) FAMINC—Family income. This variable was first set in six bands, then trichotomized to
yield upper-, mid-range, and low-income populations.

6) URBAN—A dichotomous variable indicating whether the student’s high school was located in

18
an urban area.

7) NEWCHILD—A dichotomous variable to mark students, male or female, who became parents
by the time they were 20 years old.

8) RACE—A dichotomous race/ethnicity variable, with minority (African-American, Latino, and

American Indian) = 1 and White and Asian = 0.

9) GENDER—A dichotomous variable in which male = 1 and female = 0.

10) SESQUINT—Socioeconomic status quintile. See Glossary.

High school background variables defined

1) EDUANTIC—Education anticipations. A three-level variable: (a) consistently (in grades 10

and 12) expected to earn a bachelor’s degree; (b) raised expectations to the bachelor’s degree
between grades 10 and 12; and (c) either lowered expectations from the bachelor’s level between
grades 10 and 12 or never expected to earn a bachelor’s degree.

2) CLSSRNKQ—High school class rank/GPA quintile. See Glossary and Appendix C for
accounts of construction and limited imputation, respectively.

3) SRTSQUIN—Senior year test score quintile. A test of general learned abilities was
administered to NELS high school seniors and the results set out in percentiles. For those who
did not take that test but for whom SAT or ACT scores were available, those scores were
substituted using an equipercentile concordance methodology, weighted, and set out in quintiles.
See Glossary for a more detailed account.

4) HIGHMATH—Highest level of mathematics reached in high school. A five-level variable

with calculus and precalculus at the high end and Algebra 1 and prealgebra at the low end. See
Glossary.

5) SCIMOM—High school momentum in science and mathematics. A three-level variable

combining highest level of mathematics with numbers of credits earned in core laboratory
science. See Glossary.

6) FLAN—Number of units of foreign language in high school on a five-level scale.

7) ADVANCE—Number of Advanced Placement (AP) courses. Three values based simply on

the number of AP courses recorded: three or more, one or two, and none. See Glossary for a full
account of identifying AP course work.

8) HSCURRQ—Academic intensity of high school curriculum, in quintiles. This variable is the

19
core of the analysis of students’ precollegiate histories, and, hence, is described in detail in
Part III and Appendix F.

9) ACRES—Academic Resources. A quintile index representing a composite of students’ pre-

collegiate attainment (curriculum plus class rank/GPA plus senior year test score). ACRES is the
dominant precollegiate variable in both the original Tool Box and The Toolbox Revisited. The
construction of this variable is described in detail in Part III.

Postsecondary entry variables defined

1) FIRST4—A dichotomous variable indicating whether the first postsecondary institution

attended by the student was a four-year college.

2) DOCT—Another dichotomous variable indicating whether the first postsecondary institution

attended was a doctoral degree-granting institution.

3) SELECT—A dichotomous variable indicating that the first institution attended by the student
was either highly selective or selective.11 See Glossary for details on selectivity.

4) NODELAY—A dichotomous variable marking students who entered postsecondary education

within seven months of high school graduation.

5) ACCELCRD—Acceleration credits. A sum of all college credits earned by both course work
prior to high school graduation and by examination. The values of this variable were set at three
levels: more than 4 credits, 1–4, and zero.

First-year performance variables defined

1) LOWCRED—A dichotomous variable marking students who earned less than 20 additive
credits earned in the first calendar year of attendance. For descriptive data on the relationship
between number of credits earned in the first year and highest degree, see Appendix L, table L4.

2) FRSHGRAD—GPA in the first calendar year of attendance. Grade point averages were
determined for the first full calendar year of postsecondary attendance and were then set out in
quintiles. FRSHGRAD is a dichotomous variable that divides the highest two quintiles from the
other three.

3) FREM—A dichotomous variable indicating any remedial course work in the first calendar
year of attendance.
11
Institutional selectivity in the postsecondary files of all three grade-cohort longitudinal studies conducted by the
National Center for Education Statistics has five values: highly selective, selective, nonselective, open-door, and not
ratable. The first three of these values were based on the selectivity cells developed by the Cooperative Institutional
Research Project (CIRP) at UCLA for its annual survey (since 1966) or entering freshmen. For the distribution of
NELS:88/2000 postsecondary students by selectivity of first institution attended, see Appendix L, table L3.

20
4) FCOLMATH—Another dichotomous variable indicating whether the student earned any
credits in college-level mathematics during the calendar year following first enrollment.
"College-level mathematics" was defined to include college algebra, finite math, statistics,
precalculus, calculus,12 and a category of "liberal arts mathematics" courses that require the
equivalent of high school Algebra 2 as a prerequisite, and include such topics as game theory,
basic combinatorics, and foundations of numerical methods.

Financing postsecondary schooling variables

There are three dichotomous financing variables in both Answers in the Tool Box and The
Toolbox Revisited: GRANTS, LOANS, and STUWORK. Each indicates only whether the
student used that form of financing during his/her early postsecondary career. The reader is
referred to the text of Part IV for elaboration.

Attendance pattern variables defined

1) TRANSFER—community college to four-year. With students going back and forth between
community colleges and four-year colleges, it is important to mark transfer as a permanent
change of venue, a migration that is formally recognized by system rules. A transfer student is
one who (a) started in a community college, (b) earned more than 10 credits from the community
college before (c) enrolling in a four-year college and (d) earning more than 10 credits from the
four-year college. The only time limit set for these changes of venue and credit accumulation is
the length of the longitudinal study. In the case of the NELS:88/2000, that means 8.5 years from
the modal high school graduation data of June 1992. This is a dichotomous variable.

2) FOURTRAN—A dichotomous variable indicating transfer from one four-year college to

another. The algorithm for classic transfer from a community college to a four-year college was
fairly easy to construct. But to distinguish a true four-year-to-four-year college transfer required
an indirect route. See Glossary for details.

3) MULTINS—A dichotomous variable indicating that the student attended more than one
institution. This is a macro-vision of otherwise multidirectional student behavior.

4) SUMMER—Number of credits earned during summer terms, set in three bands: more than 4,
1–4, and none.

5) PARTTIME—A dichotomous variable tagging students whose enrollment intensity was ever
part-time. The construction of this variable involves complex algorithms, and the reader is
referred to the Glossary for details. (For an account of enrollment intensity in both the

12
The category of "calculus" includes not only the standard sequence but also special "short calculus" or "brief
calculus" courses, and specialized introductions to calculus for business, economics, and life science majors.

21
NELS:88/2000 and the Beginning Postsecondary Students longitudinal study of 1995/96–2001,
see Appendix L, table L5).

Extended postsecondary performance variables defined

1) TREND—Trend in student’s GPA: rising, flat, or falling. Cumulative undergraduate GPA was
measured at three points in time—at the end of the first calendar year following initial
enrollment, at the end of the first two calendar years following initial enrollment, and at the end
of the student’s undergraduate career, no matter when that occurred. See Glossary for further
elaboration.
2) CUMMATH—Number of credits earned in college-level mathematics: more than 4, 1–4,
and 0.
3) CHANMAJ—A dichotomous change-of-major variable. See Glossary for a full description.
Final factors variables defined

1) NOSTOP—In all three NCES postsecondary transcript-based grade-cohort studies,

noncontinuous enrollment was defined as more than a one semester (or its equivalent, e.g., two
quarters) stop-out period. In the dichotomous variable, NOSTOP, the student is considered
continuously enrolled even with one semester (or two quarters) off.

2) WRPT Ratio—A dichotomous variable. On one side of the dividing line are students who
withdrew from or repeated 20 percent or more of all courses in which they enrolled (ratio of non-
penalty withdrawal and no-credit repeat grades to all grades received).

Summary: Locus of Responsibility

Figure 1 presents the basic statistical characteristics for 39 of the major variables described
above that were either tested, used in trials, and/or employed in the final logistic narrative of The
Tool box Revisited. Seven of these variables describe demographic characteristics, and another
three label types of financial aid that ultimately are offshoots of demography. Of the remaining
29 variables, 14 are principally matters of student choice (e.g., changed major), five are
indicators of student academic effort (e.g., first-year grades), seven reflect interactions of student
choice plus student effort plus opportunity to learn (e.g., highest high school math), two mark
interactions of student effort and institutional judgment or guidance (e.g., first-year remediation),
and one (education anticipations) reflects student experience, attainment, and self-assessment,
along with the encouragement of family and peers. The student is at the center of all these
representations. The locus of responsibility for the way each of these variables will tilt lies as
much with the student as with external forces. The Toolbox Revisited is optimistic that most
students can make it all come out right. We will return to this optimism in Part VII.

22
 Figure 1. Values, means, standard deviations, and use of variables considered in the logistic narrative of Academic Momentum , with
universe confined to 1992 12th-graders with complete high school records and known socioeconomic status who attended a
four-year college at any time through December 2000

Programming Relationship to variables in

a
Variable Values name Mean S.D. Used Step# Answers in the Tool Box

b
Nonnative speaker of English 2 NNSE 0.1472 0.3544 No New
First generation student 2 FIRSTGEN 0.1664 0.3725 Trial only New
Number of siblings (more than 2 = 1) 2 BROSIS 0.2912 0.4543 Trial only New
Race/ethnicity (minority = 1) 2 RACE 0.1678 0.3737 Yes 1-5 Carried forward
Gender (male = 1) 2 GENDER 0.4677 0.4990 Yes 1-7 Carried forward
Socioeconomic status quintile 5 SESQUINT 3.6909 1.2840 Yes 1-7 Carried forward
Family income 6 FAMINC 4.0792 1.5641 Trial only New
Became parent by age 20 2 PARENT 0.0282 0.1655 Yes 1-7 Carried forward
Education anticipations (consistency
and level) 3 EDUANTIC 2.8747 0.4120 Yes 1-7 Modified
Highest high school math (prealgebra
to calculus) 5 HIGHMATH 3.0030 1.3560 Trial only Carried forward
High school science momentum 3 SCIMOM 2.1146 0.8837 Trial only New
High school foreign language units 5 FLAN 3.3662 1.1617 Trial only New
Advanced Placement 3 ADVANCED 1.2493 0.5833 Trial only New
High school curriculum intensity 5 HSCURRQ 3.7424 1.1860 Trial only New
Senior year test score quintile 5 SRTESTQ 3.8533 1.1752 Trial only New
High school class rank/GPA quintile 5 CLASSRNKQ 3.6272 1.2965 Trial only New
Academic Resources quintile 5 ACRES 3.6703 1.2130 Yes 1-7 Carried forward
No delay of postsecondary entry 2 NODELAY 0.9351 0.2463 Yes 2-7 Carried forward
Acceleration credits 3 ACCEL 1.4076 0.7546 Yes 2 New

See notes at end of figure.

23
 Figure 1. Values, means, standard deviations, and use of variables considered in the logistic narrative of Academic Momentum , with
universe confined to 1992 12th-graders with complete high school records and known socioeconomic status who attended a
four-year college at any time through December 2000

Programming Relationship to variables in

a
Variable Values name Mean S.D. Used Step# Answers in the Tool Box

First school was a four-year 2 FIRST4 0.7842 0.4114 No Carried forward

First school was doctoral 2 DOCT 0.3764 0.4845 No Carried forward
First school was selective 2 SELECT 0.2348 0.4239 Yes 1-7 Carried forward
Less than 20 credits in first year calendar year 2 LOWCRED 0.2045 0.4034 Yes 3-7 Carried forward
First calendar year grades 2 FRESHGRD 0.4320 0.4954 Yes 3-7 Carried forward
Any first calendar year remediation 2 FREM 0.2204 0.4146 Yes 3-6 New
Any first year college-level math 2 FCOLMTH 0.5294 0.4992 Yes 3-5 New
Grants/scholarships 2 GRANTS 0.4972 0.5000 No Carried forward
Loans 2 LOANS 0.3180 0.4658 No Carried forward
Work-Study/campus job 2 STUWORK 0.3016 0.4590 Yes 4 Carried forward
Attend multiple institutions 2 MULTINS 0.6180 0.4859 Yes 5-7 Modified
Community college transfer 2 TRANSFER 0.1377 0.3446 Yes 5-7 Carried forward
Four-year to four-year transfer 2 FOURTRAN 0.1476 0.3547 Yes 5-7 New
Summer-term credits 3 SUMMER 2.1577 0.9184 Yes 5-7 New
Ever part-time 2 EVERPT 0.3342 0.4717 Yes 5-7 New
Trend in GPA (rising, flat, falling) 3 TREND 2.1733 0.7311 Yes 6-7 Modified
All credits in college-level math 3 MATH 2.2305 0.8388 Yes 6-7 New
Changed major 2 CHANMAJ 0.4765 0.4995 No New
Continuous enrollment 2 CONTIN 0.7984 0.4012 Yes 7 Carried forward
Ratio of withdrawal/repeat grades to all
courses attempted 2 WRPRATIO 0.0905 0.2870 Yes 7 Carried forward
Remedial problem (type and amount of
remediation) 5 REMPROB 3.896 1.4730 No Carried forward
a
Standard deviation
b
A value of 2 indicates a dichotomous variable, i.e. yes (1) and no (0)
SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement)

24
[THIS PAGE INTENTIONALLY LEFT BLANK]

25
 Part III
What Is and What Happens Before Matriculation

There are at least three distinct ways of setting up Step 1 in the progression of multivariate analyses that lead us to appreciate what
makes a difference (and how much of a difference) in completing a bachelor’s degree for students who attended a four-year college at
any time. The first step covers both student demographics and high school performance. Let us cover the demographics first because
they do not present complex analytic choices.

What If We Knew Nothing Except Demography?

The demographic background of students is marked as of a set moment in time (in our case, in grade 12). It is what students look like,
where they are living, their parents’ education(s), occupation(s), and income(s), and other features of the student’s family.
Demographic characteristics may be subject to special attention in education policy from local to national levels, but, with the
exception of the student’s marital and parental status, are not subject to change.

Demographic variables are normally considered in the context of other aspects of student experience, behaviors, and attitudes when
attainment of any kind (e.g., high school graduation, test scores, grades, college degree) is the dependent variable. Indeed, that is the
way this analysis treats demographic characteristics. But to demonstrate what happens to the demographic variables in the analysis,
this section opens with a stark presentation. If someone asked us to explain bachelor’s degree completion for 1992 12th-graders who
subsequently attended a four-year college at any time, and all we knew about them were demographics, what would the various
associations of demography with degree attainment look like?

26
Table 3 presents a logistic exploration that started with nine demographic variables described in Part II above. The socioeconomic
status quintile variable is not included, but two of its components (family income and parents’ highest level of education) are present
to serve as proxies for SES. Two of the nine demographic variables—whether the student was a nonnative speaker of English and
family immigrant status—were not accepted into the logistic equation.

Whether there were nine variables or seven, however, this logistic model fails to reach significance as a model, regardless of whether
independent variables within the model turn out to be significant. The t value for the intercept falls far below the threshold of 0.765
this study uses for keeping independent variables under consideration, and the proportion of concordant probabilities predicted at 63
percent is the lowest of any logistic model in this study.

Of the independent variables within the model, four are significant, even though that significance is undercut by the statistical
characteristics of the model as a whole. The most significant
(p < 0.01) is first generation college status, with a Delta-p statistic that says the probability of completing a bachelor’s degree is
reduced by roughly 21 percent for first generation students.

Race/ethnicity and gender are significant at p <. 05, with the messages of the Delta-p that minority status reduces the probability of
earning a bachelor’s degree by 17 percent, and being male reduces that probability by 11 percent. Falling in the highest third of family
income is marginally significant.

Table 3. Logistic account of the relationships among major demographic variables and bachelor's
degree attainment for 1992 12th-graders who attended a four-year college at any time

Variable Parameter estimate Adjusted standard error t p Delta-p

Intercept 0.2250 0.1869 0.69
Race/ethnicity -0.7257 0.1434 2.90 0.05 -0.1660
Gender -0.4760 0.1072 2.54 0.05 -0.1089
Parenthood -1.6285 0.5255 1.77 † †

27
Siblings -0.3619 0.1109 1.87 † †
Family Income 0.3870 0.1111 2.00 0.10 0.0885
First generation -0.9137 0.1420 3.69 0.01 -0.2089
Urban high school 0.1839 0.1217 0.87 † †
† Variable did not reach the minimum level of significance (p<.10) in two-tailed test.
NOTES: Statistically significant variable are in bold. G2 = 6288.09; df =5144 G2.df =1.222; X2(df) = 413.76(7);
pseudo R2 = .077. Proportion of concordant probabilities predicted = 63.0. Root design effect = 1.75

SOURCE: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement).

But the most important message is the failure of the purely demographic analysis itself. If the statistical model does not reach a
convincing level of significance, and all the variables inside the model are basically cut from the same cloth, then even if some of
those variables are significant within the model, the cloth is frayed. That failure sets up a hypothesis that will unravel in the course of
subsequent steps of the logistic narrative: Among students who attended a four-year college at any time, student demographics and
family demographics may have, at best, indirect connections with degree completion. In fact, at the moment high school academic
history is included in the multivariate account, demography plays a considerably reduced role at the same time that the logistic model
becomes more persuasive.

High School Background and History

This is where the academic momentum story begins, where the major contributions of the original Tool Box were most prominent, and
where the story continues to be of considerable importance.

There were four core high school background and performance variables in the first step of the logistic inquiry in the original Tool
Box: senior year test score, a combined index of class rank and GPA, academic curriculum intensity, and the student’s education
“anticipations.” The first three of these variables were combined in a composite index of “Academic Resources” that proved to have
lasting positive repercussions through students’ postsecondary histories.

Before we examine the results of the Step 1 logistic model, we should (a) revisit education "anticipations," and (b) provide a detailed
account of how The Toolbox Revisited deals with high school curriculum.

"Anticipations" versus "aspirations"

28
Both the original Tool Box and this study employ a variable that some people will interpret as indicating students’ education
"aspirations." But this construct is carefully labeled "anticipations," and is not based on the student’s answer to a question asked only
once (for example, in a 12th-grade survey or in a freshman registration line). Rather, "anticipations" is built from sets of questions
asked in both the 10th grade and the 12th grade, and describes the consistency and level of the student’s abstract expectations and
concrete plans.

Perhaps as a by-product of the national visibility of discussions on secondary-to-postsecondary transitions that followed the release of
A Nation at Risk in 1983, along with a burst in recruitment of minority students by both colleges and professionally-oriented
organizations such as the National Action Committee for Minorities in Engineering, the proportion of 12th-graders who consistently
expected to earn at least a bachelor’s degree more than doubled from 22.5 percent (s.e. = 0.58) among 1982 12th-graders to 59.4
percent (s.e. = 1.02) among 1992 12th-graders.. Continuing one’s education after high school, at some point and "in some form of
postsecondary education" is now a norm of expectations among high school graduates: 92.6 percent (s.e. = 0.54) of NELS:88/2000
students who graduated from high school with any kind of diploma, whether "on-time" in 1992 or later, expected to continue their
education in a postsecondary setting. That which is a norm of behavior is like breathing in and breathing out. When virtually
everyone expects to continue, the serious curricular “disconnect” between K-12 and postsecondary systems eloquently dissected by
Venezia, Kirst, and Antonio (2003) is even more fraught with hazards and ironies. "Anticipations," one can surmise, will not play as
significant a role in a logistic account of bachelor’s degree completion as it did in the original Tool Box.13

To aspire is an active concept. In our ordinary usage, it means to hope, seek, wish, or yearn. When Kao and Tienda (1998) describe
"aspirations" as "a realistic evaluation of likely out-comes" (p. 352), they are not describing "aspirations" as we normally understand
the term. Qian and Blair (1999), too, indicate that they use the "aspirations" to describe expectations or plans "rather than a wish" (p.
622). And that’s just the point: With these rare exceptions, the literature

on education "aspirations" consistently confounds an emotively-laden term applied to motivation, desire, or drive with an assessment
of possibility based in experience (Mezias 1988). Underneath the consistency-by-level construct of the educational "anticipations"
variable used in both the original Tool Box and in this study is what Kao and Tienda (1998) document as a growth from abstract
attitudes about a very distant possibility when students are in junior high school to a rational judgment based on both school
experience and input from parents and peers as the time draws closer to high school graduation. There is no doubt that the
consistency-by-level formulation of our anticipations variable is influenced by what happens to students between grades 10 and 12.
13
Even there, the variable floated in and out of significance over the steps of the logistic model (see Adelman 1999, pp. 80–81).

29
A different interpretation of expectations is implicit in Venezia, Kirst, and Antonio (2003): Expectations become assumptions, more
passive than “aspirations.” Their principal thesis—that students falsely assume that the momentum inherent in a high school diploma
is sufficient not merely to enter a postsecondary environment ("the easy part," in their words) but to complete a credential—is all the
more powerful when these assumptions end in less than completion. The principal culprit lies in poorly articulated (secondary-to-
postsecondary) curricula (Conley 2005), a topic addressed in Part VI of this essay.

Academic Intensity of Secondary School Curriculum

Think of a series of moving walkways in an airport. Passengers expect that one leads to the next, and don’t think too much about
what’s happening along the way. What the most perceptive of contemporary analysts of the curriculum paths from high schools into
colleges of any kind have observed (Venezia, Kirst and Antonio; and Conley) might be described as a disconnect in both the location
and speed of these successive moving walkways, so that the passengers stepping mindlessly off the first and heading to the second
trip, stumble into the moving hand rail, and desperately grasp at their luggage so that it doesn’t get stuck in the space between the two
segments. This issue was imperfectly highlighted in the original Tool Box and is worth extended preliminary attention here.

The “differential course work” hypothesis, originating in the work of Pallas and Alexander (1983) and Alexander and Pallas (1984) in
relation to high school test scores, and validated by Answers in the Tool Box with a longer time frame and a bachelor’s degree
attainment dependent variable, remains the core of the presentation in The Toolbox Revisited as well. With these forebears as
guidance, we are going to look at the components of the curriculum variable, alternative presentations of those components for
multivariate analyses, and some critiques of the measure. In the past decade, a number of short-hand representations of the high
school curriculum, e.g., the New Basics as advocated in A Nation at Risk (1983), "minimum college-qualified (Berkner and Chavez
1997), and "track location" (Lucas 1999; Lucas and Berends 2002) have become common in both the research literature and the
general press. The New Basics are the best known of these heuristics. The highest level of these "new basics"

—which simply counted Carnegie units1 (four of English, three each of mathematics, science, and social studies, and two of foreign
language)—has been labeled "rigorous" (McCormick 1999), even though three units of mathematics could amount to no more than
plane geometry and the science in question could lack any laboratory components. After the publication of the original Tool Box, the

1
A Carnegie unit is the basic credit system for U.S. secondary schools. It is generally recognized as representing a full year (36–40 weeks) in a specific class
meeting four or five times per week for 40–50 minutes per class session (Martinez and Bray 2002).

30
literature added criteria for highest level of math, core laboratory science, and Advanced Placement course work to the judgment of
"rigor," and sought a trichotomous presentation with labels such as "core curriculum or less," "mid-level," and "rigorous" (Horn and
Kojaku 2001).

Quite frankly, the word "rigorous" is somewhat of a misnomer since a course requiring a high concentration of intellectual effort can
be presented in a relaxed manner with comparatively low standards for success. Put another way, calculus or laboratory chemistry, for
example, can be taught in a very laid-back fashion, while an otherwise "ordinary" survey of U.S. history can require the search for,
discovery, and cataloguing of original source material, readings in archival methods, and frequent examinations and project
presentations with criterion-referenced grading standards. Both the original Tool Box and The Toolbox Revisited prefer the term,
"academic intensity," defend the continuous variable of academic curriculum intensity as a more flexible language of accounting, and
decline to invoke labels of “college qualified,” “at risk,” and others.

The academic curriculum intensity variable is called HSCURRQ. It takes a weighted distribution of NELS:88/2000 students across 31
levels of academic curriculum intensity and quality (there were 40 levels in the records of the High School Class of 1982, the subjects
of the original Tool Box), and divides the distribution by quintile. Each level of academic curriculum intensity is a distinct
configuration of numbers of Carnegie units2 earned in core academic areas and other distinct notations about students’ course of study.
For example, in both data sets, at the highest level, one finds students who, between grades 9 and 12, accumulated.3

3.75 or more Carnegie units of English;

3.75 or more Carnegie units of mathematics;
highest mathematics of either calculus, precalculus, or trigonometry;
2.5 or more Carnegie units of science or more than 2.0 Carnegie units of core
laboratory science (biology, chemistry, and physics);
more than 2.0 Carnegie units of foreign languages;
more than 2.0 Carnegie units of history and/or social studies;
more than 1 Advanced Placement course; and
no remedial English; no remedial mathematics.

These are minimums. In fact, at the highest level of academic curricular intensity, students had accumulated an average of 4.30
2
It is important to note that the Carnegie unit thresholds indicated, e.g., 3.75 units of English, are not determined a priori, but rather from the empirical
distribution of Carnegie units standardized across the NELS high school transcripts in their edited version. See the descriptive windows for all the high school
curriculum variables on the NCES data sets in CD # 2003-402 and its June 2004 Supplement.
3
For mean numbers of Carnegie units earned in major curriculum areas, see Appendix L, table L6.

31
Carnegie units of English, 4.34 units of mathematics (94 percent having reached precalculus or calculus), 3.63 units of core laboratory
science, 3.76 units of foreign languages, and 3.79 units of history and other social studies, along with an average of three AP courses.
That is an impressive portfolio.

To this particular configuration, the NELS:88/2000 version of the highest level of academic curriculum intensity adds any Carnegie
units in "computer science," a far more frequent and integrated elective in the high school curriculum by 1990, when the NELS
students were in the 10th grade, as opposed to 1980, when the High School & Beyond/So students were in the 10th grade (the mass
availability and marketing of PCs began in 1984).

Each subsequent level in a descending logical cascade subtracts something from this configuration (see Appendix F) until a relatively
smooth distribution is achieved. Nothing is perfect in these algorithmic adjustments; and whatever lumpiness in distribution remains
is assuaged in the quintile version.

In light of recent literature on the effects of AP course-taking versus scores on AP examinations (e.g., Geiser and Santelices 2004;
Klopfenstein and Thomas 2005), it is important to note that in both the original Tool Box and in this study, AP course work is not
treated as a discrete variable, rather as part of the overall index of academic curricular intensity. However, in a moment we will
demonstrate what happens when AP is invoked as a distinct independent variable in alternative constructions of the first step of our
core logistic model.

The basic critique of this account of high school curriculum has much merit: Numbers of courses (expressed in Carnegie units) in
specific subject areas are not equivalent to high content standards or performance standards in those courses (Barth and Haycock
2004). In the cascading curriculum levels of the Academic Intensity variable, a student could accumulate three units of watered down
history and social studies and still have three units. A national transcript accounting has no way of knowing what is demanded in
thousands of history and social studies courses offered in 1100 high schools, a point made in the staff research for A Nation at Risk
(see Adelman 1983). The only domestic examples available for those who argue for test-driven high content standards in individual
high school subjects such as U.S. history or chemistry or
Algebra 2 are the College Board achievement tests (now called SAT II) and the New York State Regents. Some other states (e.g.,
Michigan) have tests in place for merit-based scholarships that cover broad areas of the curriculum (e.g., math, science) and language
skills (reading, writing) but not specific subjects. In this respect, they are closer to the ACT examination, though at a less demanding
level (Bishop 2004). The high school transcripts that serve as our evidence are very inconsistent in entering SAT II subject exam
scores, so we have nothing on which to rely for an external validation of content. Getting beyond course titles to content standards is
addressed in Part VI of this study.

Altonji’s work (1996) offers a second important critique. He would have us control high school curriculum variables for school

32
effects and "quality of courses," but admits that whatever one tries for proxy measures, the results would be "imperfect." Altonji
draws our attention to the fact that more than half the variance in the mean number of Carnegie units earned in academic subjects by
the NLS-72 cohort4 was across schools, indicating disparities in opportunity to learn, teacher quality, and student talents and
proclivities. One might put Altonji’s conclusion more baldly: Students from higher SES backgrounds attend high schools that serve
similar students, schools in which parental expectations and involvement are high, and schools in which curriculum offerings and
teacher quality follow SES demands and can afford to do so because the tax base is higher. These students tend to enter college and
earn degrees at higher rates than students from "lesser" backgrounds and from schools that accompany those backgrounds, therefore,
the influence of the academic intensity of high school curriculum will be overstated.

The implicit response of the original Tool Box made explicit in The Toolbox Revisited is that there are inequities in opportunity to
learn and proclivity to learn that are reflected in the geo-demography of schools, but in a logistic account of bachelor’s degree
attainment in which the results are reported with the Delta-p statistic, socioeconomic status will modulate some of the effects of
curriculum. The message encourages all high schools to offer the requisite curricula, to make sure they have teachers who can deliver
that curricula, to believe that their students can all reach higher levels of academic intensity in preparation, and to encourage their
students to do so—no matter what their intentions for subsequent education or work may be. As the principles of the No Child Left
Behind legislation are applied in high schools, this is a large part of what they mean.

Highest level of mathematics and changes in high school course-taking

In comparing the high school records of the HS&B/So (High School Class of 1982) with those of the NELS:88/200 (High School
Class of 1992), and looking only at earned Carnegie units, what we principally witness is an improvement of precollegiate preparation
of students in mathematics and science, a change that affects the amount of remediation necessary in four-year colleges, as well as
mathematics course-taking at the college-level (Adelman, Daniel, and Berkovits 2003). We also note an unsettling increase in the
percentage of students taking a maximum of one year of foreign language course work (though that percentage spread might be
mitigated somewhat if we accounted for the increase in bilingual heritage language users between the two cohorts). Table 4 sets forth
some basic threshold markers in these curricular areas.

4
The National Longitudinal Study of the High School Class of 1972, the first, most elaborate, and longest (1972–86) of the NCES grade-cohort longitudinal
studies

33
 Table 4. Percentage of 1982 and 1992 12th-graders in academic high school programs
whose high school curriculum fell below selected content and intensity
thresholds Class of 1982 Class of 1992

All 12th- In academic All 12th- In academic

Curriculum Makers graders program graders program

Percent of students with:

Highest level of mathematics 60.2 (0.78) 35.4 (1.02) 37.7 (1.11) 24.2 (1.24)
less than Algebra 2
Maximum of one year of core 60.1 (0.82) 37.1 (1.04) 34.3 (1.17) 21.0 (1.22)
laboratory science
Maximum of one year of foreign
language 28.3 (0.88) 14.3 (0.75) 41.5 (1.30) 28.4 (1.47)
NOTES: "Core laboratory science" is confined to biology, chemistry, and physics. Standard errors are in
parentheses. Weighted N for students in academic program: HS& B/So = 1.5 M; NELS:88/2000 = 1.8M.
SOURCES: High School & Beyond/Sophomore Cohort (NCES 2000-194) and NELS:88/2000 Postsecondary
Transcript Files (NCES 2003-402 and Supplement).

Thresholds are suggestive, but not conclusive, of the shape of change in curricular preparation.
By reverse logic, table 4 says only that X percent of students are getting beyond point Q in
secondary school study. It does not say how far beyond, and that may make more of a difference
in academic momentum. The mathematics analysis in the original Tool Box was a proxy for the
general metaphor of momentum, and it may be instructive to compare the results of those
calculations—a distribution and a logistic—to what emerges from the NELS:88/2000 involving
all the rungs on the “math ladder.” Tables 5, 6, and 7 do so, confining their universes only to
students who were in the 12th grade in the year they were scheduled to be in the 12th grade.
They do not restrict the universes to those who entered postsecondary education, let alone to
those attended four-year colleges at some time. These are very inclusive groups.

Table 5 confirms other observations in these data sets: The efforts following A Nation at Risk to
move students further ahead in high school mathematics had positive results. Higher proportions
of students were reaching Algebra 2 and higher levels. Bachelor’s degree attainment rates for
those who moved at least one step beyond Algebra 2 in high school were stable, but declined at
every level at or below Algebra 2. Table 5 provides a framework that needs to be validated in a
more convincing way, and that is the purpose of table 7. Table 7 presents the results of logistic
analyses and does so in odds ratios (departing from the presentation of other logistics in The
Toolbox Revisited) to match the way in which this table was presented in the original Tool Box
study.5

From the write-ups and presentations of the original Tool Box study a principle for high school
5
The High School Class of 1982 data in table 7 are slightly different from those in the previous presentation in
Answers in the Tool Box (table 6, page 17) because a more accurate 12th-grade flag for that data set has been
developed to match the NELS:88/2000.

34
curriculum guidance arose: "One step beyond Algebra 2 doubles the odds that you will earn a
bachelor’s degree." What does that mean, i.e., how does one read table 7? An odds ratio
indicates that every unit of change in an independent variable (in this case, each step up the math
ladder) increases the odds of X happening (in this case, earning a bachelor’s degree) by Y (the
odds ratio). For table 7, the data for the full model—using only highest mathematics and
socioeconomic status quintile—are presented first, in bold. In both cohorts, the odds ratio says
that every step up the math ladder multiplies the odds of earning a bachelor’s degree by roughly
2.5 versus an odds ratio for each step up the socioeconomic status quintile measure of 1.68
(for the class of 1982) and 1.78 (for the class of 1992). These data are statistically significant
(p <.05 or better).

Table 5. Bachelor's degree attainment rate by highest level of mathematics

reached in high school by 1982 and 1992 12th-graders

Class of 1982 Class of 1992

Percentage Percentage
reaching this Earned reaching this
Level of math level of math bachelor's level of math Earned bachelor's

Calculus 5.2 (0.36) 82.1 (2.45) 9.7 (2.72) 83.3 (2.72)

Precalculus 4.8 (0.37) 75.9 (2.43) 10.8 (0.65) 74.6 (2.04)
Trigonometry 9.3 (0.51) 64.7 (2.32) 12.1 (0.81) 60.0 (3.32)
Algebra 2 24.6 (0.75) 46.4 (1.54) 30.0 (1.08) 39.3 (2.31)
Geometry 16.3 (0.65) 31.0 (1.92) 14.2 (0.87) 16.7 (1.87)
Algebra 1 21.8 (0.69) 13.4 (1.33) 16.5 (0.92) 7.0 (1.24)
Pre-algebra 18.0 (0.66) 5.4 (1.19) 6.7 (0.53) 3.9 (1.34)
NOTES: Standard errors are in parentheses. The columns for level of math may not add 100.0 percent due to
rounding.
SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore Cohort (NCES 2000-
194) and NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and Supplement).

The balance of the table takes each level of high school mathematics, and runs a logistic
regression for that level, with bachelor’s degree completion as the dependent variable, and SES
quintile as the sole control. For the class of 1982, reaching calculus in high school increased the
odds of earning a bachelor’s degree by a very impressive 8.18 to 1. For the class of 1992, the
odds ratio for calculus was still in the same range at 7.52 to 1. The parameter estimates for the
calculus line are almost identical (2.102 versus 2.018). These are consistent results.

When one looks at the columns in table 7 for the class of 1982, one notices that the sign of the
parameter estimate moves from negative to positive territory between geometry and Algebra 2,
and the value of the parameter estimate rises above 1.0 between Algebra 2 and trigonometry.
The odds ratio more than doubles in each of those steps, but only in the step between Algebra 2
and trigonometry (the "one step beyond") is the parameter also positive. For the class of 1992,
all those relationships move up one rung on the "math ladder," principally because a higher

35
percentage of this group (than the percentage for the class of 1982) reached precalculus or
calculus while a lower percentage failed to get as far as Algebra 2. The critical boundary for
math momentum now lies firmly beyond Algebra 2.

But therein lies the rub, for not everyone has the chance to reach beyond Algebra 2. Differential
lack of opportunity-to-learn was a major theme in the original Tool Box and is just as prominent a
theme in this study of a cohort a decade later. Table 6 illustrates this unhappy situation. It asks
what proportion of students—by race/ethnicity and socioeconomic status quintile—attended high
schools that offered key math courses beyond Algebra 2.

If mathematics momentum is as important as we contend it to be, one can see the ripples of
opportunity—or lack of opportunity—that start in high school offerings. In the matter of
calculus, Latino students and those from any socioeconomic status quintile other than the highest
are at a distinct disadvantage with respect to the opportunity even to confront the subject.
African-American students join the other groups in less access to both trigonometry and
statistics. While this issue will be revisited, it deserves underscoring here: The task of providing
quality secondary school curricula to everybody, the paths to AP, the paths to the kind of learning
challenges students will face in higher education, is enormous. If the promise of No Child Left
Behind is to be realized at the secondary school level, it is first and foremost through the
equitable provision of opportunity-to-learn.

36
Table 6. Percentage of 1992 12th-graders who attended high schools that offered coursesa in
statistics, trigonometry, and calculus, by race/ethnicity and socioeconomic status
quintile
Percent attending high schools that offered:
Demographic group Calculus Trigonometry Statistics

By race/ethnicity

White 58.6 (1.67) 76.9 (1.29) 27.7 (1.62)

African American 50.8 (4.14) 67.0 (3.90) 19.5 (2.71)
Latino 44.6 (4.04) 59.9 (3.55) 18.2 (2.44)
Asian 61.3 (4.31) 71.9 (3.61) 30.1 (3.94)

By socioeconomic status quintile

Highest quintile 71.6 (1.93) 83.1 (1.64) 34.0 (2.30)

2nd quintile 56.2 (2.32) 73.2 (2.13) 27.1 (2.01)
3rd quintile 54.1 (2.39) 71.4 (2.33) 24.9 (1.92)
4th quintile 49.3 (2.46) 70.3 (2.28) 20.3 (1.80)
Lowest quintile 43.5 (2.86) 63.7 (2.66) 18.5 (2.06)
a
Responses ae based on surveys of school administrators and math teachers of NELS students in 1990. Where the
administrator did not answer the question, the math teachers did not indicate that they taught the subject, and students
did not earn any credits in the sbject, the calculation assumes that the school did not offer the subject. This approach
may understimate the percentage of high schools offering the subjects at issue.
NOTES: Standard errors are in parentheses.
SOURCES: National Center for Education Statistics: NELS:88/2000 (NCES 96-130), and NELS 88:2000
Postsecondary Transcript Files (NCES 2003-402).

37
Since “highest high school mathematics” is a critical sorting component in the construction of
the composite variable describing the academic intensity of a student’s secondary school
curriculum, the ramifications of more students completing courses beyond Algebra 2, it is
hypothesized, will ironically diminish (however modestly) the explanatory power of curriculum
and raise the power of academic performance (class rank/GPA). To illustrate with a metaphor: If
it was said that a student who reaches precalculus in high school and earns a C has a better
chance of finishing a bachelor’s degree than a student who gets only as far as Algebra 2 with an
A-, we now have to raise the bar for performance in precalculus to at least a B- in order for the
prediction to hold.

38
Table 7. The math ladder for 1982 and 1992 12th-graders: Odds ratios and parameter estimates of
earning a bachelor's degree at each rung, controlling for socioeconomic status quintile

1982 12th-graders 1992 12th-graders

Adjusted Adjusted
Odds Parameter standard Odds Parameter standard
Variables ratio estimates error p< ratio estimates error p<

Highest math 2.56 0.9383 .0426 .01 2.50 0.918 .0467 .01
Socioeconmic 1.68 0.5204 .0359 .02 1.78 0.571 .0449 .05
status quintile

By highest level
of mathematics:

Calculus 8.18 2.102 .0426 .01 7.52 2.018 .1936 .05

Precalculus 6.34 1.846 .1895 .05 4.91 1.591 .1592 .05

Trigonometry 3.76 1.352 .1267 .05 3.05 0.860 .1422 .10

Algebra 2 1.82 0.599 .0867 .05 0.89 -0.115 .1030 †

Geometry 0.63 -0.468 .1108 † 0.24 -1.434 .1812 .10

Algebra 1 0.19 -1.666 .1386 .02 0.07 -2.688 .2788 .10

Pre-Algebra 0.06 -2.778 .2614 .05 0.05 -3.088 .6684 †

†Variables did nto reach the minimum level of significance (p<.10) in a two-tailed test.
NOTES: For both 1982 and 1992 12th-graders, the universe consists of all students for whom highest mathematics in
high school could be determined, SES data were available, and highest postsecondary credential at the end of the cohort
study period was known. Root design effect for the high school school class of 1982 (HS&B/So cohort) = 1.59; root
design effect for the high school class of 1992 (NELS:88/2000 cohort)=1.98. NELS:88/2000 goodness -of-fit data for
the full model: G2 = 8289.25; df = 9135; G2/df=0.91; percent concordant probabilities predicted = 84.3. HS&B/So
cohort goodness-of-fit data for the full model: G2 = 8371.74; df = 9288; G2/df = 0.892; percent concordant
probabilites predicted = 80.0.
SOURCES: High School & Beyond/Sophomore Cohort (NCES 2000-194); NELS:88/2000 Postsecondary Transcript
Files (NCES 2003-402 and Supplement).

The general improvement in the high school curriculum profile of the NELS:88/2000 cohort is
reflected, at least among students who subsequently attended a four-year college at any time,
in diminished postsecondary remediation for that population. Table 8 sets out the distribution of
what in the original Tool Box study was called the “remedial problem,” a logical cascade that
distinguishes not only the amount of remediation a student takes in college, but also the kind of
remediation at issue. The logical cascade begins with remedial reading, then sorts out those
students whose only remediation consists of one or two courses in mathematics, then divides the
balance of the remedial population between those who required more than two remedial courses
and those who required only one. The type of remediation at issue for both these residual

39
populations is usually writing, but also includes courses devoted to basic study skills and general
developmental work. If the population included students who never attended a four-year
college, this distribution would look very different.6

Table 8. Percentage of 1982 and 1992 12th-graders who attended a four-year

college at any time, by type and amount of remedial courses taken in
postsecondary education

High school High school

class of 1982 class of 1992
Type and amount of remediation

Any remedial reading 10.0 (0.56) 7.8 (0.72)

Remedial math only (1-2 courses) 14.0 (0.65) 9.8 (0.65)
More than 2 remedial courses of
any kind other than reading 14.5 (0.72) 8.9 (0.64)
Only 1 remedial course other than
reading 9.0 (0.50) 6.5 (0.41)
No remedial courses 52.5 (1.04) 66.9 (1.07)
NOTES: (1) Standard errors are in parentheses. (2) Columns may not add to 100.0 percent due to
rounding.
SOURCES: High School & Beyond/Sophomore Cohort (NCES 200-194) and NELS:88/2000
Postsecondary Transcript Files (2003-402).

6
Among all NELS:88/2000 students who participated in the 1992 survey and subsequently attended only community
colleges, 64.5 percent (s.e. = 1.88) took at least one remedial course and 43.7 percent (s.e. = 1.98) took more than
one. Among those who attended only sub-baccalaureate trade schools, which often do not require remediation, 32.4
percent (s.e. = 5.89) took at least one remedial course and 21.9 percent (s.e. = 6.07) took more than one. There is a
considerable difference between the norms and requirements of community colleges versus those of other sub-
baccalaureate institutions in the matter of remediation, and we should not mix the two in analyses of remediation.

40
Summary of high school curriculum and performance variables

To remind the reader of our tools before heading into Step 1 of the logistic narrative: For the high
school curriculum components of the analysis we have a composite variable called HSCURRQ,
and four variables set forth in Part II above describing discrete components of HSCURRQ that
we can test as proxy measures for the composite—highest level of mathematics, science
momentum (a combination of highest math and core laboratory science credits), foreign
language credits, and number of Advanced Placement courses. In a moment all these will appear
in a correlation matrix that will provide a preview of how they might play out in multivariate
analyses.

For our other performance measures, we bring forward from Part II a quintile presentation of the
student’s high school class rank/GPA, and a quintile presentation of senior year test score.

Academic Resources

The original Tool Box advanced a notion explored and developed by Karl Alexander of Johns
Hopkins and his associates in numerous exemplary contributions to the research literature: A
student’s academic background was far more important than demographic variables such as
gender, race/ethnicity, family composition, and socioeconomic status in relation to test
performance (Alexander and Pallas 1984), entering higher education (Thomas, Alexander, and
Eckland 1979), and, in one study, degree completion (Alexander, Riordan, Fennessey and Pallas
1982). How does one reflect a composite idea of "academic background" of students coming out
of high school and into postsecondary education? The key measure of Academic Resources (or
ACRES, as the original Tool Box variable was called in order to elicit the idea of academic
cultivation), combined curricular record, academic performance (class rank/GPA) as an indicator
of student effort, and an external measure in the form of performance on tests of general learned
abilities. These three components, as we have seen, are set out in quintiles. To get a preliminary
idea of their relative strength in relation to bachelor’s degree completion, table 9 takes all on-
time high school graduates in 1992 and indicates the percentage who completed a bachelor’s
degree by December 2000 by quintile of each of the three components. To be included in table
9, the student’s record had to contain positive values for all three components.

Table 9 offers some hints of what multivariate analysis will confirm. The universe consists of all
on-time high school graduates, as opposed to 1992 12th-graders, in order to equalize the condi-
tions of judging degree-completion rates by performance quintiles. Compared to the distribu-
tions for academic curriculum intensity and class rank/GPA, the test score quintile variable yields
weaker degree completion rates in its highest quintile. Comparing second quintile distri-
butions, test scores are weaker than academic curriculum. These are signs that the test score
quintile will not have as strong an association with degree completion as the other two factors.
On the surface, it looks as if class rank/GPA will be fairly dependable, but it has one minor
bump. The reader is directed in table 9 to the percentage of eventual bachelor’s recipients from
the lowest quintile of class rank/GPA (13 percent) compared to the percent of bachelor’s degree

41
 completers in the lowest quintile of test scores and academic curriculum (about 9 percent in both
cases). The differences are statistically significant in a descriptive account such as that of table
9, but how meaningful they turn out to be must await a multivariate context. In the meantime,
one would posit that the class rank/GPA measure will not prove as strong as the curriculum
measure.

Table 9. Percentage of on-time 1992 high school graduates who continued their
education in any postsecondary institution who completed bachelor's degrees by
December 2000, by quintile performance in the three component variables of
Academic Resources

Percent completing bachelor's degree

Population Quintiles of Academic Resources components

Highest Second Third Fourth Lowest

All on time 1992 high

school graduates who
continued to
postsecondary education
at any time
Curriculum 81.7 (1.50) 60.5 (2.14) 35.5 (1.93) 23.4 (1.99) 8.7 (1.37)
Class rank/GPA 78.8 (1.47) 59.1 (2.24) 40.3 (2.11) 25.7 (2.16) 13.0 (1.94)
Senior test 74.9 (1.72) 533.9 (1.92) 37.2 (2.16) 26.7 (2.41) 8.9 (1.31)
NOTES: Standard errors are in parentheses. Weighted N = 1.6M
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and
Supplement).

A multivariate analysis involving these variables, in fact, confirms our suspicions, and its results
are presented in table 10. The model is the foundation for construction of the composite
variable, Academic Resources, and is derived from ratios of the standardized beta estimates of
the three high school performance indicators produced by a logistic regression with bachelor’s
degree attainment as the dependent variable and no controls (the logistic regression itself can be
found in Appendix L, table L7). The change in these ratios between those for the High School
Class of 1982 and those for the High School Class of 1992, while still leaving academic
curriculum with the highest weight, narrows the gap between curriculum and performance, and
lowers the strength of test scores as a component of Academic Resources. Given the differences
in overall participation in the variable, i.e., the proportion of 12th-graders for whom all three
components have positive values, some of the change may be an artifice of the NELS:88/2000
data set. But the principal message is one observed when commenting on the change in the
distribution of students by highest level of mathematics studied in high school: The higher the
average level of mathematics attainment, the more student effort, reflected in grades, will count.

42
This theme will continue to play out when our story crosses the matriculation line into
postsecondary education; and it was one of the principal points made by DesJardins, McCall,
Ahlburg and Moye (2002) in their critique of the original Tool Box study.

When using all three measures—curriculum, class rank, and ACT test score—in a traditional
linear analysis predicting first-year college grades, Pike and Saupe (2002) found curriculum to be
the strongest, though their curriculum variable was dichotomous (students either meeting or not
meeting the entrance specifications for a selective state university), which would naturally
increase its power. Even when they introduced indirect effects of high school characteristics —
control (private/public), size, mean ACT score of students, and mean proportion of students who
attended the university—the parameter estimates for this curriculum variable were significantly
greater than those for test scores or class rank. While The Toolbox Revisited is not in the
business of predicting first-year grades, it is gratifying to note research that has similar respect
for the propulsive power of course of study.

Table 10. Component weights of the high school Academic Resources variable for 1982
12th-graders and 1992 12th-graders who presented positive values for all
three components

Components of Academic Resources 1982 12th-graders 1992 12th-graders

Academic curriculum intensity .41 .42

Class rank/GPA .30 .33
Senior Test Score .29 .25
NOTE: Columns will add to 1.00.
SOURCES: High School & Beyond/sophomore cohort (NCES 2000-194), and NELS:88/2000 Postsecondary
Transcript files (NCES 2003-402 and Supplement).

A second message bears repeating from its formulation in the original Tool Box: Student effort in
curriculum participation and performance over the three years of high school (grades 10 to 12)
reflected in the transcript data is worth considerably more than performance on a three-hour test
on a Saturday morning. Test scores are a natural consequence of the academic intensity of
curriculum and quality of student effort reflected in grades, and the weakening position of the
test score variable in the ACRES configuration for the NELS:88/2000 cohort is a natural
outcome of improvements in academic curriculum participation of the post-A Nation at Risk era.

Correlations

Now that we have all the precollegiate variables explained and in place, the process of bringing
zoom and macro lenses of multivariate analysis to bear begins. The first stage sets forth a
correlation matrix of precollegiate academic variables including three outcomes: on-time high
school graduation, basic postsecondary “access,” and bachelor’s degree completion. Table 11
presents the Pearson’s r results. Some of these correlations are weak, suggesting that one or both
of the variables in question will not add to the explanatory power of a logistic regression. On-
time high school graduation, for the most noted example, is not related to anything (this was also

43
true in the parallel correlation matrix in the original Tool Box). Other relationships are not
surprising, e.g., Advanced Placement has almost no bearing on entering postsecondary education
(96.7 percent [s.e. = 1.48] of NELS 12th-graders with any AP courses entered the postsecondary
sphere, versus 75.9 percent [s.e. = 1.01] of those with no AP course work).

Table 11. Correlations of major precollege Academic Resources variables and high school graduation
status, college entry, and bachelor's degree attainment by December 2000 for 1992 12th-
graders

Class On-time
Curriculum Highest Science Foreign rank / Senior HS grad, Post-
Intensity AP math (5th momentum Language GPA test standard secondary
Quintile Courses levels) (3 levels) (3 levels) quintile quintile diploma entry
(CURRQ) (APCRS) (HMATH) (SCIMOM) (FLAN) (RANK) (TEST) (ONTIME) (PSENT)
CURRQ ---- 0.368 0.777 0.774 0.645 0.572 0.581 0.167* 0.352
APCRS ---- 0.438 0.366 0.270 0.326 0.317 0.050* 0.111*
HMATH ---- 0.869 0.519 0.618 0.634 0.124* 0.301
SCIMOM ---- 0.489 0.590 0.578 0.118* 0.279
FLAN ---- 0.430 0.510 0.146* 0.338
RANK ---- 0.566 0.185* 0.298
TEST ---- 0.138* 0.313
ONTIME ---- 0.168*
PSENT ----
BACHELOR'S 0.524 0.319 0.538 0.530 0.451 0.493 0.469 0.129* 0.332
NOTE: All estimates except those noted with an asterisk are statistically significant at p<..05 or better.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

True to its position in the construction of the composite academic curriculum intensity variable
(abbreviated here as CURRQ), in fact, Advanced Placement does not exhibit as strong a
correlation with the composite as do the mathematics, science momentum, and foreign language
components. Why? Because it is invoked as a sorting criterion in only six of those 31 levels of
academic curriculum intensity (see Appendix F), while highest mathematics is invoked in 25
levels, foreign languages in 18, and core laboratory science or all science in 30. Common sense
says that we will find higher correlations of these other components with the composite
curriculum variable.

But among those other components, the correlation between the highest level of mathematics and
science momentum (which includes highest math in its definition) is so strong (0.869) as to set
off collinearity bells. The clear message is to use only one of them in any multivariate analysis.
The highest level of mathematics was dropped because the variable represents only one
curricular area, whereas SCIMOM carries two. Likewise, following Pedhazur’s (1982) rules of
thumb for identifying potential collinearity problems from correlations, table 11 advises that if
we use the high school academic curriculum index, we should not invoke either highest level of
math or science momentum in the same multivariate model.

44
The only other observations of the correlation matrix of table 11 worth special attention are:

• When bachelor’s degree completion is the reference point, the relative

correlations of the curriculum, class rank/GPA, and test score variables are in
roughly the same relationship observed in the more complex construction of the
composite Academic Resources variable: Curriculum exhibits the strongest
correlation, followed by class rank/GPA, followed by test score.
• However, when the referent variable is simply entering postsecondary education,
(a) the correlation coefficients are much lower, and (b) the test score variable is
slightly stronger than class rank/GPA.

The bottom line of these two conclusions: What counts for completion will be more potent than
what counts for mere entrance (the "easy part," in the words of Venezia, Kirst, and Antonio,
2003).
Step 1: The First Logistics

In the commentaries and critiques of these aspects of the original Tool Box, two potent questions
were raised:

1) Is a composite variable such as Academic Resources, a variable that yokes together

very different types of measures (a basically qualitative curriculum index, class rank and
GPA, test score), as convincing as these measures taken separately in any multivariate
account of degree completion (DesJardins, McCall, Ahlburg, and Moye 2002)?

2) Unlike class rank/GPA and senior test score, the academic curriculum intensity
variable is built from components, some of which are high profile in their own right, e.g.,
Advanced Placement and highest level of mathematics reached in high school. If we
substitute the major components for the composite in an account of degree completion,
will they be as convincing?

In light of these questions, three distinct ways of setting up the first step in the progression of
logistic analyses that lead us to appreciate what makes a difference (and how much of a
difference) in completing a bachelor’s degree for students who attended a four-year college at
any time were explored. The first step covers both student demographics and high school
performance.

What happens to all those demographic variables in the "demography only" logistic of table 3
when these three competing approaches were tried out? With the exception of gender,
race/ethnicity, and becoming a parent by age 20, the minute the high school performance
variables enter, the demographic variables disappear. Taken individually, none of them—first
postsecondary generation status, second language dominant, recent immigrant status, family
income, number of siblings, and urbanicity of community in which the student’s high school was
located—meet the statistical criteria for either entering or staying in the logistic models
employed in this study. But two of these variables, family income and level of parents’
education, play dominant roles in the construct of socioeconomic status, which itself more than
qualifies for the logistic treatment.

45
Repetition of a special note on the components of the high school academic curriculum intensity
variable is necessary in the context of this first step in the logistic narrative. With the exception
of Klopfenstein and Thomas (2005), a spate of recent reports and commentaries on the Advanced
Placement program (e.g., College Board 2005) claim that the original Tool Box demonstrated the
unique power of AP course work in explaining bachelor’s degree completion. To put it gently,
this is a misreading. AP is only a component of the academic curriculum intensity in both the
original Tool Box and this replication. But to test the misreadings, AP was tried out as one of
three variables that might serve as proxies for curriculum intensity. The other two variables
tested as substitutes were a measure of momentum in science (SCIMOM) which includes
elements of HIGHMATH and Carnegie units earned in both core laboratory science and all
science courses, and number of Carnegie units earned in foreign languages.

So, three distinct approaches to logistic regression models were tried out for Step 1. The first,
table 12, keeps the composite variable Academic Resources, and includes nothing else. The
second divided Academic Resources into its three components. And the third dropped the high
school academic curriculum intensity variable and replaced it with its three proxy components:
science momentum, foreign language study, and AP.

Which one of these three versions should be carried forward in the stepwise analysis? In the first
version (table 12), maintaining the composite of Academic Resources, only Academic Resources
and socioeconomic status quintile are statistically significant (though with a p value of <0.10,
SES is a barely influential presence) The Delta-p statistic says that each step up the five-quintile
ladder of Academic Resources improves the probability of degree completion by about 15
percent. The Delta-p applied to socioeconomic status quintile says that for each step the
probability of degree completion increases by about 7 percent. Expectations do not make a
difference, and the demographic variables, while all carrying negative parameter estimates, do
not meet the criteria for statistical significance either. Message: At this stage, and in this
formulation, the composite representation of student learning in high school, Academic
Resources, is a very persuasive construct.

The second and third versions of the Step 1 logistic can be found in Appendix G, and both are far
less convincing. In the second version, Academic Resources was divided into its three
components: curriculum, performance (class rank/GPA), and test score. The test score
component, with a t statistic of 0.35 (far below the threshold requirement of 0.765), disqualifies
this version from the logistic narrative. This result also reinforces our previous observation that
when both academic curriculum participation and the quality of student effort within that
curriculum (grades) improve, test scores will automatically rise and paradoxically become less
important with reference to the dependent variable of degree completion.

In the third version of Step 1, of the three proxy variables for high school academic curriculum
intensity, only science momentum (that combination of highest level of math and number of
Carnegie units in core laboratory science) emerged as statistically significant. Neither Advanced
Placement nor foreign language study reached the threshold level of significance, and, with a t
statistic of 0.75, foreign language study was on the borderline of disqualification in terms of
further consideration. The senior test score quintile variable did not even meet the minimum
statistical criterion for entrance. Had we taken this version forward into subsequent stages of
student history we would have only one proxy for high school curriculum (science momentum),
and no traces of the external measurement of general learned abilities that the senior year test

46
score quintile conveys. At least the composite index of high school performance, Academic
Resources, includes a trace of external assessment.

Table 12. Logistic account of factors associated with earning a bachelor's degree in the
history of 1992 12th-graders who attended a four-year college at any time---
demographic and high school background, version 1: Using the composite
variable for high school Academic Resources

Adujsted
Parameter standard
Variable estimate error t p Delta-p
Intercept -4.2762 0.6360 3.10 0.05
Academic resources quintile 0.6439 0.0662 4.48 0.01 0.14924
Socioeconomic status quintile 0.2912 0.0621 2.16 0.10 0.06749
Education expectations 0.6272 0.2065 1.40 † †
Race/ethnicity -0.4093 0.1941 0.97 † †
Gender -0.4633 0.1485 1.44 † †
Parenthood -1.5757 0.4790 1.51 † †
† Variables did not meet threshold criterion for statistical significance.
NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root
2 2 2 2
design effect = 2.17. G = 5315.44; df = 4919; G /df =1.081; X (df ) = 1074.9 (6); pseudo R = 0.204;
percent concordant predicted probabilities = 77.5.
SOURCE: National Center for Education Statistic: NELS:88/2000 Postsecondary Transcript Files
(NCES 2003-402 and Supplement).

Summary decision: The composite variable, Academic Resources, stays in the logistic
narrative; other versions of representing high school performance fail.

47
THIS PAGE INTENTIONALLY LEFT BLANK

Part IV

48
 Matriculation and Beyond: The Features of Postsecondary History

On reflection, the original Tool Box made a mistake in the order and boundaries of the blocks of
variables entered in its analysis. Following high school background, and tracking students who
ultimately attended a four-year college at any time, it jumped first to financial aid, for which the
High School & Beyond/Sophomore data set contained detailed information (including
unobtrusive Pell Grant files) covering the first four years of a student’s postsecondary history. It
then considered a configuration of attendance patterns that covered everything from the timing of
entry to postsecondary education to extended aspects of attendance such as whether the student
ever left his or her first institution of enrollment and never returned to that institution. Thirdly, it
followed this mixture of time periods with a block of first-year performance indicators. The
order and content of these three blocks of variables scrambled time and student history.

The Toolbox Revisited unscrambles that sequence so that momentum can be more clearly
observed. We begin with the transition from secondary school to postsecondary matriculation,
and focus on timing of entry and type of first institution. We then move on to first-year
performance, the importance of which is fully justified by the research literature. Only after
accounting for the first year do we confront financial aid (which, in the NELS:88/2000 data set,
covers only the first two years of post-high school history, and is the weakest section of the
study, by far).1 Longer-term attendance patterns and extended performance (curriculum and
trends in grades) come later, as they should.

In shaping the universe for analysis so that a full history is visible, one minor change to the
boundaries will be exercised. The NELS:88/2000 cohort was scheduled to graduate from high
school in the spring of 1992. Not all of them did, so the upper boundary for high school
graduation will now be set at December 1996. Very few cases are affected,2 but, as the NELS
history ends in December 2000, the point of changing the upper boundary is to allow enough
time for late high school graduates to build a postsecondary history.

In entering this sequence, the reader should note that the master program generating the mass of
student-level variables included on the restricted NCES data sets of transcript-based files for
both the HS& B/So cohort and the NELS:88/2000 was basically the same, hence, there were no

algorithmic differences that would account for change in student behaviors such as attendance
patterns and academic performance. There are minor exceptions, and these will be noted in the

1
Financial aid comes after first-year performance because the story line is more interested in the potential
relationship of aid to completion among those who are already enrolled. If the topic was initial access, then the
financial aid package for the first year would be part of the postsecondary entry configuration of variables. The
NELS:88/2000 unfortunately does not allow us to distinguish modes of financing postsecondary education in the
first year from those of the second year.
2
Those who earned a high school diploma of any kind after December 1996 constituted 2.7 percent of all 1992
12th-graders (s.e. = 0.16) and 0.7 percent of all 1992 12th-graders who subsequently attended a four-year college
(s.e.= 0.16).

49
narrative.

Step 2: Matriculation

The research literature has long held that where one starts out in higher education has enormous
consequences, particularly with reference to completing degrees (Velez 1985). The vast majority
(78.7 percent; s.e. = 1.15) of the universe under study here started out in a four-year college. Do
any characteristics of that first institution stand out in multivariate analysis?

Only selectivity of the first institution was admitted to the Step 2 logistic model (the fact that the
institution was a four-year college was not admitted). To be sure, there are other characteristics
of the first institution of attendance. But if the dichotomous “selective” variable turns out to
have but modest, if any, significance in the account, it is unlikely that other stock institutional
characteristics—size, control, residential/commuter ratio—will have any influence, either. The
best of the institutional characteristics variables in the literature is probably Stoecker, Pascarella
and Wolfle’s (1988) “size.” This is not a simple measure, rather a factorial scale that includes
total enrollment, student/faculty ratio, and public control. However attractive the concept,
institutional size rarely breaks through as a stand-alone factor in literature with large national
samples because of student taste—some like it large, some like it small—and taste is too variant.
All this does not mean that where one starts out is irrelevant to completion, rather it directs our
attention to other features of student academic history. More to the point, with 64.8 percent (s.e.
= 1.06) of NELS:88/2000 students who attended a four-year college at some time attending more
than one institution, and 26 percent (s.e. = 1.01) attending more than two, the task of ascribing
influence to institutional characteristics is daunting (for a discussion of this issue, see pp. 81–84).

In addition to the selectivity variable (SELECT), Step 2 includes NODELAY, a marker for direct
entry to postsecondary education following high school graduation. It also includes a new
variable, ACCELCRD, made possible by the construction of the NELS:88/2000 postsecondary
transcript file to reflect the practice of dual-enrollment that expanded during the period the High
School Class of 1992 was attending high school and that has become much more visible since
then. ACCELCRD sums all college credits earned by course work prior to high school
graduation, along with credits earned by examination—including AP, the College Level
Examination Program (CLEP), and institutional challenge exams. Most of these credits were
earned either prior to matriculation or during the first term of enrollment, though some were
earned at later points in the student’s undergraduate career. A previous brief analysis of this
phenomenon (Adelman 2004a, pp. 55–56) suggested that acceleration might have a bearing on
degree completion since the descriptive data indicated a positive relationship between the
number of "acceleration" credits earned and both (a) high school academic curriculum intensity
quintile and (b) selectivity of the first institution attended.

50
Table 13 sets forth the relationships of the variables in play at the postsecondary matriculation
stage to bachelor’s degree completion for students who attended a four-year college at any time.
Academic Resources is still in a commanding position, and the Delta-p statistic indicates that
with each step up the quintiles of Academic Resources the probability of completion increases by
12.8 percent. Socioeconomic status quintile is still significant, though again, marginally so. Of
all the new variables, no delay of entry alone is statistically significant, and its Delta-p says that
students who enter college directly from high school increase the probability of bachelor’s
degree attainment by 21.2 percent, a very persuasive marker.

Table 13. Logistic account of factors associated with earning a bachelor's degress in the
history of 1992 12th-graders who attended a four-year college at any time:
Postsecondary entry phase
Adujsted
Parameter standard
Variable estimate error t p Delta-p
Intercept -4.2124 0.6588 2.02 0.01
Academic resources quintile 0.5541 0.0715 3.45 0.01 0.1283
Socioeconomic status quintile 0.2859 0.0643 2.03 0.10 0.0662
Education expectations 0.3462 0.2032 0.78 † †
No delay of entry 0.9161 0.2224 1.88 0.10 0.2121
Selectivity of first institution 0.4470 0.2301 0.89 † †
Acceleration credits 0.1904 0.1196 0.73 † †
Race -0.4709 0.2130 1.01 † †
Gender -0.4627 0.1540 1.37 † †
Parenthood -0.9639 0.4597 0.96 † †
†
Variables did not meet threshold criterion for statistical significance
NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect =
2.19. G2 = 5060.17; df = 4913; G2/df = 1.030; X2 (df) = 1101.0 (9); pseudo R2 = 0.2127; percent concordant
predicted probabilities = 78.5.
SOURCE: National Center for Eduction Statistics: NELS:88/2000 Postsecondary Transcript files (NCES 2003-
402 and Supplement).

The selectivity of the first institution of attendance, while yielding a positive parameter estimate,
does not reach the threshold of significance, and the t value for acceleration credits, at 0.73, falls
just below the threshold for retention in the overall statistical model. The case of acceleration
credits is one for which the author hoped for a better outcome, but once a rule is set, it is
observed: The variable is dropped from subsequent steps. The demographic variables are
retained, but don’t tell us much. And education expectations barely stays under consideration
with a t value of 0.78. Beattie’s human capital analysis (2002) downplayed education
expectations as a central feature of explaining outcomes, particularly in consideration of group
differences. It turns out that, even with a more sophisticated variable (our “anticipations”) than
the customary way of marking “aspirations,” Beattie is right.

51
 Step 3: First-Year Performance

Now that we have our universe of 1992 12th-graders in college, no matter where they started out,
the most prominent initial marker of progress is a configuration of first-year academic
performance indicators—and by "first year" is meant the first calendar year following the month
in which the student first enrolled in their "true" first postsecondary institution after high school
graduation.1

Most research on the critical first year of postsecondary education takes as its subjects students
who start out in the fall term, and, by custom-and-usage, measures of retention (whether term-to-
term or year-to-year) begin with fall enrollments. The transcript data, however, teach us that not
all students commence postsecondary study in the fall term, and that there is a modest bias in
drawing a universe based on fall term beginners only. For all 1992 12th-graders who entered
postsecondary education at any time, 82.1 percent (s.e. = 0.83) started in the fall term, 5.8
percent (s.e. = 0.46) started in the summer term, and 12.1 percent (s.e. = 0.71) began in the
winter or spring terms (depending on local academic calendars). While the full data on first term
of entrance are presented in Appendix L, tables L8a and L8b, it is important to note here that
Latino and African-American students are less likely than white students to commence
postsecondary study in the fall term, and the same is true when students from lower
socioeconomic status quintiles are compared with students from the highest socioeconomic
status quintile. Any measure of retention or completion that confines its universe to students
who began their postsecondary careers in the fall term is, to put it gently, grossly incomplete.

The cumulative number of variables qualifying for the logistic narrative increases by 50 percent
when our field of vision opens out to first-year performance. The logistic model of the original
Tool Box included three first-year performance variables, two of which were major contributors
to understanding degree completion, and are carried forward: a marker for earning less than 20
credits in the first calendar year and a tag for students whose first calendar year GPA fell in the
top 40 percent of the GPA distribution for that period. The Toolbox Revisited adds dichotomous
variables indicating whether the student took any remedial courses in the first calendar year, and
whether the student earned any credits in college-level mathematics during that period. The
mathematics variable, identified as potentially potent by Astin and Astin (1993), represents a
major departure in the way The Toolbox Revisited approaches its subject, for it introduces the
potential association of postsecondary curriculum with degree completion (the original Tool Box
did not raise this issue).

The first calendar year of attendance is the year in which students’ preparation for postsecondary
education is most sorely tested (Pascarella and Terenzini 1991), and in which both remediation
and college-level mathematics study serve as indicators of that preparation. The lower division
curriculum sets out course "gateways" through which students must pass in both universities
1
The “true” first institution of attendance excludes (1) colleges and community colleges in which the student was
enrolled prior to high school graduation; (2) institutions in which the student was enrolled during the summer
immediately following high school graduation and prior to fall term postsecondary entry (unless the institution was
the same in both periods); and (3) “false starts,” that is, cases in which the student enrolled, but then withdrew
during the first term of attendance, only to enroll and complete course work in a different institution at a later point
in time (in these cases, the second institution is the “true first institution”). The true first date of attendance is the
first month of enrollment at the true first institution.

52
(Hanson 1998) and community colleges (Boughan 2001). Every school can identify its list from
high enrollment courses2 and there is no question that most of these gateways require a direct
connection to the content of high school curricula that define "readiness." Remediation stalls
student momentum toward those gateways; college-level mathematics itself is a gateway.

In a different approach, American College Testing (2004) defines “college readiness” as a

remediation-free postsecondary experience, and sets its benchmarks for “readiness” as

. . . The level of achievement [based on ACT test scores] for students to have a
high probability of success (a 75 percent chance of earning a course grade of C
or better, a 50 percent chance of earning a B or better) in such credit-bearing
college courses as English Composition, [College] Algebra, and Biology. . . (p. 1).

With all due respect to ACT, earning a C—or even a B—in college algebra is not why most
students go to school. There are better ways to benchmark, starting with the Academic
Resources index of total high school performance in which curriculum (including highest level
of mathematics reached in secondary school) dominates and test scores do not. Instead of a
grade in a single course, we then acknowledge the proportion of students who completed credits
in any of five core college-level math courses during the first year of attendance, and, among
those who became non-incidental students (i.e., earned at least an adjusted semester’s worth of
credits), the proportion who ever completed credits in those courses. In table 14, one notes right
away that among those who start in community colleges and complete college-level math
courses in the first year, college algebra is the dominant math course. Among those who start in
four-year colleges, it is not the dominant math course, even though it ranks at the top in terms of
student participation.3

Ironically, the ACT study helps explain why curriculum quality emerges as more influential than
test scores in building the Academic Resources composite, and justifies continuing to track
curriculum (in this case, using mathematics momentum) across the matriculation line. The test
scores are dependent variables that respond directly to increases in the concentration and level of
study in each of the major secondary school content areas (see, e.g., Florida Department of
Education 2005). The ACT study sees value-added in each step up the math curriculum ladder in
terms of scores on the ACT math test. So when the dependent variable is degree completion, it is
this curriculum—more than the test score—that sets the value-added tone.

2
Hanson’s (1998) list from the University of Texas at Austin ranges from Ecology, Evolution, and Society, to
Introduction to U.S. and Texas Government, to General Psychology, to Calculus I and II for Science and
Engineering Students.
3
For more discrete details on credits earned in college-level mathematics, the reader is directed to table
L9, Appendix L.

53
Table 14. Percentage of 1992 12th-graders who earned credits in five categories
of college-level mathematics courses in the first calendar year of
attendance, and among non-incidental students,a the percentage who
ever earned credits in those five categories, by type of institution first
attended

Percent completing Among all students earning

College-level during first calendar year more than 10 credits, percent
math category of attendance, starting in: who ever earned credits in

Community
4-year colleges colleges

College algebra 21.7 (1.01) 17.0 (1.19) 28.2 (0.92)

Precalculus 18.6 (0.85) 4.9 (0.77) 17.4 (0.69)
Calculus 17.6 (0.96) 1.6 (0.26) 16.8 (0.74)
Finite mathematics 4.6 (0.54) 0.9 (0.30) 6.4 (0.46)
Statistics, probability 5.3 (0.42) 2.3 (0.58) 19.6 (0.82)
a
A non-incidental student is defined as one who earned more than 10 undergraduate credits
NOTES: Standard errors are in parentheses. Weighted Ns: four-year college beginners - 1.14M; community
college beginners = 789k; all who earned more than 10 credits = 1.84M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

But does that mean that curriculum continues to have the same influence in a postsecondary
setting? The issue comes up twice in the sequence of logistic models, first in table 15, which
presents the results of the Step 3 logistic: first-year performance variables. What do we see?

• Earning less than 20 credits in the first calendar year following postsecondary
entry is a distinct drag on degree completion. The Delta-p says that falling below
the 20-credit threshold lessens the probability of completing a bachelor’s degree
by a third!

• First-year grades, a proxy for both student effort and acclimation to the academic
demands of a new environment (Pascarella and Terenzini 1991), move in the
opposite direction from low credits: If one’s first-year GPA falls in the top two
quintiles, the probability of earning a degree increases by nearly 22 percent.

54
Table 15. Logistic account of factors associated with earning a bachelor's degree in 
the history of 1992 12th­graders who attended a four­year college at any 
time:  First postsecondary year performance

Adujsted 
Parameter  standard 
Variable estimate error t p Delta­p
Intercept ­3.5834 0.6054 3.33 0.01
Academic Resources quintile 0.3419 0.0699 2.75 0.01 0.0754
Socioeconomic status quintile 0.2879 0.0569 2.84 0.01 0.0635
Education expectations 0.4040 0.1794 1.27 † †
Selectivity of first institution 0.4059 0.1979 1.15 † †
No delay of entry 0.8153 0.2779 1.65 † †
Low credit in first year ­1.5299 0.1669 5.15 0.001 ­0.3372
First­year grades 0.9919 0.1541 3.62 0.01 0.2186
College­level math in first year 0.3603 0.1479 1.37 † †
Any first­year remediation 0.4963 0.1722 1.62 † †
Race ­0.3471 0.1906 1.02 † †
Gender ­0.3414 0.1372 1.40 † †
Parenthood ­1.0277 0.3965 1.46 † †
† Variables did not meet threshold criterion for statistical significance
NOTES:  Statistically significant variable are highlighted in bold.  Standard errors adjusted by root design 
2 2 2 2
effect = 1.78.  G  = 4411.67; df  = 4764; G /df  = 0.926; X  (df) = 1516.37(12); pseudo R  = 0.2893; percent 
concordant predicted probabilities = 83.3.
SOURCE; National Center for Education Statistics; NELS:88/2000 Postsecondary Transcript Files (NCES 
2003­402 and Supplement).

• Academic Resources and socioeconomic status are the only other statistically
significant variables. The Delta-p story for socioeconomic status quintile remains
at roughly the same level: Each step up the SES ladder increases the probability of
degree completion for this population by 6 percent or so. The Delta-p strength for
Academic Resources, however, falls from 12.8 percent in the postsecondary entry
model to 7.5 percent here, a natural consequence of the introduction of other
competing curricular and grade-based variables.

• The two new curricular variables for remediation and college-level mathematics
produce t statistics that justify carrying them forward, but do not reach even a
threshold statistical significance level of p<0.10.

55
In the last observation there is a surprise that calls out for explanation. The parameter estimate
for remediation in the first year is positive, not negative as conventional wisdom would assume.
Students in this group who took any remedial courses in the first year earned bachelor’s degrees
at a 48.7 percent rate (s.e. = 2.70) compared to 69.9 percent (s.e. = 1.26) of those who did not
take remedial courses in the first year (not in table). That is a significant and meaningful
difference from which, one might assume, remediation would be a negative. But there are other
variables in this model that determine the sign and strength of the parameter estimate and its
statistical significance. In a study of the academic careers of traditional-age community college
students, the author found that remediation (one dichotomous variable marking any remedial
reading, and one multi-level variable based on type and number of remedial courses) did not
affect either transfer or (for students who did not transfer) completion of an associate degree
(Adelman 2005a). Other recent research, using a very sophisticated targeting of students whose
need for remediation may differ according to the school they enter, goes beyond the finding that
remediation is not a drag on degree completion to demonstrate that, in terms of persistence,
remediation yields decidedly positive results (Bettinger and Long 2005). Two studies do not
make for a definitive conclusion, but the evidence that students who successfully pass through
remedial course work gain momentum toward degrees is beginning to build.

As for the other variables in the Step 3 model, we note that NODELAY drops out of the
statistically significant range with an indirect message of “once a student is in college, their
performance counts more than when they arrived.” In other words, the ramifications of delayed
entry can be overcome, but only with the kind of considerable effort reflected in first-year credit
accumulation and first-year grades. The reader will note of table 16, however, that when
financial aid variables are part of the logistic model, NODELAY rises back above the
significance threshold.
Step 4. Adding Financial Aid to the Equation
There is a major difference between the High School & Beyond/So and NELS:88/2000 data sets
in the nature and temporal coverage of financial aid information. For the HS&B/So, an
unobtrusive Pell Grant file provided data on at least one type of grant-in-aid. Student reports
covered four years (1982–86) of other types of grants-in-aid, loans, and work (including, but not
confined to, college work-study from all sources) while enrolled. Three dichotomous variables
resulted: GRANT, LOAN, and STUWORK. In the original Tool Box study, based on the
HS&B/So data, GRANTS and STUWORK were modestly significant contributors to the
explanation of bachelor’s degree completion in both the financial aid and attendance pattern
steps of the model, but fell below the threshold of statistical significance as soon as first-year and
extended performance factors were taken into consideration.

The NELS:88/2000 financial aid information covered the period 1992–94 only, i.e., the first two
years following the modal high school graduation date for the cohort. In the initial input versions
of the data in 1994 (the third NELS follow-up survey), the information was provided by students
for each institution attended, and in answer to the question, simply phrased, “besides your
previous savings, how are you paying for this?” The options were grants, loans, work, parents,
and other forms of aid. For students who attended more than one institution during the 1992–94
period, it is thus possible for different combinations of financial aid types to be attached to the

discrete attendance cases. For our purposes, if a student reported using any one form of financial
aid or support (grants, loans, work-study or campus job) at any institution attended, a positive
entry was recorded for that type of financial aid or support for that student.

56
Table 16. Logistic account of factors associated with earning a bachelor's degree in the
history of 1992 12th-graders who attended a four-year college at any time:
Postsecondary financing
Adujsted
Parameter standard
Variable estimate error t p Delta-p
Intercept -3.5855 0.6048 3.33 0.01
Academic Resources quintile 0.3362 0.0701 2.69 0.02 0.0751
Socioeconomic status quintile 0.2902 0.0571 2.86 0.02 0.0648
Education expectations 0.3993 0.1788 1.25 † †
No delay of entry 0.7854 0.1980 2.23 0.05 0.1754
Selectivity of first institution 0.3962 0.1980 1.12 † †
First-year grades 0.9878 0.1541 3.60 0.01 0.2207
College-level math in first year 0.3673 0.1480 1.39 † †
Low credits in first year -1.5148 0.1674 5.08 0.001 0.3384
Any first-year remediation 0.4967 0.1722 1.62 † †
Work-study or campus job 0.1785 0.1528 0.66 † †
Race -0.3504 0.1906 1.03 † †
Gender -0.3379 0.1374 1.38 † †
Parenthood -1.0293 0.3959 1.46 † †
†Variables did not meet threshold criterion for statistical significance
Notes: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect =
1.78. G2 = 4396.88; df =4763; G2/df = 0.923; X2 (df) = 1519.14 (13); pseudo R2 = 0.2915; percent concordant
predicted probabilities = 83.4
SOURCES: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement.)

For those who ask about the importance of initial financial aid package offerings to students’
choice of where to enter the postsecondary system, the NELS:88/2000 is of little help. The
Beginning Postsecondary Students longitudinal study of 1995/96–2001, on the other hand, has
the data elements but does not provide convincing guidance: For traditional-age students (i.e.,
those who began at age 20 or younger) in the BPS95/96–2001,7 percent (s.e. = 0.7) of those who

started at four-year colleges and had applied to more than one school and 3.4 percent (s.e. = 1.6)
of those who started at community colleges and had applied to more than one school said the
financial aid package was the principal reason for their selection.1 Such percentages do not merit
further inquiry.

We come to Step 4 of the logistic model, then, with a weak collection of dichotomous financial
aid variables, and the results must be judged in that light. As table 16 reveals, only the variable
1
National Center for Education Statistics, BPS95/96–2001 Data Analysis System; NCES 2003-173.

57
indicating whether the student received College Work-Study funds from any source or held a
campus job was even admitted to the Toolbox Revisited model, and even then, produced a t
statistic of 0.66, below the 0.765 required for carrying forward to the next step. Before that next
step is taken, though, this narrative needs an important pause.

A Critical Pause for the Second Calendar Year

The literature on postsecondary persistence, retention, and attrition has historically taken its
pauses and measures at the end of the student’s first academic year. A more constructive notion
for plotting the path of academic momentum on the far side of the postsecondary matriculation
line may be to give students two full calendar years following the initial date of enrollment
before taking their progress pulses, yet the research on this possibility has been surprisingly
limited to date (see Smith 1995, and Nora, Barlow and Crisp 2005).2 Looking backwards from
the final degree completion status of our NELS:88/2000 universe to mark cumulative change to
the end of the second calendar year following entry on key indicators provides a richer
framework for advisement.

"Retention" and its paradoxes

No national database will support the assertion that a quarter of four-year college entrants and
half of community college entrants "do not return for their second year" (Kazis 2004, p. 4).
Contrary to this negative conventional wisdom, the proportion of students in the universe for this
study who "persisted" to the second postsecondary year is substantial. The definition of
"persistence" is active and student-centered, marks a calendar academic year as July 1 through
the following June 30, and runs as follows:

Whenever the student first enrolls and earns credits in postsecondary education (summer,
fall, winter, spring) marks the first academic calendar year of their postsecondary history.

If the student enrolls and earns credits at any time and at any institution during the next
academic calendar year, that student has "persisted."

"Retention," which puts the student in a passive role vis-a-vis a specific institution, is not the
right word to describe this sequence: It’s "persistence." There are two important points to
underscore: (1) This definition reveals an extremely high rate of persistence, but (2) whether we
talk about institutional “retention” or student “persistence,” the true measure is not the fact of the
event, rather the quality of the student’s record going forward. The second academic calendar
year offers students the opportunity to recapture any lack of momentum of the first. In that
respect, the second year may be even more important than the first.

2
Nora, Barlow, and Crisp (2005) offer a potentially promising analysis of a cohort at one nonselective four-year
institution that is undercut somewhat by lack of data on age distribution and the fact that retention and six-year
graduation rates are confined to that institution. But the authors include cumulative GPA to the second year,
enrollment intensity, remediation, three core courses (what this study calls "gateways"), and a rough course
withdrawal factor—all of which speak in favor of their approach.

58
The unhappy paradoxes of retention rates and retention quality3 are revealed in table 17. The
universe for the table is confined to students with complete records. The proportion of
students who enrolled sometime, somewhere in the second academic calendar year following
entrance to postsecondary education is about 90 percent for all NELS:88/2000 postsecondary
students and 96 percent for those with standard high school diplomas who subsequently attended
a four-year college at some time (and in this group, just about everybody who started in
community colleges). These data ought to destroy the mythologies of low first-to-second year
retention (which, by the formulas most commonly applied, include only students who started
full-time and in the fall semester and returned to the same school the following fall semester).
This account gives students credit for persisting, and gives students who started in sub-
baccalaureate trade schools credit for earning a certificate in their first year and then going on
their way.4

However, this account also shows that, even when one confines the universe to the group being
followed in The Toolbox Revisited, roughly one out of five entered the second year with low
credit momentum, and roughly one out of six carried low first year GPAs. The overlap, too, is
considerable: 40 percent of those coming out of their first year with fewer than 20 credits were
also in the bottom quintile of first year GPA (not in table). Low credit momentum is also due to
the number of courses taken in the first year, with 41.7 percent (s.e. = 2.46) of those who earned
less than 20 credits attempting seven or fewer courses—a proxy for part-time status—versus 2
percent (s.e. = 0.27) of those who emerged with 20 or more credits. The reason for counting
course attempts and not credit attempts is a by-product of the volume of remedial courses (which
do not carry additive credits). Half of the low first-year credit group were tied up in remediation,
versus 17 percent of those who reached or exceeded the 20-credit threshold.

3
As Tinto (1987) noted, “The point of retention efforts is not merely that individuals be kept in college, but that they
be retained so as to be further educated.” (p. 136) Poor retention quality significantly lowers the odds of further
education.
4
The persistence rates for all 1992 12th-graders noted in this paragraph are slightly different from those previously
reported (Adelman 2004a, table 3.4, p. 42), where the weight for known postsecondary participants (not the weight
for students with complete records) was used.

59
Table 17. Percentage of 1992 12th-graders with complete postsecondary records who persisted
in postsecondary education from their first calendar year of enrollment to a second
calendar year, by type of institution first attended, and, of those who persisted, percen

Of those who persisted,

first-year
performance indicators:

Earned one- In lowest

year Did not Less than 20 GPA
Student group Persisted certificate persist credits quintile

All 12th-graders 89.7 (0.57) 0.9 (0.13) 9.4 (0.55) 33.2 (1.12) 17.4 (0.81)

Type of first
institution

Four-year college 95.2 (0.59) 0.1 (0.03) 4.7 (0.59) 15.9 (0.91) 15.2 (0.86)
Community college 84.0 (1.12) 0.4 (0.10) 15.6 (1.11) 60.7 (1.93) 21.5 (1.76)
Other sub-baccalaureate 71.5 (3.06) 14.8 (2.52) 13.7 (2.01) 31.4 (5.17) 11.9 (2.70)

All with standard high

school diploma by
December 1996 who
attended a four-year
college at any time
95.8 (0.50) 0.1 (0.03) 4.2 (0.50) 21.9 (0.98) 15.5 (0.88)

Type of first
institution

Four-year college 95.2 (0.59) 0.1 (0.03) 4.7 (0.59) 15.9 (0.91) 15.2 (0.86)
Community college 97.9 (0.87) 2.1 (0.87) # 44.0 (2.93) 15.7 (2.72)
a a a a a
Other sub-baccalaureate Low N Low N Low N Low N Low N
# Rounds to zero
a
Reporting standard not met
NOTES: Standard errors are in parentheses. Row totals for the three persistence/retention columns may
not add to 100.0 percent due to rounding. Weighted N for all 12th-graders with complete postsecondary
records: 1.88M; for all 12th-graders with comple

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

60
Momentum at the end of the second calendar year

First-year background certainly helps explain some of what transpires in student histories by the
end of the second year. Tables 18 and 19 offer a skeleton of two-year performance so that future
research on the momentum hypothesis can become more sophisticated than this outing. The
information is presented in two separate tables to make sure the messages are clear. The most
overwhelming phenomenon to note in table 18 is that by the end of the second calendar year,
those who never earned their bachelor’s degree were already 25 credits behind those who did
(57.4 credits earned by those who eventually received a bachelor’s degree versus 31.6 for those
who did not earn the degree), and for those who were behind, there was no difference between
starting in a four-year college and starting in a community college. One set of "culprits" was
attendance pattern related: lower percentage full time, lower percentage continuously enrolled,
lower percentage getting to and beyond 20 credits in the first calendar year. All these factors
decrease the probability of completion.

Credit momentum is clearly in play. For the universe of this study, the mean number of credits
earned within two calendar years of the date of entry was 49.5 (s.e. = 0.51). It takes a lot of work
in the second calendar year for those whose additive credit totals in the first year were less than
20 to reach that mean, and only 3.6 percent (s.e. = 1.03) did, compared with 77.6 percent
(s.e. = 1.02) for those who earned 20 or more credits in the first year.

But a second set of negative vectors is clearly that of academic performance. In table 18 the
spread in mean grade point average between those who eventually earned the degree and those
who did not is substantial by the end of the second year, and it is not surprising that nearly one in
five of those who never earned any credential had already become status dropouts (see table 19).
The data remind us of a common sense fact that few analysts or commentators ever acknowledge
when writing about student academic careers after high school: A measurable proportion of
postsecondary students will not perform well (for GPAs of status dropouts, by timing of
permanent departure from postsecondary education, see Appendix H, table H4). Higher
education has standards, and some students will, in fact, be placed on academic probation or be
formally dismissed for academic reasons, i.e., flunk out. These compressed signals are not as
likely to be entered on official records as graduation with honors (cum laude, etc.). 5

5
Compressed signals are used to mark extremes of academic performance. The American Association of Collegiate
Registrars and Admissions Officers has reported that 90 percent of its member institutions enter graduation with
honors on transcripts (AACRAO 2002), and, in a 2005 survey of transcript practices , found that 70 percent enter
academic probation or dismissal (www.aacrao.org/pro_development/surveys/transcript05.htm).

61
Table 18. Of 1992 12th-graders who earned a standard high school diploma by
December 1996 and attended a four-year college at any time, credits
earned, GPA at the end of the second year following initial enrollment
in postsecondary education, and extended postsecondary attendance
and performance markers, by ultimate bachelor's degree status

Of those
who Of those who attended a
Selected attendance and earned four-year college, but did
academic performance markers bachelor's not earn bachelor's

Earned 20 or more credits in 90.8 (0.79) 46.8 (1.88)

first year
Always full-time 79.3 (0.58) 37.9 (1.85)
Continuously enrolled 93.5 (0.58) 45.7 (1.93)

Cumulative credits and GPA

two years for entry

average average
credits mean GPA credits mean GPA

All students in universe 57.4 (0.35) 2.91 (0.14) 31.6 (0.86) 2.13 (0.36)
Started in four-year college 58.8 (0.30) 2.92 (0.14) 32.1 (0.85) 2.03 (0.39)
Started in community college 49.2 (1.30) 2.84 (0.42) 31.6 (2.20) 2.37 (0.81)
If no remedial courses in
first year 58.8 (0.36) 2.97 (0.14) 34.1 (1.13) 2.19 (0.32)
NOTES: Standard errors are in parentheses.
SOURCE: National Center for Education statistics: NELS:88/2000 Postsecondary Trascript files (NCES
2003-402 and Supplement).

Table 19 serves another function in the narrative, namely, to look back from the end of the
longitudinal study in December 2000, and ask what percentage of postsecondary students were
no longer enrolled and had not earned a credential of any kind, by timing of the gap between
their first month and last month of enrollment? These students are called "status dropouts"
(though they may return to postsecondary education at a later date). While the general subject of
timing and reasons for departure is elaborated in Appendix H, table 19 looks at the eventual
dropout population only by their status end of the second year of attendance. It says, very simply
that:

(a) of those who never earned a credential, 23 percent became status dropouts by the end
of the second year;
(b) of all those who ultimately became status dropouts over an 8.5 year period, roughly a
third did so by the end of the second year;

62
 (c) for those who dropped out in either the first or second year, GPAs were decidedly
below thresholds required for degrees, but
(d) when asked why they left without a degree, less than 6 percent cited academic reasons
(also see Appendix H, table H2).

Yes, the status dropout rate proves higher during the first year than the second, but by the end of
the second year, the loss of academic momentum is very evident.

Table 19. Status dropout rates at the end of the second year following initial enrollment
in postsecondary education of 1992 12th-graders who attended a four-year
college at any time, and allied academic performance data

Percent of
those who Percentage of
never earned a Percent of all Mean grade dropouts who
degree who who ultimately point average said they left
became status became status at end of for academic
Timing of departure drop-outs drop-outs enrollment reasons

Left postsecondary without

credential, never returned:

during first 11 months 13.1 (1.62) 19.1 (1.33) 1.68 (0.81 5.9 (1.92)

during months 12 - 23 after

first enrollment 10.3 (1.26) 13.3 (1.56) 1.98 (0.68) 5.6 (1.40)

NOTES: Standard errors are in parentheses. Weighted N for status droputs in first two years = 103k.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

Curriculum gateways

As significant in the panorama of student performance through the end of the second calendar
year are rates of participation in specific courses identified as curricular gateways in four-year
colleges by Hanson (1998) and in community colleges by Boughan (2001). The taxonomy of
postsecondary courses used to code the transcripts for both the High School & Beyond/
Sophomore cohort and the NELS:88/2000 data files includes over 1100 discrete course
categories6 of which 645 were used to answer the following question: "What percentage of
students in the Toolbox Revisited universe successfully completed credits in each course category
between the month they first entered postsecondary education to a point two calendar years
later?" Table 20 presents the 25 academic course categories7 in which the highest percentage of
those who eventually earned bachelor’s degrees garnered credits during that time span, and for
6
The most recent version, Taxonomy of Postsecondary Courses Based on the National Transcript Samples, 2003, is
available at www.ed.gov/rschstat/research/pubs/empircurr/index.html.

63
each course category, also presents matching percentages for students who attended a four-year
college at any time but who did not complete a bachelor’s degree.

7
Though their enrollments were substantial, physical education activities, health information, physical conditioning,
orientation, and personal development courses were excluded.

64
Table 20. Of 1992 12th-graders who earned a standard high school diploma by
December 1996 and attended a four-year college at any time, course
participation rates by the end of the second year following initial
enrollment in postsecondary education, by ultimate degree status

Percentage of students who earned credits by the end of

the second year following initial enrollment

Course Earned bachelor's Did not earn bachelor's

English composition 82.3 (1.03) 53.4 (1.40)

General psychology 61.5 (1.18) 32.2 (1.19)
General biology 35.2 (1.24) 12.4 (0.87)
Introduction to sociology 34.4 (1.12) 19.6 (1.08)
U.S. history surveys 32.6 (1.22) 14.9 (0.97)
Micro/macroeconomics 30.3 (1.14) 9.3 (0.88)
General chemistry 30.1 (1.05) 7.5 (0.74)
College algebra 26.7 (1.20) 13.9 (0.94)
U.S. government 25.3 (1.12) 10.4 (0.74)
Calculus 23.7 (1.11) 3.2 (0.43)
Precalculus 22.4 (0.95) 5.8 (0.68)
Oral communication 20.4 (1.03) 11.0 (0.72)
Introduction to philosophy 18.9 (1.05) 5.0 (0.49)
Literature: general 18.9 (1.03) 5.0 (0.56)
Spanish: intro and intermed 18.8 (1.01) 5.9 (0.61)
Western civilization 17.0 (0.93) 6.5 (0.62)
a
Introduction to computing
computinga 15.8 (0.90) 10.9 (0.81)
Introduction to accounting 15.7 (0.81) 7.2 (0.56)
Statistics (mathematics) 14.4 (0.79) 3.7 (0.68)
World civilization 12.1 (0.93) 4.0 (0.50)
General physics 12.1 (0.83) 2.3 (0.42)
Public speaking 11.2 (0.78) 6.2 (0.67)
Music appreciation 10.9 (0.84) 3.8 (0.45)
Drama criticism/history 10.7 (0.73) 2.9 (0.46)
American literature 10.3 (0.71) 1.7 (0.28)
a
This is not "introduction to computer science."
NOTES: Standard errors are in parentheses. Weighted N for bachelor's recipients = 935K; for those who
did not earn bachelor's = 513k. All row estimate comparison are significant at p<.05.
SOURCE: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES
2003-402 and Supplement).

65
This is a story that moves the curriculum theme forward from high school. It is empirically-
driven, which is to say that the volume of course participation identifies the gateways for both
community college transfer students and those who started in four-year colleges in the
NELS:88/2000 cohort. Given the disparity in the number of credits accumulated by the end of
the second academic calendar year between those who eventually earned the bachelor’s degree
and those who did not, the spreads in participation rates in these 25 course categories are not
surprising. The ratios of these percentage participation rate contrasts are 7:1 in calculus,
6:1 in American literature, 5:1 in general physics, 4:1 in general chemistry, and more than 3:1 in
precalculus, micro/macroeconomics, introduction to philosophy, general literature surveys,
statistics, history/criticism of drama, and world civilization. These courses cover the full range
of traditional lower-division distribution requirements (akin to pick one from bin A, two from bin
C, etc.) in four-year colleges and general studies associate degree requirements (the bulk of the
transfer curriculum) in community colleges, with the standard non-remedial English composition
and college algebra the most visible gateways for the community college group (Boughan 2001).

The list of 25 discrete categories in table 20 invites aggregation and, with aggregation, the
inclusion of other lower division courses that serve gateway roles. For example, there are four
college-level mathematics courses among the 25, and they certainly can be aggregated as we
have already done in our assessment of first-year performance. General biology is not the only
introductory biology course category in the taxonomy used. Some institutions still use a
zoology-botany sequence; others open the study of life forms with cellular biology; others, still,
offer the option of human biology as a starting point. All of these course categories are in the
taxonomy, and joining them with general biology in an aggregate allows a broader and more
accurate coverage of participation in gateways to upper division course work requiring a
grounding in the biological sciences.

Table 21 takes this alternative approach to describing the curricular participation of students who
earned bachelor’s degrees and those who did not. Table 21 uses 14 aggregate gateways, two of
which are single-course categories repeated from table 20 (general psychology and
micro/macroeconomics). While Appendix I provides a listing of all course categories under each
aggregate, a few notes are necessary in the text:

• Humanities other than literature covers multidisciplinary introductions to the

humanities, introduction to philosophy, ethics, and introduction to religious
studies (usually a comparative religions survey). It does not include foreign
languages.

• "Foundation business" is included for two reasons. First, business fields have
historically claimed the largest percentage of majors among traditional-age
bachelor’s degree completers and a substantial proportion of associate degrees
awarded to traditional-age students by community colleges (Adelman 2004a,
tables 5.1, p. 61 and 5.4, p. 65). Secondly, prospective business majors appear to
take at least one introductory gateway course in the area during their first two
years (introduction to business, business law or business legal environment,
and/or introductory accounting).
Table 21 displays not only the percentage of students in the bachelor’s and no-bachelor’s groups

66
who earned credits in each of the 14 aggregates, but also, for those who earned any credits in the
aggregates, the average number of credits earned. The reader will notice first, that the use of
aggregates narrows the difference in participation rates between the two groups. For example, in
the category of standard English composition, the difference in participation rates between those
who eventually earned a bachelor’s degree and those who didn’t is 82.3 percent to 53.4 percent
(table 20); but when the category becomes "college-level writing," and includes technical
writing, creative writing, and advanced essay, the participation difference narrows to 84.5
percent to 68.6 percent. That is still a significant gap, though not as severe as others in table 21.
For the record, the credits earned in the aggregates of table 21 account for 59 percent of all
credits earned by students in this study during their first two years of postsecondary education.

Table 21. Of 1992 12th-graders who earned a standard high school diploma by December
1996 and attended a four-year college at any time, participation rates in lower-
division course category aggregates ad average number of credits earned in each
aggregate by the end of the second year following enrollment in postsecondary
education, by ultimate degree status
Earned bachelor's degree Did not earn bachelor's
by December 2000 degree by December 2000

Percent Average Percent Average

completing credits completing credits
Course aggregatea credits earned credits earned

College-level writing 84.5 (0.95) 4.96 (.046) 68.6 (2.05) 4.83 (.091)
Oral communication 35.6 (1.21) 3.38 (.054) 26.2 (1.59) 3.15 (.080)
Computer-related 24.5 (1.03) 3.42 (.057) 17.2 (1.52) 3.31 (.091)
Intro biological sciences 42.1 (1.25) 5.21 (.088) 22.3 (1.53) 4.96 (.160)
Intro physical sciences 40.2 (1.15) 7.46 (.142) 15.8 (1.33) 5.79 (.223)
College-level mathematics 70.5 (1.20) 6.30 (.103) 37.5 (1.87) 5.34 (.225)
Core history 56.0 (1.27) 3.04 (.132) 34.6 (1.82) 4.13 (.099)
General psychology 61.5 (1.18) 3.33 (.030) 42.0 (1.95) 3.32 (.082)
Mico/macroeconomics 30.3 (1.14) 4.69 (.088) 13.1 (1.35) 3.86 (.112)
Humanities except literature 38.2 (1.24) 4.20 (.140) 19.1 (1.50) 3.55 (.124)
Literature 45.1 (1.30) 4.48 (.087) 19.8 (1.39) 3.84 (.144)
Core social sciences 62.6 (1.27) 4.57 (.080) 42.8 (1.85) 4.22 (.115)
Visual/graphic arts 17.3 (0.96) 5.12 (.230) 10.1 (0.98) 5.47 (.488)
Foundation business 19.9 (0.88) 5.17 (.120) 14.2 (1.41) 4.86 (.227)
a
For a listing of courses under each aggregate, see Appendix I.
NOTES: Standard errors are in parentheses. Weighted N for those who earned bachelor's degrees: 935k; for
those who did not each bachelor's degrees; 513k.
SOURCE: National Center for Education Statistics: NELS:88./2000 postsecondary transcript files (NCES 2003-
402 and Supplement).

67
At the same time—and with few exceptions—the differences in average credits earned in the 14
aggregates are negligible, statistically insignificant, or meaningless—or all three of the above. In
other words, when students who did not earn a degree took courses in these aggregates, they
earned credits at the same rate as those who did earn the degree. The problem is still that a much
lower percentage of those who did not earn degrees even pass through these gateways.

What does this account of the second calendar year mean?

The sequence of steps of the logistic narrative of what matters in degree completion in The Tool-
box Revisited moves from first-year performance (with an apostrophe of financial aid
considerations) to longer-term vistas of attendance patterns and extended performance. It does
not make a distinct stop at the end of the second year. And yet by the end of students’ second
year, a significant spread in credit generation, academic performance, and curricular participation
has opened up between those who eventually completed bachelor’s degrees and those who did
not. An event history account appropriate to the analysis of withdrawal (e.g., DesJardins,
Ahlburg, and McCall 1999) would include a measurement at this point. So as the logistic
narrative heads into consideration of attendance patterns (Step 5), the reader might ask what
would happen if we took first year-credits and first-year GPA and replaced those variables with
second-year cumulative measures.

An experimental logistic was constructed in response to that hypothetical question (see Appendix
L, table L10). What happened? The Delta-p statistic for the credits variable barely budged and
the Delta-p for GPA declined but slightly, i.e., at first glance, it doesn’t make a difference
whether one uses first- or cumulative second-year measures. At the same time, though, the
strength of that cumulative second-year credits substitute8 wreaked havoc on critical variables
that would otherwise be highlighted under attendance patterns, e.g., community college transfer
and four-year-to-four-year transfer. As we will note in a moment, disentangling the
consequences of different kinds of multi-institutional attendance is more critical to the analysis
of degree completion than status at the end of the second year.

What about the gateway lower-division courses taken by the end of the second calendar year?
Are any of them key portals to degrees? Beyond college-level writing (a requirement) and
college-level mathematics, this is not a fair question, for the answer depends on a student’s major
and the requirements of the degree-granting department. Consider an English major who took
general physics, the introductory micro/macroeconomics sequence, and introduction to design to
satisfy distribution requirements. None of these courses (which are gateways to upper-division
offerings in other fields) is a portal to completing a degree in English, though one could argue
that they provide literature majors with analytic tools, reference points and contexts that can only

enrich their interpretive powers. In fact, it could also be argued that the distribution courses
gathered in the 14 aggregates of table 21 are secondary in propulsive power to the second level
8
The variable for cumulative credits to the end of the second calendar academic year was constructed in a manner
different from the dichotomous presentation of first calendar academic year credits. Cumulative credits is a
categorical variable with four levels based on empirical mean credits earned and standard deviation (49 and 19
respectively): 0–29, 30–49, 50–69, and 70 or more.

68
of course work in specific fields. For example, "everybody" takes general psychology, but those
who log experimental psychology and developmental psychology are establishing a beachhead in
the field, and those courses become the real portals to the degree.

One might use the number of curricular "gateways" through which students had passed by the
end of their second calendar academic year as an independent variable, but the national samples
with which we work in grade-cohort longitudinal studies would require consensus of academic
administrators as to the curricular locations and expected threshold numbers of these gateways,
and that consensus is simply not available. In fact, given the variety of presentations of
postsecondary curricula across over 3,000 degree-granting institutions, that consensus may be
impossible. This is ultimately a matter for local configuration and analysis (for a course cluster
approach, see Zhai, Ronco, Feng, and Feiner 2001) as well for future research that would
revisit the differential course work hypothesis in postsecondary contexts that appeared in the
literature a decade ago (e.g., Ratcliff et al. 1995; Kroc, Howard, Hull, and Woodard 1997). As
this pause in the narrative of The Toolbox Revisited demonstrates, though, those analyses cannot
be profitably engaged on the basis of the student’s first-year work alone. The second year is just
as important.

Step 5: Postsecondary Attendance Patterns

One of the principal contributions of Answers in the Tool Box was to document, on a national
scale, what college and community college registrars, institutional research officers, and
enrollment managers had known for some time: the increasingly complex enrollment patterns of
postsecondary students (Hearn 1992; Kearney, Townsend, and Kearney 1995). This complexity
has steadily accelerated since the 1970s, when the first high school cohort tracked by the
National Center for Education Statistics attended college (for historical comparisons of multi-
institutional attendance patterns, see Appendix L, table L11). It is not merely a case of the
proportion of non-incidental9 undergraduates attending more than one school increasing from 47
to 57 percent;10 it is more a question of how they attended more than one school, and in what
combinations and order (Adelman 2004a, pp. 45–50).

The most critical distinction on this dynamic landscape is between transfer and multi-institution-
al attendance. Transfer is a migration that is formally recognized by system rules, a sequential
movement from a de jure status in one institution to a de jure status at a second institution (or
third, or fourth). Furthermore, the student’s stay at the second institution is not a short visit. So,
for example, the beginning community college student who earns 46 credits at the first school
and 5 credits at the second is a multi-institutional student not a transfer student. In the context of

a four-year student’s history, earning 102 credits at the first school (ultimately, the degree-
granting school), 21 credits at a different four-year college and 9 credits at a community college
along the way, involves a situation in which credits are transferred, not the student. But the
9
"Non-incidental" means students who earned more than 10 credits in their undergraduate careers. Those who
earned 10 or fewer credits do not accumulate enough history for analysis of attendance patterns.
10
Kuh et al. (2001) found that 53.2 percent of the 32 thousand seniors who participated in the 2000 National Survey
of Student Engagement had attended more than one institution.

69
student who begins at a community college and earns 44 credits at the community college before
attending a four-year college, then earns 55 credits from the four-year college and is still enrolled
at the four-year college at the end of our tracking period is a transfer student.

Whether and how students attend more than one school may also depend on student performance
in the first calendar year of enrollment. Using the five-year Beginning Postsecondary Students
longitudinal study of 1989–94, McCormick (2003) demonstrated that for students who started in
four-year institutions, the higher the first-year self-reported GPA, the more likely the student
would stay put, whereas for students who started in two-year colleges, a higher GPA was
associated with transfer, hence, with multi-institutional attendance. Do these findings hold up in
a transcript-based longitudinal study such as the NELS:88/2000? Table 22 outlines the
corroboration: For four-year college students, McCormick’s observation is unconditionally
seconded; for community college students, the difference between attending one school and two
schools strongly supports the argument.
Table 22. Relationships of grade point average (GPA) in first-year of attendance
to number of institutions attended by 1992 12th-graders, by type of
first institution attended

Type and number of institutions First year GPA

Started in four-year college

One institution 2.74 (0.031)

Two institutions 2.61 (0.035)
Three or more institutions 2.53 (0.039)

Started in community college

One institution 2.39 (0.045)

Two institutions 2.59 (0.037)
Three or more institutions 2.45 (0.093)

NOTES: Standard errors are in parentheses.

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files
(NCES 2003-402 and Supplement).

The students in the subject universe of this study were even more mobile than the
NELS:88/2000 cohort as a whole: 35.9 percent (s.e. = 1.17) attended only one institution,
compared with 46.2 percent (s.e. =1.04) of all NELS students who earned any postsecondary
credits. And our subjects were more likely to attend more than two institutions than the larger
universe (25.3 percent (s.e. = 1.15) versus 20 percent (s.e. = 0.81). This higher degree of
mobility invites us to identify the deeper structures of attendance so that we can better sort the
relationships between starting point, performance, migration, and degree completion.

70
For those students who attended more than one school, table 23 lays out some basic patterns.
One should be very careful in reading them. The universe consists of the 64 percent of 1992
12th-graders who not only attended a four-year school at any time but also attended more than
one institution as undergraduates. Institutional mobility is built into the definition. It is not
surprising (in fact, it is outrightly redundant) that reverse transfers do not earn bachelor’s degrees
and that classic community college transfers who earn bachelor’s degrees receive them from an
institution other than the one first entered.

Those obvious observations aside, the most intriguing students in this configuration are what
de los Santos and Wright (1990), Borden (2004), and others have called “swirlers.” The basic
definition of “swirling” follows a pattern first observed in the High School & Beyond/
Sophomore cohort data and was labeled “alternating sectoral enrollment.” In this pattern, the
student starts in either a four-year college or a community college, and moves back and forth
between them for at least one cycle, accumulating more than 10 credits from both sectors in the
process. In the sample isolated in table 23, 63 percent of these cycles began in four-year colleges
Table 23. Combinations of institutions attended by 1992 12th-graders who attended a
four-year college at any time who also attended more than one school, and
percentage earining bachelor's degrees under each combination

Earned bachelor's degree

Institutional Percent From other than

combination attending Total first institution

Two or more four-year colleges 31.8 (1.41) 82.3 (1.52) 42.3 (2.57)
Reverse transfer (four-year to
community college) 9.7 (1.00) <0.1 (0.42) N.A.
Community college to four-year
college (including classic transfers) 24.0 (1.37) 58.1 (2.97) 99.7 (0.26)
Alternating sectoral enrollment
("swirling") 15.4 (1.24) 39.1 (3.77) 69.7 (4.55)
Four-year-college-based student
attending community college for
incidental course work 14.3 (1.01) 86.4 (2.32) 24.6 (4.39)
Other combinations 4.7 (0.62) 4.6 (1.42) 45.1 (13.5)
NOTES: Standard errors are in parentheses. Column for attending may not add to 100.0 percent due to rounding.
Weighted N=753k.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

and 37 percent in community colleges (s. e. = 3.98; not in table). Furthermore, 57.6 percent of
the students engaged in just one cycle of alternating enrollment while 42.4 percent rotated
through two or more alternating cycles of attendance (s.e. = 4.05; not in table). Among the major

71
categories of students who attended more than one school in table 23 there is a broad range of
bachelor’s degree completion rates (39 to 86 percent), and the “swirlers” are at the low end of
that range.

What’s the point? Attendance patterns, as accounted for in the history of the High School Class
of 1982 (the HS&B/So), presented paradoxes. As a dichotomous variable in the original Tool
Box, multi-institutional attendance evidenced no association with bachelor’s degree attainment;
classic community college transfer bore strong and positive fruit; but the more general
phenomenon of attending more than one institution and not returning to the first institution of
enrollment evidenced a strong negative relationship to degree completion. These paradoxes are
unwound in table 24, the logistic account of the associations between the increasingly complex
postsecondary attendance patterns of the 1990s and degree completion.

As befits the topic of attendance patterns, the variables considered and admitted to the Step 5
logistic model are those of time and place:

Ever part-time: flagging those students who, at some time in their undergraduate
careers, enrolled part-time in an academic term other than a
summer term;

Summer: number of credits earned during summer terms in three bands:

none, 1–4, and more than 4;

Classic transfer: community college to four-year after earning more than 10

credits from the community college;

Four-to-four: four-year college to four-year college transfer; and

Multiple schools: marking students who attended more than one institution, whether
they formally transferred or not.

Actually, one major time-related attendance pattern variable is purposefully left out of the
equation, continuous enrollment—or NOSTOP, as it is abbreviated. NOSTOP was one of the
most powerful of the postsecondary history variables in the original Tool Box. Given the length
of the grade-cohort postsecondary histories in the longitudinal studies,11 a student was allowed a
one-semester (or its equivalent, and exclusive of summer terms) stop-out period and still
considered continuously enrolled. When this variable is entered into the logistic model, its
positive value and extraordinary statistical significance do not permit us to ascertain fully how

11
Up to 12 years for the High School Class of 1972 (NLS-72), 11 years for the High School Class of 1982
(HS&B/So), and 8.5 years for the High School Class of 1992 (NELS:88/2000).

72
Table 24. Logistic account of factors associated with earning a bachelor's degree in the history of
1992 12th-graders who attended a four-year college at any time: Postsecondary
attendance patterns

Adujsted
Parameter standard
Variable estimate error t p Delta-p
Intercept -4.6208 0.7114 3.68 0.001
Academic Resources quintile 0.3648 0.0773 2.67 0.02 0.0804
Socioeconomic status quintile 0.2790 0.0621 2.55 0.05 0.0615
Education expectations 0.5165 0.1985 1.47 † †
No delay of entry 0.9468 0.3064 1.75 0.10 0.2087
Selectivity of first institution 0.5176 0.2155 1.36 † †
First-year grades 0.9295 0.1687 3.12 0.01 0.2049
College math in first year 0.3121 0.1608 1.10 † †
Any first-year remediation 0.3261 0.1876 0.99 † †
Low credits in first year -1.1934 0.1853 3.65 0.001 -0.2712
Classic community college transfer 0.9518 0.2252 2.40 0.05 0.2097
Four-to-four transfer 0.7020 0.2271 1.75 0.10 0.1547
Multiple schools -0.7509 0.1908 2.23 0.05 -0.1655
Summer-term creditsb 0.6517 0.0866 4.26 0.001 0.1436
Ever part-time -1.6067 0.1551 5.87 0.001 -0.3545
Race -0.3481 0.2096 0.94 † †
Gender -0.2955 0.1498 1.12 † †
Parenthood -0.8677 0.4246 1.16 † †
†Variables did not meet threshold criterion for statistical significance.
b
Set in three bands: 0, 1-4, and more than 4.
NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect =
1.76. G2 = 3749.31; df = 4759; G2/df = 0.788; X2 (df) = 1984.37(17); pseudo R2 = 0.3813; percent concordant
predicted probabilities = 88.1.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402
and Supplement).

73
the other variables in the model are interacting. Its strength deflates the associative contribution
of other factors in student performance that might better illuminate the dynamics of degree
completion. Hence, in The Toolbox Revisited, we set aside NOSTOP until the last possible
moment, the final Step 7.

All five attendance pattern variables introduced in table 24 break the barriers of significance at
this stage of model-building:

• Ever part-time: Whether the student was ever part-time, a decidedly negative
influence on degree completion. The Delta-p statistic tells us that part-time status
reduces the probability of completion by over 35 percent. While previous
research has documented the extent to which part-time enrollment pulls students
off the "persistence track" (Carroll 1989), the 35 percent reduction in probability
seems high given the fact that nearly half of the NELS:88/2000 postsecondary
students were part-time at some time. Students are inconsistent in their
understanding of what part-time means, and under a more generous definition of
the concept, the reduction in probability dropped to 30 percent—still an
puzzlingly high number (see definition of PARTTIME in the Glossary).

• Summer-term credits: The more, the better. That is, each step up the short ladder
of summer-term credits increases the probability of completion by 14.4 percent.

• Classic community college transfer: The significance of transfer is solid at p <

.05, and the Delta-p says that the probability of completion for a transfer student
increases by 21 percent. At one point in the construction of this variable for the
NELS:88/2000 cohort, a flag was added to indicate whether the student’s first
enrollment at the four-year college occurred within four years of entry to the
community college. This attempted refinement of a time-sensitive variable did
not change its position in the Step 5 logistic model. Given the fact that many
students who ultimately transfer move back and forth between community
colleges and four-year institutions, it is very difficult to determine the precise
moment of transfer, i.e., the moment on the academic clock at which both
institutions recognize a permanent change in status for the student. A flag for first
date of attendance at the four-year school does not help.

• Four-year to four-year transfer: This variable is not as strong as community

college transfer. Its statistical significance is marginal at p < .10. The Delta-p
says that those who started in a four-year college and transferred to another four-
year college increased the probability of earning a bachelor’s degree by 15
percent. Some 18.5 percent (s.e. = 0.86) of those who started in four-year
colleges transferred to another four-year college. In comparing those who
transferred to those who didn’t, one could say only that the more selective the first
institution attended, the less likely the student would transfer.12
12
Only 5.4 percent (s.e. = 1.54) of NELS students who started in highly selective four-year colleges transferred to
another four-year college, versus 15.7 percent (s.e. = 1.71) for those who began in selective institutions and 20.0
percent (s.e. = 0.99) for those who started in nonselective four-year schools.

74
 • Multi-institutional attendance: In the steps of the logistic model of the original
Tool Box13 in which it appeared, the parameter estimates for this variable were
-0.009, 0.015, and -0.077, and with odds ratios ranging from 0.93 to 1.02. All
these data are basically flat and inconsequential. The situation in The Toolbox
Revisited is different. In light of the two transfer variables and the ostensive
intersection of multi-institutional attendance with transfer, what does this variable
say by coming out of a logistic model with a Delta-p indicating that attendance at
more than one school reduces the probability of earning a bachelor’s degree by
about 16.5 percent? While further probing of the issue is advised, it appears that
the message of the three variables (community college transfer, four-year to four-
year transfer, and multi-institutional attendance) is that formal transfer is more
effective than broader patterns of multi-institutional attendance, including
simultaneous enrollment and “swirling.” This common sense encourages
institutions to track and guide apparent nomads toward formal transfer.

Of the other statistically significant variables in this step, even though Academic Resources and
socioeconomic status quintile are receding in time, they are not so receding in strength, and are
very much with the explanation of degree completion. In this new configuration, no delay of
entry to postsecondary education and first-year grades are actually stronger than they were in
Step 4 in their positive parameter estimates and Delta-p statistics. The negative consequences of
earning fewer than 20 credits in the first calendar year following initial enrollment is also among
the continuing statistically significant variables, though with a lower parameter estimate and
Delta-p statistic than observed in tables 15 (first-year performance logistic) and 16 (financing
logistic). What counts continues to count.

And what counts less but remains statistically acceptable for presentation includes: (a) the set of
basic demographic variables that has been in the logistic narrative from the beginning,
(b) education expectations (which are proving far weaker for the High School Class of 1992 than
they were for the High School Class of 1982), and (c) two curricular features of the first year of
postsecondary attendance (remedial course work and credits in college-level mathematics) that
will be reconstructed in Step 6, when we extend the temporal scope of curricular experience to a
student’s entire undergraduate career.
Step 6: Extended Postsecondary Performance
Both the attendance pattern variables and those covered in this step (and the next) are
distinguished from first-year performance considerations (Step 3) by the temporal reach of the
observed behaviors. All the variables from Step 5 are carried forward. We then ask what
additional key academic measures span the full range of a student’s undergraduate history—from
entrance through the first calendar year of attendance, and beyond.
As previously noted, consideration of continuous enrollment has been set aside for the last step
in the logistic model sequence. Of other extended performance variables, what was called the
“DWI Index” in the original Tool Box and is described more prosaically here as the ratio of
courses from which the student withdrew or repeated to all courses for which the student

enrolled, also is set aside for the last step, as it is another case of a very strong variable that
would overwhelm other extended performance measures that the reader should see play out.

13
Adelman (1999), table 39, pp. 80-81.

75
The block of variables entered for the first time in Step 6 includes two reconstructions of
curriculum-related factors originally presented within the confines of the first calendar year of
attendance: type and extent of remedial problems (replacing first-year remediation), and
cumulative credits in college-level mathematics (replacing first-year credits). “Remedial
Problem” (see page 34 and table 8 above and description in the Glossary, p. 190) was not
admitted to the model, while cumulative mathematics credits was easily admitted.

The variable marking the trend in cumulative student grade point averages at three points in time
is an improvement over its predecessor in the original Tool Box (which offered only two points of
reference). Event-history analysts would no doubt want to see more points than three, but more
points than three would result in the loss of more students than the 22 percent already lost by
insisting on three (due principally to cases in which the true date of first attendance could not be
determined).

Table 25 offers a descriptive account of the three trend populations (rising, flat, and declining) in
terms of average cumulative GPAs at each measurement point, the average elapsed under-
graduate time period (in years) that is covered by those three markers, and the proportion of each
group that earned a bachelor’s degree. The purpose of presenting table 25 is to demonstrate
that the permutations of relationships between GPAs at the three moments of measurement work,
and the results evidence prima facie sense. Students with rising GPAs are more likely to earn
bachelor’s degrees than those in the other two groups, and students with falling GPAs seem more
likely to spend more time as undergraduates (though the differences here are not great). The
reader will also note that for the plurality of students, grades were flat over an average
undergraduate stay of roughly four and three-quarter calendar years.

Table 25. Three trends in postsecondary grade point average (GPA) of 1992 12th-graders
who attended a four-year college at any time through December 2000 and offered
complete postsecondary records, by GPA at three points in time, and average
undergraduate time, and percentage earning bachelor's degree

Average GPA

Average Percentage
First First two At end of elapse earning Percentage
calendar calendar undergraduate undergrad bachelor's of all
GPA trend year years career time degree in group

Rising 2.43 (0.30) 2.64 (0.28) 2.93 (0.18) 4.76 (0.57) 73.5 (1.80) 37.0 (1.09)

Flat 2.72 (0.25) 2.63 (0.27) 2.73 (0.26) 4.79 (0.59) 65.5 (1.63) 43.9 (1.10)

Falling 3.09 (0.58) 2.90 (0.29) 2.70 (0.30) 4.92 (0.81) 63.8 (2.38) 19.1 (0.89)
NOTES: Standard errors are in parentheses. Columns for percent of all in group and percent earning bachelor's
degree may not add to 100.0 percent due to rounding. Weighted Ns: rising GPA = 415k; flat GPA = 486k;
falling GPA = 215k.
SOURCE: National Center for Education Statistics: NELS:88\2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).
76
Changing majors: Not a rare phenomenon

St. John et al. (2004) complained that research on persistence pays inadequate attention to
college major. Agreed. There are a number of problems with the extant research when major is
an issue, not the least of which is the definition of “persistence” as within-year, i.e., fall to spring,
and no attention to change-of-major. Stoecker, Pascarella and Wolfle (1988) paid considerable
heed to major field, found some race-by-field interactions in a structural equation model, and did
so within the context of a nine-year longitudinal study with degree completion as the dependent
variable (a better framework than that used by others). But they faced problems of tracking
change of major, moment of decision on major, the problem of students who drop out without a
major, and the exclusion of community college transfers.

In response to these difficulties, the Step 6 extended postsecondary performance model tried out
a variable that marks change of major. This variable was not accepted into the model, principally
as a by-product of overlapping characteristics with the community college transfer variable, as
virtually all community college transfers “change major” from general studies to a specific field
on entrance to a four-year college (see Glossary, p. 191). But it is worth noting that 50 percent
(s.e. = 1.26) of NELS:88/2000 bachelor’s degree recipients changed majors at least once, and
that this estimate comes very close to Simpson’s (1987) estimate of 48 percent for a 1976–84
cohort. The 50 percent mark thus appears to be a strong reference point.

All the elements are now in place for the stage of analysis that examines student histories
through the end of their undergraduate careers (with the exception of continuous enrollment and
the ratio of withdrawals and repeats to all courses). The logistic for Step 6 is presented in table
26. There are three major observations:

1) Race/ethnicity, never statistically significant in steps 1 through 5, does not even

qualify for entry under extended postsecondary performance. Performance variables
overwhelm demography. At this key inflection point in the analysis of degree
completion, it appears race/ethnicity doesn’t matter (Light and Strayer 2002), though
it may be operating as an indirect effect through other variables. But to test what
happens in a different way, the Step 6 logistic model was run defining "minority"
separately for each of the four major race/ethnicity groups. This test was run at this
point in the logistic narrative because Step 7 introduces two very powerful variables
that would bury race /ethnicity indicators. The model itself emerged significant for
white students, but not for African-American, Latino, or Asian students. For white
students, all the significant variables in table 26 were the same except for four-year to
four-year transfer, which dropped below the threshold of significance. For Latino
students, the only statistically significant variable within a nonsignificant model (t =
1.37 with 10 degrees of freedom) was whether the student had ever been part-time.
For African-American students, the only statistically significant variables within a
nonsignificant model (t =1.70 with 13 degrees of freedom) were summer-term credits
and whether the student had ever enrolled part-time. For Asian students, nothing was
significant within a nonsignificant model (t = 0.34 with 13 degrees of freedom).

77
Table 26. Logistic account of factors associated with earning a bachelor's degree in the history of 1992
12th-graders who attended a four-year college at any time: Extended postsecondary
performance

Parameter Adujsted
Variable estimate standard error t p Delta-p
Intercept -5.8188 0.7996 4.12 0.001
Academic Resources quintile 0.3147 0.0799 2.23 0.05 0.0667
Socioeconomic status quintile 0.3066 0.0628 2.77 0.02 0.0650
Education expectations 0.3825 0.2075 1.04 † †
No delay of entry 0.7798 0.3208 1.38 † †
Selectivity of first institution 0.4103 0.2225 1.04 † †
Any first-year remediation 0.2969 0.1920 0.88 † †
Low credits in first year -1.0822 0.1957 3.13 0.01 -0.2294
Classic Transfer 0.8391 0.1273 2.12 0.05 0.1779
Four-year to-four-year transfer 0.7192 0.2285 1.78 0.10 0.1525
Multiple schools -1.0523 0.2005 2.97 0.01 -0.2231
b
Summer-term credits 0.5299 0.0900 3.34 0.01 0.1123
Ever part-time -1.6696 0.1599 5.92 0.001 -0.3539
b
Cumulative college math credits 0.5456 0.0994 3.11 0.01 0.1157
Trend in grades 0.5813 0.1119 2.94 0.01 0.1232
First-year grades 1.1619 0.1860 3.54 0.01 0.2463
Gender -0.3518 0.1578 1.26 † †
Parenthood -0.9058 0.4318 1.19 † †
†Variables did not meet threshold criterion for statistical significance
b
Set in three bands: 0, 1-4, and more than 4
Notes: Statistically significant variables are highlighted in bold. Standard errors adjusted by root design effect = 1.76.
G2 = 3355.32; df = 4632; G2/df = 0.745; X2 (df) = 1965.7 (18); pseudo R2 = 0.3984; percent concordant predicted
probabilities = 89.3.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402
and Supplement.)

78
 2) The new cumulative college-level math credits variable proves its worth, with a
Delta-p statistic that says for every move up the credit-range the probability of earning a
bachelor’s degree increases by 11.8 percent, putting it in the same range of influence as
summer-term credits and trend in grades.

3) When trend in grades is added to the variable block, the influence of first-year grades
increases, in part because, by definition, a rising GPA trend requires a lower starting
point. In fact, the first-year GPA quintile factor is twice as strong, measured in Delta-p,
than the trend.

We also note variables carried forward and remaining statistically significant in the model:

• As predicted, Academic Resources continues to lose some of its power as students

move further away in time from high school performance. There are too many
intervening experiences. In Step 1, the Delta-p said that for each rung up the
quintile Academic Resources scale, the probability of earning a bachelor’s degree
increased by nearly 15 percent (see table 12 above). At this point, that increase in
probability is 6.6 percent. Tested separately (see Appendix J), the tolerance
collinearity statistic for Academic Resources dipped close to a 0.50 mark that was
set for a caution light. While still acceptable, the tolerance statistic indicates that,
at this point in a stepwise logistic with 17 other variables, a Venn diagram would
show the circle for Academic Resources overlapping (but not eclipsing) many of
the other variables in the model.

• Socioeconomic status quintile continues to evidence some explanatory power, and

with a Delta-p that basically doesn’t change from model to model. In this model,
moving up each step of the SES quintile ladder increases the probability of
earning a bachelor’s degree by 6.5 percent. SES would assume a more significant
dynamic, as Cabrera, Burkum, and LaNasa (2005) have demonstrated with the
HS&B/So cohort, if one entered the entire logistic narrative by SES quintile, and
the elaboration of that dynamic will be left to future research.

• The echoes of first-year credit generation are still noteworthy. The Delta-p says
that dropping below 20 credits will decrease the probability of bachelor’s degree
attainment by 22.4 percent.

• Whether the student was ever part-time remains a very strong negative, with a
Delta-p unchanged from its first appearance in the attendance pattern model.

• Earning credits during summer terms is still good advice, though each step of that
variable now adds 11.2 percent to the probability of earning a bachelor’s degree,
as opposed to 14.4 percent when it was first introduced under attendance patterns.

• Both the classic community college transfer variable and the four-year-to-four-
year transfer variable stay in the model with positive contributions to the

79
 probability of degree completion of 18 percent and 15 percent respectively, even
as the multi-institutional variable strengthens with a negative parameter estimate
and a restraining influence of 22 percent on the probability of degree completion.
This tension continues to challenge analysis.

The only other variable carried forward, but losing statistical significance is that indicating no
delay of entry. No delay, which has bounced above and below the line of statistical significance,
now drops below.

Lastly, what has not counted in previous steps of the model still does not count: education
anticipations, selectivity of first institution, first year remediation, gender, and parenthood.

Three major themes emerge in this penultimate step of logistic analysis:

• The combination of the strength of Academic Resources (composite high school

performance, dominated by academic curriculum intensity) and cumulative
college-level math credits underscores the power of curricular momentum on both
sides of the matriculation line.

• A group of attendance and performance factors, including part-time status at some

point, low credit momentum coming out of the first calendar year of enrollment,
and multi-institutional attendance apparently colored by "swirling" and not formal
transfer, are strong restraints on academic momentum.

• Validating the event history critique of the original Tool Box offered by
DesJardins, McCall, Ahlburg, and Moye (2002), two measures of GPA—first-year
and trend—reflecting the quality of student effort, are very positive contributors
to degree completion. The stronger position of the class rank/GPA quintile
variable within Academic Resources reinforces this message. Grades count; and
yes, there is a competitive message here, since our GPA scales and quintile breaks
are relative, not absolute.

Step 7: Final Factors

The very last step in the logistic narrative takes all 17 variables from the extended postsecondary
performance model of table 26, and adds the two strongest variables from the original Tool Box
analysis. One of these, a dichotomous variable for continuous enrollment called NOSTOP, is a
staple of analyses of postsecondary careers (e.g., Carroll 1989; Astin 1993; Horn 1998; Berkner,
He, and Cataldi 2002).

The second of these notable variables is not such a staple of analysis. It is the ratio of courses
from which the student withdrew without penalty and those the student repeated to all courses in
which the student enrolled. This is a rare topic in the higher education literature, and was last

80
seen in the major journals in Adams and Becker (1990), though the number of course
withdrawals appears on the check list of elements in analyses of time-to-degree (Knight 2002,
2004b). The ratio counts course attempts, not credits. Withdrawals without penalty are not the
same as courses “dropped” within set periods most colleges and community colleges mark for
“drop-and-add,” and dropped courses are not included in the ratio. As for no-credit repeats: Less
than 4 percent of respondents to the 2002 survey of grading practices conducted by the American
Association of Collegiate Registrars and Admissions Officers indicated their institutions did not
allow repeats; fully 55 percent indicated that a student could repeat any course for a better grade;
and 55 percent indicated that students could repeat a course as often as they liked (AACRAO
2002). These proportions suggest a very lenient environment. Nothing here involves penalty
grades. These are all cases of noncredit grades. The variable WRPT Ratio is dichotomous: On
one side of the dividing line are students who withdrew from or repeated 20 percent or more of
all courses for which they enrolled.

Table 27 presents the final step of the logistic model for The Toolbox Revisited. The power of
the last two variables entered is instantly obvious:

• Withdrawing from or repeating 20 percent or more of courses decreases the

probability of earning a bachelor’s degree by nearly half!!!

• Remaining continuously enrolled increases the probability of degree completion

by 43.4 percent.

As a consequence of the introduction of NOSTOP and the WRPT Ratio, remediation in the first
calendar year, a weakening variable from the point of its introduction, is forced out of the model.
Its attempted replacement, the more elaborate description of type and intensity of undergraduate
remedial problems (REMPROB, described in the Glossary), did not even qualify to enter the
model. The marker for earning less than 20 credits in the first year of attendance, previously a
strong contributor to the model, is turned on its head in the final step, and drops below the
threshold of significance. Why? Principally because, as table 28 demonstrates, nearly 60
percent of those who wound up withdrawing from or repeating 20 percent or more of the courses
for which they registered were already withdrawing from and repeating 20 percent or more of
their courses in the first year. When one withdraws without penalty, one earns zero credits.
When one repeats a course, one earns credits only once (assuming one passes, of course). What
the relationships in table 28 strongly suggest is that low credit production in the first year is a
logical consequence of withdrawal and repeat behavior. If we allow negative momentum to start
early, the consequences will snowball. The phenomenon argues for more intense academic
advising and monitoring, more accurate placement, and (in some cases) more sensible credit
loads in the first calendar year of enrollment.

81
Table 27. Logistic account of factors associated with earning a bachelor's degree in the
history of 1992 12th-graders who attended a four-year college at any time:
Final factors, with complete academic history

Adujsted
Parameter standard
Variable estimate error t p Delta-p
Intercept -7.6637 0.8827 4.89 0.001
Academic Resources quintile 0.2766 0.0847 1.84 0.10 0.0583
Socioeconomic status quintile 0.2974 0.0685 2.45 0.05 0.0627
Education anticipations 0.4162 0.2211 1.06 † †
No delay of entry 0.7848 0.3515 1.26 † †
Selectivity of first institution 0.4436 0.2432 1.03 † †
First-year grades 1.1020 0.1119 3.14 0.01 0.2323
Low credits in first year -0.6553 0.2165 1.71 † †
Classic community college transfer 0.7186 0.2488 1.63 † †
Four-to-four transfer 0.6832 0.2509 1.53 † †
Multiple schools -0.7306 0.2174 1.89 0.10 -0.154
Summer-term creditsa 0.5628 0.0553 3.25 0.01 0.1186
Ever part-time -1.1739 0.1009 3.71 0.01 -0.2474
Cumulative college math creditsa 0.4993 0.1075 2.62 0.02 0.1053
Trend in grades 0.5879 0.1211 2.74 0.02 0.1240
WRPT ratiob -2.3078 0.4246 3.06 0.01 -0.4865
Continuous enrollment 2.0601 0.2211 5.25 0.001 0.4343
Gender -0.3233 0.1715 1.06 † †
Parenthood -0.8511 0.4627 1.04 † †
†Variables did not meet threshold criterion for statistical significance
a
Set in three bands: 0, 1-4, and more than 4.
b
Ratio of withdrawal (W) and no-credit repeat (NCR) grades to all grades received.
Notes: Statistically significant variables are highligted in bold. Standard errors adjusted by root design effect =
1.76. G2 = 2993.12; df = 4595; G2/df = 0.651; X2 (df) = 2260.53 (18); Pseudo R2 = 0.4382; Percent concordant
predicted probabilities = 91.8.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement.)

82
Lastly, the variable marking swirling multi-institutional attendance continues to be negative —
though marginally significant with p <. 10—in table 30 at the same time that both community
college transfer and four-year-to-four-year transfer variables fall out of significance altogether. In
this final accounting, then, all three type of multi-institutional accounts fade in associative power
in the face of behaviors that transcend institutional effects: continuous enrollment and academic
performance reflected in grades.

Table 28. To what extent does the final ratio of undergraduate course withdrawal
and no-credit repeat grades reflect the ratio and volume of withdrawal
and no-credit repeat grades in the first calendar year of enrollment?
Answers from the history of 1992 12th-graders who attended a four-
year college at any time

Number of withdrawals and no-credit repeats

in first year

Final ratio of Percent for whom

withdrawals and first year withdrawal
no-credit repeats to and repeat ratio was
all courses attempt 20 percent or higher None One Two or more

Less than 20 percent 5.4 (0.60) 66.7 (1.15) 22.7 (1.06) 10.6 (0.71)
20 percent or higher 57.5 (3.93) 13.7 (2.05) 20.3 (3.17) 66.0 (3.62)
NOTES: Standard errors are in parentheses. Weighted N for those with a ratio of no-penalty withdrawals
and no-credit repeats to all courses attempted of 20 percent or higher: 106k; of those with a ratio below 20
percent; 1.08M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

83
 Summing Up the Logistic Narrative

Parts III and IV of The Toolbox Revisited have walked the reader, sometimes painstakingly,
through the consideration of independent variables for each of the seven roughly chronological
steps from high school background to extended postsecondary performance. At the end of each
step, a logistic account of what was associated with completion of bachelor’s degrees for 1992
12th-graders who attended a four-year college at any time was offered. The logistic account
applied to students’ histories as of that particular moment only. Step 7, of course, offered the
concluding account.

But to judge the full sweep of these seven steps, we ought to see them together in the same table.
Table 29 allows summative statements about the changing strength of independent variables, the
continuity of what counts, and required goodness-of-fit statistics for the whole model. Let’s start
with the goodness-of-fit statistics because they validate the model. The reader can find them at
the bottom of the second page of the table. Everything that is supposed to happen in a stepwise
logistic model (Cabrera 1994; Menard 1995) happens. The G2 (also called the "maximum
likelihood function") falls with each step; G2/df (degrees of freedom in this calculation is the
product of the unweighted number of students in the model minus the number of variables

in the model) also falls with each step, though perhaps too much in Step 7. The chi square
statistic is a little more erratic, but rises as it is supposed to. So do the pseudo R2 and the percent
of concordant probabilities predicted at each step. These results are a relief to any analyst .

Table 29 includes all the variables that were entered at any of the seven steps of the logistic
account, even if they were not carried forward to subsequent steps because they did not meet the
t statistic threshold criterion that was set at 0.765. In this manner, we can see where these
variables were introduced and where they fell out of the model (the first "X" in the table marks
the spot). Race/ethnicity is the most noted example because it was present from the beginning,
and even though it was never statistically significant, it qualified to be retained until extended
postsecondary performance variables were introduced. At that point—and again for Step 7—and
because race/ethnicity is a sensitive factor in public discourse on education—an alternative
approach was tried out whereby each of the four major race/ethnicity groups was treated as an
independent variable in the model. Only the Asian student variable qualified for entry, but
wound up with a t statistic less than 0.50 on both attempts (Step 6 and Step 7).

Another "casualty"—though for a very different reason—is the variable describing credits in
college-level mathematics in the first calendar year. This variable was purposefully replaced in
Step 6 (extended postsecondary performance) by cumulative credits in college-level mathematics
across the student’s entire undergraduate career because it was appropriate to the temporal
framework. First-year college-level math was never significant, but its more expansive
replacement was, as might be expected from comparative participation rates in college-level
mathematics by the end of the second calendar year of enrollment (table 21).

Another virtue of the presentation in table 29 is that because all statistically significant variables

84
are highlighted in bold, the reader can swiftly identify what was consistently meaningful among
the potential associations with bachelor’s degree completion for this group of students. From the
moment of their introduction, the following factors met the criteria:

• Academic Resources quintile

• Socioeconomic status quintile
• Low (less than 20) credits in the first calendar year of attendance
• First calendar year GPA
• Summer-term credits
• Ever part-time
• Trend in GPA
• Cumulative "career" credits in college-level mathematics

And no delay of entry to postsecondary education from high school evidenced positive and
significant association with degree completion in half the stages of the logistic narrative in which
it was in play.

85
Table 29. Seven steps of a logistic regression model with bachelor's degree attainment by age 26 or 27 as the outcome for 1992
12th-graders who attended a four-year college at any time
Attendance Extended Final Factors
Background Entry First Year Financing Patterns Performance
Para- Delta-p Para- Delta-p Para- Delta-p Para- Delta-p Para- Delta-p Para- Delta-pPara- Delta-p
meter meter meter meter meter meter meter

Intercept -4.28 -4.21 -3.58 -3.59 -4.70 -5.85 -7.94

Academic
Resources 0.644*** 0.1492 0.554*** 0.1283 0.342*** 0.0754 0.336* 0.075 0.371* 0.081 0.312* 0.066 0.277~ 0.058

Anticipations 0.627 0.346 0.404 0.339 0.553 0.386 0.416

SES quintile 0.291~ 0.0675 0.286~ 0.0662 0.288*** 0.0635 0.290* 0.065 0.282* 0.062 0.307** 0.065 0.297* 0.063

Race/ethnicity -0.409 -0.471 -0.347 -0.350 -0.370 X X

Gender -0.463 -0.463 -0.341 -0.338 -0.28 -0.349 -0.323
Parenthood -1.576 -0.964 -1.027 -1.029 -0.913 -0.933 -0.851
First institution
was selective 0.477 0.406 0.396 0.493 0.399 0.444
No delayed entry 0.916~ 0.2121 0.815 0.785** 0.175 0.980# 0.216 0.825 0.785

Acceleration 0.190 X X X X X
Low credits -1.53+ -0.337 -1.52+ -0.338 -1.19+ -0.263 -1.058** -0.18 0.655
First-year grades 0.992*** 0.2186 0.988*** 0.221 0.916*** 0.202 1.148*** 0.243 1.102** 0.232
First-year
remediation 0.496 0.497 0.319 0.295 X
First-year college
math 0.360 0.367 0.318 X X
Work-Study 0.179 X X X
Multiple schools 0.751* -0.166 -1.052*** -0.223 -0.731~ -0.154

Classic transfer 0.952* 0.208 0.839* 0.178 0.719

Summer credits 0.654+ 0.144 0.530*** 0.122 0.563*** 0.119

Ever part-time -1.61+ -0.354 -1.67+ -0.353 -1.17*** -0.247

Four-to-four
transfer 0.702~ 0.155 0.719~ 0.152 0.683
GPA trend 0.566** 0.119 0.588* 0.124
Cummulative
college math 0.521* 0.110 0.499** 0.105

WRPT ratioa -2.31*** -0.487

No stop 2.02+ 0.426

Root design effect 2.17 2.19 1.78 1.78 1.76 1.76 1.76
G2 5315.44 5060.17 4411.64 4396.88 3749.31 3452.61 2993.12

df 4919 4913 4764 4763 4759 4632 4595

2
G /df 1.081 1.030 0.926 0.923 0.788 0.745 0.651
1074.9 1101.0 1516.4 1519.1 1984.2 1965.7 2260.5
X2 (df) (5) (8) (11) (12) (17) (17) (18)
Pseudo R2 0.204 0.213 0.289 0.292 0.381 0.386 0.438
Percent concordant
predicted probabilites 77.5 78.5 83.3 83.4 88.1 89.3 91.8
a
Ratio of withdrawal (W) and no-credit repeat (NCR) grades to all grades received.
NOTES: Keys to significance levels: # = .10; * = .05; ** = .02; *** = .01; + = .001. X = variable did not meet criterion to be carried forward.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript files (NCES 2003-402 and Supplement).

When one adds the withdrawal/repeat ratio and continuous enrollment (which appear only in the last step of model-building), some
rough themes emerge. Of the ten independent variables, three involve uses of time (continuous enrollment, ever part-time, summer-
term credits); and four clearly reflect different aspects of academic performance (high school Academic Resources, first-year GPA,
trend in grades, low-credits in the first year, and the withdrawal/repeat ratio).
Under the temporal theme, no delay of entry floats in and out of statistical significance. Some might say, e.g., of no delay of entry, that
the fact a variable is significant in one step of the logistic and not in another "does not always mean that different effects are
operating" (Anaya 1999, p. 507). One of the reasons that this presentation of logistic regressions provides collinearity statistics (see
Appendix J) is to reassure the reader that, for whatever reason a variable ducks in and out of significance, it is not because it is
excessively tangled up with other variables.

Summary of Parts III and IV: Investment Behaviors

When we look across the series of ingredients that rise to the top of strength of association with degree completion, we can interpret
them in terms of what Noxel and Katunich (1998) called an "investment model," though this author prefers "investment behaviors," a
microscale version of human capital. Once the modest consequences of socioeconomic status are accounted for, each step offers
students a set of decisions that require the commitment of time and effort likely to yield a future benefit. These decisions move
students, sometimes smoothly and sometimes less so, toward the degree. The choices made, beginning with high school curriculum
(from the available curriculum—which is an opportunity-to-learn issue) and quality of effort in high school (reflected in class
rank/GPA), allow subsequent leverage. Entering a postsecondary institution directly from high school, earning 20 or more credits in
the first calendar year of enrollment, and performing well enough in that first calendar year to fall in the top 40 percent of a GPA
distribution build on previous investment, and are all signs of commitment. Subsequent choices that may not be reflected in a
bounded period of time, such as excessive course withdrawals, prove to be poor decisions with negative returns, breaking accumulated
momentum. Other configurations of choice, including summer-term credit generation, meeting the challenge of college-level
mathematics, effort that yields a rising GPA, and most of all, remaining continuously enrolled, all reflect continuing leverage of
attainment. This is what academic momentum is all about.

Tinto’s (1987) approach is just as direct: Students have responsibilities, and are expected to invest time on behalf of their own
learning. Yes, in his words, students have the right “to refuse education” (p. 135), but since the primary commerce of the institution is
education, those who refuse should not be surprised if (in his very delicate phrasing) the institution exercises its “right . . . to be
selective in its judgments as to who should be further educated” (p. 135). One begins to see why student choice (and the responsibility
inherent in student choice) emerges not only as a dominant theme of The Toolbox Revisited as well, but as the principal challenge to
academic advising and counseling from secondary through postsecondary education.
One grants that a great deal more is going on in the lives of traditional-age students at the same time. They are not bloodless. The
literature on persistence has included not only a mass of psychological and social considerations (Harmston 2004; Hu and St. John
2001; Kahn and Nauta 2001; Robbins et al. 2004), but also basic aspects of maturation such as physical fitness (Zhang and Richards
1998), nutrition, roommate conflicts, dating problems, time management (Purcell and Clark 2002), and personality-environment
congruences and dissonances (Feldman, Smart, and Ethington 2004). Studies such as Stage’s investigation of “motivational
orientation” (1989) are also valuable in that they override stock demographic explanations and identify “subsets of college students
who might react similarly to college experiences” and, hence, assist advisors and student service personnel. Our data sources do not
include these factors.

What the Literature Calls “Institutional Effects”

A major strand of the literature on postsecondary retention derives from Tinto’s (1987) constructs of academic and social integration,
constructs that contributed to the emergence of the National Survey of Student Engagement (Kuh 2001) and its use by hundreds of
institutions in assessing the effects of environments on student behaviors. The reason Tinto’s work generated such a continuing
elaboration and response is a combination of its elegance and prima facie common sense. At whatever age they start out, entering
postsecondary students are not empty vessels; they come with demographic characteristics and high school experiences (and, if there
is a gap of years between high school graduation and postsecondary entry, work experiences and family formation as well) that
condition and shade where and how they enter the postsecondary system. Once at an institution, these background characteristics
interact with the academic processes and social environment of that institution to yield varying degrees of determination to persist and
complete credentials. Institutional culture, including habits of faculty interactions with students outside of the formal classroom and
opportunities for a variety of peer group interactions, plays a significant role in Tinto’s models of academic and social integration.

It is not surprising that the mass of studies drawing on Tinto’s work are institution-specific or use institutional characteristics as
independent variables. To attain enough depth on all these features of student lives so that direct and indirect influences can be
mapped through path analysis requires extraordinary probing, which is best carried out in institutional contexts where adequate
samples can be assembled. Even if we employed hierarchical linear models to illuminate the relationships between nested
characteristics of individuals within institutions and outcomes such as GPA, we would need much larger samples of students within
each institution than the NELS:88/2000 provides.1

With the student as the unit of analysis, and 25 percent of the students in the universe we are examining attending more than two
institutions as undergraduates, the calculation of institutional effects becomes problematic. For the students who attended only one
1
In a case of a continuous dependent variable for student engagement, and sets of independent variables describing both student and institutional characteristics
(half of them continuous), Porter (2005) demonstrates the statistical superiority of multilevel analysis versus Ordinary Least Squares analysis. There are three
significant differences between Porter’s illustration and the story line of The Toolbox Revisited : (1) the dependent variable (bachelor’s degree attainment) and
most of the independent variables in this data essay are dichotomous, hence, our method is logistic, not linear, (2) Porter’s model assumes that students attend
only one institution, whereas more than half the students in this analysis attended two or more, and (3) while multilevel analyses can create models for
institutions "even for schools with few student observations" (Porter 2005, p. 110), of the 3,258 institutions in the NELS:88/2000 postsecondary transcript files,
1,003 had only one student in the sample. An institution-level model with a population of one is out-of-scope.
four-year college, the weighting of institutional characteristics such as those Titus (2004) admits into his analysis of three-year
persistence at the same institution (e.g., control, size, selectivity, degree of residentiality, and percent female, etc.) are fairly easy to
plot. But for those who attended more than one institution, a weighting scheme for the influence of each institutional characteristic
based on the proportion of undergraduate time each student spent in different institutions would be necessary. For a student who
earned 26 credits at a community college, 30 credits at a four-year baccalaureate residential college, and 75 credits at an urban
university, the ratios would dilute the very meaning—let alone effect—of any single institutional characteristic. This example is not a
fantasy. Consider, for example, the credit accumulation at each school attended by two sets of students from the NELS:88/2000 who
attended three institutions as undergrad- uates (fig. 2).

Figure 2. Examples of credits earned at different institutions by 1992 12th-graders who

earned standard high school diplomas by December 1996, attended a four-year
college at some time, earned bachelor's degree, and attended three institutions
as undergraduates, by type of transfer
Credits earned at each school attended

Student cases School #1 School #2 School #3

Community college transfers

#1 66 (community coll) 34 (four-year 62 (four-year)

#2 32 (community coll) 28 (community coll) 90 (four-year)
#3 41 (community coll) 18 (four-year) 64 (four-year)

Four-year-to-four-year transfera

#1 24 36 92
#2 36 21 88
#3 25 30 79
a
All three schools attended by these three students were four-year colleges.
SOURCE: National Center for Education Statistics. NELS:88/2000 Postsecondary Transcript files (NCES 2003-
402 and Supplement)

How does one evaluate indirect effects of institutional characteristics and experiences (class size, contact with faculty, peer group
interactions, etc.) for these students? Does one weight each potential effect by the percentage of credits earned at each school in
relation to all creditsearned? Can one somehow arrive at a consolidated index for each effect across schools? How would one
compare these indices to those for students who attended only two institutions? To those for students who attended only one? All six
of the students in figure 2 above earned bachelor’s degrees. For students who did not earn bachelor’s degrees but who attended three
institutions, what is the minimum threshold of credits one would set at each institution attended before trying to determine the range of
institutional characteristics and student experiences that might have some degree of association with the dependent variable? Across
all patterns of inter-institutional "traffic" (Astin and Astin 1993), is there a common effects metric that encompasses mere excursions
(incidental attendance at second or third institutions), true migration (formal transfer), and the nomadic behavior described as
"swirling"? The author tried to develop a descriptive framework of excursion, migration, fragmentation, and discovery for this task,
but with less than satisfactory results (Adelman 2004b). Certainly, there are more sophisticated models that can address the rhetorical
questions above, but they risk both neglect or devaluation of life-changing experiences at institutions where a student spent
comparatively little time and earned few credits, and false identification of effects that can be very fleeting.

Postsecondary student attendance patterns have rendered consolidated institutional-effects analyses moot. For that reason, these
analyses are best carried out within the context of individual institutions (no matter where else the student goes to school). This study
does not belittle the constructs of institutional effects and assessment efforts such as those of the National Survey of Student
Engagement (NSSE) at all. For college and community college administrators, these assessments count a great deal in determining
the kinds of environmental adjustments likely to intensify student involvement with institutional services as well as to heighten student
satisfaction with instruction, even if the student is present for only 24 credits.

Even so, many of the variables of institutional effects analyses (and that includes high schools as well as colleges and community
colleges) are sufficiently beyond the control of administrators so that the practical implications of whatever paths one delineates as
productive are limited. Yes, high schools can change the structure of opportunity-to-learn, e.g., by not even offering arithmetic or
watered-down pre-algebra and simultaneously conveying a clear message to students that they can meet the challenge of higher level
math (Lee, Croninger, and Smith 1997). And the postsecondary level can follow suit, e.g., community colleges can refuse to teach
arithmetic or basic algebra. Neither of these strategies is micromanagement, and, however some may wish otherwise, neither is likely
to be implemented on a large scale. At the postsecondary level, the very conditions of control shift: An administrator cannot change
the fact that the institution was not the student’s first choice and that he or she is determined to transfer from the moment of first
registration; there is no drug to prescribe for a student with severe homesickness; one cannot—and should not—prevent the student
from changing majors for the second time; no college authority has any influence on the romantic life and angst of 20 year-olds that
may affect their involvement with academic pursuits (Okun, Taub, and Witter 1986); administrators can’t sand down every potentially
hard edge of their schools. Furthermore, there is a natural capacity limit to truly meaningful contact with faculty outside the
classroom. To use examples from the National Survey of Student Experience, how many faculty can deans deploy for student
participation in faculty research projects or for students to work with faculty on committees and student life programs?2 The larger
and/or less residential the institution, the less the opportunity for student-faculty interaction in other than casual or ritualistic activities
(Hu and Kuh 2002).

2
Kuh et al. (2001) reported that less than 20 percent of seniors in 2000 indicated experiencing these modes of nonstandard faculty contact. By 2004, that
proportion had not changed, while 58 percent of seniors indicated they had talked often or very often with faculty outside of class about their grades (a ritualistic
interaction). See www.nsse.iub.edu/~nsse/2004_annual_ report/html/responses _senior sfi.html. (Accessed 7/31/05).
To repeat: The NELS:88/2000 does not reach the details of student development or institution-
specific student experiences accounted for in other research lines. The elaborate connections
between student background characteristics, social and psychological predispositions, initial
perceptions and responses to a particular postsecondary environment, strength of goal
commitments, etc. in relation to not only the fact but also the pace of student progress toward a
credential—all of which have been explored and documented by the touchstone giants of
postsecondary student outcomes research (Astin, Pascarella and Terenzini, Cabrera, Tinto)—lie
beyond the scope of national longitudinal studies. That is not an excuse for the "economic"
reading, broadly construed, of academic momentum. We all know that the many investment
decisions made by students along their paths through postsecondary education do not occur in
social and psychological vacuums. But the archival data on which this study draws isolate
moments in which student choice intersects the structures of opportunity offered by institutions
whose first order of business—and first order reason for existing in our society and economy—is
the generation, preservation, and distribution of knowledge. This is a story about taking
advantage of that mission; it is not a story about growing up, although that happens along the
way.

93
 Part V
Closing the Gap

One of the principal objectives of the No Child Left Behind legislation as it works its way
through grades K–8 is to close the gaps in achievement between minority and majority students.
When that objective is extended to high schools, as this essay underscores in Part III above, we
have no adolescent left behind. And when extended to postsecondary schooling, as the logistic
narrative has done, our objective is to leave no young adult behind, either. While race/ethnicity
itself was not a significant variable in that narrative, the fact of an unhappy gap remains in
degree completion—which means that there are residual echoes of experience by race/ethnicity
that affect education outcomes. It is not the intention of this monograph to explore these echoes
(there are far more sophisticated techniques and far more knowledgeable researchers than the
author to engage in that task). But if we focus on the academic story line, we can at least draw
some parameters of possibility for assisting minority students.

Two issues attendant on this observation are now addressed: our confidence in degree completion
rates, and what it would take to close the gap between majority and minority populations in
degree completion.

Bachelor’s completion rates: Some remarkable agreements

The most prescient and eloquent statement of the problems attendant on our romance with
graduation rates was offered a decade ago by Ronco (1996):

Performance, even in relatively straightforward terms like graduation and retention,

eludes our attempts to measure it. Students no longer march lockstep through four years
of college to graduation. The educational model of the new millennium will likely be
characterized more by lifelong learning, where students move among various kinds of
higher education institutions, stopping in and out as their lifestyles and educational needs
dictate. Enrollment patterns like these make indicators like four- or even six-year
graduation rates for first-time, full-time freshmen largely irrelevant. Transfer students
deprive both the sending and receiving institution of retention and graduation credit.
(p. 1).

But accepting the object of policy affections for a moment, the most enduring context for the
analyses of The Toolbox Revisited involves what its predecessor called the Dow Jones Industrial
Average accountability measure of U.S. higher education: the proportion of entering
undergraduates who earn a bachelor’s degree. There are very few truly reliable national portraits
of degree completion because there are very few data sources of convincing scope and
magnitude. The task requires a longitudinal study with a tested sampling design and matching
weights. The technology for a full census of completion does not yet exist.

94
Ewell (2004) has reminded us that a student-based indicator of completion leads to assessments
of the benefits of higher education to the entire society and economy. An institution-based
indicator does none of that, though there is no question that the institution is responsible for
contributing to the student’s discovery and path to attainment. While the student is obviously
more important, institutions are partners.
There are three recent national accounts of bachelor’s degree completion for students who started
out in postsecondary education together that can demonstrate how these two indicators, student
and institutional, play out.3 Each account comes from a different data set:
1. The NELS:88/2000 that is the subject of The Toolbox Revisited. To repeat
its major characteristics: The temporal term is a maximum of 8.5 years
(1992–2000) from the month of entrance, no matter when that happens
and at what enrollment intensity (full-time or part-time). This is a grade
cohort, with all students roughly the same age.
2. A six-year longitudinal study, based on the Cooperative Institutional
Research Project’s (CIRP) data, that followed 1994 entering college
freshmen to 2000 (Astin and Oseguera 2002). While dominated by
traditional-age students, this cohort is not wholly homogenous in terms of
age at entrance, but is confined to those who started out, full-time, at 262
four-year colleges, weighted to represent beginning full-time freshmen at
all public and not-for-profit private four-year colleges (for-profit four-year
colleges are not included).
3. Another 6-year longitudinal study, 1995/6 through 2001, conducted by
NCES as a spin-off from the massive Congressionally-mandated National
Postsecondary Student Aid Study. We have used this data set, the
Beginning Postsecondary Students longitudinal study, previously.
Beginning students of all ages and enrollment intensities (full-time, part-
time, and mixed) are included.

Let us set these three data sets side-by-side, adjust the two NCES databases to match both each
other and the CIRP database, and then assess how much they disagree with each other. We will
confine both the NELS:88/2000 and the BPS 95/96–2001 to those who started out in four-year
colleges, limit the dates of entry of the NELS:88/2000 cohort to the same year (1992) and, most
importantly, put an upper bound on age at date of entry in the Beginning Postsecondary Students
of 20. Table 30 offers these comparisons.
Given the conditions under which degree completion is judged, the conclusions of these three
data sets do not differ that much from each other. We can say that roughly a third of traditional-
age students who start in a four-year college will earn a bachelor’s degree from the same school
in the traditional four-year period, and that between 54 and 58 percent will earn the degree from
the same school in which they began within six years of entry. When the option of earning a
degree from a different four-year college than the one in which these students commenced study
is included (in the NELS:88/2000 and BPS 95/96–2001), six-year degree completion rates for
traditional-age students are in the 62–67 percent range. Only when the temporal boundaries are
extended to 8.5 years in the NELS:88/2000 does degree attainment for those who started in four-
3
There are other large longitudinal studies of degree completion, but their samples of institutions and students are
not representative. For example, Saupe, Smith and Xin (1999) used the Consortium for Student Retention Data
Exchange information from 174 public four-year colleges to track six-year (1989–96) institutional graduation rates.
While these 174 institutions enrolled nearly half of entering four-year college students in 1989, they represent a
class, not a universe. For another, but more dated example, Stoecker, Pascarella, and Wolfle (1988) used a 1980
CIRP follow-up to an initial 1971 sample of roughly 10,000 students who had attended only one four-year college in
the interim. Some 487 college and universities were represented. Stocker, Pascarella, and Wolfle were interested in
institutional effects, so their restriction of the universe to students who had attended only one school is justified, and
their findings revealing. But that doesn’t justify using the conclusions as a generalized model of degree completion
—certainly not under contemporary conditions of student mobility. For state-level postsecondary longitudinal
studies, only Florida has produced data sets comparable to the national accounts.

95
year colleges approach 70 percent, and Florida state longitudinal studies show a similar rate
(Johnson, Coles, and Thomas 2004).
Table 30. Bachelor's degree completion rates for students who began in four-year
colleges according to three different national longitudinal studies of the
1990's

Percent completing bachelor's degree

Cooperative
institutional Beginning
research postsecondary
Bachelor's degree NELS:88/2000 project (CIRP) students
completion modes 1992-2000 1994-2000a 1995-2001

Bachelor's from same

school in 4 years 30.9 (1.4) 36.4 33.1 (1.3)

Bachelor's from a
different school in 4 years 3.0 (0.30) Not available 2.3 (0.3)

Bachelor's from same

school in 6 years 52.9 (1.27) 57.6 53.7 (1.2)

Bachelor's from different

school in 6 years 11.3 (0.79) No available 8.1 (0.4)

Bachelor's from same

school in 8.5 years 55.3 (1.24) 60.6b Not available

BA from different school

in 8.5 years 14.1 (0.84) Not available Not available

Total degree completion 69.3 (1.16) 60.6b 61.8 (1.2)

a
As reported in Astin and Oseguera (2002). Standard errors are not available.
b
In Astin and Oseguera, this cumulative figure includes students who were still enrolled at their
institution of first attendance at the end of six years.
NOTES: Standard errors for the NELS:88/2000 and BPS:95/96-2001 are in parentheses.
SOURCES: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript
Files (NCES 2003-402 and Supplement) and Beginning Postsecondary Students Longitudinal
Study, 1995/96-2001, Data Analysis System. Astin and Oseguera (2002)

96
 Degree Completion: How High Can It Go?

If roughly seven out of ten traditional-age students who start in a four-year college—and roughly
two out of three who ever attend a four-year college—graduate within eight and one-half years,
how likely is it that the rate will improve, by how much, and among what groups? When looking
back at the factors consistently contributing to bachelor’s degree completion in table 29 and
asking which are most subject to change by external parties with little to modest, but creative,
effort, a few stand out, and encourage speculation. All depend on student response. In the words
of Tinto’s (1987) astute guidance:

Though . . . institutions owe each admitted student an equal degree of attention, it

does not follow that institutions should be held accountable for the equal education
of all admitted students. To absolve . . . [students] of at least partial responsibility
for their own education is to make a serious error. (p. 135)

Let’s look at each of the more promising levers, and then explore what the configuration of
change in behaviors and policy might suggest for upper boundaries of degree completion and the
closing of the gap in completion among race/ethnicity groups.

1) Less than 20 credits in the first calendar year of enrollment: Approximately nine
percent of those who entered four-year colleges at any time earned between 15 and 19
credits in their first calendar year, with African-Americans and Latinos overrepresented.4
Within this group, only 35 percent (s.e. = 3.05) earned bachelor’s degrees compared with
77.7 percent (s.e. = 0.99) of those who earned 20 or more credits. Fifteen to nineteen
credits is close enough to 20 to be optimistic: If we know more about who students are
and where they come from, colleges and community colleges can target dual enrollment
efforts to yield at least six additive credits for these students, thus pushing them across
the 20-credit line. But we do know something about these students, and our optimism
must be tempered by the facts that 15.3 percent (s.e. = 2.49) of them were assigned to
remedial reading in their first postsecondary year, an overlapping 34.5 percent (s.e. =
3.93) never reached Algebra 2 in high school, and only 27.9 percent (s.e. = 2.96) earned
any college-level mathematics credits in that first year. For the residual group (about six
percent of those who enrolled in a four-year colleges at any time), suburban high school
students from the Pacific and Southwest Central census divisions5 are overrepresented,
and provide some geo-direction for dual-enrollment policy targeting.

2) Ratio of non-penalty withdrawals and no-credit repeat courses to all courses

4
Some 7.4 percent (s.e. = 0.56) of white students earned between 15 and 19 credits in their first calendar year of
attendance, compared with 11.1 (s.e.=1.86) of African-Americans and 18.5 percent (s.e. = 3.30) of Latinos.
5
The Pacific census division includes California, Oregon, Washington, Alaska, and Hawaii; the Southwest Central
encompasses Texas, Oklahoma, Arkansas, and Louisiana.

97
 attempted: For about 10 percent of the NELS:88/2000 cohort who attended a four-year
college at any time, this ratio was 20 percent or higher. That is, this group of students
withdrew from or repeated at least one out of every five courses in which they enrolled,
behavior negatively associated with degree completion. Compared with white students,
all minority groups are overrepresented.6(s.e. = 3.56) of Latinos. What would happen if
institutions limited the number of no-penalty withdrawals and no-credit repeats students
were allowed to accumulate, converting these cases above a given threshold to penalty
grades? This is a risky proposition and difficult to model because we do not always
know why students withdraw from courses after the customary drop/add periods (Adams
and Becker 1990). Students repeat courses to earn better or passing grades. In the case
of remedial work, the repeat is required, and the list of courses with the highest ratios of
withdrawal and repeat grades is dominated by remedial offerings (Adelman 2004a, table
6.6, p. 84). In other cases, repeats are a luxury. Since no-penalty course withdrawals and
no-credit repeats increase time-to-degree measurably for those who earn degrees
(Adelman 2004a, table 6.2b, p. 79; Knight 2004b; Noxel and Katunich 1998), and since
these behaviors have a rippling effect by blocking other students from seats, often in high
demand courses, it is in institutions’ self-interest to limit the practice. Intensified
advisory care to student credit loads (Szafran 2001) and more precise placement criteria
should help.

A good example of the way this might work is provided by Eno, McLaughlin, Brozovsky,
and Sheldon (1998), who predicted difficulty ratings for specific courses likely to be
taken by entering freshmen in their institution on the basis of both overall high school
GPA and, more importantly, grades in specific high school courses. The hypotheses for
each student within each course were then set against the empirical record of students’
actual performance in the college course at issue. The information assists students’
advisors with indications of which courses a student might find particularly difficult. The
likelihood of no-penalty withdrawals or no-credit repeat grades is attenuated by delaying
students’ engagement with those courses until they have gathered the requisite
momentum elsewhere in the college curriculum.

3) Use of summer terms: As table 31 suggests, earning more than four credits during
summer terms may have a considerable influence on the degree completion rates of
African-American students in particular. That is, 78.2 percent (s.e. = 4.12) of those who
started in four-year colleges and exceeded this threshold earned bachelor’s degrees,
compared with 21.2 percent (s.e. = 4.59) of those who did not earn summer-term credits
at all. The difference was also significant for white students, but not to the same degree.
Parallel comparisons for Latinos and Asians were not statistically significant.

6
Some 8.3 percent (s.e.= 0.68) of white students were in the high ratio category, versus 12.2 percent
(s.e. = 3.04) of Asian students, 12.3 percent (s.e. = 1.96) of African-American students, and 19.6 percent

98
Table 31. Of 1992 12th-graders who started their postsecondary careers in four-year
colleges, percentage who earned bachelor's degree by December 2000, by
number of credits earned in summer terms, by race/ethnicity

Percent earning bachelor's degrees by December 2000

Number of summer term credits

Race/ethnicity None 1-4 More than 4

All 56.2 (1.95) 68.1 (2.78) 79.7 (1.29)

White 59.8 (2.22) 74.2 (2.58) 82.2 (1.19)

African-American 21.2 (4.59) 42.5 (10.3) 78.2 (4.12)
Latino 48.6 (7.14) 28.3 (7.15) 56.4 (6.21)
Asian 66.8 (10.3) 70.0 (13.0) 77.9 (7.08)
NOTES: Standard errors are in parentheses. Weighted N=1.14M.
SOURCEL: National Center for Education Statistics: NELS:88/2000 postsecondary transcript files (NCES
2003-402 and Supplement.)

Summer-term enrollment that results in more than four additive credits works for just
about everybody (we saw that in the multivariate analyses). While it surprisingly does
not shorten time-to-degree for those who earn degrees (see Appendix K), summer credits
are associated with higher graduation rates for everyone except Latino students. It is thus
in an institution’s self-interest to encourage the use of summer-term work, either at the
institution itself or at another school. Ways of approaching this goal shy of tuition
discounting include moving some of the offerings of high demand courses from the
traditional academic-year terms into the summer term, offering summative course work
in the major in the summer term, and recruiting students for credit-bearing internships
and cooperative education placements in the summer terms. Academic administrators in
cooperation with strategic enrollment managers can be very creative.

4) The high school curriculum component of Academic Resources: One of the more
dramatic illustrations of the potential influence of high school performance on bachelor’s
degree attainment in the original Tool Box isolated four major race/ethnicity groups,
confined each to those who entered a four-year college directly from on-time high school
graduation, and compared two estimates: the bachelor’s degree attainment rate for
everyone versus the bachelor’s degree attainment rate for those who were in the top 40
percent on each of the three component measures of Academic Resources (curriculum,
class rank/GPA, and senior year test score). Though undercut by the fact that, in
multivariate analyses, race/ethnicity does not play much of a role in bachelor’s degree
completion, the cross-tabulation of these estimates suggested that the criterion-referenced
curriculum variable had the potential to boost African-American degree completion by 28
percent and Latino degree completion by 18 percent (though that datum was statistically
shaky) and white students’ degree completion by 10 percent. For Asian students of the

99
 period, test scores made more of a difference than either curriculum or class rank/GPA.

Class rank/GPA and senior year test score are not criterion referenced, and while
performing better in high school in terms of grades and tests improved degree completion
in the original Tool Box, it was by lesser magnitudes. It was observed that “in a happy
paradox, everybody can be in the ‘highest 40 percent’ on the [criterion-referenced]
curriculum measure,” whereas that is not true for test scores or class rank (Adelman
1999, p. 85). Content counts, particularly for minority students, and the effect of special
preparation programs supports that conclusion (Ishitani and Snider 2004)

Does this analysis still hold, particularly as we have already observed improvements in
academic curriculum participation by the High School Class of 1992 compared with the
High School Class of 1982 (see tables 5 and 6 above)? That depends on what else is in
the mix of factors associated with degree completion, and unlike the original analysis of
this feature in Answers in the Tool Box, the other factors are matters of postsecondary
entry and postsecondary academic behavior.

How might the four factors just highlighted, along with direct entry to higher education
following high school graduation, cumulatively influence bachelor’s degree completion for the
core universe of this study, the subjects of table 29? To remind the reader just who these students
are:

• they were 12th-graders in 1992;

• they earned a standard high school diploma (not a GED) by December1996 (as
the NELS transcript records conclude in December 2000, an honest account of
degree completion should allow the student at least four years to reach that mark);
• they attended a four-year college at any time (20 percent started in community
colleges, and those who started in community colleges earned an average of 52
credits [s.e. = 1.58] from community colleges and 59.5 credits [s.e. = 1.92] from
four-year colleges); and
• their postsecondary records were complete, i.e., no transcripts were missing.

Table 32 takes this group, and first sets down a base bachelor’s degree completion rate for the
entire cohort by four major race-ethnicity groups.7 The table then enters five key conditions
influencing degree attainment that are subject to the initiatives and control of second parties, and
marks the potential bachelor’s degree completion rate for students meeting the thresholds of
those cumulative conditions. As each condition is added to the cumulation, the universe narrows
for the NELS:88/2000 cohort, and as it narrows, hypothetical degree completion rates rise. One
could choose a different sequence of these conditions, but the bottom line would be the same.

7
There were too few American Indians in the base group to follow through a narrowing population as conditions for
academic momentum were added.

100
Table 32. Hypothetical cumulative consequences of variables critical to bachelor's
degree completion for 1992 12th-graders who earned a standard high
school diploma by December 1996, attended a four-year college at any time,
and whose postsecondary records were complete, by race/ethnicity
Percent earning bachelor's degree
Cumulative African -
conditions White American Latino Asian All

1) Baseline, no 67.6 (1.18) 52.1 (4.26) 45.4 (3.74) 67.9 (4.71) 64.6 (1.12)
conditions
2) No delay of entry 71.0 (1.22) 54.6 (4.49) 50.5 (3.79) 68.2 (4.89) 67.9 (1.15)

3) No delay, top 40 85.6 (1.50) 65.9 (8,.57) 69.2 (6.33) 91.5 91.96) 84.1 (1.40)
percent of high school
curriculum, and
highest high school
mathematics above
Algebra 2

4) No delay, top 40 90.6 (1.31) 84.6 (5.95) 69.2 (8.12) 92.6 (2.27) 89.1 (1.30)
percent of high school
curriculum, and more
than four credits in
summer terms

5) No delay, top 40 92.6 (1.23) 88.2 (5.28) 71.9 (9.07) 93.9 (2.16) 91.4 (1.24)
percent of high school
curriculum, more than
four credits in summer
terms, and 20 or more
credits in first calendar
year of attendance

6) No delay, top 40 95.5 (0.98) 94.3 (4.62) 79.4 (11.1) 95.3 (2.20) 94.6 (1.07)
percent of high school
curriculum, more than
four credits in summer
terms, 20 or more
credits in first calendar
year of attendance, and
less than 10 percent of
grades were
withdrawals or no-
credits repeats

NOTES: Standard errors are in parentheses. Weighted Ns for each cumulative step: (1) 1.45M; (2) 1.33M; (3) 712k; (4)
621k; (5) 310k; (6) 273k
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcripts Files (NCES 2003-402).

101
These are statistical coincidences, not causes. The subjunctive is very strong in this presentation.
The standard errors indicate that not all these changes would be statistically significant,
particularly for minority students, but that phenomenon, too, is a consequence of a shrinking
population at each step. No delay of entry is invoked as the first condition because it results in
the least shrinkage, leaving a larger population with which to assess the other four factors.

All five conditions of table 32 are criterion-referenced. That is, everyone can meet those
conditions; everyone can cross those thresholds. The populations could be much larger. Does
that mean that future degree completion rates will look like those in this table if everyone meets
the criteria on all five counts? No: We know that not everybody will make it. We have already
marked and acknowledged that some students will flunk out or become status dropouts for other
reasons. But the table suggests just where the improvements are likely to be dramatic—and for
whom. Some examples, by race/ethnicity, would be instructive.

For Latino students, high school academic curriculum attainment provides the greatest leap in
degree completion, narrowing the gap with white students from 22.2 percent in the base rate to
16.4 percent. But nothing else narrows the gap any further. Postsecondary summer-term
participation, for an obvious case, yields something of a zero, and getting across the 20-credit
line in the first year of attendance appears to do little for Latino graduation rates. So if we are
looking carefully at this fastest growing population in the United States, we pour our efforts into
high school preparation above everything else. A decade from now, we will be able to assess
better whether postsecondary behaviors are more impressive contributors.

For white and Asian students, moving into the top 40 percent of the high school academic
curriculum intensity index and completing high school mathematics beyond Algebra 2 also is the
strongest engine among these variables for boosting college graduation rates. Earning more than
four summer-term credits adds something for white students, but nothing to speak of for Asian
students. The other factors may contribute minor momentum, but once one reaches a
hypothetical 90 percent completion rate, subsequent improvements are not meaningful.

For African-American students, who start out at a higher bachelor’s degree completion rate than
do Latinos, the high school academic curriculum factor does not improve degree completion by a
statistically significant amount, but earning more than four credits in summer terms offers a
stunning boost, narrowing the degree completion gap with white students from 15.5 percent to 6
percent. The momentum provided by this high-octane persistence behavior continues through
the first calendar year credits criterion and avoidance of no-penalty withdrawals and no-credit
repeat grades until, at the bottom line of the hypothetical rates set forth in table 32, African-
American degree completion rates are no different from those of whites and Asians.

What did African-American students in the NELS:88/2000 cohort universe who attended a four-
year college at any time study during summer terms? A transcript-based account can be very
revealing—and in this case, positively so. Using 110 aggregate course categories, table 33
displays the 15 categories accounting for the highest percentages of summer-term credits earned.

102
Table 33. Percentage of credits earned during undergraduate summer terms
by African-American 1992 12th-graders in aggregate course
categories

Category Percent of total summer term credits

Business other than accounting and administrative science 5.5

Chemistry 5.1
Physical education activities/health information 4.9
College-level mathematicsa 4.1
b
College-level writing 3.7
General biology 3.3
Psychology other than general psych 3.0
Physics 2.8
U.S. history surveys, American Civilization 2.8
Micro/macroecomonics 2.6
Accounting 2.5
Calculus 2.5
Foreign languages 2.4
Liberal studies/general humanities/general social science 2.1
Oral communication/public speaking 2.0

Total for top 15 categories 46.7

a
Includes college algebra, finite mathematics, pre-calculus, and statistics
b
Includes standard English composition, technical writing, creative writing, and advanced essay.
NOTE: The calculation is a "credit ratio" in which the unit of analysis is the course category. All
student-weighted earned credits from all course categories are added, with the total a finite glass of
100 percent that becomes the denominator for subsequent calculations. No standard errors are
produced.
SOURCE: NATIONAL Center for Education Statistics: NELS:88/2000 Postsecondary Transcript
Files (NCES 2003-402).

This is a very respectable list for anyone. It advises advisors: Point out to African-American
students that their peers have proved (a) summer-term credit production is a benefit, and (b) the
student can concentrate in the summer terms on the kind of critically demanding and critical
gateway courses that might not otherwise be in as intense focus during an academic year term.
The learning goes deeper under single-subject concentration. Examples include organic
chemistry, calculus, and experimental psychology. To repeat: There are values in summer-term
credit production for everybody, but if summer-term credit production looks like it benefits a
particular group more than others, that needs to be said. If we don’t know why the impact looks
more significant for African-American students, let us submit the issue to future research.
Is the story for low-SES students the same?

Reviewers of drafts of this document asked whether the same type of analysis could be
performed by socioeconomic status, since we know that the gap in degree completion between

103
the highest and lowest SES quintiles is far wider than any pair of race/ethnicity comparisons
(Adelman 2004a, table 3.1, p.34). The lowest SES quintile group is small: only 6 percent of the

1.43 million 1992 12th-graders who graduated from high school by December 1996 with a
regular diploma and who attended a four-year college at some time. These students completed
bachelor’s degrees at a rate of 35.9 percent, compared with 55.4 percent for students in the
middle (3rd) SES quintile, and 79.7 percent for students in the highest SES quintile. Perhaps the
best illustration of what might close the gap for them is to compare their hypothetical rates of
completion with students from the middle and highest quintiles, and table 34 does so. We use the
same levers as variables because they are subject to second-party action. But we have to stop
after the step where summer-term credits are added in because (a) in extending the data beyond
the boundaries of table 34, subsequent variables in the cumulative sequence (first-year credit
threshold and withdrawal/repeat limits) do not add anything to the degree completion rates of
low-SES students, and (b) the standard errors of the estimates have risen to the point at which
comparisons between the lowest SES quintile group and the middle group are not statistically
significant. The data of table 34, though, send clear messages: Despite obviously great
limitations inherent in low socioeconomic status, the most promising engine of momentum for

Table 34. Hypothetical cumulative consequences of variables critical to bachelor's

degree completion for 1992 12th-graders from the lowest, middle, and
highest socioeconomic status quintiles who earned a standard high
school diploma by December 1996, attended a four-year college at any
time, and whose postsecondary records were complete

Cumulative Percent earning bachelor's degree

conditions

Lowest SES Middle (3rd) Highest SES

quintile SES quintile quintile

1) Baseline, no conditions 35.9 (3.74) 55.4 (2.34) 79.7 (1.50)

2) No delay of entry 39.9 (4.18) 59.7 (2.47) 81.3 (1.46)

3) No delay, top 40 percent of high

school curriculum, and highest high
school mathematics above Algebra 2 58.7 (6.14) 72.5 (3.71) 89.0 (1.21)

4) No delay, top 40 percent of high

school curriculum, highest high school
math above Algebra 2, and more than 4
credits in summer terms 59.6 (9.73) 82.2 (5.26) 91.2 (1.25)
NOTES: Standard errors are in parentheses. Weighted Ns for each cumulative step: (1) 1.45 M; (2) 1.33 M;
(3) 431K; (4) 199k.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402).

104
these students is a strengthened high school curricular background, and achieving that objective
requires greater attention to the provision of curriculum and the quality of learning environments
in schools attended by these students, no matter where those schools are located. Even then, the
degree completion gap between the lowest and highest SES quintile would be 32 percent.

The bottom line on degree completion

In reporting the proportion of college students who complete bachelor’s degrees, there seem to
be as many answers as there are observers. A number of rules are recommended:

1. Use transcript-based data sources. They do not lie; they do not exclude.
2. Follow the student, not the institution. Community college transfer students are
followed to determine degree completion. Do it for everybody! And if one insists on
institutional graduation rates, give the institution credit for students who transfer in.
We don’t do that now.
3. Do not restrict the universe to students who entered higher education only in the
fall term and only full-time (otherwise one excludes half of an entering cohort).
4. Do not mix your 18 year-old daughter and your 34 year-old brother-in-law in the same
analytic of degree completion. They are on completely different life trajectories. In
other words, institutions should consider ledgers indicating degree completion rates
separately by age bracket at the point of entry, and national accounts should follow
suit.
5. The only people for whom bachelor’s degree completion rates should be calculated are
those who actually enrolled at a bachelor’s degree-granting institution at some time.

All these common sense rules, respectful of students and their families, were observed in The
Toolbox Revisited—and in Answers in the Tool Box before it.

The elaborate multivariate analyses of Parts III and IV of this essay provided a framework within
which academic momentum toward bachelor’s degree completion (not toward something else
such as first-year GPA or second-year retention) could be tracked and judged, but without
race/ethnicity as a vector. When we extract from this framework the most tractable variables and
examine their potential repercussions in terms of closing degree completion gaps by
race/ethnicity, there is no question that the academic intensity of secondary school curriculum
continues to serve as the engine of subsequent academic momentum, and that the use of summer
terms for substantive, highly focused postsecondary coursework is an effective booster to that
engine. Both these variables are mixtures of the student’s use of time and the mastery of content.
But what does “academic intensity” or “an academically ‘rigorous’ curriculum” really mean?
What content, postsecondary as well as secondary, could be labeled as “challenging,” that is, the
knowledge toward which students eagerly reach as opposed to unenthusiastically slog through?
These questions are part of a large missing component in this study. Part VI of this essay will
reflect on that component with the greater care it deserves.

105
 Part VI
The Missing Element of This Story

There is (at least) one major missing element of the story line in both the original Tool Box and
The Toolbox Revisited, and some important questions readers might have asked in the course of
the logistic narrative that should be addressed. The missing element, a by-product of the
limitations of the NCES grade-cohort longitudinal studies database, is content standards in high
school curricula. Postsecondary curricula, which are far more complex in organization, offer a
different framework for assessing potential student learning.

Beyond Course Titles to Content Standards

In the years following the release of A Nation at Risk (1983), a standing one-liner on the lecture
and conference circuit ran something like “we’ve changed the marquée on the theater, but the
show inside is still the same.” The marquée carried the course titles, and for better or worse, that
is what our grade-cohort longitudinal studies transcript files rely on. As Shireman (2004) has
concisely put it, "If schools just change the names of the courses . . .students will not have
learned anything more" (p. 4). In building the NELS:88/2000 data files, we discovered that in
some high schools, “precalculus” on a transcript could mean any mathematics prior to calculus,
including Algebra 1. On the postsecondary transcripts we often ran across cases of professorial
marketing with course titles such as “Tooth Brush,” “Dots to Dinosaurs,” “Time After Time,”
and (yes) “Good Books” that even online catalogues could not explain. The postsecondary
transcripts also carried 3-credit courses in topics such as social event planning, daily living skills,
and "appreciation of sports" that, when the syllabi were examined online, could be offered to
junior high school students. One has to acknowledge the limitations of the data source, in
addition to its virtues. But the limitations, in these cases, were serendipitous because the online
search for concrete clues to the content of course work was dispiriting.

Over the past two decades, states have paid increasing attention to content standards for
elementary and junior high school curricula, and the No Child Left Behind legislation has
stimulated a period of intense review and refinement of these standards. But as Achieve pointed
out in its review of state high school graduation requirements (2004), similar detailed sets of
content standards at the secondary school level are rare. We get generalized requirements that
reference course titles such as "11th-grade English" or "Applied Biology," and have no idea what
precise learning objectives will be pursued. There have been exceptions, of course, with
mathematics being an often revisited subject and in ways that demonstrated the difference
between what should be expected of all high school graduates and what should be expected of
entering postsecondary students (California Education Roundtable 1997).

The past five years in particular have witnessed the birth and expansion of a number of large-
scale efforts to get beyond Carnegie unit credit counting to comprehensive criterion-referenced
statements of what graduating high school students should know and be able to do in order to

106
"succeed" in postsecondary environments. By "succeed" is meant completing degrees (e.g.
American Association of Universities 2003; Achieve 2004). We have also witnessed the growth
of high profile alternative approaches to secondary-postsecondary transitions such as "early
college high schools." These efforts have been aided and abetted by comparative analyses of
emerging state high school exit examinations and the content and knowledge-objectives of lower
division college courses (Conley 2003; Venezia, Kirst, and Antonio 2003). The American
Diploma Project (Achieve 2004) pushes beyond general knowledge-objectives to include
samples of college assignments (e.g., profit-maximizing output analyses in microeconomics, pH
calculations for a complex solution in introductory chemistry, an essay assignment on Plato’s
distinction between thinking and belief in introductory philosophy) and workplace tasks (e.g., a
bank loan officer’s assessment of an application from a corporation for $1.7 million to purchase
two corporate aircraft, a report requiring measurements of DC supply voltage for diffusion
furnaces in semiconductor manufacturing and analysis of furnace regulator modification costs,
and the determination of dosages in an insulin therapy regimen). These are superb examples of
digging below the credit count to the stuff of learning.

Granting that the academic quality and intensity of one’s high school curriculum is a key
determinant of postsecondary success, there is no assurance that either the standards of
secondary school performance, content coverage, or challenge of the material will come close to
the threshold demands of either four-year or community colleges. For the vast majority of high
school graduates, who will not attend selective institutions, the "disconnect" is considerable.
Indeed, Venezia, Kirst and Antonio (2003) urge everyone to pay more attention than that to
which the media are accustomed to the "broad access colleges," including community colleges,
because the service of these institutions to the bulk of the nation’s postsecondary learners has the
greatest potential to better "the civic and economic well-being" of every state and region (p. 46).

It is with that in mind that the work of the American Diploma Project (2004) is noteworthy
because the sample assignments and examination questions selected signal precisely the kind of
learning expected of the bulk of the nation’s postsecondary learners. Such assignments and
questions provide clear expectations for students entering community college occupational
programs as well as those moving into the general education portions of postsecondary
education. The microeconomics problems come from a community college, the chemistry from
a research university.

Previews of the future and their risks

In fact, it could be argued that these previews of lower-division postsecondary learning object-
tives and tasks should be part and parcel of 11th- and 12th-grade curricula, equally accessible to
students intending traditional lines to a bachelor’s degree and those following career and tech-
nical education paths that may include the bachelor’s, but certainly involve the occupationally
oriented community college associate degrees. We can get to these levels through a modest
amount of dual-enrollment for many students, but for those for whom dual enrollment is
inaccessible, special postsecondary preview modules could be offered either by high school

107
faculty with the requisite knowledge or by visiting college and community college faculty.1
Somehow, students should have a taste of what will be expected, and whether their current
knowledge and skills puts them at least on the threshold of those expectations, if not beyond.
Some may argue that such previews will scare some students away from continuing their
education or that some secondary school teachers will resist the intrusion. One has to be
optimistic in both cases, and come to the task with the conviction that it’s worth the risk. The
risk to academic momentum of not providing these opportunities is greater.

While each state bears principal authority and responsibility for linking the curricular pathways
of secondary and postsecondary sectors (Venezia et al. 2005), the examples set forth by the
American Diploma Project and the “previews of the future” strategies described above do not
treat state borders as Maginot lines. And for good reason: Roughly one out of four
undergraduates (and one out of three African-American students) in The Toolbox Revisited
universe started their postsecondary careers in a state other than that in which they graduated
high school. The geographic mobility of our traditional-age postsecondary populations suggests
that, at the least, multistate regions (e.g., South Atlantic) should be considered productive
information zones for providing those concrete signals of expectations and specific examples of
quality postsecondary student work.

Moving the messages to students, families, and school teachers

Let’s take the best of what is offered by reports such as that of the American Diploma Project and
books such as Conley’s recent (2005) College Knowledge, and move it out to much larger
audiences than policymakers and others who habitually read such reports and books. It might be
more helpful for each college and community college to include in its information packages for
prospective students a sample of those examination questions and assignments in courses
typically taken by lower-division students. One might go even further and provide examples of
exemplary student responses—even less-than-exemplary student responses—to those
examination questions and assignments.

1
Dual enrollment has made huge strides since the NELS:88/2000 cohort went to high school. Some 38 states now
have formal policies (Karp, Bailey, Hughes, and Fermin 2004), though these vary widely in terms of what is offered
and by whom, who pays tuition, who gets credit toward what, academic eligibility, grade levels, course enrollment
limits, and whether credits earned are placed in escrow (though that varies more by institution than state). The
2002–03 NCES survey of dual enrollment practices at postsecondary institutions distinguished between high school
students who took courses within formal agreements and those who took courses outside of those agreements. Our
NELS:88/2000 transcript data cannot make that distinction, so they lump together all postsecondary credits earned at
colleges or community colleges prior to the date of high school graduation.
Kleiner and Lewis (2005) estimate that 813,000 high school students took courses at postsecondary institutions,
either within and/or outside of dual enrollment programs and agreements, in 2002–03, compared with the weighted
213,000 NELS:88/2000 students who earned any postsecondary credits through dual enrollment during their high
school years (1989–92). While not exactly comparable populations, those figures are testimony to the growth of
student participation in these arrangements.

108
A quarter century ago, the Educational Testing Service worked on developing an experimental
examination to assess lower college division competencies across a matrix of cognitive
operations (e.g., analysis, synthesizing) and general curricular areas (e.g., science, humanities).
Called “Academic Competences in General Education,” the questions were open-ended (not
multiple-choice) and required responses that could be composed in roughly 10 minutes. For
example, one question went something like this:

Suppose a new method for producing energy had been developed that, when brought
on-stream, would have the effect of slowing the rotation of the Earth from 24 to 26
hours. Before we can flip the switch on this new form of energy, an Environmental
Impact Statement (EIS) must be filed. What would be the chapter headings and major
sub-headings of that EIS ? You need not fill in the details, but the most important
questions about the potential effects of a 26-hour day ought to be included.

In a trial of the examination, faculty sorted student responses to questions such as “Earth’s
Rotation” into piles of descending quality. They then described what the students whose
responses were rated most highly did to earn that rating, turned to the second most highly rated
collection of responses and arrived at a consensus of what these students did not do that the first
group did—and so on down the line of the piles of student papers they had rated until a level
euphemistically described as “nonresponsive” was reached. The faculty descriptions were
subsequently aggregated and ironed out as performance criteria. One could then turn to
incoming freshmen with this question, along with an example of a first-rate response and an
example of a fourth-rate response (out of seven levels), show them the performance criteria and
say, “This is what you can expect to be able to do by the middle of next year; would you be
better than the fourth level now?” A light goes on somewhere—or should—when entering
students see that what it takes to respond effectively to the "Earth’s Rotation" question draws on
a selection of knowledge from economics and business, psychology, international relations,
world history, cultural anthropology, physical geography, technologies, visual arts—all in
addition to the core sciences. More important, the light goes on when students see a good
example of how to put all this together in an outline of a document that is as much a product of
daily work life as it is a college exercise.

And a similar light can go on among high school seniors as well, nearly all of whom think they
are going on to something called “college,” but have little idea what that means beyond the fact
that they will no longer be bound by school rules. Prospective students are not the only audience
for this information: Their families and high school teachers are equal players. Not all parents
will understand the assignments, laboratories, and examination questions, but certainly they will
sense, from the examples, that postsecondary education is serious business, that their children
will not get through with a carefree study schedule, that colleges and community colleges have
standards, and that such examples are typical of what their children will learn. Inevitably (being
optimistic about this) they will be proud enough at the prospect to ask their children to be
prepared. High school teachers reading these assessment "prods" will have to reflect on the

109
extent to which their graduating seniors possess the academic foundations and momentum to
respond successfully. To the extent to which high school teachers have doubts, they may adjust
the form and content of their own assignments, laboratories, and examinations. Students are
bound to be the beneficiaries of the resulting "postsecondary practice runs."

Postsecondary dimensions of content

At the postsecondary level, the issue of content is somewhat easier to address, but only in the
context of a student’s entire undergraduate record. We know that students completing organic
chemistry enrolled in the course not only with general chemistry as a background, but also
mathematics at at least the precalculus level. We know that students completing a course in
American economic history would have a very difficult time without prior learning in both U.S.
history writ large and the standard introductory micro/macroeconomics sequence. A course in
the psychological and cultural components of health care in a community college allied health
program will probably not be offered to anyone without previous course work in general
psychology. The chances that a fine arts degree candidate’s transcript shows an entry for a
course in color and color theory without prior foundations and studio courses are remote. In
higher education, we can use the second- and third-level courses in a field as rough
confirmations of learning at the introductory level. There is a sufficient lattice-work within
disciplines to track traces of content from one point on the scaffolding to another. No, the traces
inherent in second- and third-level courses in a field are not in themselves content standards, and
nuance is expected from school to school, but there is a generalizeable quality to these content
cues. In fact, the generalizeable portions of upper-division curricula are what one is likely to see
on the Graduate Record Examination Board’s subject matter examinations. This topic witnessed
a surge of interest in the 1980s (for a paradigmatic analysis, see Oltman 1982), and is worth
renewed exploration.

110
THIS PAGE LEFT INTENTIONALLY BLANK

111
 Part VII
Concluding Messages

Compared to its predecessor, Answers in the Tool Box, the preponderance of the Toolbox
Revisited story has been on the postsecondary side of the matriculation line. Implicitly, it calls
on colleges, universities, and community colleges to be a great deal more interventionary in the
precollegiate world, to be more self-reflective about the paths they offer from high school
through their own territories. It also calls on them both to fortify their institutional research
capacities and integrate them more intimately with academic advising and course scheduling. As
noted above, we are witnessing measurable ferment on the high school side of the passage, and
as the principles of the No Child Left Behind legislation move beyond grade 8, we will see more.
The higher education sector cannot sleep through these changes.

To students as agents of their own futures

Beyond that fundamental banner of institutional fortification, there are three sets of messages
impelled by both studies. The first set is for entering high school students who, when asked,
blithely shrug that "of course I’m going to finish college."

1. Just because you say you will continue your education after high school and earn
a college credential doesn’t make it happen. Wishing doesn’t do it; preparation
does! So . . .

2. Take the challenging course work in high school, and don’t let anyone scare you
away from it. Funny thing about it, but you learn what you study, so if you take
up these challenges, your test scores will inevitably be better (if you are worried
about that). If you cannot find the challenge in the school’s offerings, point out
where it is available on-line, and see if you can get it that way. There are very
respectable Web sites offering full courses in precalculus, introductory physics,
humanities, music theory, and computer programming, for example.

3. Read like crazy! Expand your language space! Language is power! You will
have a lot less trouble in understanding math problems, biology textbooks, or
historical documents you locate on the Web. Chances are you won’t be wasting
precious credit hours on remedial courses in higher education.

4. If you don’t see it now, you will see it in higher education: The world has gone
quantitative: business (obviously), geography, criminal justice, history, allied
health fields—a full range of disciplines and job tasks tells you why math
requirements are not just some abstract school exercise. So come out of high
school with more than Algebra 2, making sure to include math in your senior year
course work, and when you enter higher education, put at least one college-level
math course under your belt in the first year—no matter what your eventual
major.

112
 5. When you start to think seriously about postsecondary options, log on to college
and community college Web sites and look not so much for what they tell you of
how wonderful life is at Old Siwash, but what they show you of the kinds of
assignments and examination questions given in major gateway courses you will
probably take. If you do not see these indications of what to expect, push! Ask
the schools for it! These assignments and questions are better than SAT or ACT
preparation manuals in terms of what you need to complete degrees.

6. See if your nearest community college has a dual-enrollment agreement with your
school system, allowing you to take significant general education or introductory
occupational courses for credit while you are still in high school. Use a summer
term or part of your senior year to take advantage, and aim to enter higher
education with at least six credits earned this way—preferably more.

7. You are ultimately responsible for success in education. You are the principal
actor. The power is yours. Seize the day—or lose it!

Given the story lines of The Toolbox Revisited, it is obvious that students are partners in their
own education fate, and shouldn’t wait around for someone else to do something to them or for
them.

Public discourse, part 1: Dissonant data and their discontents

The second set of messages is for those who engage in public discourse on education in general,
secondary-to-postsecondary transitions, and ultimately, degree completion rates (with all stops
in-between). We have some problems here.

Foremost among these problems is the sheer volume of dissonant statistics that are thrown
around about student progress, and all the labels of “at risk,” “minimal college-qualified,” and
"failure" that get pasted to populations in the process. The “at risk” labeling default has gone so
far as to turn students into “patients,” whose “illnesses” must be diagnosed and followed up with
early intervention, intensive intervention, and continuous intervention (Seidman 2005, p. 298)
that may even continue after graduation—and for “a modest fee” (p. 299). The data dissonance
and deficit language cloud perceptions and preclude constructive policy. We all have
considerable cleaning up to do.

On any given day, the public will be offered a half-dozen different statistics on high school
graduation rates, college-enrollment rates, college completion rates, grades, and time-to-degree.
The data will appear in respectable academic journals in articles that were reviewed by peers
who often are experts on statistical technique and (at best) novices on the data sources. Or they
will appear in publications and on Web sites of respectable organizations, even though they were
never reviewed by anyone outside the organization. Anything that appears between respectable

113
covers is taken as authoritative, and once it moves into the mainstream press and onto the home
pages, we read the headlines but not the footnotes. Inference runs rampant.2

For any of these statistics, we never ask who is in the denominator: that is, who are we counting,
and who are we not counting—and how? As a consequence, what often pours out are scare
stories that make for good press and bad policy. The bad data-driven scare story, in fact, has
become the preferred narrative. We are scared by stagnant high school graduation rates over a 30
year period during which the size of the grade cohorts declined significantly then expanded
dramatically with the baby boom echo, and during which we witnessed increased immigration
from countries with mandatory school attendance ages much lower than ours. By an alternative
view, it’s amazing we have maintained a stable high school graduation rate (the quality of high
school curriculum aside). The same alternative view could be advanced with reference to rates
of postsecondary credentialing: It’s remarkable we are maintaining the same degree-granting
rates in the face of significantly higher enrollments (unless, of course, we are awarding an excess
of cheap degrees).

Dissonance by age and season: A plea for honest tracking

The source of many unnerving postsecondary stories is one of the most grievous errors in
analyses of student progress: including in the denominator students who started their
postsecondary careers at age 29, 36, or 47 along with the mass of students who entered the
postsecondary universe at age 18 or 19. Common sense says that a 19-year-old and a 31-year-
old are on completely different life trajectories, and the national data from the Beginning
Postsecondary Students longitudinal studies back up the common sense. When the newspaper
story uses the term, "college students," most adults think of their children, not their brother-in-
law or their coworker. Community and four-year college administrators know the difference,
and provide academic programs, scheduling and services for those different populations.

But what are they to do when the press and the news Web sites complain that nearly half of
entering students do not return for their second year or that the graduation rate is only 50 percent
(thus assuming everyone else is a dropout), and they are called before legislative committees and
boards of trustees to explain? There is an enormous difference by age at entry to the
postsecondary system in these measures, and an even greater distortion when one restricts the

2
For example, consider the following statement in a respectable publication: "One of the key reasons that low
income students have such low completion rates in postsecondary education is that many work long hours in order
to be able to afford college. They struggle to balance work with part-time enrollment in college . . ." (Allen,
Goldberger, and Steinberg 2004, p. 222). The data source for this assertion is the National Postsecondary Student
Aid Study of 2000, a one-year snapshot that includes no "completion rate" data. Analyses of the NPSAS 2000 data
files show that the statement does not reach the threshold of justification unless one divides the population by age
bracket. At that point, one finds that, and among traditional-age students (presumably the group referenced by the
scare), poor kids are no more likely to be working longer hours at their jobs than anyone else, though they are more
likely to use their wages for education expenses. That, at least, is an honest statement—for a snapshot population.
And it is not what we really would want to know.

114
definitions of what it means to "return to" or "graduate from" to those who started in the fall
term, full-time, and who came back to or earned a degree from the same school. That
denominator knocks out half of traditional-age students from the calculation, and denies the
realities of geographic mobility that the Bureau of the Census—let alone NCES longitudinal
studies—has documented for the 20-something population (Schachter 2004; Adelman 2005b).
Policies designed to "retain" students who have already moved to another state or who are
de facto ghosts by not being included in the retention denominator in the first place are, at best,
wastes of energy.

What is not a waste of energy is the task of developing more universal and efficient student
tracking systems, and recapturing the headlines from the mongers of scare. There are those who
will not accept NCES national longitudinal studies on the grounds that they are samples (no
matter how scientific the sampling design), that we can only afford to start one every six or 10
years, and then have to wait for people to age and accumulate academic history by which time,
the grievance goes, "the data are old." Impatient to simulate instant longitudinal cohorts, they
impute sequences of data from different sources and with denominators that include
"projections," and produce shock data that cannot be validated by any sensible reference points,
e.g., that only 18 percent of ninth-graders will earn an associate or bachelor’s degree within the
subsequent ten years (National Center for Public Policy and Higher Education 2004).3

But even the best of state tracking systems and the services of the independent National Student
Clearinghouse information system that currently (2005) covers about 2900 institutions (and cites
a burgeoning interest in including high schools in the universe), will not produce the wealth of
information that a NELS:88/2000 or a Beginning Postsecondary Students study yield. This essay
cannot recommend policy in these matters, but it can recommend creativity and cooperation,
serious reading of the papers and reports from Florida’s tracking system (e.g., Whitfield and
Howat 1999; Goodman, Latham, Copa, and Wright 2001; Goodman, Copa, and Wright 2004;
Johnson, Coles, and Thomas 2004), and reflection followed by activistic innovation, and will
wager that the long-term results look better than the scare stories assume.

Public discourse, part 2: The language we use

Language does more than reflect reality—it creates reality as well. There are considerable
problems with the language used in describing what happens to students in our education system,
and our choice of terms sets boundaries and colors of reality. The boundaries and colors, in turn,
condition the terms of policy. Let us illustrate with a few paired terms. These are contrary
rhetorics, and this study frankly admits to taking sides in their contention. But it does so in order
to urge a positive tone that, not so by-the-way, legislators, superintendents of schools, college
presidents and other leaders would prefer to use. The language of leadership is a “can do”
language, not a punitive rhetoric.

“Attrition” versus “Persistence.” When “attrition” is the governing term, we worry about
students who (it appears) leave school or college, and seek explanations for departure that have
3
There has never been a national longitudinal study of ninth-graders. But we do have a national longitudinal study
of eighth-graders—the NELS:88/2000—with transcripts, not imputations, projections, and dubious math. If we
follow these eighth-graders, including high school dropouts, all the way through to age 26, ultimately 34 percent
earned either an associate or bachelor’s degree (see the full account offered in Appendix L, table L12). That
percentage at least puts us in range of doing better. If we accept the putative (and utterly false) 18 percent, we risk
abandoning all hope and effort.

115
included theories of organizational turnover (Bean 1983) and failures of academic and social
integration (Tinto 1987). At the first sign of exit—even though the student may return—we turn
to negativity. There has to be something wrong here, we say. The student was “at risk,” the
institution did not respond—we witness a cycle of blame.

When “persistence” is the governing term, we take our directions from students. What did they
do that resulted in attainment? What structures of opportunity do we need to offer so that future
students can follow the same paths? What do we think works? Can we test it out? This is a far
more positive approach. This essay endorses it: Drop "attrition," embrace "persistence"!

“Retention” versus “Persistence.” Institutions “retain”; students “persist.” If our language is

governed by “retention” all we see are institutions determined to hold on to students, keeping
them in places that may be unproductive, at all costs, and for the sake of their public ratings. If
our language follows student “persistence,” on the other hand, we see those individuals making a
series of rational choices that take advantage of the opportunities offered by institutions so as
both to discover true interests and reach productive ends. Tinto would not object if the rhetoric
of leaving an institution was turned into a saga of discovery. Students may go elsewhere; they
may take extended time off from higher education; but ultimately they may judge the change as
positive and not a result of failure (Tinto 1987, pp. 132–33). In the rhetoric of “retention,”
students are passive: Something is done to them, and that "something" assumes a deficit model.
Under the rhetoric of “persistence” they are actors shaping their fate, with a model of success in
mind. Wouldn’t anyone rather have success?

“Pipelines” versus “Paths.” As Bach et al. (2000) noted—and others have followed—there is
no linear path to a degree, particularly for students who start out in community colleges. The
default “pipeline” metaphor, used to describe presumably linear learning experiences and
environmental sequences, is wholly inadequate to describe student behavior. Pipelines are
unidirectional closed spaces, and under the “pipeline” metaphor students are passive creatures
(as in “retention”) swept along or dropping out of the space completely through leaks at the
joints. But student behavior doesn’t look like that at all: It moves in starts and stops, sideways,
down one path to another and perhaps circling back. Liquids move in pipes; people don’t.

At the high school level, for example, a student can acquire momentum in science through a
combination of statistics and biology, on the one hand, or physics and calculus, on the other.
These are different paths, but who is to say that, once in a four-year or community college, these
students could not move in very different directions? The students entering a community college
with the statistics and biology background thinking they were heading for further study in allied
health fields could easily discover business and computer programming, and transfer to a four-
year college to pursue an academic program in management information systems with both
quantitative background and empirical habits of mind born of study in the life sciences. The
paths to degrees offer many such intersections.

Under the “pipeline” metaphor, we look for easy (sometimes glib) causalities along a single line
of explanation. “Paths,” on the other hand, allow for multiple analyses and discoveries of tools

116
that suggest (but do not predict) productive routes to education goals. This essay obviously
endorses "paths."

Reiterations

Virtually all reviewers of drafts of this study recommended a concluding reiteration of its major
themes and conclusions. Three configurations of themes and conclusions stand out in response:

First, there was a story about curriculum, the content of schooling, that was compelling in its
secondary school dimensions in the original Tool Box, and is even more compelling now on both
secondary and postsecondary stages. What you study, how much of it, how deeply, and how
intensely has a great deal to do with degree completion. All of this is common sense, but
requires equitable execution with emphasis on primary tools, which in this story means that:

• Secondary schools must provide maximum opportunity-to-learn, by which we

mean not merely course titles, but course substance. If we seek better preparation
for any kind of postsecondary education—occupational, professional or
traditional arts and sciences—we have to ratchet up the challenge of content.

• Postsecondary institutions have got to be active players and reinforcers at the

secondary school level—particularly in partnership with schools that are not
providing or inspiring students—with opportunity to learn at those ratcheted-up
levels of content. Pep talks, family visits, recruitment tours, and guidance in
filling out application and financial aid forms are not enough.

• Indeed, the first year of postsecondary education has to begin in high school, if
not by AP then by the growing dual enrollment movement or other, more
structured current efforts (for examples, see Hughes, Karp, Fermin and Bailey
2005). If all traditional-age students entered college or community college with a
minimum of 6 credits of "real stuff," not fluff, their adaptation in the critical first
year will not be short-circuited by either poor placement or credit overload.

Second, this curriculum story, joined by nuances of attendance patterns that turn out to have
significant leverage, continues into higher education. These features of the saga of degree
completion are rarely attended to.

• It’s not merely getting beyond Algebra 2 in high school any more: The world
demands advanced quantitative literacy, and no matter what a student’s
postsecondary field of study—from occupationally-oriented programs through
traditional liberal arts— more than a ceremonial visit to college-level mathematics
is called for.

• Academic advisors and counselors have to target every first-time student for at
least 20 additive credits by the end of the first calendar year of enrollment. We

117
 saw the same consequences in the original Tool Box, though now we understand
better that the chances of making up for anything less than 20 credits diminish
rapidly in the second year. Community colleges have some special challenges
here, given increasing rates of transfer among traditional-age students. With 6
credits of dual-enrollment course work, even part-time students can reach 20
credits in the first calendar year, and community colleges enroll the bulk of
traditional-age part-time students.

• Excessive no-penalty withdrawals and no-credit repeats do irreparable damage to

the chances of completing degrees. This phenomenon was also observed in the
original Tool Box. Twice advised, institutions might think very seriously about
tightening up, with bonuses of increased access and lower time-to-degree.

• More than incidental use of summer terms has proven to be a degree-completion

lever with convincing fulcrum. It’s part of the calendar-year frame in which
students are increasingly participating. Four-year and community colleges can
entice students into fuller use of summer terms with creative scheduling.

Third, in contract to their treatment in the mass of literature on academic progress, students are
explicit, rather than implicit, in The Toolbox Revisited. They are respected adults playing larger
roles in their own destinies in The Toolbox Revisited. What we call “variables” are not bloodless
abstractions: they are signs of what students do; and our messages are about where and when the
green lights and caution lights will flash along the paths toward degrees. While we trust that
school and college actions will not leave them behind, they have equal responsibilities.

Legacy

These are limited beginnings of change in the terms of the enterprise with which any reader of
this document is concerned. They are honest terms and do not pretend to predict, rather help us
draw a background tapestry against which we can judge just how well we are doing for our
children as they cross the cusp of adulthood. The terms derive from the story; the story derives
from the wisdom of the U.S. Department of Education in establishing and maintaining its
longitudinal studies; and our subsequent discussions and enlightenment derive from the
leadership of the National Center for Education Statistics in executing those studies and
providing us with archives of information that are the envy of other nations. All of this
constitutes an unmatchable legacy.

118
THIS PAGE INTENTIONALLY LEFT BLANK

119
 REFERENCES
Achieve, Inc. 2004. The Expectations Gap: A 50-State Review of High School Graduation
Requirements. Washington, DC: Author.

The American Diploma Project. 2004. Ready or Not: Creating a High School Diploma That
Counts. Washington, DC: Achieve, Inc., The Education Trust, and the Thomas B. Fordham
Foundation.

Adams, J.L. and Becker, W.E. 1990. "Course Withdrawals: A Probit Model and Policy
Recommendations." Research in Higher Education, vol. 31, no. 6, pp. 519–538.

Adelman, C. 1983. Devaluation, Diffusion, and the College Connection: A Study of High School
Transcripts, 1969-1981. Staff paper for the National Commission on Excellence in Education.
ERIC Document #ED228244.

Adelman, C. 1999. Answers in the Tool Box: Academic Intensity, Attendance Patterns, and
Bachelor’s Degree Attainment. Washington, DC: U.S. Department of Education.

Adelman, C. 2004a. Principal Indicators of Student Academic Histories in Postsecondary

Education, 1972-2000. Washington, DC: U.S. Department of Education.

Adelman, C. 2004b. The Story-Lines of Multi-Institutional Attendance. Paper presented at the

Annual Forum of the Association for Institutional Research. Boston, MA.

Adelman, C. 2005a. Moving Into Town—and Moving On: The Community College in the Lives
of Traditional-age Students. Washington, DC: U.S. Department of Education.

Adelman, C. 2005b. Geographic Mobility Among the ‘Swirlers’: Where Do They Come From?
Where Do They Go? Paper presented at the Annual Forum of the Association for Institutional
Research. San Diego, CA.

Adelman, C., Daniel, B., and Berkovits, I. 2003. Postsecondary Attainment, Attendance,
Curriculum and Performance: Selected Results from the NELS:88/2000 Postsecondary
Education Transcript Study. Washington, DC: National Center for Education Statistics.

Alexander, K., Riordan, C., Fennessey, J. and Pallas, A.M. 1982. “Social Background, Academic
Resources, and College Graduation: Recent Evidence from the National Longitudinal Survey.”
American Journal of Education, vol. 90, no. 4, pp. 315–333.

Alexander, K. and Pallas, A.M. 1984. "Curriculum Reform and School Performance: An
Evaluation of the ‘New Basics.’" American Journal of Education, vol. 92, no. 4, pp. 391–420.

Allen, D. and Nora, A. 1995. "An Empirical Examination of the Construct Validity of Goal
Commitment in the Persistence Process." Research in Higher Education, vol. 36, no. 5, pp. 509–
533.

120
Allen, L., Goldberger, S., and Steinberg, A. 2004. "Pathways to Postsecondary Credentials:
Schools and Programs That Blend Education and Employment," in Kazis, R., Vargas, J., and
Hoffman, N. (eds.), Double the Numbers: Postsecondary Attainment and Underrepresented
Youth. Cambridge, MA: Harvard Education Press, pp. 221–230.

Altonji, J. 1993. “The Demand for and Return to Education When Education Outcomes Are
Uncertain.” Journal of Labor Economics, vol. 11, pp. 48–83.

Altonji, J. 1996. "The Effects of High School Curriculum on Education and Labor Market
Outcomes." Journal of Human Resources, vol. 30, no. 3, pp. 409–438.

American Association of Collegiate Registrars and Admissions Officers. 2002. Academic

Transcripts and Records: Survey of Current Practices. Washington, DC: Author.

American Association of Universities. 2003. Standards for Success. Eugene, OR: Center for
Educational Policy Research.

American College Testing. 2004. Crisis at the Core: Preparing All Students for College and
Work . Iowa City, IA: Author.

Anaya, G. 1999. "College Impact on Student Learning: Comparing the Use of Self-Reported
Aims, Standardized Test Scores, and College Grades." Research in Higher Education, vol. 40,
no. 5, pp. 498–526.

Astin, A.W. 1993 What Matters in College: ‘Four Critical Years’ Revisited. San Francisco:
Jossey-Bass.

Astin, A. W. and Astin, H.S. 1993. Undergraduate Science Education: the Impact of Different
College Environments on the Educational Pipeline in the Sciences. Los Angeles, CA: Higher
Education Research Institute.

Astin, A.W. and Oseguerea, L. 2002. Degree Attainment Rates at American Colleges and
Universities. Los Angeles: Higher Education Research Institute

Bach, S.K. et al. 2000. "Student Attendance Patterns and Performance in an Urban
Postsecondary Environment." Research in Higher Education, vol. 41, no. 3, pp. 315–330.

Barth, P. and Haycock, K. 2004. "A Core Curriculum for All Students." In Kazis, R., Vargas, J..
and Hoffman, N. (eds.) Double the Numbers: Increasing Postsecondary Credentials for
Underrepresented Youth." Cambridge, MA: Harvard Education Press, pp. 35–46.

Bean, J P. 1983. "The Application of a Model of Turnover in Work Organizations to the Student
Attrition Process." Review of Higher Education, vol. 6, pp. 129–148.

Beattie, I.R. 2002. "Are All ‘Adolescent Econometricians’ Created Equal?: Race, Class and
Gender Differences in College Enrollment." Sociology of Education, vol. 75, no. 1, pp. 19–43.

Belsley, D.A., Kun, E., and Welsch, R. 1980. Regression Diagnostics. New York: Wiley.

121
Berkner, L. and Chavez, L. 1997. Access to Postsecondary Education for the 1992 High School
Graduates. Washington, DC: National Center for Education Statistics.

Berkner, L., He, S., and Cataldi, E.F. 2002. Descriptive Summary of 1995-96 Beginning
Postsecondary Students: Six Years Later. Washington, DC: National Center for Education
Statistics.

Bettinger, E.P. and Long, B.T. 2005. Addressing the Needs of Under-Prepared Student in Higher
Education: Does College Remediation Work? Cambridge, MA: National Bureau of Economic
Research. NBER Working Paper 11325.

Bishop, J.H. 2004. "Merit Scholarships for the Many: Doubling the Number of High School
Students Who Work Hard in High School." In Kazis, R., Vargas, J.. and Hoffman, N. (eds.).
Double the Numbers: Increasing Postsecondary Credentials for Underrepresented Youth."
Cambridge, MA: Harvard Education Press, pp. 87–100.

Boughan, K. 2001. The Rocky Road to Graduation: An Academic Career Flow Model for
Tracking Student Progress at Community Colleges. Paper presented at the annual conference of
the Northeast Association for Institutional Research. Cambridge, MA.

Braxton, J.M., Brier, E.M., and Hossler, D. 1988. "The Influence of Student Problems on Student
Withdrawal Decisions: An Autopsy on ‘Autopsy’ Studies." Research in Higher Education, vol.
28, no. 3, pp. 241–254.

Broh, C.A. 1991. "Half a Life is Better Than No Life at All: A Better Way to Measure Time to
Degree." AAHE Bulletin, vol. 44, no. 1, pp. 11–13.

Cabrera, A.F. 1994. "Logistic Regression Analysis in Higher Education: An Applied

Perspective." In Smart, J.C. (ed.), Higher Education: Handbook of Theory and Research, Vol. X.
New York: Agathon Press, pp. 225–256.

Cabrera, A.F. and La Nasa, S. M. 2001. "On the Path to College: Three Critical Tasks Facing
America’s Disadvantaged." Research in Higher Education, vol. 42, no. 2, pp. 119–149.

Cabrera, A.F., Burkum, K.R., and La Nasa, S.M., 2005. Pathways to a Four-Year Degree:
Determinants of Transfer and Degree Completion. In Seidman, A. (ed.), College Student
Retention: Formula for Student Success. Washington, DC: American Council on Education.
Praeger Series on Higher Education, pp. 155–214.

California Education Roundtable. 1997. Statement on Competencies in Mathematics Expected of

Entering College Students and Standards in Mathematics for California High School Graduates.
Long Beach, CA: Author.

Carroll, C.D. 1989. College Persistence and Degree Attainment for 1980 High School
Graduates: Hazards for Transfers, Stopouts, and Part-Timers. Washington, DC: National Center
for Education Statistics.

122
Catsambis, S. 1994. "The Path to Math: Gender and Racial-Ethnic Differences in Mathematics
Participation from Middle School to High School." Sociology of Education, vol 67, no. 3,
pp. 199–215.

Clubb, J.M., Austin, E.W., and Kirk, G.W. Jr. 1989. The Process of Historical Inquiry: Everyday
Lives of Working Americans. New York: Columbia University Press.

The College Board. 1998. High School Grading Policies. Research Notes #RN-04. New York
City: Author.

Conley, D.T. 2003. Mixed Messages: What State High School Tests Communicate About Student
Readiness for College. Eugene, OR: Center for Educational Policy Research.

Conley, D.T. 2005. College Knowledge: What It Really Takes for Students to Succeed and What
We Can Do to Get Them Ready. San Francisco: Jossey-Bass.

Curtin, T.R.., Ingels, S.J., Wu, S., and Heuer, R. 2002. User’s Manual: NELS:88 Base-Year to
Fourth Followup: Student Component Data File. Washington, DC: National Center for
Education Statistics.

Daniel v. State of California 1999. BC214156.

Dey, E.L. and Astin, A.W. 1993. “Statistical Alternatives for Studying College Student Retention:
A Comparative Analysis of Logit, Probit, and Linear Regression.” Research in Higher
Education, vol. 34, no. 5, pp. 569–581.

DesJardins, S.L., Ahlburg, D.A., and McCall, B.P. 1999. "An Event History Model of Student
Departure." Economics of Education Review, vol. 18, no. 3, pp. 375–390.

DesJardins, S.L., McCall, B.P., Ahlburg, D.A., and Moye, M.J. 2002. "Adding a Timing Light to
the ‘Toolbox.’" Research in Higher Education, vol. 43, no. 1, pp. 83–114.

DesJardins, S.L., Ahlburg, D.A., and McCall, B.P. 2002. "A Temporal Investigation of Factors
Leading to Timely Degree Completion." Journal of Higher Education. Vol. 73, no. 5,
pp. 555–581.

Dynarski, S. 2003. "Does Aid Matter? Measuring the Effect of Student Aid on College
Attendance and Completion." American Economic Review, vol. 93, no. 1, pp. 279–288.

Eaton, S.B. and Bean, J.P. 1995. “An Approach/Avoidance Behavioral Model of College Student
Attrition.” Research in Higher Education, vol. 36, no. 6, pp. 617–645.

Education Commission of the States. 2000. Building Bridges Not Barriers: Public Policies that
Support Seamless K-16 Education. Policy Brief P-16. Denver, CO: Author.

Education Commission of the States. 2001. The Progress of Education Reform 1999-2001:
High School Curriculum. Denver, CO: Author.

123
Elder, G.H. Jr., Pavalko, E.K., and Clipp, E.C. 1993. Working With Archival Data.
London (UK): Sage Publications.

Eno, D., McLaughlin, G.W., Brozovsky, P., and Sheldon, P. 1998. Predicting Freshman Success
Based on High School Records and Other Measures. Paper presented at the annual forum of the
Association for Institutional Research, Minneapolis, MN.

Ewell, P. 2004. "An Accountability System for ‘Doubling the Numbers.’" In Kazis, R.,
Vargas, J. and Hoffman, N. (eds.). Double the Numbers: Increasing Postsecondary Credentials
for Underrepresented Youth. pp. 101–112.

Florida Department of Education. 2005. Postsecondary Success Begins With High School
Preparation. Tallahassee, FL: Author. Data Trend #33 (March).

Garcia, P. 1994. Graduation and Time to Degree: A Research Note from the California State
University. Paper presented at the annual forum of the Association for Institutional Research,
New Orleans, LA.

Geiser, S. and Santelices, V. 2004. The Role of Advanced Placement and Honors Courses in
College Admissions. Berkeley, CA: Center for Studies in Higher Education. Research &
Occasional Paper Series, CSHE.4.04.

Georges, A. 2000. Financial Constraints and College Enrollment Decision. Paper presented to
the annual meeting of the Midwest Economic Association, Chicago.

Goodman, T.G., Latham, S.C., Copa, J.C., and Wright, D.L. 2001. Ready or Not, Here They
Come: A Longitudinal Analyses of the Effect of Acceleration and Remediation on Baccalaureate
Degree Attainment. Paper presented to the annual forum of the Association for Institutional
Research. Long Beach, CA.

Goodman, T.G., Copa, J.C., and Wright, D.L. 2004. Is Eight Enough? A Longitudinal Look at the
Progression of Community College Students Toward a Bachelor’s Degree. Paper presented to
the annual forum of the Association for Institutional Research. Boston, MA.

Haggerty, C. et al. 1996. National Education Longitudinal Study: 1988-1994 Methodology

Report. Washington, DC: National Center for Education Statistics.

Hair, J.F., Anderson, R.E., Tatham, R.L., and Black, W.C. 1995. Multivariate Data Analysis with
Readings. Englewood Cliffs, NJ: Prentice Hall.

Hanson G. R. 1998. Getting to the Heart of the Matter: How the Curriculum Affects Student
Retention. Paper presented at the annual forum of the Association for Institutional Research.
Minneapolis, MN.

Harmston, M.T. 2004. Cross-Validation of Persistence Models for Incoming Freshmen. AIR
Professional File, Number 93. Tallahassee, FL: Association for Institutional Research.

Haskins, L. and Jeffrey, K. 1990. Understanding Quantitative History. New York: McGraw-Hill

124
Hearn, J.C. 1992. "Emerging Variations in Postsecondary Attendance Patterns: An Investigation
of Part-time, Delayed, and Nondegree Enrollment." Research in Higher Education, vol. 33,
pp. 657–687.

Herzog, T.N. and Rubin, D.B. 1983. “Using Multiple Imputations to Handle Non-Response in
Sample Surveys,” in W.G. Madow, I. Olkin, and D.B. Rubin (eds.). Incomplete Data in Sample
Surveys. New York: Academic Press, vol. II, pp. 210–245.

Horn, L. 1997. Confronting the Odds: Students at Risk and the Pipeline to Higher Education.
Washington, DC: National Center for Education Statistics.

Horn, L. 1998. Stopouts or Stayouts? Undergraduates Who Leave College in Their First Year.
Washington, DC: National Center for Education Statistics.

Horn, L. and Kojaku, L.W. 2001. High School Academic Curriculum and the Persistence Path
Through College. Washington, DC: National Center for Education Statistics.

Houston, W. and Sawyer, R. 1991. “Relating Scores on the Enhanced ACT Assessment and the
SAT Test Batteries.” Dodge City, IA: American College Testing.

Hu, S. and Kuh, G.D. 2002. “Being (Dis)engaged in Educationally Purposeful Activities: The
Influences of Student and Institutional Characteristics.” Research in Higher Education, vol. 43,
pp. 555–575.

Hu, S. and St. John, E.P. 2001. "Student Persistence in a Public Higher Education System:
Understanding Racial and Ethnic Differences." Journal of Higher Education, vol. 72, no. 3,
pp. 265–296.

Hughes, K.L., Karp, M.M., Fermin, B.J. and Bailey, T.R. 2005. Pathways to College Access and
Success. Washington, DC: U.S. Department of Education.

Illinois Board of Higher Education. 2001. “Factors Affecting Persistence and Degree
Completion.” Report of the Commission on Persistence and Degree Completion.
Springfield, IL: Author.

Ingels, S. et al. 1994. NELS:88 Second Follow-Up Student Component Data File: Users
Manual. Washington, DC: National Center for Education Statistics.

Ishitani, T.T. and Snider, K.G. 2004. Longitudinal Effects of College Preparation Programs on
College Retention. Paper presented to the annual forum of the Association for Institutional
Research. Boston, MA.

Johnson, N., Coles, C. and Thomas, K. 2004. What Happened to the Other Half? Communicating
with Policymakers and the Public About Graduation Rates. Paper presented to the annual forum
of the Association for Institutional Research. Boston, MA.

125
Kahn, J.H. and Nauta, M.M. 2001. "Social-Cognitive Predictors of First-Year College
Persistence: the Importance of Proximal Assessment." Research in Higher Education, vol. 42,
no. 6, pp. 633–652.

Kao, G. and Tienda, M. 1998. "Educational Aspirations of Minority Youth." American Journal of
Education. Vol. 106, no. 3, pp. 349–384.

Karp, M, Bailey, T.R., Hughes, K.L. and Fermin, B.J. 2004. State Dual Enrollment Policies:
Addressing Access and Quality. Washington, DC: U.S. Department of Education.

Kazis, R. 2004. "Introduction," in Kazis, R., Vargas, J., and Hoffman, N. (eds). Double the
Numbers: Increasing Postsecondary Credentials for Underrepresented Youth. Cambridge, MA:
Harvard Education Press, pp. 1–15.

Kearney, G., Townsend, B., and Kearney, T. 1995. “Multiple Transfer Students in a Public
University: Background Characteristics and Interinstitutional Movements.” Research in Higher
Education, vol. 36, no. 3, pp. 323–344.

Kleiner, B. and Lewis, L. 2005. Dual Enrollment of High School Students at Postsecondary
Institutions: 2002-03. Washington, DC: National Center for Education Statistics.

Klopfenstein, K and Thomas, M.K. 2005. The Advanced Placement Performance Advantage:
Fact or Fiction? Paper presented at the annual convention of the American Economics
Association. Philadelphia, PA.

Knight, W.E. 2002. Toward a Comprehensive Model of Influences Upon Time to Bachelor’s
Degree Attainment. Tallahasse, FL: Association for Institutional Research. AIR Professional
File, No. 85.

Knight, W.E. 2004a. Student, Institutional, and State Policy Effects Upon Time to Bachelor’s
Degree Attainment. Paper presented to the annual forum of the Association for Institutional
Research. Boston, MA.

Knight, W.E. 2004b. Time to Bachelor’s Degree Attainment: An Application of Descriptive,

Bivariate, and Multiple Regression Techniques. IR Applications, volume 2. Tallahassee, FL:
Association for Institutional Research.

Kroc, R., Howard, R., Hull, P., and Woodard, D. 1997. Graduation Rates: Do Students’
Academic Program Choices Make a Difference? Paper presented at the annual forum of the
Association for Institutional Research. Orlando, FL.

Kuh, G.D. et al. 2001. NSSE Technical and Norms Report. Bloomington, IN: Indiana University
Center for Research and Planning.

Lee, V., Croninger, R.G., and Smith, J.B. 1997. "Course-Taking, Equity, and Mathematics
Learning: Testing the Constrained Curriculum Hypothesis in U.S. Secondary Schools."
Education Evaluation and Policy Analysis, vol. 19, no. 2, pp. 99–121.

126
Leigh, D.E. and Gill, A.M. 2003. “Do Community College Really Divert Students From Earning
Bachelor’s Degrees.” Economics of Education Review, vol. 22, no. 1, pp. 23–30.

Li, G. and Killian, T. 1999. “Students Who Left College: An Examination of Their
Characteristics and Reasons for Leaving.” Paper presented at the annual forum of the
Association for Institutional Research, Seattle, WA.

Light, A. and Strayer, W. 2002. "From Bakke to Hopwood: Does Race Affect College Attendance
and Completion?" The Review of Economics and Statistics, vol. 84, no. 1, pp. 34–44.

Long, J.S. 1997. Regression Models for Categorical and Limited Dependent Variables. Thousand
Oaks, CA: Sage Publications.

Lucas, S.R. 1999. Tracking and Inequality: Stratification and Mobility in American High
Schools. New York: Teachers College Press.

Lucas, S.R. and Berends, M. 2002. "Sociodemographic Diversity, Correlated Achievement, and
De Facto Tracking." Sociology of Education, vol. 75, no. 4, pp. 328–348.

Mallette, B.I. and Cabrera, A.F. 1991. "Determinants of Withdrawal Behavior: An Exploratory
Study." Research in Higher Education, vol. 32, no. 2, pp. 179–194.

Martinez, M. and Bray, J. 2002. All Over the Map: State Policies to Improve the High School.
Washington, DC: National Alliance on the American High School.

McCormick, A.C. 1999. Credit Production and Progress Toward the Bachelor’s Degree: An
Analysis of Postsecondary Transcripts for Beginning Students at 4-Year Institutions. Washington,
DC: National Center for Education Statistics.

McCormick, A.C. 2003. "Swirling and Double-Dipping: New Patterns of Student Attendance and
Their Implications for Higher Education." In King, J. E., Anderson, E.L., and Corrigan, M.E.
(eds.), Changing Student Attendance Patterns: Challenges for Policy and Practice.
San Francisco: Jossey-Bass. New Directions for Higher Education, no. 121, pp.13–24.

Medrich, E.A., Brown, C.L., and Henke, R.R. 1992. Overview and Inventory of State
Requirements for School Coursework and Attendance. Washington, DC: National Center for
Education Statistics.

Menard, S. 1995. Applied Logistic Regression Analysis. London (UK): Sage Publications.

Mezias, Stephen J. 1988. “Aspiration Level Effects: An Empirical Investigation,” Journal of

Economic Behavior and Organizations, vol. 10, pp. 389–400.

National Center for Education Statistics. 2002a. Statistical Standard 5-4: Tabular and Graphic
Presentations. http://nces.ed.gov/statprog/2002/std5_4.asp Washington, D.C.: Author.

National Center for Education Statistics. 2002b. Statistical Standard 4-4: Nonresponse Bias
Analysis. http://nces.ed.gov/statprog/2002/std4_4.asp Washington, D.C.: Author.

127
National Center on Public Policy and Higher Education. 2004. Educational Pipeline Policy
Alert.San Jose, CA: Author.

Nora, A., Barlow, E., and Crisp, G. 2005. “Student Persistence and Degree Attainment Beyond
the First Year in College.” In Seidman, A. (ed.), College Student Retention: Formula for
Success. Washington, DC: American Council on Education/Praeger, pp. 129–153.

Noxel, S. and Katunich, L. 1998. “Navigating for Four Years to the Baccalaureate Degree.”
Paper presented at the annual forum of the Association for Institutional Research. Minneapolis,
MN.

Okun, M., Taub, J. and Witter, R. 1986. “Age and Sex Differences in Negative Life Events and
Student Services Usage. Journal of College Student Personnel, vol. 27, no. 2, pp. 160–164.

Oltman, P. K. 1982. Content Representativeness of the GRE Advanced Tests in Chemistry,

Computer Science, and Education. GREB 81-2p. Princeton: Educational Testing Service.

O'Toole, D.M., Stratton, L.S., and Wetzel, J.N. 2003. "A Longitudinal Analysis of the Frequency
of Part-Time Enrollment and the Persistence of Students Who Enroll Part-Time". Research in
Higher Education, 44, No. 5, pp. 519–537.

Pallas, A.M. and Alexander, K.L. 1983. "Sex Differences in SAT Performance: New Evidence of
the Differential Course Work Hypothesis." American Educational Research Journal, vol. 20,
no. 2, pp. 165–182.

Pascarella, E.T. and Terenzini, P.T. 1991. How College Affects Students: Findings and Insights
from Twenty Years of Research. San Francisco: Jossey-Bass.

Patrick, W.J. 2001. "Estimating First-Year Student Attrition Rates: An Application of Multilevel
Modeling Using Categorical Variables." Research in Higher Education, vol. 42, no. 2,
pp. 151–170.

Pedhazur, E.J. 1982. Multiple Regression in Behavioral Research: Explanation and Prediction.
New York: Holt, Rinehart, and Winston.

Peng, C.-Y. J., So, T-S H., Stage, F.K., and St. John, E.P. 2002. "The Use and Interpretation of
Logistic Regression in Higher Education Journals: 1988-1999." Research in Higher Education,
vol. 43, no. 3, pp. 259–293.

Perkhounkova, Y., McLaughlin, G.W., and Noble, J.P. 2004. Factors Related to Persistence of
Freshmen, Freshman Transfers, and Nonfreshman Transfer Students. Paper presented to the
annual forum of the Association for Institutional Research. Boston, MA.

Perna, L.W. 1998. “The Contribution of Financial Aid to Undergraduate Persistence.” Journal of
Student Financial Aid. Vol. 28, no. 3, pp. 25–40.

Peterson, T. 1985. "A Comment on Presenting Results from Logit and Probit Models."
American Sociological Review, vol. 50, no. 1, pp. 130–131.

128
Pike, G.R. and Saupe, J.L. 2002. "Does High School Matter? An Analysis of Three Methods of
Predicting First-Year Grades." Research in Higher Education, vol. 43, no. 2, pp. 187–207.

Pitter, G.W., LeMon, R.E., and Lanham, C.H. 1996. Hours to Graduation: A National Survey of
Credit Hours Required for Baccalaureate Degrees. Paper presented to the Annual Forum of the
Association for Institutional Research. Albuquerque, NM.

Porter, S. 2005. "What Can Multilevel Models Add to Institutional Research?" In

Coughlin, M.A. (ed.), Applications of Intermediate/Advanced Statistics in Institutional Research.
Tallahassee, FL: Association for Institutional Research, pp. 110–131.

Purcell, J. and Clark, A. 2002. Assessing the Life and Times of First-Semester Academic
Failures. Paper presented at the annual conference of the Southern Association for Institutional
Research. Baton Rouge, LA.

Qian, Z. and Blair, S.L. 1999. "Racial/Ethnic Differences in Educational Aspirations of High
School Seniors." Sociological Perspectives, vol. 42, no. 4: 605–625.

Ratcliff, J.L. et al. 1995. The Effect of Coursework Patterns, Advisement, and Course Selection
on the Development of General Learned Abilities of College Graduates. University Park, PA:
Center for the Study of Higher Education.

Robbins, S.B. et al. 2004. "Do Psychosocial and Study Skill Factors Predict College Outcomes?:
A Meta-Analysis." Psychological Bulletin, vol. 130, no. 2, pp. 261–288.

Ronco, S.L. 1996. How Enrollment Ends: Analyzing the Correlates of Student Graduation,
Transfer and Dropout with a Competing Risks Model. AIR Professional File No.61.
Tallahassee, FL: Association for Institutional Research.

Saupe, J.L., Smith, T.Y., and Xin, W. 1999. Institutional and Student Characteristics in Student
Success: First-term GPA, One-Year Retention, and Six-Year Graduation. Paper presented at the
Annual Forum of the Association for Institutional Research. Seattle. WA.

Schachter, J.P. 2004. Geographic Mobility: 2002 to 2003. Washington, DC: U.S. Census
Bureau. Current Population Reports, P20-549.

Schutz, G. and Malo, G. 2003. Predictors of One-Year Retention in the Tennessee Degree
Attainment Tracking Database. Paper presented to the Annual Forum of the Association for
Institutional Research. Tampa, FL.

Seidman, A. 2005. "Where We Go From Here: A Retention Formula For Student Success." In
Seidman, A. (ed.), College Student Retention: Formula for Student Success. Washington, DC:
American Council on Education. Praeger Series on Higher Education, pp. 295–316.

Shireman, R. 2004. "‘Rigorous Courses’ and Student Achievement in High School: An Options
Paper for the Governor of California." Berkeley, CA: Center for the Study of Higher Education,
CSHE 13.04.

129
Simpson, W.A. 1987. "Tracking Students Through Majors." Journal of Higher Education,
vol. 58, no. 3, pp. 323–343.

Singer, J.D. and Willett, J.B. 1991. "Modeling the Days of Our Lives: Using Survival Analysis
When Designing and Analyzing Longitudinal Studies of Duration and the Timing of Events."
Psychological Bulletin, vol. 110, no. 2, pp. 268–290.

Smith, T.Y. 1995. The Retention Status of Underrepresented Minority Students: An Analysis of
Survey Results from Sixty-Seven U.S. Colleges and Universities. Paper presented to the Annual
Forum of the Association for Institutional Research. Boston MA.

Stage, F.K. 1989. "Motivation, Academic and Social Integration, and Early Dropout.” American
Educational Research Journal, vol. 26, no. 3, pp. 385–402.

Stage, F.K. and Hossler, D. 1989. "Differences in Family Influences on College Attendance Plans
for Male and Female Ninth Graders." Research in Higher Education, vol. 30, no. 3,
pp. 301–315.

St. John, E.P et al. 2004. “What Difference Does a Major Make? The Influence of College Major
Field on Persistence by African American and White Students.” Research in Higher Education,
vol 45, no 3, pp. 209–232.

St. John, E.P., Paulsen, M.B., and Starkey, J. B. 1996. "The Nexus Between College Choice and
Persistence." Research in Higher Education, vol. 37, no. 2, pp. 175–220.

Stedman, J.B. and Jordan, K.F. 1986. Education Reform Reports: Content and Impact.
Washington, DC: Congressional Research Service. Report #86-56EPW, pp. 12-41.

Stoecker, J., Pascarella, E.T., and Wolfle, L.M. 1988. “Persistence in Higher Education: A 9-year
Test of a Theoretical Model.” Journal of College Student Development, vol. 29, no.3,
pp.196–209.

Szafran, R.F. 2001. “The Effect of Academic Load on Success for New College Students: Is
Lighter Better?” Research in Higher Education, vol. 42, no. 1, pp. 27–50.

Texas Business and Education Coalition. 2002. “Master Scholars” Program. Austin, TX: Author.

Thomas, G.E., Alexander, K. L., and Eckland, B.K. 1979. "Access to Higher Education: The
Importance of Race, Sex, Social Class, and Academic Credentials." School Review, vol. 87,
pp. 133–156.

Thomas, S.L. and Heck, R.H. 2001. “Analysis of Large-Scale Secondary Data in Higher
Education Research: Potential Perils Associated with Complex Sampling Designs.” Research in
Higher Education, vol. 42, no. 5, pp. 527–539.

Tinto, V. 1987. Leaving College: Rethinking the Causes and Cures of Student Attrition. Chicago:
University of Chicago Press.

130
Tinto, V. 1988. "Stages of Student Departure: Reflections on the Longitudinal Character of
Student Leaving." Journal of Higher Education, vol. 59, no. 4, pp. 438–455.

Titus, M.A. 2004. “An Examination of the Influence of Institutional Context on Student
Persistence at 4-Year Colleges and Universities: a Multilevel Approach.” Review of Higher
Education, vol. 45, no. 7, pp. 673–699.

University of North Carolina Board of Governors. 2000. “Proposed Increase in the UNC Board
of Governors’ Minimum Undergraduate Admissions Requirements.” [No city], NC: Author.

Velez, W. 1985. "Finishing College: The Effects of College Type." Sociology of Education,
vol. 58, pp. 191–200.

Venezia, A., Kirst, M.W., and Antonio, A. L. 2003. Betraying the College Dream: How
Disconnected K-12 and Postsecondary Education Systems Undermine Student Aspirations.
Stanford, CA: the Bridge Project.

Venezia, A. et al. 2005. The Governance Divide: A Report on a Four-State Study on Improving
College Readiness and Success. San Jose, CA: National Center for Public Policy and Higher
Education.

Volkwein, J.F. and Lorang, W. G., "Characteristics of Extenders: Full-Time Students Who Take
Light Credit Loads and Graduate in More Than Four Years." Research in Higher Education, vol.
37, no. 1, 1996, pp. 43–67.

Waits, T., Setzer, J.C., and Lewis, L. 2005. Dual Credit and Exam-Based Courses in U.S. Public
High Schools: 2002-03. Washington, DC: U.S. Department of Education.

Walpole, M. 2003. "Socioeconomic Status and College: How SES Affects College Experiences
and Outcomes." Review of Higher Education, vol. 27, no. 1, pp. 45–73.

Whitfield, D. and Howat, C. 1999. Six Years on the Road: Routes to a Baccalaureate Degree.
Paper presented to the annual forum of the Association for Institutional Research. Seattle, WA.

Wirt, J. et al. 2000. The Condition of Education 2000. Washington, DC: National Center for
Education Statistics.

Wirt, J. et al. 2005. The Condition of Education 2005. Washington, DC: National Center for
Education Statistics.

Zhai, M., Ronco, S.L., Feng, X, and Feiner, A.J. 2001. Assessing Student Course-Taking Patterns
and Their Impact on College Persistence. Paper presented to the annual forum of the Association
for Institutional Research. Long Beach, CA.

Zhang, Z, and Richards, R.S. 1998. Prediction and Analysis of Freshman Retention. Paper
presented at the Annual Forum of the Association for Institutional Research. Minneapolis, MN.

Zucker, B. and Dawson, R. 2001. Credits and Attainment: Returns to Postsecondary Education
Ten Years After High School. Washington, DC: National Center for Education Statistics.

131
 APPENDIX A
Changes in Demography, Geo-Demography, and Postsecondary
Entry From the High School & Beyond/Sophomores to the NELS:88/2000

Table A1. Contrasts in the percentage distributions of selected demographic backgounds,

postsecondary plans and postsecondary entry behavior of 1982 and 1992 12th-graders

1982 (HS&B/So) 1992 (NELS:88/2000)

Parents' highest level

of education

Less than high school graduate 12.2 (0.47) 8.1 (0.59)

High school graduate 32.4 (0.66) 19.5 (0.79)
Some postsecondary 29.8 (0.58) 41.2 (0.92)
Bachelor's degree 12.7 (0.45) 17.0 (0.71)
Graduate or first professional 12.9 (0.55) 14.2 (0.74)

Race/ethnicity

White 77.5 (0.77) 72.2 (1.32)

African-American 12.6 (0.63) 12.0 (0.96)
Latino 7.1 (0.33) 10.1 (0.89)
Asian 1.5 (0.16) 4.3 (0.37)
American Indian 1.3 (0.19) 1.5 (0.43)

Proportion of same race

in childhood neighborhood

10 percent or less 4.5 (0.26) 6.3 (0.43)

11-50 percent 8.4 (0.43) 11.7 (0.63)
51-85 percent 16.8 (0.59) 24.1 (0.82)
More than 85 percent 70.3 (0.83) 57.9 (1.13)

Proportion from second

language backgrounds

All 12th-graders 5.5 (0.30) 9.4 (0.84)

White 1.8 (0.19) 2.0 (0.29)

African-American 1.2 (0.29) 3.1 (1.41)
Latino 40.5 (2.18) 50.9 (3.12)
Asian 55.1 (4.11) 48.0 (3.48)
American Indian 17.9 (5.12) 28.5 (16.7)

See notes at end of table

132
Table A1. Contrasts in the percentage distributions of selected demographic backgounds,
postsecondary plans and postsecondary entry behavior of 1982 and 1992 12th-
graders - continued
1982 (HS&B/So) 1992 (NELS:88/2000)

Became parent by age 20 9.0 (0.41) 10.8 (0.65)

Urbanicity of high school community

Urban 20.2 (1.26) 28.3 (1.53)

Suburban 49.3 (1.55) 40.9 (1.72)
Rural 30.6 (1.31) 30.8 (1.66)

Census division of high school

New England 6.7 (0.61) 4.6 (0.80)

Mid-Atlantic 16.7 (0.57) 14.6 (1.01)
East North Central 20.6 (0.60) 17.4 (0.84)
West North Central 7.9 (0.41) 8.5 (0.56)
South Atlantic 16.1 (0.62) 17.1 (0.90)
East South Central 5.3 (0.20) 6.6 (0.52)
West South Central 10.1 (0.51) 11.5 (0.70)
Mountain 4.8 (0.31) 6.3 (0.59)
Pacific 11.8 (0.55) 13.3 (0.78)

Highest level of education planned

High school or less 19.5 (0.60) 5.9 (0.52)

Postsecondary vocational 23.9 (0.58) 12.0 (0.76)
2+ year of college 15.7 (0.46) 14.2 (0.64)
Bachelor's degree 22.6 (0.53) 35.5 (0.81)
Graduate degree 18.3 (0.53) 32.6 (0.90)

Entered postseconary education

within 8.5 yearsa of scheduled 59.0 (0.67) 77.3 (0.87)
high school graduation

Of those who continued to

postsecondary, entered with
7 months of high school graduationb 77.1 (0.74) 82.6 (0.82)

a
8.5 years is the maximum time between the modal high school graduation date and the concluding date for the longitudinal
study for the NELS:88/2000, that is, June 1992 through December 2000. The HSB/So is cut to match.
b
See definition of "no delay" in glossary, p 186
NOTES: Where applicable, columns may not add to 100.0 percent due to rounding. Standard errors are in parentheses.
Weighted Ns: High School & Beyond/Sophomores = 3.3M; NELS:88/2000 = 2.6M.
SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore cohort (NCES 2000-194) and
NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402).

133
 APPENDIX B

Principal Features of the NCES Grade-cohort Longitudinal Studies

There are four grade-cohort longitudinal studies designed and conducted by the National Center
for Education Statistics. Three of these have been completed:

• National Longitudinal Study of the High School Class of 1972 (NLS-72),

started with a cohort of seniors in the spring of 1972, concluded in 1986;
• High School and Beyond, with a cohort of seniors in 1980, concluded in 1986,
and another cohort of sophomores in 1980 (HS&B), concluded in 1993;
and
• National Education Longitudinal Study of 1988, initiated with an eighth-grade
class in 1988 (NELS:88), concluded in 2000.

The data from these studies are available in both public release and restricted (license required)
form on CD-ROM, with electronic code books (ECBs) listing all variables, with descriptions and
distributions.

The fourth, the Education Longitudinal Study of 2002 (ELS:2002), starting with a sample of
20,000 10th-grade students in the spring of 2002, is in progress.

Curtin, Ingels, Wu, and Heuer (2002) offer a figure with a temporal presentation of the four
longitudinal studies,1 highlighting their component and comparison points. Each of the studies
begins with a national probability sample involving a stratified sample of schools and a random
sample of students within the target grade in those schools. Schools in minority communities are
over-sampled. In some cases, the samples are refreshed at later points in the longitudinal study
(NELS:88 in 1990 and 1992)2 and, in some cases, augmented at a later point (NLS-72 in 1973).

Each of these longitudinal studies includes a great deal more information than what is used in
The Toolbox Revisited. Not all of them are comparable in terms of the depth with which various

1
Curtin, T.R., Ingels, S.J., Wu, S., and Heuer, R (2002). National Education Longitudinal Study of 1988: Base-Year
to Fourth Follow-up Data File User’s Manual (NCES 2003-323). Washington, DC: U.S. Department of Education,
National Center for Education Statistics (http://nces.ed.gov/pubs2002/2002323.pdf, p.3).
2
As Lucas and Berends (2002) remind us, the high schools attended by students in the NELS study are not a true
representative sample. The reason stems from the decision to begin the longitudinal study in the eighth grade.
When students move into high schools from the eighth grade (or, in some districts, from the ninth grade), it is
impossible for the sampling framework to be based on high schools. Roughly 1,000 middle schools and junior high
schools became 4,000 high schools by grade 10, and there was no way to follow the students into all 4,000 of those
schools. So the NELS sample was refreshed in 10th and 12th grades to reflect the total enrolled population in those
years (Ingels et al. 1994), and , by necessity, the refreshing was based in the high schools in which the vast majority
of the existing NELS cohort students were enrolled.

134
topics are explored. The surveys of the NLS-72 were focused wholly on students, whereas those
of the subsequent longitudinal studies included parents, teachers, and secondary school
administrators. The cognitive tests administered in the 12th grade to the NLS-72 cohort were
administered in the 10th and 12th grades to subsequent cohorts, thus enabling measures of
intellectual growth. High school course-taking for the NLS-72 was summarized and reported by
the school, whereas for the HS&B/Sophomore cohort and NELS:88/2000 high school course-
taking was derived directly from transcripts. And the postsecondary transcripts for the
NELS:88/2000 were used to fill in missing information from the high school transcripts in that
cohort. Labor market histories were far more detailed in the NLS-72 and HS&B/Sophomore
cohort than they were for the NELS:88/2000. Military records exist for the NLS-72 but not for
any subsequent study. Student financial aid information included an unobtrusive Pell Grant file
for the HS&B/Sophomore cohort, and that for the NELS:88/2000 included data from the
National Student Loan Data System (though this file requires more cleaning and reconstruction
in order to be truly helpful).3

Lastly, the shift from paper-and-pencil survey response forms to computer-assisted telephone
interviews (CATI) in the1990s constricted the range of questions asked (e.g., there was no time
to ask students about reasons for changing majors, reasons for transferring from one college to
another, and degrees of satisfaction with different aspects of postsecondary experience), whereas
the NLS-72 paper survey forms covered these topics in some depth.

Nonetheless, the archives of these data sets are the richest we have to explore the nature of
secondary and postsecondary education and its consequences in the early adult life histories of
Americans over the past 30 years.

3
See the brief discussion of financial aid data in the NELS:88/2000 in Adelman, C. Principal Indicators of Student
Academic Histories in Postsecondary Education, 1972-2000. Washington, DC: U.S. Department of Education,
2004, p. 98.

135
 APPENDIX C

Differences Between the High School & Beyond/Sophomore Cohort and

NELS:88/2000 High School Record Variables, and Limited Imputation
Procedures for the NELS:88/2000
The principal portions of the data sets in question are secondary school transcripts and allied
information derived from survey questionnaires and computer assisted telephone interviews
(CATI). The secondary school transcripts were collected and coded by the same contractor for
both data sets, but the returns were very different. The High School & Beyond/Sophomore
cohort began with a stratified national sample of secondary schools, all of which agreed to
participate in the project and provide transcripts, access to teachers and administrators. The
NELS:88/2000 began with a national stratified sample of schools with eighth grades, all of
which agreed to participate in the project and provide appropriate information. As noted above
in Appendix B, when students moved into high schools, the NELS sample was refreshed in10th
and 12th grades to reflect the total enrolled population in those years (Ingels et al. 1998), and, by
necessity, the refreshing was based in the high schools in which the vast majority of the existing
NELS cohort students were enrolled. The analysis file created for this study includes the12th
grade refreshed sample, but some of the high schools that were not part of the original stratified
sample did not provide transcripts.

At the postsecondary level, on the other hand, the transcript return rate increased from 89 percent
for the HS&B/So to 93 percent for the NELS:88/2000 (Curtin, Ingels, Wu, and Heuer 2002), and
the proportion of postsecondary participants with complete records increased from 88 to 97
percent.

Editorial processes

The postsecondary transcript-based versions of both data sets were edited by the same person
(the author of this study), and with the same rules, a feature that increases reliability at the price
of intra-coder bias. If one thinks of two sets of archival documents: secondary and
postsecondary transcripts for two cohorts, i.e., four data files, in only one did the editor actually
see the paper documents submitted by institutions, namely, the postsecondary transcript files of
the NELS. These artifacts contained information that allowed corrections and fill-ins for missing
information on the secondary school transcript files, e.g., state location of high school, high
school graduation date, high school diploma type, Advanced Placement course credits (by
examination), and SAT and ACT test scores. Hence, the presentation of secondary school
transcripts on the restricted file with the postsecondary transcript data (NCES CD#2003-402) is
more accurate than it is on earlier NELS restricted files. After a year of use, a supplement to
NCES 2003-402 was issued in June 2004, containing further refinement of both high school and
college transcript-derived variables, in addition to new derived variables.

136
Missing data and limited imputation

The analysis files for both this study and Answers in the Tool Box used high school transcripts
with complete records for at least grades 10–12. But the NELS:88/2000 transcript data, as well
as test score data, fell short of the HS&Beyond/So coverage, even after preparation of the 2004
Supplement to the original NELS Postsecondary Files that included the second round of
corrections and fill-ins of missing information on the high school transcripts.

Table C1. Unweighted percentage of 1982 and 1992 12th-graders missing

precollegiate academic performance data

After
HS&B/So NELS:88/2000 Supplement
Performance Variables

Curriculum Measures 18% 14% 14%

Class Rank/GPA 13 20 17

Senior Test Score 9 14 14

One or more measures 18 24 21

SOURCE: High School & Beyond/Sophomore cohort (NCES #2000-194); NELS:88/2000 Postsecondary
Transcript Files (NCES 2003-402 and Supplement).

Class rank/GPA, the combined high school class rank/grade point average quintile, is the major
problem in the comparison displayed in table C1. In both data sets this variable (called
"CLSSRNKQ" for programming purposes) was constructed in the same way, but the input data
initially yielded a far smaller universe for the NELS than was the case for the HS&B/So.

The construction of this variable begins with class rank, expressed as a percentile, for students
whose high school graduating classes are greater than 10. Class rank was chosen for the base
reference because it overrides variability in local grading practices. Not all high schools
compute class rank, so for a significant percentage of students, this datum is missing. For these
missing cases (as well as for the students from very small high schools), the variable construction
turns to high school grade point average (where available, and only for students with three years
or more of course work in all high schools attended). In both data sets, all known cases of high
school GPA were weighted and set out in quintiles. The missing cases of class rank, by quintile,
were then substituted by the GPA quintile (the correlation between the two quintile scales was
.84).

However, there are two significant differences in data input between the HS&B/So and the
NELS:88/2000. First, the HS&B/So grade point average was for academic courses only, where-
as in the NELS, the GPA applied to all courses. Second, by the time of the NELS histories, some

137
high schools evidently were indicating neither class rank nor cumulative GPA of any kind on
transcripts (either that or the original data entry for the NELS high school transcripts did not

compute GPA). While 13 percent of the HS&B/So students were missing the combined
CLSRANKQ variable, half again of that proportion (20 percent) of the NELS students lacked a
CLSSRNKQ value (see Glossary for further details and citations).

This difference would have serious consequences in the replication of the original Tool Box
hypotheses. The universe of NELS students with positive CLSSRNKQ values was initially
smaller than those for the curriculum variable (HSCURRQ) and senior year test score
(SRTSQUIN). Since the multivariate analyses of both Tool Box studies depend on positive
values for all three variables, the composite variable built from a partition of the three would
overestimate the value of CLSSRNKQ at the expense of the other two components. In preparing
the analysis file for this study, plausible values for missing cases of CLSSRNKQ were imputed
when values for both curriculum quintile (HSCURRQ) and test score quintile (SRTSQUIN) were
positive and equal, e.g., both were in the third quintile. In these cases, CLSSRNKQ was
assigned the same value as the other two variables. The impact of this imputation strategy
lowered the unweighted missing cases of CLSSRNKQ from 20 to 17 percent, and the
unweighted cases of NELS:88/2000 students missing one or more of the high school
performance variables from 24 to 21 percent (see table C1).

Bias and potential imputation.

Even after this revision, a problem remained, and a multiple imputation procedure was followed
(Herzog and Rubin 1983). In the original edited version of the NELS high school transcript data
that appears on NCES data CD 2003-402 (March 2003), and modified by its June 2004
supplement, some 17.3 percent of the students were missing CLSSRNKQ, 13.8 percent were
missing HSCURRQ, and 13.5 percent were missing SRTSQUIN. Under NCES statistical
standards, CLSSRNKQ would have to be evaluated, since the proportion of students with non-
missing values was less than 85 percent.1

The formula used to determine the nonresponse (i.e., missing) bias in these three variables was:

A*(b-c) where A is the percent of students missing data, b is the percent of those
missing data who did NOT earn a bachelor’s degree, and c is the percent of those
not missing data who did not earn a bachelor’s degree.

To obtain the relative bias with respect to the estimate, the result of this formula is divided by b.

The initial nonresponse bias rates for the three variables were:

Bias Relative bias

HSCURRQ 0.015 0.021

1
See NCES 2002b.

138
 CLSSRNKQ 0.034 0.049
SRTSQUIN 0.039 0.048

In other words, for example, if the estimate for the curriculum variable was based only on those
with positive values, it would overestimate the percent who did not earn bachelor’s degrees by 2
percent. These are all small biases, and would not threaten tests of statistical significance. But the
missing cases were less than random. Students who never entered postsecondary education by
age 26 or 27 were overrepresented in the missing cases of each variable. This is not surprising,
since those who never graduated from high school or who were still enrolled in high school in
the year of their scheduled graduation (1992) would not show complete transcript records
in1992.

In order to increase the number of students with positive and plausible values for all three
components of the Academic Resources variable, another limited imputation was undertaken,
focused on class rank/GPA. The first question identified, of all students who were in the 12th
grade in 1992 and for whom the file showed positive values for both the curriculum
(HSCURRQ) and test score (SRTSQUIN) variables:

1) those who also showed a positive quintile value for class rank/GPA
(CLSSRNKQ), and were in the same quintile on all three measures. 16.6%
2) those who were in the same quintile on both curriculum and test score
measures, also had a positive quintile value for class rank/GPA, but the rank of this
quintile differed from the others by + 1. 13.4
3) those who were in the same quintile on both curriculum and test score
measures, but were missing class rank/GPA. 2.7
4) those who were missing class rank/GPA, and whose quintile positions
on curriculum and test score measures differed by 1. 2.5
5) those who were in the same quintile on both curriculum and test score
measures, also had a positive quintile value for class rank/GPA, but the rank of this
quintile differed from the others by > 1. 5.8
6) those who were not in the same quintile on both curriculum and test score
measures, whether or not class rank/GPA was missing. 59.0

The focus of imputation was on those members of groups 3 and 4 whose high school transcripts
showed 12 or more academic Carnegie units. This small group was then examined for prima
facie anomalies in variables such as postsecondary attendance, selectivity of first institution of
attendance (if any), and postsecondary remedial work. There were none. An algorithm was then
developed that assigned a value to CLSSRNKQ, as follows: (1) where HSCURREV =
SRTSQUIN, the class rank/GPA quintile was assigned the same value; (2) where
the difference between HSCURREV and SRTSQUIN was +1, CLSSRNKQ was assigned the
lower value in half the cases, and the higher value in the other half.

The result reduced the proportion of missing (“nonresponse”) cases of CLSSRNKQ from 17.3 to
15.8 percent. The nonresponse bias rates improved as follows:

Before imputation After imputation

139
 Bias rate 0.034 0.019
Relative bias 0.049 0.026

The consequent reconstruction of Academic Resources

The proportion of students with non-missing values for all three components of the Academic
Resources index rose from 70 to 72 percent as a result of the imputation described in above.
However small the increase, it requires a recalculation of the comparative weights of the three
components of Academic Resources, in the following steps:

a) A logistic regression is run with bachelor’s degree attainment as the dependent

variable, the three components as independent variables, and a universe limited to1992 12th-
graders. The ratios of the standardized beta coefficients for each of the independent variables
produce the weights for each variable in the composite. The changes between the earlier version
of these weightings and the post-imputation version are small, but required, as table C2 shows.

Table C2. Differences in ratios of standardized betas for the three components of the
Academic Resources composite, before and after limited imputation of
missing class rank/GPA data for 1992 12th-graders

Partition weight

Component of Academic Resources Before imputation After imputation

Curriculum 41.9 42.4

Class rank/GPA 32.9 32.5
Senior test score 25.2 25.1

SOURCE: NELS:88/2000 Postsecondary Transcript files (NCES 2003-402 and Supplement).

140
 b) Each of the three components is then multiplied by its weight brought forward from
the logistic model. So, for example, a student in the third quintile of HSCURRQ is credited with
1.272 points (3*0.424). The points are added, then weighted by the enhanced NELS high school
transcript weight for those in the 12th grade in 1992 (F4F2HWT), then set out in quintiles of
Academic Resources. The difference in distributions of the values of Academic Resources,
before and after imputation, are indicated in table C3.

Table C3. Differences in distribution of 1992 12th-graders across the values of the
Academic Resources composite variable, before and after limited
imputation of missing class rank/GPA data

Percent distribution of students

Values of Academic Resources Before imputation After imputation

Missing 29.9 27.9

Highest quintile 14.3 14.6
2nd quintile 13.3 14.3
3rd quintile 14.2 14.7
4th quintile 13.9 13.6
Lowest quintile 14.3 14.9
SOURCE: NELS:88/2000 Postsecondary Transcript files (NCES 2003-402 and Supplement).

141
THIS PAGE INTENTIONALLY LEFT BLANK

142
 APPENDIX D

Technical Issues

Accuracy of estimates and standard errors

There are different kinds of statistics in The Toolbox Revisited, and all of them are estimates
derived from student samples. Two kinds of error occur when samples are at issue: errors in
sampling itself, particularly when relatively small subpopulations (for example, American
Indians) are involved; and nonsampling errors. Even in surveys as large as the three grade-
cohort longitudinal studies used in this monograph, sampling errors can affect estimates of
statistical significance.

Non-sampling errors are more serious. A good example of a non-sampling error would be the
fact that transcripts are missing for some students in all three grade-cohort studies. The
transcripts are missing either because the student did not tell the interviewer that he or she
attended the school (and there were no transfer credits on another transcript to identify the
school); the school refused to send the transcript; the school could not find the transcript; the
information sent by the school was not really a transcript; or while the student may have enrolled
at the school he or she never registered for courses and did not generate a record. In this case,
we can mitigate the effect of missing transcripts by differential weighting of the population, and,
indeed, for both the High School and Beyond/Sophomore and NELS:88/2000 files, the analyst is
given a choice of weights, one of which is confined to students with complete records (see the
discussion of weights and flags below). Weighting, though, will not address the panoply of
nonsampling errors.

The effects of sampling and non-sampling errors ripple through databases. To judge the accuracy
of any analysis, one needs to explicate and judge these effects. When the unit of analysis is the
student, this is a straightforward issue because the original samples in the longitudinal studies
consisted of students. For example, when questions are asked about the proportion of students
who earned credits in an aggregate category of courses (e.g., table 21), the questions are about
nonrepetitive behaviors of the students who were sampled. The descriptive comparisons in The
Toolbox Revisited dealing with non-repetitive student behaviors require invocation of the
Students’t statistic to determine whether the difference between two independent estimates is
significant. The formula for computing Students’ t values is:
(P1 - P2)
t =  (se12 + se22)

where P1 and P2 are the estimates to be compared and se1 and se2 are the corresponding standard
errors. In this case, if t >1.96, one has a statistically significant difference at p <.05, a standard
marker. For the judgments of statistical significance in all cross-tabulations in this document, an
Excel template developed by MPR Associates for the production of reports to the National
Center for Education Statistics, was used.

The formula becomes more complex, however, for multiple comparisons among categories of an
independent variable such as race/ethnicity. For multiple comparisons, the critical value for t

143
rises depending on the number of comparisons that can be made in the family of the independent
variable. For race/ethnicity presented in five categories, there are10 possible comparisons, so the
significance level of each test must be p < .05/10 or p < .005. To determine the significance level
of t values in any comparison of means or proportions, the result should be matched against
standard published tables of significance levels for two-tailed hypothesis testing.

When estimates are not independent, a covariance term must be added to the Students’ t formula
(P1 - P2)_______
t =  (se12 + se22) -2(r)se1se2

where r is the correlation between the two estimates. The determination of correlations requires
a statistical software package, such as SAS or SPSS, and the invocation of proper weights for the
comparison.

Because none of the longitudinal studies invoked in The Toolbox Revisited was based on a simple
random sample of students, the technique for estimating sampling error involves a more complex
approach known as the Taylor series method. To produce Taylor series standard errors, the
estimates presented used AM, a program developed by Jon Cohen and associates at the American
Institutes for Research under contract to the National Center for Education Statistics.
Flags and weights

Each of the grade-cohort studies used in this monograph carries a complex set of flags and
weights to mark the populations for which estimates are to be generated. The selection of these
flags and weights is very important for both the accuracy and meaningfulness of estimates.

For the postsecondary transcript sample of the High School and Beyond/Sophomore cohort
(HS&B/So), the process was somewhat complex. Using the weights for the first follow-up
survey (1982, the scheduled 12th-grade year for this cohort), three postsecondary transcript
weights were developed. The first was based on a ratio of the sum of weights for all students in
the 1982 panel who subsequently (in surveys of 1984, 1986 or 1992) claimed to have attended a
postsecondary institution to the sum of weights for those for whom a transcript validating the
claim was subsequently received. The ratio was then modified by factors derived from the
stratification cells in the 1982 survey design to create multipliers that were applied to the raw
weights for the students for whom transcripts were received or for whom postsecondary
attendance was imputed from survey storylines. This is a generous formulation for all likely
postsecondary participants.

The second High School and Beyond/Sophomore weight involved the same procedure as the first
but a more restrictive ratio applied to those students for whom a true postsecondary transcript
was received. These students are more than “likely” participants; they are “known participants.”
The third weight followed the same procedure as the second, but confined the population to only
those students with complete postsecondary records (i.e., no missing transcripts). This weight is
used in analyses of credit production and grades, since complete records are necessary for the
analysis of both these features of student academic history. These weights are labeled PSEWT1,
PSEWT2, and PSEWT3, respectively.

144
To accompany these weights for the comparisons that hold the population to students who were
in the 12th grade in 1982, a special flag, SENRFLAG, was constructed from variables in the
HS&B/So that described student status in 1982. Using the given flag for participation in the
1982 cohort sample would be insufficient and not wholly accurate because not all students were
in the 12th grade in 1982, e.g., students who graduated early from high school in 1981. But there
were also students who were labeled “early graduates” on the data set (and thus candidates for
exclusion from a 12th-grade flag) whose high school graduation date was listed as 1982. Early
graduates were excluded, and erroneously labeled “early graduates” were included in the
population with SENRFLAG = 1. If these students were not participants in the 1982 panel (even
if postsecondary transcripts were received) their weight = 0. Using the 1982 panel weight alone
without this flag will not produce an accurate universe of 1982 12th-graders.

For all calculations of HS&B/So data in this document, SENRFLAG = 1, and the appropriate
PSE weight invoked.

The weights and flags for the NELS:88/2000 are more complex, still, because the cohort, established in the eighth
grade, was “refreshed” twice: first, to be representative of the census of 10th-graders in 1990,
and second, to be representative of the census of 12th-graders in 1992. The weights deriving
from the 1992 12th-grade refreshing are at the core of weights subsequently developed for the
postsecondary transcript sample. The same three postsecondary weight types developed for the
High School and Beyond/Sophomores were employed here, but in combination with the 12th-
grade (second follow-up survey, or F2) weight and the student’s presence in the final (2000)
survey panel, F4. In addition, a set of weights based on the NELS high school transcripts in
combination with the three postsecondary weight types also was developed when questions arise
concerning the relationship between secondary school variables derived from high school
transcripts and postsecondary variables derived from postsecondary transcripts.

The most frequently used NELS:88/2000 weights in The Toolbox Revisited are:

F4F2P2WT For all known postsecondary participants who were 12th-graders in 1992.
F4F2HP3W For all postsecondary participants with complete records who were 12th-
graders in 1992 and for whom high school transcripts are also part of the
file.
F4F2P3WT For all postsecondary participants with complete records who were 12th-
graders in 1992.

As in the case of the High School and Beyond/Sophomore cohort, a special flag was developed
for 12th-graders in 1992. The existing flag on the NELS:88/2000 files excluded over 250
students who, in fact, were awarded high school diplomas in the spring of 1992 and who carry
positive weights for the panel (the descriptive windows of the Electronic Code Book offer no
reasons or clues for this anomaly). These students are included in the flag, GRADE12A, used in
this monograph.

The weighted Ns for samples used in a table are provided in the notes to the tables. Even if the
same weight and flag is used on two tables, the weighted Ns may differ slightly because missing

145
values in a particular variable are excluded from the calculations.
Multivariate analyses

For all multivariate analyses in this monograph, special procedures were employed in accordance
with the complex sampling designs of NCES longitudinal studies. These procedures are in the
spirit, though not exactly to the letter, of those discussed and recommended by Thomas and Heck
(2001). Thomas and Heck recommended alternative ways of producing both regression models
and their adjusted standard errors in a single step, as opposed to the two-stage procedure used in
this study. The reader should be aware that the software employed in this study, AM, correctly
estimates standard errors associated with complex, cluster samples.

For any model, an adjusted weight based on the population with non-missing values on all
variables in the model was calculated, in the following steps:

1. A weight appropriate to the question was selected. For example, for

determinants of transfer from a community college to a four-year college, the
weight for NELS students with received postsecondary transcripts, F4F2P2WT,
was chosen.
2. A simple tabulation of the dependent variable was then run for students
who evidenced positive (non-missing) values for all variables in the model. Call
this universe A. The selected weight was invoked.
3. The log of the program for step 2 produces both unweighted and weighted
Ns for universe A.
4. The selected weight is then adjusted by what Thomas and Heck (2001)
refer to as NORMWT, i.e., the weighted N / unweighted N of universe A.
5. The selected weight is then multiplied by the NORMWT, thus adjusting
the effective sample size.
6. The example would look as follows:

F4F2P2WT / (f4f2p2wtA / unweighted N for A configuration of variables)

7. The result becomes a variable in its own right, a weight with a name, e.g., COREWT1.

A root design effect (DEFT) reflects the effect of departures from simple random sampling, and
in the methodology reports for the NELS:88/2000, is calculated for sub-populations on each of a
selection of variables (Haggerty et al. 1996; Curtin, Ingels, Wu, and Heuer 2002). For any
logistic model in The Toolbox Revisited, a DEFT based on the population with non-missing
values on all variables in the model was calculated in order to adjust the standard errors produced
by statistical packages such as SAS (used in the production of this study). The DEFT is
calculated in three steps:

1. The requisite data for a simple standard error are produced by the same equation used for
NORMWT, and set out as follows:

146
 ______
p(1-p) = s.e.
N

2. A matching Taylor series standard error is produced by AM software with the

same dependent variable and equation, with the population filtered by positive
(non-missing) values for the same variables used in the equation in step 1. The
Taylor series s.e. takes into account both stratum and primary sampling unit in
combination with the weight selected, hence accounting for the complex sampling
design.

3. The DEFT = Taylor Series s.e. / simple s.e.

Every discrete multivariate analysis has a unique DEFT. The DEFTs for the NELS:88/2000 are
rather substantial, e.g., 1.83, reflecting not only the original sampling design in 1988 but also the
successive "refreshings" of the sample in 1990 and (for the analyses in this monograph) in 1992.1
They are used to adjust the standard errors in the multivariate analyses, and hence reduce the
likelihood of overestimating the effects of independent variables. The effect of the DEFT also is
reflected in the production of the F and t statistics, for which the formulas used are:

______ 2
s.e. x DEFT =F
and
_________
F
DEFT2 = t

In the logistic models employed in The Toolbox Revisited, the level of significance of the t
statistic—p—for a two-tailed test is determined by reference to a standard table of critical values
of t that can be found in any statistics textbook.

Collinearity data

This study includes Appendix J with tolerance statistics for collinearity for all the logistic model
tables in order to assure the reader that the independent variables included in the models do not
exhibit a nearly perfect linear relationship, in other words, that they do not overlap to such an
extent that analysis of each variable, taken singly, is impossible. There are sometimes situations
in which one variable is part of the definition of another, e.g., family income in relationship to
socioeconomic status or multi-institutional attendance in relation to transfer. In cases such as
these, collinearity statistics are produced as by-products of Ordinary Least Squares regressions
for the variables in the logistic models.

Of these by-products, tolerance is the statistic of choice: It measures how well one independent
variable can be predicted by other independent variables in the model, and is equal to 1-R2 for
1
Curtin, Ingels, Wu, and Heuer (2002) report the mean DEFT for all students in the fourth follow-up (2000) survey
of the NELS as 1.954 (p. 209). The range of DEFTs in The Toolbox Revisited is 1.76 to 2.19.

147
that variable. Ideally one would want this predictive relationship to be close to 1. The literature
(e.g., Belsley, Kun, and Welsch 1980) suggests that there are no hard and fast rules for
determining collinearity using either tolerance or its inverse, variance inflation, but that, in
general, a tolerance reading of .20 or less evidences collinearity problems. This study adopted a
tolerance threshold of 0.50, i.e., any reading above that threshold indicates no serious collinearity
problem. Any indication below that point required further investigation of the independent
variable(s) in question through correlation matrices. If the variables in question were part of the
constructs of other variables, a choice was made as to which variable would be dropped.
Dropping independent variables is one of the recommended options when potential
multicollinearity arises (Knight 2004).

148
 APPENDIX E

Populations Included and Excluded From the Study Universe

The population subject to analysis in The Toolbox Revisited consisted of 1992 12th-graders who
subsequently attended a four-year college at any time through December 2000, who earned their
high school diploma by December 1996, who presented complete high school transcript records
and a senior year test score, whose postsecondary records were complete, and whose
socioeconomic status was known. But there are other students who participated in the 1992
NELS:88/2000 survey and who subsequently attended a four-year college at some time who are
not part of this population principally because (a) one or more of these data elements were
missing or (b) because they were not in the 12th grade in 1992.

It is natural to ask about the demographic characteristics of the students who are excluded from
the analysis because they are missing data or because they were not in the 12th grade in 1992,
compared with the demographic characteristics of the students who are included. In order to
compare distributions of this comparative demography, we include all 1992 survey participants.

Considering those who attended a four-year college at some time but were nonparticipants
because one or more of the key data elements were missing, key contrasts include:

• A higher proportion of minority students than white students;

• A higher proportion of students who were 20 years old or older in 1992
than those who were less than 20 years old;
• A higher proportion of students from the lowest third of family income
distribution than those from higher income levels;
• A higher proportion of first-generation students than those whose parents
had either some college or who had earned at least a bachelor’s degree;
• A higher proportion of nonnative speakers of English than those from
English monolingual backgrounds;
• A higher proportion of those with three or more siblings than those who
were either only children or had one or two siblings;
• A higher proportion of those who became parents by age 20 than those
who did not;
• A higher proportion of those who had been retained in grade at least once
than those who were never held back; and
• A higher proportion of those who came from high schools in urban areas
than those from suburban or rural areas.

Some 60 percent of this group was missing only one of the key data elements, but it was a very
important one: a high school transcript.

Of nonparticipants because they were not in the 12th grade in 1992, we notice distinctively
higher proportions by age (20 years or older), origin in a second language household (though not
among nonnative speakers of English), and parental status, and among those who had been
retained in grade and those who earned General Education Diplomas.

149
Table E1. Percentage distribution of participants in the 1992 survey of the
NELS:88/2000 by participation status in the Tool Box Revisited universe, by
demographic and schooling background characteristics
Participants in Nonparticipantsa Nonparticipanta who
in the universe who where who were not
Characteristics of this study 12th-graders in 1992 12th-graders in 1992

All students 77.8 (1.11) 20.6 (1.05) 1.6 (0.33)

By gender

Male 76.5 (1.57) 21.3 (1.46) 2.2 (0.57)

Female 79.2 (1.35) 19.8 (1.34) 1.0 (0.22)

By race/ethnicity

White 81.1 (1.08) 18.0 (1.07) 0.9 (0.20)

African-American 64.2 (4.44) 30.0 (4.04) 5.7 (2.72)
Latino 66.2 (4.11) 29.6 (3.99) 4.2 (1.19)
Asian 73.2 (5.09) 26.6 (5.09) 0.1 (0.09)
American Indian 53.6 (12.8) 43.1 (13.0) 2.4 (2.30)

By age in 1992

Missing 48.5 (9.15) 51.5 (9.15) #

20 or older 41.6 (8.56) 47.9 (9.52) 10.4 (4.86)
Under 20 79.2 (1.06) 19.3 (1.01) 1.5 (0.34)

By family income

Missing 70.8 (3.02) 27.1 (2.55) 2.2 (1.24)

Highest third 81.5 (1.44) 17.3 (1.41) 1.2 (0.37)
Middle third 80.1 (1.79) 19.2 (1.79) 0.7 (0.17)
Lowest third 71.3 (1.74) 25.2 (2.60) 3.5 (1.12)

By postsecondary
generational status

Missing 66.2 (4.08) 33.4 (4.07) 0.4 (0.24)

First generation 73.7 (2.73) 23.8 (2.74) 2.5 (0.68)
Parents had some college 78.3 (1.72) 19.2 (1.60) 2.5 (0.74)
Parents earned bachelor's 81.2 (1.34) 18.1 (1.32) 0.7 (0.27)

By second language background

Nonnative speakers 68.8 (3.78) 28.9 (3.73) 2.3 (0.81)

From second language
households 68.3 (5.48) 22.5 (4.94) 9.2 (4.06)
Monolingual English 79.0 (1.12) 19.7 (1.08) 1.3 (0.35)

See notes at end of table

150
Table E1. Percentage distribution of participants in the 1992 (second follow-up) survey of the
NELS:88/2000 by participation status in The Tool Revisited universe, by demographic
and schooling background characteristics--continued
a a
Participants in Nonparticipants Nonparticipants
the universe who were who were not
Characteristics of this study 12th-graders in 1992 12th-graders in 1992

By number of siblings

None 76.0 (4.37) 23.3 (4.39) 0.6 (0.31)

One or two 81.8 (1.15) 17.2 (1.13) 1.0 (0.24)
Three or more 71.8 (2.04) 25.5 (1.87) 2.7 (0.84)

By parenthood

Parent by age 20 64.6 95.38) 22.5 (4.07) 12.9 (4.62)

Not a parent by age 20 78.3 (1.10) 20.5 (1.07) 1.2 (0.25)

Held back in grade at least once

Missing 59.2 (6.33) 38.5 (6.21) 2.3 (1.11)

Yes 62.9 (4.85) 30.4 (4.54) 6.7 (2.64)
No 79.9 (1.07) 18.9 (1.04) 1.2 (0.31)

By urbanicity of high school

location

Urban 69.9 (2.36) 27.1 (2.15) 2.9 (0.99)

Suburban 80.1 (1.58) 19.0 (1.57) 0.9 (0.28)
Rural 83.9 (1.71) 14.9 (1.71) 1.2 (0.29)

b
By type of high school diploma

Standard 79.0 (1.07) 20.5 (1.06) 0.5 (0.13)

GED 8.7 (2.56) 23.6 (6.49) 67.7 (7.10)
a
Non-participants are defined as students who were either (a) missing data on senior year test score
and/or socioeconomic status, (b) missing an in-scope high school transcript, (c) graduated from high
school in January 1997 or later, (d) presented an in

b
A tiny percentage who received a certificate of attendance was included with the GEDs.
# Rounds to zero.
NOTES: Standard errors are in parentheses. Universe consists of students who participated in the
1992, 1994, and 2000 follow-ups of the NELS:88/2000 and who attended a four-year college at some
time. Weighted N=1.45M.

SOURCE: National Center for Education Statistics, NELS:88/2000 Postsecondary Transcript Files,
NCES 2003-402 and Supplement.

151
THIS PAGE INTENTIONALLY LEFT BLANK

152
 APPENDIX F

Gradations of Academic Intensity of High School Curriculum

The following figure sets forth the 31 gradations (in descending value) of academic curriculum
intensity and quality as used in the development of the Academic Resources index and variable
for the NELS:88/2000 cohort. The figures in the boxes represent the minimum rounded number
of Carnegie units required for the gradation on a given row. Where a cell is empty, there are no
minimum requirements. Where a cell indicates "none" (for remedial math and remedial
English), it means that no remedial work is allowed for that gradation. Where the cell for AP
courses indicates zero, that means the student did not take any AP courses, not a minimum. For
the NELS:88/2000 cohort, computer science was not nearly as widely offered as it is today.
Therefore, computer-related credits were brought into play only to disaggregate lumps in the
distribution. Total high school academic credits is an empirically-derived factor that comes into
play only in the very lowest gradations.

The basic five-subject credit thresholds were constructed in the course of examining the edited,
coded transcript data for students who were known high school graduates with graduation dates
through Dec.31, 1996. The editorial process paid particular attention to all cases that showed
less than 16 total high school credits. Where the evidence strongly suggested dissonance with
other variables in the student’s record, all transcript records from that student’s school were
examined. Where nonstandard credit metrics were found, they were adjusted with reference to
state standards for high school graduation (Medrich, Brown, and Henke 1992), and major
components (e.g., mathematics, English, etc.) multiplied or divided by as much as (but no more
than) two. For example, when a group of students from the same high school showed 40–45
Carnegie units in a state that required 20 for an academic diploma, the editorial process cut those
40–45 units in half—across all subjects in which they were given. The editorial process also
Windsorized cases of total Carnegie unit counts above 32, adjusting the major components down
one-by-one, and dropped fragmentary transcripts with less than 6 Carnegie unit counts.

As noted in the parallel appendix in the original Tool Box:

These gradations of academic intensity and quality are based on the history of
one national high school class that was scheduled to graduate in 1982. The next
graduating class for which we possess similar data is that of 1992. While the
specific number of Carnegie units, APs, and remedial indicators might change,
the basic form and principles of the gradations will probably not change. This
presentation of the possibilities of high school curricular attainment is criterion-
referenced: theoretically, everybody can reach gradation level #1. (p. 114)

The account of curriculum for the class of 1982 had 40 gradations. This account, for the class of
1992, has 31. One implication of the shrinking number of gradations is that, in fact, more
students were moving up the academic intensity ladder, clustering at higher criterion-referenced
levels.

Table F1 presents the actual mean number of Carnegie units earned in core academic fields,
irrespective of the theoretical thresholds, for students in each of the five quintiles of academic
intensity derived from the 31 more detailed gradations.

153
 Figure 3. Curriculum components of the 31 gradations of the high school academic
intensity measure of the NELS:88/2000, by Carnegie unit minimums

Figure 3. Curriculum components of the 31 graduations of the high school academic intensity measure of the
NELS:88/2000
Total
Foreign Hist or Highest Remed Remed Comput Academ
Graduation English Math Science Langs Soc Stu Math Math English APs Science Units
1 3.75 3.75 >2.0* >2.0 >2.0 >Alg2 None None >1 >0
2 3.75 3.75 >2.0 >2.0 >2.0 >Alg2 None None >0
3 3.75 3.75 >2.0 >2.0 >2.0 >Alg2 None None 0 1.0
4 3.75 3.75 3.0 >2.0 >2.0 >Alg2 None None
5 3.5 3.0 2.0 2.0 2.0 >Alg2 None None >1
6 3.5 3.0 2.0 2.0 2.0 >Alg2 None None >0
7 3.5 3.5 2.0 2.0 2.0 >Alg2 None None 0 0.5
8 3.5 3.0 2.0 2.0 2.0 >Alg2 None None 0 1.0
9 3.0 3.0 2.0 2.0 2.0 Alg2 None None >0
10 3.5 3.5 2.0 2.0 2.0 Alg2 None None 0 >0
11 3.5 3.5 2.5 2.0 2.0 Alg2 None None 0
12 3.0 2.0 1.0 1.0 >Alg2 None None >0
13 3.0 2.5 2.0 1.0 2.0 >Alg2 None None 0
14 3.0 2.5 2.0* 2.0 >Alg2 None None 0
15 3.0 2.5 2.0* 2.0 2.0 Alg2 None None 0 >0
16 3.0 2.5 2.0 2.0 2.0 Alg2 None None 0
17 3.0 2.5 1.0 1.0 2.0 <Alg2 None None 0
18 3.0 3.0 1.5 1.0 1.5 <Alg2 None
19 2.5 3.0 1.5 1.0 1.5 Alg2 None
20 2.5 2.5 1.5 0.5 1.0 <Alg2 0 >12
21 2.5 2.5 2.0 1.0 Alg2 Net O >0 >12
22 2.5 2.5 1.0 1.0 Net O >0 >10
23 2.5 2.0 2.0 1.5 <Alg2 Net O >12
24 2.5 2.0 2.0 1.5 <Alg2 Net O 1.0
25 2.5 2.0 1.0 0.5 Alg2 Net O None
26 2.5 2.0 1.0 <Alg2 None None >12
27 2.5 2.0 1.0 1.0 Net O
28 2.5 1.5 1.0 0.5 Net 1
29 2.5 1.5 1.0 0.5
30 2.0 1.0 0.5 0.5
31 =>6
NOTES: (1) Net 1 means the sum of total mathematics credits minus remedial mathematics credits was 0.5 or less; Net
0 means the sum of total mathematics credits minus remedial mathematics credits was more than 0.5

(2) An asterisk in a cell for science credits indicates core laboratory science (biology, chemistry, physics)
(3) When the distribution of students across these levels is weighted and then aggregated to quintiles, the quintile
nd rd th
breaks are as follows: 1-8 (highest quintile), 9-15 (2 quintile), 16-20 (3 quintile), 21-25 (4 quintile), and 26-31
(lowest quintil
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402).

154
Table F1. Of 1992 12th-graders with complete high school transcripts, mean Carnegie units earned in
core high school academic fields, pecent of students whose highest level of high school
mathematics was above Algebra 2, and mean number of Advanced Placement (AP) c

Core high school academic curriculum fields

Academic English Math Core lab Foreign History Computer Percent Total AP
intensity science languages and social science with courses
quintile studies highest
math
above
algebra 2
Highest 4.27 4.10 3.20 3.09 3.70 0.74 96.4 0.644
2nd 4.17 3.81 2.71 2.23 3.62 0.56 64.7 0.68
3rd 4.23 3.11 1.99 1.98 3.47 0.59 0 0.003
4th 4.10 2.98 1.36 0.74 3.44 0.61 0.71 0.019
Lowest 3.43 1.81 0.94 0.62 2.82 0.28 0.05 0.006
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES

155
THIS PAGE INTENTIONALLY LEFT BLANK

156
 APPENDIX G
Logistic Models for Two Alternative Presentations of
High School Academic Background

Table G1. Logistic account of factors associated with earning a bachelor's degree in the
history of 1992 12th-graders who attended a four-year college at any time,
version 2: Demographic and high school background using the three components
of Academic Resources as discrete variables

Adjusted
Parameter standard
Variable estimate error t p Delta-p Tolerancea
Intercept -4.7580 0.6530 3.33 0.02
Curriculum quintile 0.3782 0.0738 2.34 0.05 0.0878 0.6729
Class rank/GPA quintile 0.3886 0.0712 2.49 0.05 0.0899 0.6268
Senior test score quintile 0.0581 0.0754 0.35 † † 0.6079
Socioeconomic status quintile 0.3075 0.0638 2.20 0.10 0.0712 0.8632
Education expectations 0.5351 0.2020 1.21 † † 0.8949
Race -0.5191 0.2020 1.17 † † 0.8757
Gender -0.4076 0.1525 1.22 † † 0.9477
Parenthood -1.6210 0.4834 1.53 † † 0.9721
† Variables did not meet threshold criterion for statistical significance
a
For details on Tolerance, See Appendix D.
NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by design effect
2 2 2
= 2.17.G2 = 5206.65; df = 4943; G /df =1.0529; X (df ) = 33.8; pseudo R = 0.277; percent concordant
predicted probabilities = 78.9.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402
and Supplement).

157
Table G2. Logistic account of factors associated with earning a bachelor's degree in the
history of 1992 12th-graders who attended a four-year college at any time,
version 3: Demographic and high school background, using the three proxy
variables for high school academic curriculum intensity

Adjusted
Parameter standard
a
Variable estimate error t p Delta-p Tolerance
Intercept -5.0792 0.6795 3.41 0.01
Advanced placement 0.8931 0.2715 1.50 † † 0.8462
Science momentum 0.4005 0.0990 1.85 0.10 0.0927 0.6341
Foreign language 0.1163 0.0708 0.75 † † 0.7843
Class rank / GPA quintile 0.3317 0.0703 2.15 0.10 0.0768 0.6475
Socioeconomic status quintile 0.2957 0.0638 2.12 0.10 0.0685 0.8597
Education expectations 0.5305 0.2055 1.18 † † 0.896
Race -0.4740 0.1963 1.10 † † 0.9234
Gender -0.4022 0.1569 1.17 † † 0.9036
Parenthood -1.6042 0.4838 1.51 † † 0.9730
†Variables did not meet threshold criterion for statistical significance
a
For details on Tolerance, See Appendix D.
NOTES: Statistically significant variables are highlighted in bold. Standard errors adjusted by design effect = 2.19.
G2 = 5193.69; df = 4939; G2/df = 1.0515; X2 (df) = 33.94 (9); pseudo R2 = 0.229; percent concordant predicted
probabilities = 79.2.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402
and Supplement).

158
 APPENDIX H
The Timing of Departure
When this narrative took up the topic of the second calendar year of student histories and pointed
out that roughly one out of five of our subjects had become status dropouts by that point, the
question naturally arose: What happens to all those students who leave after the first year—or
even the first semester—when most attrition occurs? How does one account for them? We need
to provide some corrective data in this excursion, since we considered status dropout status only
in the context of the paradoxes of first-to-second year persistence.

With rare exceptions (e.g., Tinto 1987), most attrition studies are (a) short-term, as in first year to
second year, without allowing for the fact that students who skip the second year may return in
the third or fourth years, and (b) limited to attrition at the same institution, without allowing that
the student who can no longer be found at Old Siwash may, in fact, be enrolled at Greentree
Valley Community College. Students who leave one school and attend another one are not
dropouts—they are most likely transfers. Tinto (1987) made that point a long time ago, and it
should be obvious. If they are not transfers, they may be on temporary "excursions" or more
sustained voyages that end in either fragments or "discovery" (Adelman 2004b).

Given the systemwide perspective and the length of a longitudinal study such as the
NELS:88/2000, a more sophisticated question about withdrawal can be asked, looking
backwards from the last month of the longitudinal study (December 2000):

What percentage of traditional-age students who entered postsecondary education

during this period are no longer enrolled—anywhere—and never completed a
credential of any kind, by timing of the gap between their first month and last
month of enrollment?

This question follows the student—wherever the student goes. It tells us who became a “status
dropout” by age 26 or 27 (which does not preclude even these students from returning at a later
moment). And it tells us when the student became a status dropout, setting up an inquiry
concerning the reasons for departure that follows Tinto’s (1988) common sense suggestion that
these reasons change by point of departure. Table H1 sets forth these data for all 1992 12th-
graders who entered postsecondary education, by type of institution first attended.

Yes, the highest attrition rate in postsecondary education occurs in the first year (Schutz and
Malo 2003), but table H1 shows that the proportion of traditional-age students who leave in the
first year and never return is not radically higher than the proportion of those who leave at later
points in time, and differs by type of institution first attended. Consider the face validity of these
data: The mean number of credits earned by students who ultimately became status dropouts, by
period of departure, matches roughly what one would expect—half or less of the full-time norm
for the period. Proportion, however, is not the ideal way to assess the phenomenon.

Event-history accounts of attrition offer a more enlightening approach (e.g., Ronco 1996;
DesJardins, Ahlburg, and McCall, 1999). The driving force of event history is risk or hazard, as
befits a methodology derived from medical history analyses (Singer and Willett 1991), as in:

159
 given the patient’s medical momentum and other configurations of the patient’s life-style,
demography, and environment, when is the patient at risk of dying (the ultimate censoring event)
if the patient did not die last year? The flip side of risk in event history is survival. While the
purpose of this study stops short of survival analysis, the first step involved in setting up risk
modeling involves determining hazard probabilities, and these are instructive in light of the

Table H1. Percentage of 1992 12th-graders who entered postsecondary education and
withdrew without completing any credential by December 2000, by timing of
withdrawal and institution of first attendance

Institution of First Attendance

Withdrawal behavior Communit Other Mean Percent of

and its timing Four-year y college sub-bacca credits all

Left postsecondary without a

credential and did not return
Left within 11 months of
first enrollment 4.6 (0.59) 14.6 (1.13) 15.5 (2.77) 9.85 (0.54) 8.9 (0.58)

Left within 12-23 months 3.3 (0.41) 11.0 (1.32) 6.5 (1.50) 21.64 (0.99) 6.7 (0.59)

Left within 24-47 months 4.9 (0.41) 12.5 (1.11) 4.6 (1.05) 39.88 (1.62) 7.9 (0.50)

Left within 48-102 months 8.7 (0.73) 14.3 (1.23) 4.0 (1.62) 58.29 (2.68) 10.6 (0.65)
and not enrolled in 2000

No credential but still enrolled in

2000 4.7 (0.52) 9.2 (0.94) 1.8 (0.85) 70.98 (3.56) 6.4 (0.47)

Earned credential 73.8 (1.09) 38.4 (1.67) 67.7 (3.55) 124.73 (0.90) 59.5 (1.05)
a
Sub-baccalaureate institutions other than community colleges.
NOTES: Standard errors are in parentheses. Columns may not add to 100 percent due to rounding. "Credentials" can be
certificates, associate's degrees or bachelor's degrees.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and
Supplement).

160
first-year performance variables used in our analysis. That is, if we start at the first month of the first term of
attendance and, for each calendar year following that point, asked what percentage of students left the postsecondary
system with no degree and had not returned to school by December 2000, changing the denominator for each year to
the number of students who remained in the system, we have a "declining risk set" (Ronco 1996) that serves as a
rough guideline for the timing of potential dropout. The annual metric is obviously more even than the time
brackets chosen for table H1.

The "declining risk set" for this study is presented in table H2. To summarize: For all
postsecondary students, the risks of becoming a status dropout are highest in the first year,
decline through the fourth year, flatten out through the sixth year, and decline after that. For
students who attended a four-year college at any time, the risks of becoming a status dropout are
much lower across the temporal board, are slightly higher in the first year, drop and then flatten
from the second year through the sixth, and drop again after that. This is only the first step—and
hardly the final analysis—of event history, but is perhaps a more accurate way of describing the
parameters for understanding the phenomenon of when students leave postsecondary education.

Table H2. Declining "hazard probabilities" for 12th-graders who entered

postsecondary education, by calendar year following first date of
enrollment

"Hazard probabilities" for leaving without a degree

Departure timing for

those who never returned All who attended a four-year
by December 2000 All students institution at any time

First calendar year .102 .039

Second calendar year .061 .029
Third calendar year .048 .030
Fourth calendar year .039 .027
Fifth calendar year .037 .027
Sixth calendar year .041 .029
Seventh calendar year .031 .025
More than seven calendar years .026 .020
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

The reader of table H1 will note that community college beginners exit sooner than do students
who started at four-year colleges, and are more likely to still be enrolled at the end of the
longitudinal study period. While there is comparatively little variance in the pace of permanent
dropout for students who started in community colleges when the periods between the first and
the fourth year are considered, there is no question that beyond that point, attrition slows. The
same cannot be said for the pace of status dropout for students who started in four-year colleges:
The rate of permanent attrition appears to be stable over time, a finding slightly different from
that of the "declining risk set" approach, and a product of the clumping of years in table H1. As
for sub-baccalaureate institutions other than community colleges, which specialize in short-term

161
certificate programs, the status dropout rate drops dramatically from the first year to the second
and subsequent periods.

Advancing on Tinto’s (1988) suggestion, Eaton and Bean (1995) hypothesize that the factors
affecting attrition are different for first-year students who withdraw from higher education versus
the factors that come into play for students who leave at later points in time.Unfortunately, Eaton
and Bean could not investigate that hypothesis because their study was confined to first- year and
second-year students at one institution (a public research university), and bracketed only one
year’s account of retention/attrition (for a similar inquiry, see Patrick 2001). The NELS: 88/2000
data, on the other hand, offer 8.5 years and a myriad of institutional types in which to explore
and detail the boundaries of the issue. As Ronco (1996) notes, the timing and duration of
"enrollment events" is a critical framework for identifying factors associated with graduation,
transfer, and stop-out—the three modes of "exit" from an institution.

Table H3 provides a framework for future consideration of the differential attrition hypothesis.
The literature on the putatively sophisticated economic decisionmaking of adolescents with
respect to entering higher education, even in the face of uncertainty of returns (Altonji 1993), can
be extended to reasons for leaving a path to postsecondary credentials. As Beattie (2002) points
out, human capital theory is not rigid: it recognizes something called individual taste or
proclivity that may be only tangentially related to group membership (e.g., ethnicity, gender,
family income). At the least, students already in the postsecondary system know that they will
eventually be better off for their education efforts (or they wouldn’t have been there in the first
place). So, what did the students in our universe who did not complete a bachelor’s degree
program (and who were not still enrolled as degree candidates at the end of the 8.5 year period)
tell us of their reasons for leaving, by period during which they became status dropouts?

162
Table H3. Reasons for leaving postsecondary education without credentials by
timing of exit: 1992 12th-graders who entered postsecondary
education by December 1996

Percent offering these reasons for leaving postsecondary education

Timing of Job/ Personal/ Mood/

withdrawal Finances Military Academic Family Lifestyle

Left postsecondary
without a credential
and did not return

Left within 11
months of first 19.7 (3.10) 14.8 (2.03) 5.9 (1.92) 31.5 (3.51) 25.9 (3.99)
enrollment

Left within 12-23 25.2 (5.19) 11.0 (2.81) 5.6 (1.40) 42.6 (6.19) 12.7 (2.23)
months

Left within 24-47 19.4 (3.60) 17.4 (2.69) 3.6 (0.99) 35.2 (3.78) 20.5 (2.30)
months

Left within 48-102

months and not 17.8 (2.19) 18.9 (2.450 7.7 (2.46) 29.3 (4.04) 21.2 (3.21)
enrolled in 2000
NOTES: Rows will not add to 100.0 because (a) student responses to questions about reasons for departure are
not mutually exclusive, and (b) a category for "other" is not included. Standard errors are in parentheses.

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary transcript Files (NCES
2003-402 and Supplement).

163
When the NELS:88/2000 students were asked (in the spring of 2000) why they left school, they
were offered a list of reasons (though only 17 percent of students responding to the question
indicated more than one reason). As table H3 indicates, personal and family reasons for
departure dominate all other reasons for students who left postsecondary education without a
credential between the 12th and 47th month following the first month of enrollment. The same
factor ranks at the top versus all other factors except mood/lifestyle for students who left in the
first year and students who left after the fourth year. Strictly financial reasons are not as
prominent in students’ judgment as some contend. "Job/military considerations" might be
considered "financial," but there is too much ambiguity in that category of response to leap to
that conclusion. Some 6.6 percent (s.e. = 0.47) of the1992 12th-graders subsequently served in
the military, but 14.9 percent (s.e. = 4.95) of status dropouts who cited financial reasons for
leaving school had served in the military, and an identical 14.9 percent (s.e. = 3.84) who cited
"job/military considerations" for leaving had served in the military. If the NELS files had
offered a more complete labor market history, we might be able to resolve the ambiguities.

As a partial confirmation of these observations, consider the work of Li and Killian (1999), who
surveyed 622 students of all ages and levels who left a large state flagship university in 1997
(present in the winter term but not the following fall) and elicited 45 discrete reasons for
departure. First, Li and Killian discovered that 20 percent of these students were not really
dropouts at all, rather either transfers to other institutions (principally for reasons of academic
program or location) or stop-outs with intentions of returning. Of the remaining students, 43
percent left for academic reasons (e.g., didn’t like the program, classes too large, poor
performance), 34 percent left for personal reasons (e.g., unstated personal problems, location
issues including homesickness, illness, “social” reasons), and 20 percent for financial reasons.
Of the financial reasons, Li and Killian advise us that “individual patterns of money
management, more than family income” may be responsible for financially-driven attrition (p.
12), and that if we are going to understand attrition at all so that student affairs officers can
establish realistic efforts to address what is within the control of the institution to assist, we have
to dig below the level of large-scale concepts of academic and social integration. There is a
considerable range of reasons that students leave an institution, and, as Li and Killian point out,
students usually have more than one reason for departure. We can’t control homesickness; we
can help students deal with money management.

As for academic reasons for departure, table H4 reveals that compared to those who earned
bachelor’s degrees, no one in the departure universe was performing very well, no matter when
they left, and even though a tiny percentage (see table H3) cited academic reasons for walking
away. Braxton, Brier, and Hossler (1988) argue that when students are asked about their reasons
for withdrawal in post-hoc surveys, they offer "socially acceptable rationalizations." We should
hence be appropriately wary of the percentage distributions in table H3. Given the low
percentage of students who cited academic reasons, and GPAs and cumulative credits for the
same students that can be described as, at best, marginal, there is some indirect support for this
contention.

164
Table H4. Mean postsecondary GPAs of 1992 12th-graders who withdrew
without completing any credential, by timing of withdrawal, and
compared with GPA's of those who had not completed a credential
but were still enrolled in 2000, and those who earned bachelors
degrees by December 2000

Mean grade point average

Through
In first calendar second Entire
Departure behavior year of calendar year undergraduate
and timing attendance of attendance record

Left postsecondary without a

credential and did not return

Left within 1st 11 months 2.10 (0.091) 2.10 (0.091) 2.10 (0.091)
of first enrollment

Left between month 12 and 23 2.30 (0.057) 2.08 (0.058) 1.83 (0.100)

Left between month 24 and 47 2.31 (0.049) 2.26 (0.046) 2.03 (0.060)

Left between month 48 and 102 2.10 (0.059) 2.02 (0.054) 2.03 (0.050)

No credential, but still enrolled 2.14 (0.069) 2.08 (0.069) 2.19 (0.078)

Earned bachelor's degree 2.89 (0.015) 2.91 (0.014) 3.04 (0.011)

NOTE: Standard errors are in parentheses.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402
and Supplement).

165
 APPENDIX I

Course Categories Included in Aggregates for Participation Rates

In the First Two Calendar Academic Years of Attendance
Aggregate Course categories

College-level writing English composition; technical writing; creative writing; advanced

essay

Oral communications Oral communications; public speaking/debate; voice

Computer-related Introduction to computer science; introduction to

computing; computer programming: general

Introductory biological General biology; human biology; cellular biology; zoology:

sciences general; botany: general

Introductory physical General chemistry; general physics; general geology

sciences

College-level mathematics College algebra, liberal arts mathematics, finite mathematics,

statistics, precalculus, calculus

Core history Western civilization, world history, U.S. history surveys, European
history 1789–present, Asian history, African history, Latin-
American history

General psychology [single course category]

Micro/macroeconomics [single course category]

Humanities other than Humanities: general; introduction to philosophy; ethics

literature and foreign (philosophy); religious studies surveys; interdisciplinary
languages humanities; arts/humanities

Literature Literature: general; introductions to poetry, fiction, drama, non-

fiction prose; English literature; American literature; Afro-
American literature

Core social sciences Anthropology: general; cultural anthropology; geography: general;

introduction to political science; U.S. government; introduction to
sociology

Visual/graphic arts Visual communications; introduction to design; fine arts:

introduction; art history; drawing

Foundation business General business principles; business law/legal environment;

introduction to accounting

166
THIS PAGE INTENTIONALLY LEFT BLANK

167
 APPENDIX J
Collinearity Statistic (Tolerance) for the Seven Steps of the Logistic Narrative

APPENDIX J

Collinearity Statistic (Tolerance) for the Seven Steps of the Logistic Narrative

Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7

Variable table 12 table 13 table 15 table 16 table 24 table 26 table 27

Academic Resources quintile 0.8618 0.7454 0.5672 0.5638 0.5631 0.5554 0.5957
Socioeconomic status quintile 0.8761 0.8638 0.8505 0.8494 0.8452 0.8851 0.8893
c c
Race/ethnicity 0.9198 0.9231 0.9068 0.9065 0.8977
Gender 0.9712 0.9623 0.9404 0.9403 0.9351 0.9187 0.9184
Parenthood by age 20 0.9734 0.9653 0.9443 0.9441 0.9409 0.9409 0.9409
Education anticipations 0.9071 0.893 0.8517 0.8515 0.846 0.8458 0.8468
a
Selectivity of first institution 0.8296 0.8342 0.833 0.8143 0.8233 0.8195
a
No delay of entry 0.9034 0.8663 0.8599 0.8633 0.8698 0.8664
a b b b b b
Acceleration credits 0.8862
a a
Low credits in first year 0.7645 0.7611 0.7125 0.7096 0.6860
a a
First-year grades 0.8427 0.841 0.8246 0.7619 0.7646
a a c
First-year remediation 0.7586 0.7585 0.7462 0.7759
a a d d
First-year college-level math 0.7439 0.7435 0.7688
a a a b b b
Work-study 0.9563
a a a a
Multiple institutions 0.6671 0.7976 0.6601
a a a a
Classic transfer 0.7216 0.7985 0.7049
a a a a
Four-to-four transfer 0.7801 0.9426 0.7731
a a a a
Summer-term credits 0.8926 0.9030 0.8978
a a a a
Ever part-time 0.8192 0.8056 0.7130
a a a a a
Trend in grades 0.9093 0.9136
a a a a a
Cumulative college math 0.8166 0.809
a a a a a a
Continuous enrollment 0.7595
a a a a a a
Withdrawal/repeat ratio 0.7816
a
Variable not included at this step of the logistic narrative.
b
Variable did not qualify to be carried forward to the next step of the logistic narrative.
c
Variable did not meet criterion for entry into the model at this step of the logisitic narrative.
d
Replaced by cumulative college math in these steps of the logistic narrative.
NOTE: For definition of tolerance and discussion of collinearity, see Appendix D.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files
(NCES 2003-402 and Supplement).

168
THIS PAGE INTENTIONALLY LEFT BLANK

169
 APPENDIX K

So They Earned a Degree! Now, How Long Did It Take Them?

The second most frequently asked question about U.S. higher education concerns time to degree.
Despite accumulating evidence over the past quarter-century, the normative benchmark for
completing a bachelor’s degree has remained at four elapsed academic years. Our ideal seems
not only to get it over with, but to get it over with in normative time. An altruistic argument in
favor of the normative is that when students finish in a timely manner, they make room for
others, i.e., completion keeps the access channels open. Yet recipients of bachelor’s degrees are
adults who have made a variety of decisions concerning competing goals and obligations that
often do not include finishing a degree in four years. For example, if one looks at all recipients
of bachelor’s degrees in 1999–2000, and asks when they commenced their studies and how old
they were when the degree was awarded, one finds that 62.8 percent (s.e. = 1.5) of the 7.5
percent of bachelor’s degree recipients who were over 40 in 2000 started out in postsecondary
education before 1980. At a minimum, the elapsed time to degree for these people was 20 years
(in fact, the average was over 27 years). Of the 9.2 percent (s.e. = 0.4) of degree completers who
were between 30 and 39 in 2000, 73.3 percent (s.e. = 1.7) entered postsecondary education prior
to 1990, so, at a minimum, the elapsed time to degree for these people was 10 years (in fact, the
average was nearly 15 years).1

The key phrase for our purposes is "elapsed time," as that is the customary way of measuring
time to degree. It is not surprising that the older the baccalaureate recipient, then more likely
that student had taken at least two years off from higher education, and the more likely that their
principal reason for doing so was "change in family status," as in becoming a parent and raising
children. The reader might consider, in passing, when complaints are raised about excessive
time to degree, whether our system of higher education should be "blamed" for providing the
opportunity for adults with different life trajectories to return to school and complete degrees,
even if the time between initial enrollment and degree award is 15 or more years. That is the
practical meaning of elapsed time.

There is an obvious difference between elapsed and enrolled time. When DesJardins, Ahlburg,
and McCall (2002) investigated factors leading to "timely graduation," they confined one of their
universes to students who had never stopped out—not even for the one term this study allows
stop-out without labeling the student a non-continuous enrollee.2 If we followed this mode of
analysis, we could focus in on student uses of enrolled time to determine what contributed to and
what detracted from year-by-year progress. Knight (2002) includes types and timing of

remediation, majors and changes-of-major, course withdrawals and repeats, course failures,
credit hours earned and grades in general education courses, hours per week working and
1
All data in this paragraph were generated from the National Center for Education Statistics: Baccalaureate and
Beyond Longitudinal Study, 2000: Data Analysis System.
2
The research here involved a "competing risks" model in which the histories of students who had never stopped out
were compared to the histories of those who had stopped out, hence, embraced both elapsed and enrolled time.

170
studying, and first-year grade point average among variables reflecting different aspects of the
uses of time.3 Volkwein and Lorang (1996) took up the phenomenon of students with extended
time-to-degree because they purposefully carried lighter credit loads. Why did they carry less
than the 15 credit per semester norm for the institution that served as the site for this study? To
enhance GPA (an objective which precipitated dropping difficult courses), and to generate more
free time, some of which was for work or family responsibilities, or both. Some respondents
also cited difficulty of enrolling in a course "at the time I wanted," a phenomenon familiar to
anyone who has worked student registration lines.

We also know that many disciplines require credit hour production that, when translated into
standardized semester metrics, exceed the 120 credits that would be produced by students who
attended only during the regular academic year and carried a standardized full-time load of 15
credits per semester. In a survey of 91 public universities in all 50 states, Pitter, LeMon, and
Lanham (1996) found combinations of university and program requirements ranging from
122–124 credits in social sciences, foreign languages, psychology, mathematics, and protective
services, for example, to 130–142 credits in such fields as engineering, architecture, and health
professions (e.g., nursing, physical therapy, etc.). They also confirm our observation of five-year
bachelor’s degree programs in pharmacy, for a noted case, with a median credit requirement of
161. Any requirement above 120 credits will either add time to degree or encourage students to
(a) earn credits by examination, including Advanced Placement, (b) attend during summer terms,
and/or (c) carry credit loads in excess of 15 per semester in order to graduate "on time".4
As Garcia (1994) observed, whether the student is a community college transfer or a native
student in a four-year college, the “basic recipe" for completing a bachelor’s degree within the
standard "templates for time-to-degree is to maintain continuous enrollment and to earn more
than 30 semester units each academic year" (p. 8). Sounds simple, but the list of diverting
phenomena is long, complex, and often best assessed for indirect effects, as Garcia has done,
using path analysis.

What we learn from these studies of the uses of enrolled time in relation to normative models of
time to degree is what to include in both descriptive and multivariate accounts of elapsed time—
and elapsed time is the measure used in both the original Tool Box and The Toolbox Revisited.
The descriptive statistics are offered to the reader in Appendix L, table L13, but some basic
observations are worth repeating here. First, the mean elapsed time to degree for bachelor’s
recipients in the NELS:88/2000 was 4.58 calendar years, and the median time (which

is probably more meaningful) was 4.24 calendar years.5 Shorter time to degree is observable
3
Knight’s list for a “hypothesized model of effects upon total semesters elapsed” (Knight 2002, p. 6) also includes
provisions for variables derived from student surveys that are parallel to those used in the National Survey of
Student Engagement, e.g., in Kuh et al. 2001.
4
The small number of NELS:88/2000 bachelor’s recipients who completed the degree in three elapsed calendar years
or less entered with a mean of 10.6 acceleration credits (s.e. = 2.81) and subsequently accumulated a mean of 12.2
credits during summer terms (s.e. = 1.94).
5
Broh (1991) presented an intriguing argument for an alternative time to degree calculation that began with the
median, not the mean, and included students who had not finished the degree but who were still enrolled as degree
candidates. In The Toolbox Revisited, the median for students who had completed degrees is 4.24 calendar years.

171
among students from the highest quintile of high school Academic Resources, students who first
entered highly selective institutions, among those who brought more than four acceleration
credits across the matriculation line, and among those who attended only one school.
Considerably longer time to degree is observable among those who attended three or more
schools, among both community college to four-year college and four-year-to-four-year transfer
students, among those who took more than one remedial course, those who were not
continuously enrolled, and those who majored in the physical sciences. Most of these
relationships confirm common sense.

But it is the multivariate analysis that will tell us what counts, and that is the purpose of table
K1. Table K1 presents a linear (Ordinary Least Squares) regression in which the dependent
variable, time to degree, is set on a scale with four values: more than 6 calendar years, 5–6 years,
4–5 years, and less than 4 calendar years (which encompasses the normative four-year degree).
With one exception, a dichotomous variable indicating whether the student ever engaged in a
cooperative education or internship course, the independent variables selected were those
previously explored and (in most cases) used in the logistic narrative of Part IV of this study.
The statistically significant independent variables are highlighted in bold. How do we read the
table? What do we see?

On reading the table: First, the R2 indicates that this linear model accounts for half the variance
in timing of bachelor’s degree completion for the NELS:88/2000 cohort. That’s a very
convincing level. Second, the positive parameter estimates mark variables that add time to
degree, while the negative parameter estimates indicate variables that shrink it. Third, the
following variables did not meet the significance threshold for entering the model (set at p <
.05): race/ethnicity, becoming a parent by age 20, multi-institutional attendance, and freshman
year remediation.

There is no question of the major contributors to extended time to degree:

1. Excessive no-penalty course withdrawals (W) and no-credit repeats (NCR)—

Damaging to degree completion, these are just as damaging to time to degree.
Yes, we can keep students continuously enrolled, but if much of that enrollment is
nullified by withdrawals and repeats, we have what Tinto calls retention without
education. It’s worse, though, because every seat at every available hour in every
facility of an institution of higher education is at stake, and every seat marked
with a W or NCR bars another student from sitting down. When the blocked
seats reach a critical mass, general access is impeded as well. This observation is
worth repeating.
2. Transfer, both classic community college to four-year, and four-year to four-year
—As previously noted, transfer often involves a hiatus in enrollment and extra
effort to meet prerequisite requirements at the new institution.
3. Change of major—Garcia (1994) observed that the more times the student
changed major, the greater the elapsed time to degree. Knight (2002, 2004a,
2004b) also includes multiple changes of major among the set of variables with
significant impact on both enrolled and elapsed time.
4. Whether the student was ever part-time—Since our variable for part-time
attendance was constructed partly with reference to course withdrawals that, in
effect, produced lighter course loads, the result here backs up the conclusions of
Volkwein and Lorang (1996).

Including students who had not earned degrees by December 2000 but who were still enrolled as bachelor’s
candidates, the median time to degree would be 4.49—very close to the mean of 4.58.

172
 Table K1. Ordinary Least Squares regression indicating factors influencing time-to-degree
for 1992 12th-graders who earned bachelor's degrees by December 2000

Adjusted
Parameter standard Partial
Variable estimate error t p R-squarea

Intercept 3.5227 0.2703 8.41 0.001

WRPRATIO 4.5126 0.4314 6.75 0.001 0.1946
Continuous enrollemnt -1.2149 0.0854 9.18 0.001 0.1287
Community college transfer 0.4789 0.0632 4.89 0.001 0.0616
Four-to-four transfer 0.3685 0.0553 4.3 0.001 0.0358
Ever part-time 0.3813 0.0496 4.96 0.001 0.0274
Remedial problemc -0.1322 0.0344 2.48 0.05 0.0232
Freshman GPA quintile 0.0758 0.0186 2.63 0.02 0.0109
Changed major 0.1711 0.0431 2.56 0.02 0.0055
SES quintile -0.0563 0.0174 2.09 0.10 0.0028
Selective first institution -0.1227 0.0470 1.69 † 0.0045
Academic Resources quintile -0.0551 0.0233 1.53 † 0.0052
First-year low credits 0.1717 0.0800 1.39 † 0.0015
GPA trend -0.0518 0.0288 1.16 † 0.0024
No delayed entry 0.1922 0.1190 1.04 † 0.0014
Co-ops or internships -0.0736 0.0508 0.87 † 0.0007
Education anticipations -0.1324 0.0497 1.11 † 0.0006
Gender 0.0623 0.0388 1.04 † 0.0010

R2 = 0.5052
Adjusted R2 = 0.5056
†Variable did not meet minimum criterion for statistical significancne in this model.
a
Indicates the contribution of each variable to the explanation of variance, the total R2.
b
Ratio of no-penalty withdrawals and no-credit repeats to all courses attempted
c
See Glossary, p. 191
NOTES: Root design effect = 1.55. Statistically significant variables are highlighted in bold. Weighted N=922k.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

And there is no question concerning which variables will contract time to degree: continuous
enrollment and no remediation (the highest value of the "total remediation" variable).
Socioeconomic status is a marginally significant contributor to lower time to degree, while the
position of freshman year GPA quintile as a contributor to extended time to degree is somewhat
puzzling.

173
Of those variables that were admitted to the linear model but failed to reach the threshold of
statistical significance, enrollment in cooperative education or internship courses deserves
special attention because it raises contrasting policy options with respect to influences on time to
degree. Knight (2004b) observes that:

Timely degree completion is not all that matters in terms of college student outcomes.
Both analytical and student self-report evidence supports the fact that enrollment in
cooperative education classes, involvement in internships, etc., while extending time-to-
degree, significantly improves student learning and skill development, affective
outcomes, career prospects, and the like. Significantly reducing time-to-degree could
perhaps demand a trade-off against other long-term (and maybe more important)
outcomes. (p. 14).

It turns out that the NELS:88/2000 baccalaureate students who engaged in cooperative education
or internships or clinical externships did not take longer than others to complete degrees. But
Knight is really asking readers to reflect on the principal objectives of conducting a
baccalaureate enterprise. The modes of instructional delivery in professional and applied fields,
in particular, often include carefully designed, graded experience in occupational environments,
involving considerable time commitments that lie beyond the classroom. If we say to a school of
engineering, “You can’t offer cooperative education any more and cannot require more than 128
credits for a degree,” what do we accomplish in terms of the quality of student learning and the
quality of engineering graduates?

In contrasting spirit, if we observe institutions in which students can withdraw from courses,
without penalty, as much as 10 weeks into a 14-week semester, a volume of withdrawals
equivalent to 15 percent of all credits offered by the institution in a calendar year, and the
average time to degree for native students at that institution to be well over five calendar years, we
would accomplish a great deal by a tightening of course withdrawal policy. Here is a prime candidate for future
research governed by quasi-experimental design: Find two comparable institutions (mission, size, demography,
distribution of majors), one with lax withdrawal rules, the other with restrictive rules. The hypothesis, from
everything learned in this data essay: An institution that restricts course withdrawal policy will witness higher
graduation rates and shorter average time to degree.

174
THIS PAGE INTENTIONALLY LEFT BLANK

175
 APPENDIX L
Tables and Comments on Miscellaneous Topics Raised in the Text
Table L1. Percentage of 1988 eighth-graders who graduated from high school, by timing and
type of high school diploma (if any), by gender, race/ethnicity, and socioeconomic
status quintile

Graduated Graduated Graduated

by July 1992 Graduated after July 1992 after Had not
with standard by July 1992 with standard July 1992 graduated by
Population diploma with GED diploma with GED December 2000

All 1988
eighth-graders 78.3 (1..08) 2.6 (0.38) 4.7 (0.44) 5.1 (0.50) 9.3 (0.84)

Race/ethnicity

White 82.4 (1.06) 2.3 (0.40) 3.1 (0.36) 4.4 (0.54) 7.8 (0.85)
African-American 63.2 (4.59) 4.6 (1.86) 11.2 (2.39) 9.0 (2.22) 12.0 (3.27)
Latino 66.1 (3.34) 2.5 (0.44) 7.6 (1.28) 6.9 (1.16) 12.9 (3.16)
Asian 93.4 (1.90) 0.4 (1.24) 1.6 (0.57) 0.9 (0.35) 3.7 (1.81)
American Indian 61.6 (7.71) 2.9 (1.24) 11.3 (5.59) 4.3 92.39) 20.0 (7.26)

Gender

Male 77.1 (1.61) 2.2 (0.41) 5.1 (0.66) 5.2 (0.80) 10.4 (1.37)
Female 79.6 (1.28) 2.9 (0.63) 4.2 (0.58) 5.1 (0.58) 8.3 (0.92)

Socioeconomic
status quintile

Highest 94.8 (1.12) 0.5 (0.17) 2.0 (0.58) 1.6 (0.75) 1.1 (0.49)
2nd quintile 84.1 (1.90) 3.7 (1.30) 4.4 (0.96) 3.6 (0.80) 4.3 (1.02)
3rd quintile 83.4 (1.99) 2.4 (0.66) 4.2 (1.01) 5.3 (1.38) 4.7 (0.73)
4th quintile 70.5 (2.55) 2.9 (0.80) 5.9 (1.00) 7.7 (1.34) 13.0 (2.42)
Lowest 54.1 (2.61) 3.6 (0.83) 7.5 (1.25) 8.0 (1.15) 26.8 (2.80)

Urbanicity of
high school
community

Urban 73.7 (2.32) 2.8 (0.75) 7.1 (1.15) 6.7 (1.12) 9.7 (1.67)
Suburban 81.3 (1.41) 2.6 (0.74) 4.2 (0.62) 4.3 (0.77) 7.6 (0.92)
Rural 80.5 (1.85) 2.3 (0.35) 3.0 (0.46) 4.6 (0.71) 9.6 (1.76)
NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and
Supplement).
176
Table L2. Percentage distribution of high school graduation status,
postsecondary participation and degree completion of students
in the 1992 NELS:88/2000 survey, by retention in-grade

Was student ever retained in grade?

No Yes Indeterminable

Population

All Students 73.0 (0.94 17.8 (0.90) 9.2 (0.71)

High school completion

Academic diploma 90.6 (0.67) 58.6 (2.44) 57.1 (3.50)

GED and other 4.7 (0.43) 20.2 (2.28) 14.8 (2.31)
Did not complete 3.8 (0.52) 18.9 (1.86) 26.7 (3.60)
Indeterminable 0.9 (0.18) 2.2 (0.61) 1.4 (0.37)

Postsecondary entry 81.3 (0.86) 48.7 (2.23) 46.5 (3.53)

Highest degree

No postsecondary 18.7 (0.86) 51.3 (2.23) 52.5 (2.52)

No degree 35.1 (0.90) 35.0 (2.17) 29.1 (2.93)
Certificate 3.7 (0.32) 3.7 (0.65) 4.1 (1.22)
Associate 6.2 (0.41) 3.2 (0.61) 4.8 (1.33)
Bachelor's or above 36.4 (1.00) 6.8 (0.82) 9.5 (1.91)

Highest degree of those who

entered postsecondary

No degree 40.5 (1.01) 69.0 (2.60) 57.8 (5.19)

Certificate 4.8 (0.43) 9.1 (1.59) 9.8 (3.23)
Associate 8.0 (0.54) 6.2 (0.95) 10.7 (2.86)
Bachelor's 46.7 (1.10) 15.6 (1.83) 21.7 (4.05)

NOTES: (1) Standard errors are in parentheses. (2) Column totals for high school completion and highest
degree may not add to 100.0 percent due too rounding. (3) Weighted Ns for all students: Never retained in
grade = 2.298M; retained in grade = 560K; Indeterminable = 291K
SOURCE: National Center for Education Statistics: NELS: 88/2000 Postsecondary Transcript Files
(NCES #2003-402 and Supplement).

177
Table L3. Percentage distribution of 1992 12th-graders who entered
postsecondary education, by selectivity of the first institution of
attendance, by standard demographics

Selectivity of first insituion of attendance

Highly Not
Population selective Selective Nonselective Open-door ratablea

All students 3.2 (0.43) 12.2 (0.66) 40.4 (1.02) 41.6 (1.13) 2.6 (0.31)

Gender

Male 3.4 (0.58) 12.3 (0.86) 38.2 (1.40) 44.0 (1.50) 2.1 (0.39)
Female 3.0 (0.51) 12.1 (0.84) 42.2 (1.30) 39.6 (1.44) 3.1 (0.42)

Race/ethnicity

White 2.4 (0.32) 13.2 (0.86) 42.5 (1.15) 39.6 (1.24) 2.3 (0.34)
African-American 5.1 (2.30) 7.5 (1.59) 41.7 (3.36) 42.4 (3.98) 3.2 (0.82)
Latino 2.8 (1.23) 7.9 (1.23) 28.5 (3.09) 56.2 (3.40) 4.5 (1.47)
Asian 12.9 (3.09) 15.4 (2.35) 30.6 (3.11) 38.4 (3.84) 2.7 (1.37)
American Indian # 6.7 (3.02) 29.7 (8.55) 62.2 (9.33) 1.4 (1.07)

Socioeconomic
status quintile

Highest 7.3 (1.05) 25.1 (1.47) 45.6 (1.67) 20.6 (1.60) 1.3 (0.55)
2nd quintile 2.1 (0.85) 10.2 (0.99) 42.7 (2.07) 43.4 (2.07) 1.6 (0.33)
3rd quintile 0.4 (0.13) 6.0 (0.72) 40.2 (1.97) 50.3 (2.10) 3.0 (0.52)
4th quintile 0.8 (0.29) 4.8 (0.76) 35.2 (1.97) 55.7 (2.06) 3.5 (0.70)
Lowest quintile 1.3 (1.08) 3.8 (0.95) 29.9 (3.07) 58.9 (3.36) 6.1 (1.44)
# Rounds to zero.
a
Includes both less-than-two-year trade schools, conservatories (music and art), and schools of divinity.
NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

178
Table L4. Descriptive relationship of number of credits earned in the first
calendar year of attendance by 1992 12th-graders to highest degree
earned by December 2000

Highest degree earned by December 2000

Number of credits Balchelor's Percentage of
in first year None Certificate Associate or higher all students

0 - 10 84.6 (1.44) 6.4 (0.98) 5.4 (0.88) 3.6 (0.75) 24.5 (0.94)
11 - 19 63.9 (2.27) 4.9 (0.88) 12.0 (1.65) 19.2 (1.70) 16.5 (0.73)
20 - 29 27.5 (1.39) 3.8 (0.75) 8.4 (0.79 60.3 (1.60) 31.0 (0.83)
30 or more 10.6 (0.86) 5.1 (0.66) 7.9 (0.83) 76.5 (1.31) 27.9 (0.81)

Among those who

attended a four-
year college at any
time, number of
credits earned in
first year

0 - 10 75.8 (3.15) 4.9 (2.18) 8.8 (2.18) 10.4 (2.11) 12.4 (0.88)
11 - 19 53.9 (2.82) 2.1 (0.55) 12.1 (2.22) 31.9 (2.51) 14.1 (0.70)
20 - 29 24.2 (1.39) 2.0 (0.79) 5.0 (0.62) 68.9 (1.59) 38.8 (1.01)
30 or more 8.4 (0.81) 0.6 (0.16) 3.4 (0.48) 87.6 (0.93) 34.8 (0.94)

NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted N
for all 1992 12th graders =2.09M; for those who attended a four-year college at any time: 1.47M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

179
Table L5. Percentage distribution of enrollment intensity over six years
(1995/6 - 2001) of students who started out in postsecondary
education in 1995/96, by age as of December 1995

Age at entry Enrollment intensity

Full-time Part-time Both full-time

only only and part-time

All Students 47.0 (1.1) 12.2 (0.8) 40.7 (1.0)

Less than 21 51.8 (1.1) 4.4 (0.5) 43.8 (1.1)

21 - 23 40.0 (3.8) 18.7 (3.1) 41.3 (1.1)
24 - 29 30.2 (3.7) 35.5 (4.6) 34.3 (4.2)
30 and older 32.1 (3.1) 42.9 (3.5) 25.0 (2.8)

NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding.
SOURCES: National Center for Education Statistics: Beginning Postsecondary Students
Longitudinal Study, 1995/96 - 2001.

180
As the reader will notice, about 48 percent of traditional-age students in the BPS longitudinal
study of 1995/96-2001 told us that they attended part-time at some point in their undergraduate
careers. In contrast, only 38 percent of the NELS:88/2000 postsecondary cohort indicated in
their 2000 CATI interviews that they had attended part-time at any time. Are the two cohorts,
three years apart in their modal starting date, that different? No.

NELS:88/2000:

Part-time by student report 37.7

Part-time by transcript evidence 9.7

Total part-time 47.4

BPS95/96–2001 students who started

before they were 21:

Always part-time 4.4

Mixed full-time and part-time 43.8

Total part-time 48.2

So a dichotomous variable indicating whether the student ever attended part-time can be
included in the logistic model in Step 5 (attendance patterns).

181
Table L6. Mean Carnegie Unit credits earned in grades 10-12 by (a) all 1992 12th-
graders and (b) by those of known socioeconomic status who attended a
four-year college at any time and for whom high school transcript records
and senior year test scores were complete

Mean Carnegie units earned by:

All 1992 12th-graders

All 1992 who are the subjects of
12th-graders The Tool Box Revisiteda

All high school credits 17.93 (0.09) 18.74 (0.09)

All academic credits 16.06 (0.09) 17.13 (0.09)
All vocational credits 1.28 (0.03) 1.01 (0.03)

English 4.11 (0.02) 4.18 (0.02)

Mathematics 3.39 (0.03) 3.61 (0.03)
Foreign language 1.98 (0.04) 2.31 (0.04)
Science 3.14 (0.03) 3.39 (0.03)
Social sciences 3.55 (0.03) 3.64 (0.03)
a
These students are the same as those described under (b) in the title of this table.
NOTES: Standard errors are in parentheses. Weighted N for all 1992 12th-graders; 2.62M.
Weighted N for 1992 12th-graders who graduated from high school by December 1996, attended a
four-year college at any time, and were of known socioeconomic status; 1.41M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files
(NCES 2003-402 and Supplement).

182
Table L7. Logistic account of the three components of Academic Resources is relation
to bachelor's degree attainment for all 1992 12th-graders with complete
records for all three components

Adjusted
Parameter standard Standardized
Variable estimate error F t p estimate Odds Ratio

Intercept -5.3515 0.2348 529 11.12 0.01

Curriculum intensity 0.6273 0.0568 122 5.39 0.02 0.4672 1.87
Classrank/GPA 0.4700 0.5290 79.1 4.34 0.05 0.3686 1.60
Senior test score 0.3649 0.0561 42.3 3.17 0.10 0.2785 1.44

NOTES: Standard errors adjusted by Design Effect = 2.05; Universe consists of 1992 12th graders with high
school transcript records and senior year test scores. Percent of concordant probabilities predicted; 85.1;
G2=7536.7; df =8283; g2/df =90.0; X2 (df)=2977.32 (3)
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-
402 and Supplement).

183
Table L8a. First season of postseconday attendnace of NELS:88/2000 students, (a) who
participated in the 1992 NELS survey and (b) who were 12th-graders and
subsequently attended a four-year college

12th-graders in 1992
with full high school
All students in records who attended
1992 survey a four-year college

First season of attendance

Summer term 5.8 (0.46) 4.4 (0.49)

Fall term 82.1 (0.83) 91.3 (0.65)
Winter or spring terms 12.1 (0.71) 4.3 (0.37)

Table L8b. Percentage of two groups of NELS:88/2000 postsecondary

students who entered in the fall term: (a) All who participated in
the 1992 NELS survey and (b) those who were 12th-graders in
1992 and subsequently attended a four-year college, by
race/ethnicity and socioeconomic status quintile

Percent Entering in the Fall Terms

12th-graders in 1992
with full high school
All students in records who attended
1992 survey a four-year college
By race/ethnicity

White 84.3 (0.84) 92.3 (0.54)

African-American 73.8 (3.53) 85.7 (4.28)
Latino 73.9 (2.19) 86.1 (3.49)
Asian 83.3 (3.54) 89.9 (3.77)
American Indian 85.1 (5.82) 97.1 (2.87)

By socioeconomic
status quintile

Highest 92.2 (0.73) 93.7 (0.81)

2nd quintile 83.3 (1.85) 90.6 (1.66)
3rd quintile 79.6 (1.68) 90.8 (1.04)
4th quintile 77.4 (1.83) 89.9 (1.32)
Lowest 66.0 (2.96) 82.7 (3.62)
NOTES: Standard errors are in parentheses. Weighted Ns: 1992 Panel =2.18M; 1992 12th graders =
2.06M; 1992 12th graders who attended a four-year college at any time and who presented full high
schools records = 1.18M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files
(NCES 2003-402 and Supplement). 184
Table L9. Percentage of 1992 12th-graders earning credits in college-level math courses, both
in the first calendar year of attendance and cumulatively, by number of credits
earned in college-level mathematics and type and selectivity of first institution of
attendance

Number of credits in all college-level math courses

earned in the first calendar year of attendance

Characteristics of
first institution None 1-4 More than 4

Type of first institution

Four-year 44.2 (1.20) 30.2 (1.02) 25.6 (0.98)
Community college 82.1 (1.22) 12.0 (1.00) 6.0 (0.75)
Other sub-baccalaureate 89.2 (2.72) 9.5 (2.70) 1.3 (0.51)

Selectivity of first institution

Highly selective 25.1 (5.69) 26.9 (4.24) 48.0 (5.32)
Selective 26.9 (1.98) 32.0 (2.38) 41.1 (2.28)
Nonselective 51.3 (1.26) 30.1 (1.15) 18.6 (0.91)
Open-door 82.4 (1.17) 11.8 (0.96) 5.8 (0.73)
Not ratable 92.4 (2.51) 3.7 (2.09) 3.9 (1.42)

Number of credits in all college-level math courses earned at any time

in the undergraduate careers of students who earned more than 10
creditsa
None 1-4 5-9 More than 9

Type of first institution

Four-year 23.2 (0.96) 26.3 (0.96) 28.9 (0.92) 21.5 (0.99)
Community college 52.8 (1.82) 21.4 (1.47) 16.1 (1.70) 9.6 (1.01)
Other sub-baccalaureate 79.1 (3.88) 12.4 (3.62) 6.1 (1.70) 2.4 (0.83)

Selectivity of first institution

Highly selective 18.1 (5.44) 17.8 (3.53) 18.0 (3.34) 46.1 (5.33)
Selective 16.5 (1.74) 21.1 (2.21) 31.9 (2.13) 20.5 (2.16)
Nonselective 25.4 (1.02) 29.1 (1.14) 28.7 (1.09) 16.8 (0.94)
Open-door 54.2 (1.77) 21.2 (1.43) 15.6 (1.24) 9.1 (0.92)
88.0 (3.10) 2.7 (1.14) 7.9 (2.65) 1.4 (0.81)
a
The "more than 10 unergraduate credits" criterion excludes incidental students.
NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted N for all
1992 12th graders =2.09M; for those who earned more than ten credits: 1.83M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES 2003-402 and
Supplement).

185
Table L10. Logistic account of factors associated with bachelor's degree
completion for 1992 12th-graders who attended a four-year college at
any time, using second-year cumulative credits and second-year
cumulative GPA

Adjusted
Parameter standard
Variable estimate error t p Delta p

Intercept -6.2118 .7595 4.63 .001 ----

GPA through second year 0.7585 .1826 2.37 .05 0.1672
Credits through second year 1.1785 .1054 6.34 .001 0.2597
No delay of entry 0.8852 .3240 1.55 † †
SES quintile 0.2636 .0651 2.29 .05 0.0581
Education expectations 0.5402 .2031 1.51 † †
Academic Resources quintile 0.3345 .0821 2.31 .05 0.0737
Ever part-time -1.3259 .1611 4.66 .001 -0.2923
Summer-term credits 0.5273 .0929 3.29 .01 0.1162
Race/ethnicity 0.3327 .2186 0.86 † †
Gender -0.1998 .1574 0.72 † †
Parenthood by age 20 -0.8960 .4535 1.12 † †
Multiple institutions -0.5354 .1874 1.62 † †
Community college transfer 0.6251 .2151 1.65 † †
First institution was selective 0.4008 .2248 1.01 † †
College-level math in first year 0.2064 .1692 0.69 † †
First-year remediation 0.2940 .1929 0.86 † †
†Variables did not meet threshold criterion for statistical significance.
NOTES: Standard errors adjusted by Design effect=1.71. Delta-p computed only for statically significant
variables. Universe consists of all 1992 12th graders who participated in postsecondary education, whose
records indicated positive values for all variables in the model, and whose transcript records were complete.
Weighted N = 1.5M. G2 = 3477.1; df =4776; G2/df = 0.73; X2 (df )=2272.2 (17); p = 0.0001. Proportion
concordant predicted probabilities = 90.4 percent.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

186
Table L11. Institutional attendance patterns of 1982 and 1992 12th-graders who
subsequently earnd more than 10 postsecondary creditsa

Attendance characteristics Class of 1982 Class of 1992

Number of schools attended

One 46.9 (0.82) 43.5 (1.06)

Two 33.8 (0.75) 35.1 (0.93)
More than two 19.3 90.63) 21.5 (0.84)

For those who attended more than

one school:

Attended in more than one state 39.5 (1.21) 35.7 (1.36)

Institutional combinations:

Four-year only 38.9 (1.21) 27.5 (1.17)

b
Reverse transfer 7.6 (0.60) 8.2 (0.76)
Community college transfer 18.1 (0.90) 22.1 (1.12)
c
Alternating/simultaneous 10.2 (0.67) 12.8 (1.01)
Four-year student with 12.7 (0.90)
incidental community colleged e

Four-year plus other sub- 4.9 (0.46) 1.8 (0.36)

baccalaureate
Community college only 10.2 (0.69) 9.1 (0.71)
Community college and other 5.3 (0.50) 3.0 (0.33)
sub-baccalaureate
Sub-baccalaureate trade school 1.1 (0.22) 0.3 (0.20)
only
Other combinations 3.6 (0.51) 1.5 (0.31)
a
The reason for excluding incidental students (those who earned 10 or fewer credits) is that they provide an
insufficient history for judging multi-institutional attendance.
b
Student began in four-year institution, did not earn a degree, and transferred to a community college.
c
Student moved back and forth between four-year and two-year institutions, and/or attended two institutions at
the same time, or did both.
d
Student was based in a four-year institution, and earned 10 or fewer credits, principally during summer terms,
from community colleges. This measure was not available in the High School & Beyond/Sophomore cohort
transcript file.
e
Category was not available in the High school & Beyond/Sophomore cohort transcript files (NCES 2000-
194).
NOTES: Standard errors are in parentheses. Rows may not add to 100.0 percent due to rounding. Weighted
N for those earned more than ten credits: 1.83 M.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript Files (NCES
2003-402 and Supplement).

187
Table L12. Percentage distribution of final (December 2000) educational status of 1988
eighth-graders, by type and timing of high school diploma (if any), and
including those who did not earn high school diplomas

Secondary/postsecondary status Percent

1) Earned standard high school diploma by July 1992 and

Earned at least a bachelor's degree 29.1 (0.94)

Associate degree was highest earned credential 4.7 (0.28)
Certificate was highest earned credential 2.8 (0.24)
No degree, but still enrolled in 2000 4.6 (0.33)
No degree, not enrolled in 2000 23.6 (0.79)
Never entered postsecondary education 13.1 (1.10)

2) Earned standard high school diploma after July 1992 and

Earned at least a bachelor's degree 0.2 (0.12)

Associate degree was highest earned credential 0.3 (0.14)
Certificate was highest earned credential 0.2 (0.04)
No degree, but still enrolled in 2000 0.4 (0.14)
No degree, not enrolled in 2000 1.9 (0.29)
Never entered postsecondary education 1.6 (0.23)

3) Earned GEDs or Certificates of Attendance and

Earned at least a bachelor's degree 0.1 (0.03)

Associate degree was highest earned credential 0.2 (0.05)
Certificate was highest earned credential 0.4 (0.14)
No degree, but still enrolled in 2000 0.8 (0.22)
No degree, not enrolled in 2000 2.8 (0.38)
Never entered postsecondary education 3.9 (0.46)

4) Others
Did not graduate from high school, but entered postsecondary 1.0 (0.42)
Did not graduate from high school, no postsecondary 6.7 (0.65)
Indeterminable high school graduation status 1.7 (0.37)
NOTES: Standard errors are in parentheses. Percent column may not add to 100.0 due to rounding. Weight used
throughout this table is the F4BYWT with a base year (1988) flag. F4BYWT covers NELS:88/2000 students who
were in both the base year (1088) sample and the 2000 follow-up survey sample. Weighted N=2.93M.

SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript files (NCES 2003-
402 and Supplement).

188
 Table L13. Mean elasped time to bachelor's degreea for all 1992 12th-graders who
earned the degree by December 2000, by demographics, academic
background, and attendance pattern characteristics

Mean elapsed
calender years Standard Standard
Student characteristic to degree deviation error

All NELS:88:2000 bachelor's 4.58 1.098 0.028

By Academic Resources quintile

Highest quintile 4.23 0.914 0.042

2nd quintile 4.63 1.089 0.045
3rd quintile 4.94 1.091 0.071
4th quintile 4.98 1.206 0.137
Lowest quintile 5.54 1.095 0.161

By socioeconomic status quintile

Highest quintile 4.40 1.018 0.044

2nd quintile 4.66 1.131 0.056
3rd quintile 4.74 1.089 0.060
4th quintile 4.83 1.201 0.084
Lowest quintile 4.87 1.113 0.120

By education anticipations

Bachelor's consistent 4.49 1.051 0.029

Raised to bachelor's 4.90 1.138 0.067
Lowered from bachelor's 5.14 1.303 0.218

By selectivity of first institution

Highly selective 3.92 0.741 0.093

Selective 4.32 0.987 0.071
Non-selective 4.53 1.026 0.031
Open door 5.39 1.165 0.070
Not rates 4.85 1.412 0.481

By number of acceleration credits

None 4.67 1.105 0.034
1-4 4.46 1.006 0.083
5-8 4.38 1.005 0.081
9 or more 4.28 1.101 0.088

189
 Table L13. Mean elapsed time to bachelor's degreea for all 1992 12th-graders who earned
the degree by December 2000, by demographics, academic background, and
attendance pattern characteristics -- continued
Mean elapsed
calendar years to Standard Standard
Student characteristic degree deviation error

By number of schools attended

One 4.30 0.825 0.034

Two 4.66 1.143 0.048
Three or more 5.04 1.254 0.063

By transfer status

Community college to four-year 5.43 1.162 0.071

Four-year to four-year 5.09 1.197 0.060
No transfer 4.43 1.015 0.030

By continuity of enrollment

Continuously enrolled 4.44 0.941 0.024

Non-continuous 6.68 1.114 0.101

By number of remedial courses

None 4.43 1.042 0.033

One 4.94 1.117 0.071
More than one 5.33 1.092 0.068

Bachelor's degree major

Business 4.56 1.083 0.072

Education 4.72 1.193 0.088
Engineering/architecture 4.71 1.019 0.089
Math/computer science 4.45 0.915 0.161
Physical sciences 4.99 0.293 0.194
Biological sciences 4.41 0.998 0.063
Health sciences/services 4.66 1.131 0.081
Humanities 4.35 1.134 0.143
Fine and performing arts 4.61 1.830 0.144
Social sciences 4.50 1.062 0.058
b
Applied social sciences 4.68 1.033 0.066
a
From first true date of attendance to degree award date.
b
Includes communications, public administration, criminal justice, social work, family and
community services, and leisure studies/recreation.
SOURCE: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript files
(NCES 2003-402 and Supplement).

190
THIS PAGE INTENTIONALLY LEFT BLANK

191
 GLOSSARY
VARIABLES AND THEIR CONSTRUCTION

This glossary serves not only to expand the definition of the variables considered for The Tool-
box Revisited, but also to provide details on their construction and allied guidance for
interpretation that the reader may find helpful. As in Part II of this study, the presentation
follows the order of the seven steps of the logistic narrative.

Where the initial definition of the variables in Part II was simple and transparent, e.g., GENDER,
no further elaboration is provided here. Nearly all these variables were either taken directly or
derived from other variables on NCES 2003-402 and its June 2004 Supplement.

Step 1a: Demographic Background

NNSE (Nonnative speaker of English)—New in this analysis.

The most notable demographic change between postsecondary students from the High School
Class of 1982 and those from the High School Class of 1992 was a doubling of those who not
only came from households where English was a second language, but who themselves were
users of the heritage language. More than half of both Asian and Latino postsecondary students
were so classified (Adelman 2004a, table 1.3, p. 9).

IMMIG (Family immigrant status)—New in this analysis.

Allied to second language background is the immigrant status of families. A dichotomous

variable, IMMIG, was created to indicate whether the student’s parents were immigrants to the
United States within the previous 10 years. Among those who attended a four-year college at
any time, 27.5 percent (s.e. = 3.33) of Asian students and 8.8 percent (s.e. = 1.61) of Latino
students came from recent immigrant families. The dichotomous variable did not meet the
minimum t criterion for retention in the demographic logistic model. Nonresponse bias
undermines this variable: Responses from nearly 15 percent of the NELS:88/2000 families are
missing.

BROSIS (Number of siblings)—New in this analysis.

Some research has indicated that students from larger families have more difficulty entering and
successfully participating in higher education (Stage and Hossler 1989; Leigh and Gil 2003;
Perkhounkova, McLaughlin, and Noble 2004). The dichotomous variable, BROSIS, marks
students with three or more siblings versus those with one, two, or none.

FIRSTGEN (First generation postsecondary student)—New in this analysis.

A dichotomous variable under which the criterion for first-generation status is that neither parent
had attended a postsecondary institution. Parents’ education status was reported by parents
themselves, not by students. The discrepancy between students’ understandings of their parents’
education status and parental accounts was demonstrated in the original Tool Box (p. 36): For the
High School Class of 1982, only 50 percent of the students with at least one parent who had

192
earned a bachelor’s degree (but not a graduate degree) could correctly describe their parents’
achievement. Parents’ highest level of education is a key component of socioeconomic status,
and the original Tool Box did not test it separately.

FAMINC (Family income)—New in this analysis.

Family income in 1991 (the year before NELS students were scheduled to be in the 12th grade)
was reported by parents, and set in six bands with a fairly even distribution of all respondents
across those bands. In 2005 dollars (adjusted for a Gross Domestic Product deflator6 of 1.288),
as shown in figure 4, these bands were:

Figure 4. 1991 family income bands used in the NELS:88/2000 in 2005 dollars

Income band Income range ($000)

Highest 96.6 k and higher

Second 64.4 - 96.5
Third 45.1 - 64.3
Fourth 32.2 - 45.0
Fifth 19.3 - 32.1
Lowest Less than 19.3 k
SOURCES: National Center for Education Statistics: NELS:88/2000 Postsecondary Transcript files
(NCES 2003-402 and Supplement); Budget of the United States Government: Historical Tables,
Fiscal Year 2005, table 10.1.

For purposes of our analysis, the bands were combined in pairs—highest two, middle two, and
lowest two—to produce a trichotomy. This version is used in the Step1a logistic model
(demographic characteristics only), along with FIRSTGEN, as one of the proxies for
socioeconomic status.

NEWCHILD (Parenthood by age 20)—Carried forward from Answers in the Tool Box.

The variable, abbreviated as NEWCHILD, is a dichotomous marking of students (of either sex)
who became parents by age 20. It was a powerful, negative, and consistently significant variable
through all stages of the logistic story of the HS&B/So. A decade later, the rate of early
parenthood had changed. For the entire cohort of the HS&B/So, 11.6 percent (s.e. = 0.43)
became parents by age 20, a proportion that rose to 16.4 percent (s.e. = 0.92) for the
NELS:88/2000. For those who ever entered a postsecondary institution, these percentages drop
in half. Georges (2000) would remind us of the common sense that students who became parents
by age 20 were not likely to enter college at all by that time. Indeed, 59.4 percent (s.e. = 2.30) of

6
For the GDP deflator calculator, go to http://www1.jsc.nasa.gov/bu2/inflateGDP.html

193
the NELS:88/2000 participants who had children by 1994 did not enter any postsecondary
institution by 2000, compared with 20.9 percent (s.e. = 0.89) of those who did not become
parents by 1994.7

SESQUINT (Socioeconomic status quintile)—Carried forward from the original Tool Box.

In NCES longitudinal studies, socioeconomic status (abbreviated SES) is a composite index of

family income, parents’ highest level of education, prestige of parents’ occupations, and the
presence of items such as books, regular newspapers, and a computer in the student’s household.
Parents provide this information, and the 1992 parent interview was the primary source. The
NELS:88/2000 files presented this index in centiles, as did the HS& Beyond/So file before it. To
match quintile presentations of high school curriculum intensity, high school class rank/Grade
Point Average, 12th-grade test score, and the composite variable, Academic Resources (see
below), the centile scale was collapsed to quintiles. SES is an anchor of the education attainment
literature, with recent studies enlightening our understanding of the complex ways students of
different SES backgrounds experience higher education (e.g., Walpole 2003).

Step 1b: High School Background

EDUANTIC (Education anticipations)—Modified from Answers in the Tool Box.

In both the original Tool Box and The Toolbox Revisited, this is a composite variable describing
the consistency and level of the student’s education expectations, hence, goal commitment (Allen
and Nora 1995). It departs considerably from measures of student education “aspirations” that
are based on the student’s answer to a single question at one moment in time. The variable was
built from matched pairs of questions asked in grades 10 and 12. The High School &
Beyond/Sophomore cohort surveys included six pairs of questions: highest level of education
expected, planned dominant activity in the year following high school graduation, timing of
college entry, choice of a two-year or four-year college as entry point, lowest level of education
with which the student would be satisfied, and whether the student would be disappointed if he
or she did not graduate from college.

The NELS:88/2000 version does not have as strong a base. Only two pairs of questions were
asked in grades 10 and 12 that feed into the construction: highest level of education expected and
timing of college entry.8 In grade12 only, the respondent was asked what was the most likely
type of postsecondary institution he or she would enter: four-year, two-year, or trade school, and
the answer to this question also was used in building the variable. But the lack of parallelism

between the HS&B/So survey and the NELS:88/2000 lead to revisiting the NELS categories and
applying stricter algorithmic standards to yield an education anticipations variable closer to the
7
These percentages are based on the entire panel of participants in 1992 who also participated in the 1994 and 2000
follow-up surveys.
8
Georges (2000) used a similar level-of-expectations by consistency-of-expectations construct for parental visions of
their children’s future education in the NELS:88/2000 cohort. Parents were interviewed on this issue when students
were in the eighth grade and again when students were scheduled to be in the 12th grade.

194
structure of its predecessor, as displayed in figure 5.

Even after this construction, the education anticipations variable required an adjustment. Unlike
the quintile variables for curriculum, class rank/GPA, and senior year test score, in which
meanings are linear, the five-level anticipations variable is categorical. For purposes of
multivariate analysis, the five levels were reduced to three as follows: (1) consistently expected
to earn a bachelor’s degree; (2) raised expectations to the bachelor’s degree between grades 10
and 12; and (3) either lowered expectations from the bachelor’s level between grades 10 and 12
or never expected to earn a bachelor’s degree. The result is still a categorical variable, but the
reference point—expectations for earning a bachelor’s degree—is now constant in all three
values.

Figure 5. Comparison of the education anticipations composite variable in the High

School & Beyond/Sophomore cohort and the NELS:88/2000: Changes in
students’ expectations between grade 10 and grade 12

HS& B/So NELS:88/2000

Bachelor’s consistent Bachelor’s consistent

Raised to bachelor’s Raised to bachelor’s

Lowered from bachelor’s to some college; Lowered from bachelor’s to some

associate consistent college, inconsistent

Lowered to no degree; Raised to some college; consistent

associate inconsistent “some college”

No degree plans, don’t know Sub-baccalaureate, including high

school diploma or less

SOURCES: National Center for Education Statistics: High School & Beyond/Sophomore
cohort, NCES 2000-194; NELS:88/2000 Postsecondary Transcript Files, NCES 2003-402.
CLSSRNKQ (Class rank/GPA quintile)—Carried forward from the original Tool Box.
This variable is a composite of high school class rank and GPA quintile. In both the HS&B/So
and NELS:88/2000 data sets the composite was constructed in roughly the same way. The
construction is described in detail in Appendix C, but the key features are worth repeating here.

The construction of this variable began with class rank, expressed as a percentile, for students
whose high school graduating classes were larger than 10. Class rank was chosen for the base

reference because it overrides variability in local grading practices,9 and responds to the
9
Among the divergent policies found by the College Board (1998): (a) 9 percent of high schools do not use A-F or
numeric grading systems; (b) 8 percent of high schools do not give credit for grades lower than C-; (c) 19 percent of

195
divergence between stable scores on standardized tests such as the SAT or ACT and rising GPAs
among the college bound (College Board 1998). Not all high schools compute class rank,10 so
for a significant percentage of students, this datum is missing. For these missing cases (as well
as for the students from very small high schools), the variable construction turns to high school
GPA (where available, and only for students with three or more years of course work). Basing a
percentage scale first on class rank, and filling in the missing values with available GPAs from a
matched scale using an equipercentile concordance methodology (Houston and Sawyer 1991),
weighting the combined scale and cutting it by quintiles solves some of the bias and validity
problems that would result from relying on GPA alone.

SRTSQUIN (Senior year test score quintile)—Carried forward from the original Tool Box.

SRTSQUIN is the quintile version of a senior year test score. Comparing the HS&B/So to the
NELS:88/2000, this variable had the least variation in construction among the precollegiate
academic performance variables. In both data sets, composite scores on an “enhanced mini-
SAT” given to survey participants in the 12th grade were set out in percentiles. For missing
cases, ACT scores were first converted to the SAT scale using ACT’s equipercentile
concordance methodology (accounting for the 1989 revision of the ACT test battery), and a
combined SAT/ACT variable constructed. Scores on this variable were set out in percentiles,
and filled in for missing cases of the senior year test score percentile using the equipercentile
concordance method. SRTSQUIN reduces the noise in the lumpy distribution with a quintile
presentation. Despite these efforts, 14 percent of the students in the NELS:88/2000 database are
missing a senior year test measure, compared to 9 percent in the HS&B/So (see Appendix C). It
must be acknowledged that the construction of SRTSQUIN combines low-stakes test scores with
high-stakes test scores, though it is unclear how that ultimately affects a quintile presentation.

HIGHMATH (Highest level of mathematics reached in high school)—Carried forward from

the original Tool Box.

In the original construction of this variable in both the HS&B/So and NELS:88/2000 data sets,
some editing of the original coding of high school mathematics courses was required in light of
postsecondary transcript evidence. For example, if the original coding in 1992 judged the
student’s highest level of high school mathematics to be "geometry," and the student’s
postsecondary transcript record of 2000 showed enrollment and completion of "Calculus III" in
the first semester of college, then it was obvious that the original coder did not know the

difference between "geometry" and "analytic geometry," and the student’s high school record had
to be adjusted accordingly. HIGHMATH is considered a categorical variable (as opposed to a
continuous variable) because the distance between its levels cannot be judged as ordinal.

high schools are in districts where grading systems vary; and (d) 43 percent of high schools exclude some courses
when calculating GPA.
10
In its survey of high school grading practices, the College Board (1998) found that 19 percent of high schools did
not calculate class rank.

196
SCIMOM (High school momentum in science and mathematics)—New in this analysis.

This variable has three values: (1) A student who reached a level of math beyond Algebra 2 and
who earned three or more Carnegie units in core laboratory science was labeled as having
sufficient momentum in STEM (Science, Technology, Engineering, and Mathematics)-oriented
fields to enhance that momentum even further in a postsecondary environment (whether or not
they chose to do so). (2) Students who reached Algebra 2 but no higher, and accumulated more
than 1.5 units of core laboratory science exhibited "modest" science momentum, and (3) those
whose high school records fell short of "modest" on either the mathematics or science criteria
were labeled "weak-to-minimal" in terms of science momentum. SCIMOM is one of three
variables tested together as proxies for the combined measure of the academic intensity of a
student’s high school curriculum, HSCURRQ.

FLAN (Number of units of foreign language in high school)—New in this analysis.

This variable was developed as part of a configuration of proxies for academic curriculum
intensity so that we could test the proxies against the consolidated academic curriculum index,
HSCURRQ, described on pages 24–27 and in Appendix F. FLAN indicates the number of
Carnegie units earned in high school foreign language study, and has five values: more than 3
units, from 2.01 to 3 units, from 1.01 to 2 units, from 0.1 to 1 unit, and 0 units. FLAN is one of
three variables tested as proxies for HSCURRQ.

ADVANCE (Number of Advanced Placement courses)—New in this analysis.

This variable was constructed with three values based simply on the number of AP courses
recorded: three or more, one or two, and none. It was tested, along with SCIMOM and FLAN,
as a proxy for the consolidated academic curriculum index, HSCURRQ.

The identification of Advanced Placement course work in the NELS:88/2000 was a complex
task. The version of the NELS:88/2000 high school transcripts to which researchers have
referred in the past understates participation in AP courses, though it was difficult to say precise-
ly by how much. Two proxy measures were employed to create the first version of an AP varia-
ble on the data file used in this study:11 AP test scores recorded on the high school transcripts and
AP entries on the postsecondary transcripts (which do not include test scores). These were com-
bined to yield 7 percent of the NELS:88/2000 12th-graders, compared with the College Board’s

estimate of approximately 11 percent of 1992 12th-graders having taken AP courses.12 Not

everyone who takes AP courses takes AP examinations, and while that might explain some of the
gap in the estimates, it still left an uncomfortable discrepancy between the NELS and College
Board accounts of Advanced Placement volume.
11
NCES CD 2003-402.
12
See indicator #14 in The Condition of Education, 1999. Washington, D.C.: National Center for Education
Statistics, 1999, p. 56.

197
In light of this low yield and anomalies in the data, a new approach was taken in the work that
produced the 2004 Supplement to the original NCES 2003-402 files. Enough new cases of
AP course taking (not courses labeled "honors" or just "advanced") were identified to raise the
proportion of NELS 12th-graders with AP courses to 9.4 percent.13

Step 2: Postsecondary Entry

FIRST4 (First school attended was a four-year college)—Carried forward from Answers in
the Tool Box.

The dichotomous variable indicating whether the first institution attended by the student was a
four-year school was active in the original Tool Box account, but did not qualify for entry into the
model in this replication. It is not clear why this should be the case. Some 81.9 percent
(s.e. = 0.82) of the class of 1982 who attended a four-year college at any time started at a four-
year college, versus 78.1 percent (s.e. = 1.04) of the class of 1992. There is no meaningful
difference in those numbers.

DOCT (First school attended was a doctoral degree-granting institution)—Carried forward

from the original Tool Box.

Another dichotomous variable indicating that the first institution attended by the student was not
only a four-year college, but a doctoral degree-granting school. In the original Tool Box, this
variable was admitted to the steps of the logistic model but was never statistically significant. In
The Toolbox Revisited analysis, this variable did not qualify for entrance in the model, even
though roughly similar percentages of students who attended a four-year college at any time
started in a doctoral degree-granting institution (33.8 percent; s.e. = 1.06 for the HS&B/So
versus 34.1 percent; s.e. = 1.11 for the NELS:88/2000).

The variable indicating the selectivity of the first institution of attendance (SELECT) is, to a
large extent, confounded with doctoral degree-granting status (55 percent of the NELS:88/2000
students who started at doctoral degree-granting institutions started at either selective or highly
selective schools).

SELECT (Selectivity of first institution attended)—Carried forward from the original Tool
Box.

A dichotomous variable indicating that the first institution attended by the student was either
13
This approach began by isolating students who were consistent in their claims to have taken an "advanced
placement course," planned to take an AP test, and reported taking at least one AP test between 1990 and 1993. From
this group, those for whom AP course work had already been recorded were dropped. The balance were first
matched against the HIGHMATH variable. For students who reached calculus or precalculus, responses to another
question as to whether they took the course to earn college credit were invoked. If the student responded positively,
the record was credited with an AP course. College transcripts were also examined for foreign language study at
advanced levels in the first year of attendance (along with more than 2 years of foreign language study in high
school), etc. Where matches were found, the student record was credited with an AP course.

198
highly selective or selective.1 Some 4.1 percent (s.e. 0.49) of the universe for this study started
in highly selective institutions (e.g., Princeton, the University of California at Berkeley, and
Harvey Mudd). Another 17.8 percent (s.e. = 0.98) began their postsecondary careers in selective
institutions (principally flagship state universities such as Michigan-Ann Arbor, North Carolina-
Chapel Hill, Wisconsin-Madison, and Texas-Austin). More than half of the universe of students
in this study (56.3 percent; s.e. = 1.32) commenced study in nonselective four-year colleges, and
20.7 percent (s.e. = 1.19) in open-door institutions, a category that is just about identical to
community colleges. For the previous cohort of students from the High School Class of 1982,
selectivity of first institution had a modestly positive and statistically significant association with
degree completion until the last step of that analysis, extended postsecondary performance.
Readers interested in the distribution of first institutional selectivity by race/ethnicity, gender,
and socioeconomic status are referred to Appendix L, table L3.

NODELAY (Direct entry to postsecondary education following high school)—Carried

forward from the original Tool Box.

This dichotomous variable was built from a simple distinction: Among students for whom we
know both the month and year of high school graduation and the month and year of a transcript-
documented attendance in a postsecondary institution following high school graduation, is the
difference (a) seven months or less, (b) 8–18 months, or (c) more than 18 months? The first of
these options is regarded as "direct entry." Dual enrollment courses do not count in this
determination. The student who graduates from high school in June has until the following
January to enroll and be judged "no delay." To those who might question the seven month
allowance and argue for four months, one would point to a small percentage of graduates with
April commencement dates who would be judged late entrants in September under a four-month
rule.

ACCELCRD (Credits earned prior to high school graduation and by examination)—New

in this analysis.

ACCELCRD sums all college credits earned by course work prior to high school graduation,
along with credits earned by examination—including Advanced Placement, College Level
Examination Program (CLEP), and institutional challenge exams (the majority of these—in the
records of the NELS:88/2000—in foreign languages). Most of these credits were earned either
prior to matriculation or during the first term of enrollment, though some were earned at later
1
Institutional selectivity in the postsecondary files of all three grade-cohort longitudinal studies conducted by the
National Center for Education Statistics has five values: highly selective, selective, nonselective, open-door, and not
ratable. The first three of these values were based on the selectivity cells developed by the Cooperative Institutional
Research Project (CIRP) at UCLA for its annual survey (since 1966) of entering freshmen. Community colleges and
area vocational-technical institutes (AVTIs) were assigned the value of “open-door.” Theological seminaries, music
conservatories, and sub-baccalaureate vocational schools were considered “not ratable.”

199
points in the student’s undergraduate career. The ACCELCRD values were set at three levels:
more than 4 credits, 1–4, and zero.

Step 3: First-year Performance

LOWCRED (Less than 20 additive credits earned in the first calendar year of
attendance)— Carried forward from Answers in the Tool Box.

A dichotomous variable built on an analysis of the mean number of additive credits (those that
count toward degrees) earned by all postsecondary students in the first calendar year following
the date of first enrollment. The threshold at which this broad population began to earn any
credentials (including certificates and associate degrees) was 20 credits (in the NELS:88/2000
population, 20.252 credits, to be precise; s.e. = 1.016). LOWCRED is an early momentum
indicator with enduring consequences. As table L4 in Appendix L demonstrates, the more
additive credits earned in the first year, the more likely a degree will be completed.

FRSHGRAD (Grade point average in the first calendar year of attendance)—Carried

forward from the original Tool Box.

Grade point averages were determined for the first full calendar year of postsecondary
attendance, and were then set out in quintiles. FRSHGRAD is a dichotomous variable that
divides the highest two quintiles from the other three. For the history of the HS&B/So used in
the original Tool Box study, the dividing line was 2.70. For that of the NELS:88/2000 used in
The Tool- box Revisited, the line is drawn at 2.88. The difference reflects the general trend in
GPAs of the two cohorts (see Adelman 2004a, table 6.1, p. 78).

FREM (Any remedial course work in the first calendar year of attendance)—New in this
analysis.

FREM is a dichotomous variable marking students who took any remedial courses during the
first calendar year of attendance. Three variations on first-year remediation were tried out in the
process of arriving at this formulation: one that focused only on remedial reading, another on
remedial math, and a third on all types of remediation. The dichotomous version of the latter
was the only form that could meet the minimal statistical criteria for entrance into the stepwise
logistic model and allow us to track any association of early remediation with degree completion.

Step 4: Financial Aid

In general, the NCES grade-cohort longitudinal studies have been weak on financial aid
information, where as the event-cohort Beginning Postsecondary Students longitudinal studies
have offered a panoply of financing data. For example, if we were to ask about grants or
scholarships received in the first year of attendance (1995–96) for the BPS:95/96–2001 by
traditional-age students who attended a four-year college at some time, we would find the
information indicated in table Glossary-1.

200
We would know, too, that for those who received grants/scholarships in 1995/96, the amount
constituted an average of 69.3 percent (s.e. = 1.0) of all aid received, and covered 33.4 percent
(s.e. = 0.7) of the price of attendance for the 1995–96 academic year. We could easily break
these averages out by type of institution first attended. And we would know more, as the
BPS:95/96–2001 covers PLUS (Parent Loan for Undergraduate Students) loans, Perkins loans,
Stafford loans (subsidized and unsubsidized), need-based aid by source, merit-aid by source, Pell
grants, SEOG (Supplementary Educational Opportunity Grants) grants, and other types of
financial aid. If the grade-cohort longitudinal studies such as the NELS:88/2000 had gathered
even a third of this information, a much richer analysis would be possible—though not
necessarily an analysis demonstrating a significant association of types, amounts, and ratios of
financial aid to degree completion.

Table Glossary-1. Percentage of 1995/96 beginning postsecondary students 20 years old and younger
who attended a four-year college at any time, and received grants/scholarships in
1995/96, by source and average amount of award, and bachelor's degree attainment
rate by 2001

Percent earning bachelor's degree by

2001
Percent Did not receive
receiving Average amount Received grant or grant or
Source of grant grant of grant scholarship scholarship

All sources 54.6 (1.3) $4506.00 (170.7) 59.8 (1.4) 49.1 (1.9)
Federal 23.2 (1.0) 1958.60 ( 34.0) 47.8 (1.9) 57.1 (1.4)
State 19.1 (1.0) 1757.40 ( 68.8) 58.6 (2.0) 54.0 (1.4)
Institution 31.9 (1.3) 4312.00 (208.0) 67.5 (1.7) 49.0 (1.5)
Other 16.1 (0.7) 1529.60 ( 73.5) 71.1 (2.0) 51.9 (1.4)
NOTES: Standard errors are in parentheses.
SOURCE: National Center for Education Statistics: Beginning Postsecondary Students Longitudinal Study,
1995/96-2001. Data Analysis System

Step 5: Attendance Patterns

FOURTRAN (Transfer from one four-year college to another)—New in this analysis.

The algorithm for classic transfer from a community college to a four-year college was fairly
easy to construct. But to distinguish a true four-year-to-four-year college transfer required an
indirect route. In the final survey of NELS longitudinal study (2000), respondents who attended
any postsecondary institution(s) were asked whether they ever "transferred credits." One cannot
equate the yes/no answer to that question with institutional transfer. One group of four-year-to-
four-year transfers was easy to identify: those who started in a four-year college and earned a
bachelor’s degree from a different four-year college. The second group was defined by the

201
following attendance characteristics: They started in a four-year college, attended at least two
four-year colleges, earned more than 30 credits from four-year colleges, were not reverse
transfers, and earned less than 20 credits from community colleges and other sub-baccalaureate
institutions. For the most part, these students did not earn bachelor’s degrees, but they
accumulated an average of 85.6 credits (s.e.=3.57) from four-year colleges. Of those who did
not earn a degree, 34.6 percent (s.e. = 4.17) were still enrolled and degree-candidates in 2000.

MULTINS (Student attended more than one postsecondary institution as an

undergraduate)—Carried forward from Answers in the Tool Box.

MULTINS is a dichotomous variable indicating that the student attended more than one
institution. This is a macro-vision of otherwise multidirectional student behavior. One might
suspect that when MULTINS, TRANSFER, and FOURTRAN are in the same logistic model, at
least one of these variables will evidence an unacceptable degree of collinearity. But within the
context of the universe under study in The Toolbox Revisited, the correlation between community
college transfer (TRANSFER) and MULTINS is 0.3363 and that between four-year-to-four-year
transfer (FOURTRAN) and MULTINS is 0.3246. These are modestly positive, but not so high as
to indicate collinearity problems. What that may mean is that the three independent variables are
making distinctly different statements in the context of the logistic regressions in which they
appear.

SUMMER (Number of credits earned during summer terms)—New in this analysis.

Attendance and credit generation during summer terms turned out to be a strong proxy for high-
octane persistence among community college students (Adelman 2005a), and was, hence,
brought into the model for the population of those who attended a four-year college at any time.
The variable was set by three bands of the number of credits earned in this manner: more than 4,
1–4, and 0. Some 54.8 percent (s.e. = 0.92) of all 1992 12th-graders who continued their
education earned credits in at least one summer term; 61.1 percent (s.e. = 0.96) of those who
earned more than 10 credits did so, as did 63.2 percent (s.e. = 1.10) of the universe on which this
essay focuses. Given the magnitude of those percentages, it is not so much the fact of using
summer terms, but the volume of credit generation that should count.

PARTTIME (Was the student’s enrollment intensity ever part-time?)—New in this analysis.

The original Tool Box declined to confront part-time status and its effects. If one is using
transcripts as evidence, there are a number of problems in determining which students are part-
time and when. This issue deserves extended attention, with support drawn from very different
data sources (see, e.g., O’Toole, Stratton, and Wetzel 2003). Even if one standardizes credits on
a semester metric across over 3000 institutions in the NELS:88/2000 postsecondary transcript
files, it is often difficult to know what is considered “part-time” in term X at institution Y in
program Z. With over 60 percent of students earning credits in summer terms, we would be
obligated to include such terms in an account of enrollment intensity, and the question, “What is
‘full-time’ in a summer term?” would have to be answered in each institution. In the last survey
of the NELS:88/2000 cohort in 2000, students were asked whether they had ever “attended less
than full time.” Despite the ambiguity of the phrasing and students’ understanding of what "less
than full time means," student responses were accepted as the first stage of creating a variable

202
indicating part-time status at some point in their careers. In their 1994 interviews, students were
also asked about their enrollment intensity status at the first institution they attended. When they
indicated part-time, that fact overrode any denial of part-time status in their 2000 interview.
Proximate judgment supercedes retrospection.

The second stage in developing this rough proxy draws on the transcripts and the notion of
reducing credit load to part-time status. To identify students who may start with a full-time
schedule but reduce it to part-time intensity by withdrawing from courses, we take the average
credit load per semester for bachelor’s degree recipients—14.2 (average annual load of 30.8
minus the sum of summer-term credits, credits by examination, and dual enrollment credits
divided by two)—and ask by what percentage that figure would have to fall to bring the student’s
credit load below the threshold for full-time status (12 credits). The answer is 16.2 percent. So
for the sake of this crude proxy measure, it is assumed that students who withdrew from
attempted credits at 16.2 percent rate or higher were—or became—part-time at some time. With
that group added to students who indicated part-time status in the course of their 2000
interviews, the proportion of NELS students who were part-time at some time in the
undergraduate careers was 47.4 percent—compared with 48.2 percent of traditional-age students
in the BPS longitudinal study of 1995/96–2001 (for an account of enrollment intensity in both
data sets, see Appendix L, table L5).

Step 6: Extended Postsecondary Performance

REMPROB (Nature and intensity of remedial work)—Carried forward from Answers in the
Tool Box.

In both the original Tool Box and The Toolbox Revisited, REMPROB describes the nature and
extent of a student’s remedial course work without the boundary of the first calendar year of
attendance. For that reason, when REMPROB is tried out in the logistic narrative at the stage of
extended postsecondary performance, it replaces the freshman year remediation variable
(FREM).

REMPROB is created by an if-then-else logic that starts with remedial reading as the most
serious of remedial problems,2 and works its logical steps as follows:

1. Any remedial reading

2. Two or more remedial courses (not reading, and, if math, more than two)
3. One or two courses of remedial mathematics only
4. One remedial course (not reading, not math, thus leaving writing,
language arts, and general basic skills)
5. No remedial course work

For the sake of presentation in the logistic sequence, REMPROB is a dichotomous variable in
which remedial reading plus more than one remedial course falls on one side of the border, and
all other remediation (including no remediation) on the other side. In the original Tool Box,
REMPROB was admitted into the model, but did not reach the threshold of statistical signifi-
2
For an elaboration of REMPROB in relation to student demographic backgrounds, high school performance, and
degree attainment, see Adelman 2004a, pp. 87–94, and Wirt, et al. (2000), indicator 14, p. 52 and supplemental
tables (p. 152).

203
cance. In The Toolbox Revisited, REMPROB did not meet the minimum statistical standard for
entry into the model when it replaced the freshman year remediation variable in Step 6.
TREND (Trend in student’s grade point average)—Modified from previous presentation in
the original Tool Box.
This variable describes the trend in a student’s postsecondary grade point average (GPA), and
applies only to those students with complete undergraduate transcript records and computable
GPAs. In its original Tool Box version, two reference points were used: end of the first calendar
year GPA, and final undergraduate GPA. From these two markers, one could observe whether
GPA was rising, flat, or falling, using a ratio of final GPA to first-year GPA. Ratios in the .95 to
1.05 range were judged to indicate no change, with ratios above 1.05 indicating a rising GPA,
and ratios below 0.95 indicating a falling GPA. For a slightly better set of markers, the
NELS:88/2000 postsecondary transcript files thus added a variable for GPA two years after entry,
thus providing three points of reference. With three points of reference, ratios are not invoked,
rather relationships (greater, less than, or equal to)—and at three points in time. From the
permutations of these relationships one can determine whether the trend was rising,
flat/inconsistent, or falling.

CUMMATH (Number of credits earned in college-level mathematics)—New in this analysis.

This variable, setting out cumulative credits earned in college-level mathematics in three bands
(more than four, 1–4, and 0) replaces the parallel variable for first-year college-level
mathematics credits in the context of an extended postsecondary history account. The
motivation for including this curriculum marker is the same: Even if postsecondary students do
not concentrate their studies in science, technology, engineering, or mathematics/computer
science (STEM), their quantitative background will be called upon (in business and the social
sciences, in particular). Student academic momentum, it is hypothesized, will be as much
enhanced by quantitative study on the post-matriculation side of education history as it was in
secondary school.

CHANMAJ (Change of major)—New in this analysis.

If transcripts are our principal source of evidence, the only moment at which we are sure of a
student’s major is the moment of degree award. Sometimes, the NELS:88/2000 transcripts with
no degree indicated a major, which was verified by hand-and-eye reading by two judges only
when the transcript carried a minimum of 15 credits and the major field indicated carried a
minimum of 12. But that still leaves a large swath of missing accounts of students’ major. It is
wholly possible, for example, that a student earning a B.S. in computer science changed majors
twice along the way, and in two different institutions. It is also possible that a nondegree
transcript had no disciplinary character to it at all while the student indicated he or she was
majoring in X. In the matter of major—and change-of-major—the transcript is not as credible
as the student. So the last NELS survey in 2000 asked students whether they had changed
majors at any time, but unfortunately did not ask “from what to what.”

A dichotomous change-of-major variable was developed and tried out for this study. Basically, it
took the universe of students who said they changed majors, and added to that universe
community college transfer students who moved from general studies to a discrete major, and
cases where the student presented two transcripts with no degrees but different majors. It also
took student responses in 1994 to questions about their major at the first institution attended and
that at the most recent institution attended. Where the student reported two different disciplinary
majors, e.g. electrical engineering at the first institution and biology at the most recent

204
institution, that, too, was marked as a change of major no matter what the student told
interviewers in 2000. Unfortunately, all these steps created a collinearity problem with the
TRANSFER variable, and CHANMAJ did not qualify for entrance into the logistic narrative.
On the other hand, it was admitted and played a notable role in the linear account of time to
degree (see Appendix J).
Step 7: Final Model
NOSTOP (Continuous enrollment)—Carried forward from Answers in the Tool Box.

Given the extended potential undergraduate periods of 12 years (National Longitudinal Study of
the High School Class of 1972), 11 years (High School and Beyond/Sophomores), or 8.5 years
(NELS:88/2000), it makes no sense to describe stop-out behavior as one semester or its
equivalent. In all three NCES postsecondary transcript-based grade-cohort studies, non-
continuous enrollment was defined as more than a one semester (or its equivalent, e.g., two
quarters) stop-out period. In the dichotomous variable, NOSTOP, the student is considered
continuously enrolled even with one semester (or two quarters) off. From the moment of its
introduction, NOSTOP was the strongest independent variable in the logistic narrative of the
original Tool Box.

WRPT Ratio (Ratio of non-penalty withdrawal and no-credit repeat grades to all grades
received)—Carried forward from the original Tool Box.

In the original Tool Box, this variable was called the DWI (drops, withdrawals, and incompletes)
index. DWI is not a wholly accurate acronym. What is really described is a ratio in which the
number of courses from which the student withdrew without penalty plus those the student
repeated is the numerator, and the total number of courses in which the student enrolled is the
denominator. The ratio counts course attempts, not credits. Withdrawals without penalty are not
the same as courses “dropped” within set periods that most colleges and community colleges
mark for “drop-and-add.” Courses “dropped” are not included in the ratio. Nothing here
involves penalty grades. All cases at issue are noncredit grades. The variable WRPT ratio is
dichotomous: On one side of the dividing line are students who withdrew from or repeated 20
percent or more of all courses in which they enrolled.

205