Journal of Experimental Psychology: © 2013 American Psychological Association

Learning, Memory, and Cognition 0278-7393/14/$12.00 DOI: 10.1037/a0034057

2014, Vol. 40, No. 1, 41– 65

Dual-Retrieval Models and Neurocognitive Impairment

C. J. Brainerd, V. F. Reyna, C. F. A. Gomes, Alzheimer’s Disease Neuroimaging Initiative

A. E. Kenney, C. J. Gross, E. S. Taub,
and R. N. Spreng
Cornell University

Advances in dual-retrieval models of recall make it possible to use clinical data to test theoretical
hypotheses about mild cognitive impairment (MCI) and Alzheimer’s dementia (AD), the most common
forms of neurocognitive impairment. Hypotheses about the nature of the episodic memory declines in
these diseases, about decline versus sparing of specific processes, and about which individuals will
become impaired over time can all be rigorously tested. Basic theoretical principles, such as whether
recollection and reconstruction are distinct retrieval processes, can also be evaluated. In 3 studies,
measurements of recollective retrieval, reconstructive retrieval, and familiarity judgment were extracted
from standard clinical instruments, for healthy subjects and for subjects with MCI and AD diagnoses.
Differences in reconstructive retrieval consistently distinguished MCI and AD, in nationally represen-
tative subject samples as well as in highly educated samples, and recollective retrieval also distinguished
them in highly educated samples. Dual-retrieval processes were accurate predictors of future
conversion to MCI and AD over periods of 1.5– 6 years and were better predictors than the best
genetic marker of these conditions (the ε4 allele of the APOE genotype). The standard recollection-
deficit account of memory declines in MCI and AD was not supported, but the data were consistent
with an alternative account that stresses the increasing importance of reconstruction deficits as older
adults convert to these diseases.

Keywords: dual-retrieval processes, memory deficits, Alzheimer’s dementia, mild cognitive impairment,
APOE genotype

Supplemental materials: http://dx.doi.org/10.1037/a0034057.supp

In this article, we illustrate how research on dual-process models The work that we report had two aims, one theoretical and the
of retrieval can be advanced by extending those models to the most other empirical. The theoretical objective was to connect dual-
common forms of neurocognitive impairment in older adults and process models to neurocognitive impairment by using them to
to the types of clinical data that are used to diagnose these diseases. pinpoint the nature of memory declines in mild cognitive impair-

This article was published Online First August 26, 2013. LLC; NeuroRx Research; Novartis Pharmaceuticals Corporation; Pfizer
C. J. Brainerd, V. F. Reyna, C. F. A. Gomes, A. E. Kenney, C. J. Gross, Inc.; Piramal Imaging; Servier; Synarc Inc.; and Takeda Pharmaceutical
E. S. Taub, and R. N. Spreng, Department of Human Development, Cornell Company. The Canadian Institutes of Health Research is providing funds
University; Alzheimer’s Disease Neuroimaging Initiative. to support ADNI clinical sites in Canada. Private-sector contributions are
Preparation of this article was supported by National Institutes of Health facilitated by the Foundation for the National Institutes of Health (www
(NIH) Grant 1RC1AG036915 to C. J. Brainerd and V. F. Reyna and by the .fnih.org). The grantee organization is the Northern California Institute for
CAPES Foundation (BEX 0328/12-0) to C. F. A. Gomes. The HRS (Health Research and Education, and the study is coordinated by the Alzheimer’s
and Retirement Study) is sponsored by the National Institute on Aging Disease Cooperative Study at the University of California, San Diego.
(Grant NIA U01AG009740) and is conducted by the University of Mich- ADNI data are disseminated by the Laboratory for Neuro Imaging at the
igan. Data collection and sharing for this project were funded by the University of California, Los Angeles. This research was also supported by
Alzheimer’s Disease Neuroimaging Initiative (ADNI; National Institutes NIH Grants P30 AG010129 and K01 AG030514.
of Health Grant U01 AG024904). ADNI is funded by the National Institute Some of the data used in preparation of this article were obtained from
on Aging, by the National Institute of Biomedical Imaging and Bioengi- the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (ad-
neering, and through generous contributions from the following: Alzhei- ni.loni.ucla.edu). As such, the investigators within the ADNI contributed to
mer’s Association; Alzheimer’s Drug Discovery Foundation; BioClinica, the design and implementation of ADNI and/or provided data but did not
Inc.; Biogen Idec Inc.; Bristol-Myers Squibb Company; Eisai Inc.; Elan participate in analysis or writing of this report. Individual members of the
Pharmaceuticals, Inc.; Eli Lilly and Company; F. Hoffmann–La Roche Ltd. Alzheimer’s Disease Neuroimaging Initiative are listed in the supplemental
and its affiliated company Genentech, Inc.; GE Healthcare; Innogenetics, materials.
N.V.; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research and De- Correspondence concerning this article should be addressed to C. J.
velopment, LLC; Johnson & Johnson Pharmaceutical Research & Devel- Brainerd, Department of Human Development, Cornell University, Ithaca,
opment LLC; Medpace, Inc.; Merck & Co., Inc.; Meso Scale Diagnostics, NY 14853. E-mail: cb299@cornell.edu
41
42 BRAINERD ET AL.

ment (MCI) and Alzheimer’s dementia (AD). Although there has sively with old/new item recognition (for a review, see Yonelinas,
been prior work on dual-process conceptions of such declines, the 2002). The dominant methodology, first used by Strong (1913),
present research (a) implemented recall-based methods of measur- has been to enrich recognition with metacognitive judgments that
ing dual processes that avoid limitations of older recognition-based supply information about whether the basis for old/new decisions
methods and (b) measured those processes with data from low- is recollective or nonrecollective. Three procedures account for the
burden clinical instruments that are part of diagnostic batteries. bulk of the literature: (a) remember/know judgments (Tulving,
The empirical objective was to determine whether measurements 1985), which is by far the most extensively used method; (b)
of dual-retrieval processes in well-characterized samples of inclusion versus exclusion judgments, which allow the recollection
healthy control (HC) individuals, individuals with MCI, and indi- and familiarity parameters of Jacoby’s (1991) process-dissociation
viduals with AD have predictive power in identifying older adults model to be estimated; and (c) confidence judgments, from which
who are at risk of future HC ¡ MCI and MCI ¡ AD transitions. estimates of recollection and familiarity components of the
Here, we investigated how well those theory-driven measurements receiver-operating characteristic (ROC) are extracted (Yonelinas,
fare, relative to the best genetic predictor of such transitions, the ε4 1994).
allele of the apolipoprotein E (APOE) genotype. Although these procedures have generated a vast literature, they
In the first section below, we sketch recently developed tech- have stimulated a series of validity challenges (e.g., Heathcote,
niques for measuring dual-retrieval processes with recall tasks. Raymond, & Dunn, 2006). Such challenges, along with other
That section begins with a brief reprise of well-known criticisms of sources of evidence (e.g., Ratcliff, Van Zandt, & McKoon, 1995),
recognition techniques, continues with some problems that are were examined in a seminal review by Malmberg (2008), who
posed when those techniques are used with impaired populations, concluded that available evidence converges on the view that a
and ends with a discussion of how recall-based measurement can single familiarity process is adequate to handle item recognition.
be applied to the clinical memory instruments that figure in neu- Because remember/know, inclusion/exclusion, and confidence are
rocognitive batteries. In the second section, we summarize current defined over recognition, they become problematic as dual-process
theoretical ideas about which processes are responsible for mem- methodologies. In order to make progress on evaluating dual-
ory declines as individuals convert to MCI or AD. In the remaining process conceptions of memory declines in AD and MCI, then,
sections, we report three studies in which recall-based measure- alternative measurement procedures are desirable. However, they
ment was used with samples of HC individuals and individuals must be within the capabilities of subjects who have neurocogni-
with MCI or AD in order to tie particular retrieval processes to tive limitations. We discuss the second problem before summariz-
memory declines in particular conditions (Studies 1 and 3) and ing the alternative procedure that we used.
then to predict future conversion to MCI and AD over periods of
1.5 to 6 years (Studies 2 and 3). Measuring Dual-Memory Processes in Subjects With
Cognitive Limitations
Dual-Retrieval Processes in Recall
Validity challenges aside, customary methods of separating
A key feature of the present research is that we used a recall-
recollective from nonrecollective retrieval share the practical lim-
based technique for measuring dual-retrieval processes, which
itation that they are high-burden methodologies. They require
allows those processes to be measured with the clinical memory
subjects to comprehend and remember instructions as to how to
tests that are included in neuropsychological batteries. The most
introspect on the phenomenological qualities of remembering and
widely used of those instruments, such as the Consortium to
to perform those introspections reliably, which exceeds the capa-
Establish a Registry for Alzheimer’s Disease (CERAD; Morris et
bilities of some populations—with young children (e.g., Ghetti &
al., 1989) recall test and the Rey Auditory Verbal Learning Test
Angelini, 2008) and older adults with neurocognitive impairments
(RAVLT; Rey, 1941), involve a small number of study–test trials
(e.g., Brainerd et al., 2009) being prime examples. Nevertheless,
(e.g., 3–5) on supraspan lists (e.g., 10 –15 words). Although rec-
those populations are important targets of dual-process research.
ognition tasks have traditionally been used to measure dual-
For instance, dual-process hypotheses have been formulated for
memory processes, alternative procedures that rely on recall (as-
MCI and AD (Bugaiska, Morson, Moulin, & Souchay, 2011), with
sociative, free, serial) have recently been developed (Brainerd,
the standard proposal being that conversion to each condition is
Aydin, & Reyna, 2012; Brainerd & Reyna, 2010; Brainerd, Reyna,
characterized by reductions in recollective retrieval coupled with
& Howe, 2009; Gomes, Brainerd, & Stein, 2013), for two reasons.
sparing of familiarity (see below).
The first was a series of critiques of conventional recognition
The high-burden problem can be illustrated by three features of
methodologies (for a review, see Malmberg, 2008), and the second
remember/know, the dominant methodology. First, remember/
was that those methodologies are beyond the capabilities of subject
know instructions are long and complex (see Rajaram, 1996),
groups with substantial cognitive impairment or dementia. We
requiring high school levels of reading comprehension. Second,
comment briefly on these two matters before sketching how dual
although instructions can be read aloud (e.g., Billingsley, Smith, &
processes are measured with recall tasks and discussing similari-
McAndrews, 2002), doing so ensures neither comprehension nor
ties and differences between the recall and recognition conceptions
correct implementation when subjects have cognitive limitations.
of dual processes.
It also creates a new obstacle, relative to written instructions that
can be consulted throughout a recognition test: An additional
Limitations of Recognition Techniques
memory load is imposed, which is a key consideration with cog-
The distinction between recollective and nonrecollective (usu- nitively limited subjects. It might seem that both obstacles could be
ally called familiarity) retrieval has been studied almost exclu- overcome by simplifying and compressing the instructions. How-
 DUAL-RETRIEVAL MODELS 43

ever, the simplifications that are necessary to ensure comprehen- pecially semantic information (e.g., “animal” and “farm” for
sion by subjects with dementia are so drastic that it would be horse). That subjects are able to reconstruct items in this manner
dubious to assume that the same processes are still being mea- is well documented in research on tip-of-the-tongue and feeling-
sured, and when such simplifications are effected, subjects may of-knowing phenomena. In both cases, subjects have been found to
still fail to comprehend the instructions. The third obstacle is that access a range of partial-identifying information about list items
remember/know judgments demand that subjects introspect on the before they can be recalled (e.g., Brown & McNeill, 1966; Hicks
contents of their mental states. Obviously, it is questionable to & Marsh, 2002; Koriat, 1993, 1995; Kurilla & Westerman, 2010;
assume that subjects with the forms of brain atrophy that are Schacter & Worling, 1985). For example, Koriat, Levy-Sadot,
associated with AD and MCI can introspect reliably. Edry, and de Marcas (2003) reported that the semantic features of
It might also seem that these obstacles could be overcome by Osgood’s (1952) model of meaning can be accessed before items
resorting to recognition tasks other than the conventional ones; that are recalled. Reconstruction searches for items that match partial-
is, tasks that do not demand comprehension of complex instruc- identifying features and generates sets of candidate items (e.g.,
tions about how to introspect on the phenomenological qualities of horse, cow, goat, sheep) that are small enough to be processed
remembering. Limited work of that sort has been reported in older within the time constraints of a recall test. As the features that
adults with neurocognitive impairments (e.g., Gallo, Cramer, generate such sets do not uniquely identify studied targets, the sets
Wong, & Bennett, 2012), but such research raises the questions normally include nontargets (cow, goat, sheep). To avoid high
about process comparability that were just mentioned. The main- intrusion rates, a judgment operation performs familiarity checks
stream literature on dual processes in recognition is based squarely on reconstructed items before outputting them, which is how the
on remember/know, process dissociation, and dual-process ROC dual-retrieval model implements the familiarity notion of dual-
data, which means that these procedures are, in effect, the opera-
process conceptions of recognition. On analogy to signal detection
tional definitions of recollection and familiarity. Without validity
models, familiarity signals from reconstructed items are processed
studies linking these traditional metacognitive measures to simpli-
by setting a decision criterion and outputting items whose famil-
fied alternatives, there is a significant risk that very different
iarity exceeds that criterion.1
processes are being measured (e.g., Ghetti & Angelini, 2008).
Recollective and nonrecollective retrieval are quantified by fit-
ting two-stage Markov chains to the data of standard recall para-
Recollective and Nonrecollective Processes in Recall digms, including clinical instruments such as the CERAD and the
RAVLT. Such models deliver tolerable fits to a wide range of
In contrast to metacognitive judgments, simple recall of lists is
within the capabilities of impaired subjects. There is, of course, a recall data throughout the life span (for a review, see Brainerd et
large literature on associative, free, and cued recall in impaired as al., 2009). With respect to clinical instruments, Brainerd et al.
well as healthy older adults, and such tests are part of clinical (2012) and Gomes et al. (2013) showed that tasks in which
neuropsychological batteries that are used to diagnose dementia subjects participate in only three study–test trials per list are
(e.g., Langa et al., 2005). In the research that we report, a model adequate to conduct model fits and obtain identifiable estimates of
that separates recollective and nonrecollective components of re- its parameters. As the statistical machinery for fitting various
call was fit to data from two of the most widely implemented permutations of the dual-retrieval model to data, estimating pa-
clinical recall tests, the CERAD and the RAVLT, using large-scale rameters, and conducting parameter significance tests has been
studies of late-life impairment in which those tests were adminis- presented in prior articles, it is not reprised in this paper. Here, the
tered to well-characterized samples of HC individuals and indi- focus is on the retrieval processes that are measured by the model’s
viduals with MCI or AD. In addition to being low burden, the parameters, which are defined in Table 1. It can be seen that there
dual-retrieval model has the advantage that nothing is added to are separate parameters that measure recollective retrieval (D),
existing neuropsychological instruments. The method of data anal- reconstructive retrieval (R), and familiarity judgment (J). These
ysis is the only thing that changes. parameters do not map with simple, observable features of recall
The dual-retrieval model (Brainerd et al., 2009) posits that items performance; that is, there are no observable aspects of perfor-
are recalled via a recollective operation, called direct access, and a mance that one can point to and say that they are uniquely due to
nonrecollective one, called reconstruction. Reconstruction is ac- one of the parameters. Instead, parameters must be estimated with
companied by a slave judgment operation that evaluates the fa-
miliarity of reconstructed items before outputting them. (It is a 1
Reconstructive retrieval should not be confused with the notion of
“slave” operation in the sense that it is not activated unless recon- “partial recollection,” which figures in recent dual-process recognition
struction is successful.) The recollective operation accesses verba- models (e.g., Rotello, Macmillan, & Reeder, 2004; Wixted & Mickes,
tim traces of list items’ prior presentations directly, without search- 2010). As mentioned, reconstructive retrieval involves regenerating the
ing through traces of other items, and is therefore the faster of the studied item itself (e.g., horse), using stable stored information about it
two forms of retrieval. Direct access is also more accurate than (e.g., some of its semantic features, such as “farm” and “animal”). Partial
recollection shares neither property. The item itself is not regenerated
reconstruction because it produces errorless recall: When an item’s because it is presented to subjects as a recognition probe. The stored
verbatim trace is directly accessed, its surface form is symbolically information that subjects retrieve about that item is not stable stored
reinstated, so that the item can be recalled by simply reading it out information but, rather, consists of arbitrary contextual cues that were
of consciousness. Direct access is a recollective operation because associated with its presentation on the study list. Thus, if horse were
presented in blue script font inside a hexagonal box on the left side of the
it reinstates vivid, realistic details of prior presentations. computer screen, retrieving some but not all of these contextual details
The nonrecollective operation, reconstruction, regenerates items (e.g., “blue” and “script” but not “hexagonal” or “left”) is an example of
from stable episodic traces of partial-identifying information, es- partial recollection.
44 BRAINERD ET AL.

Table 1
Retrieval Processes That Are Measured With Repeated Recall Data and Delayed Recall Data

Process/parameter Definition

Learning to recall
Direct access (recollection):
D The probability that a verbatim trace of an item’s presentation on a study
list can be accessed on a recall test that follows a study cycle
Reconstruction:
R For any item whose verbatim trace cannot be accessed on a recall test
following a study cycle, the probability that it can be reconstructed on
that recall test
Familiarity judgment:
J For any item that is reconstructed on a recall test that follows a study cycle,
the probability that the reconstruction is judged to be familiar enough to
output

Forgetting
Direct access (recollection):
FD On a forgetting test, the probability that the direct access operation fails for
items that could be directly accessed following the last study cycle
Reconstruction:
FR On a forgetting test, the probability that the reconstruction operation fails
for items that could be reconstructed following the last study cycle

a model that takes into account the fact that observed performance administering immediate versus delayed recognition tests. In recall
is not process pure. experiments with the dual-retrieval model, all of these manipula-
tions were found to increase the D parameters (Brainerd et al.,
Dual-Retrieval Processes in Recognition Versus Recall 2009, 2012; Brainerd & Reyna, 2010; Gomes et al., 2013).
In the same vein, some instructive findings on the relation
Dual-process conceptions of recognition versus recall are sim- between recognition and recall indexes of recollection were re-
ilar in some respects and different in others. The major difference ported by Gomes (2013), who combined recall- and recognition-
lies in nonrecollective retrieval, where the recall process is obvi- based measures in a single experiment and correlated them. His
ously more complex than the corresponding recognition process. subjects learned to recall word lists, using a procedure that allowed
In recognition, subjects need not recover target items because they the model in Table 1 to be fit to the data and its parameters to be
are presented as test probes. When probes fail to provoke recol- estimated. After the last recall test, items that subjects had recalled
lection, familiarity checks are executed to determine whether were represented as test probes, and subjects made the traditional
global memory strength is sufficient to warrant accepting them as types of metacognitive judgments that separate recollection from
old. In recall, targets are not presented as test probes, and, conse- familiarity in recognition. Some subjects made remember/know
quently, targets that cannot be recollected must be reconstructed judgments, and others made source judgments (targets had been
from partial identifying information if familiarity checks are to presented in distinctive contexts). For the first group, model pa-
occur. Thus, nonrecollective retrieval simply consists of familiar- rameters were used to compute the proportion of items that were
ity checks in recognition, whereas it consists of item reconstruction recalled recollectively, and remember/know scores were used to
plus familiarity checks in recall. estimate the proportion of items that were recognized recollec-
On the other hand, dual-process conceptions of recollective tively. The recall and recognition indexes were strongly correlated.
retrieval are similar in recognition and recall. In both instances, For the second group, model parameters were used to compute the
recollection involves becoming consciously aware, during the test proportion of items that were recalled recollectively, and accurate/
phase, of what happened when particular target items were pre- inaccurate source judgments were used to estimate the proportion
sented during the study phase. If recollection is indeed similar in of items that were recognized recollectively. Again, the recall and
recognition and recall, the straightforward prediction is that it recognition indexes were strongly correlated. Beyond the theoret-
should react similarly in the two domains to selected manipula- ical similarity between the recall and recognition conceptions of
tions: Manipulations that increase conventional indexes of recol- recollection, then, data on the effects of recollection-oriented ma-
lection in recognition ought to affect the D parameters of the recall nipulations and on correlations between recall and recognition
model. We have reported several experiments that are consistent indexes make a presumptive case for similarity.
with this prediction. For example, consider six manipulations that
have been consistently found to increase recollection in recogni- Dual-Process Accounts of Memory
tion (e.g., Yonelinas, 2002): (a) studying shorter versus longer
Impairment in AD and MCI
lists; (b) studying lists of word pairs versus singletons; (c) studying
lists of low- versus high-frequency words; (d) studying lists of AD is the most common variety of dementia, accounting for
emotionally valenced versus neutral words; (e) administering rec- more than two thirds of dementia diagnoses after age 70 (e.g.,
ognition tests to younger adults versus healthy older adults; and (f) Plassman et al., 2007) and affecting roughly 20% of individuals
 DUAL-RETRIEVAL MODELS 45

after age 75 (Hebert, Scherr, Bienias, Bennett, & Evans, 2003). It that conversion to AD is marked by declines in recollective re-
is often preceded by MCI, a form of cognitive impairment no- trieval, coupled with sparing of familiarity (Dalla Barba, 1997;
dementia (CIND) that is more prevalent than AD. Both are quint- Tse, Balota, Moynan, Duchek, & Jacoby, 2010; Westerberg et al.,
essentially diseases of episodic memory, inasmuch as memory 2006; for a review, see Bugaiska et al., 2011). Recollective decline
impairment is their diagnostic hallmark. With respect to AD, the has also been used to explain collateral deficits that individuals
diagnostic criteria of the Diagnostic and Statistical Manual of with AD display in executive function, language, and object iden-
Mental Disorders (4th ed.; DSM–IV; American Psychiatric Asso- tification—the idea being that tests of these abilities require sub-
ciation, 1994) require clinically significant impairment in episodic jects to recollect specific details in order to perform the focal tasks
memory coupled with clinically significant impairment in at least (see Baudic et al., 2006; Creamer & Schmitter-Edgecombe, 2010).
one of four other domains (executive function, language, motor However, the dominant hypothesis about executive function is that
function, or object identification). In other words, individuals with there are declines in that ability that go beyond memory-induced
AD must show memory impairment, which is why performance on deficits (e.g., Storandt, 2008). It has recently been proposed that
clinical recall instruments such as the CERAD and the RAVLT is conversion to MCI is also characterized by declines in recollection
the best neuropsychological marker of this diagnosis (e.g., de but not familiarity, with recollective deterioration simply being
Jager, Hogervorst, Combrinck, & Budge, 2003). Observed impair- less severe than in AD (Anderson et al., 2008; Serra et al., 2010;
ments must represent declines from earlier levels of functioning in Westerberg et al., 2006). There is, of course, a large literature,
order to exclude individuals who simply perform poorly on mem- using traditional recognition methods of separating recollection
ory tests. Also, impairments cannot be due to other diseases that and familiarity, that favors a recollective locus for age declines in
cause memory declines (e.g., alcoholism, cardiovascular disease, episodic memory among healthy individuals (for reviews, see
diabetes, stroke), but individuals may be assigned possible Light, Prull, La Voie, & Healy, 2000; Reyna & Mills, 2007;
rather than probable AD diagnoses when such diseases are Yonelinas, 2002). Thus, this hypothesis about MCI and AD posits
present but judged not to be causing impairments (e.g., McK- continuity in the recollection deficits that underlie episodic mem-
hann et al., 2011). ory declines in these diseases and in healthy aging.
Diagnostic criteria for MCI are weaker versions of AD criteria An alternative scenario that we have proposed (Brainerd et al.,
(see Petersen, 2004). Of those just mentioned, the last two (de- 2009; Reyna & Brainerd, 2011) involves qualitative shifts in the
clines from earlier levels of functioning and exclusion of other processes that are responsible for declines during healthy aging
diseases) are the same, but the first is different. For MCI, the first versus MCI and AD. That hypothesis is predicated on the follow-
criterion requires impairment in episodic memory or one of the ing considerations. Before healthy adults reach age 70 and the MCI
other domains, with the former designated as amnestic (a-MCI) conversion rate accelerates, cumulative declines in traditional mea-
and other forms as nonamnestic (n-MCI).2 Thus, memory impair- sures of recollection are already quite large. Further declines that
ment is not necessary for MCI. Empirically, however, it accounts would be commensurate with the levels of impairment in a-MCI
for roughly two thirds of such diagnoses (Petersen et al., 2010), (the diagnostic criterion is at least a 1.5 SD difference between
which is why performance on clinical recall instruments is also the individuals with a-MCI and HC individuals on episodic memory
best neuropsychological marker of MCI diagnoses (e.g., Harel et tests; Petersen, 2004) could drive recollection to near-floor levels.
al., 2011). Further, a-MCI but not n-MCI is prodromal to AD; That might rule it out as a source of further memory decline in
there is a developmental progression from HC to a-MCI to AD transitions to AD—leaving nonrecollective processes, such as fa-
(see Brainerd et al., 2013). miliarity or reconstruction, as remaining possibilities.
The memory declines in AD and MCI diagnostic criteria are Comparisons of the two scenarios require measurement of rec-
purely descriptive, consisting simply of scores on memory instru- ollective and nonrecollective remembering in samples of well-
ments and self-reports of memory complaints. The obvious theo- characterized HC individuals, individuals with MCI, and individ-
retical questions are: Which underlying processes are responsible uals with AD. Such samples are essential because diagnostic error
for those declines? Are these processes different for MCI than for is a persistent source of discrepant findings about these conditions
AD? Is there some sequence of process deterioration during the (Brainerd, Reyna, Petersen, Smith, & Taub, 2011). Because MCI
progression from HC to MCI to AD? Such questions are central to is a recent diagnostic category, there has been only limited time for
any theoretical characterization of these diseases. However, they data sets that meet that criterion to accumulate. There is a recent
are also of great clinical interest because theoretical characteriza- data set, however, that supplies large samples of well-
tion illuminates disease mechanisms, and disease mechanisms characterized HC individuals, individuals with MCI, and individ-
must be understood to produce successful treatments (Brainerd et uals with AD: the Aging, Demographics, and Memory Study
al., 2009). To make progress on such questions, researchers have (ADAMS) of the National Institute on Aging’s Health and Retire-
used dual-process conceptions to characterize memory impair- ment Study (HRS; Health and Retirement Study, 2011). Another
ments. That is a logical approach, because the brain regions that major advantage of the ADAMS is that it is the only nationally
have been foci of dual-process research with healthy subjects (the representative sample of HC individuals, individuals with MCI,
perirhinal, parahippocampal, and entorhinal cortices and the hip- and individuals with AD extant (Langa et al., 2005), so that what
pocampus; e.g., Diana, Yonelinas, & Ranganath, 2007; Ranganath,
2010) are regions that exhibit pathology in postmortem studies of
2
individuals with MCI or AD (e.g., Braak & Braak, 1995; Nelson, Clinically, the n-MCI diagnosis is further subdivided into individuals
who exhibit impairment in a single nonmemory domain (sn-MCI) versus
Braak, & Markesbery, 2009). multiple nonmemory domains (md-MCI; see Panza et al., 2005). This
As discussed in a prior review (Brainerd et al., 2009), the other, less common, variety of MCI was not a focus of the present research,
predominant hypothesis about AD is a recollection-deficit notion: as it is not prodromal to AD.
46 BRAINERD ET AL.

is true for ADAMS diagnostic groups should be true in general for future AD and MCI was compared to the best genetic predictor of
older adults in the United States. We used that data set in Studies these conditions, the ε4 allele of the APOE genotype.
1 and 2. Another data set whose subject sample was not represen- Because Studies 1 and 2 revealed that dual-retrieval processes
tative was used in Study 3 to examine some important follow-up differentiated the ADAMS diagnostic groups and predicted future
questions that could not be studied with the ADAMS. disease, it was important to investigate whether those processes
could be successfully measured at the level of individual subjects.
That was the aim in Study 3, which used another large-scale study
Overview of the Research of late-life impairment (Alzheimer’s Disease Neuroimaging Initia-
In the first two studies, the dual-retrieval model was fit to tive; ADNI) that provides recall data that are more extensive than
ADAMS subjects’ recall on a standard clinical recall instrument those of the ADAMS. With these richer data, the model could be
(CERAD) in order to measure levels of recollection, reconstruc- fit to the recall of individual HC, MCI, and AD subjects, and the
tion, and familiarity judgment in nationally representative HC, questions that were examined in the first two studies could be
MCI, and AD groups. The aim in Study 1 was to pit different reexamined with individualized measurements of dual-retrieval
theoretical accounts against each other by pinpointing the process processes.
differences among the diagnostic groups during immediate and
delayed recall. That was done, first, by showing that the dual-
Study 1
retrieval model provides acceptable fits for all three groups and,
second, by comparing recollection, reconstruction, and familiarity The HRS involves a sample of over 30,000 subjects from over
judgment parameters (a) between the HC and MCI groups and (b) 70,000 households that encompass all geographical regions, racial
between the MCI and AD groups. groups, and ethnic groups in the United States. Subjects were
The next study focused on the problem of forecasting future sampled with a multistage clustered area probability model that
disease. That problem, whose scope can be illustrated by studies of ensured that the sample would accurately represent older adults
longitudinal progression to AD and CIND (e.g., Tabert et al., from all regions, racial groups, and ethnic groups. ADAMS sub-
2006), is that the neuropsychological tests that are used to diagnose jects are a nationally representative subsample of 856 individuals,
MCI and AD do not perform especially well at predicting the age 70 and older, from the larger HRS pool (for details, see
future emergence of these diseases. At a minimum, neuropsycho- Plassman et al., 2007, 2008). HRS subjects participate in biennial
logical batteries contain six types of tests (see, e.g., Langa et al. interviews that gather information on a variety of demographic,
2005): general cognitive ability tests, such as the Mini-Mental employment, wealth, caregiving, family structure, and health mea-
State Exam (MMSE; Folstein, Folstein, & McHugh, 1975) and the sures (Juster & Suzman, 1995). The ADAMS subsample was
Shipley Vocabulary Test (SVT; Zachary, 1986), and tests of the constructed to include five levels of cognitive ability, ranging from
five specific abilities that are stipulated in the DSM–IV dementia low-functioning to high-normal functioning, based on subjects’
criteria (episodic memory, executive function, language, motor performance during their most recent biennial interview (Langa et
function, and object identification). In a review of 73 studies of al., 2005).
neuropsychological predictors of future AD, Twamley, Ropacki, ADAMS subjects received extensive neuropsychological testing
and Bondi (2006) found that (a) predictive relations were weak, (b) and were diagnosed for the presence of neurocognitive impair-
only five of the six types of tests displayed significant predictive ment. Subjects participated in up to four separate waves (A, B, C,
relations in some studies (general cognitive ability, executive and D) of testing and diagnosis over 6 years. During Wave A,
function, language, memory, and motor function), and (c) none of which is the focus of Study 1, 856 subjects (mean age ⫽ 81.6,
the six types of tests displayed consistent predictive relations range ⫽ 70 –110) completed a 3- to 4-hr assessment. They (a)
across all or even most studies. Concerning (c), the percentages of received a battery of neuropsychological tests, (b) received med-
studies producing significant predictive relations were 38% for ical examinations to identify conditions that must be considered in
general cognitive ability, 50% for verbal memory, 28% for visual diagnosing neurocognitive impairment (e.g., cardiovascular dis-
memory, 44% for executive function, 33% for language, and 17% ease, diabetes), and (c) provided buccal tissue samples for geno-
for motor functioning. Given that predictive relations were weak typing. Concerning (a) and (b), these data were reviewed by a
and were only reliable about one third of the time, the need for diagnostic team, who classified subjects according to three levels
reliable predictors, especially ones that are theoretically motivated, of functioning (HC, CIND, and demented [D]). Concerning (c),
is clear. tissue samples were analyzed for the APOE genotype. The genetic
That was our aim in Study 2, in which we focused on dual- data of 14 subjects were not usable, leaving a final sample of 842
process prediction of future AD in individuals with MCI diagnoses subjects. The chronology of the ADAMS testing waves and the
and future MCI in HC individuals. With regard to AD, we deter- diagnostic composition of the subject samples of Study 1 and
mined whether the measurements of recollection, reconstruction, Study 2 are summarized in Table 2.
and familiarity judgment in individuals with MCI that had been In Study 1, we used immediate and delayed recall data of HC
made in Study 1 would predict who would transition to AD after subjects, CIND subjects with MCI diagnoses, and D subjects with
16 –18 months and after 4.5– 6 years. Concerning MCI, we deter- AD diagnoses. Those data were generated by one of the clinical
mined whether the measurements of recollection, reconstruction, recall instruments mentioned earlier, the CERAD. To pinpoint
and familiarity judgment in HC individuals that had been made in process-level differences among the three groups, we fit the dual-
Study 1 would predict who would transition to MCI after 4.5– 6 retrieval model to the data, estimated its parameters (see Table 1)
years. As a baseline, the ability of these processes to forecast on an age-adjusted basis, and compared parameter values among
 DUAL-RETRIEVAL MODELS 47

Table 2 simple free recall task, in which subjects study a list of 10 familiar
Diagnostic Composition of the Subject Samples in Study 1 and concrete nouns. (One list is arm, butter, cabin, engine, grass,
Study 2 for the Four Waves of ADAMS Testing letter, pole, queen, shore, ticket.) The list is studied three times,
and each study cycle is followed by an oral free recall test.
Study and testing wave Approximately 5 minutes after the third test, a delayed recall test
Study 1 Study 2 is administered, without further opportunities to study the list, as a
B C and D
forgetting measure. Other neuropsychological tests are interpo-
Diagnostic (16- to 18-month (4.5- to 6-year lated during the delay. ADAMS subjects responded to construc-
group A follow-up) follow-up) tional praxis tests, which measure the ability to draw two- and
HC 304 7 122 three-dimensional figures, during this interval. Responses se-
MCI 98 49 40 quences for recall Tests 1–3 were input to the dual-retrieval
AD1 119 15 20 learning model in Brainerd et al. (2012; Equations A1–A9) and
AD2 105 6 11 Gomes et al. (2013; Equations A1–A9), in order to fit it to the
Note. These values are a combination of incident and prevalent diagno- performance of the HC, MCI, AD1, and AD2 groups. As the fits
ses. ADAMS ⫽ Aging, Demographics, and Memory Study; HC ⫽ healthy were acceptable, the recollection, reconstruction, and familiarity
control; MCI ⫽ mild cognitive impairment; AD1 ⫽ probable Alzheimer’s judgment parameters were estimated for each group on an age-
dementia; AD2 ⫽ possible Alzheimer’s dementia.
adjusted basis, and parameter significant tests were computed to
determine which ones differentiated the groups. Next, the data
the groups.3 As the HC group was large and covered a 24-year age sequences for recall Tests 2, 3, and 4 (delay) were input to the
range (70 –94), we conducted follow-up analyses of how dual- forgetting model in the Appendix, in order to estimate forgetting of
retrieval processes varied throughout this age range by estimating recollection and reconstruction for each of the four groups on an
the parameters separately for different age groups. Here, the ob- age-adjusted basis. Parameter significance tests were computed to
jective was to determine whether, late in life, there are continuing determine which ones differentiated the groups.
age declines in recollection, reconstruction, or familiarity judg- Finally, the 304 HC subjects were split into six chronological
ment when there is no evidence of disease. age groups, in order to identify any process-level declines that
occur very late in life when there is no evidence of disease. Each
group’s Trial 1–3 data were input to the learning model to measure
Method
age variability in recollection, reconstruction, and familiarity judg-
Subjects. The Wave A ADAMS sample with genetic data ment. Each group’s Trial 2– 4 data were input to the forgetting
consists of 304 HC subjects (mean age ⫽ 78), 237 CIND subjects model to measure age declines in forgetting of recollection and
(mean age ⫽ 84.5), and 301 D subjects (mean age ⫽ 84.3). Within reconstruction.
the latter two classifications, the diagnostic team also assigned
subdiagnoses to CIND subjects and D subjects. There were 10 Results and Discussion
sub-CIND diagnoses, most of which contained very few subjects.
Before detailed results are presented, the qualitative patterns
There was a total of 17 sub-D diagnoses, most of which contained
were as follows. At the most general level, the results provided no
very few or no subjects.
support for the continuity hypothesis that memory declines over
In research with the ADAMS genetic data, Brainerd et al. (2011)
HC ¡ MCI ¡ AD transitions are entirely recollection driven. In
found that the CIND group contained 98 MCI subjects (mean
line with the hypothesis that loss of recollective ability among
age ⫽ 83.2) and that the D group contained 224 AD subjects
healthy older adults is too extensive for it to be the sole source of
(mean age ⫽ 84.5). They also found that although the ADAMS
memory declines in neurocognitive impairment, absolute levels of
MCI group is a mixture of a-MCI and n-MCI, the group’s average
recollective retrieval were low in this nationally representative
performance on episodic memory tests meets the clinical criterion
sample of HC subjects. However, model fits showed that recol-
for a-MCI (ⱖ 1.5 SDs below the performance of healthy age-
lective as well as nonrecollective retrieval were needed to account
mates). Last, Brainerd et al. reported that the AD subjects could be
for the data of all subject groups. During initial learning, there was
further subdivided into 119 subjects with probable AD (AD1)
only a slight decline in recollective retrieval for HC ¡ MCI
diagnoses (mean age ⫽ 86.1) and 105 subjects with possible AD
transitions. Instead, memory declines were dominated by deterio-
(AD2) diagnoses (mean age ⫽ 86.1).
ration in nonrecollective retrieval, especially the reconstruction
Procedure. Summary data for the Wave A neuropsychologi-
component. Both reconstruction and familiarity components de-
cal tests, genetic results, and psychiatric classifications are avail-
clined in HC ¡ MCI transitions, and there were further and much
able from HRS. Our focus was on the immediate and detailed data
larger declines in the reconstruction component in MCI ¡ AD
of the clinical recall instrument, the CERAD. The immediate
transitions. With the retention data, we found that reconstruction
CERAD test consists of three trials of study plus free recall on a
10-word list, and the delayed CERAD test consists of one addi-
tional free recall test, which is administered after a 5-min filled 3
Individuals with AD are usually older than individuals with MCI or HC
retention interval. individuals, and individuals with MCI are usual older than HC individuals;
The dual-retrieval model analyzes complete error–success se- hence, it is standard practice in neuropsychological research to use age-
adjusted data in comparing these groups. Failure to do so has been a source of
quences for individual list items (see Brainerd et al., 2009, Equa- both false positive and false negative results (e.g., Brainerd et al., 2011).
tion A1), and for the CERAD items, such sequences consist of Although we report results for age-adjusted data, we also analyzed the data
three immediate tests plus one delayed test. The CERAD is a without age adjustment and found that the qualitative patterns were the same.
48 BRAINERD ET AL.

was again the key process that declined over diagnostic transitions. following a recall success, or following either. Tutorials and soft-
There was no reconstructive forgetting in HC subjects; preserva- ware for all of the modeling analyses that are reported in this
tion of that form of retrieval was perfect. However, such forgetting article may be found at www.human.cornell.edu/hd/brainerd/
increased to nearly 50% in MCI subjects and to above 80% in AD research.cfm
subjects. There were also declines in recollective forgetting asso- Fits to the data of the HC, MCI, AD1, and AD2 groups showed
ciated with AD but not MCI. Thus, in contrast to the continuity that a single submodel fit the data of all groups. In this submodel,
hypothesis, MCI and AD diagnoses were dominated by deteriora- (a) recollective learning occurred following errors, and (b) there
tion in nonrecollective retrieval. Further, these changes were was a single reconstruction parameter for all three trials. Model
unique to neurocognitive impairment: When HC subjects were fitting involved two steps, necessity tests followed by sufficiency
stratified by age, levels of recollection, reconstruction, and famil- tests. Necessity tests evaluate whether the dual-retrieval model is
iarity remained constant from age 70 onward. not parsimonious because recall data can be fit by a simpler
Differences in dual-retrieval processes: Learning. Mean one-process model; specifically, a model with a reconstructive
probabilities of correct recall for the four recall tests appear by retrieval operation and a familiarity judgment process. The fit test
diagnostic group in Table 3. The percentage of total correct recall for this one-process model is provided in Brainerd et al. (2012;
declined reliably from HC (60%) to MCI (38.5%) to AD (AD1 ⫽ Equations A12–A15). When it is fit to the data of any group, it is
16% and AD2 ⫽ 20.2%), F(3, 622) ⫽ 414.66, p ⬍ .0001, partial fit with two degrees freedom. As there were four groups (HC,
␩2 ⫽ .67. Post hoc tests (Tukey honestly significant difference MCI, AD1, and AD2), there were eight degrees of freedom, so that
[HSD]) showed that MCI performance was worse than HC per- the .05 critical value for the G2 test of the null hypothesis that the
formance (mean difference ⫽ 6.40), that MCI performance was model fits the data was 15.51. The observed value of G2(8) was
better than either AD1 performance (mean difference ⫽ 6.74) or 64.52, and, thus, the data were not generated by a single recon-
AD2 performance (mean difference ⫽ 5.48), and that the two AD struction process. An important implication is that although esti-
groups did not differ reliably (ps ⬍ .0001). mates of the recollection parameters were low in absolute terms, as
To measure differences in recollection, reconstruction, and fa- expected, they were nevertheless reliable because the data could
miliarity judgment between diagnostic groups, we fit the dual- not be accommodated by reconstructive retrieval alone.
retrieval model to the age-adjusted Trial 1–3 CERAD data of each
When necessity tests yield poor fits, sufficiency tests ask
group. There are six slightly different versions of this model (see
whether recall data are more complex than the dual-retrieval model
Brainerd et al., 2012; Gomes et al., 2013), which form a 2 ⫻ 3
posits and, therefore, cannot be fit by a model with two retrieval
matrix of submodels that differ along two dimensions. One dimen-
processes. The appropriate fit test is provided in Brainerd et al.
sion refers to the set of reconstruction and familiarity judgment
(2012; Equation A11). For the study as a whole, the model was fit
parameters: A submodel may have either two of each (R1, R2, J1,
with four degrees of freedom (one per subject group), so that the
J2), or it may have a single reconstruction parameter and three
.05 critical value of the G2 test was 9.48. As the observed value
familiarity parameters (R, J1, J2, J3). The second dimension refers
(9.27) did not exceed the critical value, model fit was acceptable.
to whether a submodel allows items that are reconstructable but
The retrieval processes that differentiate HC, MCI, and AD
not recollectable to become recollectable following a recall error,
subjects can be seen in Table 4, where maximum likelihood
estimates of the recollection, reconstruction, and familiarity judg-
ment parameters are reported by diagnostic group. Although sep-
Table 3 arate parameter estimates for the AD1 and AD2 groups are re-
Mean Correct Recall by Trial in Study 1 and Study 3 ported, a preliminary analysis revealed that they did not differ
reliably between these diagnostic groups. Consequently, another
Trial
set of estimates is reported for the pooled AD1 and AD2 data, and
Diagnostic group 1 2 3 4 5 6 those values were used to test for differences in retrieval processes
Study 1 between MCI and AD subjects.
HC .43 .61 .73 .60 The procedure for identifying between-group differences in
MCI .25 .42 .48 .25 parameter values involves three steps (test statistics in Brainerd et
AD1 .10 .18 .20 .04
al., 2012, Equations A11–A11c): a likelihood ratio test of the
AD2 .14 .23 .24 .07
AD pooled .12 .20 .22 .05 omnibus null hypothesis that none of the parameters differed
reliably among the three groups (HC, MCI, AD), followed by
Study 3 likelihood ratio tests of the null hypothesis that none of the
HCNC .35 .52 .62 .70 .74 .33
parameters differed reliably between specific pairs of groups (HC
HCC .29 .43 .55 .60 .66 .34
a-MCINC .30 .40 .44 .51 .55 .26 vs. MCI and MCI vs. AD), followed by likelihood ratio tests of the
a-MCIC .25 .34 .39 .42 .44 .22 null hypothesis that specific parameters (say, R) did not differ
AD .23 .29 .33 .33 .35 .19 reliably between specific pairs of groups (say, HC vs. MCI). The
Note. HC ⫽ healthy control; MCI ⫽ mild cognitive impairment; AD1 ⫽ test of the omnibus null hypothesis was a G2 statistic with 12
probable Alzheimer’s dementia; AD2 ⫽ possible Alzheimer’s dementia; degrees of freedom and a .05 critical value of 21.03. This null
HCNC ⫽ healthy control not converted to amnestic; HCC ⫽ healthy control hypothesis was rejected, G2(12) ⫽ 3,221.66. The tests of the
converted to amnestic; a-MCINC ⫽ amnestic mild cognitive impairment
not converted to Alzheimer’s dementia; a-MCIC ⫽ amnestic mild cognitive
two-group null hypotheses were G2 statistics with 6 degrees and a
impairment converted to Alzheimer’s dementia; AD ⫽ Alzheimer’s de- .05 critical value of 12.59. As the observed values for the HC–MCI
mentia. test (342.35) and the MCI–AD test (953.68) were both more than
 DUAL-RETRIEVAL MODELS 49

Table 4 subjects (.47 vs. .36), though the parameterwise tests showed that
Estimates of Recollective Retrieval, Reconstructive Retrieval, only J3 was reliably smaller among AD subjects.
Familiarity Judgment, and Forgetting for ADAMS Subjects Differences in dual-retrieval processes: Forgetting. The
During Wave A fact that the CERAD includes a retention test 5 minutes after the
third learning trial allows two forgetting processes, recollective
Diagnostic group forgetting (FD) and reconstructive forgetting (FR), to be measured
Process/parameter HC MCI AD1 AD2 AD pooled with the forgetting version of the model (see Appendix). As
background, the percentage of forgetting (decline in recall from the
Recollection
D1 .10 .04 .03 .03 .03 third learning trial to the retention test) increases as we move from
D2 .04 .00 .02 .05 .03 HC (17%) to MCI (48%) to AD (AD1 ⫽ 74% and AD2 ⫽ 76%),
MD .07 .02 .02 .04 .03 F(3, 622) ⫽ 262.53, p ⬍ .0001, partial ␩2 ⫽ .56. Post hoc tests
Reconstruction
(Tukey HSD) revealed that MCI forgetting exceeded HC forget-
R .69 .54 .18 .28 .22
Familiarity judgment ting (mean difference in recall probability ⫽ .31), that AD1 and
J1 .53 .40 .40 .43 .42 AD2 forgetting both exceeded MCI forgetting (mean differences
J2 .65 .50 .44 .36 .39 in recall probabilities ⫽ .26 and .28), but that AD1 and AD2
J3 .70 .51 .34 .24 .28 forgetting did not differ reliably (ps ⬍ .0001).
MJ .63 .47 .40 .34 .36
Forgetting: Recollection To measure differences in recollective and reconstructive for-
FD .32 .59 .87 .82 .84 getting, we analyzed the Trial 2– 4 data, which provided estimates
Forgetting: Reconstruction of both types of forgetting, FD and FR. Those estimates appear at
FR .00 .45 .75 .65 .70 the bottom of Table 4. First, however, we fit the model to the data
Note. ADAMS ⫽ Aging, Demographics, and Memory Study; HC ⫽ of the HC, MCI, AD1, and AD2 groups. Because that analysis had
healthy control; MCI ⫽ mild cognitive impairment; AD1 ⫽ probable eight degrees of freedom, the critical value of the G2 statistic to
Alzheimer’s dementia; AD2 ⫽ possible Alzheimer’s dementia.
reject the null hypothesis of fit at the .05 level was 15.51. The
observed value, G2(8) ⫽ 14.22, was below the critical value.
Before considering differences among diagnostic groups, we
25 times the critical value, these null hypotheses, too, were re-
note a key validity result for the two forgetting parameters. Under
jected.
their theoretical definitions, reconstruction is more resistant to
At the level of retrieval processes, the HC–MCI and MCI–AD
forgetting than is recollection (Brainerd et al., 2012). If FD and FR
parameter comparisons identified specific processes that differed
are valid measures of those processes, the former should be larger
between diagnostic groups. The HC–MCI comparisons showed
than the latter, which has been found with young adults (Brainerd
that two processes differed between these groups, reconstruction
et al., 2012). Estimates of these parameters for the ADAMS reveal
and familiarity judgment. The R parameter was significantly larger
the same pattern. Across diagnostic groups, the mean value of FD
in HC subjects, as were all three J parameters (see Table 4), so that
was .58 and the mean value of FR was .38.
MCI memory decline was not due to loss of recollective ability.
Inspection of the parameter estimates reinforces the previous
Rather, it was harder for MCI subjects to learn how to reconstruct
items, and once they had done so, the reconstructions seemed less finding that declines in reconstructive retrieval are hallmarks of
familiar to them. Although neither of the recollection parameters neurocognitive impairment, because, on the retention test, group
(D1 and D2) differed reliably for the HC–MCI comparison, differences were dominated by reconstructive forgetting. The most
follow-up analyses showed that there was a reliable difference in dramatic change occurred in HC ¡ MCI transitions. Among HC
the proportion of items that could be recollected by Trial 3. Using subjects, there was no reconstructive forgetting, which is consis-
the parameter estimates for these groups, one can calculate the tent with the notion that the memory representations that it pro-
total proportion of items that could be recollected on any learning cesses are very stable in healthy individuals (e.g., semantic fea-
trial. By Trial 3, those values were .14 for HC and .04 for MCI, tures). In sharp contrast, the forgetting rate for reconstruction was
which was a reliable difference (p ⬍ .01 by a proportions test). nearly 50% in MCI subjects, and a parameterwise significance test
Thus, although the parameter comparisons failed to confirm that (see above) confirmed that this increase was highly reliable,
MCI memory decline is due to loss of recollective ability, there G2(4) ⫽ 16.13, p ⬍ .003. With respect to recollective retrieval, it
was some support for a recollective contribution at the level of can be seen in Table 4 that forgetting was substantial among HC
cumulative performance. subjects (FD ⫽ .32) and was higher still among MCI subjects
The MCI–AD parameter comparisons failed to provide any (FD ⫽ .59). However, the difference was not reliable. The per-
support for the continuity hypothesis: Neither of the recollection centage of items that MCI subjects learned to retrieve recollec-
parameters differed reliably between the MCI and AD groups, and tively (4% by Trial 3) was so low that there was little power to
follow-up analyses showed that the proportion of items that could detect reliable HC–MCI differences in FD.
be recollected by the last learning trial was the same. The process- For AD subjects, because forgetting rates did not differ between
level difference between these groups was simply that it was far the AD1 and AD2 groups, estimates of FD and FR are reported for
harder for AD subjects to learn how to reconstruct items. The value the pooled data as well as for the individual groups. First, it can be
of R in the MCI group was more than twice that in the AD group seen that the forgetting rate for reconstructive retrieval rose to 70%
(.54 vs. .22). Also, reconstructed items were somewhat less famil- in AD subjects. Parameterwise tests showed that the MCI ¡ AD
iar to AD subjects than MCI subjects. The mean value of the J increase in FR was highly reliable, G2(1) ⫽ 52.26, p ⬍ .0001.
parameters was smaller among AD subjects than among MCI Second, it can be seen that the forgetting rate for recollective
50 BRAINERD ET AL.

retrieval, which was .59 in MCI subjects, rose to .84 among AD cross-sectional, and, hence, they may be contaminated by selective
subjects. Again, however, the percentage of items that MCI and survival effects. It may be that subjects who survive to older and
AD subjects learned to retrieve recollectively was so small that older ages are progressively healthier, on average, than age-mates
there was little statistical power with which to detect group dif- who do not survive and that this is masking age declines that
ferences in FD, and this difference was not reliable. would be detected with longitudinal comparisons.
Developmental changes in healthy older adults. The find- Summary of process-level results. Application of the dual-
ings reported so far show that dual-retrieval processes— especially retrieval model to the data of HC, MCI, AD1, and AD2 subjects
reconstruction—are sensitive to neurocognitive impairment. What produced three general patterns. First, as subjects were learning to
about the other dimension of diagnostic separation, specificity? recall, declines in reconstruction (R) were the most salient markers
The findings might not be specific, and, instead, the same declines of HC ¡ MCI and MCI ¡ AD transitions, with the J parameters
that were detected in HC–MCI and MCI–AD comparisons may also declining by smaller amounts. Second and similarly, over the
also be occurring as healthy individuals continue to age. The retention interval, increases in reconstructive forgetting (FR) were
specificity question can be answered with the data of the HC hallmarks of HC ¡ MCI and MCI ¡ AD transitions over a
sample, which covers a broad age range (70 to 94). retention interval. Third, changes in reconstruction and, to a lesser
We split the HC sample into six adjacent age groups that extent, familiarity judgment were specific as well as sensitive to
contained reasonably equal numbers of subjects. Their mean ages neurocognitive impairment. As long as older adults remained
were 71.5 years, 73.5 years, 75.5 years, 79 years, 81 years, and 89 healthy, aging itself did not produce reliable changes in the mod-
years. We fit the learning model to each group’s Trial 1–3 data to el’s parameters.
estimate levels of recollection, reconstruction, and familiarity
judgment, and we fit the forgetting model to each group’s Trial
Study 2
2– 4 data to estimate levels of forgetting for recollection and
reconstruction. Third, we computed an omnibus likelihood ratio Now that process-level differences among HC, MCI, and AD
test of the null hypothesis that none of the learning model’s diagnoses have been identified, we turn to whether such differ-
parameters differed reliably among the six age groups, and we ences will forecast future disease. Concerns about possible cogni-
computed the same test for the forgetting model. Neither test tive impairment or dementia are among the most frequent reasons
produced a null hypothesis rejection. Thus, the earlier findings are that older adults present for medical treatment (Petersen, 2004).
specific to neurocognitive impairment because these age group Hence, current data that predict future impairment are of great
comparisons showed that dual-retrieval processes did not decline clinical interest and are a key test of the clinical applicability of
after age 70 as long as subjects remained healthy. basic research on memory declines (Brainerd et al., 2009). Con-
Estimated levels of recollection, reconstruction, familiarity sidering that AD is the most prevalent form of dementia and that
judgment, recollective forgetting, and reconstructive forgetting MCI is its prodrome, data that predict future MCI in HC individ-
appear by HC age group in Table 5. Consistent with the omnibus uals and future AD in MCI individuals are of special interest
tests, visual inspection reveals that these processes, as measured by (Summers & Saunders, 2012).
the CERAD at least, were remarkably constant from age 70 to the We shift attention to such questions in Study 2 by investigating
early 90s. The caveat should be added that these results are whether measurements of dual-retrieval processes will predict
longitudinal transitions to MCI and AD, over periods of 16 –18
months and 4.5– 6 years. This was done by combining dual-
retrieval measurements that were made during Wave A of the
Table 5 ADAMS with Wave B, C, and D diagnostic data. We also com-
Estimates of Recollective Retrieval, Reconstructive Retrieval, pared the predictive power of dual-retrieval processes to that of the
Familiarity Judgment, and Forgetting for ADAMS Healthy best genetic marker of MCI and AD, the ε4 allele of the APOE
Control Subjects During Wave A genotype.
Age group (years)
Recall that during Wave A, older adults had been diagnosed as
HC, CIND, or D on the basis of neuropsychological tests and
Process/parameter 71.5 73.5 75.5 79 81 89 medical examinations. Wave B focused specifically on transitions
Recollection from CIND to D over a 16 –18 month interval following Wave A,
D1 .08 .07 .22 .11 .14 .09 the aim being to retest and rediagnose all subjects who received
D2 .00 .00 .35 .04 .00 .10 CIND classifications. Ultimately, three quarters of them partici-
MD .04 .04 .29 .07 .07 .04 pated, with death being the modal reason for nonparticipation. Our
Reconstruction:
R .69 .69 .91 .66 .83 .63 interest lies with the subset of CIND subjects who were diagnosed
Familiarity judgment with MCI because they account for roughly two thirds of new AD
J1 .60 .57 .32 .50 .40 .54 diagnoses (Brainerd et al., 2013). The objective was to determine
J2 .67 .69 .40 .63 .60 .63 whether the dual-retrieval measurements that were made during
J3 .71 .74 .44 .71 .67 .67
MJ .66 .67 .39 .61 .54 .61
Wave A would predict who would be more likely to progress to
Forgetting: Recollection AD 16 –18 months later. As genetic data were available for these
FD .30 .21 .38 .33 .32 .36 subjects, the predictive power of dual-retrieval processes was
Forgetting: Reconstruction compared to that of the ε4 allele.
FR .00 .00 .00 .00 .00 .00 Waves C and D focused on transitions from CIND to D and
Note. ADAMS ⫽ Aging, Demographics, and Memory Study. from HC to CIND or D by retesting and rediagnosing subjects who
 DUAL-RETRIEVAL MODELS 51

had been previously classified as CIND or HC (see Plassman et al., ments to predict Wave B, C, and D diagnoses of HC and MCI
2011). Slightly more than half of the Wave A HC subjects partic- subjects, and we did likewise for the Wave A genetic data.
ipated in Wave C, with death being the modal reason for nonpar-
ticipation, and slightly less than half of the Wave A CIND subjects
Results and Discussion
participated in Wave C, with the modal reasons for nonparticipa-
tion being death or a dementia diagnosis during Wave B. (ADAMS There were three important qualitative patterns for Wave B.
subjects who received dementia diagnoses in a given wave did not First, mean recall accuracy did not differ reliably among the three
participate in subsequent waves.) We were concerned with the transition groups (MCI ¡ MCI/CIND, MCI ¡ AD, and MCI ¡
following questions about HC subjects: (a) Can dual-retrieval HC) on either the learning or the forgetting parts of the CERAD.
processes predict which of them are at risk of converting to MCI However, second, the three groups did differ reliably at the level of
4.5– 6 years later? (b) Can dual-retrieval processes predict which retrieval processes, and as in Study 1, reconstructive retrieval
of them are at risk of converting to AD 4.5– 6 years later? (c) How differentiated them. In other words, measurements of reconstruc-
does the predictive ability of dual-retrieval processes compare to tive retrieval that had been made during Wave A were able to
that of the ε4 allele? With respect to MCI subjects, although predict which MCI subjects would be more to likely to convert to
dual-process and genetic prediction had already been examined for AD over the next 16 –18 months. Third, in contrast, the best
the Wave B data, we reexamined these questions over the entire genetic marker of AD failed to predict conversion to AD among
6-year interval with the Wave C and D data. the same subjects.
There were four important qualitative patterns for Waves C and
D. First, as with the Wave B data, mean accuracy on the learning
Method
and forgetting portions of the CERAD did not differ among the
Subjects. With respect to the Wave B data, 77 of the 98 Wave groups of MCI subjects (MCI ¡ MCI/CIND, MCI ¡ HC, and
A MCI subjects were rediagnosed during Wave B. We focused on MCI ¡ AD), and it also did not differ reliably among the groups
those who either received a second cognitive impairment diagnosis of HC subjects (HC ¡ HC, HC ¡ MCI, and HC ¡ AD). Second,
(MCI or some other form of CIND) or were rediagnosed as AD as with the Wave B data, measures of reconstructive retrieval
(AD1 or AD2). There were 49 subjects who received a second during Wave A predicted which MCI subjects would convert to
CIND diagnosis, 35 of whom received a second MCI diagnosis AD over the next 6 years, and, now, recollective forgetting also
and 14 of whom were diagnosed with some other form of CIND. predicted such transitions. Third, reconstructive retrieval did not
The latter subjects were ones for whom some medical condition predict future transitions to impairment among HC subjects, but
that can cause impairment was present that had not been detected those transitions were predicted by another nonrecollective pro-
earlier. There were 21 subjects who received a Wave B AD cess, familiarity judgment, and also by recollective forgetting.
diagnosis, 15 of whom were diagnosed as AD1 and 6 of whom Hence, measurements of dual-retrieval processes were able to
were diagnosed as AD2. The mean age of subjects who progressed predict which HC subjects would be more likely to transition to
did not differ significantly from the mean age of subjects who did cognitive impairment or dementia 4.5– 6 years after those mea-
not progress. In addition, some subjects (N ⫽ 7) were rediagnosed surements were taken. Fourth, the best genetic marker of neuro-
as HC during Wave B. Although that number is too small for direct cognitive impairment did not perform as well. The frequency of
statistical comparisons, indirect statistical comparisons are possi- the ε4 allele failed to predict either HC ¡ AD or MCI ¡ AD
ble, as we show. transitions, although it did predict HC ¡ MCI transitions.
With respect to the Wave C and D data, the subjects of interest Predicting future disease: Learning. For the Trial 1–3 data,
were those who had either (a) a Wave A MCI diagnosis or (b) a the percentages of total correct recall for the MCI ¡ MCI/CIND
Wave A HC diagnosis. Concerning the first group, of the 77 MCI and MCI ¡ AD transition groups were virtually the same (39% vs.
subjects who were rediagnosed during Wave B, 36 were rediag- 37% for Wave B subjects; 39% and 43% for Wave C/D subjects)
nosed during Wave C and/or Wave D. Of these 36 subjects, 15 and did not differ reliably (by t tests). With respect to HC subjects,
received a final diagnosis of MCI in Wave C (if they were tested the percentages of total correct recall for the three HC transition
only in that wave) or Wave D (if they were tested in both waves), groups were 64% (HC ¡ HC), 57% (HC ¡ MCI), and 56% (HC
14 received a final diagnosis of AD1 or AD2, and 7 received a ¡ AD). An F test showed that these values did not differ reliably.
final HC diagnosis in Wave C or D. In short, recall data by themselves, before retrieval processes were
Concerning the second group, of the 304 subjects with a Wave measured, did not predict future transitions to MCI or AD. It
A HC diagnosis, 223 were retested and rediagnosed during Wave should be noted that the ADAMS contains two other tests of
C and/or Wave D. The data of 164 subjects were of interest. Of episodic memory, a word list recognition test and the Wechsler
these subjects, 122 received a final diagnosis of HC in Wave C (if Story Memory Test (Wechsler, 1997), and two tests of general
they were tested only in that wave) or Wave D (if they were tested cognitive ability that are administered in most neuropsychological
in both waves), 25 received a final diagnosis of MCI, and 17 batteries, the MMSE and the SVT. Performance on those tests also
received a final diagnosis of AD1 or AD2. failed to predict future transitions to MCI or AD. However, re-
Procedure. During Waves B, C, and D of ADAMS testing, trieval processes were reliable predictors.
the procedures (neuropsychological testing, medical examination, The relevant data appear in Tables 6 (Wave A MCI ¡ Wave B
and diagnosis by a consensus panel) were the same as during Wave groups), 7 (Wave A MCI ¡ Wave C/D groups), and 8 (Wave A
A. The diagnostic team that reviewed the data of each wave and HC ¡ Wave C/D groups). First, however, we fit the learning
assigned diagnoses was blind with respect to subjects’ earlier model to the data of the three Wave B groups (MCI ¡ HC, MCI
diagnoses and data. We used the Wave A dual-retrieval measure- ¡ MCI/CIND, and MCI ¡ AD) and the five Wave B/C groups
52 BRAINERD ET AL.

Table 6 between-group differences in the D or J parameters was reliable. In

Estimates of Recollective Retrieval, Reconstructive Retrieval, contrast, it appears that reconstructive retrieval predicted future
Familiarity Judgment, and Forgetting for ADAMS Wave B dementia, as well as transitions back to normal functioning in
Diagnoses of Wave A MCI Subjects Wave B, in the same manner that it differentiated HC, MCI, and
AD groups during Wave A: The ordering of the values of the R
Wave B diagnosis of Wave A MCI subjects parameter was (MCI ¡ HC) ⬎ (MCI ¡ MCI/CIND) ⬎ (MCI ¡
Process/parameter MCI/other CIND not AD AD1/AD2 HC AD) in Wave B and (MCI ¡ MCI/CIND) ⬎ (MCI ¡ AD) in
Wave C/D. Only the MCI ¡ MCI/CIND and MCI ¡ AD transi-
Recollection
D1 .04 .04 .06 .08
tion groups contained sufficient subjects for direct statistical com-
D2 .00 .00 .00 .18 parison, and likelihood ratio tests showed that the estimate of R
MD .02 .02 .03 .13 was larger in the MCI ¡ MCI/CIND group in both Wave B and
Reconstruction Wave C/D, G2(1) ⫽ 6.51, p ⬍ .01, and G2(1) ⫽ 6.03, p ⬍ .02.
R .58 .60 .42 .98
Although the small number of subjects in the Wave B MCI ¡
Familiarity judgment
J1 .40 .39 .43 .21 HC group precludes direct statistical comparison to the MCI ¡
J2 .47 .48 .52 .42 MCI/CIND group, an indirect comparison is possible. In particu-
J3 .50 .50 .54 .41 lar, a parameter-invariance test can be computed for the Wave B
MJ .46 .46 .49 .33 MCI ¡ MCI/CIND group in which the value of R is fixed at its
Forgetting: Recollection
FD .66 .61 .69 .44
estimated value for the MCI ¡ HC group, rather than letting R be
Forgetting: Reconstruction a free parameter (whose value is .58 for MCI ¡ MCI/CIND
FR .00 .00 .00 .01 subjects). The fit of the model is recalculated for the MCI ¡
Note. ADAMS ⫽ Aging, Demographics, and Memory Study; HC ⫽
MCI/CIND group under this constraint, which yields a G2 statistic
healthy control; MCI ⫽ mild cognitive impairment; AD1 ⫽ probable with two degrees of freedom that is then compared to the G2
Alzheimer’s dementia; AD2 ⫽ possible Alzheimer’s dementia; CIND ⫽ statistic obtained when the model was originally fit with one
cognitive impairment no-dementia; not AD ⫽ pooled MCI/CIND and HC degree of freedom (i.e., when R was free to vary). The difference
groups. between the two statistics is a G2(1) test, with a critical of value of
3.84 to reject the null hypothesis. G2(1) ⫽ 4.49, and, thus, the R
value that was observed for MCI ¡ MCI/CIND subjects was
(MCI ¡ MCI/CIND, MCI ¡ AD, HC¡HC, HC¡MCI, and reliably smaller than the value that was observed for MCI ¡ HC
HC¡AD). Thus, for the study as a whole, the model was fit with subjects. Thus, there was statistical support for the conclusion that
eight degrees of freedom, so that the goodness of fit test was G2(8), although total correct recall did not differ among the groups, the
with a critical value of 15.51 to reject the null hypothesis of fit at ability to retrieve items reconstructively declines steadily as one
the .05 level. As before, fit was acceptable, the observed value of moves from MCI subjects who later transition back to HC to
the G2(8) statistic, 9.85, being well below the critical value. subjects who remain impaired to subjects who progress to AD.
MCI transitions. That the retrieval processes that MCI sub-
jects used 16 –18 months to 6 years earlier predicted who would
Table 7
be rediagnosed as MCI/CIND versus AD can be seen in Tables 6
Estimates of Recollective Retrieval, Reconstructive Retrieval,
and 7, where maximum likelihood estimates of the recollection,
Familiarity Judgment, and Forgetting for ADAMS Wave C or D
reconstruction, and familiarity judgment parameters are reported
Diagnoses of Wave A MCI Subjects
for the Wave B transition groups and the Wave C/D transition
groups. Also, parameter estimates are reported for MCI ¡ not- Type of transition
AD, which is an amalgamation of the MCI ¡ MCI/CIND and
MCI ¡ MCI ¡ MCI ¡
MCI ¡ HC groups. The MCI ¡ not-AD group is actually of Process/parameter MCI/CIND not AD AD
greater clinical interest than the MCI ¡ MCI/CIND or MCI ¡ HC
group, because once an MCI diagnosis is assigned, the crucial Recollection
D1 .05 .07 .08
question is the likelihood of a later dementia diagnosis relative to D2 .02 .06 .00
all other possibilities that do not involve dementia. MD .04 .07 .04
As before, the procedure for identifying between-group differ- Reconstruction
ences in parameters consisted of (a) a likelihood ratio test of the R .50 .65 .48
null hypothesis that none of the parameters differed reliably among Familiarity judgment
J1 .45 .32 .40
the groups, and (b) likelihood ratio comparisons of parameter J2 .54 .45 .51
estimates for the MCI ¡ MCI/CIND and MCI ¡ AD groups only J3 .48 .47 .53
(the Wave B MCI ¡ HC group was too small for direct compar- MJ .52 .41 .48
isons). Concerning (a), this test produced a null hypothesis rejec- Forgetting: Recollection
FD .58 .52 .71
tion for both Wave B, G2(12) ⫽ 48.97, p ⬍ .0001, and Wave C/D, Forgetting: Reconstruction
G2(6) ⫽ 16.86, p ⬍ .01. Hence, some of the Wave A parameters FR .00 .00 .00
could predict future transitions in both sets of data. With regard to
Note. ADAMS ⫽ Aging, Demographics, and Memory Study; MCI ⫽
(b), a glance at the values in Tables 6 and 7 shows that recollective mild cognitive impairment; CIND ⫽ cognitive impairment no-dementia;
retrieval and familiarity judgment did not predict which MCI not AD ⫽ pooled MCI/CIND and HC groups; AD ⫽ Alzheimer’s demen-
subjects would transition to AD. Indeed, none of the G2 tests for tia.
 DUAL-RETRIEVAL MODELS 53

Finally, with respect to the clinically important MCI ¡ not-AD groups. The second pair of tests produced null hypothesis rejec-
group, we repeated the sequence of likelihood ratio tests for this tions for HC ¡ HC versus HC ¡ MCI and for HC ¡ HC versus
group versus the MCI ¡ AD group, in order to determine whether HC ¡ AD, both G2(6) tests ⬎ 12.59, establishing that these
there was support for a conclusion that seems apparent from transitions could be predicted by at least some of the parameters.
inspecting the groups’ parameter estimates: Reconstructive re- Glancing at Table 8, it appears that (a) the recollection and
trieval differentiates these transition groups, but recollective re- reconstruction parameters do not differentiate any of the groups,
trieval and familiarity judgment do not. There was support for that but (b) the three familiarity judgment parameters differentiate the
conclusion. For both data sets, the MCI ¡ AD versus MCI ¡ HC ¡ HC group from the HC ¡ AD group but not from the HC
not-AD parameterwise G2 test for R was reliable, but the corre- ¡ MCI group. Parameterwise likelihood tests confirmed this.
sponding tests for the D and J parameters were not. With respect When the J parameters were compared between the HC ¡ HC and
to the critical question of which MCI subjects will not progress to HC ¡ AD groups, J1, J2, and J3 were larger in the group that
dementia over the next 6 years, then, the answer was those who remained healthy, the values of the G2(1) tests being 7.87, 13.62,
were better at reconstructive retrieval. and 10.32, respectively (all ps ⬍ .01). The values of G2(1) for the
HC transitions. With the Wave C/D data, we investigated parameter tests that did not produce null hypothesis rejections
whether dual-retrieval processes also forecast future transitions to were all ⬍ 2.50, which is well below the critical value of 3.84.
impairment and dementia among subjects who are currently Predicting future disease: Forgetting. As in Study 1, we
healthy. The retrieval processes that, 4.5– 6 years earlier, predicted used the Trial 2– 4 data to estimate recollective forgetting (FD) and
which HC subjects would convert to MCI or AD can be seen in reconstructive forgetting (FR) for each group. The percentage of
Table 8, where parameter estimates are reported for each of the forgetting was virtually the same for the MCI ¡ MCI/CIND and
three transition groups and also for the pooled data of the HC ¡ MCI ¡ AD groups (13% vs. 10% for Wave B subjects; 28% and
MCI and HC ¡ AD groups. The procedure for identifying reliable 28% for Wave C/D subjects). The same was true for the for HC ¡
parameter differences among HC transition groups was the same HC, HC ¡ MCI, and HC ¡ AD groups in Waves C/D (11%,
as before; that is, an omnibus likelihood ratio test of the null 19%, and 16%). t and F tests showed that none of the between-
hypothesis that none of the parameters differed reliably among the group differences was reliable. Thus, raw forgetting, before re-
three groups, followed by omnibus likelihood ratio tests of the null
trieval processes were measured, did not predict future transitions
hypothesis that none of the parameters differed reliably between
to AD. Estimates of the two forgetting parameters are reported by
the HC ¡ HC and HC ¡ MCI groups and of the null hypothesis
transition group at the bottoms of Tables 6, 7, and 8.
that none of the parameters differed reliably between the HC ¡
For Wave B, the values in Table 6 reveal little between-group
HC and HC ¡ AD groups, followed by likelihood ratio tests of the
variability in recollective or reconstructive forgetting parameters.
null hypothesis that specific parameters (say, R) did not differ
Consistent with that suggestion, a likelihood ratio test of the null
reliably between specific pairs of groups. The first test produced a
hypothesis that neither forgetting parameter differed reliably
null hypothesis rejection, G2(12) ⫽ 43.02, p ⬍ .0001, establishing
among the three Wave B groups, which was a G2(6) statistic with
that at least some of the parameters differed reliably among the
a critical value of 12.59, did not produce a null hypothesis rejec-
tion.
The picture is different for Wave C/D, with recollective forget-
Table 8 ting predicting future disease in both MCI and HC subjects. An
Estimates of Recollective Retrieval, Reconstructive Retrieval, omnibus likelihood ratio test indicated that at least some of the
Familiarity Judgment, and Forgetting for ADAMS Wave C or D parameters differed reliably between the MCI ¡ MCI and MCI ¡
Diagnoses of Wave A Healthy Control Subjects AD groups, G2(5) ⫽ 36.63, p ⬍ .0001. For HC subjects, some
Type of transition parameters differed reliably among the three diagnostic groups,
G2(10) ⫽ 20.02, p ⬍ .03, some differed reliably between the
HC ¡ HC ¡ HC ¡ HC ¡
Process/parameter HC MCI/AD MCI AD HC ¡ HC and HC ¡ MCI groups, G2(5) ⫽ 15.95, p ⬍ .007, and
some differed reliably between the HC ¡ HC and HC ¡ AD
Recollection groups, G2(5) ⫽ 15.55, p ⬍ .009. Glancing at the values of the
D1 .16 .10 .09 .12
recollective forgetting parameter in Table 7, the rate of recollective
D2 .17 .14 0 .29
MD .17 .12 .05 .21 forgetting was higher for MCI ¡ AD subjects than for MCI ¡
Reconstruction MCI/CIND subjects, and a parameterwise test showed that the
R .63 .70 .71 .71 difference was reliable, G2(1) ⫽ 5.61, p ⬍ .02. Glancing at the
Familiarity judgment
values of the recollective forgetting parameter in Table 8, it also
J1 .58 .46 .51 .41
J2 .63 .52 .61 .39 seems to predict future transitions to MCI and AD among HC
J3 .70 .59 .68 .45 subjects: FD was larger in the HC ¡ MCI group than in the HC ¡
MJ .64 .52 .60 .42 HC group, and it was larger in the HC ¡ AD group than in the HC
Forgetting: Recollection
¡ HC group. Parameterwise tests confirmed that FD was larger in
FD .24 .43 .48 .47
Forgetting: Reconstruction the HC ¡ AD group than in the HC ¡ HC group, G2(1) ⫽ 19.03,
FR 0 0 0 0 p ⬍ .0001, and that it was larger in the HC ¡ MCI group than in
Note. ADAMS ⫽ Aging, Demographics, and Memory Study; HC ⫽
the HC ¡ HC group, G2(1) ⫽ 19.20, p ⬍ .0001. Qualitatively, HC
healthy control; MCI ⫽ mild cognitive impairment; AD ⫽ Alzheimer’s subjects who will progress to MCI or AD a few years later forget
dementia. about half the items that they learn how to retrieve recollectively,
54 BRAINERD ET AL.

whereas HC subjects who will remain HC forget about one quarter reliable individualized parameter estimates can be obtained, even
of those items. with fairly short lists, using a sliding window bootstrap with
Predicting future disease with APOE. If measurements of resampling procedure. The resulting estimates of D, R, and J are
dual-retrieval processes predict MCI ¡ AD, MCI ¡ HC, HC ¡ averages of the separate parameter values for each of these se-
MCI, and HC ¡ AD transitions, what about the ε4 allele? For quences.
Wave B, we computed the frequencies of carriers of this allele in With respect to the first of the three objectives, we sought
the MCI ¡ MCI/CIND, MCI ¡ not-AD, and MCI ¡ AD groups. clinical data in which the RAVLT had been administered to large
The frequencies were .31, .37, and .38, respectively. The MCI ¡ samples of HC individuals, individuals with MCI, and individuals
not-AD versus MCI ¡ AD frequencies and the MCI ¡ MCI/ with AD. Although establishing individualized model fits for
CIND versus MCI ¡ AD frequencies did not differ reliably, with healthy older adults would be significant progress by itself, the
␹2(1) ⬍ 1 in each instance. For Wave C/D, we computed frequen- ultimate question is whether the model fits the performance of
cies of ε4 carriers in the MCI ¡ MCI, MCI ¡ not-AD, and MCI individuals with impairment or dementia as well as healthy indi-
¡ AD groups, which were 43%, 44%, and 38%, respectively. viduals, which would allow direct comparisons of retrieval pro-
Neither the MCI ¡ MCI versus MCI ¡ AD difference nor the cesses among diagnostic groups. A data set that meets this spec-
MCI ¡ not-AD versus MCI ¡ AD difference was reliable, ification is the ADNI, in which the RAVLT, along with some of
␹2(1) ⬍ 1 in each instance. Also for Wave C/D, we computed the same ADAMS neuropsychological tests, was administered to
frequencies of ε4 carriers in the HC ¡ HC, HC ¡ MCI, and HC older adults. With respect to the second objective, a consistent
¡ AD groups, which were 20%, 50%, and 29%, respectively. The finding of the ADAMS HC–MCI and MCI–AD comparisons was
difference in ε4 frequencies was reliable for HC ¡ MCI conver- that recollective retrieval did not decline with increasing impair-
sion, ␹2(1) ⫽ 12.32, p ⬍ .005, but not for HC ¡ AD conversion, ment. Although that is a valid normative pattern, because the
␹2(1) ⫽ 1.46. subject sample on which it is based is nationally representative, it
Thus, the predictive story for dual-retrieval processes versus ε4 might not hold for some important subgroups of older adults. In
was simple. The former were successful predictors, but the latter particular, it might not hold for more highly educated subgroups,
mostly were not. Whereas the memory processes forecast all who exhibit higher levels of recollection than representative sam-
transitions, including MCI ¡ HC reversion, ε4 predicted only HC ples (Brainerd et al., 2009). Mean educational level is considerably
¡ MCI conversion. higher for ADNI subjects than for ADAMS subjects (see below).
With respect to the third objective, predicting future MCI and
AD, the ADNI, like the ADAMS, has a longitudinal component.
Study 3
Subjects who were initially diagnosed as HC or MCI were fol-
In Study 2, measurements of dual-retrieval processes identified lowed for 2 years and were rediagnosed at intervals of 6, 12, 18,
subgroups of HC subjects who were at increased risk of conversion and 24 months. At the end of 2 years, 44% of subjects who had
to AD and MCI and subgroups of MCI subjects who were at initially been diagnosed as MCI had converted to AD, and 8% of
increased risk of conversion to AD. Measurements of those pro- subjects who had initially been diagnosed as HC had converted to
cesses proved to be better predictors of future AD and future MCI MCI. Consequently, it was possible to determine whether individ-
than the best genetic marker of these conditions. Such findings ualized measurements of dual-retrieval processes, which were
suggest that dual-retrieval processes can be useful tools when it obtained at the start, were able to predict future HC ¡ MCI and
comes to identifying individuals who are at increased risk of MCI MCI ¡ AD conversion. ADNI subjects, like ADAMS subjects,
or AD, but predictions of that sort require parameter estimates for were genotyped, so it was also possible to compare the predictive
individuals. Although the dual-retrieval model is applicable to power of individualized measurements of dual-retrieval processes
individuals as well as groups (Brainerd et al., 2009), the CERAD to that of the ε4 allele. Because measurements of dual-retrieval
does not generate sufficient data to fit the model to the perfor- processes were individualized, they could be combined with the
mance of individuals; hence, only group-level analyses are possi- genetic data in logistic regressions to compute the levels of sen-
ble with the ADAMS. While that suffices to test theoretical hy- sitivity and specificity with which future disease was predicted.
potheses about AD and MCI and to study predictive power, it is
unsatisfactory because predicting impairment is a question about
Method
individuals.
To make progress on that question, we report a final study in ADNI design and subjects. Data used in the preparation of
which another standard clinical instrument, the RAVLT, was used this article were obtained from the ADNI database (adni.loni.u-
to (a) fit the model to the performance of individuals and obtain cla.edu). The ADNI was launched in 2003 by the National Institute
individualized estimates of D, R, and J, (b) test theoretical hypoth- on Aging, the National Institute of Biomedical Imaging and Bio-
eses about AD and MCI with individualized rather than group engineering, the Food and Drug Administration, private pharma-
parameter estimates, and (c) test the ability of individualized ceutical companies and nonprofit organizations, as a $60 million,
estimates to predict future transitions to MCI and AD. The 5-year public–private partnership. The primary goal of ADNI has
RAVLT generates considerably more data for individuals than been to test whether serial magnetic resonance imaging (MRI),
does the CERAD, for two reasons: The study list consists of 15 positron emission tomography (PET), other biological markers,
words rather than 10, and there are five study–test cycles rather and clinical and neuropsychological assessment can be combined
than three. As noted previously, three cycles are sufficient to to measure the progression of MCI and early AD. Determination of
produce identifiable estimates of all six parameters. With more sensitive and specific markers of very early AD progression is
than three, as we have shown elsewhere (Brainerd et al., 2012), intended to aid researchers and clinicians to develop new treat-
 DUAL-RETRIEVAL MODELS 55

ments and monitor their effectiveness, as well as lessen the time were required to have scores in the 2– 8 range on the delayed part
and cost of clinical trials. The principal investigator of this initia- of this test (the maximum score is 25) to be included in the MCI
tive is Michael W. Weiner, MD, VA Medical Center and Univer- sample. As a result, although ADAMS MCIs were a mixture of
sity of California–San Francisco. ADNI is the result of efforts of a-MCI and n-MCI, ADNI MCIs were exclusively a-MCI. This is
many co-investigators from a broad range of academic institutions important when it comes to testing theoretical hypotheses about
and private corporations, and subjects have been recruited from the retrieval processes that differentiate HC, MCI, and AD groups
over 50 sites across the United States and Canada. The initial goal and that predict future HC ¡ MCI and MCI ¡ AD transitions.
of ADNI was to recruit 800 subjects, but ADNI has been followed Those processes might be different when the MCI group is re-
by ADNI-GO and ADNI-2. To date these three protocols have stricted to subjects with significant memory impairment and when
recruited over 1,500 adults, ages 55 to 90, to participate in the the AD group is restricted to probable AD.
research, consisting of cognitively normal older individuals, peo- As mentioned, ADNI subjects were rediagnosed at regular in-
ple with early or late MCI, and people with early AD. The tervals over the next 2 years. Nearly half the a-MCI subjects
follow-up duration of each group is specified in the protocols for converted to AD, and the rest did not (designated as the a-MCIC
ADNI-1, ADNI-2, and ADNI-GO. Subjects originally recruited for and a-MCINC subgroups below). A small proportion (8%) of the
ADNI-1 and ADNI-GO had the option to be followed in ADNI-2. HC subjects converted to a-MCI, and the rest did not (designated
For up-to-date information, see www.adni-info.org as the HCC and HCNC subgroups below).
The ADNI resembles the ADAMS in that older adults were
administered a battery of neuropsychological tests, which con-
Results and Discussion
sisted of the RAVLT, a word list recognition test, the Wechsler
Story Memory Test, the MMSE, forward and backward digit span, Our principal concern, of course, was whether the model fit the
the Boston Naming Test, the Clocks Test, the Geriatric Depression recall data of individual subjects, and, crucially, whether fit was
Scale, and the Category Fluency Test. Except for the RAVLT, all acceptable for impaired and demented subjects as well as for
of these instruments were administered in the ADAMS, the main healthy ones. Assuming that fit could be established, we intended
difference between the two batteries being that the ADAMS in- to reexamine the question of which retrieval processes differenti-
cluded other instruments that were not in the ADNI. Similar to the ated those groups and the question of whether any of the retrieval
ADAMS, subjects received a diagnosis at the start of the ADNI, processes were able to predict future disease; this time with indi-
but the subject sample was then restricted to individuals who vidual rather than group parameter estimates.
received one of just three diagnoses: HC (N ⫽ 207), a-MCI (N ⫽ Individualized model fits and parameter estimates.
368), and probable AD (N ⫽ 173). Individualized data are inherently noisier than group data, and
The focal recall task, the RAVLT, differs from the CERAD models that work well with the latter often fail with the former.
in three ways. First, as mentioned, the list consists of 15 words When the dual-retrieval model is applied to the recall of an
rather than 10, and, second, learning consists of 5 study–test individual subject, the test has one degree of freedom, just as with
cycles rather than three. Third, the forgetting test occurs 30 group data. As there were RAVLT data for 748 subjects, the total
minutes after the last learning trial. As with the CERAD, the degrees of freedom for fitting the model to the complete data space
RAVLT lists consist of familiar concrete nouns, free recall tests were 748, and the critical value of the G2 statistic to reject the null
are oral, the second forgetting test occurs without further op- hypothesis of model fit at the .05 level was 812.76. The observed
portunities to study the original list, and the forgetting interval value was 700.88, so that the model delivered acceptable fit to
is filled with interpolated neuropsychological tests. The inter- individual recall protocols, for the study as whole. Equally impor-
polated tests included the Wechsler logical memory test, for- tant, fit was acceptable for each diagnostic group, considered
ward and backward digit span, category fluency, digit-symbol separately, allowing direct parameter comparisons among them.
substitution, Boston naming, and rating scales for depression There were 173 AD subjects, 368 a-MCI subjects, and 207 HC
and dementia. subjects, so that the critical .05 values of the G2 statistic were
There were four notable differences between the ADNI and the 204.69, 413.73, and 241.57, respectively. The observed values
ADAMS subjects. First, the ADAMS sample was nationally rep- were 182.92, 354.67, and 183.31, respectively.
resentative, but the ADNI sample was not. ADAMS subjects were Distributions of observed values of the G2 statistic for individual
obtained via representative sampling from a national pool of over subjects in each diagnostic group are shown in Figure 1. As nearly
60,000 older adults, whereas ADNI subjects were individuals who half of the a-MCI group ultimately progressed to AD, the distri-
responded to recruitment efforts at various clinical sites. Second, butions for the a-MCIC and a-MCINC subgroups are shown sepa-
ADNI subjects were more educated than ADAMS subjects. The rately. For individual subjects, the critical value of G2 to reject the
average ADAMS subject had not completed high school (mean null hypothesis of model fit is 3.84, which appears as a dashed line
education ⫽ 11.2 years), but the average ADNI subject had com- in each panel. It can be seen that the mean G2 values of these
pleted nearly 4 years of college (mean education ⫽ 15.6 years). distributions (AD ⫽ 1.06, a-MCIC ⫽ 0.99, a-MCINC ⫽ 0.95, and
Third, although the mean age of both samples was above 70, HC ⫽ 0.78) are all far below this cutoff, with virtually all of the
ADAMS subjects were roughly five years older than ADNI sub- distribution for each diagnostic group falling below it. It can also
jects (mean ages ⫽ 80 years 11 months vs. 75 years 5 months). be seen that each distribution has a pronounced positive skew.
Fourth, the ADNI MCI sample was restricted to the AD prodrome, Diagnostic differences in dual-retrieval processes: Learning
a-MCI, and the AD sample was restricted to probable AD. The and forgetting. Mean probabilities of correct recall for the six
Wechsler logical memory test was used to distinguish a-MCI recall tests appear by diagnostic group in Table 3. Naturally,
subjects from other MCI subjects during initial testing: Subjects average recall differed among the diagnostic groups. During the
56 BRAINERD ET AL.

Figure 1. Distributions of the individualized goodness-of-fit statistics for healthy control subjects (A), a-MCI
subjects who remain a-MCI (B), a-MCI subjects who would later convert to probable AD (C), and probable AD
subjects (D). The dotted line in each panel is the critical value for rejection of the null hypothesis that
dual-retrieval model fits the data. a-MCI ⫽ amnestic mild cognitive impairment; AD ⫽ Alzheimer’s dementia.

learning phase, mean recall (out of 15) was 8.71 (HC), 6.20 lective retrieval, (b) rates of forgetting were much higher for
(a-MCI), and 4.54 (AD), F(2, 724) ⫽ 219.89, MSE ⫽ 3.16, p ⬍ recollection than for reconstruction, (c) familiarity judgment did
.0001. During the forgetting phase, it was 6.58 (HC), 3.77 (a- not differentiate any of the diagnostic groups, and (d) estimated
MCI), and 2.30 (AD), F(2, 724) ⫽ 209.57, MSE ⫽ 3.61, p ⬍ values of the familiarity judgment parameter for HC and a-MCI
.0001. Mean values of individualized parameter estimates for the subjects were close to their values in the ADAMS data.
AD, a-MCIC, a-MCINC, and HC groups are reported in Table 9, With respect to differences between the estimates in Table 9
and separate estimates are reported for the HCC and HCNC sub- versus Table 4, there are three notable ones. First, concerning
groups. Visual comparisons reveal similarities and differences recollective retrieval, we thought that the ADNI data would in-
with the earlier comparisons for the ADAMS diagnostic groups crease the values of the recollection parameters and that this
(see Table 4), which were confirmed with significance tests. With process might now differentiate diagnostic groups. It did. The
respect to similarities, the most obvious one is that reconstructive mean values of D in Table 9 were larger than the corresponding
retrieval is again the retrieval process that consistently discrimi- values in Table 4, and it can be seen that MD declined as subjects
nates all diagnostic groups from each other: As we move from HC transitioned from HC to a-MCI and from a-MCI to AD, though the
to a-MCI to AD, its value declines from .48 to .38 to .28. Another latter decline was small relative to the former. Second, although
similarity is that recollective forgetting was considerably lower for reconstructive retrieval was again easier than recollective retrieval,
HC than for a-MCI or AD. Additional similarities with earlier the values of the reconstruction parameter were smaller for all
findings are that (a) reconstructive retrieval was easier than recol- diagnostic groups in this study— especially for the HC and a-MCI
 DUAL-RETRIEVAL MODELS 57

Table 9
Individualized Estimates of Recollective Retrieval, Reconstructive Retrieval, Familiarity
Judgment, and Forgetting Parameters for ADNI Subjects

Diagnosis
Process/parameter HCNC HCC HC a-MCINC a-MCIC a-MCI AD

Recollection
D1 .27 .20 .26 .19 .15 .17 .14
D2 .20 .16 .20 .11 .07 .09 .06
MD .24 .18 .23 .15 .11 .13 .10
Reconstruction
R .48 .34 .48 .40 .36 .38 .28
Familiarity judgment
J1 .72 .85 .73 .74 .74 .74 .81
J2 .51 .55 .51 .48 .46 .47 .45
J3 .43 .44 .43 .39 .30 .38 .30
MJ .56 .61 .56 .54 .50 .53 .52
Forgetting: Recollection
FD .16 .33 .17 .30 .32 .31 .39
Forgetting: Reconstruction
FR .01 .00 .01 .02 .03 .02 .04
Note. ADNI ⫽ Alzheimer’s Disease Neuroimaging Initiative; HC ⫽ healthy controls; HCNC ⫽ healthy
controls who did not convert to a-MCI; HCC ⫽ healthy controls who converted to a-MCI; a-MCI ⫽ amnestic
mild cognitive impairment; a-MCINC ⫽ a-MCI subjects who did not convert to AD; a-MCIC ⫽ a-MCI subjects
who converted to AD; AD ⫽ Alzheimer’s dementia.

groups. Item selection is a likely explanation. The RAVLT has two a-MCI than for AD, as in Study 1. The ANOVA for recollective
more learning trials than the CERAD, so that more items can be retrieval produced a main effect for diagnostic group, F(2, 742) ⫽
recollected by the end of learning. This means that the average 84.05, MSE ⫽ .01, p ⬍ .0001, no main effect for ε4, and no
difficulty of the remaining items, which must be reconstructed to interaction. Post hoc comparisons revealed (all ps ⬍ .001) that MD
be recalled, will be higher for the RAVLT than the CERAD. Third, was larger for HC than for a-MCI and larger for a-MCI than for
forgetting levels for both recollection and reconstruction were AD. The ANOVA for familiarity judgment produced no main
lower than before, and reconstructive forgetting was near floor in effects and no interaction. The ANOVA for recollective forgetting
all diagnostic groups. Again, a likely explanation lies with the produced a main effect for diagnostic group, F(2, 742) ⫽ 30.25,
additional learning trials of the RAVLT, which should make both MSE ⫽ .07, p ⬍ .0001, no main effect for ε4, and no interaction.
retrieval operations more resistant to forgetting because learning is Post hoc comparisons for the diagnostic group factor revealed (all
more thorough. ps ⬍ .001) that FD was larger for HC than for a-MCI and larger for
Turning to significance tests of these similarities and differ- a-MCI than for AD.
ences, because parameters were estimated for individuals, standard In summary, the process-level differences among the diagnostic
analysis of variance (ANOVA) procedures can be used rather than groups resembled those in Study 1 in three major respects: Recon-
likelihood ratio comparisons. To simplify the ANOVAs, we first structive retrieval separated all three diagnostic groups from each
computed a mean value of the two D parameters and a mean value other, recollective forgetting separated HC from a-MCI and AD,
of the three J parameters for each subject, which meant that there and familiarity judgment did not separate the diagnostic groups.
would be four parameters to submit to ANOVA, three learning There were two notable differences as well. First, recollective
parameters (MD, R, and MJ) and a forgetting parameter (FD). retrieval now separated all of the diagnostic groups, and, second,
(ANOVAs for the reconstructive forgetting parameter were not recollective forgetting now separated a-MCI from AD. In connec-
computed because values were near floor.) Using each parameter tion with the latter finding, remember that there were large differ-
as a dependent variable, we computed a 3 (diagnostic group: HC, ences between MCI and AD FD estimates for the ADAMS, but
a-MCI, AD) ⫻ 2 (ε4: noncarrier vs. carrier) ANOVA to determine they were not reliable owing to low levels of initial recollective
if it differed reliably as a function of diagnosis. The APOE factor learning among AD subjects. Thus, the only result that is substan-
was included because we planned to investigate the retrieval tially different than the ADAMS data is that recollective retrieval
processes’ ability to predict future disease (see below), and to also separated the diagnostic groups.
interpret those data, it is essential to know whether the processes Because retrieval processes were estimated for individuals, for-
interact with genotype. (In Study 2, retrieval processes and geno- mal statistical measures of the accuracy with which they separate
types were independent predictors.) the HC group from the a-MCI group and the a-MCI group from the
The ANOVA for reconstructive retrieval produced a main effect AD group can be computed. This is done via logistic regressions,
for diagnostic group, F(2, 742) ⫽ 32.63, MSE ⫽ .05, p ⬍ .0001, in which diagnostic groups are the dependent variables and indi-
no main effect for ε4, and no interaction. Post hoc comparisons vidualized estimates of the parameters that differentiate those
(Tukey HSD) for the diagnostic group effect revealed (all ps ⬍ groups are the predictor variables. This delivers measures of sen-
.0001) that R was larger for HC than for a-MCI and larger for sitivity and specificity, from which an overall accuracy estimate,
58 BRAINERD ET AL.

the mean of the two measures, can be calculated. We computed Table 10

two such analyses, one for HC–a-MCI discrimination and one for Mean Estimates of Recollective Retrieval, Reconstructive
a-MCI–AD discrimination. Estimates of the model’s learning and Retrieval, Familiarity Judgment, and Forgetting for ADNI
forgetting parameters were the predictor variables in both analyses. Healthy Control Subjects
To conduct the logistic regression for HC-a-MCI, we first de-
leted the data of the small group of subjects (N ⫽ 25) whose fit test Age group (years)
equaled or exceed the critical value, leaving a total of 550 subjects. Process/parameter 70.4 73.7 76.9 82.6
The logistic regression for those subjects produced a reliable fit
Recollection
statistic, ␹2(8) ⫽ 173.06, p ⬍ .0001. The overall accuracy with
D1 .28 .28 .25 .24
which dual-retrieval processes discriminated the two groups was D2 .23 .19 .20 .16
good (73.5%), with good sensitivity (88.8%; chance ⫽ 63.8%) and MD .26 .24 .23 .20
specificity (58.3%; chance ⫽ 32.6%). The frequency of ε4 differed Reconstruction
reliably among the three diagnostic groups (HC ⫽ 25.1%, R .48 .51 .47 .47
Familiarity judgment
a-MCI ⫽ 54.9%, AD ⫽ 64.7%), and we saw in the earlier J1 .72 .69 .72 .73
ANOVAs that the effects of ε4 carrier status were independent of J2 .51 .50 .51 .52
retrieval processes (i.e., there was never an APOE ⫻ Retrieval J3 .43 .44 .43 .44
Process interaction). Therefore, it was possible that sensitivity MJ .55 .54 .55 .56
Forgetting: Recollection
and/or specificity might improve if ε4 carrier status were added as
FD .16 .13 .16 .18
a predictor variable. They did not improve. When ε4 carrier status Forgetting: Reconstruction
was added, the new sensitivity and specificity values, 86.3% and FR 0 .01 .02 .01
59.8%, were virtually the same as without this predictor. Thus,
Note. ADNI ⫽ Alzheimer’s Disease Neuroimaging Initiative.
retrieval processes alone did a good job of differentiating the two
diagnostic groups, and the genetic data could be dispensed with
because they did not improve differentiation. hypothesis that dual-retrieval processes decline with age during
To conduct the logistic regression for a-MCI–AD, we first late adulthood, as long as subjects remain healthy. Hence, the
deleted the data of the small group of subjects (N ⫽ 26) whose fit declines that were observed for the HC versus a-MCI and a-MCI
test equaled or exceed the critical value, leaving a total of 515 versus AD are specific to disease.
subjects. The logistic regression for those subjects produced a Predicting HC ¡ a-MCI and a-MCI ¡ AD conversion with
reliable fit statistic, ␹2(8) ⫽ 98.94, p ⬍ .0001. The overall accu- dual-retrieval processes. Recall that nearly half of the a-MCI
racy of group differentiation was moderately good (65%). How- subjects converted to AD over the 2-year interval, while a small
ever, although the specificity of group differentiation was good proportion of HC subjects converted to a-MCI. Thus, the question
(89.7%; chance ⫽ 68%), sensitivity was not (40.2%; chance ⫽ of whether dual-retrieval processes predict future disease reduces
31.8%). As before, it was possible that sensitivity and/or specific- to whether any of them are able to distinguish a-MCIC subjects
ity might improve if ε4 carrier status were added as a predictor from a-MCINC subjects or to distinguish HCC subjects from HCNC
variable. However, when it was added, sensitivity and specificity subjects. To answer these questions, we computed a 5 (diagnostic
values, 89.5% and 39.6%, were virtually unchanged. In differen- group: HCC, HCNC, a-MCIC, a-MCINC, AD) ⫻ 2 (ε4: noncarrier
tiating the a-MCI group from the AD group, then, retrieval pro- vs. carrier) ⫻ 4 (parameter: MD, R, MJ, FD) ANOVA, which
cesses did a good job of avoiding false positives but an unsatis- produced a main effect for diagnostic group and a Diagnostic
factory job of avoiding false negatives. As with HC–a-MCI Group ⫻ Parameter interaction. We then analyzed the interaction
differentiation, the genetic data could be dispensed with because to determine whether any of the retrieval processes distinguished
they did not improve differentiation. MCIC from a-MCINC subjects or HCC from HCNC subjects. They
Developmental changes in HC subjects. In Study 1, we did. Recollective retrieval (MD) predicted future transitions from
found that although dual-retrieval processes differed among HC, a-MCI to AD, whereas reconstructive retrieval (R) and recollective
MCI, and AD groups, they did not differ among HC subjects of forgetting (FD) predicted future transitions from HC to a-MCI.
different ages. To determine whether this was also true for ADNI As with the ADAMS data, then, measurements of dual-retrieval
subjects, we split the HC group into four adjacent age levels with processes that were taken at the start were able to predict future
reasonably equal numbers of subjects per group: mean ages ⫽ 70.4 transitions to a-MCI and to AD over a period of 2 years. Recol-
years (range ⫽ 62–72), 73.7 years (range ⫽ 73–75), 76.9 years lective forgetting was more common among subjects who ulti-
(range ⫽ 76 –78), and 82.6 years (range ⫽ 79 – 89). We fit the model mately converted from HC to a-MCI, similar to the ADAMS, but
separately to the recall protocols of individual subjects in each age there was a further predictor, reconstructive retrieval. Note that
group. The critical value for rejection of the null hypothesis of fit was reconstructive retrieval was not reliable predictor in Study 2. With
3.84 for each subject, and the mean values of the fit statistic for the respect to AD conversion, in contrast to the ADAMS data for
age four groups were .90, .78, .62, and .82, respectively. a-MCI ¡ AD, where learning how to retrieve reconstructively was
The means of the individualized estimates of the D, R, J, FD, a reliable predictor, it was not so for ADNI subjects. However,
and FR parameters for each age group appear in Table 10. Visual recollective retrieval was: a-MCI subjects who converted to AD
inspection indicates that they do not vary perceptibly. That was were less able to retrieve recollectively at the start of the 2-year
confirmed with a 4 (age group) ⫻ 8 (parameter) ANOVA, which interval.
produced neither an age main effect nor an Age ⫻ Parameter Because dual-retrieval processes were estimated for individuals,
interaction. Once again, therefore, there was no support for the it is possible to compute formal statistical measures of the sensi-
 DUAL-RETRIEVAL MODELS 59

tivity and specificity with which they predict future disease. As General Discussion
before, this was done with logistic regressions, in which diagnostic
We began with two broad objectives. One was to extend current
groups supplied the dependent variables, and the predictor vari-
work on dual-process models of recall to theoretical hypotheses
ables were individualized estimates of the model parameters. Also
about neurocognitive impairment, a domain in which recall decline
as before, the small groups of subjects whose fit statistics equaled
is the most reliable neuropsychological marker of MCI and AD.
or exceeded the critical value did not figure in the logistic regres-
There, our focus was on comparing different dual-process concep-
sions.
tions of memory declines in MCI and AD. The other objective was
For HC ¡ a-MCI transitions, computation of a binary logistic to use dual-process measurements to predict future transitions to
regression in which the dependent variable was HCC versus HCNC MCI and AD and to compare their predictive power to that of the
and the predictor variables were estimated values of the learning ε4 allele. Thus, our research may be thought of as a proof of
and forgetting parameters produced a reliable fit test, ␹2(8) ⫽ concept—the concept being that mathematical models of recall are
21.04, p ⬍ .008. However, that analysis is not meaningful. Be- productive tools for pinpointing the retrieval processes that define
cause the number of HCC subjects (16) was so small, relative to the “memory decline” in MCI and AD and for predicting future
number of HCNC subjects (191), the level of chance prediction of emergence of these diseases. To conclude, we discuss what was
a future HCNC diagnosis is 92.3%. Hence, it is impossible to accomplished in each of these spheres.
achieve a statistically reliable level of specificity with HC subjects,
which means that the formal predictive power of dual-retrieval Dual-Process Conceptions of MCI and AD
processes cannot be evaluated with these data when it comes to Dual-retrieval models of recall were introduced in response to
future conversion to a-MCI. two uncertainties that are posed by recognition-based measure-
In contrast, a formal evaluation is possible with respect to ment. One, which is explicated in Malmberg’s (2008) review, is
prediction of future AD, because the sizes of the a-MCIC (N ⫽ that the standard methods of separating dual processes in recog-
158) and a-MCINC (N ⫽ 93) groups are not as extremely unbal- nition data (remember/know, ROC, process dissociation) may not
anced as those of the HCC and HCNC groups. When we computed do so. Many investigators (e.g., Heathcote et al., 2006) have
a binary logistic regression with a-MCIC versus a-MCINC as the shown, for one or another of these methods, that the resulting data
dependent variable and the model’s learning and forgetting param- can be handled by models that posit only a single familiarity
eters as predictor variables, the fit test was reliable, ␹2(8) ⫽ 47.81, process. The other uncertainty (Brainerd et al., 2009; Ghetti &
p ⬍ .0001. The overall accuracy with which dual-retrieval pro- Angelini, 2008) is that these methods require subjects to make
cesses predicted future AD was moderately good (66%), with complex metacognitive judgments, and, hence, they are too de-
sensitivity being reliably above chance (62%; chance ⫽ 45%) and manding for some populations—notably, older adults with CIND
specificity also being reliably above chance (69.9%; chance ⫽ or dementia. Dual-retrieval models of recall avoid both problems:
55%). As in prior logistic regressions, we added ε4 carrier status as Recall indexes two distinct forms of retrieval, and the simple tasks
a predictor variable to determine if it contributed further predictive over which these models are defined are within the capabilities of
variance. It did not. Sensitivity (60.1%) and specificity (68.9%) subjects with impairment or dementia.
were virtually the same as before, and, hence, the genetic data These models have been applied in recent developmental studies
could once again be dispensed with. (Brainerd et al., 2009, 2012) and experiments with young adult
Summary. Overall, application of the dual-retrieval model to samples (Brainerd & Reyna, 2010; Gomes et al., 2013). Levels of
the ADNI data set produced four patterns that are worthy of note. fit have been good, and model parameters have behaved in accor-
By far the most important one from the perspective of theory- dance with theoretical expectations (e.g., D estimates are smaller
driven research on neurocognitive impairment is that individual- than R or J estimates, forgetting rates are steeper for D than for R
ized model fits were good. The model fit individual RAVLT or J, D responds to verbatim cuing manipulations while R responds
to semantic cuing manipulations). The present studies yielded a
protocols as well for individuals with a-MCI and individuals with
similar picture for healthy and impaired older adults. At both the
AD as it did for HC individuals (see Figure 1). Second, as in Study
group and individual levels, fit statistics did not exceed critical
1, dual-retrieval processes differentiated the diagnostic groups.
values for model rejection and were usually well below critical
Reconstructive retrieval again declined steadily as one moved
values. Also, parameter behavior was theoretically coherent inas-
from HC to a-MCI to AD. In these more educated subjects, who
much as the patterns among D, R, J, FD, and FR estimates that have
exhibited higher levels of recollective retrieval than those in the been observed in young adults were observed in MCI and AD
ADAMS sample, recollection declined and recollective forgetting subjects, as well as in healthy older adults. Study 3 provided data
increased, from HC to a-MCI to AD. Sensitivity–specificity anal- on these points that went well beyond prior research, because prior
yses showed that by themselves, dual-retrieval processes produced research has not involved individualized fits or individualized
good HC–a-MCI differentiation, though sensitivity was not reli- parameter estimates.
able for a-MCI–AD differentiation. Third, dual-retrieval processes Turning to theoretical hypotheses about memory declines,
again predicted future disease, with reconstruction and recollective across the three studies, several parametric patterns that bear on
forgetting predicting conversion to a-MCI and recollection pre- those hypotheses were reported, and for convenience of reference,
dicting conversion to AD. Dual-retrieval processes’ levels of sen- all of them have been summarized in qualitative language in Table
sitivity and specificity in predicting future AD were both reliable. 11. Hypotheses about memory declines in neurocognitive impair-
The genetic data contributed no predictive variance beyond what ment are motivated by the fact that the brain regions of main
was contributed by dual-retrieval processes alone. interest in dual-process research with healthy subjects are also
60 BRAINERD ET AL.

Table 11
Diagnostic Differentiation and Prediction of Future Disease by Dual-Retrieval Processes

Diagnostic differentiation Prediction of future disease

Process HC vs. a-MCI a-MCI vs. AD HC ¡ a-MCI HC ¡ AD a-MCI ¡ HCc a-MCI ¡ not ADc a-MCI ¡ AD

Studies 1 and 2 (ADAMS)

Retrieval
Direct access Yesa No No No No No No
Reconstruction Yes Yes No No Yes Yes Yes
Familiarity judgment Yes Yesb No Yes No No No
Forgetting
Recollective No No Yesd Yesd No No Yesd
Reconstructive Yes Yes No No No No No

Study 3 (ADNI)
Retrieval
Direct access Yes Yes No NA NA NA Yes
Reconstruction Yes Yes Yes NA NA NA No
Familiarity judgment No No No NA NA NA No
Forgetting
Recollective Yes Yes Yes NA NA NA No
Reconstructive No No No NA NA NA No
Note. HC ⫽ healthy controls; a-MCI ⫽ amnestic mild cognitive impairment; AD ⫽ Alzheimer’s dementia; NA ⫽ not analyzed; ADAMS ⫽ Aging,
Demographics, and Memory Study; ADNI ⫽ Alzheimer’s Disease Neuroimaging Initiative.
a
The only reliable difference was in the proportion of items recalled recollectively by the last trial (Trial 3). b Based on reliable differences in the J3
parameter only. c Based on indirect statistical comparisons necessitated by small sample sizes. d Predicts only 4.5- to 6-year transitions.

regions that exhibit pathology in postmortem studies of individuals larger amount in MCI versus AD, and the familiarity judgment
with MCI and individuals with AD (e.g., Nelson et al. 2009). We component declined by a much smaller amount. On the forgetting
saw that the dominant continuity view extends a well-established side, reconstruction was again the dominant discriminator of the
trend in healthy aging, positing that conversion to MCI and AD are three diagnostic groups. First, there was a qualitative difference
characterized, respectively, by further declines in and almost com- between healthy and impaired subjects in the sense that there was
plete loss of recollective capability, coupled with sparing of non- no reconstructive forgetting among HC subjects. There was
recollective processes (e.g., Bugaiska et al., 2011; Serra et al., marked reconstructive forgetting in impaired subjects, however,
2010). The alternative discontinuity view posits that the shift from with the level being higher for AD than MCI (80% vs. 50%). All
healthy aging to neurocognitive impairment is accompanied by a of these results are more consistent with the discontinuity view of
shift from recollective to nonrecollective declines (e.g., Brainerd et memory declines than the continuity view. Another noteworthy
al., 2009). Here, a key consideration is that recollective declines in datum is that the process-level changes that were hallmarks of
healthy subjects are so substantial by age 70 that recollection may MCI and AD were specific to disease. When we divided the HC
not be a realistic source of the severe declines in episodic memory sample into adjacent age groups, recollection, reconstruction, and
that are specified in diagnostic criteria for MCI and AD. Another familiarity judgment were invariant between age 70 and the early
consideration is that the brain atrophy that is associated with these 90s.
conditions, particularly with AD, has spread beyond the regions Because the subject sample was nationally representative, these
that are associated with recollection. Nevertheless, the hypothesis results provide a normative picture of process differences among
that nonrecollective declines become prominent in conversion to diagnostic groups for the average older adult. In Study 3, we contin-
MCI and AD is surprising from the perspective of the traditional ued to examine process differences with the data of the ADNI, which
continuity hypothesis. provides another large sample of HC individuals, individuals with
When the subjects were nationally representative samples of HC MCI, and individuals with AD. This sample is not nationally rep-
individuals, individuals with MCI, and individuals with AD (Stud- resentative, and the MCI and AD groups are restricted to a-MCI
ies 1 and 2), none of the dual-process comparisons among the and probable AD individuals. Unlike the ADAMS, then, the ADNI
groups could be said to favor a continuity view. We measured both does not provide a valid normative picture. However, it has a key
learning and forgetting of recollective and nonrecollective re- advantage over the ADAMS—namely, that with the ADNI data,
trieval. On the learning side, there was little support for the notion one can address the question of whether the model performs well
that recollective retrieval is connected to neurocognitive impair- with individual recall protocols. The ADNI used a recall instru-
ment. (The only evidence of that was a small difference in the total ment (the RAVLT) that generates sufficient data for individualized
amount of recollective retrieval by the last learning trial for HC fits and individualized parameter estimates. We found that the
versus MCI but not for MCI versus AD.) Instead, both the HC– individualized fits were good and that they were as good for
MCI and MCI–AD comparisons were dominated by declines in subjects with impairment or dementia as they were for healthy
nonrecollective retrieval, especially reconstruction. Both the re- ones (see Figure 1). Further, the individualized parameter esti-
construction and familiarity judgment components declined in HC mates differed reliably among diagnostic groups. Another useful
versus MCI. The reconstruction component declined by a much feature of the ADNI is that one can evaluate the extent to which the
 DUAL-RETRIEVAL MODELS 61

components of the ADAMS normative picture of process differ- not predict conversion to AD among individuals with CIND diag-
ences among diagnostic groups hold for different subpopulations noses (Brainerd et al., 2013).
of older adults. Explicitly, although declines in recollective re- With respect to conversion to MCI, the predictive picture is
trieval are not markers of MCI and AD in nationally representative weaker still and is complicated by the fact that following MCI
samples, they might be in more highly educated samples, in which diagnosis, more than 20% of individuals usually transition back to
baseline levels of recollection are higher. HC (Fisher et al., 2011). On the neuropsychological side, the fact
In that connection, we found that most process differences were that MCI is a relatively new diagnostic category means that a
the same as in the ADAMS, but some were different. With respect substantial literature on its predictors has not yet had time to
to similarities, the process difference that dominated group com- accumulate (Summers & Saunders, 2012). On the genetic side,
parisons in the ADAMS was also present in the ADNI: Recon- there is no established marker of MCI, and there is an ongoing
structive retrieval declined steadily from HC to MCI to AD. Also, controversy as to whether ε4 is elevated in MCI as well as AD
familiarity judgment, which displayed small diagnostic group dif- (Brainerd et al., 2011; Small, Rosnick, Fratiglioni, & Backman,
ferences in the ADAMS, was completely spared in the ADNI. 2004).
Thus, the ADAMS findings for reconstructive retrieval are candi- Against this backdrop, we examined the ability of dual-retrieval
dates for universal aspects of memory decline in MCI and AD. As processes to forecast future MCI and future AD in ADAMS
expected, the main difference between the ADNI and ADAMS subjects in Study 2 and to forecast future a-MCI and probable AD
data lay in the findings for recollection. During learning, there was in ADNI subjects in Study 3. With the ADAMS, a finding of
a substantial HC–MCI difference in the mean value of the D general significance is that recollective retrieval, which failed to
parameters (.23 vs. .13) and a small but reliable MCI–AD differ- differentiate the diagnostic groups during Wave A, also failed to
ence. During forgetting, the differences among diagnostic groups predict future AD among MCI subjects and failed to predict either
were entirely due to differences in recollective forgetting, whereas future AD or future MCI among HC subjects. We studied predic-
reconstructive forgetting had contributed to differences among tion of MCI ¡ AD conversion over a 16 –18 month interval and
diagnostic groups in the ADAMS. prediction of MCI ¡ AD, HC ¡ MCI, and HC ¡ AD conversion
With ADNI data, the availability of individualized parameter over a 4.5– 6 year interval. With respect to MCI ¡ AD, one of the
estimates allowed us to conduct formal analyses of the ability of processes that differentiated the MCI group from the HC and AD
dual-retrieval processes to differentiate individuals with different groups during Wave A was a reliable predictor of conversion after
diagnoses by computing sensitivity and specificity statistics. Sta- 16 –18 months and after 4.5– 6 years; namely, reconstructive re-
tistically speaking, the model did a good job of differentiating HC trieval. Values of the R parameter were smaller for MCI subjects
individuals from a-MCI individuals, because sensitivity and spec- who would convert to AD than for those who would not. Further,
ificity were both well above chance, but not of differentiating the predictive power of reconstructive retrieval was better than the
a-MCI individuals from individuals with AD, because sensitivity best genetic marker of AD, because the statistical association
was not above chance (although specificity was excellent). Adding between MCI subjects’ ε4 carrier status and their tendency to
genetic data as a further predictor variable failed to increase diagnos- progress to AD was not reliable.
tic differentiation, relative to dual-retrieval processes alone. With respect to conversions over a 4.5- to 6-year interval,
reconstructive retrieval was again a reliable predictor of which
Dual-Process Prediction of Longitudinal Transitions to MCI subjects would convert to dementia over that interval. Rec-
ollective forgetting was also a reliable predictor. With respect to
MCI and AD
HC subjects, the familiarity judgment parameters predicted which
There is a need for reliable, theoretically motivated predictors of subjects would convert to dementia over the 4.5- to 6-year interval,
future conversion to impairment among healthy older adults and of and the recollective forgetting parameter predicted which subjects
future conversion to dementia among older adults with CIND would convert to MCI or dementia. Once again, the dual-retrieval
diagnoses. With respect to future dementia, some neuropsycholog- model did a better job of forecasting longitudinal transitions than
ical tests have been found to be reliable predictors, but predictive ε4 did. Whereas the model’s parameters predicted all of the tran-
relations are weak and variable (for a review, see Tabert et al., sition types, ε4 carrier status did not predict HC ¡ AD transitions
2006). On the latter point, tests of verbal memory are the only ones or MCI ¡ AD transitions. It did predict HC ¡ MCI transitions,
that have proved to be reliable predictors in even half of extant however. For this transition group, additional analyses showed that
studies. With respect to genetic predictors of conversion to demen- ε4 carrier status was unrelated to model parameters, which meant
tia, early-onset AD is associated with defects on chromosome 21, that the two were independent predictors of conversion to MCI.
particularly mutation of the amyloid precursor protein gene, and on Turning to Study 3, once again dual-retrieval processes were
chromosomes 1 and 14, which are due to mutations in the reliable predictors of future conversion, both to a-MCI and to
presenilin-2 and presenilin-1 genes (Kawas & Katzman, 1999). probable AD, and prediction was not improved by the addition of
Combined, however, all three mutations account for only 2% of genetic data. Although dual-retrieval processes were reliable pre-
early-onset AD diagnoses (Pericak-Vance et al., 2000), and in any dictors of both HC ¡ a-MCI and a-MCI ¡ probable AD, the
event, the great preponderance of AD diagnoses are late-onset proportion of subjects who made the former transition over a
(after age 65). There, the ε4 allele fares better, with roughly 50% 2-year period was far too small to conduct a formal sensitivity–
of individuals with probable AD and roughly 30% of individuals specificity analysis. However, the proportion of a-MCI subjects
with possible AD being carriers as compared to roughly 20% of who converted to probable AD was adequate for such an analysis.
healthy age-mates, in nationally representative samples (Brainerd Both sensitivity and specificity were above chance, and overall,
et al., 2011). However, recent research suggests that this allele may the dual-retrieval model was a moderately good predictor of con-
62 BRAINERD ET AL.

version from a-MCI to probable AD. As in the sensitivity– ADNI, ε4 was associated with conversion to AD, but it did not
specificity analysis of diagnostic group differentiation, the genetic improve the sensitivity or specificity of prediction, relative to
data added nothing to dual-retrieval processes’ ability to predict dual-retrieval processes alone.
future AD: When ε4 carrier status carrier status was added as a The final item is concerned with healthy rather than impaired
further predictor variable, there was no improvement in either older adults, though it illustrates the role of impairment in late-life
sensitivity or specificity. cognitive declines. In the nationally representative ADAMS sam-
Summing up the predictive picture for MCI and a-MCI, the ple, when the model was fit to the memory performance of
process that did the best job of differentiating the MCI group from adjacent HC age groups, dual-retrieval processes remained stable
the HC group during Wave A of the ADAMS, reconstructive between age 70 and the early 90s. Likewise, in the more highly
retrieval, also predicted future a-MCI ¡ HC and a-MCI ¡ not AD educated ADNI sample, fitting the model to the memory perfor-
transitions during Waves B, C, and D, and in the ADNI, recon- mance of adjacent HC age groups showed that dual-retrieval
structive retrieval even predicted the small (N ⫽ 16; 8%) group of processes did not vary with age. Thus, although there are substan-
HC ¡ a-MCI transitions. Summarizing the predictive picture for tial declines in episodic memory among healthy individuals prior
AD and probable AD, reconstructive retrieval also predicted MCI to age 70, subsequent declines seem to be associated with disease
¡ AD transitions during Waves B, C, and D of the ADAMS, but rather than aging.
recollective retrieval predicted a-MCI ¡ probable AD transitions
in the ADNI. References
American Psychiatric Association. (1994). Diagnostic and statistical man-
Concluding Comments ual of mental disorders (4th ed.). Washington, DC: Author.
Anderson, N. D., Jennings, J. M., Cabeza, R., Ebert, P. L., Grady, C. L., &
A large number of findings about dual-retrieval processes in Graham, S. J. (2008). Recollection- and familiarity-based memory in
MCI and AD have been reported, and to avoid losing the forest for healthy aging and amnestic mild cognitive impairment. Neuropsychol-
the trees, we close by highlighting five items of general signifi- ogy, 22, 177–187 doi:10.1037/0894-4105.22.2.177
cance. The most important one, surely, is that mathematical mod- Baudic, S., Dalla Barba, G., Thibaudet, M. C., Smagghe, A., Remy, P., &
els of memory can extract reliable measurements of underlying Traykov, L. (2006). Executive function deficits in early Alzheimer’s
retrieval processes from simple instruments that are administered disease and their relations with episodic memory. Archives of Clinical
every day in clinics worldwide. It is difficult to overestimate the Neuropsychology, 21, 15–21. doi:10.1016/j.acn.2005.07.002
Billingsley, R. L., Smith, M. L., & McAndrews, M. P. (2002). Develop-
importance of the fact that massive clinical databases can now be
mental patterns in priming and familiarity in explicit recollection. Jour-
analyzed to pinpoint process loci of diseases. Indeed, such analy- nal of Experimental Child Psychology, 82, 251–277. doi:10.1016/
ses could easily be incorporated as routine features of existing S0022-0965(02)00007-3
neuropsychological batteries. Braak, H., & Braak, E. (1995). Staging of Alzheimer’s disease-related
The second item is the theoretical importance of the overall neurofibrillary changes. Neurobiology of Aging, 16, 271–284. doi:
picture that emerged for reconstructive retrieval. Those results 10.1016/0197-4580(95)00021-6
provide broad support for the hypothesis that the shift from healthy Brainerd, C. J., Aydin, C., & Reyna, V. F. (2012). Development of
aging to neurocognitive impairment is accompanied by a shift to dual-retrieval processes: Learning, forgetting, and reminiscence. Journal
of Memory and Language, 66, 763–788. doi:10.1016/j.jml.2011.12.002
declines in nonrecollective processes, rather than a continuation of
Brainerd, C. J., & Reyna, V. F. (2010). Recollective and nonrecollective
the sparing that characterizes healthy aging. Declines in recon- recall. Journal of Memory and Language, 63, 425– 445. doi:10.1016/j
structive retrieval differentiated all diagnostic groups in both the .jml.2010.05.002
ADAMS and the ADNI, whereas declines in recollective retrieval Brainerd, C. J., Reyna, V. F., & Howe, M. L. (2009). Trichotomous
usually did not. Declines in reconstructive retrieval also predicted processes in early memory development, aging, and neurocognitive
future MCI and future AD in the ADAMS and future a-MCI in the impairment: A unified theory. Psychological Review, 116, 783– 832.
ADNI. Indeed, declines in reconstructive retrieval, always on the doi:10.1037/a0016963
learning side and also on the forgetting in the ADAMS, could be Brainerd, C. J., Reyna, V. F., Petersen, R. C., Smith, G. E., Kenney, A. E.,
Gross, C. J., . . . Fisher, G. G. (2013). The apolipoprotein E genotype
fairly said to be the prime memory marker of neurocognitive
predicts longitudinal transitions to mild cognitive impairment but not to
impairment in these studies.
Alzheimer’s dementia: Findings from a nationally representative study.
The third item of general significance, considering the mixed Neuropsychology, 27, 86 –94. doi:10.1037/a0030855
and inconsistent literature on neuropsychological predictors of Brainerd, C. J., Reyna, V. F., Petersen, R. C., Smith, G. E., & Taub, E. S.
MCI and AD, is that dual-retrieval processes so consistently pre- (2011). Is the apolipoprotein E genotype a biomarker for mild cognitive
dicted future emergence of these diseases. Moreover, when the impairment? Findings from a nationally representative study. Neuropsy-
data allowed a formal sensitivity–specificity analysis to be con- chology, 25, 679 – 689. doi:10.1037/a0024483
ducted, sensitivity and specificity values were both well above Brown, R., & McNeill, D. (1966). The “tip of the tongue” phenomenon.
chance. The fourth item of general significance, considering the ε4 Journal of Verbal Learning and Verbal Behavior, 5, 325–337. doi:
10.1016/S0022-5371(66)80040-3
allele’s stature as a biomarker of AD and the expense of perform-
Bugaiska, A., Morson, S., Moulin, C. J. A., & Souchay, C. (2011).
ing genetic tests on older adults, is that the genetic data added Metamemory, recollection and familiarity in Alzheimer’s disease. Revue
nothing to dual-retrieval processes when it came to predicting Neurologique, 167, 3–13. doi:10.1016/j.neurol.2010.03.001
future disease. In the ADAMS, ε4 was simply not associated with Creamer, S., & Schmitter-Edgecombe, M. (2010). Narrative comprehen-
conversion to AD, whereas dual-retrieval processes were. In the sion in Alzheimer’s disease: Assessing inferences and memory opera-
 DUAL-RETRIEVAL MODELS 63

tions with a think-aloud procedure. Neuropsychology, 24, 279 –290. Koriat, A. (1993). How do we know that we know? The accessibility model
doi:10.1037/a0018107 of the feeling of knowing. Psychological Review, 100, 609 – 639. doi:
Dalla Barba, G. (1997). Recognition memory and recollective experience 10.1037/0033-295X.100.4.609
in Alzheimer’s disease. Memory, 5, 657– 672. doi:10.1080/741941546 Koriat, A. (1995). Dissociating knowing and feeling of knowing: Further
de Jager, C. A., Hogervorst, E., Combrinck, M., & Budge, M. M. (2003). evidence for the accessibility model. Journal of Experimental Psychol-
Sensitivity and specificity of neuropsychological tests for mild cognitive ogy: General, 124, 311–333. doi:10.1037/0096-3445.124.3.311
impairment, vascular cognitive impairment and Alzheimer’s disease. Koriat, A., Levy-Sadot, R., Edry, E., & de Marcas, S. (2003). What do we
Psychological Medicine, 33, 1039 –1050. doi:10.1017/ know about what we cannot remember? Accessing the semantic attri-
S0033291703008031 butes of words that cannot be recalled. Journal of Experimental Psy-
Diana, R. A., Yonelinas, A. P., & Ranganath, C. (2007). Imaging recol- chology: Learning, Memory, and Cognition, 29, 1095–1105. doi:
lection and familiarity in the medial temporal lobe: A three-component 10.1037/0278-7393.29.6.1095
model. Trends in Cognitive Sciences, 11, 379 –386. doi:10.1016/j.tics Kurilla, B. P., & Westerman, D. L. (2010). Source memory for unidentified
.2007.08.001 stimuli. Journal of Experimental Psychology: Learning, Memory, and
Fisher, G. G., Franks, M. M., Plassman, B. L., Brown, S. L., Potter, G. G., Cognition, 36, 398 – 410. doi:10.1037/a0018279
Llewellyn, D., . . . Langa, K. M. (2011). Caring for individuals with Langa, K. M., Plassman, B. L., Wallace, R. B., Herzog, A. R., Heeringa,
dementia and cognitive impairment, not dementia: Findings from the S. G., Ofstedal, M. B., . . . Willis, R. J. (2005). The Aging, Demograph-
Aging, Demographics, and Memory Study. Journal of the American ics and Memory Study: Study design and methods. Neuroepidemiology,
Geriatrics Society, 59, 488 – 494. doi:10.1111/j.1532-5415.2010 25, 181–191. doi:10.1159/000087448
.03304.x Light, L. L., Prull, M. W., La Voie, D. J., & Healy, M. R. (2000).
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-Mental Dual-process theories of memory in old age. In T. J. Perfect & E. A.
State: A practical method of grading the cognitive state of patients for Maylor (Eds.), Models of cognitive aging (pp. 238 –300). Oxford, Eng-
the clinician. Journal of Psychiatric Research, 12, 189 –198. doi: land: Oxford University Press.
10.1016/0022-3956(75)90026-6 Malmberg, K. J. (2008). Recognition memory: A review of the critical
Gallo, D. A., Cramer, S. J., Wong, J. T., & Bennett, D. A. (2012). findings and an integrated theory for relating them. Cognitive Psychol-
Alzheimer’s disease can spare local metacognition despite global ogy, 57, 335–384. doi:10.1016/j.cogpsych.2008.02.004
anosognosia: Revisiting the confidence–accuracy relationship in epi-
McKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T., Jack,
sodic memory. Neuropsychologia, 50, 2356 –2364. doi:10.1016/j
C. R., Kawas, C. H., . . . Phelps, C. H. (2011). The diagnosis of dementia
.neuropsychologia.2012.06.005
due to Alzheimer’s disease: Recommendations from the National Insti-
Ghetti, S., & Angelini, L. (2008). The development of recollection and
tute on Aging–Alzheimer’s Association workgroups on diagnostic
familiarity in childhood and adolescence: Evidence from the dual-
guidelines for Alzheimer’s disease. Alzheimer’s & Dementia, 7, 263–
process signal detection model. Child Development, 79, 339 –358. doi:
269. doi:10.1016/j.jalz.2011.03.005
10.1111/j.1467-8624.2007.01129.x
Morris, J. C., Heyman, A., Mohs, R. C., Hughes, J. P., VanBelle, G.,
Gomes, C. F. A. (2013). On the relation between memory and metamemory
Fillenbaum, G., . . . Clark, C. (1989). The Consortium to Establish a
in free recall: The effects of list length and word frequency on dual
Registry for Alzheimer’s Disease (CERAD): Part I. Clinical and neuro-
processes (Unpublished master’s thesis). Cornell University.
psychological assessment of Alzheimer’s disease. Neurology, 39, 1159 –
Gomes, C. F. A., Brainerd, C. J., & Stein, L. M. (2013). Effects of
1165. doi:10.1212/WNL.39.9.1159
emotional valence and arousal on recollective and nonrecollective recall.
Nelson, P. T., Braak, H., & Markesbery, W. R. (2009). Cognitive impair-
Journal of Experimental Psychology: Learning, Memory, and Cogni-
ment in Alzheimer disease: A complex but coherent relationship. Jour-
tion, 39, 663– 677. doi:10.1037/a0028578
Harel, B. T., Darby, D., Pietrzak, R. H., Ellis, K. A., Snyder, P. J., & nal of Neuropathology and Experimental Neurology, 68, 1–14. doi:
Maruff, P. (2011). Examining the nature of impairment in visual paired 10.1097/NEN.0b013e3181919a48
associate learning in amnestic mild cognitive impairment. Neuropsy- Osgood, C. E. (1952). The nature and measurement of meaning. Psycho-
chology, 25, 752–762. doi:10.1037/a0024237 logical Bulletin, 49, 197–237. doi:10.1037/h0055737
Health and Retirement Study. (2011). Unpublished dataset, University of Panza, F., D’Introno, A., Colacicco, A. M., Capurso, C., Del Parigi, A.,
Michigan. Caselli, R. J., . . . Solfrizzi, V. (2005). Current epidemiology of mild
Heathcote, A., Raymond, F., & Dunn, J. (2006). Recollection and famil- cognitive impairment and other predementia syndromes. American Jour-
iarity in recognition memory: Evidence from ROC curves. Journal of nal of Geriatric Psychiatry, 13, 633– 644.
Memory and Language, 55, 495–514. doi:10.1016/j.jml.2006.07.001 Pericak-Vance, M. A., Grubber, J., Bailey, L. R., Hedges, D., West, S.,
Hebert, L. E., Scherr, P. A., Bienias, J. L., Bennett, D. A., & Evans, D. A. Santoro, L., . . . Haines, J. L. (2000). Identification of novel genes in
(2003). Alzheimer disease in the U.S. population: Prevalence estimates late-onset Alzheimer’s disease. Experimental Gerontology, 35, 1343–
using the 2000 census. Archives of Neurology, 60, 1119 –1122. doi: 1352. doi:10.1016/S0531-5565(00)00196-0
10.1001/archneur.60.8.1119 Petersen, R. C. (2004). Mild cognitive impairment as a diagnostic entity.
Hicks, J. L., & Marsh, R. L. (2002). On predicting the future states of Journal of Internal Medicine, 256, 183–194. doi:10.1111/j.1365-2796
awareness for recognition of unrecallable items. Memory & Cognition, .2004.01388.x
30, 60 – 66. doi:10.3758/BF03195265 Petersen, R. C., Roberts, R. O., Knopman, D. S., Geda, Y. E., Cha, R. H.,
Jacoby, L. L. (1991). A process dissociation framework: Separating auto- Pankratz, V. S., . . . Rocca, W. A. (2010). Prevalence of mild cognitive
matic from intentional uses of memory. Journal of Memory and Lan- impairment is higher in men: The Mayo Clinic study of aging. Neurol-
guage, 30, 513–541. doi:10.1016/0749-596X(91)90025-F ogy, 75, 889 – 897. doi:10.1212/WNL.0b013e3181f11d85
Juster, F. T., & Suzman, R. (1995). An overview of the Health and Plassman, B. L., Langa, K. M., Fisher, G. G., Heeringa, S. G., Weir, D. R.,
Retirement Study. Journal of Human Resources, 30, S7–S56. doi: Ofstedal, M. B., . . . Wallace, R. B. (2007). Prevalence of dementia in the
10.2307/146277 United States: The Aging, Demographics, and Memory Study. Neuro-
Kawas, C., & Katzman, R. (1999). Epidemiology of dementia and Alzhei- epidemiology, 29, 125–132. doi:10.1159/000109998
mer disease. In R. D. Terry, R. Katzman, K. L. Bick, & S. S. Sisodia Plassman, B. L., Langa, K. M., Fisher, G. G., Heeringa, S. G., Weir, D. R.,
(Eds.), Alzheimer disease (pp. 95–116). New York, NY: Raven Press. Ofstedal, M. B., . . . Wallace, R. B. (2008). Prevalence of cognitive
64 BRAINERD ET AL.

impairment without dementia in the United States. Annals of Internal Storandt, M. (2008). Cognitive deficits in the early stages of Alzheimer’s
Medicine, 148, 427– 434. doi:10.7326/0003-4819-148-6-200803180- disease. Current Directions in Psychological Science, 17, 198 –202.
00005 doi:10.1111/j.1467-8721.2008.00574.x
Plassman, B. L., Langa, K. M., McCammon, R. J., Fisher, G. G., Potter, Strong, E. K. (1913). The effect of time-interval upon recognition memory.
G. G., Burke, J. R., . . . Wallace, R. B. (2011). Incidence of dementia and Psychological Review, 20, 339 –372. doi:10.1037/h0072087
cognitive impairment, not dementia in the United States. Annals of Summers, M. J., & Saunders, N. L. J. (2012). Neuropsychological mea-
Neurology, 70, 418 – 426. doi:10.1002/ana.22362 sures predict decline to Alzheimer’s dementia from mild cognitive
Rajaram, S. (1996). Perceptual effects on remembering: Recollective pro- impairment. Neuropsychology, 26, 498 –508. doi:10.1037/a0028576
cesses in picture recognition. Journal of Experimental Psychology: Tabert, M. H., Manley, J. J., Liu, X. H., Pelton, G. H., Rosenblum, S.,
Learning, Memory, and Cognition, 22, 365–377. doi:10.1037/0278-7393 Jacobs, M., . . . Devanand, D. P. (2006). Neuropsychological prediction
.22.2.365 of conversion to Alzheimer’s disease in patients with mild cognitive
Ranganath, C. (2010). Binding items and contexts: The cognitive neuro- impairment. Archives of General Psychiatry, 63, 916 –924. doi:10.1001/
science of episodic memory. Current Directions in Psychological Sci- archpsyc.63.8.916
ence, 19, 131–137. doi:10.1177/0963721410368805 Tse, C. S., Balota, D. A., Moynan, S. C., Duchek, J. M., & Jacoby, L. L.
Ratcliff, R., Van Zandt, T., & McKoon, G. (1995). Process dissociation, (2010). The utility of placing recollection in opposition to familiarity in
single process theories, and recognition memory. Journal of Experimen- early discrimination of healthy aging and very mild dementia of the
Alzheimer’s type. Neuropsychology, 24, 49 – 67. doi:10.1037/a0014887
tal Psychology: General, 124, 352–374. doi:10.1037/0096-3445.124.4
Tulving, E. (1985). Memory and consciousness. Canadian Psychologist,
.352
26, 1–12. doi:10.1037/h0080017
Rey, A. (1941). Psychological examination of traumatic encephalopathy.
Twamley, E. W., Ropacki, S. A. L., & Bondi, M. W. (2006). Neuropsy-
Archives de Psychologie, 28, 286 –340.
chological and neuroimaging changes in preclinical Alzheimer’s disease.
Reyna, V. F., & Brainerd, C. J. (2011). Dual processes in decision making
Journal of the International Neuropsychological Society, 12, 707–735.
and developmental neuroscience: A fuzzy-trace model. Developmental
doi:10.1017/S1355617706060863
Review, 31, 180 –206.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale—Third Edition.
Reyna, V. F., & Mills, B. A. (2007). Interference processes in fuzzy-trace
San Antonio, TX: Psychological Corporation.
theory: Aging, Alzheimer’s disease, and development. In C. MacLeod & Westerberg, C. E., Paller, K. A., Weintraub, S., Mesularn, M. M., Hold-
D. Gorfein (Eds.), Inhibition in cognition (pp. 185–210). Washington, stock, J. S., Mayes, A. R., & Reber, P. J. (2006). When memory does not
DC: American Psychological Association. doi:10.1037/11587-010 fail: Familiarity-based recognition in mild cognitive impairment and
Rotello, C. M., Macmillan, N. A., & Reeder, J. A. (2004). Sum– difference Alzheimer’s disease. Neuropsychology, 20, 193–205. doi:10.1037/0894-
theory of remembering and knowing: A two-dimensional signal detec- 4105.20.2.193
tion model. Psychological Review, 111, 588 – 616. doi:10.1037/0033- Wixted, J. T., & Mickes, L. (2010). A continuous dual-process model of
295X.111.3.588 remember/know judgments. Psychological Review, 117, 1025–1054.
Schacter, D. L., & Worling, J. R. (1985). Attribute information and the doi:10.1037/a0020874
feeling-of-knowing. Canadian Journal of Experimental Psychology, 39, Yonelinas, A. P. (1994). Receiver-operating characteristics in recognition
467– 475. doi:10.1037/h0080074 memory: Evidence for a dual-process model. Journal of Experimental
Serra, L., Bozzali, M., Cercignani, M., Perri, R., Fadda, L., Caltagirone, C., Psychology: Learning, Memory, and Cognition, 20, 1341–1354. doi:
& Carlesimo, G. A. (2010). Recollection and familiarity in amnesic mild 10.1037/0278-7393.20.6.1341
cognitive impairment. Neuropsychology, 24, 316 –326. doi:10.1037/ Yonelinas, A. P. (2002). The nature of recollection and familiarity: A
a0017654 review of 30 years of research. Journal of Memory and Language, 46,
Small, B. J., Rosnick, C. B., Fratiglioni, L., & Backman, L. (2004). 441–517. doi:10.1006/jmla.2002.2864
Apolipoprotein E and cognitive performance: A meta-analysis. Psychol- Zachary, R. A. (1986). Shipley Institute of Living Scale: Revised manual.
ogy and Aging, 19, 592– 600. doi:10.1037/0882-7974.19.4.592 Los Angeles, CA: Western Psychological Services.
 DUAL-RETRIEVAL MODELS 65

Appendix
Forgetting Model

The data space that was analyzed for the CERAD delayed test of-fit test of this model is obtained by computing a likelihood ratio
was the event sequence S2T2S3T3T4. (A similar data space, statistic that compares the likelihood in A9 to the likelihood of the
S4T4S5T5T6, was analyzed for the RAVLT delayed test, and the same data when all 7 free empirical probabilities are free to vary.
developments in this section apply to both the CERAD and That test statistic, which is asymptotically distributed as ␹2(2), is
RAVLT data spaces.) The S2T2S3T3 part is the last two learning
trials, and the T4 part is the delayed recall test. The dual-retrieval G2 ⫽ ⫺2ln[L5 ⁄ L7], (A10)
model for this event sequence has five memory parameters: D, R, where L7 is the likelihood of the data when all empirical proba-
J, FD, and FR. The first three parameters have the same definitions bilities are free to vary.
as in Table 1. FD is a forgetting parameter that measures loss of the This test statistic is used to evaluate within- and between-
ability to retrieve an item’s verbatim trace between T3 and the condition hypotheses about differences in parameter values. For
delayed test, and FR is a forgetting parameter that measures loss of between-condition tests, consider an experiment that contains k
the ability to reconstruct an item from traces of partial identifying conditions. As a test of hypotheses about whether a parameter (say,
information between T3 and the delayed test. Over the three recall FD) differs between a pair of conditions, (a) an experimentwise test
tests, there are eight error–success patterns for any item: C2C3C4, is computed to determine whether there is global statistical evi-
C2C3E4, . . . , E2E3E4. The probabilities of these patterns can be dence that the parameter differs among the k conditions, and (b) if
expressed as functions of the memory parameters: that test yields a null hypothesis rejection, conditionwise tests are
computed to determine whether the parameter differs between
P共C2C3C4兲 ⫽ D(1 ⫺ FD) ⫹ (1 ⫺ D)RJ3(1 ⫺ FR); (A1)
specific pairs of conditions. The first test statistic is
P共C2C3E4兲 ⫽ DFD ⫹ (1 ⫺ D)RJ2[FR ⫹ (1 ⫺ FR)(1 ⫺ J)]; (A2) G2 ⫽ ⫺2ln兵Li5 ⁄ [L5(1) ⫻ L5(2), ⫻ . . . , L5(k)]其
P共C2E3C4兲 ⫽ (1 ⫺ D)RJ (1 ⫺ J)(1 ⫺ FR);
2
(A3) (A11)
P共C2E3E4兲 ⫽ (1 ⫺ D)RJ(1 ⫺ J)[FR ⫹ (1 ⫺ FR)(1 ⫺ J)]; (A4) where the denominator contains the values of the numerator of
A10 that are computed for the data of each of the k conditions and
P共E2C3C4兲 ⫽ (1 ⫺ D)(1 ⫺ R)RJ2(1 ⫺ FR) ⫹ (1 ⫺ D)R(1 ⫺ J)
the numerator contains a single value of the numerator of A10 that
D(1 ⫺ FD) ⫹ (1 ⫺ D)R(1 ⫺ J)(1 ⫺ D)J2(1 ⫺ FR) is computed for the pooled data of the k conditions under the
constraint that the value of the target parameter is the same in all
⫹ (1 ⫺ D)(1 ⫺ R)D(1 ⫺ FR); (A5) conditions. The G2 statistic is asymptotically distributed as ␹2(2k).
The second test statistic, for two conditions i and j, which is
P共E2C3E4兲 ⫽ (1 ⫺ D)2(1 ⫺ R)RJ[FR ⫹ (1 ⫺ FR)(1 ⫺ J)] asymptotically distributed as ␹2(2), is
⫹ (1 ⫺ D)(1 ⫺ R)DFD ⫹ (1 ⫺ D)R(1 ⫺ J)DFD G2 ⫽ ⫺2ln[Lij5 ⁄ (Li5 ⫻ L j5)]. (A12)
⫹ (1 ⫺ D)R(1 ⫺ J)(1 ⫺ D)J[FR ⫹ (1 ⫺ FR)(1 ⫺ J)]; (A6) The numerator is a value of the numerator of A10 that is computed
for the pooled data of the two conditions under the constraint that
P共E2E3C4兲 ⫽ (1 ⫺ D)2(1 ⫺ R)R(1 ⫺ J)(1 ⫺ FR)J the value of the parameter is the same in those conditions, and the
⫹ (1 ⫺ D)R(1 ⫺ J)2(1 ⫺ D)(1 ⫺ FR)J; (A7) denominator is the two values from the denominator of A11 for
these conditions.
P共E2E3E4兲 ⫽ (1 ⫺ D)2(1 ⫺ R)2 ⫹ (1 ⫺ D)2(1 ⫺ R)R(1 ⫺ J)[FR For within-condition tests, these tests compare the values of
different parameter pairs (say, FD vs. FR) within a condition. Such
⫹ (1 ⫺ FR)(1 ⫺ J)] ⫹ (1 ⫺ D)2R(1 ⫺ J)2[FR a test stipulates that a relation of equality or inequality holds
⫹ (1 ⫺ FR)(1 ⫺ J)]. (A8) between the members of the pair. The test statistic is just

The likelihood of any sample of data and estimates of the five G2 ⫽ ⫺2ln[L4 ⁄ L5], (A13)
parameters are obtained by maximizing the following likelihood which is asymptotically distributed as ␹ (1) because the likelihood
2
function: in the numerator is estimated with one less degree of freedom than
L5 ⫽ P共pi兲N(i) . (A9) when all five parameters are free to vary.

The pi are the eight expressions on the right sides of Equations Received March 19, 2013
A1–A8. Because 5 memory parameters are estimated, the likeli- Revision received June 13, 2013
hood in A9 is computed with 2 degrees of freedom. A goodness- Accepted June 17, 2013 䡲