Papers by Christina Christie
New Directions for Evaluation, Mar 1, 2017
Rooted in ideas from operations research in the 1930s, improvement science bloomed in the healthc... more Rooted in ideas from operations research in the 1930s, improvement science bloomed in the healthcare literature during the 1990s and has since then spread rapidly across fields such as management, social work, behavioral economics, and most recently education (Lewis, 2015). So what is thing called "improvement science"? What is the intellectual foundation of improvement science? And what does it look like in real-world applications? What, if anything, might we, as evaluators, learn from the techniques and tools of improvement science? These are questions that will be addressed in this chapter.
Evaluation and Program Planning, Oct 1, 2023
American Journal of Evaluation, Mar 4, 2021
Since 2007, two taxonomies have been proposed to identify the components of evaluation practice t... more Since 2007, two taxonomies have been proposed to identify the components of evaluation practice that may be specified in an evaluation policy. Little is known, however, about how these taxonomies align with evaluation policies developed by philanthropic foundations. Through thematic analysis, this article first compares 12 foundation evaluation policies to the components of the two existing taxonomies. Based on this analysis, an updated taxonomy is proposed, with 11 components to inform evaluation policies in the philanthropic sector and beyond. The 12 foundation policies are then reexamined in relation to the new taxonomy to compare their length, breadth, and depth. This article concludes with implications for practice and suggestions for future research.
New Directions for Evaluation, Sep 7, 2018
The present chapter provides the conceptual foundation on which the remainder of this special iss... more The present chapter provides the conceptual foundation on which the remainder of this special issue is grounded. First, the chapter considers the idea of an evaluation marketplace and reflects on the limited attention awarded to the commercial aspects of evaluation. Second, the scarce-yet significant-literature on the evaluation market and industry is considered, identifying three distinct strands of contributions: the evaluation market composition, market dynamics, and strategies for navigating these. In the third section, the chapter offers working definitions of the evaluation market and evaluation services as well as presents the evaluation market framework, an emerging framework for better understanding the central components of the evaluation marketplace, including its context, composition, and dynamics. A brief set of concluding notes brings the chapter to a close.
Evaluation, Apr 1, 2018
The push for evidence-based practice is persistent in the public sector-what counts is what works... more The push for evidence-based practice is persistent in the public sector-what counts is what works. One central premise for evidence-based practice is the existence of an evidence base; that is, an accumulated and generalizable body of knowledge. Informed by a recent systematic review, we examine the promises and pitfalls of meta-analysis (the statistical workhorse of systematic reviews) as the primary blueprint for cumulative knowledge building in evaluation. This analysis suggests that the statistical assumptions underlying the meta-analytic framework raise issues that, at least in regards to producing generalizable knowledge, may cut even deeper than is suggested by common criticisms. Advancing beyond meta-analysis, we consider alternative approaches for knowledge building and reflect on the implications of these for individual evaluations.
New Directions for Evaluation, Mar 1, 2017
In this chapter, we discuss the similarities and points of departure between improvement science ... more In this chapter, we discuss the similarities and points of departure between improvement science and evaluation, according to use, valuing, and methodsthree dimensions of evaluation theory to which all theorists attend (Christie & Alkin, 2012). Using these three dimensions as a framework for discussion, we show some of the ways in which improvement science and evaluation are similar and how they are different in terms of purposes, goals, and processes. By doing so we frame the illustrative cases of improvement science that follow in this issue.
American Journal of Evaluation, Apr 7, 2019
Many organizations now have evaluation policies. Because formal evaluation policies intend to fra... more Many organizations now have evaluation policies. Because formal evaluation policies intend to frame evaluation practice, it is important to understand the aims of policies and their influence on practice. Evaluation theory is also intended to guide practice; therefore, the extent to which policies are informed by theory is also important to consider, given the related purposes of evaluation theories and policies. Because evaluation policies have emerged with greater occurrence relatively recently, the conceptual and empirical work on the theory-policy-practice connection is limited. Our aim in this article is to flesh out the intersection between evaluation theory, policy, and practice. As illustrative examples, we discuss the three policies in terms of methods, use, and valuing-three core dimensions of evaluation theory. The article concludes with a set of reflections on the different strategies for and implications of a stronger integration of evaluation theory in evaluation policies.
New Directions for Evaluation, 2018
This chapter describes the main features of the demand side of the U.S. federal evaluation market... more This chapter describes the main features of the demand side of the U.S. federal evaluation market. The chapter is structured according to three sections. Section one provides a historical account of the overarching developments of the U.S. federal evaluation market, awarding special attention to the legislative drivers of federal funding for evaluation. In logical extension, the second section documents the current federal evaluation funding across the twenty‐two non‐defense departments subject to the provisions in the 2010 Government Performance and Results Modernization Act. Advancing toward a better understanding of the federal procurement and contracting of evaluation, a more fine‐grained analysis of evaluation contracts in the Department of Health and Human Services is provided. Finally, the third section concludes the chapter by summarizing and considering the implications of the findings in relation to evaluation practice.
Journal of Multidisciplinary Evaluation, Aug 6, 2021
Background: Unlike scientific theories, evaluation theories are prescriptive: a set of actions an... more Background: Unlike scientific theories, evaluation theories are prescriptive: a set of actions and approaches that should be followed when conducting an evaluation. While evaluation theorists have offered a variety of writings describing their theories and approaches, few have offered a specific outline of what the theory looks like in practice. Thus, Alkin and Christie formulated a book to aid evaluators in how to apply theories in evaluations (Alkin & Christie, forthcoming). This book culminates in a series of prototypes that outline each theory's goals, appropriate contexts, prescriptions, and observable actions in application. Purpose: In order to aid evaluators in applying theories, this article seeks to provide a basis for comparison that can be used to help evaluators select which theory would be most appropriate in their practice. Setting: This comparison can be applied in any setting where evaluations fit the context prescribed by each of the theories. Intervention: Not applicable. Research Design: Not applicablre. Data Collection and Analysis: Not applicable. Findings: In order for theories to influence practice effectively, theories must be displayed in a way that allows for easy comparison. This comparison of three theory prototypes demonstrates that prototypes can be an effective way for selecting a prescriptive theory when conducting an evaluation.
New Directions for Evaluation, Jun 1, 2022
for the J. Paul Getty Museum. 1 The Explorer and Detective families featured in this story reflec... more for the J. Paul Getty Museum. 1 The Explorer and Detective families featured in this story reflect a composite of observed families. The Alexanders, for example, reflect Detective families 506D4 and 507D8 (pointing); 429D2, 507D3, and 507D5 (gesturing); 428D1 (handing the card to a parent); and 528D7 (seeking verification of found artwork from parent).
New Directions for Evaluation, Sep 7, 2018
This chapter reflects on the findings from a comparative analysis of the cases comprising this is... more This chapter reflects on the findings from a comparative analysis of the cases comprising this issue, summarizing and expanding on key findings from these in relation to the evaluation market framework. Major implications of the findings for evaluation practice are considered. The chapter concludes with an outline of a future research agenda for the evaluation marketplace.
American Journal of Evaluation, Dec 1, 2006
... how AI has the potential for building evaluation capacity and contributing to an organization... more ... how AI has the potential for building evaluation capacity and contributing to an organization's desire to be a learning organization. Christina A. Christie at Google Indexer on July 28, 2010 aje.sagepub.com Downloaded from
New Directions for Evaluation, Dec 1, 2015
The importance of context in shaping evaluation practice is broadly accepted. Still, there is muc... more The importance of context in shaping evaluation practice is broadly accepted. Still, there is much to be learned about context, and we remain at the early stages of this process. In this chapter, we describe a framework for organizing and focusing ideas for conducting research on evaluation context. We hope that this tool will be a source of inspiration for future studies of context, thus contributing to a deeper and higher quality evaluation knowledge base.
New Directions for Evaluation, Sep 1, 2019
SAGE Publications Ltd eBooks, Apr 30, 2012
Evaluation and Program Planning, Feb 1, 2023
SAGE Publications, Inc. eBooks, 2015
New Directions for Evaluation, Mar 1, 2017
This chapter focuses on features of improvement science that the evaluation field could benefit f... more This chapter focuses on features of improvement science that the evaluation field could benefit from, drawing from case examples. Themes addressed include learning from error, learning from and reducing variation, systems thinking, and learning for scale. We also consider opportunities for professional development, whereby evaluation professionals and people in practice can successfully apply improvement science techniques and methods within and across fields and in the complex systems that shape health, social, and educational outcomes. Finally, we describe resource and management challenges of building improvement science into the evaluation repertoire.
Uploads
Papers by Christina Christie