Models as Epistemic Anchors: A Framework for Model-based Instruction

DEVELOPING THE MODELING FRAMEWORK

Philosophy and psychology of science

The recognition that model--based reasoning is central to scientific practice is a relatively recent perspective in the philosophy of science. Historically, philosophers of scientists have been concerned with the nature and structure of scientific theories, and only recently have they actively turned towards the cognitive processes involved in theorizing (e.g Downes, 1992). More modern, "practice--based" accounts of philosophy of science have argued that models sit at the center of scientific activity (Giere, 1988;1994;Godfrey--Smith, 2006). Both historical and contemporary science studies suggest that much of what scientists do involves the construction of hypothetical models, the use of these models to make sense of natural phenomena and the continual evaluation and revision of these models (Clement, 1989;Nersessian, 1992;2002).

We take two important lessons from model--based accounts of scientific practice. The first is the recognition that model--based reasoning is intensely creative and dynamic. Rather than march students through a series of unchanging steps, modeling is a much more contingent process. The implication is that to be successful modelers, students need to be flexible reasoners, able to move between levels of causality and description and capable of monitoring the sense--making they are doing.

The second point is that the nature of modeling practices varies with discipline, or more accurately, modeling practices tend to vary with epistemic culture (c.f. Knorr--Cetina, 1999). In other words, different scientific disciplines tend to pursue different kinds of knowledge. For example, evolutionary biologists seek to reconstruct past events, while molecular biologists attempt to uncover the details of molecular pathways. While both are biologists, the former may be epistemically more closely related to a historian and the latter to an engineer. Thus, some explicit attention to the purpose of modeling is needed in order to guide both students and instructors.

Taken together these two points suggested to us the need for a modeling framework that makes explicit the variety of cognitive processes that comprise modeling and some way of organizing those processes with respect to the purpose of modeling.

Empirical observations of student modelers

The importance of an explicit account of modeling practice became further clarified by our studies of undergraduate modelers -- students in the Collaborative Learning at the Interface of Mathematics and Biology (CLIMB) program. In this program, a small cohort of seven to nine upper--division mathematics and biology majors worked together with science faculty for a full year. The year began with a series of structured modeling tasks from a range of biological disciplines and culminated with a student--driven group modeling project.

The CLIMB program, because it features scientists mentoring students, has been a particularly productive context for observing modeling practices. In the context of instruction, the scientists are more likely to make some of their tacit knowledge about modeling explicit. Furthermore, by directly observing the successes and struggles of students we have been able to refine our understanding of the utility of clear epistemic aims. This insight is supported by two patterns that have emerged from our analyses of CLIMB.

First, we noticed that science faculty repeatedly asked the students, "What is the question?" It was only by framing their instruction in terms of the type of question that the students wanted to answer with their model that the scientists could give them productive advice about how to proceed. Second, we observed that, in many ways, the nature of the question and the corresponding epistemic aim largely dictated the kinds of activities in which the CLIMB students were engaged.

When students were pursuing precise quantitative predictions with their model, their reasoning patterns differed from when they were using their models to clarify concepts. Perhaps even more importantly, we found that when students lost sight of the purpose guiding their modeling, that they fell into relatively unproductive patterns of activity -concerning themselves with superficial features of the representation or searching for a "correct" model that matched what the faculty had published. In sum, our empirical observations further reinforced the need to attend to the aims of modeling and how they might influence opportunities for learning.

INTRODUCING THE MODELING FRAMEWORK: MODELS AS EPISTEMIC ANCHORS

Modeling can be roughly divided into the three categories of model--based activity: model construction, model use, and model evaluation. These activities form the backbone of the Modeling Framework ( Figure 1). Crucially, they do not represent prescribed steps in the modeling process; modeling can begin at any of these points.

Furthermore, this framework makes no commitments about the length of time spent in any particular phase of activity. The transitions between them may take days or seconds. Neither are these phases meant to occur completely independently of one another. Often they occur simultaneously -the construction of a new model occurring in tandem with the evaluation of a prior model for example.

Given these caveats, we find this division a useful way to organize different aspects of modeling for pedagogical purposes. The focus of a lesson might be defined in terms of either construction, use or evaluation. Nevertheless, the Modeling Framework is meant to show how these activities are truly integrated, and perhaps most importantly, that they are linked through interactions with a particular aim. In science, modeling supports a variety of sense--making activities, helping scientists organize ideas, bound their investigations, identify gaps, and make conceptual progress. In this sense, models are best conceived of as "tools for thinking." However, because models are flexible tools, our main contention is that it is important to specify how a model is to be used, for what purpose, in order to productively design model--based instruction. Because modeling requires making specific epistemic commitments about the scope and purpose of inquiry, it can productively shape the inquiry activities by tying them to a particular scientific aim.

In this sense, scientific practice can be viewed as being epistemically anchored to models, while models themselves are anchored to some epistemic aim. We propose an explicit framework ( Figure 1) that highlights these anchoring functions and discuss the assumptions inherent in it and the pedagogical implications of it. This framework demonstrates epistemic anchoring in three ways: meet the desired aim. For example, even if the model is "false," it may still make an important conceptual contribution to the problem. Rather than discard knowledge from the model in this case, one approach is to recognize that a different aim has been achieved. Thus, even as the modeling process evolves over time it remains connected to an aim. 3. Integration among the three phases of modeling. The external arrows show how each of the three phases of modeling is related to one another. The decisions made during model construction will be influenced by the modelers understanding of the appropriate criteria for model evaluation and by the particular use for which the model is being built. The way in which the model is used will be constrained by the decisions made in its construction, while the output, in the form of predictions,

Each phase of modeling activity is tied to a particular aim. The arrows that

ANIMATING THE MODELING FRAMEWORK WITH A VIGNETTE

In order to make the main conceptual points that have emerged this framework more salient, we illustrate them with an empirically derived vignette. In addition, we draw out of this account some productive implications for model--based classroom instruction more generally. We derived this vignette from our research of They then worked as a group to conduct their own research project over the spring and summer. This vignette summarizes the events that took place from early May, as the CLIMB students were beginning to converge on a modeling topic, to late August, as they students were finishing their modeling activities and beginning to prepare a final write up and presentation of their work.

The vignette begins after almost five months of instructor--led modeling activities. The group has now begun the process of defining the research project that will be the focus of their work for the next seven months. After a month of reading scientific papers and brainstorming, the group has begun to converge on a topic.

Originally inspired by a paper on the eradication of polio, the group has decided to explore the possibility of eradicating vaccine--preventable diseases in the case that vaccination is voluntary. In such cases it can be difficult, if not impossible, to completely eradicate a disease because there always remains a reservoir of unvaccinated individuals for the disease to exploit. By jumping between these pockets of non--vaccinators the disease can persist, flaring up on occasion before receding into obscurity, but never is completely extinguished. The group found this problem particularly interesting both from a modeling and from a social perspective. The CLIMB mentors encouraged them to choose a specific disease on which to focus their attention. After several more weeks of research the group narrowed their interests down to two diseases: influenza and measles. At the start of this vignette the group is undecided, and a subgroup of students is presenting their ideas about modeling the dynamics of the influenza.

A Vignette: Modeling the dynamics of vaccine--preventable diseases

In early May, the CLIMB students presented the following idea to their mentors: the 1 incidence of the influenza (flu) virus could be tied to the efficacy of the flu vaccine in 2

the previous year. The students knew that because the flu virus evolves rapidly, the 3 vaccine developers must try to anticipate the characteristics of the virus each year 4 and modify the vaccine accordingly. In some years they do a better job than others, 5 and as a result, the efficacy of the flu vaccine can vary greatly from year to year. The 6 CLIMB students reasoned that if the vaccine was effective people would continue to 7

vaccinate, but if it was not effective people would decide not get vaccinated the 8

following year. This would create a delayed correlation between efficacy in one year 9 and the disease prevalence in the following year. 10 11

The students were initially pleased with this idea. It seemed right, and they planned 12 to build a model to prove it. They were somewhat dismayed when their mentors 13

were not as impressed. The response from all three mentors was similar: What do 14 you want to know? The model was logical and not necessarily incorrect, but it lacked 15 a purpose. Was it going to help them explain some phenomenon? Was it going to 16 make usable predictions? Was it going to help them make theoretical progress on 17 some poorly understood problem? 18 19

The mentors suggested one strategy for grounding their model to an aim -try to 20

find an empirical pattern that needs explaining. The students' initial flu idea 21 assumed that there was a connection between vaccine efficacy and voluntary 22

vaccination. The mentors suggested they begin by finding out if such a correlation 23 actually existed. After a few days of research, the students found no such 24 correlation. They did however find data on rising measles prevalence in the UK that 25 correlated well with the timing of a major autism--related vaccine scare. The focus of 26 the project became building a model that could account for the UK data pattern. 27 28

The group now began the process of constructing a model. One of their first steps 29 was to come up with a long list of questions to consider: how would they model 30 disease transmission? Would they treat all people the same or include different age--31 classes in their model? How would potential vaccinators learn information about 32 vaccine risk or disease prevalence? Should they account for immigration and 33 emigration? After several days the mentors once again intervened and suggested 34 that the students look carefully at existing models in the science literature. 35 36

For the next several weeks, the students read and evaluated a number of disease 37 modeling papers. They assessed the simplifications and assumptions made by the 38 authors, and they began to generate a spreadsheet of key modeling decisions along 39 with the author's justification (if given) for his choice. For example, one author 40

wanted to construct a model that could accurately replicate the empirical pattern of 41 biennial cycling of measles incidence. He found that in order to do this he had to 42 include different parameters for different age--classes (such models are said to 43

include "age--structure"). Another author wanted to explore how imitative learning 44 could impact disease spread, but omitted age--structuring for the sake of simplicity. 45 46

The students decided that they wanted to include age--structure to make their model 47 more realistic, but that they also wanted to explore different ideas about human 48 learning. They decided that they didn't think imitation was a reasonable mechanism 49 of how information about vaccination spreads through human populations. Instead, 50 they decided to explore two kinds of information transfer: one based on dyadic 51 interactions, which they termed "social learning," and a second "environmental" 52

form of learning in which information would be simultaneously available to a large 53 fraction of the population, a process meant to mimic learning from media reports. 54 55

By late summer, the CLIMB cohort had developed a computer simulation that was 56 producing output about the proportion of vaccinators predicted by their model and 57

the corresponding level of disease prevalence over time. However, they noticed that 58 they were still far from replicating the original empirical patterns that inspired their 59

work. The students were disheartened by this lack of fit. They had wanted their 60 model to be able to make accurate predictions about how measles prevalence in the 61 UK might change in the coming years. 62 63

The CLIMB mentors, on the other hand, were not overly disappointed by this lack of 64 fit to the data. Instead, they took the opportunity to redirect the students towards a 65 different aim. They suggested that a worthwhile feature of their model was that it 66 allowed them to explore two possible mechanisms of information transfer ("social" 67

and "environmental" learning) and the implications of these mechanisms for 68

patterns of disease spread in general. 69 70

This shifted how the students used the model. Instead of trying to tweak their 71 simulation so that it matched the data, the students now systematically varied 72 different parameters with the purpose of exploring how different combinations of 73 parameter values changed the simulation output. By looking for patterns in this 74 output, the group was able to make general conceptual statements about how 75 different forms of information spread could, in theory, influence disease dynamics. 76

Applying the Modeling Framework

We now examine the key ways in which models helped structure the CLIMB students' inquiry project in ways that kept it anchored to authentic scientific practice and clear epistemic aims. In the above vignette, the CLIMB mentors used several strategies to keep students reasoning productively about their project. In the above example, three key features of the modeling framework are evident: the importance of a clear aim, the dynamic link between model and aim, and the integration of model construction, use and evaluation.

The importance of articulating the aim. When the CLIMB students first

began to think about a modeling project, they proposed a number of topics that they could build models "of." The above vignette began with the students describing how they might build a model of flu dynamics (lines 1--10). A crucial shift occurred when the CLIMB mentors asked them to consider what they might build a model "for" (lines 12--18). This is an example of how the mentors helped the students anchor their model construction firmly to an epistemic aim (lines 20--27). Ultimately, the aim helped the students make appropriate decisions about how to construct their model. For example, because they initially wanted their model to be realistic enough to generate accurate predictions, they decided it would be important to include age-structure in their model (lines 47--49).

The dynamic link between model and aim.

Just as the ultimate aim of the project influenced the students' modeling, the ultimate form of their modeling created the need to shift aims. While the focus of the project shifted several times in more subtle ways, one of the largest shifts came when students realized that their model output did not match the empirical data (lines 57--63). At this point in the project the students were evaluating their model. Because they perceived that the primary aim of their modeling was to make accurate predictions about disease dynamics, they were disappointed to find that their model, when evaluated for empirical accuracy, did not stand up to the data. Nevertheless, the CLIMB mentors were able to see that the model could be used to fulfill a different kind of aim. They suggested that the students use the model in a more exploratory way (lines 65--70).

In this way, the evaluation of the model led to a shift in aim. This shift in aim then changed the criteria for evaluation. Instead of holding the model up against data, the ability of the model to generate new ideas about the interaction between information spread and disease dynamics became the new criteria for success (lines 72--78).

3. Integrating model construction, use and evaluation. At a course grain the modeling process of the CLIMB students could be divided into three phases: first, the students constructed a model, they then used the model to develop and run computer simulations, and finally, they evaluated the success of their model. While each of these phases are somewhat distinct, the above vignette demonstrates how at each phase the CLIMB students and their mentors were both anticipating the next step and drawing on the previous one. As they constructed their model they drew extensively from their evaluations of existing models in the literature (lines 37--45).

By developing an understanding for the choices made by prior modelers, the CLIMB students were better prepared to make and justify the choices that went into their own model construction. These choices were made in anticipation of the model's intended use; the students opted to include age--structure so that it would be realistic enough to generate accurate predictions (lines 47--49) and included different modes of "learning" to explore possible alternatives to imitation dynamics (lines 48--55). When the students prepared to use their model to make predictions, they recognized that they would evaluate their model output against the UK data.

Later, when they shifted to using their model to explore possible mechanisms of information spread, they were less concerned about fitting data since they were now using their model to explore more conjectural elements of their model (lines 72--78). Finally, the ways in which they evaluated their model informed future iterations of model construction. Once the group accepted that their model could not match the empirical data, they decided to spend more time carefully articulating the mechanisms of learning, specifying the details of dyadic social learning and environmental learning in the construction of their revised model.

IMPLICATIONS FOR MODELING IN CLASSROOMS

Studying upper--division undergraduate students and scientist mentors has numerous advantages, but we acknowledge the insights gained from this context need not be applicable to K--12 classrooms. Nevertheless, we believe the use of the Modeling Framework has allowed us to distill, from the CLIMB context, several key points that are worth bringing into the discussion of model--based instruction in classroom settings that can, in fact, apply to K--12.

The need for clear epistemic aims.

Modeling means different things in different contexts. Even within the scientific community, models are used for a variety of different purposes (Svoboda and Passmore, in review). Moreover, it is sometimes the case that these purposes are not mutually compatible; there are often tradeoffs to consider when constructing a model (Levins, 1966). As the CLIMB example illustrated, a model that is conceptually powerful may not be able to yield accurate predictions.

The Modeling Framework suggests that regardless of the type of modeling task that is presented to students, there should be a clear sense of for what purpose the model is being constructed, used or evaluated. This is important because it gives focus to what could otherwise be a confusing task. Modeling requires making a number of important decisions and choices. A clear sense of purpose was important for the CLIMB students, but we argue that is perhaps even more important in the context of classroom tasks. For example, asking students to "build a model of [x]" is not a very specific task. The way that students approach this task will depend on what they think a model is. We propose that it might be more productive to categorize modeling task by epistemic aim. "Construct a model that can explain how [x] happens" is a different kind of task than "construct a model the makes an accurate prediction about the value of [x] at time t," which is in turn different from, "construct a model that explores the possible ways in which [y] might affect [x]."

In science, models are tools for sense--making. Scientists use models to make sense in different ways: sometimes by crafting detailed explanations, sometimes by making predictions, and sometimes by exploring the space of possibilities. In order for models to be used like this in classrooms, it needs to be clear to both students and teachers how a model will help them make sense of a particular question or problem. Ultimately giving students a sense for how models can help them reason is as important as the reasoning itself.

A similar argument applies to the context of model evaluation. Instead of presenting students with a list of evaluation criteria, we need to link these criteria to specific aims. We might tell students that the ideal model is simple, explanatory, predictive, clear, accurate etc. However, maximizing all of these criteria is rarely possible. Sometimes a simple model is better, but often simple models, while elegant, cannot make accurate predictions. Sometimes an accurate model is better, but often models that can closely match data are not conceptually powerful. In the CLIMB example, the students desperately wanted their model to match the data.

Had they focused on this goal it is likely that their model would have lost some of the conceptual power that made it so interesting.

Integrated modeling creates productive constraint.

The extended model--based inquiry of the type undertaken by the CLIMB students can seem daunting. It was challenging for both CLIMB students and CLIMB mentors. We do not necessarily advocate that classroom modeling tasks take students through iterative cycles of model construction, use and evaluation.

Nevertheless, we do think it is important for both instructors and students to be able to situate what they are doing in the larger context of model--based inquiry.

That is, while we can imagine student tasks that focus on each phase independent of the others, in order to give these tasks meaning, they need to be understood in the context of the others. In fact, we argue that integrating model construction, use and evaluation can productively constrain students' modeling activities. When each phase is informed by the previous and conducted in anticipation of the next phase, the number of choices students must make is reduced. As we saw in the CLIMB example, knowing how they were going to use their model, helped them make specific choices about what to include in their model, and evaluating prior models helped the students better understand the kinds of choices they would make as model builders.

Following productive lines of inquiry.

One of the primary advantages of modeling is that it is often a generative activity. However, the insights that emerge from modeling are not always anticipated ahead of time. While it is important to be clear about modeling aims, it may not always be necessary to be rigid about them. Modeling is a creative and generative process, like writing or designing, modeling can take the modeler in unexpected directions.

In the CLIMB example, neither the students nor the mentors anticipated that one of the most important contributions of their model would be an exploration of how information about vaccination decisions moves through human populations.

Nevertheless, once this contribution was recognized, the mentors encouraged the students to pursue it. We can imagine this happening in classrooms, though perhaps in less dramatic ways. As students and teachers use models to make sense, they may encounter unanticipated questions, fuzzy concepts or alternative explanations. We would encourage instructors to pursue some of these productive detours from the original aim.

Doing so requires having the flexibility, for example, to let go of the aim of getting students to generate the "correct" model, or at least letting go of this as the sole aim of the activity. If models are used only as vehicles for teaching known content, students will miss out on the many opportunities for constructive reasoning. Moreover, a focus on "correct" models can perpetuate epistemological beliefs that are antithetical to authentic science. This is not to say that we should never present students with historical or contemporary consensus models. But as we have argued, we should do so in ways that situate these models in the context of their use. For example, the model of Mendelian inheritance is particularly useful for making predictions about the expected phenotypic frequencies of crosses for single-locus traits. In the context of simple breeding experiments, the Mendelian model is a "good" model. However, as the number of genes underlying a trait grows, the model breaks down -it is unable to explain the inheritance patterns of multi--locus traits.

Allowing the model to vary with context will require taking a meta--cognitive perspective to teaching modeling. As other author's have argued, in addition to teaching students the modeling "process" we should be helping them to understand the nature and purpose[s] of modeling (e.g. Schwarz & White, 2005). Such knowledge will allow students to make the most of modeling by allowing them to participate in the sophisticated reasoning, deep content understanding and rich discussions about the nature of science itself that model--based instruction has the potential to support.