Papers by Molly Stuhlsatz
Research in Education
In this paper, we critically examine the way in which scholars have traditionally defined and pro... more In this paper, we critically examine the way in which scholars have traditionally defined and problematized attrition. Through a series of examples of large-scale intervention impact studies, we share insights about the sources and consequences of attrition that expand our notion of how and why attrition occurs. We also discuss potential steps for anticipating, mitigating, and responding to attrition in the dynamic context of schooling. By expanding our understanding of attrition, we hope to engage the field in further dialogue that could lead to policies and practices that might lessen the potential impacts not only on our ability to conduct research, but also our ability to advance the learning of teachers and their students.
International Journal of Science and Mathematics Education
Social Sciences & Humanities Open
2021 ASEE Virtual Annual Conference Content Access, Jul 26, 2021
Science Educator, 2019
This longitudinal impact study examined the effects of a leadership development program for distr... more This longitudinal impact study examined the effects of a leadership development program for district-based teams of educators leading science education reform efforts. Propensity score matching was used to construct a comparison group of demographically similar districts who received either none or extant leadership development programming. Treatment effects were estimated at one-year intervals for eight years after the onset of the treatment program. The longitudinal pattern of treatment effects suggests that outcomes were similar for students in treatment and comparison districts in the early phases of implementation followed by a dip in outcomes for treatment districts that eventually recovers to be higher than comparison districts by 7 years from treatment onset. Analyses of data from treatment districts suggest a strong positive association between higher student outcomes and both the existence of highly effective professional learning communities and higher retention of leadership team members.
2021 ASEE Virtual Annual Conference Content Access, Jul 26, 2021
Presentation made by Louise Mead et al. as part of the "Bringing Research Data to the Ecolog... more Presentation made by Louise Mead et al. as part of the "Bringing Research Data to the Ecology Classroom: Opportunities, Barriers, and Next Steps" Session at the Ecological Society of America annual meeting, August 8th, 2017, Portland Oregon
Author contributions: Donovan (PI) was responsible for the study concept, design of all RCTs, fun... more Author contributions: Donovan (PI) was responsible for the study concept, design of all RCTs, funding acquisition, development of all intervention materials, teaching the intervention in RCT 1, all statistical analyses, and manuscript writing. Semmens co-developed the study design for RCT 1 and the curricular framework for the first iteration of the intervention materials. Keck codeveloped all intervention materials, taught the interventions in RCT 1, and assisted in the thinkalouds and edits that produced the intervention materials for RCTs 2-3. Brimhall edited all intervention materials, taught the intervention in RCT 1, and implemented RCT 3 at her school site. Busch co-developed the first iteration of the intervention materials for the climate change control condition in RCT 1. Weindling and Duncan co-developed the intervention materials for RCTs 2-3, recruited teachers for RCTs 2-3, and aided in conducting RCTs 2-3. Stuhlsatz, Buck Bracey, Bloom, and Co-PI Kowalski assisted in ...
All of the interventions described in the included studies are science education interventions.
While scientific argumentation in central to examining and evaluating how data transforms into ex... more While scientific argumentation in central to examining and evaluating how data transforms into explanations of phenomena, few students have the opportunities to do so. The claims-evidence-reasoning (CER) framework has been productive in helping students develop strong scientific arguments, but students who use this framework often do not provide adequate/sufficient evidence or reasoning to support the claims they make when explaining phenomena. In this mixed methods case study, we examine students' written responses to a series of CER-scaffolded activities to investigate how their constructed claims, evidence and reasoning statements change over time, and to identify features (or types of implementations) of the activities that may have contributed to those changes.
Advances in STEM Education, 2018
In this chapter, we describe an approach to professional development grounded in a model of PCK t... more In this chapter, we describe an approach to professional development grounded in a model of PCK that centers around a teacher’s knowledge of how to elicit and respond to student thinking, to engage students in the practices of science, and to develop coherent and effective learning experiences. This approach has been shown to be effective in increasing PCK in both in-service professional development and pre-service teacher education contexts and has resulted in increased student achievement when compared to more traditional approaches. Current and future directions for this work include examining the pathways between teacher professional knowledge and student learning, further development of our approach to measuring PCK using automated analysis of teacher writing, and exploring how to effectively scale up this somewhat resource-intensive approach to teacher learning.
Frontiers in Education, 2021
Estimating and monitoring the construct-irrelevant variance (CIV) is of significant importance to... more Estimating and monitoring the construct-irrelevant variance (CIV) is of significant importance to validity, especially for constructed response assessments with rich contextualized information. To examine CIV in contextualized constructed response assessments, we developed a framework including a model accounting for CIV and a measurement that could differentiate the CIV. Specifically, the model includes CIV due to three factors: the variability of assessment item scenarios, judging severity, and rater scoring sensitivity to the scenarios in tasks. We proposed using the many-facet Rasch measurement (MFRM) to examine the CIV because this measurement model can compare different CIV factors on a shared scale. To demonstrate how to apply this framework, we applied the framework to a video-based science teacher pedagogical content knowledge (PCK) assessment, including two tasks, each with three scenarios. Results for task I, which assessed teachers’analysis of student thinking, indicate ...
Studies in Educational Evaluation, 2020
Machine learning has been frequently employed to automatically score constructed response assessm... more Machine learning has been frequently employed to automatically score constructed response assessments. However, there is a lack of evidence of how this predictive scoring approach might be compromised by construct-irrelevant variance (CIV), which is a threat to test validity. In this study, we evaluated machine scores and human scores with regard to potential CIV. We developed two assessment tasks targeting science teacher pedagogical content knowledge (PCK); each task contains three video-based constructed response questions. 187 inservice science teachers watched the videos with each had a given classroom teaching scenario and then responded to the constructed-response items. Three human experts rated the responses and the human-consent scores were used to develop machine learning algorithms to predict ratings of the responses. Including the machine as another independent rater, along with the three human raters, we employed the many-facet Rasch measurement model to examine CIV due to three sources: variability of scenarios, rater severity, and rater sensitivity of the scenarios. Results indicate that variability of scenarios impacts teachers' performance, but the impact significantly depends on the construct of interest; for each assessment task, the machine is always the most severe rater, compared to the three human raters. However, the machine is less sensitive than the human raters to the task scenarios. This means the machine scoring is more consistent and stable across scenarios within each of the two tasks.
Science & Education, 2020
Conflation of sex and gender is implicated in the development of essentialist thinking, which has... more Conflation of sex and gender is implicated in the development of essentialist thinking, which has been linked to the justification of systems of prejudice in modern society. This exploratory study presents findings from a person randomized control trial conducted with 460 students in 8th-10th grade that investigated the extent to which students conflate sex and gender in their writing about genetics. Students were randomly assigned to one of three short readings that either (1) explained the genetics of sex in plants; (2) explained the genetics of sex in humans; or (3) refuted neuro-genetic essentialism, offering instead a social explanation for why women receive fewer PhDs in science, technology, mathematics, and engineering than men. While previous findings from the authors suggest links between the condition students were assigned to and psychological indicators related to essentialist thinking, no work was done to investigate how students' use of language might implicate cognitive conflation as a possible factor in understanding these results. In this study, student responses to a constructed response writing task given after the reading were analyzed to investigate the use of sex and gender language. Students in all three conditions used both sex and gender language. However, students in the refutational text condition tended to use sex and gender language deliberately in order to explain PhD attainment, while students in the traditional genetics conditions used the terms interchangeably, suggesting subconscious conflation. Students in the genetics of human sex condition were more likely to manifest this conflation than students in the genetics of plant sex condition. Implications for instruction are discussed.
Journal of Research in Science Teaching, 2020
Recently, it has been argued that improving students' genomics literacy could prevent student... more Recently, it has been argued that improving students' genomics literacy could prevent students from developing erroneous beliefs about social identity, such as the belief that racial groups differ cognitively and behaviorally because of their genes; a belief called genetic essentialism. To date, however, little research has explored if or how a conceptual understanding of genomics protects against the development of genetic essentialism. Using a randomized control trial (RCT) (N = 721, 9th–12th graders), we explore if students with more genomics literacy are more able to conceptually change their genetic essentialist beliefs after engaging in a learning experience designed to refute essentialist thinking. The results of the RCT demonstrated that students with higher genomics literacy (relative to those with lower genomics literacy) exhibited greater reductions in the perception of racial differences and greater reductions in belief in genetic essentialism after learning how patt...
AERA Open, 2018
In the past two decades, there has been extensive focus on how to calculate power for cluster-ran... more In the past two decades, there has been extensive focus on how to calculate power for cluster-randomized trials, or CRTs (e.g.
Science Education, 2019
When people are exposed to information that leads them to overestimate the actual amount of genet... more When people are exposed to information that leads them to overestimate the actual amount of genetic difference between racial groups, it can augment their racial biases. However, there is apparently no research that explores if the reverse is possible. Does teaching adolescents scientifically accurate information about genetic variation within and between US census races reduce their racial biases? We randomized 8th and 9th grade students (n = 166) into separate classrooms to learn for an entire week either about the topics of (a) human genetic variation or (b) climate variation. In a cross‐over randomized trial with clustering, we demonstrate that when students learn about genetic variation within and between racial groups it significantly changes their perceptions of human genetic variation, thereby causing a significant decrease in their scores on instruments assessing cognitive forms of prejudice. We then replicate these findings in two computer‐based randomized controlled trial...
Uploads
Papers by Molly Stuhlsatz