Mixed Methods Design

Council for Research in Music Education First-Generation Mixed Methods Designs in Music Education Research: Establishing an Initial Schematic Author(s): Chad West Source: Bulletin of the Council for Research in Music Education, No. 199 (WINTER 2014), pp. 53-67 Published by: University of Illinois Press on behalf of the Council for Research in Music Education Stable URL: http://www.jstor.org/stable/10.5406/bulcouresmusedu.199.0053 . Accessed: 06/10/2014 16:26 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. . University of Illinois Press and Council for Research in Music Education are collaborating with JSTOR to digitize, preserve and extend access to Bulletin of the Council for Research in Music Education. http://www.jstor.org This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 © 2014 Board of Trustees University of Illinois First-Generation Mixed Methods Designs in Music Education Research: Establishing an Initial Schematic Chad West Ithaca College Ithaca, NY Music education research is rooted in long traditions of quantitative and, more recently, qualitative research techniques; both of these paradigms have well-established research techniques that are well understood in the profession. Less understood are so-called mixed methods designs that combine techniques and indings from both paradigms (Leech, Dellinger, Brannagan, & Tanaka, 2010). Since the profession has relatively few examples of published mixed methods designs, researchers are left to work mainly with textbooks that may lead to only vague and supericial understandings of how to implement a mixed methods study (Bergman, 2011). Researchers need practical examples illustrating ways mixed methods studies have been used. Creswell (2009a) argues, We need a current map of the ield of mixed methods to help authors who are submitting manuscripts. . . . Scholars looking to write a manuscript that contributes to the mixed methods literature may want to start by examining this map and considering where their work may contribute to our current understandings of the ield. (p. 105) While scholars in varying disciplines such as social sciences (Alise & Teddlie, 2010; Bryman, 2007), health sciences (Collins, Onwuegbuzie, & Jiao, 2007) and math education (Hart, Smith, Swars, & Smith, 2009) have analyzed examples of published mixed methods studies, no such analysis has been published within music education. herefore, the purpose of this article was to identify and describe the methodological nuances used by what could be considered the irst generation of American music education researchers who used the term mixed methods to describe their dissertations, thus establishing an initial schematic from which music education researchers could draw to further conversation within the profession regarding mixed methods research design. he irst part of this article reviews the philosophical underpinnings and historical development of mixed methods research. he second part examines music education 53 BCRME_199 text.indd 53 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 mixed methods dissertations and discusses them in the context of what scholars have written about mixed methods research techniques with particular focus on design features including the elements of timing, weighting, and mixing. he article concludes with suggestions for music education researchers interested in conducting future mixed methods studies. PHILOSOPHICAL UNDERPINNINGS AND HISTORICAL DEVELOPMENT Onwuegbuzie (2000) provided an account of the epistemological underpinnings of mixed methods research. He explained how logical positivism (i.e., objective veriication of systematic data collection using statistical procedures to explain, predict, and control phenomena) dominated scientiic philosophy up until the late nineteenth century. Positivists of this era argued that social phenomena existed independent of the observer’s values, and thus generalizations could be made regardless of time and context. Not until the turn of the twentieth century did social scientists begin questioning the exclusive use of the scientiic method in understanding social issues; interpretivists argued that generalizing the complex processes and products of the human mind was unattainable. In the 1950s and 1960s post-positivist researchers conceded that multiple realities exist in the social sciences and that research is inluenced by the values of the investigators, but persisted in their belief that some universal relationships among social phenomena could still be identiied.1 It was during this time that the social sciences became increasingly receptive to using multiple (quantitative) methods with ofsetting strengths and weaknesses to examine the same phenomenon. According to several accounts (Creswell, 2009b; Creswell & Plano Clark, 2007; Greene, 2007; Onwuegbuzie, 2000) it was Campbell and Fiske (1959) who irst introduced the idea of using multiple quantitative research methods as a source of triangulation. In the 1960s and 1970s researchers began combining surveys and interviews and, inally, qualitative and quantitative techniques. From about the mid-1980s to the mid-1990s researchers began discussing whether qualitative and quantitative data could be combined (Creswell & Plano Clark, 2007). Prior to this time, many qualitative researchers maintained that the two approaches assumed diferent worldviews and were thus incompatible. Rossman and Wilson (1985) referred to those who believed the two methods to be incompatible as purists, those who adapted their methods to the situation as situationalists, and those who believed that research problems could be addressed using a variety of lenses as pragmatists. Pragmatic researchers (e.g., Bryman, 1988; Reichardt & Rallis, 1994) challenged the notion of incompatibility, suggested connections between the two paradigms, and urged researchers to move beyond the paradigm debate. In the 1980s and 1990s researchers began deining types of mixed methods systems and describing procedures for conducting mixed methods research; elaboration of these classiications continues today. 54 BCRME_199 text.indd 54 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research MIXED METHODS DESIGNS F O U N D W I T H I N M U S I C E D U C AT I O N D I S S E R T A T I O N S ( 2 0 01 – 2 011) In the early spring of 2012, the author searched ProQuest Dissertations and heses using the terms “mixed methods” and “music” within abstracts and keywords. he author then reviewed all of the returns to identify only those that were actually conducted within the ield of music education. his resulted in 24 self-described music education mixed methods dissertations from 2001–2011. It is likely that other studies also combined qualitative and quantitative techniques, but did not include these terms as described and thus were not included in these igures. his section of the article examines these dissertations through the lens of what scholars have written about mixed methods research techniques. While many have contributed to the emerging mixed methods design classiications in the past two decades, the music education dissertations presented in this article are examined in relation to the writings of Creswell and Plano Clark (2007) since these authors are among the most widely cited in the profession. Creswell and Plano Clark’s Designing and Conducting Mixed Methods Research (2011) is now in the 2nd edition; however, the 2007 version is used in this analysis since the dissertation authors examined in this article would not have had access to the 2011 edition. Creswell and Plano Clark (2007) suggest four instances when a mixed methods design might be an appropriate approach for addressing a research problem: (a) when a need exists for both qualitative and quantitative approaches (triangulation design); (b) when a need exists to enhance the study with a second source of data (embedded design); (c) when a need exists to explain the quantitative results (explanatory design); and (d) when a need exists to irst explore a phenomenon qualitatively (exploratory design). Along with determining the mixed method design (i.e., triangulation, embedded, explanatory, or exploratory) the mixed methods researcher must determine (a) in what order the data will be collected and organized (timing), (b) how much emphasis will be given to each method (weighting), and (c) how the data will be merged and combined in the inal analysis and discussion (mixing). In the current article, the author examined the mixed methods dissertations within music education, compared the researchers’ classiications with those of Creswell & Plano Clark, and highlighted instances of ambiguity. Table 1 illustrates the ways mixed methods designs have been used in music education, including weighting, timing, and mixing of data. Triangulation Design When a need exists to examine both qualitative and quantitative data as a means of adding validity to a study, a researcher might use a triangulation design. he purpose of this oldest, most common, and well-known mixed methods design (Creswell & Plano 55 BCRME_199 text.indd 55 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions BCRME_199 text.indd 56 Table 1 Music Education Mixed Methods Matrix Mixed-Methods Research Design Triangulation Embedded he “Gray Areas” Explanatory Exploratory Lack of Rigor Design Components Content Analysis MultiMethod Worldviews not Matching Methods Timing This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions C Rhee (2001) Huang (2004) Ferguson (2005) Rodriguez (2005) Danserau (2005) Bazan (2007) Lee (2007) homas (2007) Fitzpatrick (2008) Scruggs (2008) Hendricks (2009) Menard (2009) Wexler (2009) Wheelhouse (2009) Wilson (2010) Neokleous (2010) Baer (2010) Bazzy (2010) Gavin (2010) Legutki (2010) Stringham (2010) Mozeiko (2011) Cushinery (2011) West (2011) (x) X X (x) X X X S Weighting QUAL X X Mixing QUAN C E I X X X X X X X X X X X X X X X X X X X X X (x) X X X X X X X X X X X X (x) X X X X X X X X X X X X X X X X X (x) (x) X X (x) X X X X X X (x) (x) X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 8/22/14 2:36 PM Note. For the element of “Timing:” “C” indicates that both forms of data were collected concurrently, and “S” indicates that data were collected sequentially. Note. For the element of “Weighting:” “QUAL” indicates that more weight was given to the qualitative data, and “QUAN” indicates that more weight was given to the quantitative data. Note. For the element of “Mixing:” “C” indicates one form of data was connected to the other, “E” indicates one form of data was embedded in the other, and “I” indicates both forms of data were mixed in the interpretation. Note. (x) denotes an approximation of the element listed. An author’s design is considered to be an approximation of one of Creswell and Plano Clark’s designs when it contains some, but not all of the elements outlined, shares characteristics with another design, or otherwise diverges from the characteristics outlined by Creswell and Plano Clark. Instances of these approximations are discussed within the text. X X X X X X X X X West Mixed Methods Designs in Music Education Research Clark, 2007) is “to obtain diferent but complementary data on the same topic” (Morse, 1991, p. 122). Creswell and Plano Clark (2007) explain, he combination of qualitative and quantitative data provides a more complete picture by noting trends and generalizations as well as in-depth knowledge of participants’ perspectives. . . . One type of evidence may not tell the complete story, or the researcher may lack conidence in the ability of one type of evidence to address the problem. Further, the type of evidence gathered from one level in an organization might difer from evidence looked at from other levels. (p. 33) With this design, researchers typically implement the data collection phase of both methods concurrently and merge the two datasets in the analysis phase or interpretation section. he triangulation design is the most common mixed method design found to date in music education dissertations. Danserau (2005) collected both qualitative and quantitative data to investigate the nature and development of musicality among three-year old children. Two years later homas (2007) used a triangulated design to analyze team teaching in a large choral performance setting, and Lee (2007) converged quantitative and qualitative data collection and analysis techniques to examine two diferent beginning cello methods. Other triangulation designs include Fitzpatrick (2008), who converged indings to explore ways that instrumental music teachers operate within urban settings; Hendricks (2009), who observed the inluence of competitive musical environments upon students’ self-belief structures; Menard (2009), who explored students’ attitudes toward and abilities in music composition; Bazzy (2010), who studied boys’ group singing participation in a single-sex setting; and Neokleous (2010), who examined the singing skills of preservice kindergarten teachers. All of these researchers collected data concurrently, weighted the methods equally, and mixed the indings through discussion. Embedded Design Sometimes a need exists to enhance the study with a second source of data. In this instance, a researcher may choose to implement what Creswell and Plano Clark (2007) refer to as an embedded design. he purpose of an embedded design is to address questions that cannot be answered by a single dataset. For instance, if the major qualitative research problem includes subproblems that are best addressed through quantitative approaches, a researcher may embed a quantitative question in an otherwise qualitative study. Conversely, qualitative subquestions can also be embedded in larger quantitative studies. Creswell and Plano Clark (2007) explain “the investigator includes qualitative data for several reasons, such as to develop a treatment, to examine the process of an intervention or the mechanisms that relate variables, or to follow up on the results of an experiment” (p. 67). Since the data are used to answer diferent research questions within the study, data collection can be completed either sequentially or concurrently. Unlike the triangulation design, the intent is not to 57 BCRME_199 text.indd 57 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 merge the two datasets to answer the same research question, but rather to “keep the two sets of results separate in their reports” (p. 71). Within music education, Scruggs (2009) compared the learning outcomes of a learner-centered environment with those of a more teacher-centered classroom culture. he researcher used quantitative instruments to measure student attitudes and student performance and used qualitative techniques to uncover student perceptions of the classroom environment. Two years later, Baer (2010) studied the efect of multiple intelligence instructional strategies on sight-singing achievement as well as the efect of sight-singing ability on overall audition scores for a select chorus. While the majority of the study addressed quantitative questions, the researcher also embedded qualitative techniques (observations and interviews) to identify the techniques that students used when performing the sight-singing component of the audition. hat same year, Stringham (2010) examined music achievement and described personal perspectives of high school students who learned to improvise and compose using a sequential music curriculum in a non-auditioned wind and percussion ensemble. he researcher used quantitative instruments to reveal relationships between student aptitude and achievement in composition and improvisation, and used qualitative techniques to identify student perceptions about improvisation and composition and to identify teaching techniques. Last, Mozeiko (2011) used an experimental design to detect the efect of a treatment on violinists’ performance and case studies to determine the participants’ experience of the treatment. While Scruggs, Baer, and Stringham collected data concurrently and reported their indings according to the research question, Mozeiko collected her data sequentially and mixed her indings through discussion. Explanatory Design Explanatory designs can be implemented when a researcher seeks to build upon, explain, or enhance previously collected quantitative results. In the irst phase of collection, the researcher collects and analyzes the quantitative data. In the second phase, the researcher collects qualitative data that follows up on the initial quantitative results. Morse (1991) explains “this design is well suited to a study in which a researcher needs qualitative data to explain signiicant (or nonsigniicant) results, outlier results, or surprising results” (p. 72). In explanatory designs, the data is mixed at the design stage in that the quantitative data informs the qualitative method, and qualitative indings are presented as the end product. Within music education, Wheelhouse (2009) examined minority student participation in public school music programs in Minority Student Achievement Network (MASN) districts. he researcher irst created and administered a survey to examine the music programs in such districts and then conducted follow-up interviews of selected participants to further explain the indings from the survey. Wilson (2010) investigated the attitudes of high school choral students and their directors regarding the relative 58 BCRME_199 text.indd 58 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research value placed on treble clef choirs versus mixed choirs to determine whether female singers were devalued in choral programs. After administering a survey to collect a breadth of information, the researcher conducted factor analysis to develop qualitative questions and explore them through observations and interviews. Both Wheelhouse and Wilson collected data sequentially, connecting their quantitative indings to their qualitative design and mixed their indings through discussion. Cushinery (2011) examined both external and internal factors (quantitative) leading to retention of six music teachers and the interaction that exists between those factors (qualitative). Data collection consisted of irst surveying 260 music teachers and then qualitatively studying six participants. While the author described the study as an “embedded mixed-methods design,” the design might be considered an explanatory design since “the quantitative data informed the data collection [of the qualitative] portion of this study” (pp. 38, 39). he study indeed has characteristics of an embedded design, as the author describes, in that a qualitative research question is embedded among two other quantitative questions; the quandary here seems to be whether it is the nature of the research questions or the role of the indings that determines the design classiication. Creswell and Plano Clark might suggest that since the role of Cushinery’s quantitative indings was to inform the qualitative data collection, the intent of the study was to explain, and thus could be considered primarily an explanatory mixed methods design. Bazan (2007) explored teaching strategies of middle school band directors who reported a student-directed teaching style. To identify the most student-directed teachers, Bazan irst gathered quantitative data using a researcher-designed demographic questionnaire and Gumm’s Music Teaching Style Inventory (MTSI). hen, the researcher observed, videotaped, and interviewed the three participants identiied by the MTSI in the irst stage as being the most student-directed. Similar in design to Bazan was Gavin (2010) who examined persistence to degree completion of undergraduate music education students at a single university. he researcher used secondary quantitative data from the registrar to determine the withdrawal rates and patterns of withdrawal, and to purposefully select participants for the qualitative portion of his study. He then interviewed the participants regarding their reasons for attrition. While the quantitative data in both studies (Bazan, 2007, and Gavin, 2010) were used to inform the qualitative design, as is characteristic of explanatory designs, they might also be categorized as embedded designs since both researchers’ reasons for collecting quantitative data were to identify participants for the qualitative portion of the study, rather than to use qualitative data to explain the initial quantitative results. Legutki (2010) surveyed 274 respondents regarding their motivation in high school band. he researcher used the survey results to both explain motivation in the high school band and to purposely select outliers to be interviewed regarding their motivational beliefs for the qualitative phase of the study. While data were collected sequentially, were interactive at the design stage, and thus had elements of an explanatory design, the impetus for collecting both forms was convergence: “he questionnaires and 59 BCRME_199 text.indd 59 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 interviews were used together to elaborate, enhance, deepen, and broaden the overall interpretations and inferences, which might not be possible to accomplish through only one of the methods” (p. 42) and thus also contain elements of a triangulation design. Exploratory Design Finally, Creswell and Plano Clark (2007) explain that exploratory designs are useful when “measures or instruments are not available, the variables are unknown, or there is no guiding framework or theory” (p. 75). Since the intent of the exploratory design is that the qualitative results help develop the quantitative method, data collection is conducted in two phases; irst, the phenomenon is explored qualitatively and from its analysis, the researcher forms quantitative questions. In exploratory designs, the data are mixed at the design stage where the qualitative indings inform the quantitative method, and quantitative indings are presented as the end result. An example of such a design is Wexler (2009) who explored college instrumental studio teachers’ shared goals, values, and teaching strategies. he researcher irst interviewed participants and developed qualitative themes that he then used to form survey questions to administer to a larger population. Since the qualitative data were already mixed at the design stage, Wexler only reported the quantitative results as the end product. West’s (2011) dissertation approximates the exploratory design. he irst phase of his study was a qualitative exploration of middle school jazz education for which observation, ield note, interview, and artifact data were collected. hemes generated from the qualitative portion regarding participants’ perceived ability to teach middle school jazz and current thoughts about and teaching of middle school jazz informed the development of survey items used to collect data from a larger population of middle school music teachers. Whereas Wexler reported only the quantitative indings, as is standard in exploratory designs, West also mixed data from both phases in the inal analysis and presented them through discussion; thus, West’s study most closely resembles the exploratory design, but includes elements of the triangulation design as well. DISCUSSION hough many of the studies presented in this article do not explicitly identify their categorization, reviewing the method and purpose for collecting and mixing the data reveals elements of their designs, and inferences can be made about how they might be classiied. Most music education studies (eight) have been triangulation designs, followed by embedded designs (six), explanatory designs (three), and exploratory designs (two), although many of the studies examined here also include elements of other designs. his is not surprising since a reciprocal evolutionary process between design and typology exists; that is, theory compels practice and practice compels further theory. Since mixed methods as a “third methodological movement” (Tashakkori & Teddlie, 2003) is still evolving, it is diicult to pinpoint exactly what is and what is not 60 BCRME_199 text.indd 60 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research mixed methods research. Sometimes studies conform to part, but not all, of Creswell and Plano Clark’s (2007) deinition, and thus fall into “gray areas” such as when a study lacks rigor in one method or the other. Studies that only collect a few open-ended questions as part of the survey rather than the rich contextualized data, prolonged engagement in the ield, triangulation of sources, observation, artifact data, and other elements associated with rigorous qualitative research, might be considered to fall within one of these gray areas. Examples of such would be Ferguson (2005), who collected quantitative data from only three sites out of a population of over 100, making generalization (as the author admits) of the quantitative results impossible, and Lee’s (2007) quantitative sampling of 10. However, while Creswell and Plano Clark would consider these designs to be gray areas of mixed methods research, this sentiment is not universal among mixed methods writers; Morse and Niehaus (2009) contend that lack of rigor in one method is exactly what makes the study mixed methods rather than two separate studies: he deining characteristic of mixed methods research is that it involves a primary or core method combined with one or more strategies drawn from a second, different method for addressing the research question by either collecting or analyzing data. he supplemental component is partially complete [emphasis mine] and not conducted rigorously enough to stand alone or to be published by itself. (p. 14) Morse and Niehaus seem to be talking here about the topic of weighting by suggesting that one data set is supplementary to the other; what makes their argument a bit diferent is that they are discussing weighting in the context of rigor, whereas most other writings within the mixed methods literature discuss weighting in the context of scope. Certainly, one form of data often plays a larger role in the study than the other, as Creswell and Plano Clark (and most other mixed methods authors) suggest, but music education mixed methods researchers and music education editorial review board members must determine whether they also believe that the supplemental form of data should be incomplete or less rigorous than the other, as Morse and Niehaus seem to suggest. Another type of study that Creswell and Plano Clark (2007) consider to be a gray area is content analysis. Gavin’s (2010) study might be considered such in that he analyzed secondary data supplied through a university registrar’s oice. While both types of data were analyzed, both typed of data were not collected, thus Creswell and Plano Clark would consider this study to fall into a gray area of mixed methods research (2007, p. 12). he so-called multimethod research (Morse, 1991), such as when Campbell and Fiske (1959) used multiple quantitative techniques to triangulate indings, would also be considered by Creswell and Plano Clark to be a gray area. Within music education, Rodriguez (2005) used only qualitative techniques (i.e., personal narrative, documented observations, and interviews) to explore the transmission of Canary Islands folk music. While Rodriguez identiied her study as a “qualitative mixed method case study” (p. ii), Creswell and Plano-Clark would classify such a study as a multimethod design since it only utilized qualitative data. 61 BCRME_199 text.indd 61 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 Last, research utilizing a method to collect and analyze data associated with a diferent paradigm would fall into Creswell and Plano Clark’s (2007) gray areas of mixed methods research. An example of this would be where a researcher utilizes a standardized instrument to study participants’ lived experiences. he same would apply if a researcher used ethnographic techniques to study causal relationships to a broad population. Instances of worldview not matching methods can be seen in certain music education mixed methods dissertations such as Huang (2004) who used an open-ended questionnaire to collect demographic information, and Rhee (2001) who asked closedended questions in the qualitative portion of the study. Similarly, Bazan’s (2007) last two research questions asked for closed-ended information (what, how many, what percentage), but were collected qualitatively through video, ield notes, and participant interviews. CONCLUSION he purpose of this article was to identify and describe the methodological nuances used by what could be considered the irst generation of American music education researchers who used the term mixed methods to describe their dissertations, thus establishing an initial schematic from which music education researchers could draw to further conversation within the profession regarding mixed methods research design. It can be argued, however, that examining completed research through the lens of typology is problematic in itself. Such an endeavor could inadvertently reify methodological constructs when the purpose of mixed methods research, rooted in pragmatism, is to utilize any and all methods that would efectively address the research problem. If we are to consider that the research problem should determine the method, then categorizing researchers’ procedures according to any a priori schematic may capture neither the author’s intent nor the complexities of the study. Further, pragmatic researchers relying on typologies to design their research may ind that their choices are prematurely constrained (Bergman, 2011); it may be that a need for a mixed methodology becomes apparent only upon collection and analysis of the irst form of data. Typologies should serve to elucidate completed research rather than to reify theoretical constructs, and it is important that readers keep this in mind when interpreting the studies presented in this article. Still, Natasi, Hitchcock, and Brown (2010) argue that the “continued wrestling with mixed methods typologies [is] a useful endeavor, even if the issue remains problematic” (p. 306). his article examined self-reported mixed methods dissertations within music education. here are, however, a few mixed methods articles also published in music education journals (e.g., Heinrich, 2012; Ho & Law, 2009; Ivaldi & O’Neill, 2008; King, 2008; Rowe, 2009). hese articles are all self-described as using a mixed methods design and all are published in journals housed outside of the United States. As of yet, mixed methods articles within music education journals in general, and American music 62 BCRME_199 text.indd 62 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research education journals in particular, are relatively scarce when compared to the number of the mixed methods dissertations that have been conducted. I do not presume to know whether this is a result of such dissertations not being submitted, seasoned researchers not conducting mixed methods research, or review boards being reluctant to accept mixed methods research. Certainly, the very nature of searching for articles that are self-described as “mixed methods” may be problematic. Guest (2013) found that the PubMed database in 2000 contained no articles using the term, “mixed methods” in the title, but by 2010, this number had grown to 103 (p. 144), suggesting that, “researchers in various disciplines were integrating qualitative and quantitative methods long before the ield of mixed methods formally emerged and typologies were established. Similarly, within music education, some authors do not use the term “mixed methods” to describe their research, such as Vitale (2011) who described the study as “multiple-methods.” Other researchers omit the term completely and simply indicate that they collected and analyzed both qualitative and quantitative data (e.g., Butler, 2001; Hendel, 1995; Spohn, 2008; Teixeira & Del-Ben, 2010; Welch, Purves, Hargreaves, & Marshall, 2011; Wrigley & Emmerson, 2013). Sometimes mixed methods articles can be identiied only by closely reading the method to determine that the researcher actually utilized a mixed method approach (e.g., Clark, 2013; Johnson, Price, & Schroeder, 2009). Some mixed methods articles on topics relevant to music education are published outside of music education journals such as Burt-Perkins and Mills (2009) and Yim and Ebbeck (2009), all of whom published in general education journals. Last, it appears that some authors may conduct a mixed methods study and then report only one type of data in its publication, such as St. George, Holbrook, and Cantwell (2012) and Lau and Grieshaber (2010), who both reported only the qualitative indings. While teams of researchers consisting of both qualitative and quantitative experts often partner in ields such as medicine and other physical sciences to conduct mixed methods research, music education researchers thus far have worked alone. his is not surprising since tenure and promotion structures within our profession are generally such that coauthorship is discouraged. Critics of single-author mixed methods research contend that high degrees of competency in both methods are unlikely to exist within a single researcher. his criticism also raises questions about the implications for graduate research education. Tashakkori (2009) indicated that this is emerging as a major concern within the mixed methods community, arguing that there is often ambiguity in which courses and skills are needed before a student takes a mixed methods course. Further, students are often frustrated by the inconsistent presentation of mixed methods concepts and designs ofered by diferent textbooks and scholars. In light of the review of these 24 dissertations, I ofer a few suggestions for conducting future mixed methods research. First, it is important to identify, describe, and cite the overall design and speciic procedures; the past ten years has ushered in a plethora of literature written on mixed methods methodology, yet only a few of the dissertation authors 63 BCRME_199 text.indd 63 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 reviewed in this article situated their studies within such literature. Second, considering that much mixed methods research is grounded in pragmatism, researchers should take care to not allow typological constructs to dictate the best possible approach for exploring a problem. Instead, researchers should simply cite the available literature and describe how the current method difers and why. hese studies will then serve as counter examples that illustrate new ways of utilizing mixed methods research. Bergman (2011) explains: here exist enough successful applications of research methods that do not follow the contemporary ideologies and conventions. he counter examples may help formulate a better understanding about these groups of methods—if they still can be grouped in this way. With this, it is time to bring in a second generation of theoretical considerations about the shape and reasons for mixed methods research. (p. 101) hird, mixed methods researchers within music education should explore creative ways of representing the mixed indings so that meaningful conclusions can be drawn without betraying the philosophical underpinnings of either form of data. And last, just as qualitative researchers challenged the research community throughout the 1980s and 1990s, so should mixed methods researchers persist today. NOTES 1. Here, “post-positivism” refers to a “softened” version of positivism that addresses some postmodern criticisms, but preserves the basic assumptions of positivism (i.e., the possibility of objective truth and the use of the experimental method). REFERENCES Alise, M. A., & Teddlie, C. (2010). A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. Journal of Mixed Methods Research, 4(2), 103–126. Baer, J. (2010). he relationship of multiple intelligence instruction to sight singing achievement of middle school choral students (Unpublished doctoral dissertation). Walden University, Minneapolis, MN. Bazan, D. E. (2007). Teaching and learning strategies used by student-directed teachers of middle school band. Dissertation Abstracts International, 68(5). Bazzy, Z. (2010). Upper elementary boys’ participation during group singing activities in single-sex and coeducational classes (Unpublished doctoral dissertation). University of South Florida, Tampa, FL. Bergman, M. M. (2011). he politics, fashions, and conventions of research methods. Journal of Mixed Methods Research, 5(2), 99–102. Bryman, A. (1988). Quantity and quality in social research. London: Routledge. Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8–22. Burt-Perkins, R., & Mills, J. (2009). Pitching it right? Selection and learning at a music conservatoire. British Educational Research Journal, 35(6), 817–835. Butler, A. (2001). Preservice music teachers’ conceptions of teaching efectiveness, microteaching experiences, and teaching performance. Journal of Research in Music Education, 49(3), 258–72. 64 BCRME_199 text.indd 64 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitraitmultimethod matrix. Psychological Bulletin, 56, 81–105. Clark, J. (2013). A qualitative exploration of higher self-eicacy string students preparing for a competition. International Journal of Music Education, 31(1), 4–14. Collins, K., Onwuegbuzie, A. J., & Jiao, Q. G. (2007). A mixed methods investigation of mixed methods sampling designs in social and health science research. Journal of Mixed Methods Research, 1(3), 267–294. Creswell, J. W. (2009a). Editorial: Mapping the ield of mixed methods research. Journal of Mixed Methods Research, 3(2), 95–108. Creswell, J. W. (2009b). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). housand Oaks, CA: Sage. Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. housand Oaks, CA: Sage. Cushinery, C. (2011). Factors inluencing music teacher retention: A mixed methods study (Unpublished doctoral dissertation). University of Nevada, Las Vegas, NV. Dansereau, D. R. (2005). he musicality of 3-year-old children within the context of research-based musical engagement (Unpublished doctoral dissertation). Georgia State University, Atlanta, GA. Ferguson, D. A. (2005). Music education in Edison Schools: An evaluation of K-5 music education in three schools managed by Edison Schools, Inc. Dissertation Abstracts International, 66(7), 2524. Fitzpatrick, K. R. (2008). A mixed methods portrait of urban instrumental music teaching. Dissertation Abstracts International, 69(3). Gavin, R. (2010). An exploration of potential factors afecting persistence to degree completion in undergraduate music teacher education students (Unpublished doctoral dissertation). Florida State University, Gainesville, FL. Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass. Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods Research, 7(2), 141–151. Hart, L. C., Smith, S. Z., Swars, S. L., & Smith, M. E. (2009). An examination of research methods in mathematics education (1995–2005). Journal of Mixed Methods Research, 3(1), 26–41. Heinrich, J. (2012). he provision of classroom music programs to regional Victorian primary schools. Australian Journal of Music Education, 2, 45–58. Hendel, C. (1995). Behavioral characteristics and instructional patterns of selected music teachers. Journal of Research in Music Education, 43(3), 182–203. Hendricks, K. (2009). Relationships between the sources of self-eicacy and changes in competence perceptions of music students during an all-state orchestra event (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, IL. Ho, W., & Law, W. (2009). Sociopolitical culture and school music education in Hong Kong. British Journal of Music Education, 26(1), 71–84. Huang, H. J. (2004). A study of the relationship between music learning and school achievement of sixth-grade students (Doctoral dissertation). Dissertation Abstracts International, 65(2), 338. Ivaldi, A., & O’Neill, S. A. (2008). Adolescents’ musical role models: Whom do they admire and why? Psychology of Music, 36(4), 395–415. Johnson, C. M., Price, H. E., & Schroeder, L. K. (2009). Teaching evaluations and comments of preservice music teachers regarding expert and novice choral conductors. International Journal of Music Education, 27(1), 7–18. King, A. (2008). Collaborative learning in the music studio. Music Education Research, 10(3), 423–438. 65 BCRME_199 text.indd 65 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions Bulletin of the Council for Research in Music Education Winter 2014 No. 199 Lau, W., & Grieshaber, S. (2010). Musical free play: A case for invented musical notation in a Hong Kong kindergarten. British Journal of Music Education, 27(2), 127–140. Lee, A. J. (2007). Two non-traditional cello methods for young beginning cello students: A mixed methods study. Dissertation Abstracts International, 69(1). Leech, N. L., Dellinger, A. B., Brannagan, K. B., & Tanaka, H. (2010). Evaluating mixed research studies: A mixed methods approach. Journal of Mixed Methods Research, 4(1), 17–31. Legutki, A. (2010). Self-determined music participation: he role of psychological needs satisfaction, intrinsic motivation, and self-regulation in the high school band experience (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign, IL. Menard, E. (2009). An investigation of creative potential in high school musicians: Recognizing, promoting, and assessing creative ability through music composition (Unpublished doctoral dissertation). Louisiana State University and Agricultural and Mechanical College, Baton Rouge, LA. Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40(2), 120–123. Morse, J. M., & Niehaus (2009). Mixed methods design: Principles and procedures. Walnut Creek, CA: Left Coast Press. Mozeiko, K. (2011). he efects of participation in the Alexander Technique on female violinists and violists: A mixed-methods study (Unpublished D.M.A.). Boston University, Boston, MA. Natasi, B., Hitchcock, J., & Brown, L. (2010). An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 305–338). housand Oaks, CA. Sage. Neokleous, R. (2010). Tracking preservice kindergarten teachers’ development of singing skills and conidence: An applied study (Unpublished D.M.A.). Boston University, Boston, MA. Onwuegbuzie, A. J. (2000, November). Positivists, post-positivists, post-structuralists, and postmodernists: Why can’t we all get along? Towards a framework for unifying research paradigms. Paper presented at the Annual Meeting of the Association for the Advancement of Educational Research, Ponte Vedra, FL. Reichardt, C. S., & Rallis, S. F. (Eds.). (1994). he qualitative quantitative debate: New perspectives. San Francisco, CA: Jossey-Bass. Rhee, E. (2001). he incorporation of technology into music education in Korea: A mixed method study. Dissertation Abstracts International, 62(11), 3620. Rodriguez, S. E. (2005). Transmission of traditional musics and music teaching and learning in the Canary Islands: A perspective. Dissertation Abstracts International, 66(10). Rowe, V. (2009). Using video-stimulated recall as a basis for interviews: Some experiences from the ield. Music Education Research, 11(4), 425–437. Scruggs, B. (2009). Learning outcomes in two divergent middle school string orchestra classroom environments: A comparison of a learner-centered and a teacher-centered approach (Unpublished doctoral dissertation). Georgia State University, Atlanta, GA. Spohn, C. (2008). Teacher perspectives on No Child Left Behind and arts education: A case study. Arts Education Policy Review, 109(4), 3–11. St. George, J., Holbrook, A., & Cantwell, R. H. (2012). Learning patterns in music practice: Links between disposition, practice strategies and outcomes. Music Education Research, 14(2), 243– 263. Stringham, D. (2010). Improvisation and composition in a high school instrumental music curriculum (Unpublished doctoral dissertation). University of Rochester, Eastman School of Music, Rochester, NY. 66 BCRME_199 text.indd 66 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions West Mixed Methods Designs in Music Education Research Tashakkori, A. (2009). Are we there yet?: he state of the mixed methods community. Journal of Mixed Methods Research, 3(4), 287–291. Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social & behavioral research. housand Oaks, CA: Sage. Teixeira Dos Santos, R., & Del-Ben, L. (2010). Quantitative and qualitative assessment of solfege in a Brazilian higher educational context. International Journal of Music Education, 28(1), 31–46. homas, M. P. (2007). Efects of team teaching in the massed secondary choral setting. Dissertation Abstracts International, 68(3). Vitale, J. (2011). Music makes you smarter: A new paradigm for music education? Perceptions and perspectives from four groups of elementary education stakeholders. Canadian Journal of Education, 34(3), 317–343. Welch, G. G., Purves, R. R., Hargreaves, D. D., & Marshall, N. N. (2011). Early career challenges in secondary school music teaching. British Educational Research Journal, 37(2), 285–315. West, C. (2011). Teaching middle school jazz: An exploratory sequential mixed methods study (Unpublished doctoral dissertation). University of Michigan, Ann Arbor, MI. Wexler, M. (2009). A comparative survey of goals and strategies of college music performance teachers across instrumental groups (Unpublished doctoral dissertation). Teachers College, Columbia University, New York, NY. Wheelhouse, P. (2009). A survey of minority student participation in music programs of the minority student achievement network (Unpublished doctoral dissertation). University of Rochester, Eastman School of Music, Rochester, NY. Wilson, J. (2010). Practices of and attitudes toward high school treble clef choral ensembles. (Unpublished D.M.A.). Boston University, Boston, MA. Wrigley, W. J., & Emmerson, S. B. (2013). Ecological development and validation of a music performance rating scale for ive instrument families. Psychology of Music, 41(1), 97–118. Yim, H., & Ebbeck, M. (2009). Children’s preferences for group musical activities in child care centres: A cross-cultural study. Early Childhood Education Journal, 37(2), 103–111. 67 BCRME_199 text.indd 67 8/22/14 2:36 PM This content downloaded from 147.129.130.129 on Mon, 6 Oct 2014 16:26:36 PM All use subject to JSTOR Terms and Conditions