Research Commentary: The Promise of Qualitative Metasynthesis for Mathematics Education

Mathematics education has benefited from qualitative methodological approaches over the past 40 years. Stanley Erlwanger’s (1973) seminal case study of Benny opened the door to qualitative methodologies (Steffe & Kieren, 1994) by showing researchers the understanding and interpretations that can be learned through using methodologies such as interviews, case studies, and observations. Although the number, type, and quality of qualitative research studies in mathematics education has increased since Erlwanger’s work (Berry, Pinter, & McClain, 2013), little is known about how a collective body of qualitative research findings contributes to our understanding of a particular topic within the field. In other words, there is a lack of knowledge about how to integrate or synthesize findings across qualitative studies in mathematics education in order to influence policy and practice.

In the broader field of education, there is a growing interest in integrating findings across qualitative studies to discover patterns and common threads within a specific topic or issue as well as to deepen our understanding of evidence-based practices (Yore & Lerman, 2008). Through a process of qualitative research metasynthesis, our knowledge base can be broadened to provide insights into attitudes, perceptions, interactions, structures, and behaviors relevant for mathematics teaching and learning. The purpose of this Research Commentary is to provide readers with a rationale for, the definition and purpose of, and a summary of how to conduct qualitative metasynthesis as a means of synthesizing and interpreting qualitative studies in the field of mathematics education. The promise of qualitative metasynthesis is in its utilization and valuation of collective bodies of qualitative studies for synthesis and application in policy and practice.

Research Synthesis

Research synthesis is an attempt to integrate systematically a large body of related research literature. Often a review of literature, meta-analysis, and qualitative metasynthesis are considered similar; however, they are distinct. In order to define qualitative metasynthesis, it is important to distinguish between a review of literature and a meta-analysis. A review of literature is a convention in which researchers locate their original inquiry within the context of what has previously been studied so as to convince the reader that this additional study is justifiable and that the results of the study will have relevance to some aspect of advancing the body of literature (Thorne, Jensen, Kearney, Noblit, & Sandelowski, 2004). Sometimes a review of literature is designed to summarize the strengths and weaknesses of previous research for the purpose of establishing that previous findings and claims are relevant to the current focus of inquiry. The conclusion of such a review of literature purports that this new study will fill a strategic gap or provides knowledge relative to a previously poorly understood aspect (Téllez & Waxman, 2006; Thorne et al., 2004). A qualitative metasynthesis is not a review of literature; it is an analysis and interpretation of the findings from selected studies. Researchers conducting qualitative metasynthesis use a deliberate process of selecting studies with an emphasis on synthesizing, analyzing, and interpreting findings across the selected studies. This differentiates qualitative metasynthesis from the summary, description, and critique of a review of literature (Thorne et al., 2004).

A meta-analysis is a procedure first applied to quantitative group-experimental research data (Glass, McGaw, & Smith, 1981). Meta-analysis uses a statistical procedure that aggregates and condenses a body of quantitative research studies to a common standard metric, such as a mean effect size (Finlayson & Dixon, 2008). Typically, a meta-analysis is used to summarize, replicate findings, or determine cause and effect, whereas qualitative metasynthesis seeks to interpret findings for deeper understanding of meaning across a pool of selected studies. Qualitative metasynthesis is a procedure for qualitative research synthesis that produces interpretative results from integrating, comparing, and interpreting patterns and insights systematically across qualitative research studies while maintaining the integrity of the individual studies (Erwin, Brotherson, & Summers, 2011). Although thousands of meta-analytic investigations have been conducted in educational research (Forness, 2001), qualitative metasynthesis has received much less attention (Erwin et al., 2011).

Qualitative metasynthesis first emerged in the 1970s and has gained considerable attention in the field of nursing (Finlayson & Dixon, 2008). Qualitative metasynthesis is sometimes referred to as “meta-ethnography” (Noblit & Hare, 1988), “metasynthesis” (Sandelowski, Docherty, & Emden, 1997), or “metastudy” (Paterson, Thorne, Canam, & Jillings, 2001). Although some focused synthesis work has been conducted in the areas of educational leadership and desegregation (Noblit & Hare, 1988) and coteaching in the special education literature (Scruggs, Mastropieri, & McDuffie, 2007), to date, there has been one (Berry & Thunder, 2012) integrative review of qualitative mathematics education research using a research synthesis technique. Noblit and Hare (1988) were one of the first to introduce qualitative metasynthesis to the broader field of education research by describing a method that they identified as metaethnography, or “the synthesis of interpretive research” (p. 10).

Rationale for Qualitative Metasynthesis

Erwin, Brotherson, and Summers (2011) described three reasons for using qualitative metasynthesis in education. First, synthesizing a collective body of qualitative research to identify common themes or to compare and contrast different groups on a topic provides deeper insights and makes for a greater contribution to understanding more about how a collective body of research contributes to our understanding of a particular topic within the field. Second, in this milieu of evidence-based support, qualitative metasynthesis allows for a broader approach to evidence-based research, practice, and policy by expanding how knowledge can be generated and used. Third, qualitative metasynthesis responds to moving from knowledge generation to knowledge application by helping researchers to not only make sense of a collective body of research but also identify gaps and omissions in a given body of research.

Types of Qualitative Metasynthesis

Finlayson and Dixon (2008) delineated four types of metasynthesis: metaethnography, grounded formal theory, cross-case analysis, and metastudy. Each approach is an extension of a single-study version of the qualitative methodology. The metasynthesis level of study follows the same theory of methodology as its single-study counterpart while using the data of many qualitative studies. In metaethnography, the researchers synthesize individual ethnographic studies to describe broader relationships using metaphors (Noblit & Hare, 1988). In grounded formal theory, the researchers use coding and categorizing to develop an abstract, general theory to explain relationships (Strauss & Corbin, 1997). In cross-case analysis, the researchers systematically code, refine, and cross-reference descriptive metathemes and metacategories (Miles & Huberman, 1994). In metastudy, the researchers sample, evaluate, and analyze studies following a highly linear and structured procedure (Paterson et al., 2001).

Finlayson and Dixon (2008) described bidirectional dichotomies concerning the development of qualitative metasynthesis in which “a tension exists between the interpretive methods used to synthesise qualitative studies and scientific demands for transparency and reliability” (p. 61; see Figure 1). Their bidirectional dichotomies describe five tensions: (a) the use of iterative or a priori approaches, (b) the reflexive or detached role of the researchers, (c) the exhaustiveness of the literature search, (d) the nonuse or use of formal appraisal criteria, and (e) the systematic nature of the process. Metasynthesis can vary within these dichotomies. Finlayson and Dixon aligned metaethnography and grounded formal theory with iterative approaches, reflexive nature, nonexhaustive literature searches, lack of appraisal criteria, and nonlinear procedure. Metastudy is closer to the other end of the dichotomies because of its a priori approach, detached nature, exhaustive literature search, formal appraisal criteria, and linear procedure. Cross-case analysis varies within the dichotomies because its approach is linear and a priori; however, it lacks formal appraisal criteria and may not involve an exhaustive literature search.

Figure 1. Dichotomies of qualitative metasynthesis methodologies. This figure is adapted from “Qualitative Meta-Synthesis: A Guide for the Novice,” by K. W. Finlayson and A. Dixon, 2008, Nurse Researcher, 15(2), p. 61. Copyright [2008] by Nurse Researcher. Adapted with permission.

Methods: Qualitative Metasynthesis

A qualitative metasynthesis is a process that uses rigorous qualitative methods to synthesize and interpret data across a pool of qualitative studies. This process consists of six discrete steps: (a) identify a specific research metaquestion, (b) conduct a comprehensive search, (c) select initial relevant studies, (d) appraise the quality of initially selected studies, (e) synthesize findings of selected studies using qualitative techniques, and (f) present synthesis findings across the studies to address the research metaquestion (Sandelowski & Barroso, 2007). Methodologically, the process of qualitative metasynthesis is not different from a single-study qualitative research study; however, the significant difference is in the gathering of data and in the evidence-based synthesis findings.

Throughout this section, we discuss each of the six steps for conducting a qualitative metasynthesis. We first describe the process in general terms and then illustrate the process with a specific example. All of the examples are drawn from a qualitative metasynthesis that we conducted to study the mathematics experiences of Black learners (Berry & Thunder, 2012).

Identify a Specific Research Metaquestion

The formulation of a research question for a metasynthesis is similar to the formulation of a research question for a qualitative research study. A qualitative research question encapsulates the purpose of a qualitative study (e.g., to explore, understand, generate, or discover) and identifies the central phenomenon to be studied. A qualitative metasynthesis research question must also be a metaquestion—a question that has already been studied qualitatively. A metaquestion may not overlap completely with the individual research questions of all relevant studies, but it overlaps enough to necessitate further investigation that can result in an answer to the question. It should be broad enough to capture the studies that will contribute to an answer but not so broad that the answer is unattainable. The researcher conducting a metasynthesis poses a research metaquestion that is answered through the analysis of existing, relevant qualitative research studies.

By way of example, the research metaquestion that guided our metasynthesis (Berry & Thunder, 2012) was: In what ways do Black learners negotiate their experiences with mathematics across time? Our qualitative metasynthesis research question articulated the purpose of our study (to understand and discover) and identified the central phenomenon to be studied (Black learners’ experiences with mathematics). This question had already been studied qualitatively. There existed a body of qualitative research related to this question; however, this question had not been studied at the metalevel to move from knowledge generation to knowledge application. Through qualitative metasynthesis, we sought to understand the phenomenon within the existing, collective body of work and to develop a theory of success.

We relied on our expertise in the field of research related to Black learners’ experiences with mathematics in order to identify our specific research metaquestion. Initially, our research metaquestion was: In what ways do Black learners experience success with mathematics? As we conducted our search for studies to include in the metasynthesis, we found studies that explored Black learners’ experiences involving moments of both “success” and “nonsuccess” in mathematics. Through our iterative approach, we realized that all of these experiences were significant in the journeys of our participants and that we could not solely focus on, and therefore solely value, their experiences with academic success at certain moments in their lifetime. In addition, we realized that the phrase “experience success” relied on our narrow, outsider definition of success; however, our grounded theory approach demanded that we unpack insiders’ definitions. As a result, we revised the phrase to “negotiate their experiences,” which allowed for the participants’ own definitions of success to be examined and to become central to our development of a theory of success. We also added “across time” to emphasize the exploration of participants’ experiences as a cohesive whole rather than discrete events.

The process of identifying and revising our research metaquestion reflects an issue that is unique to qualitative metasynthesis. In qualitative methodologies, the definition of the phenomenon emerges from the data. Thus, when examining multiple qualitative studies, as in qualitative metasynthesis, the interpretations of the phenomenon may vary across studies (Jensen & Allen, 1996). We addressed this issue through our iterative approach. We revised and refined our question and allowed the definition of the phenomenon (Black learners’ experiences with mathematics, both successes and nonsuccesses) to emerge from the data.

Our final research metaquestion became: In what ways do Black learners negotiate their experiences with mathematics across time? We abandoned our definition of success and relied on participants’ varied definitions of success and nonsuccess. We used the verb negotiate to mean the various ways that participants found to overcome obstacles while experiencing mathematics, compromised and agreed on ways to experience mathematics, and made meaning of their own experiences with mathematics. “Across time” means that we explored the interconnectedness of participants’ experiences prior to and after their moments of success and nonsuccess.

Conduct a Comprehensive Search

The comprehensive search for studies to include in a metasynthesis results in the sample for the overall study. This process of gathering data for qualitative metasynthesis is significantly different from the process of gathering data for qualitative studies. Qualitative studies gather data through interviews, focus groups, observations, and document analyses. Qualitative metasyntheses rely on the data from previously conducted qualitative studies. The data are gathered through a comprehensive search for existing, relevant studies. A comprehensive search can be an exhaustive literature search, but it is not an exhaustive literature review. This search, retrieval, and validation process is systematic yet iterative, and it requires that the researchers keep track of each decision point. This creates both an audit trail and directions for replication.

Topical, population, temporal, and methodological parameters need to be defined by researchers at the onset of the search in order to have high recall and precision (Sandelowski & Barroso, 2007). Recall refers to the retrieval of relevant articles from the overall database. Precision refers to the level of relevance of retrieved articles. At this stage of the metasynthesis, high recall is most important. Precision will be further addressed through the appraisal stage. Researchers’ identification of the four parameters is highly contingent on their knowledge of the field. In broad fields in which metasynthesis will be informative and meaningful, researchers must have both a deep and a broad understanding of the landscape in order to define, explain, and defend the parameters. In this way, the four parameters serve as initial inclusion criteria.

The development of the topical parameter is closely linked to the identification of the research metaquestion. Researchers begin with a working definition of the topic based on their knowledge of the field. This definition can be revised and refined throughout the search as researchers encounter unanticipated findings. In our example metasynthesis, our initial topic was the experiences of Black learners of mathematics who experienced success in mathematics classes. Through the iterative search process, we revised and refined our topic to the experiences of Black learners in mathematics, in and out of the mathematics classroom as well as in and out of the school setting, and their definitions of success.

The population parameter is also informed by the researchers’ knowledge of the field and must be explained and defended within the sample selection of the study. Initially, our population was Black or African American PreK–Grade 12 students of mathematics attending school within the United States. We expanded our population to include participants older than PreK–Grade 12 who reflected back on their PreK–Grade 12 experiences as well as participants who were nonlearners (such as parents and teachers) whose reflections were focused on the experiences of learners (their children and students).

The temporal parameter defines both retrieval and publication time frames for the article search. In our metasynthesis, we limited the publication dates from January 2000 to May 2012 because previous work suggested that qualitative research on Black learners has increased significantly since the early 2000s (Berry et al., 2013). We conducted the retrieval from May 11 to June 28, 2012.

Finally, the methodological parameter requires the researchers to answer the question: What is qualitative research? Qualitative metasynthesis integrates the findings of qualitative research; thus, the studies included in qualitative metasynthesis must use qualitative methodologies (Sandelowski & Barroso, 2007). Our study’s methodological parameter included empirical qualitative research and excluded studies employing quantitative or mixed methods, dissertations, reviews of literature, summaries of research, policy documents, calls for research, book reviews, op-ed pieces, and pedagogical or practitioner articles describing the implementation of teaching, tools, or practice with learners. Mixed methods studies were excluded because, by their nature, their qualitative and quantitative methodologies and findings are overlapping, highly reliant on each other, and therefore inseparable (Sandelowski & Barroso, 2007). Dissertations were excluded because they are not peer reviewed; however, published peer-reviewed reports of dissertations were included as empirical qualitative research (Sandelowski & Barroso, 2007).

Bates (1989) described six berrypicking strategies: footnote chasing, citation searching, journal runs and hand searching, area scanning, author searching, and subject searches in electronic bibliographic databases. Berrypicking strategies rely on information retrieval by which researchers refine and make decisions about the search process. High recall in a search depends on using each of these strategies. The interdisciplinary nature of qualitative research demands that researchers use multiple indices to conduct their search and to ensure that it is comprehensive. Like other aspects of the comprehensive search, developing high-recall search terms for electronic databases depends on the expertise of the researchers. In our metasynthesis of Black learners’ mathematical experiences, we first conducted a search using EBSCO to simultaneously search the following five databases for peer-reviewed journal articles: (a) Academic Search Complete, (b) Education Research Complete, (c) ERIC, (d) Teacher Reference Center, and (e) OmniFile Full Text Mega. We purposefully chose to search using subject terms because articles in the selected databases are indexed by subject terms. We conducted an additional search using the Sociological Abstracts database using the same search terms. See Table 1 for a summary of the six databases used to search for studies. Within each database, we chose the special limiters options to limit our search to peer-reviewed, scholarly journal articles (see Table 1). Our initial search produced 391 documents using the five EBSCO databases and 310 documents using the Sociological Abstracts database.

Select Initial Relevant Studies

With each retrieved source, at least two research team members should review and decide whether the citation is relevant. The validation process begins with the title of the source but may require the researchers to review the abstract and possibly the full report. This, in turn, serves to revise and refine the inclusion criteria. Throughout the search, retrieval, and validation process, researchers systematically track their decisions about citations by recording exact databases, search term protocol, special limiters, and each decision point. Researchers can also check their audit trail and replicability by completing dual searches. Figure 2 visually displays the various decision points encountered during the search, retrieval, and validation process of our study of Black learners negotiating their mathematical experiences. In our study, using this iterative process, we selected 53 relevant studies from the 391 EBSCO documents. There were a significant number of studies that overlapped between the EBSCO documents and the Sociological Abstract database. We selected three additional studies that were only found in the Sociological Abstracts database for a total of 56 studies.

Figure 2. Decision points encountered during the search, retrieval, and validation process of metasynthesis. This figure is adapted from the Handbook for Synthesizing Qualitative Research (p. 51), by M. Sandelowski and J. Barroso, 2007, New York, NY: Springer. Copyright [2007] by Springer. Adapted with permission.

Appraise the Quality of Initially Selected Studies

After selecting initial relevant studies, researchers appraise the quality of these studies individually as well as comparatively.

Individual appraisal. The individual appraisal involves reading each study and evaluating it using a systematic but dynamic, intrareport reading guide.

The individual appraisal of studies also serves as an opportunity to address an issue that is unique to qualitative metasynthesis, the issue of comparability of qualitative studies. Although qualitative research cannot be treated as a unified field because of the plurality of methodological approaches (Dixon-Woods, Shaw, Agarwal, & Smith, 2004), each relevant qualitative study contributes to the understanding of a phenomenon regardless of the type of qualitative methodology (Jensen & Allen, 1996). Additionally, qualitative research studies should include basic quality criteria for methodological aspects such as research problem, purpose, and question; data collection techniques; data analysis; report of findings; and implications and conclusions.

In Figure 2, it is evident that the individual appraisal step is an integral part of the iterative process to finalize inclusion and exclusion criteria. Our initial inclusion and exclusion criteria were based on our definitions of the topical, population, temporal, and methodological parameters for our study. Through the search, retrieval, and validation process, we formulated additional permissible inclusion and exclusion criteria. See the Appendix for a summary of our final inclusion and exclusion criteria.

We chose to include longitudinal qualitative studies with participants older than PreK–Grade 12 if the participants’ reflections focused on their PreK–Grade 12 educational experiences. We also chose to include studies in which the participants were not the learners (i.e., parents and teachers), if the participants’ reflections focused on the learners’ experiences rather than their own experiences. Studies focused on additional learning opportunities, such as after-school mathematics clubs and out-of-school mathematics programs, were included because these were settings in which Black learners negotiated experiences with mathematics and contributed to the exploration of participants’ experiences as a cohesive whole rather than discrete events. Studies with non-Black participants were included if most of the participants were identified as Black and there was a presentation in the findings focused on the Black participants. We discovered multiple publications that used the same data; in that case, we conducted the individual appraisal of each study and then excluded one or both depending on their quality. Based on our additional permissible inclusion and exclusion criteria, we excluded 23 of the 56 studies and included 33 studies.

We addressed the issue of the comparability of qualitative studies by including qualitative studies relevant to our central phenomenon that shared high-quality qualitative methodology but varied in type of qualitative methodology. To evaluate the quality of each study’s qualitative methodology, we adapted an appraisal checklist reported by Erwin et al. (2011). Figure 3 represents the adapted checklist providing points for each indicator with a maximum of 15 points. This appraisal checklist served as our reading guide for individual appraisal of the 33 relevant studies. We used Erwin et al.’s point distribution and ranges for overall standards of quality and credibility: A score of 11–15 points indicates high overall standards, a score of 6–10 points indicates moderate overall standards, and a score of 0–5 points indicates low overall standards. We included the 26 studies that scored in the high range (11–15 points) and excluded 7 studies that scored in the middle range (6–10 points). These 26 studies were the final ones included in our metasynthesis.

Figure 3. Appraisal criteria for assessing quality of qualitative research process. This figure is adapted from “The Promise of Qualitative Metasynthesis: Mathematics Experiences of Black Learners,” by R. Berry and K. Thunder, 2012, Journal of Mathematics Education at Teachers College, 3(2), p. 46. Copyright [2012] by the Journal of Mathematics Education at Teachers College. Adapted with permission.

Comparative appraisal. Comparative appraisal involves creating cross-case displays and summaries of the selected, relevant studies. The purpose of comparative appraisal is to prepare for synthesizing findings, to notice initial trends and patterns, and “to include items directly relevant to the integration of findings you want to produce” (Sandelowski & Barroso, 2007, p. 81). The comparison appraisal also allows for the identification of missing information, confirming and negative cases, and duplicate reports. In our metasynthesis, we created a cross-case display of the 26 selected studies organized within four categories.

After reconciling our individual appraisals, we discussed initial trends among the studies. We noticed that the studies varied in the time frame of the participants’ experiences: 14 studies focused on participants’ experiences within a finite period of time, such as one grade level or specific years spent in special mathematics or academic programs, after-school programs, or mathematics clubs, whereas 12 studies engaged participants in reflecting on their experiences spanning their entire lives, such as describing how experiences from early time periods contributed to their current positioning with mathematics. We also noticed that the studies varied in the setting of the participants’ experiences: Seven studies focused on participants’ experiences in school only, two studies focused on participants’ experiences out of school only, and 17 studies focused on participants’ experiences both in and out of school. The patterns in these contexts were significant because they narrowed or broadened the participants’ lenses for the way they framed mathematical experiences and their awareness of connections among experiences to mathematics. Based on these initial patterns, we categorized the research papers across two domains: temporal (time) and settings (in school or out of school). Then, we further categorized papers four ways: (a) across time and in school and out of school, (b) finite time and in school and out of school, (c) finite time and out of school, and (d) finite time and in school. These relationships emerged from our data and were not a priori. This comparative appraisal supported our initial analysis and integration of findings.

Synthesize Findings of Selected Studies Using Qualitative Techniques

The findings sections from each article serve as the data for qualitative metasynthesis. The data for the qualitative metasynthesis include each study’s participants’ voices through quotes as well as each study’s researchers’ voices through their descriptions. Consequently, each article’s entire findings sections are extracted into a single document to be coded. Researchers should select a method for analysis that is informed by the purpose of the study, the theoretical framework of the study, and the type of qualitative metasynthesis along the continuum (see Figure 1). Data analysis follows the same process for single-study qualitative research and qualitative metasynthesis.

In our example study, the grounded theory approach was used to code, categorize, and constantly compare data to develop a general theory of success (Strauss & Corbin, 1997). We open coded the findings independently and then negotiated our independent, open coding to reach a shared set of initial codes and definitions to be used consistently throughout the analysis of data. The initial codes were then categorized. We reread and recoded to refine and verify coding and to assure consistency. After this, we sorted the data by codes and reread, looking for themes within each code to see if there were dimensions that required the data to be further discriminated. Through this process, themes emerged from the data.

Present Synthesis Findings Across the Studies to Address the Research Metaquestion

From this categorization and classification of the data, researchers provide and describe visual data displays. In our study (Berry & Thunder, 2012), we created a cross-case display of the 26 selected studies organized within four categories, a table with representative quotes from each article for each of our five findings, four additional tables that organized and visually displayed details and definitions for each finding, and finally a graphic of the defining qualities of learners’ experience pathways. Miles and Huberman (1994) provide additional strategies and possible tables and graphics for creating visual data displays.

The synthesis findings across the studies should be used to answer the research metaquestion. By synthesizing a collective body of work, the researcher reports evidence-based findings that can be used to drive policy and practice. In our sample qualitative research study, we used the existing body of research on Black learners’ experiences with mathematics across time to build a theory of success. We found that success is a continuous negotiation, which includes nonsuccess, and that this negotiation is built on values informed by experiences. Black learners with a high sense of agency chose actions and behaviors that embodied an active pursuit of success based on their own definitions of success. In order to support Black learners with a high sense of agency, all educators (including teachers, guidance counselors, administrators, and policy makers) should serve as advocates to help all learners access the broadest academic options and select challenging academic options that lead to the learners’ definitions of success. Additionally, in order to shift learners with a low sense of agency to a high sense of agency, educators must choose language and practices that reflect a growth mindset and that cultivate identities with positive self-efficacy. In other words, informed student choice with an eye to long-term planning toward learners’ definitions of success should be central to transparent, challenging, and student-centered academic advising and academic engagement.

Validity criteria. High quality presenting and reporting of qualitative metasynthesis establishes validity. Validity criteria for qualitative metasynthesis methods are comparable to validity criteria for single-study qualitative research methodologies yet reflect the larger grain size of metasynthesis data. There are four types of validity to address in qualitative metasynthesis: descriptive, interpretive, theoretical, and pragmatic (Sandelowski & Barroso, 2007). Similarly, Shenton (2004) described four criteria to establish validity in single-study qualitative research: credibility, dependability, confirmability, and transferability.

In single-study qualitative research, credibility refers to establishing the accuracy with which the researchers record the phenomenon studied. When integrating multiple qualitative studies through qualitative metasynthesis, descriptive validity evaluates how accurately the data collected represent all relevant studies. In our sample metasynthesis, descriptive validity was verified through our exhaustive literature search, independent replication of our literature search, independent individual appraisal, and a thick, rich description of synthesis findings (in particular, the table of representative quotes from all 26 studies).

Dependability in single-study qualitative research is determined by the replicability of the study. In qualitative metasynthesis, the original findings and interpretations of multiple studies go through a second round of analysis integrated with each other; therefore, interpretive validity verifies that the phenomenon is interpreted accurately, completely, and with regard to multiple perspectives through member checking. Member checking in our metasynthesis consisted of the researchers independently replicating the literature search, meeting to reconcile individual appraisals, collaborating to create the comparative appraisal, refining and revising our metaquestion and inclusion and exclusion criteria, and negotiating our independent open codes in order to recode the data consistently.

In single-study qualitative research, confirmability is established through reflective commentary to demonstrate the objectiveness of the researchers and their interpretations. Similarly, in qualitative metasynthesis, theoretical validity is established through reflective commentary combined with an audit trail in order to demonstrate the credibility of the researchers’ decisions, including the research methods and integration of findings. Figure 2 represents the highly systematic nature of our iterative approach, documents each decision point, and creates an audit trail despite our nonlinear path. In addition, our metasynthesis included reflective commentary.

Finally, the transferability of a single qualitative study evaluates whether the context has been described in enough detail to transfer the findings to similar contexts. The pragmatic validity of a qualitative metasynthesis evaluates whether the synthesis findings are useful, timely, applicable, and translatable to other contexts. We established pragmatic validity in our metasynthesis through our exhaustive literature search; our thick, rich descriptions of synthesis findings; and our discussion of the implications for our synthesis findings to drive policy and practice.

Figure 4 outlines the components to include when presenting and reporting qualitative metasynthesis. Each of the steps for conducting a qualitative metasynthesis is evident in this outline.

Figure 4. Steps for reporting qualitative metasynthesis.

Conclusion

Qualitative research in mathematics education has exposed new relationships among familiar ideas by pushing the field to pay closer attention to the context and processes of mathematics teaching and learning. Researchers’ views of hmathematics teaching and learning have been complicated by qualitative researchers’ efforts to unpack complex phenomena, such as interconnected contexts and situated and sociocultural perspectives in mathematics teaching and learning. These researchers are pushing the field beyond process–product research toward considering the complexities and nuances significant for understanding factors that influence outcomes. Since the Benny study (Erlwanger, 1973), the field of mathematics education has benefited from a growing body of qualitative research. Some topics in the field have reached a degree of maturity from which qualitative studies can be synthesized to reveal patterns and common threads.

Our knowledge of the field suggests that there are several topics in mathematics education that could benefit from further investigation through qualitative metasynthesis. Our nonexhaustive list of topics includes: (a) qualitative research on teaching practices (e.g., mathematics discourse, use of tools and technology), (b) qualitative research on the mathematics experiences of marginalized learners (including students with special needs and English language learners), (c) qualitative research on learners’ thinking and understanding on mathematics topics (e.g., fractions and algebraic representations), (d) qualitative research on professional development, and (e) qualitative research on novice or preservice teachers’ experiences. By implementing qualitative metasynthesis methodologies, researchers can pose metaresearch questions to a common pool of studies for further investigation. Qualitative metasynthesis allows researchers to interpret and articulate deeper understanding of meaning across a pool of studies.

It is plausible that findings and implications from qualitative metasynthesis can provide new understanding about the complexities and nuances across a pool of studies that may not be as evident in a single study. The field of health science provides an example of how findings and implications from qualitative metasynthesis have influenced public policy and clinical practice. Neubeck et al. (2012) reported that findings from a qualitative metasynthesis focused on patients with heart disease in cardiac rehabilitation had implications on policies and practices for participation in cardiac rehabilitation. Because the findings of qualitative metasynthesis are presented across a common pool of studies rather than one study, qualitative metasynthesis is better positioned to have an influence on practices and policies in mathematics teaching and learning.

The use of qualitative metasyntheses in education, specifically in mathematics education, remains inchoate. Because of this, we looked to the field of health sciences, particularly nursing, to inform our methodological framing. One goal of this commentary is to transfer, detail, and explain the methodological framing for qualitative metasynthesis within mathematics education. As researchers engage in qualitative metasynthesis, we hope that our detailed, step-by-step process as well as our sample study will inform their methods and prepare them for the significant decision points along the way.

On a practical level, we suggest that qualitative metasynthesis requires at least two researchers who are experts in the research topic area. Expertise is needed to formulate a research metaquestion that is answerable through qualitative metasynthesis, to make sense of varying interpretations of the phenomenon across studies, to understand the meanings of contributing theoretical frameworks across a pool of studies, and to identify essential topical, population, temporal, and methodological parameters. Also, experts are familiar with some range of the relevant literature, which allows them to search the literature exhaustively, to create appropriate inclusion and exclusion criteria, and to select relevant studies. Researchers with expertise are more readily able to negotiate varying theoretical frameworks and qualitative methodologies across the pool of studies. Most important, expertise in the research topic facilitates the researchers’ ability to interpret and apply synthesis findings in order to answer the research metaquestion and to knowledgeably identify implications for research, policy, and practice; in other words, expertise facilitates the researchers’ ability to meet the goals of qualitative metasynthesis.

With the expansive and ever-growing number of qualitative research studies in mathematics education, it is time to move beyond knowledge generation to knowledge application. There is a wealth of qualitative research in mathematics education. Findings and implications from qualitative metasynthesis hold the potential to influence policies and practices to improve mathematics teaching and learning. Applying the rigor of qualitative metasynthesis to the existing body of research broadens the notion of evidence-based practices. That is the promise of qualitative metasynthesis in mathematics education.

References

Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407–424. doi:10.1108/eb024320

Berry, R., & Thunder, K. (2012). The promise of qualitative metasynthesis: Mathematics experiences of Black learners. Journal of Mathematics Education at Teachers College, 3(2), 43–55.

Berry, R. Q., III, Pinter, H. H., & McClain, O. L. (2013). A critical review of American K–12 mathematics education, 1900–present: Implications for the experiences and achievement of Black children. In J. Leonard & D. B. Martin (Eds.), The brilliance of Black children in mathematics: Beyond the numbers and toward new discourse (pp. 23–53). Charlotte, NC: Information Age.

Dixon-Woods, M., Shaw, R. L, Agarwal, S., & Smith, J. A. (2004). The problem of appraising qualitative research. Quality & Safety in Health Care, 13(3), 223–225. doi:10.1136/qshc.2003.008714

Erlwanger, S. H. (1973). Benny’s conception of rules and answers in IPI mathematics. Journal of Children’s Mathematical Behavior, 1 (2), 7–26.

Erwin, E. J., Brotherson, M. J., & Summers, J. A. (2011). Understanding qualitative metasynthesis: Issues and opportunities in early childhood intervention research. Journal of Early Intervention, 33(3), 186–200. doi:10.1177/1053815111425493

Finlayson, K. W., & Dixon, A. (2008). Qualitative meta-synthesis: A guide for the novice. Nurse Researcher, 15(2), 59–71. doi:10.7748/nr2008.01.15.2.59.c6330

Forness, S. R. (2001). Special education and related services: What have we learned from meta-analysis? Exceptionality, 9(4), 185–197. doi:10.1207/S15327035EX0904_3

Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

Jensen, L. A., & Allen, M. N. (1996). Meta-synthesis of qualitative findings. Qualitative Health Research, 6(4), 553–560. doi:10.1177/104973239600600407

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.

Neubeck, L., Freedman, S. B., Clark, A. M., Briffa, T., Bauman, A., & Redfern, J. (2012). Participating in cardiac rehabilitation: A systematic review and meta-synthesis of qualitative data. European Journal of Preventive Cardiology, 19(3), 494–503. doi:10.1177/1741826711409326

Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.

Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001). Meta-study of qualitative health research: A practical guide to meta-analysis and meta-synthesis. Thousand Oaks, CA: Sage.

Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. New York, NY: Springer.

Sandelowski, M., Docherty, S., & Emden, C. (1997). Qualitative meta-synthesis: Issues and techniques. Research in Nursing & Health, 20(4), 365–371. doi:10.1002/(SICI)1098-240X(199708)20:4<365::AID-NUR9>3.0.CO;2-E

Scruggs, T. E., Mastropieri, M. A., & McDuffie, K. A. (2007). Co-teaching in inclusive classrooms: A metasynthesis of qualitative research. Exceptional Children, 73(4), 392–416. doi:10.1177/001440290707300401

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75.

Steffe, L. P., & Kieren, T. (1994). Radical constructivism and mathematics education. Journal for Research in Mathematics Education, 25(6), 711–733. doi:10.2307/749582

Strauss, A. L., & Corbin, J. M. (Eds.). (1997). Grounded theory in practice. Thousand Oaks, CA: Sage.

Téllez, K., & Waxman, H. C. (2006). A meta-synthesis of qualitative research on effective teaching practices for English language learners. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 245–277). Language Learning & Language Teaching monograph series. Amsterdam, the Netherlands: John Benjamins Publishing. doi:10.1075/lllt.13.12tel

Thorne, S., Jensen, L., Kearney, M. H., Noblit, G., & Sandelowski, M. (2004). Qualitative metasynthesis: Reflections on methodological orientation and ideological agenda. Qualitative Health Research, 14(10), 1342–1365. doi:10.1177/1049732304269888

Yore, L. D., & Lerman, S. (2008). Metasyntheses of qualitative research studies in mathematics and science education. International Journal of Science and Mathematics Education, 6(2), 217–223. doi:10.1007/s10763-008-9128-9

Authors

Kateri Thunder, Burnley-Moran Elementary School, Charlottesville City Schools, 1300 Long Street, Charlottesville, VA 22901; [email protected]

Robert Q. Berry III, Department of Curriculum, Instruction and Special Education, Curry School of Education, University of Virginia, P.O. Box 400273, Charlottesville, VA 22904; [email protected]

Submitted July 3, 2014

Accepted January 21, 2015

APPENDIX