Tarr, James E., Reys, Robert E., Reys, Barbara J., Chávez, Óscar, Shih, Jeffrey, and Osterlind, Steven J. “The Impact of Middle-Grades Mathematics Curricula and the Classroom Learning Environment on Student Achievement.” Journal for Research in Mathematics Education 39, (May 2008): 247–280.
What influences students’ learning in the classroom? Is it the selection of textbooks, the learning environment, or the teacher’s professional development? This study sought to compare and contrast the systemic effects of adopted textbooks, teacher practices, and standards-based learning environments on student achievement in mathematics.
In this era of high-stakes testing and public accountability, school personnel are searching for ways to improve mathematics learning opportunities for all students. Although there is no single “magic bullet,” one avenue for strengthening school mathematics programs is the selection and implementation of high-quality mathematics textbooks.
Researchers note that textbooks have historically played a prominent role in U.S. classrooms, often defining the mathematics curriculum that students have an opportunity to learn. Studies also underscore the need to give careful attention to the mathematics content that the textbooks emphasize and how they present it.
Despite the dominant role that mathematics textbooks have played, drawing a direct link from the textbook to the students’ learning is complicated by other factors, including the teacher’s choices and actions, the organization of the school and classroom, and the students’ readiness and willingness to learn.
Analyzing the impact of mathematics textbooks on student learning is complex and difficult for many reasons. These challenges include gaining access to schools, documenting the extent to which teachers follow the textbook, collecting data over a sustained period, identifying appropriate comparison groups, isolating variables, and obtaining valid measures of student achievement.
Notwithstanding the challenges inherent in studying the relationship among teachers, curriculum, and student learning, a growing number of studies provide evidence that curricular materials funded by the National Science Foundation (NSF) positively influence teacher decisions and actions and student learning.
Who Was Studied?
The study examined the achievement of 2,533 students receiving instruction in 10 middle schools from 33 teachers over a two-year period. Rural, small community, suburban, and urban schools were chosen to reflect the diversity of the U.S. school population. Consideration was also given to ethnic representation among students as well as a range in the percentage of students qualifying for free or reduced lunch. In addition, the selection of schools for the study was based on the number of years since adoption of the current mathematics textbook series, organizational structure, and community size.
James E. Tarr, Robert E. Reys, Barbara J. Reys, Óscar Chávez, Jeffrey Shih, and Steven J. Osterlind conducted the study and report their findings in an article in the May 2008 issue of the Journal for Research in Mathematics Education (see below for full citation).
The authors examined the students’ achievement in relation to the implementation of textbooks, comparing achievement resulting from instruction conducted with texts developed with funding from the NSF with achievement resulting from instruction conducted with textbooks developed by commercial publishers.
The authors determined their findings by measuring student learning on two tests, the Balanced Assessment in Mathematics (BAM) and the TerraNova Survey (TNS). The BAM is an open-ended test, and the TNS is a multiple-choice test.
The study was designed to examine how variables at the classroom level—such as the use of one curriculum type, implementation of district-adopted textbooks, and presence of a standards-based learning environment—affected middle school students’ mathematical achievement. This focus necessitated classroom observations by the research team.
The study concentrated on finding the answers to three questions:
- Does the implementation of district-adopted textbooks differ by curriculum type (NSF funded vs. publisher developed)?
- In classrooms where textbooks are a strong determinant of the content and activity of mathematics lessons, to what extent does curriculum type predict student achievement?
- Does the relationship between curriculum type and student achievement in mathematics depend on variations in the standards-based learning environment?
For this study, a standards-based learning environment is defined as an environment in which students make conjectures about mathematical ideas and explain their responses or strategies. Student statements are used to build discussion or work toward a shared understanding for the class. Moreover, multiple perspectives are encouraged and valued, and enacted lessons foster the development of conceptual understanding.
Comparison of the two sets of teachers (those using NSF-funded or publisher-developed textbooks) indicated that they were comparable in educational background, familiarity with national professional standards, and teaching licensure and experience. The authors’ analyses of data also found similar levels of textbook implementation across curriculum types.
However, the teachers differed in recent professional development as well as beliefs about teaching and learning mathematics. Teachers of NSF-funded curricula reported more time spent attending workshops or recent coursework in mathematics. They also reported participation in professional development focused on learning how to use their district’s textbook with particular emphasis on learning inquiry/investigation-oriented methods.
Another distinct difference was that teachers of publisher-developed curricula were more likely to believe that students learned best in classes in which students were grouped according to ability. They also believed that students learned best with teacher-led lessons that provided students with opportunities for practice and reinforcement.
The interaction between the curriculum and a standards-based learning environment was associated with a significant impact on students’ achievement. Specifically, this study provides evidence that a standards-based learning environment has a positive impact on students’ achievement on performance assessments that measure mathematical reasoning, problem solving, and communications skills, but only when such an environment is coupled with a curriculum that embodies this pedagogical orientation.
In particular, when NSF-funded curricula were implemented in a manner consistent with their design specifications, the influence on students’ achievement on the BAM was compelling; when this was not the case, BAM performance levels were lower. These results make the case that what is needed for higher student achievement is consistency between curriculum and instruction.
Nevertheless, in using the TNS as the dependent measure, the study found no significant impact of either curriculum type when coupled with varying levels of standards-based learning environments.
The study further concluded that standards-based learning environments were more prevalent in the classrooms of teachers using an NSF-funded curriculum. However, introducing a standards-based learning environment did not guarantee that teachers attained a high level of success in this area. The fact that a low level of standards-based learning environment was evident among 30 percent of teachers using NSF-funded curricula is troubling, given that these teachers received significantly more professional development than teachers of publisher-developed textbooks. In addition, the professional development focused on learning how to implement the NSF-funded textbooks.
These results point to further evidence that standards-based curricula are more challenging to teach and teachers require more professional development to teach them.
The researchers acknowledge several limitations in their study related to sample and analyses, observation data, and challenges associated with equating groups, as well as school, teacher, and student attrition.
Bay-Williams, J. M., B. J. Reys, and R. E. Reys. “Effectively Implementing Standards-Based Mathematics Curricula in Middle Schools.” Middle School Journal 34 (March 2003): 36–41.
Reys, R. E., B. J. Reys, R. Lapan, G. Holiday, and D. G. Wasman. “Assessing the Impact of Standards-Based Middle Grades Mathematics Curriculum Materials on Student Achievement.” Journal for Research in Mathematics Education 34 (January 2003): 79–95.
Tarr, J. E., Ó. Chávez, R. E., Reys, and B. J. Reys. “From the Written to the Enacted Curricula: The Intermediary Role of Middle School Mathematics Teachers in Shaping Students’ Opportunity to Learn.” School Science and Mathematics 106 (April 2006): 191–201.