By NCTM President Diane J. Briars
November 2014

Adoption of curriculum materials is one of the most important decisions a teacher, school, or district can make. While state standards describe what students are expected to learn and be able to do, what is taught in classrooms—the implemented curriculum—is heavily influenced by textbooks and other instructional materials. The instructional materials affect lesson content, depth and duration of instruction for particular topics, and topic sequence. So, while we may talk about curriculum materials as just "resources," the fact is that they strongly influence classroom instruction—for better or worse. 

Not surprisingly, evaluating curriculum materials has been a hot topic of conversation at recent meetings I've attended. "Which materials are best aligned with 'the Standards'—Common Core or other state standards?" "What criteria, rubrics, or evaluation processes will result in the selection of the 'best' curriculum materials for implementing 'the Standards'?"

During my tenure as mathematics director for the Pittsburgh Public Schools, I led many mathematics materials adoption committees—and I learned a great deal about productive and nonproductive practices. From that work and my experiences with other districts and states, large-scale materials review projects, and national recommendations, I offer my "Top Lessons Learned" about effective curriculum materials evaluation. 

Review Criteria and Process: Top Lessons Learned  

1. Focus on the central evaluation question: What curriculum materials best support students' learning of the standards? Wording the question in terms of students' learning of content, rather than implementation of standards, puts students' learning front and center. What students learn and how well they learn it depend on both mathematics content and instruction. Framing the review in terms of students' learning makes support for effective teaching and learning a critical feature for review, along with content.

2. Remember that content analysis is much more than alignment. Alignment of content with standards is often represented through "crosswalks" that connect the two, indicating whereand when content addresses particular standards. While such an approach can be useful, effective content analysis examines how materials address standards, that is, it looks for the following:

• The treatment of content is consistent with that described in the standards. For example, the grade 7 Common Core State Standards for Mathematics (CCSSM) emphasize understanding and using unit rates and equivalent ratios to solve problems involving proportional relationships, building a foundation for understanding slope. Thus, a critical content "look for" is whether materials build this understanding and emphasize use of these methods, instead of emphasizing solving proportions by using cross multiplication, with little or no attention to unit rates and equivalent ratios.
• The development of conceptual understanding, procedural fluency, and applications is balanced, with explicit connections among the three ("rigor" in CCSSM). A critical review criterion is the extent to which procedural fluency builds on conceptual understanding. With respect to applications, important "look fors" include applications that require problem solving and reasoning, as well as more routine use of concepts and skills; the use of applications to introduce new content, as well as to apply concepts and skills after initial instruction; real-world" applications; and, especially at the high school level, opportunities for using mathematics to model real-world situations.
• The development of content reflects what is known about how students learn that content most effectively. Ideally, this knowledge is incorporated in the standards, so it would be addressed in content treatment review. (CCSSM's attention to learning progressions, especially in grades K–8, is one of its strengths.) If it is not, or standards do not provide sufficient detail to reflect this knowledge, it is an important review criterion. For example, research clearly indicates that students learn their basic facts more efficiently and effectively when instruction focuses on fact families and strategies that relate unknown facts to known facts (doubles plus one, for example), instead of rote memorization of individual facts. Although CCSSM explicitly includes such strategies, other college- and career-ready standards may not. Regardless, the treatment of basic facts is an important consideration in materials review for grades K–4.
• The development of students' problem solving, reasoning, and other mathematical habits of mind—the set of processes identified in the CCSSM Standards for Mathematical Practice—receives explicit and regular attention. These experiences should be embedded in content development, not separate activities or lessons that can easily be skipped. This analysis is also part of the review of support for effective instructional practices described in #3 below.
• The materials are focused. Curriculum materials should give sufficient attention to the critical topics identified in the standards for each grade (in CCSSM, the "major work" of the grade), so that students have the time and support to develop the identified proficiencies. That does not mean simply adding more content to each grade so the books become larger! It means devoting more attention to focus topics and less to secondary topics, while omitting topics that are not in the standards.
• Content treatment is coherent. The content is effectively organized so that students can clearly see how ideas build upon, or connect with, other ideas both within and across grades.  This analysis requires looking at the development of content across grades and courses, in addition to looking at the development within a grade or a course.
• The mathematics in the materials is accurate. That the materials should be as close to error-free as possible goes without saying.

Student Achievement Partners' Publishers Criteria provides a more detailed discussion of the preceding criteria.

3. Analyze the nature of the instructional tasks and activities—this is as important as analyzing content. This analysis examines how the materials support students' learning though opportunities to engage in tasks that promote reasoning and problem solving and teachers' implementation of effective teaching practices as described in NCTM's Principles to Actions: Ensuring Mathematical Success for All. Critical questions include the following:

• To what extent do lessons regularly feature tasks that engage students in problem solving, reasoning, and making sense of mathematics as core instructional activities, rather than special features that can be omitted?
• What is the quality of these tasks? Do they permit multiple entry points and approaches? To what extent do they address the learning goals of the lesson?
• Do the tasks constitute a coherent series designed to address specific mathematical goals across lessons? Do the tasks build procedural fluency from conceptual understanding across lessons?
• What supports do the teachers' editions provide for effective implementation of these lessons? Do they provide, for example, information about likely student solutions, questions to support students as they work on tasks and in subsequent debriefing discussions, and suggestions about ways to structure the summary discussion?

Understanding the intended instructional model is essential for this analysis. Be sure to read the teacher's edition or other explanatory materials, view supporting webinars, etc., that describe the instructional model and where particular supports are located. Reviewing only the student materials may not provide sufficient understanding of how the materials are intended for use in the classroom to support an adequate analysis. 

4. Focus initial reviews on student materials and teacher editions of the materials.These have the primary influence on classroom teaching and learning. Analyze ancillary materials and other supports for effective teaching and learning—such as assessments, technology integration, additional practice, and professional learning—after you have narrowed your choices to materials that adequately meet the content and instructional support criteria. All the flashy supplementary materials in the world won’t make up for flawed content or lack of high-quality instructional activities.

5. Consider equity, diversity, and access. High-quality content and instructional practices are critical for the success of all students; therefore, reviews of these aspects are essential first steps in addressing equity and access. After narrowing your choices, however, consider specific ways in which materials promote equity and access. To what extent, for example, do they—

• provide teachers with strategies and materials for meeting the needs of a range of learners, including both struggling and advanced learners?
• suggest accommodations and modifications for English language learners that will support their regular and active participation in learning mathematics?
• provide a balanced portrayal of various demographic and personal characteristics?

See the CCSSO-NCSM Common Core State Standards (CCSS) Mathematics Curriculum Materials Analysis Tools for a more complete list of equity, diversity, and access criteria. 

6. Recognize that all omissions or gaps are not the same. No materials are perfect. Inevitably, an evaluation process will uncover gaps, omissions, or inadequate treatment of some content. The key question is how easily teachers, the school, or the district can fill the gaps. For example, providing additional practice on a skill may be relatively easy; providing lessons to address a gap in concept development is probably more difficult. Gaps that are most difficult or impossible to fill are consistent lack of instructional tasks that engage students in problem solving, reasoning, and the mathematical practices. Expecting teachers, schools, or districts to create or find high-quality tasks for almost every lesson is unreasonable—and, most likely, will not provide the consistent quality or coherence needed for effective teaching and learning.

7. Recognize that additional content is less problematic than gaps that are difficult to fill. Given the variation in standards across states, materials are likely to contain content beyond that addressed in your standards. The issue is how that extra content affects the treatment of content addressed in the standards. If the extra content can easily be skipped, or if it contributes positively to students' learning the content addressed in the standards, then it doesn't matter. It does matter, however, when it decreases time and attention on content addressed in the standards, disrupts the focus and coherence of the materials, or is so great that the books are huge.

8. Request all series and materials produced by each publisher. When you call for materials to review, remember that some of the large companies publish more than one program, so you may have to ask to see them all. Also, request programs from smaller, alternative publishers and developers as well as the large publishers. You want to review all the options, not just the traditional best sellers.

9. Allocate sufficient time for your review process. Thoughtful analysis of the content, instructional activities, and other features of curriculum materials described above takes time. Materials that are adopted are likely to be used—and to influence instruction—for a number of years. So time spent reviewing materials carefully is time well spent.

10. Use a "narrowing choices" strategy to make the review process as efficient as possible. Clearly, thorough content analyses are time-consuming—and may seem overwhelming. To make the process manageable, first review all materials for their treatment of only one or two key content domains. Retain for further review only those materials that give adequate treatment to those domains. Then make a second cut based on your evaluation of the nature of the instructional tasks and support for effective teaching practices within those domains. After these cuts, you're likely to have a manageable number of materials for further review. For example, to review middle school materials with respect to CCSSM, you might first review all materials for their treatment of ratios and proportional relationships (grades 6 and 7) and functions and expressions and equations related to proportional relationships (grade 8). Then review materials that treat that content well from the standpoint of the nature of their instructional tasks, and so on, for that content. Submit the materials that adequately address both criteria to additional review, starting with the remaining content domains, instructional tasks, and other review criteria such as equity, diversity, and access, ancillary materials, and so on.

11. Rate and discuss rather than score. Analysis of materials is qualitative, rather than quantitative; that is, reviewers are judging the quality of content treatment, instructional activities, and so forth, in different materials. Consequently, qualitative rubrics with categories such as "Not Found," "Low," "Marginal," "Acceptable," and "High" can be more useful than numeric scales. Qualitative ratings also provide useful guidance for subsequent within- and across-grade discussions of the quality of different materials.

12. Provide adequate professional learning for the members of the review team. It is essential that all reviewers both understand the standards and are knowledgeable about the effective teaching practices for implementing them. To ensure this common base of knowledge and understanding, consider engaging reviewers in collaborative study of the standards. For CCSSM, read and analyze the progression documents in addition to the standards themselves. APrinciples to Actions book study can be a good way to build knowledge of the effective teaching practices.

13. Try out your top choices in the classroom. The real test of the quality of any materials is the learning that they support in the classroom. If at all possible, try out at least a unit or two from the materials under final consideration in several classrooms. Even if the review committee is in unanimous agreement, using the materials in some classrooms is important before finalizing the decision. When you test the materials in this way, recognize that they may use unfamiliar instructional models, so students—and teachers—will need some adjustment time. My experience has been that trying out materials has been invaluable in helping review committees adopt materials that strongly support effective teaching and learning.

A number of rubrics and tools are available to support materials evaluation. I have used the CCSSO-NCSM Common Core State Standards (CCSS) Mathematics Curriculum Materials Analysis Project Tools referred to earlier. The strengths of these tools are that they provide qualitative rubrics for analysis of different review criteria, along with worksheets that are specifically designed to support cross-grade as well as within-grade analysis of treatment of core content domains. As you consider rubrics for your process, be sure that they (1) support cross-grade analysis of content coherence as well as the quality of individual lessons or units and (2) promote discussion of strengths and weaknesses of particular materials rather than only numerical ratings.

Even though this list of review criteria and processes may seem overwhelming, in practice, these "lessons" have worked very well to guide the review process and support adoption of materials that will promote all students' learning of the standards. Selection of curriculum materials is one of the most important responsibilities of teachers, schools, and districts. And careful analysis of how materials address standards and instruction is a necessary foundation for this work and critical to the learning of all students.