Response to SBAC's Review Draft of Content Specifications with Content Mapping for the Summative Assessment of the Common Core State Standards for Mathematics Download

Andy Isaacs, Martin Gartzman, and Others
Center for Elementary Mathematics and Science Education (CEMSE)
University of Chicago

On August 29, 2011, the SMARTER Balanced Achievement Consortium (SBAC) released its Content Specifications with Content Mapping for the Summative Assessment of the Common Core State Standards for Mathematics (Content Specifications) for public comment. We reviewed the Specifications and submitted feedback to SBAC via their online survey. However, the importance of SBAC's charge—as well as the elegance of their proposed solution—warrants an additional public response.¹

One of the key implementation challenges of the Common Core State Standards for Mathematics (CCSSM) is to find valid ways to assess both the Standards for Mathematical Practice (SMPs) and the Standards for Mathematical Content. Historically, the tendency in large-scale assessment is to retreat to what is easy and cheap to measure. As such, concerns about optimizing the technical characteristics of the tests often end up overwhelming concerns about measuring what is important. This generally leads to an overemphasis on assessments involving routine problems that often do not reflect the rigorous mathematics of the type that is now expected in the CCSSM, including the practices required in the SMPs.

SBAC's approach confronts this problem head-on. SBAC's approach utilizes a spectrum of items balanced along a number of dimensions, including task length and level of expertise. Evidence is gathered about four major claims:

Claim #1 — Students can explain and apply mathematical concepts and carry out mathematical procedures with precision and fluency.

Claim #2 — Students can frame and solve a range of complex problems in pure and applied mathematics.

Claim #3 — Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.

Claim #4 — Students can analyze complex, real-world scenarios and can use mathematical models to interpret and solve problems.

We see some extremely valuable benefits to this approach, but we have several concerns as well.

Key Benefits to the Approach Outlined by the Content Specifications

• By organizing its assessment framework around the above four claims, SBAC has created an elegant model for integrating the SMPs and the content standards. For school districts and teachers struggling with how to understand the connections between content and practices in the CCSSM, the SBAC document is a great leap forward. It offers a practical means to translate the content and practice standards into viable assessment.
• The four claims provide much-needed coherence. A danger in any list of standards, such as CCSSM, is that curriculum, instruction, and assessment based on those standards will fragment into disconnected bits. Focusing assessment on the four claims should be helpful in counterbalancing this tendency toward fragmentation inherent in any attempt to write assessments for a set of content standards. The four claims will encourage the very coherence called for in the CCSSM.
• By organizing assessment around the four claims, the document frames a balanced and viable model for mathematics assessments. It would be very easy to create a large-scale assessment that focuses on the same old procedural fluency with paper-and-pencil arithmetic. This has been done repeatedly over the past thirty years. However, the CCSSM has challenged us to use and assess the mathematical practices and content in meaningful ways.
• The SBAC document acknowledges that procedural knowledge and skills can be assessed in the course of solving problems and generalizing mathematics from rich tasks. This reduces the need to assess students using large numbers of routine problems to ascertain whether they can use basic knowledge and follow simple procedures.
• Grounding the approach in J. Pellegrino's assessment triangle and the Depth of Knowledge framework strengthens the document. The cognitive rigor and depth of knowledge classification scheme will discourage an unhealthy focus on routine procedural fluency items and help assure a balance of assessment items that reflect the learning trajectories and complexity that underlie much of the CCSSM.

Major Concerns with the Content Specifications

• The prioritization of mathematical content clusters in Appendix A, in effect, narrows the curriculum beyond what is outlined in the CCSSM. We have observed the same problem with the prioritization schema used in PARCC's Content Frameworks. The CCSSM are presented as the minimum core standards—and we believe a further narrowing of the CCSSM endangers the integrity of the standards as a whole. In Grades 3–6, it appears that SBAC's intent is to emphasize every single topic from the arithmetic curriculum of the 1950s, while other essential content in the CCSSM receive lower priorities. If SBAC's tests for Grades 3–6 reflect the prioritizations in Appendix A, there is a danger that the implemented curriculum will fail to prepare students for college and careers in a world in which routine arithmetic calculations are routinely carried out by machines. The assessment consortia were charged (and funded) to assess the standards in the CCSSM, not to redefine them.
• There is a lack of clarity about interim assessments—when they will be administered and how they will be factored into the accountability formula. We have some concern that formative and interim assessments may constrain innovation in curriculum design by dictating particular instructional sequences and pedagogical approaches. The document's proposal to provide "curriculum- embedded assessments that offer models of good curriculum and assessment practice" (p. 8) is similarly unclear.
• The Content Specifications need clarity about the use of technology that students can use while taking the exam. In particular, the call to ban calculators before Grade 5 (p. 24) seems out of character with an earlier discussion in the document about the importance of a strategic use of tools by students in solving problems.
• The relationships among the standards, the content prioritization in Appendix A, the assessment targets for the four claims, and the proposed score-reporting categories are not transparent and are not aligned. For example, CCSSM focuses on arithmetic and geometry in the elementary grades, and several of the assessment targets for Grade 4 involve geometry. However, there is no reporting category for geometry in Grade 4. Even if geometry items are included on SBAC's total scores but not listed in its scoring reports, it is likely that instruction will focus on reported categories and geometry will be neglected. There are several other examples where the assessment targets, reporting categories (at least for Claim #1), and priorities do not align.

Despite our concerns, we consider the approach taken by SBAC promising. An assessment built on these specifications can help maximize the potential of the Common Core State Standards to transform mathematics education in a positive way.

Contact Information

Andy Isaacs, Co-Director
Martin Gartzman, Executive Director
Center for Elementary Mathematics and Science Education (CEMSE) University of Chicago
Email: aisaacs@uchicago.edu; mgartzman@uchicago.edu