[Testing Testing] SBAC Grade 11 Practice Test #4

Each “Testing Testing” post analyzes a released test item, focusing on both the mathematics and interface involved in the new breed of exams. Our goal is to help improve the quality of these exams, especially if they may be used to inform student graduation or teacher merit pay. The mathematical analysis here is from Al Cuoco, director of the Center for Mathematics Education at EDC, from the Trevi Fountain in Rome. I provide the interface analysis.

Here’s Problem 4 from the SBAC Grade 11 Math Practice Test.

There are several issues with this item. By far the greatest issue is that it doesn’t assess a Common Core standard. The closest standard is HSA-APR.B.3, and read it carefully:

• HSA-APR.B.3: Identify zeros of polynomials when suitable factorizations are available, and use the zeros to construct a rough graph of the function defined by the polynomial.

This problem doesn’t ask students to do that. It asks students to use a rough sketch of a graph to build a factorization, identifying the linear factors of the underlying polynomial.

Besides this, there are some serious mathematical issues with this item as presented. The correct answer to this type of problem should start out in one’s mind as a multiple of $(x+3)(x+1)(x-2)^2(x-4)$, applying the Remainder Theorem (HSA-APR.B.2). Then, you check a sixth point to find the multiple, making sure the polynomial agrees with the nonzero points on the graph (a polynomial function of degree n is determined by n+1 function values, not just its n roots). But no coordinates of a sixth point are given. A student could simply not think of this and get the problem right, or estimate the value of the function at 0 and see if the graph is approximately correct (and get it right), or assume that this is a test and read the minds of the test writers (and get it right). It’s a good thing the graph seems to pass through (0, -48), or there would be no way to give a correct answer!

Even if you could get the exact coordinates of f(0), there are other polynomials functions whose graphs contain these same points and that have an extra x-intercept off the picture. The phrase “the function for the graph” is not accurate: there is more than one. Better wording: “Find a polynomial function that could have this graph.” The phrase “Create the function” should also be avoided, since it’s not clear what it means; Common Core has students “define” a function or “build” a function for modeling or transformation.

Still, the core issue is that this problem does not directly address a standard. The problem would be much better if it assessed HSA-APR.B.3 directly: Give a function defined by a polynomial that has been partially factored, revealing some zeros, allowing for complete factorization. For example:

$g(x) = 3(x^2-1)(x-2)^2$

Students who can sketch the graph of g (or answer questions about the graph) have met several HSA-APR standards.

Separate from the mathematics, there are a few major interface issues with this problem. Students are only allowed to drag linear factors to the right, so it is not possible for a student to enter $(x-2)^2$. They must instead enter the clumsy $(x-2)(x-2)$. Some students will wonder why they have to do it this way, others will think the interface is broken. It’s an unnecessary hurdle that will prevent some students from answering correctly even though they are capable of completing the task on paper.

Like #11, the interface does a poor job of overwriting when new objects are placed. Dragged objects “snap” into one of five positions, and if an object is already there, conflict:

We can picture a student trying to do this on purpose, attempting to get the $(x-2)^2$ term. It easily happens by accident and the interface for “deleting” objects is not obvious. Would the answer above be marked correct or incorrect? It’s not clear, and that is a big problem.

The SBAC Practice Tests are available for public viewing, and we are grateful to have these problems available for public comment.

Bowen is a mathematics curriculum writer. He is a lead author of CME Project, a high school curriculum focused on mathematical habits of mind, and part of the author team of the Illustrative Mathematics curriculum series. Bowen leads professional development nationally, primarily on how math content can be taught with a focus on higher-level goals. Bowen is also a champion pinball player and once won \$1,000 for knowing the number of degrees in a right angle.

8 Responses to [Testing Testing] SBAC Grade 11 Practice Test #4

1. Not assessing any of the standards seems like a major problem. Thanks for these Testing Testing posts btw. Enjoying your insight.

2. Sam says:

If we’re trying to catch common misunderstandings, it seems strange to omit x as a potential factor in the box.

3. l hodge says:

The problem does not directly assess the standard. But, what is wrong with indirect assessment? Responses to this question certainly provide some information on the student’s ability to do what is specified in the standard.

Directly testing the standard by going from a partially factored equation to the graph would be a more difficult question. Perhaps the authors were aiming for a relatively easy question that addresses, indirectly, at least part of the standard.

Providing a means for entering (x – 2)^2 could prevent some possible confusion, but is also is a big hint for those students that may not have otherwise thought about the possibility of repeated factors.

I might request “create the equation for a function that could have this graph” just to acknowledge that the graph itself is a function. But, fixing up some of these subtle wording issues may create additional points of confusion for students. Not sure where to strike the balance.

4. Bowen Kerins says:

In my opinion, every problem on such an exam should align to a standard, otherwise it’s not testing the Common Core. The test is short enough and needs to cover a wide range of standards, so a question that only indirectly assesses a single standard is inadequate.

If it’s a matter of difficulty, simpler problems could address the standard directly, such as asking students to match graphs with given polynomial functions.

• l hodge says:

I don’t know. To me, the original question is already a slightly less structured version of a multiple choice question that asks the students to match one of several functions to a single graph. Do you think that matching one of several graphs to a single function addresses the standard, but matching one of several functions to a single graph does not? The question is also less of a visual overload than the corresponding multiple choice would be with half a page full of choices involving parenthesis and x’s.

5. I agree that the problem is flawed in some of the ways that Bowen and other commenters have suggested, but I think it is fixable.

As for alignment, I would encourage people to look not only at the individual standards, but also the cluster headings. They, and the domain headings, and the hierarchy into which these are all arranged, are as much part of the standards as the individual standard statements. Alignment to cluster headings is not only possible but desirable, to avoid the atomization of the curriculum that is so often drive by tests. The new notation for standards which explicitly includes upper case roman letters for the clusters is intended to encourage this.

Thus, if a task touches on many standards in a cluster, it would align to the cluster heading. This case is a little more complicated, because although the task clearly aligns to the cluster heading A-APR.B: Understand the relationship between zeros and factors of polynomials, there is no standard within the cluster to which it aligns exactly. So it’s a judgement call. Still, the cluster headings distribute their meaning across the standards within the cluster, and it seems reasonable to me to claim at least partial alignment in this case (just my opinion, of course).

• Bowen Kerins says:

I think your opinion carries a fair amount of importance, sir…! I will work to include cluster and domain headings in future items, and agree completely that those should be the focus of curriculum. Thanks also for explaining the new breakdown of the standards, which I did not previously understand.

• dpsinha says:

As we think about granularity and clusters, we should also recognize that because the CCSSM writers stayed away from conflating standards with assessment targets or curriculum maps, the consortia have rightfully done their own clustering into targets (in part to avoid “micro-assessment” as discussed above). In this case however, things don’t get clarified. In the SBAC Grade 11 Content Specs, A-APR.3 isn’t associated with any target. The item would probably align to the broad Target M “Analyze functions using different representations”, which cites F-IF.7&8, and in particular F-IF.7c “Graph polynomial functions, identifying zeros when suitable factorizations are available, and showing end behavior”. But as with A-APR.3, a narrow reading doesn’t yield alignment, while saying “if they should be able to identify zeros from factorizations, shouldn’t they be able to also identify factorizations from zeros?” would.