Assessment Principles in Illustrative Mathematics 6-8 Math

A wide-ranging team worked together to develop the Illustrative Mathematics Grades 6-8 Math curriculum. As Assessment Lead, it was my responsibility to write and curate the Shared Understandings document about assessments we used throughout the writing process, and I thought you might be interested to read some of the key features.

This quote drives a lot of the ideas about assessment:

“You want students to get the question right for the right reasons and get the question wrong for the right reasons.” – Sendhil Revuluri

The statement above is particularly true for multiple-choice items, and is a shift from the ways I used to write as a high school teacher. If there were a likely “sign error” I would include that as a multiple-choice distractor, because surely some students will make that mistake. But this is the wrong reason to be wrong: the item is meant to test a particular standard. Distractors should have good reasons for being selected that are relevant to the standard(s) being addressed.

  • In general, assessment items should be targeted and short.

This particularly is true for application problems, which frequently have sentences or paragraphs that are meaningless to the task at hand.

  • Items must exist in isolation, never using the result of another item. These “double whammy” items penalize students who make an error or skip items. Each part of an extended response item must not depend on answering a previous part correctly. Each part should be answerable even if all previous parts have been skipped completely, to give students all possible opportunities to show proficiency. When you think a “double whammy” is unavoidable, think about what information would put the student in the position that a student correctly answering the first part would be. Typically, a “restart” of the item with a different name, object, equation, or example of the same context can avoid the “double whammy”.

Specifically, never ask students to use their work in part (a) to do part (b), because if they could not solve part (a), they have no ability to demonstrate the skill intended by part (b).

  • Items must be method-agnostic whenever possible. Avoid “Use [method] to solve [problem]”, because this may force students to use a method or representation that runs against their preferences.

Just write “Solve [problem]”.

  • Assessments as a whole should reflect a varied depth of knowledge including items that would be rated as DOK 1, 2, or 3 on this chart. In general, an assessment should have about 40% DOK 1, 40% DOK 2, and 20% DOK 3. The most typical error is not enough DOK 1.

It’s okay to include a few fastballs on a test!

  • A student who has mastered the target skill should ideally be able to answer a multiple-choice item without looking at the options, then find the answer among the options. In some cases it is necessary to have the student discriminate among the options, but if this can be avoided, do so. For example, “Which of these points is in Quadrant II?” can be improved by asking “Which quadrant is (-3,4) in?”

And this is the biggest one for multiple-choice:

  • Think carefully about the logic a student might use to respond to the item, and whether there are significant and relevant conceptual errors a student could make but still arrive at the correct response. Pick correct responses accordingly, or use distractors to catch these errors. For example, consider “Which of these fractions is largest? 1/3, 1/4, 2/7, 3/8”. It seems fine, but a student whose process is “a fraction with a larger numerator is larger” will select 3/8 and would be correct for the wrong reason. In this case 2/5 would be a better correct answer (replacing 2/7); if 3/8 is the intended correct response, use at least one distractor with a larger numerator.

There is a lot more to say, but hopefully this gives you some flavor for the depth of thought the Illustrative Mathematics team put into these materials. Go here for access to the materials. Thanks for reading!

Advertisements

Is 2501 prime?

While walking the dog I encountered the license plate “A 2501”, and because I’m a weirdo my immediate question was to wonder if 2501 is prime.

It’s not a multiple of 3… or 7… or 13.

So, let’s give you some spoiler space, to try the problem without any techmology. Is 2501 prime, or not?

 

 

 

This spoiler space is sponsored by the Illustrative Mathematics Grades 6-8 curriculum, coming very soon! It will be CC-BY, which means you can download and use it free for any purpose, forever.

 

 

 

Enough space yet?

 

 

So, having failed to find a simple multiple that divides 2501, I noticed it was one more than a multiple of 4, and a sum of squares:

2501 = 50^2 + 1^2

This is a sign that it might be prime, because all primes that are one more than a multiple of 4 can be written as the sum of two squares (try 17 … 21 … 29). But it’s also a sign that it might not be prime, because all primes that are one more than a multiple of 4 can be written as the sum of two squares in exactly one way (try 25 … 65).

So the search changed from finding a factor to finding another sum of squares, and the search ended quickly after remembering 2401 was a perfect square:

2501 = 49^2 + 10^2

What’s interesting here is we know from this that 2501 is not prime, but we don’t know what its factors are! Having tested and not finding any prime factors less than 17, this is enough to know that 2501 is the product of exactly two prime factors.

But which ones? To do this, I tried to find a way to write 2501 as a difference of squares. And again, the search ended much more quickly than I expected:

2501 = 51^2 - 10^2

51^2 - 10^2 = (51 + 10)(51 - 10)

It made me wonder if there are other numbers with the same property, where the number is 1 more than a perfect square, and also has an equal gap to the next square up and next square down… guess what.

26 between 16 and 36

37 between 25 and 49

n^2 + 1 between (n-1)^2 and (n+1)^2

Let me know if you find anything interesting here, or if you have any favorite numerical calculations like these.

All-New Mathematics of Game Shows (NCTM 2016)

Thanks to those who attended my talk at NCTM 2016 at 8am Saturday! Yow, that’s early. At the same time as many other great talks, it was an honor to have a good audience.

Here are the slides from my presentation. Hopefully they make sense to those who weren’t there, but if you have any questions, please let me know.

Slides from NCTM 2016 (PDF)

Ed Begle’s Laws of Mathematics Education

This post from Dan Meyer discussed Ed Begle’s two laws of mathematics education, which feel as relevant today as they did during his presentation in October 1970.

  1. The validity of an idea about mathematics education and the plausibility of that idea are uncorrelated.
  2. Mathematics education is much more complicated than you expected even though you expected it to be more complicated than you expected.

You can read all of Begle’s remarks beginning at page 27 of this document, which also includes many other presentations and papers from the same conference.

Another conversation about The Arrow Way

A little more with my second-grade son, age 7, about how he does addition and subtraction, compared with “the Arrow Way” from EngageNY.Screen Shot 2015-11-17 at 6.46.53 PM

“Would you rather do an addition problem or a subtraction problem?”

“Ummmmm…”

“Alright I’ll pick one of each. Tell me how to do 39 + 23.”

“It equals 62.”

How did you do it?”

“By adding the 9 and the 3 first, and then adding the tens.”

“How many tens?”

“Six tens.”

“But I only see five tens.”

“That’s because 9 plus 1 equals 10.”

“Oh, that’s really cool. Hey, how would you do 62 – 23?”

“Ummmm… 62 minus 23… 62 minus 23… it equals (whisper) 39.”

How did you do it?”

“Just by … well, I just knew the way I did it before, and minused it.”

“I don’t know what you mean. Tell me more about how you did it!”

“I just counted down.”

“So, you counted down one at a time? Like 62, 61, 60, 59…?”

“Yeah yeah!”

“Really, you counted all 23 numbers down? You didn’t skip any? I didn’t hear you count all those numbers. I think you used a shortcut.”

“No, I just counted them in my brain. All the way down.”

“Ok, what would you do for 62 – 51?”

(Very fast.) “Uh, 11.”

“So, how did you do that so fast? You didn’t count all the numbers, right?”

“Yeah I counted them really fast!!”

“Come on, what did you really do?”

“Well, I counted it down really fast. 62 minus 50 equals 20, minus 1 equals 11. Wait wait wait, 62 minus 50 is 12, not 20! Silly.” (He often mixes up “twelve” and “twenty” in speech, but writes it correctly.)

“Could you do 62 – 51 using the arrow way?”

“No, you can’t. It’s impossible. You would get bored. Minusing 10s is boring.”

“Well, what about writing a minus 50 arrow?”

“You can’t do that! It’s not in the rule.”

“What if I told you that you could change the rule and use whatever numbers you want on the arrows?”

“Nah, you can’t do that. The rule stays as a rule. You can’t change it.”

“Let’s try one more subtraction, then we’re done. But you have to tell me out loud everything you do. Ok? It’s 100 minus 37.”

“Ok. 100 minus 30, then minus 7, equals 77.”

“What do you think?”

“77 is the answer.”

“How did you get that?”

“100 minus 30 is 70, then … no no no wait a second … it’s 63 instead of 77!”

“How did you get that?”

“Wah wah wah wahhhhhh.”

“Really?? Ok you got two different answers. 77 and 63. Are any of those the right answer?”

“63 is the real right answer.”

“What would you say to someone who got 77, to help them get the right answer?”

“Dude, just minus 7 and you’ll get the right answer.”

“What would this problem look like with arrows?”

“100 arrow -10 90 arrow -10 80 arrow -10 70 arrow -1 69 arrow -1 68 arrow -1 …”

“Okay wait. How many arrow -1s are there going to be?”

“Seven.”

“So why not just make one arrow -7 instead of seven arrow -1s?”

“No. You just can’t do that. It’s not in the rule. Boom bum bum!”

“Ok let me show you a choice. 100 arrow -30 70 arrow -7 63.”

“That’s not the way we do it in math class!!”

The Arrow Way

On the way home tonight, my second-grade son Aaron starting talking about something he’s learning called The Arrow Way. It’s part of the EngageNY curriculum.

Screen Shot 2015-11-17 at 6.46.53 PM

There’s a lot of positives in this representation, especially when it’s also coupled with number lines. But why couldn’t the problem above just skip the words “using the arrow way”? Or in problems like this, why require students to use this method?

Screen Shot 2015-11-17 at 6.55.46 PM

The result is that some kids think this is a “rule” or “law” instead of a representation/aid/solving method. This can cause misconceptions as kids focus on weird details like what is and isn’t allowed in the representation, rather than on tying the representation to existing knowledge and problems. The visual is nice, but once it’s named, it becomes as important as the other named things kids know in math, and at this point there aren’t many.

Here’s the conversation with Aaron (age 7).

“We have this new rule, it’s called The Arrow Way. The arrow way isn’t a story, it’s a math law.”

“What is it?”

“I don’t know.” (laughing)

“Come on, tell me for real.”

“The rule is +1, -1, +10, -10, and you gotta put one of these above the arrow.”

“So if I started with 50, what could I do?”

“Add 10 to make 60!” (He asked me to make the 60 bigger, “like I’m shouting.”)

“Okay. What else could you do, starting from 50?”

“Minus 10 to make 40! Or you could minus 1 to make 49! And there’s nothing else.”

“Ok. Could you start with 50 and make 70?”

“No. You can’t add 20, because it’s not in the rule.”

“Could you add 100?”

“No! That’s not in the rule!”

“Could you use more than one arrow to make 70?”

“No! Not at all.” (laughing)

“Can you use more than one arrow together?”

“I don’t know, but you can only start from zero, in the rule.”

“Ok. Is that true? Why can you only start from zero?”

“Because that’s a math law, also.”

 

On speed and fluency

I really enjoyed this article by former NCTM president Cathy Seeley, “Faster Isn’t Smarter”. It’s been interesting over the years to get to talk to many mathematicians that did not feel like they were good at math as children, because of their relatively slow speed.

Fluency matters, but it’s not real surprising that students expect all math problems to be solvable within seconds if they are given thousands of problems that are solved within seconds, and rewarded for how quickly or how many they can do within a time limit. Seeley also discusses issues that emerge in a timed, high-stress environment:

“Some students respond well to competitive and timed situations, thriving on the pressure to bring out their best; others have quite a different reaction. This particular boy received a clear message that some students are good at math and some are not—and he knew which group he was in. He also was prevented from finishing the test, something that causes some students tremendous frustration.”

The article offers some simple suggestions for changing this dynamic in favor of fluency without focusing on speed. Thanks!