I’ve lost track of who on social media pointed me to this, but this blog post about testimony to the Michigan Legislature is a brilliant demonstration of what’s so difficult about teaching even simple subjects. Deborah Ball, the Dean of the education school at the University of Michigan gives the legislators a simple grading exercise from elementary school math. The video here is worth a watch:
(This also includes one of the greatest failed SNL references ever. It flops badly enough that the guy responsible is nearly as embarrassed as he ought to be…)
The problem she’s illustrating is one that plagues graders at all levels: while there are a finite number of ways of arriving at the correct answer, the number of ways to do even simple questions incorrectly is essentially infinite, and some of them are perversely ingenious. And one of the key tasks of an educator is to provide useful feedback, which in many cases means trying to figure out what the hell the student did to get the wrong answer they got.
This is one of the many reasons why I try to get students not to plug numbers into their calculator from the very start of a problem. If I get a problem set that works things through algebraically, I can usually spot the error fairly easily, but when students are manipulating nine-digit decimals, going from one to the next via their calculator, it’s nearly hopeless. The occasional arrogant student employing the Feynman Method (“First you write down the problem. Then you think very hard. Then you write down the answer.”) really is hopeless.
This is also a tricky problem for exam design. When I have time to make up exam questions at leisure, I try to go through some of the obvious wrong methods as well– punching angles in radians into a calculator set to degrees, using sine instead of cosine, etc.– to make sure that the wrong answers are clearly distinguishable. Sadly, most of the time, I don’t really have the time to do that, and on occasion I’ve ended up with questions where one of the common wrong paths leads to an answer that’s within 10% or so of the correct answer. That’s a grading nightmare, as it becomes really hard to sort out genuine math and physics errors from idiosyncratic rounding behavior.
I will admit that I’m not as good about this as I probably should be– I frequently punt and write “Math Error” in the margin, meaning “You’ve ended up with the wrong numerical values, and I don’t understand why.” In my current standards-based grading scheme, I’ve also added a separate standard for “Correctly performing calculations to get numerical answers,” that shows up on any question with a numerical answer. That gets me out of trying to figure out what weird combination of calculator keystrokes could lead from correct equations to wrong numbers.
(I had a student once who was off by an order of magnitude on all his calculations, which I initially guessed must be a units issue. He was adamant that he was right, though, and we spent fifteen minutes going over his calculations before I realized that when he had to enter a number like “6×107” into his calculator, rather than hitting “6 EE 7” he was putting in “6 x 10 EE 7” and thus actually calculating 60×107. The calculation in question involved multiplying three numbers that were in scientific notation, and dividing by the product of two other numbers in scientific notation, which is how he ended up with all his answers off by a factor of 10…)
The infinite variety of wrong answers and the need to figure out the process behind them is a constant and under-appreciated source of angst for faculty at all levels, and it’s nice to see it demonstrated so clearly in a public forum. And it’s something to keep in mind when people start ranting about what a cushy job teachers have and how anybody could do that.
I have also encountered that calculator error. Don’t they get to use calculators in k-12? How did they get a high school diploma while getting everything wrong in math class?
Wait, don’t answer that.
Alex, at least with the students I had it was because my calculators weren’t like the math teacher’s calculators. They weren’t an order or two off, they would be 10^46 off calculating moles to atoms, or vice versa, and say “but that’s what the calculator said.”
they would be 10^46 off calculating moles to atoms, or vice versa
It’s easy to see how they get an error like that: flipping the sign on the exponent of Avogadro’s number would do the trick.
Figuring out what the student who turned in example A from the original post was thinking is a harder problem. I saw the video, but had trouble following the line of reasoning the witness laid out for that one. To me it looked as if the student multiplied 45 * 9 and 54 * 2–though why he thought that was the correct approach, I have no idea.
The Feynman method reminds me of the Ramanujan method. When someone once mentioned a certain problem to him (while he was cooking dinner) after a second or so Ramanujan gave some complicated infinite continued faraction as the answer. When the astonished questioner asked how he could figure out the answer so quickly Ramanujan replied – It seemed to me that the answer should be some sort of infinite continued fraction so I simply asked myself what continued fraction could it be and the answer instantly occurred to me.
Grading math problems can be very time consuming when one has to give partial credit for answers. When I graded math exams and homework long ago almost nobody did the problems correctly so after going through them once I had to go back over them to make sure I was reasonably consistent in how I was giving partial credit.
I come across the 6 × 10 EE 7 error pretty regularly.
Eric, I knew exactly what they did wrong, what bothered me was the insistence that it had to correct because it was what the calculator said (but they didn’t put it in the calculator correctly) and the complete lack of estimation skills (large number of atoms is still a small number of moles).
Definitely not as hard as figuring out what the students in those examples got wrong. I figured out a and b but not c, yet.
Marciepooh@6:
C) is doing 50×25 because it’s easier than 49×25, then subtracting 25.
But he’s ADDING instead of subtracting in the last step.
Apparently, besides a class in arithmetic, there now has to be a class in calculator…
Thanks, John. I’d guessed that but got stuck by why he’d accidentally add instead of subtract at the end. Being very far removed from 3rd/4th grade math, it seems to me any one who realizes (50×25)-(1×25) = (49×25) wouldn’t forget to subtract, but, again, I’m not a 3rd or 4th grader.
More on the topic of teachers needing to understand why something is correct, or incorrect, is important – I thought it was odd when my undergraduate program didn’t require comprehensive science ed majors take physics with calculus. Seemed to me the teacher needed to understand the math, even if she never taught it. I also had a head-desk moment when I heard a one of the CompSciEd students on field trip talking about how she didn’t see why her advisor had a problem with her being placed in a physics classroom for her student teaching* while she was taking the 1st semester of physics at night.
*It is possible the placement she was referring to was not her actual student teaching but one of the many other in-school placements, but from what she said it sounded like she was approaching graduation.
Eric, did you eventually figure out the issue in example A?
For those who haven’t:
5 x 9 = 45; write down 5, carry the 4. But instead of carrying the 4 to the tens column beside that 5, they carried it back to the original multiplication problem (the 49) and added it to the 4 to make 8 (well, 80).
Then they multiplied 5 x 8 (the ‘4’ of 49, plus the 4 they carried) to get the ’40’ part of ‘405’.
Then they did exactly the same thing for the next part, carrying (and adding) the ‘1’ from 2 x 9 to the ‘4’ of 49, which is how they got the ’10’ (2 x 5) of 108.
As Chad put it, rather ingeniously wrong. And at least they were consistent about it.
what bothered me was the insistence that it had to correct because it was what the calculator said (but they didn’t put it in the calculator correctly)
My response to any such student would be to point out the old GIGO rule: Garbage In, Garbage Out. It doesn’t matter whether they get the result from a calculator, a spreadsheet, or some more specialized program. The computer will always do what you tell it to do, which may or may not be what you want it to do. Every student needs to learn this lesson.
I don’t know what to do about people who have no sense of “That can’t possibly be right”, but it’s far more common than it should be. I’ve even seen it in physics grad students. Back when I was taking E&M (Jackson), we had a homework problem in which I obtained a result I knew couldn’t be right (there was a discontinuity at the origin which should not have been there), but failed to find the error in my math. So I asked a fellow student, who walked me through the problem to his solution, which I promptly informed him was the same wrong answer I had obtained–he didn’t notice the discontinuity. It wasn’t until a class session where the problem was discussed that we discovered the error we had independently made: if the coordinate system is non-Cartesian, unit vectors have nonzero spatial derivatives, and if you include those terms in the calculation you get a physically reasonable result.
@Wilson: Yes, it makes sense once it’s written out. But the explanation in the video went by very quickly, and since I have not been involved in a multiplication drill in mumblety-mumble years, that kind of mistake would never have occurred to me. Whereas in B it was obvious what went wrong (failure to shift an intermediate result over by one column), and the shortcut the student who did C attempted (and botched) was something I might have done.
The educational curriculum gets a lot of ridicule, and much of it is deserved. But there are reasons prospective elementary school teachers are required to take some of the classes they do, and if one of them is to learn to spot this kind of error, then that’s a good reason for taking that class.
Remember when it was all done with logarithm tables and slide rules? 9.3010-10 was good old .2? We were still doing that when I first started teaching.
I tend to tell my students that the calculator is their enemy. They don’t usually believe me, no matter how many times they screw up using it.
@Alex: Yup. They use calculator in K through 12. In many places, that is ALL they do. No arithmetic without. The complete lack of number sense means that not only do they still make mistakes with the calculator, like everyone does, they don’t have the skills to recognize that the answer is ridiculous.
Unfortunately, many (I would say most, in my experience) grammar school teachers have no knowledge or understanding of math. I know several that are proud of their lack of knowledge. Arithmetic skills are unimportant, in their view, because the calculator `does’ the math. This makes for a very difficult time by the time the student sees algebra and geometry, as the basic tools are the same, and if a student is using a calculator for 6*8, he or she is breaking focus on the actual problem at hand.
That said, I have told students, when doing vector field problems, substitute as early as possible, but no earlier, as otherwise the algebra may become cumbersome. For example, iIf all you need is the unit vector for curl at a point, take the partials and sub in the point. Then evaluate…
I suspect the reason they get through high school not knowing how to do scientific notation on a calculator is a combo of (1) not doing much with scientific notation in k-12 and (2) the test on scientific notation is graded by a teacher who didn’t know how to use the calculator either and came up with a flawed answer key.
I have mentioned this elsewhere but I used to teach undergraduate science students. It always amused me to stand around as they worked out the answer to a problem on their calculators. They would furiously click away doing the several serial calculations and come up with some answer such as 1.2 (which may have been correct). I would just look at them and frown slightly and they would furiously go through the same calculation and come up with 1500 say. Another frown and another furious key punching session and yet another completely different number. And so on. Some eventually realized the problem, but a lot never seemed to get the idea that they should be able to get the correct answer (with their set of data) and expected me to tell them which of their attempts was the correct one.
I bet the abacus (soroban) would be better pedagogically than calculators. And cheaper, too!
One observation this post brings to mind is that students are often not taught how to do tests, and so miss out on points they could easily get by setting out their approach to test problems clearly on paper. Of course, the students need to have an idea of how much points they can gain from this, or else time constraints will lead them to maximise the amount of questions answered, with obvious consequences to their noting of the methods they used to arrive at the answers.