Here's something that I'm finding frustrating: for all the mountain of ink spilled on the issue of remedial math in colleges (including enormous numbers taking them, the fact that it's the critical determination of whether people get a college degree or not, dim prospects of existing placement tests, etc., etc., etc.), when I search for papers where someone has tried to correlate specific math skills of incoming students to success in college remedial algebra -- I come up totally empty.
Weirdly, I can find studies that correlate specific diagnostic test questions in basic math skills to other classes. Here's one relating specific math skills to success in college statistics classes. Here's another. Here's a study relating basic math skills to success in economics classes.
But predicting success in basic algebra classes? I'm coming up totally empty. I'm truly bewildered at this -- part of me can't possibly believe that no one has published results like that, but part of me is stewing from returning to this futile search many days over and over again.
Does anyone know of such research linking specific skill questions to success in college remedial algebra? Or any college algebra classes?
2014-02-17
Precipitation Probability
This winter module I've had a batch of students in my introductory statistics course who are so aggressively intelligent that they've spotted every single spot where I had any gray area or ambiguity in my lectures. In places I do this knowingly to simplify the subject, and prepare backup answers in case anyone asks -- this semester is the first time where every single one of backups got used, and then some. This will definitely benefit my classes in the future, and in fact, I learned a few things myself along the way. For example:
At the end of the probability concepts section, the major thing I want students to do is to interpret probability statements (which for some is the most difficult part of the course, never having encountered probability concepts before). I give a quiz question on the classic weather forecast precipitation probability: "Interpret this probability statement: 'There is a 40% chance of rain today in the New York area'". So personally, I always took this to mean that there was a 40% chance of getting any rain at all in New York today (40% chance for a drop of measurable rain somewhere in New York; i.e., over many days like today 40% of such days will get a drop of rain or more in New York).
But one of my students not only started researching this on her own, she actually called the New York weather service to ask a meteorologist how this was computed. She still didn't get the interpretation quite right (one of the few questions she missed all semester), but the discussion was enlightening for both of us.
The truth is that the weather forecast statement is in regards to rain at any random location in New York, not actually the rain for New York as a whole. I suppose that is really a more useful statement, after all. The publicized percentages are computed by multiplying the expected coverage area percent by the probability of rain occurring in that area (so if it's 40% likely that 100% of the area gets rain, you report the same result as an 80% chance that 50% of the ground gets rain). Therefore: What's being reported is the chance that any arbitrary point in New York gets measurable rain; i.e., 40% indicates that for any random point in New York, if we observe many days with conditions like today, 40% of such days get a measurable drop of rain on that point-location.
Links to more information:
At the end of the probability concepts section, the major thing I want students to do is to interpret probability statements (which for some is the most difficult part of the course, never having encountered probability concepts before). I give a quiz question on the classic weather forecast precipitation probability: "Interpret this probability statement: 'There is a 40% chance of rain today in the New York area'". So personally, I always took this to mean that there was a 40% chance of getting any rain at all in New York today (40% chance for a drop of measurable rain somewhere in New York; i.e., over many days like today 40% of such days will get a drop of rain or more in New York).
But one of my students not only started researching this on her own, she actually called the New York weather service to ask a meteorologist how this was computed. She still didn't get the interpretation quite right (one of the few questions she missed all semester), but the discussion was enlightening for both of us.
The truth is that the weather forecast statement is in regards to rain at any random location in New York, not actually the rain for New York as a whole. I suppose that is really a more useful statement, after all. The publicized percentages are computed by multiplying the expected coverage area percent by the probability of rain occurring in that area (so if it's 40% likely that 100% of the area gets rain, you report the same result as an 80% chance that 50% of the ground gets rain). Therefore: What's being reported is the chance that any arbitrary point in New York gets measurable rain; i.e., 40% indicates that for any random point in New York, if we observe many days with conditions like today, 40% of such days get a measurable drop of rain on that point-location.
Links to more information:
- Comments from the National Weather Service, reposted at the University of Texas at Austin website: here.
- Video from a meterologist in Boulder, including citation to the 2005 Gigerenzer et. al. paper in Risk Analysis which surveyed people for their understanding of these statements (where I got my quiz question in the first place): here.
2014-02-03
Teacher Guilt and Grading Workload
Is this a closely-kept secret? I think that all of the college math instructors I know really intimately (I'm counting about four, including myself) have at some point admitted to an overwhelming stack of grading assignments that they've procrastinated on, and a painful amount of guilt that they've experienced over that apparent failure. One instructor told me in passing once that he basically had a nervous breakdown over Thanksgiving break, over not being able to accomplish all the grading he needed to. Which was freakishly familiar, because the exact same thing basically happened to me, several years ago.
I think this advice (like much of what I write here) goes out to new instructors, just starting their career, on the off chance they internet-search for this one day. So here we go: Your students absolutely deserve prompt feedback on their work, ASAP. But you also deserve a reasonable quality of life, not absolutely drowning in work and falling behind all the time. If you're not getting your grading done promptly, then you've got to be sensitive to that and change your assignments such that they're gradable in reasonable time.
Note this goes regardless of whatever pedagogical fashion is currently happening, or whatever suggestions you've received for proper assignments to give. It must be doable in the time that you have, full stop. The top priority basically always has to be honesty about time management; if you can't do it, then you can't do it, and you need to admit that and change it.
Here's how it started happening for me: When I was graduate student and given a few sections of college algebra to teach for the first time, my adviser (who was a really great guy) recommended what he did for homework -- have students keep a notebook journal of homework problems, turn in the notebook every week or two, and "check it off". Well, this turned into me lugging a giant box of notebooks home from school in the second week and it just sitting at home. Probably I attempted to "check it off" a few times and was so appalled and bewildered at the entirely undecipherable, jumbled scribblings in the first several books (possibly with answers from the back of the book transcribed at the end) that I just didn't see how I could possibly read, decipher, assess, and adjudicate this giant mass of outrageous nonsense (as I now interpret it). My mentor said it was "easy" for him, but it seemed utterly impossible for me.
Moreover, what counted as acceptable work? If a student came and challenged me on not being "checked off", how could I defend that, or explicate what line needed to be crossed? All absolutely reasonable concerns. Well, then I was committing to assessing each problem in some fashion individually, scoring up a particular ratio of acceptable problems for a pass/fail check, providing rubrics, peering closely at every line of written transformations, etc. Or at least in theory I was: it was impossible, and every day I'd slump into class and mumble a shameful excuse about why the notebooks weren't coming back, and probably get around to it about twice a semester. A task that put dread over me literally all semester long.
And you know -- this tradition more or less lasted for some years after I started teaching professionally for work. The process evolved by my first cutting down the number of problems to a fairly specific list that I expected to assess problem-by-problem. I went and ordered custom rubber pass/fail stamps to try and expedite the system; but an ever-present problem was even being able to find specific problems on some students' jumbled-up paper. Then I reduced it to about 3 specific problems I assigned each week on a one-page worksheet I designed, with dedicated space to put each problem, a completely worked-out example to show the correct format, and exactly 10 lines of work that I would assess line-by-line (passing would need 6/10 lines exactly correct). But even this requirement overwhelmed students in the basic algebra class, and there was constant combativeness around the assessment. Some could never learn to use an equals sign on each line. Most classes would find one person who understood the assignments and all be copying their page when I walked into the classroom.
There was some weekend a few years ago where it felt like I went basically berserk over the issue -- I just couldn't deal with it any more. I think this would be when I first switched from part-time to full-time, so my courseload doubled. Even the reduced one-page assignments were not manageable, I still had a sense of dread all the time, it didn't seem to help my students at all, and mostly all I got for my feedback was grief.
End of the story -- those assignments simply had to freaking stop. The most honest truth that I finally realized was this: my remedial students need a monumental amount of work and practice to overcome their deficiencies, and I don't have anywhere near the amount of time in my life to assess all of that vast amount of work that they have to do. The responsibility has to be put on them -- even if the majority of remedial math students are going to fail at the challenge.
The new protocol is this: There is a list of homework assignments that they're expected to do, all with answers to check at the back of the book, and if something doesn't work right or they have questions, then they can ask in class. I don't collect or grade this homework; there is too much for me to deal with. I'm usually needling just a bit at the start of every class; if no one asks about any exercises then there's some uncomfortable silence that I let settle. But usually I get one or two students who are asking questions, and then my time spent responding is actually helping someone who does want it, as well as the rest of the class, and also setting an example for proper study skills. As semesters pass, it seems like I get somewhat better traction and momentum with this, with more students actually participating. (I guess I write this tonight after multiple students in my statistics class asked about a bungled textbook problem that's been on my syllabus for 3 years now -- tonight was the first time anyone brought it up, slightly embarrassing for me, but otherwise beneficial to my future classes.) Also, I use our Blackboard system to deliver 5 multiple-choice quiz questions to the students every week -- entirely automated grading, so it pops up in my digital gradebook without any effort on my part, complete with comprehensive statistics on what the hardest parts were -- keeping the students to some required attention every week, without spending any more of my home or class time on the process.
As another example, this semester I switched my college algebra tests from multiple-choice to open-response (grading on quality of writing/justifications as well as raw answers). Right before I did this, I had another instructor warn me to not make it too burdensome on myself (a reasonable concern!). But I didn't just add work for myself: I cut the size of the tests to a level that would be easy for me to grade. Instead of 20-question multiple choice tests (like most instructors here use), I give 10-question open-response tests. Namely, the hardest 10 questions in the block -- no rinky-dink warm-up problems (like trivial linear equations or simply adding polynomials). But the scoring system is simply for each each question: 1st point for the correct answer, and a 2nd point for well-written justification (or maybe 1 point for a single error, 0 points on the second error). Each point is 1/20 = 5% of the test, absolutely laser-fast to score and add up. (There's no fiddling with granularities less than 5%; I don't have time for that.) It takes me about an hour, maybe two, to grade and give feedback to all the problems for all the students in a section on one test cycle.
So here's where I am today: When I give a test I cannot wait to get started grading it. I'm almost over-eager to see how my students are doing, and curious to see what's working well and what we can brush up and improve in the future. I know that the grading will go quickly and be productive, and I will be getting data about how the class is progressing very soon. Separately, I'm almost addicted to checking in on the online quiz progressions, following the statistics of which problems are hardest.
The workload has flipped from dreadful to highly desirable, and I look forward to getting student work whenever I can now. I think I've had some test problems that were hard to grade (I can't think of what they were right now), but then I pull them out of rotation and replace them with something more reasonable to assess. I basically don't have arguments about grading anymore; the points are specified in advance on a practice test, and it's all very easy to see where everything is coming from.
Last semester, I actually had one student express surprise and near-disbelief at how quickly I got graded tests back to students; namely, the very next day without fail. She asked me how I could do that when all of her other professors took at least a week to do the same thing. My answer was something like I was really curious about how my students were doing and couldn't wait to find out. (Truth be told, my tests are usually graded a few hours after I give them, and results are online usually around 2-3am after my night classes.)
So I offer that as a success story of someone who's gone from crushing years-long ever-present guilt and dread over grading, to where I almost can't get started at it soon enough to satisfy me. The key is to budget time first, to be honest about what you can do, and to cut and design assignments to a level where you can grade them with a sense of joy.
Have we all gone through this trip through the valley of dread? Have you?
Related question on Stack Exchange: Academia, regarding my ongoing inability to understand how anyone makes the "check for completion" protocol work: "What are the minimum criteria when checking homework for completion only?"
I think this advice (like much of what I write here) goes out to new instructors, just starting their career, on the off chance they internet-search for this one day. So here we go: Your students absolutely deserve prompt feedback on their work, ASAP. But you also deserve a reasonable quality of life, not absolutely drowning in work and falling behind all the time. If you're not getting your grading done promptly, then you've got to be sensitive to that and change your assignments such that they're gradable in reasonable time.
Note this goes regardless of whatever pedagogical fashion is currently happening, or whatever suggestions you've received for proper assignments to give. It must be doable in the time that you have, full stop. The top priority basically always has to be honesty about time management; if you can't do it, then you can't do it, and you need to admit that and change it.
Here's how it started happening for me: When I was graduate student and given a few sections of college algebra to teach for the first time, my adviser (who was a really great guy) recommended what he did for homework -- have students keep a notebook journal of homework problems, turn in the notebook every week or two, and "check it off". Well, this turned into me lugging a giant box of notebooks home from school in the second week and it just sitting at home. Probably I attempted to "check it off" a few times and was so appalled and bewildered at the entirely undecipherable, jumbled scribblings in the first several books (possibly with answers from the back of the book transcribed at the end) that I just didn't see how I could possibly read, decipher, assess, and adjudicate this giant mass of outrageous nonsense (as I now interpret it). My mentor said it was "easy" for him, but it seemed utterly impossible for me.
Moreover, what counted as acceptable work? If a student came and challenged me on not being "checked off", how could I defend that, or explicate what line needed to be crossed? All absolutely reasonable concerns. Well, then I was committing to assessing each problem in some fashion individually, scoring up a particular ratio of acceptable problems for a pass/fail check, providing rubrics, peering closely at every line of written transformations, etc. Or at least in theory I was: it was impossible, and every day I'd slump into class and mumble a shameful excuse about why the notebooks weren't coming back, and probably get around to it about twice a semester. A task that put dread over me literally all semester long.
And you know -- this tradition more or less lasted for some years after I started teaching professionally for work. The process evolved by my first cutting down the number of problems to a fairly specific list that I expected to assess problem-by-problem. I went and ordered custom rubber pass/fail stamps to try and expedite the system; but an ever-present problem was even being able to find specific problems on some students' jumbled-up paper. Then I reduced it to about 3 specific problems I assigned each week on a one-page worksheet I designed, with dedicated space to put each problem, a completely worked-out example to show the correct format, and exactly 10 lines of work that I would assess line-by-line (passing would need 6/10 lines exactly correct). But even this requirement overwhelmed students in the basic algebra class, and there was constant combativeness around the assessment. Some could never learn to use an equals sign on each line. Most classes would find one person who understood the assignments and all be copying their page when I walked into the classroom.
There was some weekend a few years ago where it felt like I went basically berserk over the issue -- I just couldn't deal with it any more. I think this would be when I first switched from part-time to full-time, so my courseload doubled. Even the reduced one-page assignments were not manageable, I still had a sense of dread all the time, it didn't seem to help my students at all, and mostly all I got for my feedback was grief.
End of the story -- those assignments simply had to freaking stop. The most honest truth that I finally realized was this: my remedial students need a monumental amount of work and practice to overcome their deficiencies, and I don't have anywhere near the amount of time in my life to assess all of that vast amount of work that they have to do. The responsibility has to be put on them -- even if the majority of remedial math students are going to fail at the challenge.
The new protocol is this: There is a list of homework assignments that they're expected to do, all with answers to check at the back of the book, and if something doesn't work right or they have questions, then they can ask in class. I don't collect or grade this homework; there is too much for me to deal with. I'm usually needling just a bit at the start of every class; if no one asks about any exercises then there's some uncomfortable silence that I let settle. But usually I get one or two students who are asking questions, and then my time spent responding is actually helping someone who does want it, as well as the rest of the class, and also setting an example for proper study skills. As semesters pass, it seems like I get somewhat better traction and momentum with this, with more students actually participating. (I guess I write this tonight after multiple students in my statistics class asked about a bungled textbook problem that's been on my syllabus for 3 years now -- tonight was the first time anyone brought it up, slightly embarrassing for me, but otherwise beneficial to my future classes.) Also, I use our Blackboard system to deliver 5 multiple-choice quiz questions to the students every week -- entirely automated grading, so it pops up in my digital gradebook without any effort on my part, complete with comprehensive statistics on what the hardest parts were -- keeping the students to some required attention every week, without spending any more of my home or class time on the process.
As another example, this semester I switched my college algebra tests from multiple-choice to open-response (grading on quality of writing/justifications as well as raw answers). Right before I did this, I had another instructor warn me to not make it too burdensome on myself (a reasonable concern!). But I didn't just add work for myself: I cut the size of the tests to a level that would be easy for me to grade. Instead of 20-question multiple choice tests (like most instructors here use), I give 10-question open-response tests. Namely, the hardest 10 questions in the block -- no rinky-dink warm-up problems (like trivial linear equations or simply adding polynomials). But the scoring system is simply for each each question: 1st point for the correct answer, and a 2nd point for well-written justification (or maybe 1 point for a single error, 0 points on the second error). Each point is 1/20 = 5% of the test, absolutely laser-fast to score and add up. (There's no fiddling with granularities less than 5%; I don't have time for that.) It takes me about an hour, maybe two, to grade and give feedback to all the problems for all the students in a section on one test cycle.
So here's where I am today: When I give a test I cannot wait to get started grading it. I'm almost over-eager to see how my students are doing, and curious to see what's working well and what we can brush up and improve in the future. I know that the grading will go quickly and be productive, and I will be getting data about how the class is progressing very soon. Separately, I'm almost addicted to checking in on the online quiz progressions, following the statistics of which problems are hardest.
The workload has flipped from dreadful to highly desirable, and I look forward to getting student work whenever I can now. I think I've had some test problems that were hard to grade (I can't think of what they were right now), but then I pull them out of rotation and replace them with something more reasonable to assess. I basically don't have arguments about grading anymore; the points are specified in advance on a practice test, and it's all very easy to see where everything is coming from.
Last semester, I actually had one student express surprise and near-disbelief at how quickly I got graded tests back to students; namely, the very next day without fail. She asked me how I could do that when all of her other professors took at least a week to do the same thing. My answer was something like I was really curious about how my students were doing and couldn't wait to find out. (Truth be told, my tests are usually graded a few hours after I give them, and results are online usually around 2-3am after my night classes.)
So I offer that as a success story of someone who's gone from crushing years-long ever-present guilt and dread over grading, to where I almost can't get started at it soon enough to satisfy me. The key is to budget time first, to be honest about what you can do, and to cut and design assignments to a level where you can grade them with a sense of joy.
Have we all gone through this trip through the valley of dread? Have you?
Related question on Stack Exchange: Academia, regarding my ongoing inability to understand how anyone makes the "check for completion" protocol work: "What are the minimum criteria when checking homework for completion only?"
Subscribe to:
Posts (Atom)