The Calculus of the Classroom
beckyfisher73's posterous
Evaluating teachers and quarterbacks
As a kid, I sort of cracked the code for the
quarterback rating formula when I was in 4th grade or so. If you're
interested in this story, you can check out my Learning As a Hobby post. I also had what I considered to be good and bad teachers
throughout my school career. I have spent a fair amount of time
thinking about what makes teachers "good," but I haven't thought of this
in terms of a formula. Until now.
Here's a graphic from an article in the New York Times (login to a free account is required):
This is interesting to me. How do you measure and assign a value to "student characteristics"? Does a poor black kid living with his grandmother who makes him do his homework and be respectful to his elders count higher or lower than a kid who has two professional parents who will always side with their son because he, like them, is entitled to the best in life? How do you measure "true total school effect"? What is the "student error term"? Subscripts, superscripts, Greek letters as variables and summations. I don't buy it.
Here's how the NFL rates passers (per Wikipedia):
where mm(x) = max(0,min(x,2.375))
when
It's not the subscripts, superscripts, Greek letters, and mathematical operations that make these two formulas fundamentally different, it's the nature of the "data" and the interrelationships between the values that make one formula comprehensible and the other not for me.
In a 16 game regular season, an NFL quarterback may have a few passes where there is a difference of opinion as to whether it was caught or incomplete or intercepted. A few touchdowns may be called back because of a penalty or some other factor. But, by and large, there is wide acceptance to what it means to complete a pass, to attempt a pass, to intercept a pass, or to score a touch down. There is an accepted method for measuring the distance gained when a pass is completed. The commisioner, owners, players, coaches, referees, announcers, and fans share in this body of knowledge and pretty much accept the outcome when the final decision comes down.
Teaching and learning is not this calculable. What works for one student may not work for another. Not all kids have the same resources at home or at school. Not all teachers have the same level of support from their principals. Not all principals have access to a supportive central office or others who can help improve learning in their school. Not all PTOs can raise the same dollar amounts. Not all school libraries are powerful centers for meaning making.
Another thing that I find amazing in all of this is the notion of winning or losing a game regardless of how well the quarterback rated. Having a perfect quarterback rating for a game does not guarantee victory, which is what really counts in the end. Just ask Chad Pennington and Bobby Hebert - they both had perfect games according to the formula but walked away losers. And, the highest rated quarterback doesn't always make it to the playoffs. How will this play out for our teacher rating formulas?
Here's a graphic from an article in the New York Times (login to a free account is required):
This is interesting to me. How do you measure and assign a value to "student characteristics"? Does a poor black kid living with his grandmother who makes him do his homework and be respectful to his elders count higher or lower than a kid who has two professional parents who will always side with their son because he, like them, is entitled to the best in life? How do you measure "true total school effect"? What is the "student error term"? Subscripts, superscripts, Greek letters as variables and summations. I don't buy it.
Here's how the NFL rates passers (per Wikipedia):
where mm(x) = max(0,min(x,2.375))
when
It's not the subscripts, superscripts, Greek letters, and mathematical operations that make these two formulas fundamentally different, it's the nature of the "data" and the interrelationships between the values that make one formula comprehensible and the other not for me.
In a 16 game regular season, an NFL quarterback may have a few passes where there is a difference of opinion as to whether it was caught or incomplete or intercepted. A few touchdowns may be called back because of a penalty or some other factor. But, by and large, there is wide acceptance to what it means to complete a pass, to attempt a pass, to intercept a pass, or to score a touch down. There is an accepted method for measuring the distance gained when a pass is completed. The commisioner, owners, players, coaches, referees, announcers, and fans share in this body of knowledge and pretty much accept the outcome when the final decision comes down.
Teaching and learning is not this calculable. What works for one student may not work for another. Not all kids have the same resources at home or at school. Not all teachers have the same level of support from their principals. Not all principals have access to a supportive central office or others who can help improve learning in their school. Not all PTOs can raise the same dollar amounts. Not all school libraries are powerful centers for meaning making.
Another thing that I find amazing in all of this is the notion of winning or losing a game regardless of how well the quarterback rated. Having a perfect quarterback rating for a game does not guarantee victory, which is what really counts in the end. Just ask Chad Pennington and Bobby Hebert - they both had perfect games according to the formula but walked away losers. And, the highest rated quarterback doesn't always make it to the playoffs. How will this play out for our teacher rating formulas?
Chris Frank responded:
I love it! Do the algorithms have an opinion parameter? I'm just
thinking if I had to rank people, the best way is to get other people's
opinions about them... students, parents, other teachers. It wouldn't
be perfect but it would be uncommon for a really good teacher to score
low like this, or for a really bad teacher to score high. And
ranking teachers should be easier than ranking pro quarterbacks. The
difference between pro quarterbacks is tiny compared to differences
between teachers.
What would you do if you could invent the teacher ranking system?
What would you do if you could invent the teacher ranking system?
Guy Brandenburg responded:
Chris, I think you have it exactly backwards. Ranking teachers is much
harder than ranking quarterbacks. There are plenty of cases where a
teacher who I really disliked nevertheless did a good job of teaching me
stuff. In college, I found out the hard way that I could transfer out
of classes with professors I disliked or had big political/moral
disagreements with. But even the ones I strongly disliked, I learned
something in their class.
And the US Airforce Academy did an extremely valuable experiment in some of their classes (physics?) where they randomly assigned studednts to professors, for years, so the students couldn't pick and choose the way i used to do. They found, very consistently,t hat professors who got the highest evaluations were the ones who were able to earn the highest grades for their students -- grades largely being assigned by department-wide final exams and tests and so on -- but those high-ranking professors by those short-term measures were, quite surprisingly to many folks -- the absolute worst in promoting long-term understanding. And yes, this LTU was measured!!!
So what do you want? Long-term understanding and poor grades and low student evalua5tions, or high grades and good testr scores and positive s5tudent evaluaions, coupled with a serious lack of thoughtful insight and long-5term understanding of the basic fundamental concepts of the discipline?
And the US Airforce Academy did an extremely valuable experiment in some of their classes (physics?) where they randomly assigned studednts to professors, for years, so the students couldn't pick and choose the way i used to do. They found, very consistently,t hat professors who got the highest evaluations were the ones who were able to earn the highest grades for their students -- grades largely being assigned by department-wide final exams and tests and so on -- but those high-ranking professors by those short-term measures were, quite surprisingly to many folks -- the absolute worst in promoting long-term understanding. And yes, this LTU was measured!!!
So what do you want? Long-term understanding and poor grades and low student evalua5tions, or high grades and good testr scores and positive s5tudent evaluaions, coupled with a serious lack of thoughtful insight and long-5term understanding of the basic fundamental concepts of the discipline?
No comments:
Post a Comment