Three tiers of classrooms

Via Arnold Kling, I see Jesse Rothstein trying to prove that you can't measure teaching ability, or perhaps even that teaching ability doesn't matter:

Like all quasi-experiments, this one relies on an assumption that the treatment – here, teacher switching – is as good as random. I find that it is not: Teacher switching is correlated with changes in students’ prior-year scores.

It's important to figure out which kind of classroom we are talking about. There are at least three tiers of classroom styles. If you measure only in the middle tier, then I can believe that teacher skill would have only a small effect. However, it's really easy to tell the difference between the tiers if you look, especially for the bottom-most tier compared to the other ones.

At the bottom tier, some classes are just zoos. The teacher is ignored, and the students talk to each other. At best, they work on material for another class. Teacher skill doesn't matter within this tier, from at academic perspective; one zoo teaches students just as much as another zoo. I am sad to say that classrooms like this do exist. It's a potential bright note that such teachers are very easy to identify in an objective way; their students have absolutely terrible results on standardized tests such as Advanced Placement (AP). There's no need for sophisticated statistics if all the students are scoring 1-2 out of 5.

At the middle tier, some classes involve the teacher walking the students through standardized textbooks and other material. Basically, the textbooks are software and the teachers are the hardware that runs it. It's not an inspiring kind of classroom, but at least it is inexpensive. Within this tier, I could see teacher skill not mattering much, because the students spend all their time glued to the course materials. However, you'd certainly like to find out who is in this tier versus in the zoo tier.

Worth a brief mention is that there's an upper tier as well. Maybe "style" is a better word in this case. Sometimes the teacher actually understands the course material, and so is able to respond to the questions with anecdotes and exercises that are tailored for that particular student. For this tier, teacher evaluation is especially important. Among other things, some teachers are fooling themselves, and would be better off staying closer to the book.

Learning can be measured

I've served as a teacher in a number of roles. I've taught math in a private high school, and I've taught undergraduate and graduate computer science. I also tutored constantly throughout all of my own schooling. Based on this experience, I'd like to emphasize one stance in the discussions that are going around about education reform this (and every) election season.

Learning can be measured.

Teachers know how to test their students to see whether they're learning what is intended. When I taught trigonometry and linear algebra, it really wasn't that hard to figure out which students were able to do it and which weren't. I gave them sample problems, gave them an hour, and then look at how they did on it. This gave tremendous insight into what the capabilities of the people in the class were. Any teacher who can't do this is basically failing at their job. It's just part of what teachers do.

Standardized tests are also pretty good. Granted, they have their problems. The questions leave little room for the grader to use judgment, and the graders don't have any extra information about the students than what is on the test. However, standardized tests also have benefits. The questions are much better devised and worded. They probe the student's skills in more ways, and so that answers to the questions more clearly indicate how the student is doing. The test makers have a larger view of their field than any individual teacher, so they avoid the temptation to grind an ax about some particular sub-sub-sub-topic. Additionally, the same lack of judgment that the graders have means that the grades are more objective. It is a more subtle story than I should get into in this post, but suffice to say that an apple a day for your teacher really does make a difference. Standardized tests can pierce through the reputation bubbles within a school and see how each student is really performing.

As it works out in practice, I have to say that standardized tests are quite good at measuring knowledge level, possibly even better than the home-grown tests. Most of my experience with standardized tests is at the high school level, but in that experience they're pretty good. I and my fellow students got exactly the grades that would be expected based on what we knew: we did well on standardized tests in our best areas, and we did badly in areas we didn't know so well. Further, from the talking I've done with more experienced high school teachers, they believe the tests, too. They can, more often than not, guess the exact grade on a scale of 1-5 that any student will get on an AP exam.

In short, measuring learning isn't too hard if you are willing to use standardized tests. Look at how the students do at the beginning and end of the year, and you'll know how well the teacher taught them.

I believe most teachers would agree with all of the above, but they say the opposite when it comes to measuring teachers themselves. I suppose no one likes oversight.