Learning can be measured

I've served as a teacher in a number of roles. I've taught math in a private high school, and I've taught undergraduate and graduate computer science. I also tutored constantly throughout all of my own schooling. Based on this experience, I'd like to emphasize one stance in the discussions that are going around about education reform this (and every) election season.

Learning can be measured.

Teachers know how to test their students to see whether they're learning what is intended. When I taught trigonometry and linear algebra, it really wasn't that hard to figure out which students were able to do it and which weren't. I gave them sample problems, gave them an hour, and then look at how they did on it. This gave tremendous insight into what the capabilities of the people in the class were. Any teacher who can't do this is basically failing at their job. It's just part of what teachers do.

Standardized tests are also pretty good. Granted, they have their problems. The questions leave little room for the grader to use judgment, and the graders don't have any extra information about the students than what is on the test. However, standardized tests also have benefits. The questions are much better devised and worded. They probe the student's skills in more ways, and so that answers to the questions more clearly indicate how the student is doing. The test makers have a larger view of their field than any individual teacher, so they avoid the temptation to grind an ax about some particular sub-sub-sub-topic. Additionally, the same lack of judgment that the graders have means that the grades are more objective. It is a more subtle story than I should get into in this post, but suffice to say that an apple a day for your teacher really does make a difference. Standardized tests can pierce through the reputation bubbles within a school and see how each student is really performing.

As it works out in practice, I have to say that standardized tests are quite good at measuring knowledge level, possibly even better than the home-grown tests. Most of my experience with standardized tests is at the high school level, but in that experience they're pretty good. I and my fellow students got exactly the grades that would be expected based on what we knew: we did well on standardized tests in our best areas, and we did badly in areas we didn't know so well. Further, from the talking I've done with more experienced high school teachers, they believe the tests, too. They can, more often than not, guess the exact grade on a scale of 1-5 that any student will get on an AP exam.

In short, measuring learning isn't too hard if you are willing to use standardized tests. Look at how the students do at the beginning and end of the year, and you'll know how well the teacher taught them.

I believe most teachers would agree with all of the above, but they say the opposite when it comes to measuring teachers themselves. I suppose no one likes oversight.

Scientific medicine

Thorfinn of Gene Expression has a great post up on the difficulty of generating knowledge, even in a relatively hard science like medicine:

Doctors believe in breaking fevers, though there is no evidence that helps. Flu shots also don’t seem to work. I’ve also mentioned how uclers came to be declared a disease due to “stress”, when in fact they were clearly due to bacterial infection. Meanwhile, several large-scale tests of medicine use — from the RAND insurance study, or the 2003 Medicare Drug expansion — find minimal evidence that more medicine leads to better health.
[...]
I think our body of medical knowledge does illustrate how hard it can be to generate reliable knowledge, even in cases when we can easily run numerous experiments on a randomized basis.

Softer sciences have an envy of the hard sciences. Their researchers envy how reliable the experimental results are in a physics or chemistry experiment. In the hard sciences, it's possible to do controlled experiments where all of the relevant variables are controlled. Further, the models are simple enough that there aren't a host of alternative models that can explain any experiment. For example, if your theory is that the acceleration due to gravity is the same for all masses of objects, and your experiment is consistent with that theory, it's hard to come up with any simpler theory that would explain the same thing. "It doesn't matter" is already as simple as it gets.

I spent a lot of time with the Learning Sciences group at Georgia Tech. While they put an admirably high effort into careful experimental validation of their tools, methods, and theories, they were quite frank that the experimental data were hard to draw inferences from. They could describe a situation, but they couldn't reliably tell you the why of a situation.

The problem is that even with randomized trials, there are so many variables that it's hard to draw any strong conclusions. There is always a plausible explanation based on one of the uncontrolled variables. For learning sciences, a particularly troublesome variable is the presence of an education researcher in the process. Students seem to always do better when there's an experimenter present. Take away the experimenter, and the whole social dynamic changes, and that has a bigger effect than the particular tool. Seymour Papert's Mindstorms is a notorious example. Papert paints a beautiful picture of students learning deep things in his Logo-based classrooms, a picture that has inspired large numbers of educators. I highly recommend it to any would-be teacher. However, nobody can replicate exactly what he describes. It seems you need Papert, not just his tools, and Papert is darned hard to emulate.

All too often we focus on a small effect that is dwarfed by the other variables. The teacher, the software engineer, and the musician are more important than the tools. In how many other areas of knowledge have we fallen into this trap? We ask a question that seems obviously the one to ask--Logo, or Basic? Emacs, or vi? Yet, that question is framed so badly that we are doomed to failure no matter how good are experiments are. We end up comparing clarinets to marimbas, and from that starting point we'll never understand harmony and rhythm.

Against an Internet Blacklist

There is a bill in the U.S. Senate to set up a blacklist for American citizens:

The main mechanism of the bill is to interfere with the Internet's domain name system (DNS), which translates names like "www.eff.org" or "www.nytimes.com" into the IP addresses that computers use to communicate. The bill creates a blacklist of censored domains; the Attorney General can ask a court to place any website on the blacklist if infringement is "central" to the purpose of the site.

To draw an analogy, this is like ordering someone's phone line to be disconnected based on a simple court order. It's not a good plan even if it were limited to sites that were clearly infringing copyright. Shouldn't the site owner get a day in court before their access is cut off?

Needless to say, I don't think we should have a DNS blacklist in America. We shouldn't adopt totalitarian information control just to prop up the current crop of companies that are in industry. Indeed, why should we work so hard to prop up yesterday's business models, anyway? We may as well try to bring back the horse and buggy.

Visa restrictions strike again

The band Incognito is not visiting Atlanta this year:

We did all I could to make this happen, but my band and I were not given the deposits that were agreed and after much toing a froing severe delays to our arrangements and our visa applications has made our deadlines impossible.

American visa requirements are holding hostage all sorts of beneficial social activity. It stops economic, intellectual, and in this case cultural improvement.

Fay and I had a wonderful time hearing Incognito play in Switzerland. They are an extraordinarily international band, having members from several different continents. Maybe they'll have better luck next year getting past the American border control.

The simple way to use CUPS from Windows

In the hopes of steering someone else away from a tar pit, don't bother messing around with Samba if all you want to do is print over the network to a Linux box. Just do two simple things:

1. Set the printer up using CUPS, the standard print software for Linux.
   
2. In the Windows "Add Printer" dialog, click the option that allows a URL, and paste in the URL to your printer. It will be like: http://<computer-ip>:631/printers/<cups-printer-name> . When it asks for a printer maker, specify "Generic" "MS Publisher Imagesetter".
   

Gah! I lost over an hour fooling around with Samba docs, because I hadn't realized Windows supports CUPS directly.

Hat tip to the BSDpants blog. They write:

So, I know!, Windows means samba and there's a port named "cups-samba". This must be just what I'm looking for. ... Well, it was kind of what I was looking for. But, it was too much trying to do things the Windows way. A little painful. What I discovered in digging around was that I didn't need cups-samba or even samba. CUPS by default leaves a port open for printing and you can use IPP to print directly to a CUPS printer via the network, even from Windows, and using generic, Windows Postscript printer drivers already on your Windows machine (probably).

I never understood why Windows network printing was any more complicated than this. Just have a standard protocol, and then any client can speak to it. All you should have to do on the client is paste in a URL.

Even now, I note that Windows is dying to have you select a driver. What's up with "MS Publisher Imagesetter" ? I wish they'd let go of the concept. If the user has joined the 1990s and specified a URL for their printer, that should be all they have to say about the matter. Anything so specialized that a URL alone is not enough could use some other mechanism.