Crowdsourced department ranking

So, I was reading The Monkey Cage this morning, and happened upon a post about crowdsourcing rankings of political science and sociology departments.  Basically, there’s a website that lets you put in an arbitrary list of options (which it somewhat unfortunately insists on calling “ideas”) and gives the internet as a whole to vote on them pairwise.  Of course, the next step was obvious (and while, yes, it was procrastination, in my defense it was actually incredibly easy), so I set up such a listing for math departments.  If you have nothing better to do with your time, you can go and vote a bit.  You can also see the results, though of course, at the moment they are pretty meaningless (not that they won’t be meaningless after lots of people vote, but I think at the moment, some schools have received no votes either way).

Before anybody complains about the schools listed: I just took the listing of graduate programs in mathematics in the United States and Canada from the Notable Math Wiki.  Obviously, it was a little unfortunate to have to be so nationalistic (continentalistic?) but otherwise, I think the overwhelming number of pairs for anyone would have been two schools they had never heard of.  If somebody else wants to set up an option for schools in different parts of the world, of course, they are free.

EDIT: I decided the full list was just too unwieldy; I eliminated all the schools whose “score” (roughly their probability of being liked better than a random school) was below 40 (though, of course, the remaining ones are going to spread out now).  Interestingly, the results are not nearly as “conventional wisdom” as I expected; Northwestern is a lovely school, but I don’t think many would rank it above Harvard, MIT, and Princeton as it is at the time of writing.  If that’s a statistical fluke, presumably it will go away a lot faster now, as the remaining schools will get voted on more often.

5 thoughts on “Crowdsourced department ranking

  1. They more or less match other rankings. Maybe you can be more specific. Who has better 3-manifold topology? Who’s better at topological dynamics? Those might actually help someone pick a graduate school or a postdoc.

    Crowdsourcing itself is great. You take a room full of people who are not that well-informed and yet they come to a very clear conclusion. Sometimes, you put a little betting money to keep them honest (a prediction market).

  2. John-

    I disagree, quite vehemently in fact (not about the fact that these match other ratings; that was almost unavoidable). When you’re choosing grad schools, you shouldn’t be looking for something as specific as topological dynamics (because the probability is just too high that you’re wrong), and when choosing a postdoc, you should be looking for something much more specific (you should know the individual people in the field, and not need to look at a department ranking).

  3. Hey Ben,

    When considering grad schools, I might look for places good at mathematical physics and 3-manifold topology (which I did) or one of the centers for logic or dynamics or mathematical biology or number theory. That specificity level might be help a future grad student narrow down their choices. By the end of undergrad, you have a rough sense of what you like and dislike.

    My point was that such general rankings don’t mean much because they try to average across too much information. This survey does just as well as other ranking systems in aggregating what people collectively know but not individually.

    Looking for postdocs (eventually), I will have to be more specific about what I want to do and who I will work with. You are right. At that point, I had better base those decisions on rankings.

Comments are closed.