Websites someone else should make: how to write a letter of recommendation

It occurred to me while I was reading the comment of Anonymous that it is an incredibly serious problem that no one is ever trained to write letters of recommendation. In large part, the problem is that people have very little opportunity to see such letters before one serves on a hiring committee, and it’s not so clear that people can pick up the necessary skills just from reading other people’s letters. There really needs to be a tutorial online explaining how to do so, hopefully with examples (obvious the examples would have to be hypothetical, but that’s fine).

I doubt that this will ever happen, but I thought I would throw the idea out there. You never know when someone might bite.

36 thoughts on “Websites someone else should make: how to write a letter of recommendation

  1. Steven Krantz’s book “A Primer of Mathematical Writing” has a section of some length on this topic, though I definitely agree that a more extensive tutorial along the lines you suggest would be good.

  2. Really?

    The closest book I can find is Steven Krantz’s “Mathematical Publishing: A Guidebook”, which doesn’t seem to cover letters of recommendation at all. The table of contents (at least) is available at amazon .

  3. Another reason this should be publicized: more than one person I’ve asked to write letters has asked me to write a draft for them. At which point I have no clue what to do.

  4. Things NOT to say in a letter of recommendation:

    “You’ll be lucky to get him/her to work for you. (S)he rarely comes to class drunk. […]
    I cannot express my praise any better than this.”

    Seriously, I look at the first paragraph. {\bf Jane Doe} has asked me to write a letter. I am happy/honored/pleased/flattered to do so.

    Jane’s 2024 Big State University dissertation concerned itself with left adjoint bifunctors over a closed n-category with no dual objects. The interest in Jane’s work arrises since it has applications to the [….].

    When she was developing her work, we met at […]. I asked her […], and watched her mind quickly grasp the nature of the problem.

    etc.

    The problem becomes when one is reading a hundred of these, they tend to run together. Whether you like it or not, the majority of candidates will be short listed because someone knows of the work, or someone in the department wants to learn about left-adjoint bifunctors over closed semi-strict n-categories.

    Also, some committee members will look at WHO wrote the letter.

    One might think that short letters are better. They are when there is specific mathematical content: This problem was the most important problem in [Stateyourarea].

    Use action verbs when possible. Avoid double meaning phrases when speaking about women candidates.If they can be taken
    in an incorrect manner, they will.

  5. 5. I think when people ask you to write a draft probably the main thing they want from you is a concise description of your work. My reason for guessing this is that my advisor explicitly asked me for a short description of all my papers and projects to help him with writing my rec. Basically I printed out for him a copy of what I had on my webpage. The point is to refresh the recomender’s memory and give them a skeleton of facts on which to build the letter.

  6. There is a deeper problem than mere lack of advice. The system of letters of recommendation is inconsistent and grade-inflated. Everyone is fabulous according to almost every letter; the only question is how fabulous. If a letter fails to imply that the applicant’s research is fabulously broad, say, then someone reading the letter might wonder if the research is as narrow as a bookmark.

    Okay, the system is not really so terrible — yet — that’s it’s impossible to write a standout letter. But the situation is annoying, and it’s hard to give advice because it’s a moving target. The standout verbiage from a few years back becomes the standard verbiage for the current year.

    I wonder if it would make things better to have non-confidential letters of recommendation, even though some people insist that it would make things worse. Letter-writers often assume that applicants can get wind of their own letters, but not each others’.

  7. So the question is: is there any way to return credibility to letters of recommendation? Some sort of artificial scarcity of recommending we can impose? I’m thinking of things like this.

  8. The reason that letters of recommendation are not entirely a quagmire is that they still carry real information amid the red herrings. You can look at who wrote the letter and how long it is. Sometimes letters have indirect comparisons that are credible. Direct comparisons, like X is as good a mathematician as Y and better than Z, can still work, but they are increasingly hokey as well as risky for the letter-writer. In many cases the best that you can expect from the content of a letter is simply to attribute credit to the applicant for his or her results. This too can be hokey (applicant brilliantly demolished the following conjecture that you never heard of), but it is relatively credible.

    In some cases the arXiv helps a lot, because it can reveal a trail of results and citations apart from what the letters say. Of course journal publications can do that too, but the publication cycle is too slow for the job market. The situation in which that comes up short is joint papers. You don’t want to hire someone just because you wish that you could have hired his advisor. Unfortunately the advisor might also game the system with his letter of recommendation for his student.

    I think that might help if the three level system that we have now, the arXiv + journals + Math Reviews, were simplified to direct reviews of arXiv papers. But that reform is easier said than done.

    The link to “signals” that you mention is interesting, but it solves a different problem. Even though the math jobs wiki is very incomplete, I think that it is has enough to reveal chaos in the job market. Somehow the math job market should be better coordinated, but how and with what rules?

  9. I don’t see how the job market can be really well coordinated; for one thing, it is an international one, even if many US-based people do not necessarily see this, and different countries have very different systems.

    So there’s a fair number of French PhDs, for instance, who go for postdocs outside France or even Europe (though not a majority), and similarly for many other European countries.

    Not so many people come to France from the US because the teaching has to be done in French in most cases, but if you know how the system works there, it may have its attractions (e.g., permanent positions more or less immediately after a PhD, including some CNRS permanent _research_ positions, with no teaching obligations whatsoever, at a comparable time — if you’ve never heard about that, you’re probably be amazed to learn that the interview for such positions lasts all of 7 minutes, including two (2) minutes of presentation, and five for questions…)

  10. Greg: “You don’t want to hire someone just because you wish that you could have hired his advisor.”

    Is this how joint papers with one’s advisor are generally perceived? It’s probably a wrong and unfair assumption in many cases.

  11. Is this how joint papers with one’s advisor are generally perceived? It’s probably a wrong and unfair assumption in many cases.

    I think you’re reading a little too much into one sentence. I think he was saying that it’s generally known to be a possibility that the important ideas behind a joint paper are the advisor’s, so if the advisor doesn’t go out of his/her way to dispel the idea, people may surmise that that’s the case.

  12. some CNRS permanent _research_ positions, with no teaching obligations whatsoever, at a comparable time

    My understanding was that the pay for these positions was quite poor though, poor enough that people often give them up to take teaching positions at universities.

  13. Yes, the downside of the CNRS is that the pay is not very high — however, at the beginning, it’s not worse than the teaching positions (in France — it’s still less than a good postdoc in the US).
    The reason many people do not stay there is that promotion within CNRS is extremely restricted: there are some senior positions with the same features and better salary, but very few, so most good CNRS researcher naturally look for promotion as professors in universities.

  14. Googling for “letter of recommendation” found many websites that do exactly what you ask (though I don’t know how good they are at hitting the *moving* target that Greg Kuperberg pointed to).

    For example, the first one that looks like its more than just a collection of samples is: http://www.boxfreeconcepts.com/reco/

  15. Josh,

    I don’t think that site is really what I had in mind. On the whole, job-hunting advice aimed at a general audience can’t be relied upon for the academic market. In fact, I would even be careful about advice from other academic disciplines, or even advice that didn’t take which segment of the market you’re aiming for into account (research universities, 4 year schools, etc.).

    Of course, many things will be applicable in all situations, but one can’t know too reliably which ones.

  16. Ignoring for a moment the question of how to write a letter, I’m wondering when to do it. I’ve had 3 people ask me to write letters for postdocs this year, but since I’m still a postdoc, I wasn’t sure if it was a good idea. From what I’ve heard, the main thing about a letter is not what it says, but who wrote it; e.g. a raving letter from an unknown isn’t worth nearly as much as a mediocre letter from a big wig. Does anyone have a good litmus test for when to decline to write a letter for someone?

  17. Sami, I’m the same vintage as you, and I wouldn’t write a letter of recommendation for a Ph.D. graduate. I’d have some trepidation about doing it for an undergrad. Instead I would try to “launder” my recommendation through someone more senior.

  18. Letters of recommendation are always painful to write, for me at least. As I’m reminded by the earlier comments,
    there is grade inflation at work. Search committees do
    struggle to identify the unusually strong letters among
    the routinely strong ones. (On our side, we have to
    emphathize with advisors compelled to write simultaneous
    recommendations for several students of theirs who are
    of differing promise in research, teaching, whatever. But
    we do want to get an accurate picture.)

    There are, alas, no rulebooks to consult. Generally
    very short enthusiastic recommendations like “Hire
    this person!” are unhelpful, as are multipage technical
    treatises. After all, even a person with a few postdoc
    years is unlikely to have yet done a lifetime’s worth of
    interesting stuff. But the seriousness of a letter does
    signal something to hiring committees, however it is
    transmitted. Did this letter take a while to write or
    was it done during a more absorbing conversation?
    Does it really characterize the applicant well among
    the blur of others?

    A bottom line for letters: writers of high reputation
    get more attention than others by hiring committees,
    provided they do know the candidate and have something
    to say. (As I know from years of work in graduate
    admissions, this makes some sense. Professor X may
    value an undergraduate student most highly, but
    Professor Y may have lived in the world of mathematics
    more deeply and have a more sober assessment of
    the student’s potential.)

  19. It is most definitely a good idea to have the majority of one’s recommendation letters be written by the most senior and eminent mathematicians you can find. This is one way to combat grade inflation: Professor X of Berkeley/Harvard/Princeton can’t say “The best student ever!!!” every year for twenty years. (Or more precisely, if she does, then people will figure it out eventually and not take the Professor’s letters seriously. I do know of some “wolf-crying” superstar mathematicians, but only a few.) Eminent people can also make knowledgeable binary comparisons (which I find very helpful, so long as the writer can be trusted) to people that the majority of readers are likely to have heard of. “Best student ever!!!”
    is not necessarily high praise if none of your other students ever got tenure at a research university, for instance.

    The counterbalance is that it is good for some of the people who write your letters to have real insight into your work. For graduating PhD
    students, I think it is fairly standard for a couple of letters to be written by other faculty at your institution in the same general research area. Often you have a single meeting with these people and/or give them a copy of their thesis, but they may not have anything really insightful to say about you.

    I would say that if there is a more junior mathematician who has something really important to say about your work then you might as well solicit a letter from this person IN ADDITION TO the other letters you were getting anyway. It is generally understood that the 3-4 letters asked for are a minimum, and most people don’t mind at all reading at least a couple more. (However, two years ago there was a candidate who had 9 letters of recommendation that were pretty similar to each other. This is weird enough that I still remember it.)

    Getting younger people who know you well to talk to your other letter writers is also a good idea. I have myself gotten recommendation requests and said, “I’m flattered that you ask, but most of the people reading my letter would never have heard of me. Who you really want to write your letter is X. I know her pretty well and will talk to her about your work.”

  20. Great thread. I have recently written and recently read plenty of letters of recommendation. Some observations:

    The reputation of the writer counts most. However, some writers, as noted, have spent their political capital through loose praise. Hiring committees will try to parse the meaning of the letters based on all the information they have about the writer. Such exegeses can be veritably Talmudic.

    Most letters are generically positive. The good ones can concisely specify the candidate’s best contribution, and for a general readership. This usually requires that the writer is actually knows what he/she is talking about, but not always.

    US writers are more cryptic with their criticisms. Words that hurt: “good,” “knowledgeable.” Words that kill: “capable,” “competent.”

    Generally, committees will try to calibrate for the differences between US and European letter writers, but I would guess (stress guess) that letters from the Continent are still a gambit for candidates at US institutions.

    Direct comparisons are great for committees, and shouldn’t pose a risk for any but the most junior writers.

  21. 20, 21. I don’t think that a postdoc at a top school should have any reservations about writing a 4th recommendation for a graduate student. Assuming you know the applicant better than anyone else does. Someone who is a postdoc at MIT should be qualified to have opinions on what makes someone a successful graduate student.

    A postdoc, however, is not someone who has the experience to know what makes someone a successful postdoc, so writing a rec for a graduate student seems like an iffy proposition. I don’t know what your job situation is, if you already have a permanent job lined up then you at least have the bare qualifications. Without that it seems weird to me to write even a fourth letter for a grad student. Laundering seems like a much better bet.

    Of course, I don’t know much about this stuff other than my undergrad advisor thinking it was a good idea to have a fourth letter from a law student with a math Ph.D. who I’d taken a mat tutorial from and who knew me very well. The other 3 letters were from famous people though.

  22. Of course, more knowledgeable people can correct me, but I suspect even for a undergrad, a recommendation by a postdoc who isn’t very well known runs a strong chance of just being ignored. I think what Pete said above has some serious weight: people don’t just give more weight to more senior letter writers because they’re elitists; they’re almost much more likely to have a feel for how that person writes their letters and how to calibrate them. Of course, one has to start somewhere, but I think this is a good argument for waiting until one is somewhat established, for the sake of one’s recomendees.

  23. Ben,

    I know I am presenting an extreme case, but surely you are more likely to be known to someone on a graduate admissions committee than a professor at Simon’s Rock. Depending on the school, calibration of your letters is unlikely to be that much harder either.

  24. Alex,

    I feel like that’s not a very helpful comparison. I mean, how many people are choosing between a recommendation from a professor at a 4 year school, and a postdoc at a research school. Typically, there are postdocs around at a school, there will also plenty of senior professors with better connections.

  25. I think he was saying that it’s generally known to be a possibility that the important ideas behind a joint paper are the advisor’s, so if the advisor doesn’t go out of his/her way to dispel the idea, people may surmise that that’s the case.

    The first half of what you say is more in the direction of what I meant. Yes, it’s fine a priori to have joint papers with your advisor; but yes, it could happen that the advisor is the stronger collaborator. My impression is that search committees tend to be fair about this situation. They tend to be enthusiastic if the work is good; they can also believe that the student contributed significantly to the work.

    What I meant to say is that you might want to use letters of recommendation to better interpret joint work, not to dispel suspicions about joint papers. But if the letters for everyone are all ambiguously fabulous, it won’t resolve the matter. I think that if anything, the system rewards joint work, sometimes more than it should.

  26. Words that kill: “capable,” “competent.”

    For those here who have not yet seen typical letters of recommendation, the suggestion that a word like “competent” would be the kiss of death should show you that something basic has gone wrong. “Ivanovich not only proved the Chen-Jacobs conjecture, he is also competent at quantum field theory.” Red flag! Can’t hire him!

    My impression is that no one word is necessarily a red flag; it depends on context. But Eric is completely correct at a more general level, that it is too easy for a letter of recommendation to damn an applicant with faint praise. The basic reason, of course, is that the other applicants are the greatest thing since sliced bread according to their letters.

    Certainly any overt criticism in a letter is deeply damaging. Even an overt qualification of praise reads more like a disqualification of praise. Praising the wrong thing can be damaging, e.g., “Ivanovich works in algebraic geometry, and by the way, he plays a mean game of tennis!”

    Not everyone is irritated by the coded style of letters of recommendation, and few people have lost all faith in them. But a lot of people are irritated or at least jaded. I don’t know whether Ben Webster’s pointer to employment “signals” is the solution or not, but it does have one very important aspect: It is suggested by game theory and economics; it is not just an evolved social practice.

  27. The important thing about the signals is that they’re constrained; you can only send them to two schools (for those of you who didn’t follow the link, in economics they now have a system for applicants to send exactly two schools a “signal” expressing interest in that school). The school knows that by sending a signal to them, you gave up the chance to do it for other schools. Similarly, letters of recommendation would be a lot more credible if giving lavish praise to one applicant had an actual cost of some sort. It seems impractical to make praising applicants some kind of zero-sum game, but it hard to know how else to contain inflation.

  28. In fact there is an ultimate cost, and it has something to do with why letters of recommendation still work at all. The coded style of letters is weird and shifts over time. Nonetheless, there is a standard range in any given year. If you lavish praise beyond accepted bounds, people won’t believe you. You’ll have a little less credibility as a letter-writer and the applicant will not be helped as much either.

    This points to a paradoxical feature of recommendation letters. Even though you have to write in a style of exaggerated praise, the content written in this strained style is ideally entirely truthful.

    Joke letter: “This is the greatest young mathematician EVER, working in the area of arithmetic foliations, who I have met personally, and who happens to be Icelandic and left-handed.”

    Anyway, the NSF has run up against the same limitations as letters of recommendation in grant proposal reviews, and in fact they did come up with another method of forced compromises: Panel rankings. Of course a search committee is itself a kind of ranking panel. The problem is that search committees cannot be expert in all areas; it’s a redundant and otherwise inefficient evaluation system. If letter writers could be organized into panels by area of research — rather than organized by university as search committees are — then in principle job market applicants would be evaluated more accurately and more efficiently.

    The problem with signals is that many letters writers only write one or two letters in any given year, and there is enough room in the system for all of the letters to be like that. Moreover, prolific letter writers are already careful to make their letters consistent, both to be fair to their juniors and to retain credibility. So rationed signals would mend the ankle that isn’t sprained.

  29. A brief postscript to what I wrote before: We all need to
    keep in mind the fact that recommendation letters are
    used to persuade hiring committee members outside the
    research area, but especially to persuade deans and provosts not in daily contact with the faculty involved
    to fund the position (not a given here at UMass no matter
    what the department wants). That’s why physicists
    and chemists and others like to have Nobelists write letters, a pastime for the latter which no doubt
    fills their declining years nobly enough. Those close to the
    research area can actually look at the candidate’s papers
    and preprints without taking someone else’s word for it.
    Teaching and related activities involve more guesswork
    on everyone’s part, along with value judgments.

  30. So rationed signals would mend the ankle that isn’t sprained.

    Greg, I feel like this isn’t quite consistent with what you’ve said in previous comments. I mean it’s one thing to say “Any attempt to rein in the inflation of letters of recommendation through artificial restrictions would do more good than harm.” This is almost certainly true. There’s a reason I didn’t suggest a scheme for doing this; I certainly can’t think of a good one.

    But you seem to saying that it isn’t a problem at all, which I find hard to believe. I mean, it may be true that there is some “reasonable range” for these things, but presumably it’s not so hard to miss and write a letter that is too negative or so positive it won’t be believed. I mean, if it were really true that across the board “prolific letter writers are already careful to make their letters consistent, both to be fair to their juniors and to retain credibility” then there wouldn’t be inflation! Presumably some of them don’t do such a great job, and their recomendees suffer for it.

  31. Ben, I concede that I haven’t been all that clear on this point. I also agree with you that artificial scarcity, taken broadly, is the right concept to improve letters of recommendation. All I meant is that imposing artificial scarcity on individual letter-writers would be ineffectual. Most letters do not come from prolific letter-writers. Moreover, prolific letter-writers tend to be self-consistent, albeit not always consistent with each other, and this is in fact a stabilizing force. The real problem is a letter that can’t easily be compared to other letters from the same senior mathematician.

    Panel rankings, for instance, are another kind of artificial scarcity. They would limit the number of letter-writers, but if anything individual letter-writers would be doing more.

Comments are closed.