text:
Should I use ChatGPT to create cover letter for assistant professor jobs in math?
I am in the job market this year for tenure-track positions.
I was wondering if you could tell me how good/bad it is to create cover letters using ChatGPT. I am making edits after I generate my cover letter, but I am worried that my applications are not getting picked up because of AI-generated cover letters.
What kind of impressions hiring committee get after looking at such a letter? I am in Math.

Why are you doing this?

What kind of impressions hiring committee get after looking at such a letter? I am in Math.

For me, the effect would be similar to you writing
I AM A PERSON WHO LIKES TO TAKE SHORTCUTS
in big bold letters in your cover letter. Given that this is the sort of person you are, I would know that if my department hired you, we should fully expect you to continue to use AI assistance to write your course syllabi, committee reports, grant proposals, emails, and many other types of work products.
My likely reaction would be to be grateful to you for disclosing so clearly what kind of a person you are, and for saving me the work of reading the rest of your application and thinking how I should rate it.

Whoa, bit harsh. Accurate though. Worth an upvote, anyway.

Is taking shortcuts really that bad? Why is adapting to AI seen as a bad thing? Have you never taken any shortcuts? If the syllabus or curriculum can be generated using AI and addresses everything that a student or the coordinator needs to know , then what is the problem? Moreover, isn't that one of the purposes of using AI? It helps us to save some time and be more efficient? And, if English is not my first language, then why is it wrong for me to use AI to help me with the grammar and write more coherent sentences, and convey my thoughts more clearly?

@It'sMe, why hire a human at all, if the AI can do everything just as well?

@It'sMe Those are a lot of big ifs that nobody has shown to be true.

@It'sMe I would classify AI use into responsible and irresponsible use. Responsible AI use is where you put a lot of thought into writing a document but use the AI to improve your writing quality and/or overcome language deficiencies. That’s fine and is not what I would call taking a shortcut. Irresponsible use is where you try to dishonestly create a superficial impression (which invariably collapses under minimal scrutiny) that you put a lot of thought into writing something, when you actually didn’t. That’s what I call taking a shortcut. And yes, it’s bad.

@It'sMe also: if taking shortcuts is not bad, then writing on your cover letter that you are a person who likes to take shortcuts should also be not bad. Right? Well, feel free to write that in your cover letters and see where that attitude gets you…

"I AM A PERSON WHO LIKES TO TAKE SHORTCUTS" What is wrong with that? This says that I am a person looking for innovations and ways to optimize my work. I see this as a positive trait.

@pabouk-Ukrainestaystrong "my work" -- this is a problem. In academia, a lot of work is shared work (writing papers, grants, designing teaching approaches). When one team member takes shortcuts, the quality often struggles, and colleagues have to pull harder to have this paper published, the grant approved, classes delivered, etc.

@pabouk-Ukrainestaystrong Taking shortcuts can end in two ways. If one cares about their work, they may indeed optimise it to achieve the same results in an easier way. But the other possibility is that they take shortcuts to avoid any work, hoping nobody will notice. I dare say the latter is more common.

@pabouk-Ukrainestaystrong The reason it's called a shortcut is because the outcome has worse quality. It is fine to typewrite instead of manually write your course materials. That is not a shortcut. It is not fine to AI generate your course materials, and in general anything someone spends time to read in detail because AI does not produce anything at that level yet.

@PasserBy ok, it is my misunderstanding of the term "shortcut" in this context which comes from the fact that using AI for writing a text is not necessarily worse. The quality of the result strongly depends on the way you use the AI. --- 1. Cover letter is a course material. --- 2. Generating the base of the text in one go without enough input, using current AI, is worse. Going through multiple iterations with the AI and providing enough input could give better results (compared to writing without AI) if done wisely.

"You only get one chance to make a first impression" is a trite, but true proverb. And a cover letter is part of the first impression. By having a LLM do this - obviously important - task makes comes with a high risk that result is generic of lesser quality than if the time to write it themselves were to be invested, leading to the implication that this "shortcut" is not the "works smart to save unnecessary effort" but rather the "cuts corners to get things done faster" kind, and that the applicant does just this - cutting corners if they think they can get away with it.

"AI assistance to write your course syllabi, committee reports, grant proposals, emails, [...]" Is this necessarily a bad thing? I use chatGPT to write PHP code: it is not perfect, but it saves a lot of time for other, more valuable activities.

@JFabianMeier see my comment above about responsible vs irresponsible AI use. For coding, AI is definitely a timesaver and efficiency-booster. For letter-writing, my impression is AI use (of the “responsible” type) can help improve the quality of your writing a bit but only if you invest comparable amounts of time to what you’d need anyway to write your document. Alternatively, you can save time by using AI to write pleasant-sounding documents that are devoid of any meaningful content. If that is your goal, then I guess you too are a person who likes to take shortcuts.

@DanRomik you're really talking as if you have never taken any shortcuts. What about copy-paste? Would you count count that as shortcut? Have you never done that? When submitting an article to a journal, some journal give you two options- either directly submit your PDF file or submit all the files using their source file. Have you never directly submitted just the PDF file? Do you always sit and spend few hours converting everything into their source file?

Maybe the above two examples don't apply to you, but I find it hard to believe that you have never taken any shortcuts in your life or academic life. I'm sorry to say this.

Don't send emails either. Hand deliver the message/letter to the person. Earlier people used to mail the documents to the university. But now they just upload the PDF file and click the send button. They are not putting the effort of arranging the documents, sealing it in an envelope, writing the proper address and mailing it to the university. Why submit applications electronically? Why allow that? Ask the interested candidate to hand deliver the documents in you office.

@It'sMe I think there is a bit of misunderstanding here. DanRomik appears to talk about "responsible vs irresponsible AI use" and how responsibly using LLMs for this particular task is not a timesafer in their eyes as responsible use would require similar time for post-processing as performing the task "by hand" would and using it as a "shortcut" would require not spending the time for post-processing, making the use irresponsible.

Let me give a common example of my own regarding responsible vs irresponsible use of tools: Automated grammar- & spellchecking. Having potential mistakes highlighted makes all our lives a lot of easier, but - especially when "official" correspondence is concerned - should not be solely relied upon. It is a fine tool that will help us catch typos and similar issues, but it is inadvisable to simply assume "there are no red squiggly lines, everything is fine" and proceed to skip re-reading what one wrote.

@CharonX I agree with his comment about "responsible and irresponsible use of AI" that he clarifies later. But he doesn't give that clarification in the answer. His answer, especially the statement in big, bold letters, is expressing his dislike for taking shortcuts by using AI and it seems to suggest that he's going to judge/rate the application harshly. Also look at this second comment (the comment just after responsible and irresponsible use).

I feel like we're expressing a humanities vs. engineering divide here, and note that math is a humanities discipline, not an engineering one (even if engineers use math).  In the humanities, we care how beautifully something is done, not just that it works.  I haven't asked him, but imagine how Knuth (note the title of his magnum opus) would react to someone using AI to generate literate programs.  I think he would say it's besides the point.

As someone who recently read 20+ variations on pretty much the exact same cover letter for our department's open job, and was able to recapitulate almost that exact same cover letter by inputting a request to ChatGPT to make a cover letter for our job listing, I recommend against this.
ChatGPT is very good for "punching up" or proofreading a draft that you have already written, but left to create letters on its own usually results in pretty generic letters that will sound exactly like everyone else that requested ChatGPT write the letters for them. You want to stand out (in a good way) with your cover letter, and not blend in with everyone else.

Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed.

This seems like an unwise idea. Meaningless pablum, whether ChatGPT-generated or not, is distasteful to most mathematicians I know, and is unlikely to go unread.
In my experience at a US research university, pro forma letters that are brief and don't say much anything of substance are commonplace and fine; however, I've heard that at more teaching-oriented universities a tailored cover letter is very important.
If you do write a longer, tailored cover letter, then the purpose is to demonstrate that you've put some thought into the position and why it's a good fit for you. My guess is that ChatGPT-generated output is unlikely to convince anybody of this.

"the purpose is to demonstrate that you've put some thought into the position and why it's a good fit for you", I'd add that it is indeed a good idea to put some thought into why you think the position is a good fit. This can only be beneficial to the application process. And if the result is a well written cover letter: fine!

Hello, thank you for the answer. I am editing parts of it, but some words are generic and if I change with other synonym, it just gets weird sounding.

@Topology How would chat GPT produce a cover letter that explains why you in particular would be a good fit for that position in particular? Editing individual words to synonyms is not the point here, the point is that a LLM generated letter will be pointlessly generic. You need to think about what information you are conveying to the hiring committee about why YOU would be a good fit for their particular position.

If you use AI as a tool and carefully edit what it produces, then you should be fine. But, it depends on what your prompt to the AI was and what data it was trained on. If it produces generic text then it won't be very useful and might trigger readers that you aren't a serious contender. It might be obvious to some that it wasn't written by a human, even without a check.
I'd recommend against its use for producing cover letters (and other things), but only as a source of ideas. If I received a letter that I thought was AI generated, I'd certainly have doubts about the sender.
One normally writes a "generic" letter on their own and then tailors it for every application, based on the stated needs of an institution. You don't really need an AI for that, assuming you are applying to a reasonable number of positions. But the tailoring has to be sensible. If it is overly fawning or obsequious then it might be less than helpful, but more likely to be auto-generated.
And, if you aren't a native speaker, then an AI is especially dangerous to use as it may say things that aren't really proper, but that you don't "catch".

Hello @Buffy: Thank you so much for the detailed answer. I sometimes add specific details to my cover letter, generated by AI (where you ask AI to specifically pull out information from the job Ad etc). How does that sound? The only thing is: the language gets repetitive sometimes.

That "repetition" is exactly what you don't want.

I agree with this. There is nothing wrong in using AI or anything else, as long as the letter is of high quality.

It's worth remembering that identifying (or even suspecting) artificially-generated isn't that difficult once you know what you're looking for. To use it successfully, at the moment, for anything much more detailed—or nuanced—than a list or basic description, you'd need to adjust the language so much that you might as well have written it in the first place.
In other words, I suspect that using ChatGPT would either harm your approach or just not save you the time you might think it does.

(+1) For me at least (fortunately, a native English speaker), this is what I would expect happen if I did this. However, after having written an n'th draft, I suppose it wouldn't hurt to see what ChatGPT would write and see if it includes anything that I might have overlooked, in which case I'd proceed to write an (n+1)'th draft.

And once the AI-generated cover letter is detected, there's a decent chance it goes straight in the trash. Submitting an AI-generated application is basically saying, "Please use your limited time as a human to read my resume, even though I did not find it worthwhile to spend my limited time to write it." No, thank you.

identifying (or even suspecting) artificially-generated isn't that difficult once you know what you're looking for. - Can you provide a reference?  I thought with good prompting it can be hard to tell the difference.

It is likely rather disadvantageous for you to use LLMs to create your cover letter for you.
The hiring committee will use your cover letter (alongside the other information you submitted) to determine if you are a good candidate for the position. Using a LLM both obscures most of the information (as the LLM can and will - by nature - only generate generic text) while - should they notice that it was created by a LLM - make some likely negative implications about you instead.
If I were part of a hiring committee and notice an applicant submitted LLM generated content, I would need to assume that they don't take the job seriously, are likely to continue using LLMs to generate text for them, or (most likely) both - which would make them a very bad fit.
In short: It is a bad idea, you will leave a bad impression, don't do it.

Before I go on I note that the instant I started an answer, the message "Answers generated by artificial intelligence tools are not allowed on Academia Stack Exchange".  Bit of hint there, perhaps.
I had a job coach, years ago, who told that the cover letter is the most important part of the application.  It's the thing that stops your application being instantly binned (or shredded in more GDPR conscious places).
Every cover letter should be crafted and targeted at the specific role you want.
They won't even glance at your CV if they don't like the cover letter.
One day you'll be the poor sod going through a hundred or more of these and, trust me, AI generated crap will be dumped.
Writing a cover letter (which extends into things like looking for grant money for research and a hundred other things) is a vital skill.  Learn it.

+1 for the last paragraph, which makes a very good point no one else here has thought of making.

Chat-GTP can help with improving an existing draft. It says things things like 'you write this, can you give an example?' or it may suggest better wording (in particular for non-native speakers, it is really good at improving text in this way).
View it in the same way as giving the letter to a friend who may give some suggested improvements, but do not use it to generate a letter from scratch.

I'm maybe a bit more pro-LLM than some other perspectives. My feeling is that using an LLM to help you generate text is fine, but using it as a replacement for thought is not.
In other words, a good cover letter should explain why you are a good fit for the position. There is an implicit question here, that requires thought. Which of your qualifications match the description in the letter? What accomplishments do you have that will make you stand out from the crowd? What are your goals for your research and teaching? If all you do is prompt ChatGPT with "write me a cover letter," it will not know the answers to those questions, and it will show that you have put no thought into it.
However, if you have put in the legwork to have answers for those questions, then you can use a language model much more effectively than simply saying "write me a cover letter." You can write a much more specific prompt like, "Write a letter that talks about my  accomplishments X, Y, and Z, and that fits into University A which is looking for B, C, D. Make sure to emphasize my experience in super-duper-modern-technology and explain how it relates to Prof E's research in FGH." You could feed it a cover letter that you've written highlighting your skills and the job description and ask it to modify your cover letter to emphasize things in the job description. If you use it well, the result will be something that reflects the thought you have already put into answering the key questions for yourself.
Of course, once you get text from an LLM, you should not simply copy/paste it and call it done. You should read what it has written carefully, and use it for inspiration. Rewrite sentences to put them in your words (you may emphasize some ideas in a subtly different way), correct errors, add pieces that it has missed or that you have realized are important. In the end, you are still responsible for what you submit, and the text should reflect your thought and your work. But you can certainly use the LLM as an assistant to help you express your ideas.
On some level, the thought is the hard part, and coming up with text perhaps "should" be easy once you have done all that legwork. But you might have writer's block, or English may not be your first language, or just want some help since editing is often easier than starting with a blank page.

What I don't understand about this approach (as well as about many other uses cases of LLMs that people are talking about) is what's the benefit of it? The time consuming thing about writing anykind of non-trivial text is not typing a few paragraphs, it's (a) coming up with the right contents and (b) expressing those contents in a way that is most appropriate for the purpose and the audience of the text (due to (b) I disagree that coming up with the text were easy). How can an LLM save me a significant amount of time with those things?

@JochenGlueck For a cover letter I probably agree this is not going to help much. But, sometimes it is more time efficient or at least more pleasant for me to edit text than to start from a blank page. And I've also been able to use LLMs to take a "vomit draft" that contains the ideas but is too long and informal and make the text more concise, or to convert an outline into a first draft. I think if used properly it's just a tool, and like any tool there's a startup cost for using it that might not be worth it for everyone, and it's not the only way.

I agree that it's merely a tool. It's just that, with few exceptions, I haven't found the tool particularly useful so far, so I'm trying to understand what other people find useful about it. It might depend on one's personal working style, though. For instance, I would almost always prefer starting from a blank page over editing something that someone (or something) else has written.

(For the record: I'm not the one who downvoted.)

I'm not surprised this answer is controversial/downvoted, and lord knows I don't think I'm the source of truth and I could be wrong. But, I would say that LLMs are a big, new technology. One approach is to try to effectively ban a particular use case on moral grounds. Another is to try to accept that people will use the tools available to them and reward effective uses of it. We will see which one turns out to be more successful in the long term.

@Andrew 1) LLMs are not in fact "new technology". They're an overfed autocomplete. 2) Calling the objections "moral grounds" suggests that you've never actually paid even remote attention to what people who actually understand AI research are saying and why they're objecting to it. I hadn't downvoted your answer yet, but for that comment I am.


text:
Can a professor fix a mistake in grading after the semester has ended?
Let us assume that a professor is handling a course on statistics for undergrads.
They conduct exams and evaluate the answers and award marks to the students.
Suppose that the professor committed some mistakes, unknowingly, during the evaluation of a student's answers in one of the exams, and as a result, there is an error in the final grade of the student in statistics, and it is not noticed by anyone, including the student.
If the student realizes the mistake done by the professor in the next semester, then is the professor still responsible for the mistake that was committed? Or is the professor not responsible for their actions since the student was calm throughout the semester?
If the professor is responsible, then what is the action they need to do if the administration does not allow to edit the grades after freezing them?
Assume that the professor distributes the answer sheets after their evaluation for clarifying doubts of students regarding her evaluation. And the student can approach them at that time in case of any issues i.e.,  wrong marking.

Rules on grade changes very a lot between different institutions- there may or may not be a formal process to appeal for a grade change in this case, but if there is one, the details will be institution specific.

(1) Personally, I find the hypothetical way of writing unnecessary and irritating to read. (2) You may be able to avoid closure by cutting off after "...that was committed?" Since the institution-specific questions are unanswerable here, but the general question may be on-topic

At my university, I can request a change after submission (even years later), but I have to fill out several forms that detail exactly why a change should be considered.
Like @BrianBorchers said, this will vary by institution -- you need to ask them.

Yes, the professor is responsible for mistakes she made. Not noticing a mistake doesn't excuse it. But the current situation is no longer under the control of the professor as you state the question. It is up to the administration to correct the error.
Of course, the professor should make an appeal, along with the student, for an exception to any rule.
So, in giving an evaluation to the professor you can validly complain about the mistake, but not about the rules made by others. For that, you need to complain elsewhere.
In general, people are responsible for their own actions and omissions, but not those of others. But they should also work to correct past mistakes however they can.

The lead question has been edited, so let me add:
While the professor should try to help the student fix any errors that were made, the rules might not allow it. However, I would consider systems with inflexible rules, not permitting errors to be repaired, as unethical. It saves someone a bit of work to forbid such changes, but the accuracy of the record and the fairness to students are values that far outweigh such considerations.

Let me add that there are often regulations like "Students have X time to make a complaint" or, similarly, "after X months, the exam has to be (securily) trashed". Then, it might or might not be possible to appeal the grade (or maybe possible for the prof, but not for the student).

@user111388, and a sensible system will permit exceptions to avoid unfairness. Overly rigid systems serve no one and lead to injustice.

Sure. But the OP did not say that they are in a sensible system.

At my university, professors are "obligated" to post their final grades within a certain deadline.  I said "obligated" because this has generally been my experience, but I have heard stories about how some didn't post their grades until the start of the next semester.
That said, there is a possibility that there could be a grading mistake.  When professors do upload grades, they upload them into a central database.  However, they make sure to get it right the first time since it is not as easy to change the final grades once they have been entered.  In the rare situation that a grade does need to be changed, professors can submit a "grade change request" form to our registrar, which will allow a grade to be changed in the central database.  One example I saw first hand, was that in an undergrad distributed systems/cloud computing course, the professor had extended our final project deadline to essentially coincide with the ending of our semester, but he still had to get grades in by a certain deadline.  To compensate, he submitted whatever grades he had before a certain day, and any grade changes necessary after that day he would fill out forms to submit to the registrar.
So yes, professors CAN change grades, but they will likely have to go through a process that is more tedious than it may seem.

I think the level of tedium varies from place to place.  If I need to correct a grade after the semester ends, I only have to fill out a simple web form: semester, course, student (picked from the final roster), new grade.  Before it was electronic, it was a simple paper form.  Formally, the form is only a request, but in 20 years, I've never seen a grade change request denied, sent back for more information, or take more that a week to take effect, including one change seven years after the course ended. (To be fair, I've never tried to change a grade downward.)

To clarify, I did mean that it wasn't as simple as changing a test grade or something the way they would normally do since they upload final grades to a central database as I had mentioned.  But yeah, it does vary by the institution but it takes time, that is what I meant.

My impression, at least at the universities that I've been involved with, is that grades can be corrected after the fact. The procedure would be something like the following:

The student contacts the professor, says "I believe there was a mistake in my grading, for the following reason: [Reason goes here.] Would you be willing to double check this?"
If the professor agrees that he/she made a mistake, then the professor follows an official process and requests of the administration that the grade be changed. Generally, I would expect this request to go through.

In particular, I would recommend (politely) contacting the professor as soon as possible, but whenever the mistake is noticed.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What’s the current state of the art in searching for journal articles?
I haven’t been a student for many years. When I was in school at first we used portals like

JSTOR
Science Direct
Elsevier
Galileo

Next, Google Scholar came out and seemed to be a superset of all of those, so it was the best when I left off.
I was on a phone call with a university class a couple of years ago and they told me there was some newer “meta portal” (my term) that was along the same lines but even better than Google Scholar. I can’t recall the name and it’s not obvious which one it is when I try to figure it out by searching the web.

When I began reading your first sentence (e.g. "for many years"), I was expecting you to mention things like bibliographies of papers relevant to your interests, library stacks, department reading room, etc., but not JSTOR and Science Direct!

@DaveLRenfro I may be old but I'm not ancient ;) Just a little out dated. I left academia in 2013.

I'm not aware of any database search that is equivalent yet better than Google Scholar (except perhaps Scihub, which is illegal). But to help people help you, please specify your academic discipline because some disciplines have special databases that serve them better.

@Tripartio I'm a Data Scientist. My degrees are in econ but my work cut across many fields these days. Do you know of one that's at least like Google Scholar in the sense that it searches many sub portals/journals and which came out more recently than 2013?

Web of Science is very useful (if your institution/employer has access to it)

@Superbee This seems like it might be it, thanks! I guess, technically, it's older than I thought, but perhaps it gained traction since I left academia. Yes, I recall that I didn't have access, but I was just thinking perhaps after remembering what it's called I could see if there's some way for me to obtain access. If you want to make it into an answer I'll accept it.

This question is rather vague, since it is asking us to tell you what you might have heard about a couple of years ago (kind of like Nebuchadnezzar asking his sages to tell him his dream). Effectively, it is probably inappropriate for Academia SE for the reasons explained here: https://academia.meta.stackexchange.com/questions/3657/why-was-my-question-put-on-hold-for-shopping.

Honestly, the search tool that helps me the most is just plain-vanilla Google.

@Tripartio I’m not asking you to tell me what I might of heard, I’m asking for the name of portals that are more comprehensive than Google Scholar which came into widespread use in academia since I left (which therefore might be the same as what a group of college students and professors were telling about). Respectfully, I think it was objective and within scope.

Web of Science is a subscription-based citation database that came out shortly after Google Scholar. I can't compare it to other services such as JSTOR and Galileo (which I have never tried) but at least anecdotally I find it to be better in some respects compared to Google Scholar (better UI and more trustworthy citation counts, at least in my field). However it is subscription-based so whether or not you have access will depend on your employer/institution.
Edit -- There is another citation tool that I feel is criminally underutilized so I feel compelled to promote it. The arXiv has a fantastic browser add-on called the arXiv Bibliographic Explorer which provides detailed information about citations for nearly every article on the arXiv. Even though this is not directly related to your question, I have found it to be an extremely useful tool that not many people know about, so it deserves mention.

I doubt Galileo is still around. It was a state funded tool in the state of Georgia.

@Hack-R https://www.galileo.usg.edu/

More comments on Web of Science: launched 1997 (according to the box that comes up when you google it; can't find that info elsewhere).  That box also says "90 million +" records indexed. Google Scholar didn't come out until 2004 (Wikipedia). GS does full-text search, IIRC WoS only does keyword/etc.  GS is crippled in some ways (downloads, sort by date, etc). Also: see this comparison of GS/Scopus/WoS

@BenBolker Thanks for pointing out it came out in 1997 and not "shortly after Google Scholar" as I indicated in my answer. That was based on the earliest archived snapshot I could find on the Wayback Machine

For the history buffs among us, as discussed on the Wikipedia page etc., WoS goes back to the ISI Science citation index.  An online version was available in the UK as early as 1991. https://www.sciencedirect.com/science/article/abs/pii/016975529290038R

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Meta-analysis using Google Scholar - are all years equally complete?
Like so many others, I scrape data from Google Scholar as a part of my lit review process, so that I can have a structured data set for meta-analysis of the literature.
I noticed that for a couple of many topics of interest, the # of articles per year seems to be increasing until 2017, then it drops off sharply.
I wonder if it's really safe to assume that fewer articles were published in 2018?
Is it possible that this means that the data through 2017 is relatively "complete", whereas journals and authors for 2018 may still be in the process of being added to Google's index, thus the total number is under-reported?
Has anyone encountered this?

maybe you could check and compare this observation with other search index's like scopus and ISI web of knowledge. Also arxiv.org has a statistics section to check this https://academia.stackexchange.com/a/126982/41661

@user847982 I like that idea, though I guess there's a hurdle of needing to write a separate data scraper for those. I was also wondering if I could try using the same scraper with Google Scholar and different topics, to see if I see this happening across the board within Google Scholar

Google Scholar has its strong points (e.g. indexing of grey literature that is not available in any regular scholarly database), but data quality is not one of them. Of course, this is not because Google lacks the ability to create a high quality database; it is rather because publishers refuse to grant it permission to create a high-quality database that it distributes for free. Google's index is based on Google Scholar's web spider whose completeness depends on what is available from public websites (Google strictly respects websites permissions; it makes no attempt to index anything where the websites ask it not to do so with a robots.txt entry). I would not be surprised if some publishers restrict Google's permission to index details of some of their most recent publications.
With that perspective, then for any given topic, if there is a sharp dropoff during or after 2017 (its unclear which is the case the way you worded the question), I would not consider that evidence of anything. That is, it is not necessarily evidence that people suddenly stopped publishing on that topic; it is only evidence that Google's index no longer contains that topic, for whatever reason. I know that I've seen quite a few articles that have charts like that and make claims like that, but I don't consider such claims reliable. (And when I peer-review articles that make such claims, I tell the authors so.)
To make any concrete, serious claim about change in publishing patterns of topics, you would need a more rigorous and systematic database source (such as Web of Knowledge, Scopus, etc.) and at least a two-year lag to make sure that all data is complete.

Thank you very much (+1). I agree with you on the 2 year lag and much of what you said. Are you sure that Google Scholar creates its create index in the same what that regular Google search does? It would be good to have a link to something discussing that. I was under the impression that they combined self-reported results from the journals and authors with some special/different data scrapers/connectors built for the journal sites. That was just my perception though, so I may well be wrong. Either way, the 2-year lag is a good idea. How do we know that WoK or Scopus are better?

@Hack-R as a meta-analyst myself, I would say to be careful with Google Scholar, especially regarding it's limitation (no wildcards, limits of characters in search). From my experience, WoK has way more possibilities. You should check published meta-analysis in your field and note where did they do their systematic search.

@Emilie Thank you as well; that is useful. Based on your recommendation and the one in this answer, I will write a data scraper for it today.

@Hack-R, about WoK or Scopus being better, I don't have any citations to give you, but I have colleagues who research bibliometrics and for them it's not even a question. Even though the Clarivate (WoK) and Elsevier (Scopus) teams have already done a lot of cleanup from what the journals give them, bibliometricians still need to do a lot of their own cleanup. In contrast, Google Scholar is hopelessly unusable for precise scientometric purposes, to paraphrase one of my bibliometrician colleagues. It's great for general searches, but not very useful for scientific bibliometrics.

@Hack-R, no, I'm pretty certain that Google Scholar uses custom spiders different from those of the regular Google Web search; however, it is all Google technology. (Please note I said "spiders", not "scrapers".) I'm editing my answer to make this more explicit.

You might find this open access resource helpful: Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources (http://dx.doi.org/10.1002/jrsm.1378)


text:
Pros and cons of writing a book review?
I am a postdoc in a biology department. I am friends with a professor in the philosophy department who is the editor of a Nature journal which publishes book reviews. A 150-page popular science book on biology has just come out and he has asked me to write a 3-page review for it. I would have 6 months to read the book and write the review.
What are the pros and cons of doing this?
An obvious con is that it takes away time from me doing research. There are no obvious pros to me, although it does not seem like much work and it might be 'fun'.
(Wasn't sure what tags to use, if someone has suggestions then please edit.)

How is a Philosopher writing book reviews in Nature magazine? Isn't Nature about biology and maybe physics?

@einpoklum It is a journal published by the Nature Publishing Group

There are several benefits of writing book reviews for publication:

Develop an understanding of the publications that are out there. Just like how reading academic papers affords you the benefit of being abreast with the latest research, writing book reviews allows you the opportunity to learn more about a field.
Develop a relationship with a journal. Journals need people to write book reviews. It can never hurt to build a relationship with a journal and its editors.
Develop a profile in the academic community. Book reviews are in no way akin to writing an academic article on a research subject. But they can put your name out there and allow you opportunities to become better known to you academic community.

All told book reviews allow your voice to be heard on a subject.

It's also a good way to work on improving your writing.

It's also an advantage to be able to do a friend a favor.

“It can never hurt to build a relationship with a journal and its editors.” Well you can receive spam requesting more of the same until you die or your e-mail address changes, for one. I wouldn't call that part a relationship, relationships are between humans, not between a human and a database of poor souls who were helpful once.

Since Vladhagen only mentions pros, let me list cons also. They may or may not apply to you. Feel free to edit my answer to add more cons.

As you've noted, It takes away time from your research. Moreover, it can also take time away from other important things like preparing for teaching or preparing for meetings, talks etc.
As some researchers feel the need (or the pressure!) to do the same amount of "real work" (i.e. research) every week regardless of what else they have to do, it could also take away free time and/or time you could spend with your partner, children, friends or hobbies. Especially if you procrastinate and do all the work in the last week. Don't underestimate the amount of work such a review is!
(I assume the review is public, not private.) Depending on the attitudes of the people in your field, you could step on someone's toes: If you write something, some people may not like what you have written, get angry and you could lose reputation with them. Of course, you can also make the authors themselves angry if you write something bad about the book or show some misunderstanding in the review.
If the book is controversial, so will your review be; you can probably not do anything right with the review. I heard about a math research book (in Model Theory, I think) where the author inserted in every chapter a (unrelated) pornographic/sexual picture. I wouldn't want to be a reviewer of this book -- it's hard to do such a review right, especially if the author is much more well-known than you. For a popular science book, scientists often argue about whether a book is "too scientific written" or "too less scientific written".
It can be hard and dull to read a book carefully for a long time, especially if the author does not write too well.
Readers who buy/read the book because of your review **may get angry if they don't share your opinion. So many times I wanted to learn from books where the reviews said they were "didactically well written" only to find out that the reviewer had no idea about didactics and the book was not didactic at all.
You may lack the knowledge about (some of) the topics the authors write about.
People may look at your CV and think of you as unproductive because you "wasted" your time with this review.  Unfortunately, those people exist -- on this site, there are even stories about hiring people who see good teaching evaluations as negative! Of course, one could say "those people are stupid, I don't want to work for them anyway" -- in reality, in academia one often does not have so many choices (especially if you are not geographically flexible).
You are make a commitment to complete the review. While most people are hopefully understandable, some people might not like it if you cannot finish your review because of something more important to you which might come up (new position, new partner, new children, sickness). Most likely, reneging is worse than not accepting to do the review.

Of course, some of those points may not apply to you but I found it important to list them all anyway.

What is the name of the book you mentioned ?

The book with the pornographic pictures is the self-published French edition of Bruno Poizat's Groupes Stables.  The English translation, published by the  American Mathematical Society, omits them.  (Poizat is a model theorist, which is probably what user109301 was thinking of; his other well-known book is Cours de théorie des modèles / A Course in Model Theory.)

Disappointingly, James Loveys' review of the French edition (J. Symbolic Logic, 1989) ends, "And the pictures are nice." Thomas Scanlon's review of the English edition (Bull. AMS, 2002) is very positive about the book itself but scathing about the pornography (which he describes as "tasteless" and "cross[ing] the line between offensive and insulting") and Poizat's self-justifying, faux-apologetic foreword to the English edition.

"the author inserted in every chapter a (unrelated) pornographic/sexual picture"  I'm almost afraid to ask, but what would be related pornographic images to Stable Groups?

@TripeHound Models engaging in group activities in a stable?

@TripeHound Well, stability is closely related to *cough* model theory.

Writing a review for a controversial book can be a very strong pro instead of, as you say, a con. There are several scientists who are known to the wider public mostly because of their acerbic reviews of controversial books. Yes, this can go either way but it’s definitely not unambiguously negative.

One small advantage is that you get a copy of the book "for free" (admittedly not really worth it in terms of the time you'd put into writing the review) ... This is more of an advantage for expensive technical books that you'd like to own.


text:
What happened to the Journal of Artificial Intelligence?
I'm writing a survey paper in my own research area, and came across this publication. It was published in the "Journal of Artificial Intelligence", and has a fair number of citations. The doi link for it is a site called "scialert.net", which is certainly not the journal's page.
However, when searching for this journal, I am unable to find any information about it. Does anyone have information about what happened to the journal?

I see that the web site hosts many journals: https://scialert.net/journals.php

Googling...
https://www.ansinet.com/guideline.php?issn=1994-5450
https://scialert.net/jhome.php?issn=1994-5450
Some random recent article:
https://scialert.net/fulltextmobile/?doi=jai.2022.1.8&org=11
The article you were looking for: https://scialert.net/fulltextmobile/?doi=jai.2011.156.165&org=11
Looks like it's still in business.


text:
Why is it bad to judge a paper by citation count?
I hear a lot from the experts that citation count is a bad idea as a measure of judging a paper. This seems simply counter-intuitive to me. I would like to know if any study has been done in this direction?

Of course papers in medicine are 20 times more valuable than papers in geology.  That's why they receive 20 times more citations!

I don't have time now to find them, but there are some studies.  Not just "opinions" like the answers we have up to now.

For the exactly the same reason that judging intelligence by income is a bad idea.

Because I need to cite paper X if I want to say that its method sucks.

Why would it be a good idea?

Why does this have at least 13 upvotes? It looks to me as a silly question… not worth closing, sure, and deserving a good answer (which it has), but upvoting?

If citation count is the measure of a good paper, and (as you seem to assume) all citations are positive, how would any paper ever get its first citation?  At best, you've got the relationship backwards.

@Lohoris Because it is about a popular subject, and it is simple enough that everyone understands it. Many views means many upvotes, and upvotes attract views. See answers for further explanation of this effect.

One reason experts agree that citation counts can be misleading is personal experience.  For example, I certainly don't think my most highly cited paper is the best, most important, or most interesting of my papers (in fact, everyone would agree it isn't even close).  In all the cases I can personally judge, there's only a loose correlation between citations and quality.  One striking example is Einstein's papers: his joint paper with Podolsky and Rosen has far more citations than his discovery of special relativity, but the idea that this makes it superior is laughable.

Dear Amrith, as you can see you asked a controversial question. It's one of the subjects people in academia get all emotional about. Your question is perfectly valid, cannot be answered with 'ask your adviser' for a change, and, as you can see, harder to answer with references than it seems. It has the potential to attract a 'canonical' answer that we can cite in the future. Thanks, and keep asking such questions on this site.

@Lohoris how deserving a question is of an upvote is not a binary thing. Poor questions might get fewer upvotes, but it's a destructive notion to say that any question that is not good deserves zero upvotes.

@Moriarty actually it's quite productive. The whole point of having points is to incentivize good post and to deter bad ones.

@CapeCode doesn't seem controversial at all to me. Everyone agrees it's a silly thing to do under any circumstance, from what I've read so far.

This famous paper http://www.ncbi.nlm.nih.gov/pubmed/8137688 "discovers" an approximation method known since Newton, and I think it has about 200 citations.

@Lohoris, isn't the question the OP is asking and the question about the question itself you are talking about approximately the same things? The cite count/upvotes say very little about the quality of the paper/question!

Citation counts are like the stock market. They tell you something about people's value perceptions without necessary telling you whether these are based on valid "fundamentals". To quote Keynes: "We have reached the … degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be"

Even if citations be a good metric in nature but It suffers from "richers get richer" and also aging problem (the old papers get more citation just because they are online a lot of time).

Perhaps just agree that it is a bad measure, just better than the others that have been attempted.

Well, in many respect using citation count to e.g. choose prize winners would be akin to giving the Nobel prize in literature to the author who sold the most books. Sales count do have a relation with quality in many senses, but a very complicated one. Same goes with citation counts.

I do not know of any study in this field, but I have a hunch that the following points apply (without any particular order, just numbered for convenience in the case of comments/replies):

Opportunity for citation: A paper that presents the ultimate answer to a problem is probably more valuable than a paper that presents just an unfinished attempt at solving the same problem. Yet, the latter is much easier to build upon (because it still leaves obvious extension points) and thus might be cited much more frequently.
Meaning of citations: There is no rule that a paper needs to be cited positively. If a paper has an obvious flaw, this may give rise to a number of other papers that just cite the flawed paper to point out what they are about to prove wrong or do better.
Citation scope: Referencing a paper does not mean referencing the whole paper, or its core finding. A reference to a paper might mean that the whole paper is pointed out as related work, or it could just mean that the second half of a particular statement in the introduction section is pointed out as related. A citation can well point out a remark that is completely marginal to the referenced paper, but that the citing author deemed to explain the fact in question particularly well, or that made a particularly recent mention of an old topic (to show the topic is still of interest).
Not all citations are equal: The question whether something is or is not cited hinges on its relative relevance compared to other citations, based on the available space left in the current paper format, not on the global importance of the work.
Citation habits differ: Even within one (sub-)field, citation habits differ wildly between conferences and authors. This is in part due to different paper formats and length restrictions (see previous point), but also simply because some authors are much more willing to throw in citations every now and then than others. Such habits can even vary for a single author, based on "strategic" considerations for the intended final target audience of the paper.
Citations can be transitive: Authors occasionally face situations where they can either list several examples of related work, or cite a single overview paper (e.g. a state of the art report), to give an overview over some basics or otherwise related topic. The latter case is clearly more space-efficient and possibly more helpful (as the overview paper is designed to give an overview while at the same time conveying the information somewhat in-depth, which often cannot be done in a single sentence of a related work section), but does not increase the citation count of the works referenced in the overview paper.
Papers are not atomic: There is no "natural" clustering of research questions, solutions, works and experiments. How many or how few findings go into/are allowed in a single published work is mostly a matter of style and personal preference of authors and program chairs. With that in mind, any particular finding may be presented in a single paper (which gets N citations), or split up into N papers (each of which may get only one citation). Thus, while the number of citations in this example may give a hint about the importance of each of the single written documents, it does not say much about the importance of the actual set of research questions.
Citation count is not normalized by field/community: As correctly pointed out by Cephalopod, '"What constitutes many depends on the field". Groundbreaking work in a narrow field might never get many citations.' Likewise, fields where publishing small bite-sized papers at a high frequency is common naturally amass a much larger total amount of citations than fields where publications are very seldom and, when they occur, cover a large amount of progress, discussion and findings.
Citation tweaking: As any metric, defining such a single factor for measuring performance, citation count is prone to abuse. If an author suspects citation count to be beneficial, there is an incentive to self-cite (or cyclically cite each other within an institute), and possibly even break up publications into smaller parts (related to the practice of salami slicing) to have more opportunities for those extra citations. As it can be assumed that some authors are already doing that, this renders citation count even less reliable as a quality indicator, as the "training data set" that would be used for comparison is already skewed.
Citation count ≠ citation count: The citation count itself is not a reliable measure to start with, as different sources will provide different citation counts, depending on what citations (and what kinds of documents that contain citations) are counted:

Citations in a book can be counted by chapter (point in favour of this: If each chapter is written by a different (group of) author(s), every chapter has its individual list of citations), or for the whole book (point in favour of this: A book tends to have a certain focused topic, and - more so than e.g. conference proceedings - all chapters tend to follow a common "narrative", so some central works are likely to be referenced in various chapters).
If possibly predatory journals are included, a variant of item 9 can ensue in which citations in less reputable sources (that are counted nonetheless) are bought, as pointed out by ff524.
Citation counts can include citations from totally legitimate works, which are nonetheless of a vastly different status compared to peer-reviewed publications. For example, Google Scholar sometimes seems to count student Bachelor and Master theses and other documents created while studying if the university's digital library that provides a copy of every such submitted document happens to be crawled by Google.

External factors influence selection of cited works: If there is one particular finding in one particular work that is appropriate to cite, there is no question that that particular paper will be cited. However, often enough, things are much more vague - there are various eligible papers that allude to the same statement in different ways, and it is not at all clear which one is preferrable to cite. At this point, external factors that are rather arbitrary can directly influence whether or not a work is cited. For instance, the older paper might be cited to show the original finding ... but the more recent paper might also be cited instead to show that the topic in question is still of interest in recent works. The older paper might be chosen because it is the original reference, or the more recent paper at the more highly-valued venue might be chosen to make the reference stronger, or yet another paper might be chosen because it contains the more comprehensible explanation. Lastly, depending on the citation style, even space may be an issue - if an author is fighting for the last few lines to stay within the allowable page limit, at least in APA style, a paper by Li et al. has an inherent advantage over one by Miller and Bowman, one by Miller and Bowman has an advantage over one by Erdenebileg and Papadopoulos, and one by Erdenebileg and Papadopoulos has an inherent advantage over a paper by Russell-Rodriguez and Pennington-Kiesewetter.

You missed Well laid out papers (or am I just thinking of posts :-) ).

@Cephalopod: Very true, added it.

An example on point 5: I'd say in my field is it typical to have 5-30ish references on a normal length paper. One author usually cites more like 80-100.

There's also 9b: authors who bypass the subtle method and just buy citations

Your hunch has significant overlap with what is reported in publications on the subject.

Very nice! Also, addition to the fist point (opportunity for citation): a paper in a widely circulated journal will much easier cited than a a paper in a local / expensive / new / less prestigious journal for the simple fact that more reader can reach them independently from the actual quality of the paper.

@Greg: I think that's included in item 11, "the more recent paper at the more highly-valued venue might be chosen to make the reference stronger". I am reluctant to add that as a point of its own, because it could be argued that while a paper on a better venue is not automatically better, there might be a certain correlation between better venues, higher visibility, and a higher likeliness for high quality of a paper. It's not at the same level of arbitrariness as the other points.

@O.R.Mapper You are right. But I was also referring to cases when society journals and for profit journal have different pricing and availability. E.g in Japan, American society journals are much more available than UK or German ones, and people prefer to publish in American journals, too. That has nothing to do with the quality of the journal, but more about other cultural/political factors. Or high impact new journals, e,g Nature-family journals can be valuable, high impact yet hardly available ones. Again, it is nothing with the quality, but budgetary considerations.

I would also say that the citation count isn't normalized by time. A brilliant paper published six months ago simply hasn't had time to have it's citations start to appear in the literature, compared to a paper that's been gathering them for years.

In support of Point 2, see Is it ethical for authors to reference another paper, but not cite it formally, because they consider it unscientific?. It is about trying to write about the defects in a paper without adding to its citation count, presumably in case people equate citation count with quality.

Regarding point 11, there's also some evidence that papers with longer titles or names are more likely to get cited. See https://www.sciencemag.org/news/2015/08/brief-papers-shorter-titles-get-more-citations-study-suggests https://royalsocietypublishing.org/doi/10.1098/rsos.150266 . There's some evidence also that longer titles are not not get counted in citation counts because they are more likely  to have the citation garbled to an extent that software like Google Scholar doesn't recognize it as a citation for the work in question.

@JoshuaZ: Based on your two links, it seems you meant to write "shorter" (?) Good point about longer titles providing more opportunities for citation mistakes that prevent an automated recognition of the citation.

@O.R.Mapper Yes, sorry. Shorter, not longer.

Citation count is a good example of a phenomenon subject to the Matthew Effect: a feedback process in which privileged individuals become more privileged as a result of their privilege.  Quality, of course, does have a significant correlation with a paper drawing citations.  Its citations are also strongly affected, however, by the fame of its authors, the fame of its venue, and also simply by the fact that other people cite the paper (thus making people like yourself judge it as "higher quality" and therefore more likely to cite it). Likewise, the mere fact that a paper is obscure does not mean that it is bad.  Mostly, it simply means that nobody is paying attention to it.  Finally, the fact that people are happy with a result doesn't necessarily make it true, as any number of scientific shifts in thinking (not to mention scandals) can attest; a paper may even draw citations precisely because other people are criticizing it.
Citation count is still a good way of getting a good first impression of which papers other people consider important, but you really need to judge the quality yourself, as a scientist.

tl;dr.
It only works under certain conditions.

There is actually a whole field of study that explores the scientific process via publication metrics, it's called bibliometrics. It is true that citation count within a field is sometimes used as a proxy to estimate an article's quality. See for example this recently published study about peer review:
Siler K, Lee K, Bero L (2015) Measuring the effectiveness of scientific gatekeeping PNAS  Jan 13;112(2):360-5. doi: 10.1073/pnas.1418218112
From which the highlighted paragraphs below are copied.
Arguments in favor
First, the reasons given to consider citation count as a measure of quality (I edited the references to match the ones given at the bottom of this post):

Scientists cite work for a myriad of reasons (1, 2). However, the
  vast majority of citations are either positive or neutral in nature
  (3). We worked with the assumption that scientists prefer to build
  upon other quality research with their own work. As Latour and Woolgar
  (4) suggested, citation is an act of deference, as well as the means
  by which intellectual credit and content flows in science. Relatedly,
  we also assumed that most scientists want to produce quality work and
  will seldom attempt to garner credit and attention by blatantly doing
  bad work. Thus, on the whole, the attention and impact associated with
  citations provides a reasonable measure of quality. Citations provide
  an objective and quantitative measure of credit and attention flows in
  science.

Arguments against
They also discuss the limitations of using citations as a measure of quality, the logarithmic distribution of citations:

Because citations are often distributed exponentially, with a few
  articles garnering disproportionate attention (5), we also used the
  logarithm of citation counts as a dependent variable to diminish the
  potential influence of a few highly cited outlier articles

And the 'social status' effect:

scientists often rely on heuristics to judge quality; status of
  scholars, institutions, and journals are common means of doing so (6,
  7). Unsurprisingly, citations received by manuscripts were positively
  correlated with the impact factor of the journal in which it was
  eventually published.

In short, comparing citation count of articles published in journals with a very different impact factor is a bad idea.
Pages 46 and 47 of reference number 3 give an exhaustive list of limitations of citation count as a proxy for quality (I edited the list for the sake of brevity):

Time-dependent factors Due to the exponential increase in scientific output,
  citations become more probable from year to year.
Field-dependent factors Citation practices vary between science and social
  sciences fields [...] and even within different areas (or clusters) within a single subfield [...], small fields attract far fewer citations than more
  general fields [..].
Journal-dependent factors [...] journal accessibility, visibility, and
  internationality [...] as well as the impact, quality, or prestige of the journal may influence the probability of citations.
Article-dependent factors [...] There is also a positive correlation
  between the citation frequency of publications and the number of co-authors of
  the work [...]. And, as longer articles have more content that can be cited than
  shorter articles do, the sheer size of an article influences whether it is cited [...].
Author/reader-dependent factors The language a paper is written in [...] influence the probability of citations. [...] citations are affected by social networks: authors cite primarily works by authors with whom they are personally
  acquainted. [...]  men receive substantially more citations to their work than women do.

Many regular users of this site will love that one:

Availability of publications Physical accessibility [...], free online
  availability of publications [...], and the publishing media [...] influence the probability of citations.

And lastly:

Technical problems [...] The incorrect citing of sources is
  unfortunately far from uncommon: Evans et al. (1990) checked the references in papers in three medical journals and determined that 48 percent were incorrect:
  “The data support the hypothesis that authors do not check their references or
  may not even read them” [...] In a similar investigation,
  Eichorn and Yankauer (1987) found that “thirty-one percent of the 150 references
  had citation errors, one out of 10 being a major error (reference not locatable)” [...] Additionally, problems stemming from homographs and synonyms can arise
  when researching publications and deriving citations from citation databases
  using authors’ names.

All these are linked to plentiful citations that you can find in the original document available for download here.
References

Lynn FB (2014) Diffusing through disciplines: Insiders, outsiders and socially
influenced citation behavior. Soc Forces 93(1):355–382.
Hargens LL (2000) Using the literature: Reference networks, reference contexts, and the social structure of scholarship. Am Sociol Rev 65(6):846–865.
Bornmann L, Daniel H-D (2008) What do citation counts measure: A review of studies on citing behavior. J Doc 64(1):45–80.
Latour B, Woolgar S (1979) Laboratory Life: The Construction of Scientific Facts (Sage, Los Angeles).
Lotka AJ (1926) The frequency distribution of scientific productivity. J Wash Acad Sci 16:317–323.
Long JS, Fox MF (1995) Scientific careers: Universalism and particularism. Annu Rev Sociol 24:45–71.
Lee CJ, Sugimoto CR, Zhang G, Cronin B (2013) Bias in peer review. JASIST 64(1):2–17.

First off, don't trust generalizations too much. The number of citations is not a perfect measure by far. First, more citations take time to accumulate, once could possibly look at citations per year or something similar. Second, it is possible to cite your own work so without filtering out so-called self-citations you may see inflated values. Self-citation, in itself is not necessarily an evil either, there are many reasons why one must reference ones own earlier work. One obvious reason is that later work often builds on earlier work and part of that is usually earlier work by the same person. Third, the number of citations are field dependent and in bibliometrics methods exist to remove such bias. Hence an article in a hot topic with much research will receive more hits than an article, no matter how excellent, in a small field.
So, when judging an article from its citations, it is useful to keep the problems in mind and not over-interpret. This is essentially not different from any normal approach to any data set.

"First off, don't trust generalizations too much." is a generalization you expect me to trust :)

The reason citation counts are not good for judging paper quality is that there are a great many reaons (15, 28, 26 - depending on the study) for citing, and citation practice vary between disciplines. See:
Case, D O & Higgins, G M 2000 How Can We Investigate Citation Behavior? A Study of Reasons for Citing Literature in Communication. JASIS 51(7) 635-645
for references to earlier studies, and report of a study in the discipline of communication.

Others have mentioned, but it bears repeating - citations are not necessarily positive. Plenty of papers use faulty methodologies and are cited as examples of what not to do.
Also, at least within the social sciences, citation is correlated with age and 'first mover advantage'. Because any academic paper worth its salt is going to cite previous work on the topic in the literature review, older papers will naturally garner higher citation counts, all else held equal. This has very little to do with the quality of those papers.

To me, if a paper is cited as an example of what not to do, then that problematic paper is definitely an important contribution to the scholarly conversation. I expect that truly bad and insignifcant papers would have very low citations, but if a paper is wrong but heavily cited, then it apparently has an important influence on the field. So, according to this logic, high citations do not necessarily reflect "quality", but they do reflect "impact" on the field.

There have been many good answers to this question, but I wanted to frame what I see as the most important points.
Citation counts measure impact not quality:
The citation count of an article is a measure of research impact, and not quality. Even as a measure of research impact, it has issues, but in general, "research impact" is closer to what it represents. A high quality new paper will have minimal citations, because it has had minimal time to have an impact. Similarly, the quality of a paper is only loosely related to research impact. Some high quality papers are difficult for the literature to digest. Some low quality papers evolve into standard publications that are commonly cited in a given context. Entire literatures may use questionable methods but build a whole ecosystem of mutually reinforcing citations.
Citation count is confounded by time since publication:  One of the biggest points is that:

citation_count = average_citations_per_year x years_since_publication

Time since publication is unrelated to quality, yet it is one of the main drivers of article citation count. Thus, even if you did see citations as an index of quality, average citations per year would be a better index than total citations. Even there, it gets complex because articles in a given field tend to have a general distribution of citations over time where for example, citations per year typically tends to peak early and then gradually declines over time.  So if you were trying to get a pure measure of quality based on citations, you would probably try to estimate the expected citation count after 30 or more years based on how many years has passed and known citation distribution characteristics.
Assorted other points:

Fields differ in citation practices: Some fields have short citation half-lives so that more of the total citations an article will receive typically accrue in a shorter period of time (e.g., 5 years versus 15 years). Some fields include more references and therefore there is a greater sum of citations in the system. In general, this is more of an issue if citations are being used to do comparisons across disciplines. If you operate in a particular field, then this is less of an issue. And if you are aware that STEM fields tend to have shorter citation half lives and more references relative to social sciences, then you can incorporate this into your perspectives.
Self-citations can be removed: Many systems like Scopus allow for the removal of self-citations. This can be useful where you suspect such gaming is occurring (or generally where there is an academic that publishes a lot and self-cites a lot without getting citation traction with other academics).

Using article citation count as an index of quality
While most answers here point out the problems with citation counts, it is worth taking a balanced perspective.
After five or ten years after publication, average citations per year provides some information regarding the quality of a publication. Actually, it is a measure of impact potential as indexed by achieved impact over a finite period of time.
It's not definitive, but it is a useful bit of information. If you want to assess quality, either you or someone suitably skilled needs to read the article and appraise it.
In particular, if after five or ten years, an article has almost no citations, then this is a bit of a red flag for the quality of the paper. Likewise, if an article has hundred of citations per year, this suggests that it is likely to be an important piece of work. None of this is definitive, but it is suggestive and useful information.
General points on how to use citation counts

Citation counts are a measure of impact, not quality
Average citations per year is a better measure of the "impact potential" of a paper than total citations
Citation counts become more informative as more time since publication accrues. As a rough rule of thumb, five years post publication is probably a rough minimum to get a somewhat accurate sense of citation potential.

First of all, what does it mean to better?
I know this may sound like nitpicking, but I can provide an example that gives a strong negative correlation between citations and a certain measure of better. I typically write algorithms for my research. Generally speaking, you don't get to publish unless your algorithm is an improvement in some meaningful way over previous algorithms. But you certainly need to cite previously published algorithms! Thus, for a given problem, the early algorithms will have a very large number of citations, while the later algorithms will have a much smaller number of citations.
So if your definition of "better" is that the algorithm is faster (a very reasonable definition), then the papers with large numbers of citations are likely to be the worst in the literature (since they are the oldest).
As an anecdote, I just published a paper on an algorithm that I have reason to believe will close the book on optimization for that particular problem (partly because it's really fast, and partly because it's not a very hot topic). As such, I don't expect that paper to get many citations, even when compared with previous algorithms on that same problem.

On top of all the other good points made already, there is also Goodhart's law to consider:

A measure ceases to be a good measure once it becomes a target.

I think it boils down to the question:
Do you use citations as a measure or as a metric.
A measure is a value you get by measuring a quantity (e.g. miles, seconds, citations, words). A measure is a mostly objective attribute. A measure does not have any meaning, it is only a quantisation.
A metric uses usually multiple measurements and combines them. After the combination the metric can be used and compared to a set of rules or other metered objects in order to sort or value them. The combination of measurements is the step were a) meaning and b) subjectivity enters the whole process.
It is therefore absolutely understandable to use citations as a measure but you will most likely never use it as a metric itself. When the number of citations is used as a metric for paper-quality, assumptions need to be made which cannot be proven (except by measuring something else as well).
Only because one can measure something does not mean that the measurement contains any information.

Could you elaborate on what the difference is?

Could you clarify the purpose of a measure that is not used as a metric, please? As far as I understand, your answer basically says a measure is the measured value - at which point the the statement "citations can be used as a measure" seems to make little sense, as the number of citations is a measure. Citations can be used for obtaining a measure (their number), but no-one doubted that fact itself, whereas the question whether that measure should be used remains unanswered. Concretely: How does the "understandable (...) use of citations as a measure" happen without using them for ...

... a metric? How is something used as a measure, but not as a metric? If a measure has no meaning, as you write, how is it "used" on its own?

That is exactly the point I am trying to make. The question is if one can use the number of citations to judge and the answer is: since it is a measure you can use it but only if you compile it into a meaningful metric. Basically everything can be a measure, but a measure does not have a meaning. If you want to judge something you need meaning and therefore (in this case) the number of citations alone is meaningless and needs to be combined with other measures.

A metric uses usually multiple measurements and combines them — I think you misspelled "is a function that is symmetric, non-negative, positive for distinct pairs, and sub-additive".

Doe, Umanskaya, Milton and Bennt suggest in [DUMB 1999] that it is indeed acceptable to evaluate papers based on their citation count, and that paper of theirs has over 5,000 citations, so we can probably take their word for it.

@corey979 - I think the intent of the joke is, please don't bother to examine the validity, since it has 5000 citations. It must be bona-fide, so we can take their word for this. As for einpoklum - Well played, sir :)

Human beings, regardless of intelligence level, generally tend to subscribe to the "herd mentality". I guess it's because we, as animals like any other animal, want to conserve energy. For example, if everyone believes Stephen Hawking or Lisa Randall is rarely wrong, then it's in my best interest to focus on what he/she has to say. It will save me the energy of exploring other sources and finding answers for myself.
At someone's request I will try to clarify my answer.
Let's say I'm writing a paper about black holes. Obviously, I want to write the best possible paper but I'm not the best physicist on planet Earth. The best physicists on planet Earth will probably have the most citations on the subject matter. Does having the most citations about a subject mean that the highly cited individual is always correct concerning the subject? No. I have to realize that if I'm referencing that person's work. Assuming that Stephen Hawking is right all the time about black holes or even assuming that he is right most of the time is a huge and dangerous assumption.

I am not sure what you are trying to say with respect to this question. Can you make your answer more explicit, please?

Ok ... I think you are providing an example to the Matthew Effect mentioned in jakebeal's answer. I see. Sort of.

You are correct. I had no knowledge that there was a named "effect" for what I was describing. I've understood this concept most of my life without ever realizing some guy understood this phenomenon before me and had an "effect" named after him. I guess you can say I independently discovered this "effect" and should share the notoriety.

Actually, Hawking is quite controversial in some of his theories (that, of course, adds more citations).

@IsaacWannabeeNewton Actually, the Matthew Effect is not named after its discoverer, but rather is named after a biblical passage talking about the rich getting richer.

I think your first paragraph is not quite the same as the    Matthew Effect. I think you touch on the fact that, for most people, it's more cost-effective (energy-wise) to just 'pick-up' the general consensus on a subject than think for themselves. It is, however, somewhat unsettling that this is present here, of all places. But politics affects science as well. Try, for instance, publishing a paper (however well researched) on a topic that's not considered 'politically correct' and you'll see what I mean.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Comparison of current work to the state of the art in a (journal) paper
My last paper evaluations in journals (rejects/major reviews decisions) emphasized I should make more clear the novelty and strengths of my work in comparison to the state of the art.
Which style is preferred?

After presenting each related work, make a brief comparison with my work when possible, or:
Make a summary of the strengths/novelties of my work as the last paragraph in the "Related Works" section, and point the differences and advantages over the cited works

This is likely field dependent. In the Intro/motivation one would cover the previous work, and point out that you are doing something different. You then discuss your work in detail. Then I'd have a section discussing how your work improves on (or not!) the other approaches, detailing when (and why) it would be best to take your approach. It isn't something you just bolt onto the paper, it has to become part of the paper throughout.

Here are some suggestions based on things I learnt from a seminar on academic writing I got back from just yesterday:

First, what follows assumes that the work you did is truly novel. However, unless you are doing a straight replication of some other work, then there must be some novelty in your work. Your first job is to clearly identify in your own mind what is truly novel, unique and valuable about your work. You need to be very clear about this before you can persuade anyone else.
For the literature review, rather than looking for what literature exists and then comparing your work to the literature, try to write your presentation of the literature to surround what you actually did. Think of presenting the literature like a doughnut. You want to arrange the literature to look like a doughnut with a conspicuous hole in the middle of it--the goal of your article is to fill in that hole.
In your literature review, it is extremely important to not mention research that is not part of the doughnut. I am certainly not asking you to hide any relevant research (if you try that, reviewers will skewer you); I am rather recommending that you not distract the reviewers with any work that is not directly related to the problem that your article is solving. (And to be clear, the problem you are solving should be what you consider novel and valuable, not more than that.) Your work in the literature review is absolutely not to review all the existing possibly related research; it is only to review the research that is directly related to the problem that you are trying to solve, that is, only research that is part of the doughnut with a hole in the middle of it. Less is more: that is, you do a much better job when you present the least amount of research necessary; too much research that is not directly related will lose the reviewers and make them lose sight of what you're really trying to do.
After framing the literature review section, then go back to the introduction. In the introduction, be sure to summarize in one paragraph maximum the top two points from the literature review that demonstrate that you are filling a real gap in the literature. Note that I recommend that you write this part of the introduction last--because the introduction is absolutely the most important part of the entire article, I recommend that you write it last. I do realize that some excellent writers prefer to write it first, but for less than excellent writers like myself, it is usually best to write the most important part of the article last.

I hope these general tips are helpful.

In some areas the discussion section is prominently present, in others you can use a part of the results to explicitly and practically compare yourself to other methods. So mentions and "dry exercise" separation from other's work goes the way you describe. The actual comparison does not go into the literature review.

@OlegLobachev, one of the strong points of the seminar is that by the time they finish reading the introduction, reviewers in the top journals (in management) have pretty much already decided in their minds whether they want to reject the article or recommend revisions. So, if you wait until the discussion to present your strongest points, it's too late. Although the context was top management journals, I think the principle that the introduction sets the reviewers' emotional attitude towards evaluating the rest of the article is universally applicable.

@Alice I would simply add one or two sentences wherever in the article it fits best, just to make the reviewers happy. This minor concession to reviewers would not make the article unbalanced.

I think that your abstract should feature a statement about in which domain(s) your approach works better. When listing related work, emphasise on their weakness compared to your work (was that mentioned by someone before? cite their work!). Then, in your results/discussion, make clear that you indeed found an approach superior to existing work. If possible, show that through simulations/experimental results/surverys/data.
Also, when introducing your method in detail, focus on pointing out differences to prior work. Keep in mind that since papers are densely populated with complex information, it is easy for readers to skip a sentence or two. (it does not help that a lot of reviewers seem to be very busy).
My current advisor holds the opinion that writing a paper is like selling a used car: while you should not keep quiet about downsides, you should still be very convincing.
Clear and structured writing is also important: to keep the readers entertained, the paper should work like an essay where you argue that your approach is indeed novel.


text:
How can I help a Ph.D. student who is dissatisfied with my department?
I am supervising a mathematics Ph.D. student who is technically strong, inquisitive, curious, creative, and came in with a very positive attitude. He's been dealing with discouragement, and I'd like to help him.
He expressed frustration with our department, and said that he'd like to tune it out entirely and focus on working with me. He wishes that we had more "fire in the belly", and is frustrated that we're resigned to the realities of being a second-rate department and having to make ends meet. He has been frustrated with the remedial teaching he's been asked to do, he hasn't really made friends with his peers, and he's also upset with the ongoing political situation here in the US.
It has always been my belief that, as it takes a village to raise a child, it takes an entire department to mentor a Ph.D. student. This was certainly true for me; I attribute my success in grad school not only to my advisor, but to many other professors and fellow grad students who helped me along the way.
As such, I'm not sure how effectively I can help him. Especially since I am at best an average "manager" when it comes to time management, planning, etc.
What can I do to help my student achieve his goals?

What are his goals?

Most of what they are frustrated about are not the department per se. Not making friends is a real issue.

Does your college have a strong physics department?

What are the ramifications of "tuning out" the department?  The department will probably not "tune out" the student, even if the student tunes out the department.  FWIW, if the student can find a department not worried about making ends meet, the student should run away from it.

He wants to be a researcher, but I haven't pushed him too much on specifics, and maybe I should. I think our physics department is roughly comparable to ours

he'd like to tune it out entirely and focus on working with me

That seems like a reasonable approach to the overall issue. It's unlikely possible to effect wholesale change in the short period of a PhD career, might as well focus on what's most important and strive to make the next stop (post doc, etc) at a department more fitting.
The rest, though, seems entirely at odds with this approach.

He wishes that we had more "fire in the belly", and is frustrated that we're resigned to the realities of being a second-rate department and having to make ends meet

Well, okay, are we tuning it out or planning to fix everyone else's problems? You can't really control other people's belly fire. A PhD student certainly isn't in a position of leadership that might have some minor influence. If the department is second-rate, well, welcome to the party, Student, this is the department you joined rather than a first-rate one: get used to it. For that matter, even first-rate academic departments are just making ends meet. There's not a lot of glut, any excess there is would be used in any department.

He has been frustrated with the remedial teaching he's been asked to do

Well, given the horrid state of high school education in the US this work needs to be done, and it's work that pays the bills for an academic. Seems like an attitude that needs to change.

he hasn't really made friends with his peers

Well of course not, he thinks he's better than everyone else as they are all second-rate and beneath him.

he's also upset with the ongoing political situation here in the US

That surely doesn't make the student unique, especially in US academia.
So, overall, I think I'd coach the student to indeed focus on their research but if he wants to tune out the rest that means actually tuning it out. I'd recommend you try to help the student take a broader view of the world and be careful of grass-is-greener falsehoods, but that may or may not be productive. I don't think your post meant to be negative at all, but yet I get the impression this person may be a bit on the insufferable side; some people like that grow out of it as they get some humility. Others never will, and you won't be able to help, so don't be too hard on yourself if it doesn't work.

+1 because this is a very useful answer, but I disagree so much that I wrote mine :) (especially about A PhD student certainly isn't in a position of leadership that might have some minor influence)

"If the department is second-rate, well, welcome to the party, Student, this is the department you joined rather than a first-rate one: get used to it." This is such a good sentence that it should be made into a plaque and framed.

Thanks for your answer. In my opinion he's a little cocky and entitled, but nothing extreme, and I was certainly guilty of this myself when I was younger. I've never seen him act mean-spirited. I'm trying to encourage the student to look at the big picture and make the best of the situation he's in, but I haven't managed much success so far.

@academic I'd encourage them that at their next stages, the main things they'll be judged on is research output. Anything they've done before now won't really matter: grades in courses, skill on engineered problems, none matters unless they're able to show something new, so might as well focus on that if they want a place in the next phase of the big picture. When they wander into other complaints, redirect them back to those goals. If they're stubborn about it, I'd try to get them to articulate an actual goal for e.g. the department, something more than being just "better", and work on that.

I'd also coach about realities of the world: being experienced in teaching courses, even remedial ones, is important for getting a job where that's where your salary comes from. There are precious few jobs in math anywhere, and effectively none that let you skip teaching.

Try to harness the "fire" of the student into something productive
Some of these problems are things that can be mitigated and some are beyond the control of you and the student.  Since this student obviously has a lot of "fire" the trick here will be to try to harness that enthusiasm into something productive rather than letting it deteriorate into cynicism and withdrawal.  For this reason, I recommend that you have some discussions with your student on potential avenues by which the student could form opportunities for improving the department in small but meaningful ways.  This might involve uplifting the process for remedial education, adding new seminars or groups for students to meet and discuss ideas, etc.
As part of this approach it will be important to separate out things that are beyond the control of the student.  Here the Serenity Prayer offers good advice: "... grant me the serenity to accept the things I cannot change, the courage to change the things I can, and the wisdom to know the difference".  As part of this, I recommend you teach this student that he will need to learn to put up with some frustrations in any job/career and push ahead with his own work regardless.  It is fine to express frustrations at annoying things beyond your own control, but it is also rational to expect that institutions you work in will not always function as you would like them to and that you will inevitably have to put up with some "bullshit" parts of your job.

From your profile, I'll guess that this is mathematics, which was my field initially. I was in a very large department (~50 professors) and most of them were irrelevant to my needs and goals. My advisor was my main research contact, but there was also a small (3-4 professor and a few students) working/seminar group that met periodically. People could ask and answer questions about their research, though each of the students had a distinct problem to work on.
Outside that group there were a couple of other professors that I could relate to, though they were in different fields than mine. But it wasn't my advisor that set up these contacts. It was nothing more than the fact that I admired them for various reasons including their teaching style. I had some contact with them through required advanced courses leading to comprehensive exams.
A student can't, however, address the issue of thinking the department isn't energetic enough, or is rated too low. Complaining about it won't be helpful, whether it is true or not. Those sorts of things take both time and an infusion of money to solve.
But, I think it would serve everybody well to create such a working seminar that meets, say, weekly. Having an open-door policy within the group, especially for students, might also be workable, and would probably be helpful.

+1 especially for the last paragraph, which if carried out has to potential to solve more than one of the OP-student's issues.

I'm not so sure about the last paragraph. I think you owe it to yourself to put a limit on the time you give to this student's issue. Also, I would advise against an open-door policy to disgruntled PhD students with other advisors -- I suspect almost all PhD students become disgruntled at some stage. The difficult cases can be very difficult. I applaud you for wanting to help, and I think all the answers here are worth exploring. However, I would avoid getting locked into an extra weekly commitment. [I am an ex-academic, and arguably one reason I left is that I did too much firefighting.]

@EdWynn, right. Not always workable.

I was, to some extent, this kind of PhD student. I wanted to push brand new methods into my physics department (more computers, neural networks - it was the early 90s).
I made many friends (usually young physicists) and a few hard enemies. At some point, I realized that either I would go for the dean's chair, or leave academia. I left academia because the neo-medieval system was hard to change and I realized that I could do fantastic things in the industry as well (and be paid 5 times what I would be in academia).
Nice story - but what is the link with you as the supervisor? Well, my supervisor was extraordinary in helping me (often behind my back) to navigate the muddy waters of academia. He "used" (← this is very positive) me to push changes and pour oil where there was too much friction.
He was an average physicist but a truly wonderful supervisor. he did not try to stop me and let me drive my (somehow unusual) PhD. This PhD was really different from others but he said "WoJ, this is your baby and you have to face the music. I think that this is great music but tastes differ".
So what is the conclusion, looking back 25 years? If I were him I would do the same: not interfere but help with my experience and knowledge of people.

It has always been my belief that, as it takes a village to raise a child, it takes an entire department to mentor a Ph.D. student.

However true, it also takes one person to hold responsibility for the final outcome. Otherwise, the result can be death by committee.

What can I do to help my student achieve his goals?

You can help him define targeted specific aims with focused objectives that lead to tangible outcomes for a primary goal of achieving a PhD in mathematics in a defined period of time. Essentially, help him become adept at writing a solid dissertation proposal.
Otherwise, you might act somewhat as a sounding board to his current frustrations if only by noting that each level of responsibility in a career path has its own associated types of frustrations and that sometimes the only lesson is simply in learning which frustrations are battles to fight versus battles to leave well enough alone.


text:
Can I replicate Thesis using a different subject of study?
Let’s say there is this dissertation:

The impact of interest rate on economic growth in country X

Can I write my dissertation about:

The impact of interest rate on economic growth in country Y

This seems far too shallow of a contribution for a dissertation. What if you generalized their approach such that it could be applied to any country? That might be interesting. Talk to your advisor about it!

can I follow the same chapters the previous author did? That seems like a really strange thing to do. Surely the more meaningful thing to do is follow the same methodology.

Also, what is your goal? Replicating previous work on a different country is unlikely to suffice for a PhD thesis, but it might form part of your PhD or be a Masters thesis.

Will it be detected by Turnitin? – This is not he question you should ask.

It is plagiarism only if you do it without citing that previous dissertation.

It is not (necessarily) plagiarism (unless you actually copy parts, or merely rewrite the previous thesis), but it would make a pretty boring thesis. A PhD should add something new to a field; investigating the same thing in a different group/country would probably not be enough. Therefore, you can, probably, not write your thesis about it (i.e. the thesis committee/advisor/... would probably not accept it as being sufficient for a PhD thesis).
Unless, that is, you add something interesting. One possibility could be (though keep in mind that I am in a very different field) to say that:

According to previous work, the effect of foo is bar, as shown by studies in Country X. Country X is a dictatorship/christian/... country. There are reasons A, B and C to suggest that foo⇒bar has been observed because of Country X being a ... country. According to our theory, in a democratic/pastafarian/... country the effect of foo should be buu, not bar. We test our theory in Country Y, which has such a system. Furthermore we show that the effect foo depends on [country property] by doing some more studies.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/