text:
Should unreadability or sloppiness of students' work be reflected in their evaluation?
When evaluating and grading students' work in technical fields (like computer science or mathematics), I often face ones that are really messy, sometimes even unreadable (I mean structure, not - handwriting). It's not clear at all what part belongs to what or what is the final result. Instead of spending a lot of time trying to figure out if there is a good result buried somewhere in the mess, I'm tempted to just fail such a work as unreadable.
While these (I'd say) formal aspect don't reflect the ability of the student to solve the problem, it strongly reflects his/her ability to present the solution. And in their future professions, it won't be much good that they can solve problems, if they're not able to present the solution to someone else. Or for example grant agencies simply reject proposals that fail even slightly given formal criteria.
My questions are:

Should the evaluation include evaluation of readability, proper structure etc. for example in a textual form (without impacting the final grade perhaps), something like "The solution is completely unreadable." or for a larger work "The solution lacks a proper structure - there is no introduction or conclusion of what have been accomplished." etc.?
Should such deficiencies be also reflected in the grade? If so, how much? Would it be acceptable to even fail a work just because it's completely unreadable?

Are you talking about hand writing or organisational presentation?

@JackAidley I'm talking about written works - both hand-written (like homeworks or exams) and typed (like semestral projects).

Yes, please! Do what you can to stop the deluge of people who believe being simply being technically competent is all that is required in the real world. Communication is often more important.

I would be careful of penalising students for poor hand writing because of the number of people with specific disabilities that affect ability to write neatly. Where it's a matter of organising work, of course you should.

As somebody with really poor handwriting... I request if you're going to grade on neatness, only do so if typing is allowed.  My poor handwriting does not affect my performance in the workplace because there is really no such thing as hand written communication; it is all typed or verbal, at least where I work.  Note: That said, even with poor handwriting, organization of an answer has nothing to do with handwriting ability and there are other ways of making your answer clear.

@PetrPudlák Can they give your their reports (so you can red-ink it or give some pointers on the structure) before the final submission?

@MikeS By unreadability I didn't mean poor handwriting, I was  concerned about things like giving no explanation for derivations of their result, no organization of an answer (like different parts of an answer scattered chaotically around a sheet of paper), etc.

@PiotrMigdal Yes, I encourage that.

just return the work to them and tell them you can't read it. offer them an opportunity to clean it up.

Should such deficiencies be also reflected in the grade? — This is already answered by the first question.  If a student can submit deficient work without affecting their grade, that student can correctly conclude that the deficiency doesn't matter.

I don't know if your institution has a flag to mark students as dyslexic, but if it does you should check it. It's one of those dyslexic tendencies that people don't understand and you could make a big difference to someones education by giving the right kind of feedback.

My professor in college had a policy - misformatted code = 50% off.. Perhaps that was a little on the harsh side..

@PetrPudlák in case of chaotic answering during exams (or other time-critical task) - people are sometimes thought to jump from question to question during exam to avoid getting stack on first one (or answering just one question perfectly and not having time for rest) and even if one reserve space to avoid it it is easy to miscalculate (this doesn't apply to homeworks though).

Have you told your students you expect them to turn in work that is readable and coherent?  If so then absolutely.  If not then it seems like it is a good time to express this, give then express this expectation now, and give the student a very limited window to correct his deficiencies.

For the Computer Science side, are you talking theory (writing papers) or practical application (writing programs)?  Or both?  (Really, I'd say the same, but my reasons apply primarily to the practical application side, which is what I have experience with)

@Izkata I'm talking about theory. But you're right, the same applies to code as well. I think I'd be even stricter with badly documented code.

@PetrPudlák You profile says your location is Czech. Is it the case that your students are required to write their works in English which is not their native language?

@scaaahu No, they're writing in their native language.

I think the most important part of your post is in your comment

Or for example grant agencies simply reject proposals that fail even slightly given formal criteria.

If a student isn't corrected for sloppy work while in school, when will he or she improve? The question really boils down to

How best can we as instructors give guidance to students so that they will succeed in the future?

Evaluations in an educational setting can serve many purposes, but fundamentally they should be used as a tool to help students succeed.  Obviously, you have to be careful because a poor grade on a transcript can have long-term ramifications, but ignoring sloppy or unreadable work by trying to see through it is doing a disservice to the student on a particularly key issue: communication of an idea is as important as the idea itself.
To answer your questions:

1. Should the evaluation include evaluation of readability, proper structure etc...?

Absolutely.  To ignore this would be bad pedagogy.  Determining how to relate this to the student without causing a misunderstanding about the idea itself is the difficult part.  Encouragement or direction to seek out writing help is a good idea, as is having students re-write or re-submit work that is sloppy.  Hopefully the work improves throughout the course; being picky at the beginning of a course can set a good standard for the rest of the course.

2. Should such deficiencies be also reflected in the grade? If so, how much? Would it be acceptable to even fail a work just because it's completely unreadable?

The answer to this ties directly back into the first question.  I would suggest letting students re-write or submit material without penalty (or with minimal penalty) early on in a course, with the understanding that they must improve by the end.  If your only assessment is a final project, consider allowing a re-write with an incomplete grade.  But, if that is your policy, I'd strongly suggest providing a time for draft review well before the final project is due in order to minimize the need for this route.  As always, I would also spell out in a syllabus your policy and the fact that presentation is a part of the course, and I would also discuss this on the first day of class (with examples of past work that is sloppy!).
You've posed a hard question, but an extremely important one.  I probably suggest rejecting 30% of the papers I review for conferences simply because they are unreadable.  Most of the time I don't even get to the idea behind the work before I realize that it doesn't matter what the idea is because it is too poorly written (and this ends up in the review).  I want more papers that are well presented, because I'm certain some of the papers I give poor reviews to because of this problem have great ideas behind them!  Teachers that encourage students to improve their writing and presentation are necessary for this to occur.

As a side note, on my undergraduate course the single best lesson from writing technical reports from experiments was not its scientific value, but how to make it clear, readable and conclusive. And they were rarely accepted on the first submission - they were rather returned with a lot of red ink.

I would suggest letting students re-write or submit material without penalty (or with minimal penalty) early on in a course — This is a great idea if you have few students or infrequent assignments.  For a 200-student course with weekly narrative homework, not so much.

While I do agree with Peter Jansson's answer that we should make it completely clear to students what they need to do to have their work understood, I feel differently about the impacting on assessment.
In industry, the ability to write readable (which means maintainable) code is critical. The ability to write a really cool, super efficient block of logic which nobody can understand is simply producing a ticking time bomb which will explode as soon as someone else goes to edit it. Students need to know that you are not just being a 'stuffly old prof' but rather that there are certain expectations from the profession and they must live up to them. Coding is not just about making a solution - it's about making a solution which will not explode 12-18 months later.
On the academic side, I also have students who have similar problems where they do not care much about being careful in their writing. I generally handle them like this:

If I can understand it, but it is difficult, then I will mark them based on the overall quality of their work. If it is difficult to understand then the quality of their work is not great. Even if they are far smarter than me, they should be able to write in such a way that they can be understood.
If I cannot understand it, then I simply fail them. They are responsible for showing me that they understand and if they do not do that, then I must fail them - this is my responsibility.

So, yes, educate them. However, I would not let poor quality slide.
I try to remind my students of the story of the family. Every happy family is happy in the same way. Every unhappy family is unhappy in a different way. The students are usually lost at first, then I explain to them that for the family to be happy they must get 100 things right. But, get anyone of them wrong and the family is unhappy. If they want to be happy then they must get it all right.

Proper presentation of results is a vitally important part of modern research. It's also a necessary skill for any field of work into which an academically trained individual will enter after graduation.
If one does not learn how to exchange information with other people and organizations, the results can be catastrophic. Edward Tufte's Visual Information uses the Challenger explosion as a case study in exactly what can go wrong when communication skills are ignored at a fundamental level. The launch was not aborted because the engineers just provided a bunch of disorganized data to management, who couldn't parse it out to understand that it was too cold to launch.
So, as much as possible, presentation and organization of material should be considered when assigning grades. Now there are situations when it would be unfair to penalize: for example, in the context of a timed exam, where students may feel rushed and may not be writing neatly. (However, I also feel that this is a largely contrived circumstance, which doesn't reflect true understanding in many important ways.) In any situation where they have had time to independently prepare their work and submit it, then presentation and writing style should definitely be taken into account.
However, at the same time, unless the class is a writing class, then it's unfair to fail or strongly penalize a student who has turned in work that would otherwise be satisfactory, but was not put together well. Personally, when I am grading project reports, I "control" for this by making a decision on what the grade for the technical content of the project. I then give a bonus "partial" grade level for a well-organized and well-written report, and similarly deduct a "partial" grade if it's substandard. So, for instance a report that would have gotten a B+ could become either an A- or a B.

However, at the same time, unless the class is a writing class, then it's unfair to fail or strongly penalize a student who has turned in work that would otherwise be satisfactory, but was not put together well. — Strongly disagree.  The point of class work is not just to solve problems, but to effectively communicate those solutions to another human being.  Every class is a writing class.

Every class is a writing class: Many of my students are writing in a foreign language. I will not fail them for poor grammar and typos.

In my experience, the students that have the most trouble with writing are actually writing in their first language.  And the biggest problem isn't "poor grammar and typos", but inconsistency (complete lack of grammar) and total disorganization.  If I can't understand precisely what a student means, I can't give them credit, and (unlike many of my teaching colleagues, perhaps including you) I adamantly refuse to read their mind.

If the writing is so bad as to be incomprehensible, that's not the same thing. I guess I've been fortunate enough not to really have to deal with that—at least not in the classroom. (But if they can't write at all, how did they get admitted to the university in the first place?)

Ideally, no. The main problem when evaluating is expressed by a saying in my native language "muddy writing, muddy thinking". Essentially, if you cannot decipher what is written, either because it is sloppy writing or poorly expressed, then one faces problems when grading. Students often complain about grading where they sense they are misinterpreted etc. This is when we cannot understand what they are writing. Often, I believe, we also give people the benefit of the doubt and grade more leniently than what the verbatim answer would state. This is, of course, never picked up by a student. Hence a catch 22.
What can we do? The problem lies in the examination.
First, we can provide clear guidelines about clarity of language and writing. This should be pointed out long before the exam, in fact early in their education. A study of Swedish students showed that they learn two ways of writing in pre-university levels. In languages, focus is on grammar, spelling and language but not content; whereas in other subjects focus is on content and not language. When they arrive at the university they do not realize that both are important; we need to tell them.
Second, we can chose other means of examination where those with poor hand-writing can do better. Examples can be many: from essays through take homes to verbal and of course highly depending on the course, contents, level etc. Examples of the means of grading is to have external graders (more resources!) or to do as one of my teachers did; ask students to request everyone writes the exam with the pencil, and then after the exam use a pen to add corrections to their own exame as the teacher went through the exam. The exam was then collected and graded by the teacher. I mention this as an example, fully realizing it would not work in every situation.
So the answer for me is no, but our assessment is always tainted by many factors whether we want it or not. To reduce the element of subjectivity is what we would be looking for.

When I set marking rubrics for my students' work, I usually include two main areas: Clarity and Contribution.
Clarity specifically addresses whether the student has communicated their ideas (or answered the questions) in a clear and unambiguous manner.  As such, poor presentation or (really) bad grammar impacts this portion of the mark.
I generally don't detract from the student mark for speeling errurs [sic]; even though I am a native English speaker, I am an Australian working in the US and am frequently in a quandary over whether to use Australian or American spelling.
My present students are generally from non-English speaking backgrounds, and I find their clarity generally very good, even though their turns of phrase sometimes seem awkward to me.  I give feedback on this awkwardness but, unless it impacts the clarity of their presentation, I don't dock marks for it.
Contribution is used to address the correctness or fitness for purpose of the work.
I also add a further C, Collaboration, if the work is a group project.

To directly answer your questions: Yes, and Yes.
As a side-issue:  Because many of my students are non-native English speakers, I also tend to allow double-submission of assignments.  The students may submit their assignments twice before the due date.
I give an undertaking that I will do my best to give early feedback on an early submission (because I am an adjunct, I make it clear that "early" means the submission has to happen before a weekend before the due date).  The students can then take my feedback and implement it on their new submission.  I find this improves the quality of the presentation (clarity), but not so much on the contribution (unless they were really, really out of line on the first submission).
Unless the students do a complete re-write, the second marking is usually just a "delta" / "diff" on the first.

Beauty is in the eye of the beholder.
You as an instructor may see hundreds upon hundreds of papers a term.  The students do not have that same luxury for comparison.  If penmanship and other elements of style are important to you as an instructor, then as a student, I suggest you express your standards and expectations up front, even if it seems that is something I should know and do coming into your class.
I expect no less from an employer as an employee.

Note the degree of impact for presentation on the syllabus, even if there is no plus points for exemplary, and only negative points awarded for substandard.
Provide a sanitized modicum of 'bad', 'passable', and 'exemplary' presentation for comparison.
Encourage them to seek clarification or special consideration up front, as opposed to seeking remedy to penalties after the fact.  They may have certain existing conditions which precludes them from generally meeting those standards.  Or more commonly, they may not have been exposed or acclimated to the technical writing style you are accustomed to in that particular field..

Absolutely -- if you do ANYTHING differently than what is commonly expected, you must say so. (There's a lot of things that you should say regardless of whether it's common or not.) This goes for style and/or penmanship.

I particularly like the answer that suggested having rubrics to identify criteria such as clarity.
When it comes to a situation where work is completely unreadable, I apply the pedagogical pattern known as Grade it again, Sam. Very often, the student who submits unreadable work is simply not making an effort. When I refuse the work using this pattern and give the student a second chance to improve readability, the second work is always better and subsequent works are usually also better.

I totally agree!

As a current student of computer science (master level) I can say that commenting on the readability of work from students, providing pointers and even taking it into account when grading is of great importance. It will benefit the student a lot in the long run.
At my university, many professors point out bad readability and incoherent structure, although mostly on project/paper type assignments and less on weekly homework assignments.
However, it is considered important in all cases. I'm now writing my thesis and have written two research papers as part of research projects (for publication) and the comments I got on structure and readability greatly improved my writing.
Also, I see from fellow students that they write solutions without providing reasoning and without introducing the topic and the problem they are solving. I believe this is very bad style (for group work this has unfortunately led to me rewriting the group documents many times simply to improve the structure). However, students don't often see a problem because it gets accepted by the teacher.
An important note though is that I think grammar and spelling should be of significantly less importance. For instance, the Chinese students in our class often forget the word "the" in writing (all documents are written in English for my master). Although this affects readability and is often pointed out by teachers, it is generally not considered an error significant enough to deduct points.
The line between decreased readability due to grammar/spelling and actual incomprehensibility is hard to determine, and should be determined on a case by case basis. Still it is important that students also learn this.
In short, definitely help students in improving their structure and on communicating their ideas and force students to do so by making it affect the grade! It will only help them like it helped me!
EDIT: As a side note, handwritten solutions are generally not accepted where I study, but if YOU are requesting handwritten solutions, it would be unfair to deduct points for bad handwriting. Some people just don't have a nice handwriting, an unfortunate result of the modern computer era. If students CAN but don't have to hand in handwritten solutions, then it should affect the grade as it is the students choice to write instead of type (if it significantly affects the readability of course).

What is your evaluation measuring?  Are you grading a 2nd grade math test, or are you evaluating a doctoral thesis on neuroscience?
If it's the former, then no; sloppiness and structure take a backseat to getting the subject matter correct.  The reason is your focus should be on evaluating the crucial element of the subject being evaluated.  At early levels of development, there must be some leniency given to presentation in order to focus on the fundamentals.
Of course, there is a reasonable limit to how sloppy or unreadable something can be before the evaluation ceases to be useful.  Intentionally sloppy responses relying on confusion in the hopes of gaining partial credit don't deserve any credit at all.
If you're evaluating a student in a near-professional setting, e.g. university and higher, then it makes sense to consider presentation as part of the grade.  Well structured, carefully written responses should get more credit than sloppy and careless ones.  Presentation becomes important because in the "real world" people evaluate you based on the whole package.  Nobody is going to buy a bottle of Tylenol with a hand-scribbled, barely legible label.  Likewise, even superficial details should be considered important in high level evaluations.
I was a sloppy student through primary school and even most of high school.  Many teachers simply refused to grade some papers that were unacceptably sloppy, and others graded what they could within reason.  None of those teachers who refused to grade a paper stick out in my mind, nor did their insistence on neatness change my behavior.  I excelled in the subjects where my understanding of the core materials mattered most, and did not focus my attention on subjects where presentation was considered more important than the concepts being taught.
Overall, I graduated in the top 5% of my high school and went on to a good college.  By the time I got my masters degree, my value of structure and presentation has improved sufficiently that I can survive in a professional environment.  Becoming a professional, I believe, is the goal of "teaching" organizational skills.

I don't see a reason to ignore sloppiness on a 2nd grade math test. It's much easier for the student to correct those habits early on than later. And it frees teachers from the avoidable drudgery of slogging through slop, to focus on the deeper things and deliver a better education.

I am not a teacher, but I am in my second year of my associates working toward my bachelors. My opinion is that yes you should, it will only hurt them later in life. In high school I passed my classes with C or a C+, now in college I am getting As and Bs. The reason is not that I am magically smarter, it is that I apply myself at least twice as hard.
 There is so much information out there to learn anything, APA or MLA, and sites like Grammar Girl and reference generators. I think spelling should and grammar should be graded hard, maybe not at the first of the semester.
 If they are bad at first, correct them and give them places to go for examples of the correct way(s) to do it. If by mid-term they have not figured it out then grade them hard. All work should flow; if it does not, let them know now because in their career they will not get a nice correction and time to fix it. I work my butt off in school and feel as if I get graded hard, but it is when the teacher gives good feedback that I learn and make the change. So giving them a grade lower than an A needs an explanation of what they need to fix.
 This is why we are seeing on job descriptions -Bachelors from a reputable University- teachers need to be hard so that the student walks away with a real education not just a piece of paper that cost them thousands of dollars.

When you discuss formatting constraints of the grant application process, you're getting close to a solution, I think. There are really two components here; one of these is structural/formatting criteria, and the other is clarity. When you're in a situation where structural requirements and great care are part of the lesson you're teaching, I find it best to refuse to grade work until it has been presented in the proper format, and apply whatever late-work structure you normally use.
This teaches the lesson that formatting is not optional fluff, but is instead a precondition to being taken seriously. That said, if you're going to follow this method, your instructions on format should themselves be crystal clear and unambiguous. Ask a competent peer to follow your formatting instructions and see if they can arrive at the right format.
The second question about messiness is more troublesome, and to some extent your response should depend on where you're situated within the broader track your students will follow, and what constitutes messy/sloppy. I think your response should be two-pronged:

I wouldn't advocate failing students outright for unreadable work, but let them know you can't grade the assignment until you can understand it, and ask them to discuss their work with you one-on-one. Like above, allow your late-work policy to degrade the value of their work if they don't come address it swiftly. I ran into a few students who had a sense of what academic writing should sound like, with the result that they were completely incomprehensible when trying to parrot a voice they had no real command of. But they were able to explain it in person. This method allows you to show how imperative clear communication is--it's a precondition to being graded--and avoid invalidating what may have otherwise been good work. The ultimate penalty is based on how quickly the students resolve the problems with their work.
Include some metric of style/quality/organization on your rubric. While I understand the inclination to see writing as something students should already know how to do--and that you probably don't see yourself as an instructor of writing--I think rubrics should always always always reflect the qualities you actually expect of good work in your field. Your rubric telegraphs your own priorities, and if you don't communicate clarity and style as a concern, it shouldn't come as a surprise when students think it doesn't matter.


text:
How much is submission acceptance to high-profile conferences influenced by internal politics?
I was speaking with several professors in my department about submission to high-profile conferences like CHI in human-computer interaction. Most of the professors were of the opinion that acceptance in most of these high-profile conferences (typical acceptance rate of less than 20%) is heavily influenced by politics: even though you may have comparatively good research, the editors will most likely be more favored toward accepting papers published by Microsoft Research or Google Research or from other high-profile universities where they have close relationship with the faculty and researchers.
The professors advised me against spending time and resources on publishing to these conferences and instead concentrate on second-tier journals (like those published by Springer) for better return on investment for my efforts.
I assume in good faith that most of the editors may not reject papers without reading them, but does anybody concur with the opinion of professors at my department?

CHI uses a relaxed form of double-blind reviewing.  Of course that doesn't make insider bias impossible, but it does at least encourage reviewers to read the actual submissions.

It is an often-heard prejudice in CS that many of the top conferences are relatively closed "boys clubs". Indeed, if one just looks over accepted papers for multiple years, one typically ends up seeing the same affiliations over and over again, strengthening this impression.
However, this could be due to a number of (good or bad) reasons:

Papers from professors / labs that are well-known in the field may simply be of much higher technical quality, making it quite natural that they also get accepted much more often.
Professors / labs that are well-known in the field generally know how to write papers for this specific conference. They know what the TPC values, and how to present their results in a way that is appreciated in the field. As these professors are typically in the TPC themselves, they know what kind of papers usually get accepted and which are rejected.
Professors / labs that are well-known in the field often have a better grasp on the existing state of the art in the field, making it easier for them to identify what is good and novel. In my experience, "outsiders" have a tendency to overestimate the novelty of their contributions significantly. Further, well-known labs know what problems are currently en vogue in the community.
It is of course also perceivable that papers from well-known professors / labs are just not judged as critically. For instance, a reader may very well think that a paper is not applying a given technique correctly, but as the paper comes from the group that invented this technique, he gives them the benefit of doubt and assumes that they will know the technique better than him. He would probably not extend the same favorable thinking to an outsider.
Finally, for some topics, it is just easier for some labs to do good research than others. A common example are the web search tracks at the WWW conference. These tracks typically require the validation of new algorithms on real data, to which mostly only industry labs from Yahoo! etc. and their close collaborators have access.

Note that none of these reasons is really politics. Indeed, I would argue that all of the reasons above are significantly more likely than a paper getting rejected for the reasons you cited.

I assume in good faith that most of the editors may not reject papers without reading them

I would say, at a top conference such as CHI, you can rely on your paper at least getting reviewed thoroughly, yes.

but does anybody concur with opinion of professors at my department?

Well, given all the reasons above, it is indeed quite likely that your paper will be rejected. By definition, if a conference has a <20% acceptance rate, rejection is always a real possibility (clearly, it happens to most submissions). However, I am wondering why it would be better to not even try if you think your work is good enough for CHI. If the paper gets rejected there, you can still re-submit to a lower-tier venue, and you receive a number of hopefully helpful reviews. The only disadvantage I see is that it prolongs the publication process by half a year, but if you see any chance of the paper begin accepted at the top-tier venue, I think this should be worth it.

I agree; the attitude expressed by the professors seems overly defeatist. Whatever happened to the "college try"?

One final aspect that is not mentioned here is the issue of scoping for a conference. Reviewers tend to give extra points for things that they are interested in (which makes sense). However, this causes a feedback loop where conferences tend to get more and more restrictive in scoping.  In an interdisciplinary conference that I'm involved with, it's a constant battle to keep a variety of research areas well-represented, because if the balance of reviewer interests goes too far in one direction, good research on the other sides gets marginalized (e.g., excellent papers relegated to posters).

Good answer.  End of the day, some places are simply resource rich.
They have talented and experienced people, easy access to data and funds, and a supportive environment.  If you don't belong to these places, it makes it harder or your idea must be truly outstanding or outshine those from big league players.

For what it's worth, none of the reasons listed here seem overly "bad" to me. Even #4, which is slightly quesitonable at first sight, is based upon the assumption that "the group that invented [a] technique (...) will know the technique better than [the reviewer]" - and that assumption, for better or worse, is probably simply true.

Having served on the CHI program committee, many other program committees, and having been program chair of a couple of other HCI conferences, I think your professors are overly cynical.
First, as JeffE pointed out, CHI (and many other conferences) use blind or semi-blind reviewing, so at least the reviewers do not know whose paper they are reviewing.
Second, I have never been in a PC meeting where the identities of the authors was a point of discussion. Of course that doesn't mean that the associate chair (AC) for the paper was not influenced by who the authors were, but it would certainly imply that there is no institutionalized bias for or against particular authors or institutions.
I would definitely not worry about your papers being rejected without being reviewed; in fact, when you submit you will get the reviews so you know why your paper was or was not accepted.

While I think this is all true, to play devil's advocate, I would say that anyone who knows a field well can often infer which group (if not which authors) wrote a given paper. There are just certain research topics or strategic advantages (e.g., proprietary data sets) that groups are known for. I agree that I haven't seen this impact decisions though: nobody cares who the group or researcher is.

I completely agree with almost everything xLeitix wrote, except the conclusion.  One thing that was not addressed is why your professors told you not to submit to CHI.  It's possible that they're giving their honest opinions, or that they themselves had a bad record submitting there.
But it's also possible that they are telling you that your work is not good enough for CHI, but in a gentle manner.  Also, they are familiar with where you are in your career track and may believe that losing time and effort on a failed CHI attempt is not good for you.  Both of these require perspective that you cannot have on yourself.
Consequently, I would suggest that you take the advice you were given.

But if your professors think your work isn't good enough for CHI, they should say so directly.  Being "gentle" is not actually doing you any kindness.  You deserve an unvarnished appraisal of your work.  Demand it.

This kind of advice is often given to people, but in my view it's fundamentally misguided.
It's true to some extent that many of the top conferences have a particular way of doing things, and that people in the community associated with a conference will often have more experience with that way of doing things and find it easier to get papers accepted there. However, it's untrue that the communities associated with particular conferences are a closed shop: it's entirely possible to adapt your way of doing things to what you know different conferences expect, and that will improve your acceptance rate at those conferences. Moreover, it's absolutely worth your while to do this, because publishing at the top conferences massively improves your chances of having a successful academic career (not to mention that it puts you in contact with a large number of good researchers in your field).
In that sense, the people who are suggesting that you eschew the top conferences and simply send your work to second-tier venues are doing you a serious disservice, because they're basically ensuring that you're not even on the pitch where you could compete with your peers (indeed, the heavy focus that top groups put on publishing at top conferences tends to mean that they are underwhelmed by people who don't compete). If you're good and you put in sufficient effort, you can definitely play at the right level, whereas counting yourself out of the game before you've even tried to succeed is deeply unwise in my view.
TL;DR The people giving you this advice are leading you astray in my view. It's not wise to spend your time and energy submitting papers to venues that many researchers will view as being less than top-flight, when you could be refining your papers to make sure that they get into the top venues.
More generally, people unfortunately tend to give similar advice about all kinds of things, not just academic conferences. A classic example would be applying to university, or applying for top jobs. The general form of such advice tends to be:

This venue is biased against you specifically, or against people like you (whatever that means) in general.
This venue wouldn't suit you if you were accepted -- you'd enjoy it way less than a lower-tier venue.
This venue isn't all it's cracked up to be anyway -- they just have a high opinion of themselves.

There are certainly situations in which people might be a bit biased, or in which you wouldn't enjoy some place, or in which the place isn't as good as it thinks it is, but these things are by no means universally true. If people tell you "don't apply to X because I have specific, verifiable evidence of Y", then that may be one thing (check the "verifiable evidence"), but if they tell you not to apply to top venues in general, then you need to seriously question that.

Another good answer. My only gripe here is that there is a whole world (so to speak) of conferences between the high-profile conferences with acceptance rates around 20% mentioned by the OP and "venues that many researchers will immediately discount". Not going for the flagships does not automatically mean turning to the "conference-equivalent of predatory journals".

@O.R.Mapper: Yes, true, I wasn't trying to imply that it meant that (I've edited it accordingly). Nevertheless, many jobs do mention publications at specific top conferences explicitly, and publishing at second-tier (though non-predatory) conferences will definitely help you less for those kind of jobs.

I agree with xLeitix's answer and would like to elaborate on how some high-profile researchers manage to constantly appear at certain high-profile conferences.
I'd say the method to reliably appear on a given conference are resources and focus:
First, to professor X, conference A is not simply another conference. It is their subfield's "flagship" conference, their "home" meeting. They will and can put a lot of effort into making sure they get published there. Thus, they do not just submit one paper. They make sure they have at least a short paper about each of their hottest ideas to submit. With ten or more submissions, a few acceptances are likely even at a low acceptance rate.
Related to that, note that the conference topic is probably not just X's personal hobby. More likely, they have an entire research group specialized on the conference's core topic at their disposal. X is probably already involved with half a dozen projects with different members of their group at any time, all of which are thematically suitable for conference A.
Lastly, there is simply a strong overlap between X's topics of expertise and A's topics. Even if X does, somehow, not manage to publish in A, they will actively participate nonetheless (thereby adding to the impression that their name appears all the time with respect to A):

They may be among A's chairs.
They might chair one or more sessions at A.
They might be invited to give a keynote talk at A.
They might offer a workshop co-located with A. Note that even without anything questionable going on, this could lead to a certain additional bias towards contributions by X's group: While less frequently so for workshops at very large conferences, workshops often rely extensively on word-of-mouth marketing. As a consequence, people in X's direct vicinity are most likely to know about the workshop and its precise focus.
As a well-known member of the audience whose work is being built upon by other submissions to A, they might be mentioned in some of the talks. This can happen in totally informal remarks: "Last year, Alice, who is sitting over there, presented technique Y and asked Z, so I have been looking into Z."

And even if all of these fail, X will still attend A and attract enough attention during coffee breaks to reinforce the memory: X was actively part of A again.

Politic is everywhere, of course, especially in situations where the resource giving out to people is limited. Bear in mind, taking a look at the program committee before submitting! And please don't stick to one conference! Even some well-known professors get rejection at some conferences, let alone others.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
How to ask PhD advisor to work remotely on thesis for extensive periods?
I am in a PhD program in a top US institution but my SO,family,friends are somewhere else. I am considering a thesis that would require minimal/no field or lab work. This would be a big compromise as I was hoping to do intensive field work but there is no funding for field work. I thought that a compromise would be that I can work on it remotely. I haven't mentioned yet to my advisor that I am home sick as we don't really connect on a personal level, but we have a good connection on a research level.
How do I approach my advisor saying that I would like to work remotely for 6 months out of the year? I have worked remotely before and it has always worked but I don't know how this type of setup is seen at a top tier US institution.
How do I approach my advisor about this issue?

Both answers are equally satisfying and helpful, I wish I could pick both.

I would break this down into two parts. First, discuss your circumstances with your advisor and explain that you want to spend more time "at home", and ask how he feels about you working remotely. Go into the discussion with a concrete proposal for how the practicalities might work: how will you do your work? Do you have access to the resources you'll need? How will you stay in touch with your advisor/colleagues/the research community? Take on board any reservations your advisor might have; don't go into the discussion with too fixed an idea of what the "right" outcome is. Agree to review the arrangements once they've been tested for a while.
Once you have agreement in principle, then you can hash out the dates. I would try and start with relatively short stretches of remote working - a couple of weeks - to allow any teething problems/concerns (on either side) to be resolved. I suspect everyone will be happier if you can spend your six months away in short chunks, rather than in one big block. Also, recognise that your advisor may have reasons to want you present for certain occasions/portions of the year - involve him in your planning.

I think you should aim for a shorter period at first, to show that it doesn't affect your productivity.  My wife and I aim to both work remotely 25% of our time, to address the two-body problem.  If each of us works remotely for two weeks every two months, that aim is fulfilled.  A colleague of mine just returned from a 10-week remote work trip.
Six months, however, is very long to work remotely in one go.  It is of course more expensive to work remotely in shorter segments, but if your advisor doesn't know yet whether it will work out, six months is too long.  Explain that you are ultimately interested in working remotely up to half of the time.  Propose to work remotely for 3-4 weeks first.  Be available at any normal local time for your supervisor, even if that might mean inconvenient times on your end.  After the remote work trip, return and evaluate how it went.  Then propose to plan a longer remote-work trip, perhaps 10 weeks, and return again.  If all goes well after a couple of 10-week blocks, you could bring up the idea for an extended six-month period.
The worst that can happen is that they say no, and you may have to find a different way.  Perhaps they don't agree with 6 months in one block, but are fine with 3 2-month blocks spread over a year.  It will cost more money, but it is safer in many ways too.
Good luck.

thanks, that sounds more reasonable, I think bringing up the 6 months right away it's too much. I think I should start by saying that I would like to extend my stay during the breaks.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Generating a connection of researchers graph out of DBLP
I'd like to know the connection of researchers. Similar to the connection of LinkedIn. In the field of computer science, the DBLP database seems to be useful. Does anyone know a tool to generate a FoaF graph out of DBLP data and a FoaF (Friend of a Friend) graph rendering tool?

There are many possibilities to generate a graph. E.g. parse it with Python and then use http://gephi.org to visualize it or e.g. in http://flowingdata.com/2012/08/02/how-to-make-an-interactive-network-visualization/. However, most likely you won't be able to visualize all at once, but rather - e.g. a graph for a given person.

Currently, I'm looking for a ready-to-run-solution instead of coding it for myself :)

If there is a need, I can write one :). See https://github.com/stared/tag-graph-map-of-stackexchange/wiki and I want to play a bit with D3.js. But... not this week.

Microsoft Academic Search can do it too. Search for the first author, klick on it's name (orange background), click "Geanology Graph", Click "Co-author Graph", enter the name of the other person on the right and press enter.

Microsoft Academic Research can do something similar for you, you just need to have Sliverlight plugin installed in your browser. Search for the first author, click on it's name (orange background), click "Geanology Graph", Click "Co-author Graph", enter the name of the other person on the right and press enter.

Did the graph visualization move someplace else?

A bachelor thesis of interest for you:
Remo Lemma, Ebony - Visualizing the DBLP Database. University of Lugano, 2010
url : http://www.inf.usi.ch/faculty/lanza/Downloads/Lemm2010a.pdf

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How should I go about making CV as an undergraduate?
I'm just an undergraduate math student, and I need to make a CV for a research application. However, from the CVs that I have already seen I have nowhere near enough to fill probably half of a page. I mean I can really only put my education history which is 3 schools. What else should I put?

Highly related: Why is an academic CV not the place to sell?

Before starting research, An undergraduate student really only has a résumé. A curriculum vitae is generally intended for someone who is intending to pursue a career in research, and has a somewhat "standardized" format (although there are still many variations).
But as a young student just beginning to do research, I would recommend that you focus on listing skills that you have that would make you appealing to someone interested in hiring you. If you have more than the standard training for someone at your level in university (more courses, have already done research projects, etc.), then you should list that information. Similarly, if you have special skills (programming languages, familiarity with specific techniques or tools, etc.), then those should also be listed.
Any jobs that you've held that might also prepare you for such an experience would also be of interest.

In Europe, 'CV' doesn't have this connotation and is used in the same sense that Americans use résumé.

CVs and resumes vary greatly by field and level of experience. A CV of an undergraduate applying to graduate school is very different than the CV of a associate professor applying to become a full professor. I would suggest talking to your classmates and the career services at your university.


text:
Good, neutral source for ISBN linking? (on citing books)
In my bibliography, I want to link the ISBN number to an actual website. My question is: which website is recommended to be used? Is there some 'neutral' source that I can expect to be still functioning in 5 years and that will return reliable results?
IBSN numbers are apparently provided by the International ISBN Agency, so this would clearly be the best source. Turns out, however, that they do not offer an ISBN lookup! I am currently considering two options:
Google Books: http://www.google.comco.uk/search?tbo=p&tbm=bks&q=isbn:0060930314&num=10
ISBNSearch.org: http://www.isbnsearch.org/isbn/0060930314
Both appear to have their commercial interest, but they appear to be reliable enough. I would prefer Google. Or is there any other more recommended website?

You could use worldcat.org, it is built and maintained collectively by the participating libraries around the world (see Wikipedia article).
They offer ISBN search, e.g. www.worldcat.org/search?q=isbn:0060930314

It's worth mentioning that they offer permalinks, e.g. http://www.worldcat.org/oclc/1419841.

Google Books is a great website and there’s nothing wrong with using it for your purpose. But if you don’t want to be dependent on Google you could use the following sites:
Wikipedia: http://en.wikipedia.org/w/index.php?title=Special%3ABookSources&isbn=0060930314
The downside for Wikipedia is that it doesn’t show the book title and one has to search another time for the book. But it’s ad free and certainly still online in five years.
But I’d suggest OpenLibrary:  http://openlibrary.org/search?isbn=0060930314
Non-profit, ad free, part of the Internet Archive. Also high possibility of availability in five years.

Thanks. I see that OpenLibrary lists two results though... that doesn't look right, does it?

http://www.bookfinder.com/ has been around for a long time and they claim they never charge a markup from search results, so although not neutral as such maybe useful?
There is also http://www.openisbn.com/ which says

OPENISBN is a personal project dedicated to provide its users ISBN search services and ISBN data, introduction, book reviews, book preview, free ebooks and book price comparison for your specific book.

Possibly something that will not be around for decades, but it sounds at least very neutral.


text:
"Looking for a postdoc" ad inside a conference talk?
I got my PhD one year ago; I am looking for a position (e.g., postdoc).
I am going to a fairly attended conference, and was pondering about how to make audience aware that I am on the market.

An explicit, dedicated slide (maybe at the end)?
A verbal statement?
A caption on my T-shirt? :)

Would you find any of this appropriate?
What would you advice me to do?
EDIT
In case you think it's not a good idea, I'd be interested in knowing why.
Otherwise, I'd be interested in any additional implementation advice (how to formulate that, or whatever comes to mind).

We can't hire you if we don't know who you are.

@JeffE Do you mean that apart from the I am available, there should be a slide containing biographical or personal information of some sorts?

I've even heard advisors mentioning that their students are on the market ("... this other case was solved by my student Mr. X, who, by the way, is on the market right now ..."), so I wouldn't worry too much about it "not being a good idea".

Don't be shy about advertising. As you and Dave suggest, place it at the end, but you don't need to be very subtle about it. I've seen a number of talks where people advertise the fact that they're on the market, and everyone understands the need to do this. Presumably you're worried about it seeming desperate, but it isn't.

Au contraire, a little self-depricating humor could get a laugh, break the ice, and get a conversation going.

I would add to the end of your conclusion a simple statement:

By the way, I'm looking for a postdoc position.

I've seen it done before, and I don't think people minded.
A T-shirt could be a fun way of broaching the topic when talking to people.

Be personal (if possible). Try to research the person or people that may be interested in you and approach them directly.


text:
When should a student attempt unsupervised research?
I just finished my masters and am currently working as a research assistant before starting a PhD in applied mathematics. I was doing some preliminary work for a possible project and derived some minor results which I find interesting, but in the end, we decided not to go ahead with the project.
I still find the minor results interesting and thought of digging a little bit deeper by myself. However, since the group is no longer interested in this, I would have to do this by myself, unsupervised. Is this a very bad thing? Or a good learning experience?
I had a bad experience doing unsupervised research (wasted a lot of time for no results). However, I know that as a PhD student, I must eventually publish first-authored papers. How do I know when to make the switch?

You can work with your advisor and get a first-authored paper.

It is a common thread - see "Go rogue too soon/too late" in 10 easy ways to fail a Ph.D. and Going Rogue at The Chronicle of Higher Education. An excerpt from the first:

The advisor-advisee dynamic needs to shift over the course of a degree.
Early on, the advisor should be hands on, doling out specific topics and helping to craft early papers.
Toward the end, the student should know more than the advisor about her topic. Once the inversion happens, she needs to "go rogue" and start choosing the topics to investigate and initiating the paper write-ups. She needs to do so even if her advisor is insisting she do something else.
The trick is getting the timing right.
Going rogue before the student knows how to choose good topics and write well will end in wasted paper submissions and a grumpy advisor.
On the other hand, continuing to act only when ordered to act past a certain point will strain an advisor that expects to start seeing a "return" on an investment of time and hard-won grant money.
Advisors expect near-terminal Ph.D. students to be proto-professors with intimate knowledge of the challenges in their field. They should be capable of selecting and attacking research problems of appropriate size and scope.

Personally, I have a lot of experience with unsupervised research (yes, usually wasted a lot of time for no results), starting from high school.
Sure, doing advisor's project is much safer (as, in some sense, (s)he is responsible for the big picture, references, judging which result is good enough) and allows to forget about doubts (for good or bad) and concentrate on work.
However, what I've discovered after 1.5 of my PhD (mostly "going rogue") is that there are two important things:

asking experts (or peers), and
day-to-day collaboration.

Sure, you can have an expert and a collaborator in the same person of your advisor, but as long as you are not afraid of talking to other people and have some knowledge in the topic, it shouldn't be hard to ask experts appropriate questions.
When it comes to collaboration - it may be trickier (but not if you locally have people with an interest overlap).
On contrary, working alone for months on a theoretical problem, with little feedback, may be very hard (both technically and psychologically). I've learnt it the hard way. It is not a problem for one research line, but it is when all research is done alone (may be inefficient and depressive, perhaps unless you are in a deep love with it).

My former Ph.D. advisor's philosophy is that a student should become a teacher of their supervisor. He seems to give the green light when the student

explains their own research problem clearly,
shows a clear sign of familiarity with all relevant work,
puts their work in context,
gives a convincing argument about why it's important and interesting,
looks they can make it, and, somewhat more importantly,
chose a topic that isn't too unfashionable to the extent that devoting their time during their Ph.D. student phase to that kind of topic is a career suicide.

I wrote my first paper when I was a second year grad student, and my first joint paper with my  former Ph.D. supervisor is after I got my degree. My friend who got his Ph.D. from the same advisor at the same time as me was also like this, and he's doing pretty well in academia. So probably all that matters when it comes to starting your own math research is if you're ready for it. And if you're ready, I don't think your advisor can talk you out of it anyway. At least, randome guys on the internet like us surely can't.
So, I'd say if you're being honest to yourself and still confident that you know about your research topic better than your advisor, and if you can get him interested in your research, it should be fine.

It is not necessarily a bad thing to forge ahead on your own and it will certainly be a learning experience. It is definitely a good thing to try this, because it will demonstrate and help reinforce that you are an independent researcher, something which you need to be at the end of your PhD studies.
But this is not without some risk.
Part of the role of the supervision that would otherwise be provided is to improve your skills as a researcher and writer of scientific texts. Without this guidance, your skills at the end of your PhD may not be as good as they could be.  You may have gaps in your knowledge that would need to be filled. The presentation of scientific material in your writing may end up being suboptimal, because you won't have someone to proposed better ways of writing. And worse still, your ideas may not end up being the best they could be, because you won't have someone questioning them and your assumptions, ultimately forcing you to produce better versions of your ideas.
I've seen it before. Some highly independent students I've seen in action fail to follow recognised mathematical conventions, produce new notations for existing concepts, miss key references and lack understanding of key concepts that would help improve their work, and so on.
Independence is a good trait, but try to take advantage of opportunities to receive guidance from your supervisor(s) while you can.

The risk Dave describes comes more from not working with your advisor than from working on something without your advisor.  See the difference?  You can (perhaps even should) work independently on X while working with your advisor on Y.

I don't think your description of highly independent students is extremely accurate. What you wrote seems to me describes myopic students who don't learn others' work by reading papers and attending conferences/seminars. Being independent doesn't mean you don't listen to others or ask for advice. In my obnoxiously arrogant opinion, it simply means you decide. You should take into account every piece of information you can get, including what your advisor says.

@YuichiroFujiwara: You are right. It is a description of some highly independent students, not all of them. I reworded a little.

Basically, the only restrictions are issues related to funding. If your funding is intended for you to work on a specific topic, then your primary efforts should be given to the designated topic. However, you can, if so motivated, pursue the research topic independently. In mathematics, this might not be such a huge issue, as the infrastructural demands are not so large (except for some applied math projects). However, in other disciplines, where this "unsupervised research" may require experimental supplies and other resources, this is a much bigger issue. In the latter case, you must obtain approval from the PI whose equipment, material, and other resources you will be using!
If your time is "unrestricted," in the sense that it is not tied to any particular project or program, then you should talk with your supervisor to see if you can use some of your "normal" time to devote to the "side" project. If you can, great; if you can't, then again, this needs to be "on the side"—beyond your main project.
As for the question of doing "first-authored" research, well, that transition process, in many respects, is the goal of the PhD! If you are to be a successful researcher, then during your doctoral studies you will learn how to do research independently, and eventually unsupervised. (Moreover, you will hopefully also have the chance to learn how to supervise others!)

Think of research as a learning paradigm, whenever you learn something new spend time asking yourself how you would develop it if what you just learned was the state of the art, and what interesting directions would be. As you learn more about the topic you can see whether you were right. This makes learning to do research a constant part of your life rather than something you are learning to do when you start your thesis or dissertation.


text:
How do you cite a Github repository?
I am working on a honours thesis and have developed a Fortran library that I would like to cite in it.
In some places they suggest to quote the documentation of the project but this is something I plan to do in the future and isn't done yet. The only documentation I have right now is the README file and the code itself.
What would be an acceptable way of directing a reader to my work on Github?
(bonus) Is there a BibTeX way to do it?

I think energynumbers answer is correct in terms of what is "correct" in academic literature. However, if the immediate issue is "I need to let a reader of the thesis/dissertation that I'm working on right now see the software", then (low-tech as it seems) you could consider including the source code in an appendix. How practical this is may depend on its length... Or ask your supervisor whether they're happy for you to provide a github link and what the format should be. Also, welcome to Stackexchange :-)

@SimonWaldman: 7K lines of code spread over multiple modules is definitively not something I can include in an appendix. My colleague did include his MATLAB code in his thesis but I found it to be a terrible way to share code as you cannot reuse it easily (copying it in matlab would ignore the tabbing, etc.).

@charlespwd ha, fair enough. And I agree that it's a terrible way to share code. But if this is for an examinable thesis rather than a published paper, I think the correct answer is "whatever your supervisor is happy with" ;-)

See also this discussion: http://lists.software-carpentry.org/pipermail/discuss_lists.software-carpentry.org/2013-October/001048.html

^ There are so many great points I think they are worth compiling into an answer, much better than all current ones!

Take a look at this: [https://github.com/blog/1840-improving-github-for-science]. This might answer at least a part of your question.

What should the year be? The most recent commit? The first commit? The repo's creation date on Github...also if it was moved there from another site?

GitHub has now added built-in citation support (https://twitter.com/natfriedman/status/1420122675813441540). Simply add a CITATION.cff file (
https://citation-file-format.github.io/) to your repo and a citation widget will be added to the sidebar:

Here's a handy cff file generator: https://citation-file-format.github.io/cff-initializer-javascript/#/

I would go with:

P.W.D. Charles, Project Title, (2013), GitHub repository, https://github.com/charlespwd/project-title

Or a bit different, depending on your citation style. Just is is crucial to include: author, title (if style includes it), year and link.
Of course if you do have a publication related to this piece of code (even if only by fact it is your first publication basing on this code), don't forget to cite is as well! (In the current academic system it is all that counts.)
And respective BibTeX entry:
@misc{Charles2013,
  author = {Charles, P.W.D.},
  title = {Project Title},
  year = {2013},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/charlespwd/project-title}},
  commit = {4f57d6a0e4c030202a07a60bc1bb1ed1544bf679}
}

Beware, this above are improvisations (especially the BibTeX entry), not standards.
Even for more established and citable things there are not set standards, see e.g.:

How to cite a sequence from The On-Line Encyclopedia of Integer Sequences (OEIS)? - MathOverflow

See also:

How to cite sourcecode from an svn repository - TeX.SE
How to cite software in LaTeX - SuperUser.SE

EDIT:
See also how to assign DOI to a particular commit in your repository.

Maybe we can agree on something like this for a standard. I don't see why traditional journals are better than GitHub repositories to publish papers. In this case I'd upload the LaTeX sources to the repository and provide a link to the PDF. I'm seriously thinking about doing this, there is even some review process, and reviewers can open issues in the repository :-m This would be free for authors and provide open access, promote collaboration, etc.

If the repo owner doesn't make their full name known, would you suggest using their GitHub user name instead?

No. I would search his real name for that username elsewhere.

Some github users do not use their real name and make it very difficult to find it.

Why not use url={https://github.com/charlespwd/project-title}

What's the year in the case that I'm not referring to a specific commit but to the repo in general?

@Trylks - realize your comment is from 2013, but the Journal of Open Source Software (JOSS) does all of their review on github. See https://github.com/openjournals/joss-reviews/issues/4368 for instance.

Now they become somehow citable by providing DOI references. That was on their 14th of May news:
https://github.com/blog/1840-improving-github-for-science

GitHub now offers citation as a service, at least with Zenodo. This guide instructs how to connect your accounts and get a DOI with your work:
https://guides.github.com/activities/citable-code/

I ran into this as well but it does not help to cite someone else's repository.

The underlying question seems to be how to get citable software.
There is a traditional method for doing this.
You write a methodology paper describing the software, how it was built, what it does. You can provide the source code as supplementary information.
And then you cite that paper, in future papers where you apply the software.

I think that as a scientific society, it is crucial to develop a way of citing other things than journal publication (e.g. code, data). Otherwise science will never move into the 21st century.

@PiotrMigdal I agree and I hope there will be better ways to cite non-journal knowledge. However this is what there is now.

@Trylks I know. So mine is a comment, not a downvote or anything. Anyway, the question is what to do when there is no accompanying journal entry (BTW: as of now I have the same problem).

I hope though that you don't imply that we get rid of peer review in the 21st century? The good thing with this answer is that if a linked code repository and data are connected with a paper, then it was part of the review process. While when I'm citing just somebody's stuff somewhere on the web, it could be anything really.Not saying peer review is perfect,but that we really need to integrate it with the more modern interconnected work styles.

What I've seen most often is a footnote with the URL you want to provide, in this case the URL of your GitHub repository. This will point people even better because it will be in the same page, and sure it will not count for bibliographic statistics and your h-index, but it wasn't going to count anyway, at least not yet.

Figshare provides a way to import a GitHub release and assigns a doi to it. I haven't tried this yet but it seems to be a good middle path between a software paper and citing just the repository.


text:
When did it become commonplace for data set providers to ask users to cite their paper(s)?
Nowadays data set providers typically ask data set users to cite their paper(s) introducing the data set.  When did this practice become commonplace?

(Given what I read on this page, I feel the need to emphasize that I do not intend to open a debate around the usefulness of such citations.)

Since when has it become common to write papers based on someone else's data set?

@keshlam Since people share data sets?

In that case, my answer would be the same. If someone has made a major contribution to your paper, you owe them recognition... and you owe your readers the ability to trace the data back, in case there was something unusual about it. Credit or citation would seem the obvious ways to do that. I'm not sure why you believe this wouldn't have been done in the past; i'd think it was simply "best practice".

@keshlam I am curious.

Your question is quite unclear; can you please ask more explicitly what you are looking for? Are you asking since when datasets should be cited? Are you asking since when dataset authors asked explicitly for being cited? By "citing datasets", do you specifically ask about referring to papers that describe the datasets, or about providing any kind of attribution for datasets? If referring to papers, are you asking about dataset authors pointing out a particular paper that, in their opinion, is most preferrable to represent the dataset, or about citing any of the papers that mention the dataset?

@O.R.Mapper I am asking since when data set providers ask users to cite their paper(s) introducing the data set, i.e. one or several particular papers that, in the data set provider's opinion, is most preferrable to represent the dataset.

One reason people seem to react differently to this question than you expected is that any sentence beginning with "Since when..." is usually interpreted in a somewhat confrontational way (compared to the neutral phrasing JeffE edited into the question).

@TobiasKildetoft Good point, thanks, question edited, feel free to edit further.

Much better now. It is now clear that this is a question about timing rather of best practices.

Question to the OP: Is the emphasis in your question on the "issue" that the authors of a data set ask a certain paper to be cited instead of the dataset itself? Or is rather that you are interested in from when onwards they ask for to get any type of citation at all?

@DCTLib  I am interested in from when onwards they ask for to get any type of citation at all.

I don't see a logical distinction in citing used algorithms vis-a-vis data sets. Both seem perfectly natural to get cited. Both are an integral part in the results you are reporting in a paper and hence should be attributed.
You're probably thinking in terms of machine learning papers, where typically a new method has to be benchmarked on a large variety of data sets. While it may seem like overkill to cite each and every data set that was used, lets not forget that whoever provided that data also put in effort and deserves credit for it. Again, this isn't that much different from citing the competing algorithms you are comparing against.
(Academic) software packages should also get cited, or at least mentioned, when they are used. This isn't really a new thing, for example the old 4-clause BSD license essentially demanded the same thing.

"I don't see a logical distinction in citing used algorithms vis-a-vis data sets. Both seem perfectly natural to get cited." - and yet, software is often not cited. This paper, for instance, contains frequent mentions of Microsoft Word, and yet the bibliography totally lacks a reference to something like "Microsoft Corp. Microsoft Word. ." This is analogous to just mentioning the name of a dataset, rather than actually citing it as a bibliographic reference.

@O.R.Mapper I know that happens often, but both are bad practice in my opinion.

I fully agree. It is just that this answer (and also some of the comments on the question) make it sound like we had to explain to the OP why datasets (and software, and similar) should be cited (which may or may not be necessary to explain), while the explicitly asked question, when dataset providers started to exlicitly suggest particular papers on the dataset for citation, is not really answered.

@O.R.Mapper as far as I know there is no when, it was always the right thing to do. I think that's why the comments and my answer are what they are.

Again, the question as it is written is not when it became the right thing to do, but when it became common practice to explicitly write something like "If you use this dataset, please cite paper XYZ." on the website from where the dataset can be obtained, rather than relying on the authors to do the right thing. Did the very first authors publishing datasets already do so? Or were they rather completely happy with providing this novel service to the community by providing a dataset, whereas the idea that you could actively ask for a citation thereof evolved only at a later time?

Since Fisher's Iris data set had an accompanying paper I guess it has always been quite common to get cited for data. I reckon Iris is one of the first heavily reused data sets. Even if it wasn't explicitly asked in the past, source attribution has always been important in academic writing.

As we seem to interpret the question rather differently, I have asked the OP for clarification in a comment.

Yes I am curious to know around when data set providers started asking explicitly to get cited.

Shortly:y after someone was rude enough to use data they should have credited without doing so, I presume. If you want to know when it became common in a particular field, I invite you to do a study of the literature and report back to us... or figure out what field might have already conducted such a study and do a literature search therein.


text:
What to do with students coming to office hours asking to check their homework?
As a teaching assistant (TA), how to handle the situation where a student comes to office hours asking to check their homework? I.e. the student hands in their solution to the homework and ask the TA to check for any mistake. Assume homeworks are worth somewhere between 20% and 50% of the grade.
If country-specific, I am interested in the United States. If field specific, I am interested in computer science, linguistics, neuroscience and maths.

Related: http://academia.stackexchange.com/q/16422/9934

Excuse me, but how exactly is "a student seeking out available resources to improve the quality of their work and understanding of the subject matter" EVER a bad thing? What on earth could a TA be there for, if not for exactly such a task as checking/reviewing homework assignments with students? The least significant part of a TA's job is proctoring and grading. The most significant part is providing peer-review and guidance.

@dwoz I've never said it's a bad thing.

@dwoz a lot of technically advanced people are put in TA roles with the expectation that they're just going to immediately know all the "people stuff" that goes with it. I'm not (at all!) suggesting Franck is socially inept, just that any time you're managing people you really need training, even if you're already good with people. And my experience has been there is literally 0 budget for the kind of training they need, despite the fact they're managing the next generation of engineers and researchers. Since he'll never get that training, he's asking for help with the things he needs it for.

@corsiKa Actually I did have some great training from my department, just always glad to get more feedback :-)

Wow, that's fortunate, and refreshing. Consider yourself lucky!

@corsiKa, I understand the premise here.  I am in an academic university situation.  I'll try to add a more "nuanced" answer below.

@FranckDernoncourt your question reads as "those pesky students, always interrupting my work with their requests to look at their homework and questions about it, why can't they just go away".

@jwenting this certainly isn't my intent. Which sentence make the question look non-neutral?

you help them because you're a TA. i guess i don't understand the question ?

@sgroves Might seem unfair to other students, amongst other things.

@FranckDernoncourt it might seem that way, but it's not unfair. Other students can ask you too. That's what you're there for ...

I don't know what "comes to office hours" means?

@LightnessRacesinOrbit Some schools require professors and TAs to maintain (and post) a set of regular times when they will be in their office and available to respond to student questions (as opposed to working on research, committee work, or preparing for class). Those times are called "Office Hours", and students are invited to come during those times if they have questions.

@dmckee: I see. I would call it "come to the office, during 'office hours'".

@LightnessRacesinOrbit It might be an American English thing?  It's similar to how we say we're "going to the movie" instead of "going to the movie theater to see the movie".  We encourage students to "come to office hours".

@CoreyHarris: Could be. Like how you also [frustratingly] say silly nonsense things like "I'm waiting on a train" when in fact you're waiting for a train, on a train station platform ;)

Answering from the point of view of a physical scientist, and in keeping with my personal take on what kinds of help are appropriate or inappropriate.
(I haven't been a TA for a long time, but I teach at an all-undergrad department so I have to be my own TA.)

These are great teachable moments if you have the time (and it can take a lot of time).
You don't check their homework, you ask them to explain their solutions to you.
But there is a catch: you don't let them get away with "I used this formula, and I solved for [variable]"; instead, make them explain their logic and the conceptual basis of their work step by step. When they are stuck or are proceeding incorrectly, you probe their understanding of the problem in a Socratic style.
The kind of questions you might ask include:

Conceptual basis

What physical principles apply to this problem?
Are their any hidden assumptions here? If so, what are they?
What statements in the problem or facts about the world brought you to select that particular approach? And why did you use that expression of the principle in mathematical form?
How well do the explicit assumptions in the problem match the real world (i.e. is this a PhysicsLand (tm) cartoon or a somewhat realistic treatment)?
Are these results reasonable for the real world? If so, why? If not how do you expect the world to differ and what neglected effects would make the answer more realistic?

Math basis (where they'll want to spend all their time at first)

Why is is that the right value for that variable?
Why is that substitution allowed in this case?
What does it mean that there are two solution to that equation? Which one do you use and why? Is the other one meaningful as well?
Does this situation actually meet the preconditions used in deriving the result you want to use?
Can this problem be stated as a special case of one we're already done? Why or why not?

Problem solving basis

What is the goal here? (You'd be surprised how often they lose sight of that.)
What can you learn from what we've been given? (If you don't know how to proceed just try shotgunning it; forward version.)
What list of thing would get you in position to find the answer? (If you don't know how to proceed just try shotgunning it; backward version.)
Do we already know the correct solution to a simpler version of the problem? Does that hint at a way to proceed here?

Most students will find those questions very difficult at first, but as they become more adept at handling the questions they should see their homework and exam scores improve markedly.
If they have the patience they will solve the problem for themselves right there in your office.
Though this is very time consuming most students will not be regulars. Some will simply become frustrated at what they see as your unwillingness to "help" and look elsewhere; and others will get better at the discipline: as the term progresses they'll bring you fewer problems and ask more perceptive questions about them.

In addition to wanting to talk about formulas first, they are going to want to talk about values ("and then I plugged in the 18 from the problem..."). Don't let them do that either. Make they say what quantity it was ("and then inserted the given initial velocity ...").
Beginners are all about numbers and formulas, but learning the discipline is about principles first, problem solving process second, and particular results last.

Philosophical note: As I see it the purpose of homework is to facilitate learning. I wouldn't even grade it except that there is no other way to insure that they will do it. That's why it doesn't bother me that they are getting a lot of support doing the homework for my class: when they come to me I get to make it a learning experience for them.
(The first time they come in they may think they are going to get one for free, but they're in for some skull sweat.)
Admittedly, I get to decide that for my own class and when you are a TA, you may have to abide by the professor's opinion about how much and how directly you should help with assignments.

don't let them get away with "I used this formula, and I solved for [variable]" -  See highly relevant video

Ouch. Painfully true to life, except that I was never as perky and attractive as the TA in the video.

How about giving them a blog of SE wirtten by Jeff about how SE has improved his life?

For CS assignments I wouldn't point out specific mistakes but would gesture vaguely in the direction of anything glaring or get them to talk me through the logic step by step and make sure they knew what they were being asked for. If there was a significant mistake I might put the assignment to the side and go through some of the basis material with them related to the mistake.

This is eerily similar to Pólya's "How to solve it"...

Re grading: One of my favorite professors has a homework grading policy I have always liked. If you turned it in, seriously attempted every problem (and explained why you got stuck if you couldn't get it) and got at least 75% of the questions right, it was a 100%. If you attempted them all but got less than 75% right (even if it was 0 right), it was a 66%. If you didn't seriously attempt them all, no matter how many were right, it was a 0%.

I love using a technique like this on people who come to me for help with questions when I'm not the TA. The only real danger I've found is that it's too easy to accidentally go overboard and suddenly realize that three hours have disappeared on what should have been a pretty simple problem!

I never understood why my fellow TAs took the attitude of "I won't help you with homework problems".  My approach was always that, if you're willing to show up to my office hours and actively try to solve the problem, I'd be happy to keep dropping hints until you finally get the answer.

Your students are lucky.

When I was a TA, my preferred approach to this dilemma was to offer to work a problem that was thematically similar to the problem, but different in its specifics.  For example, if the problem was working out an application of a particular algorithm to a scenario by hand, then I would make up another scenario with similar properties and we would work the algorithm on that.  In those cases where a student tried to then turn us back to the homework problem again, I would say something like: "I'm not going to work on homework problems, but we can work another similar one if you like."
I found this approach valuable because it let us address the skill and content questions and concerns that the student had, yet at the same time made sure I wasn't doing their homework for them.

This is what I do with my kids to help them with their homework. For example, give the following problem.
2x +5 = y, solve for x
Then I may instead work out this problem with them.
9 + 6y = 2x

The answer may very much depend on departmental practice. I've been in departments where the TA's office hours were essentially going over the homework and helping you arrive at an answer, and departments where that's absolutely not the practice.
Assuming it's not the practice, something simple like "I'm sorry, I can't tell you if your answer is right or wrong. If you have a specific question about a step, I'd be happy to go over that with you."

This is essentially what I do.  However, I'll also offer to discuss other example problems that are not part of the homework assignment if the student would like to do that.

s/can't/won't/ in the suggested reponse

"If you have a specific question about a step, I'd be happy to go over that with you." Then they could just ask about every step of the problem.

@user8001 "How do I do the next step" is not a specific question. If they want to solve their homework during my office hours and ask a question when they grind to a halt, I don't mind. And if someone wants to try and rules lawyer my help...well, that's certainly a strategy.

Standard answer around here is "I don't know" or more politely "I won't check your work other than for grading it".
Want to ask about a knotty point? Go ahead. I'll even solve a very similar problem with you. Still outstanding homework is off limits.

I asked vonbrand and not even the TA knows how to resolve this problem!

@Ángel , it's the lecturer in my case

Maybe set 3 homework problems, only grade problem C, but tell the students that if they completely understand how to do problem A and B, and then C will be easy for them.   Give them as much help as they need on A and B.
Even publish a worked solution for A and B a short time before C has to be handed in.

How does this answer the question?

@corsiKa "What to do with such students?" "Solve problems A and B, but not C." Actually it is a variation of jakebeal's answer.

This is a good time to teach a student good study practices.  E.g. remind them to review their work regularly, do their work on schedule, use course resources such as texts.  Tell the student how learn to get the right answer on their own.

Aren't TA office hours supposed to be one of those resources? I've found that working through a problem that actually counts towards their grade gets students' attention in a way that very few things do.

A TA who holds office hours for students should by all means consider homework assignments "fair game" for discussion during student interactions. Typically, the best approach during those meetings isn't to correct/grade the assignment, but rather to confirm whether the student's approach and methodology was appropriate, and deliver guidance or identify resources that will aid the student.
However, "proofreading" a homework assignment for a student who wants you to catch their "sloppy" is not really why a TA is there.

I tell my students:

I will mark your assignment any time you like. If you want me to mark it well before it's due, I will be happy to do that. But I will only mark it once. Hand it in, and I'll mark it,

So that rules out "please look this whole thing over and let me know how close it is to being complete." I think that's a real imposition and it interferes with learning too. However, if someone wants to come to me and ask whether one particular thing is headed in the right direction, or if they are stuck and want me to unstick them, I will. I will answer any specific, crisp question that they ask. But I won't just take the whole thing, put energy and effort into looking it over as a whole, and then hand it back for them to improve. That request comes from a desire to get 10/10, not a desire to learn the material of my course.

In a practical point of view, it would be best to teach the student at this stage. Neglecting might affect the interest of the student within the subject. This is the time when the student can learn the most as he/she is readily prepared to accept inputs. Doing so would greatly boost your reputation among eager students since a true teacher grows the interest of her/his students.
As it may not be ethical to solve homework problems of individuals per se as you stated that this makes up as part of the academic score. But it would be better to make the student understand how to solve the problem. If several students have the similar doubt, then it would be best to solve the problem as a group intimating the remaining students of this special solving session. After all, the motive of homework is to make students learn and remember as well as verifying what they have learnt.

A lot of these responses seem to be based on the idea that education is somehow a battle between teacher and student, and that they can only learn from suffering.  People learn from one on one instruction, and anything else is an expensive waste of time.  Denying them that so as to make absolutely sure that they don't benefit unfairly ... I suppose that's a sign of the relationship between students and teachers in the university setting - the student pays but the teacher tries to avoid being responsible for the outcome, acting instead as though he's the point of the whole thing.  That none of this is clear to a lot of university staff makes me wonder exactly how much intellectual life goes on at your various institutions.

In case you did not understand the question, homework problems weigh 20 - 50% of the grades. Checking the correctness of homework for an individual student may be unfair to the other students.

This answer comes very close to saying "I am paying for the education, so the instructors should do what I say." Among the many things students are paying for is the opportunity to be certified in a particular subject. This question is asking how to provide effective instruction without undermining the assessments that go into this certification. I don't see how this post answers that question.

Also, the tone of the question suggests that at least some of the students are wanting a "premark" - getting a grade from the TA before actually submitting the exercise. Depending on how much time the TA has for such things (in particular - is there enough office hours for all students to do this?) it could be quite the advantage...

I am having great difficulty where this notion of "unfairness" comes from.  It would seem to be a misplaced notion...or a lack of understanding of what "fairness" means.   If a student is allowed additional resources that are denied to others, for example given additional time to complete, then that's possibly unfair.  If student A's paper is graded to a different standard than student B, that's unfair.  If student A avails herself of resources that student B could access as well, but declines, then that is not an example of unfairness.

If each solution counts as a negative score to them, and if you have a solution manual, tell them the problems they got wrong. Students sometimes know the subject and are just trying to get the best grade they can.
If you don't have some sort of solution manual of course it may be difficult to tell them every question they got wrong. However if you don't have other students during office hours who have actual questions about the subject, what's the point in not receiving their request. Office hours are for the student. If you don't have any other student requests, go through the homework with them.
Of course if each question doesn't hurt the student's grade if they get it wrong, then you should expect them to give real questions about how to solve a problem.

"Check my homework" is an unreasonable request.  This is a teachable moment that can help you teach a student how to ask a question, which is more important than anything going on in an idiot dual class.    You need to get the student to ask questions that can inform you where the student is conceptually weak, so you can try to patch the holes.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Patent trial data set
I am looking for a data set that would list patent trials that involve an academic institution, if possible with the date, the parties involved and the outcome.
I am aware of this list of number of US patents awarded by academic institutions in 2014, as well as (1), which lists the evolution of yearly aggregate costs for resolved lawsuits, unresolved lawsuits,
and nonlitigated assertion for patent trials (non-academic specific), but I couldn't find a similar data set for academic patent trials.

(1) Bessen, James, and Michael J. Meurer. "The Direct Costs from NPE Disputes" Cornell L. Rev. 99 (2013): 387.

Have you tried asking a law library's reference desk how you could research this?

@keshlam No, there is no library near me.

If you're asking about a (presumably, preferably, open) data set, this question might be a better fit for the Open Data SE site. However, since you express some interest in references and I was quite curious about what information on the topic is available, I will try to answer your question here.
I would start searching for the relevant data sets, at least, on U.S. Patent and Trademark Office (USPTO) and European Patent Office (EPO). Doing so reveals some data sets that certainly could be useful for the task at hand. In particular, the following data sets & services might be of interest:
Data sets and databases

USPTO Patent Assignment Dataset
USPTO Historical Patent Data Files
USPTO's PatentsView Datasets (query tool and APIs seem to be unavailable at present)
Electronic Document Information System (EDIS) by U.S. International Trade Commission
EPO Datasets
Databases by Darts-ip (commercial; see "Patent focused" quadrant)
LITALERT (LITigation ALERTs) Database (commercial)

Services (search + analytics, etc.)

Lex Machina
Docket Navigator
Patexia Litigation Search

Papers / Books (bonus)

The effects of patent litigation on university licensing efforts
Innovation and litigation: Tensions between universities and patents and how to fix them
The trials of academe: The new era of campus litigation

P.S. Obviously, for most data sets and databases, you would have to perform data pre-processing to select/associate data relevant to academic institutions and, perhaps, some additional steps. Considering your background, it should not be a big problem. I hope that my answer is helpful.