text:
What should I let students do when they finish exams early?
I organized my students to take their exams in the school’s computer labs. Some students finished well before others. In each section, I tried different policies for what students should do in this situation, but none worked very well.

I let one section of students leave when they finished, but make much noise while moving their chairs and gathering their things. Furthermore, I am concerned that students will find ways to cheat, e.g. leave and access the Web-based testing system from their iPod.
I let one section do as they like on the computers when they finished, but this made it difficult to determine who was finished and who was using resources to cheat on the exams.
I gave one section some extra credit work to do if they finished early, but many students did other homework instead, which led to similar difficulty in determining who was finished or cheating.

What is a good policy that does not cause interruption to other students but also does not hinder proctoring efforts?

In a written exam, you could allow the student to hand in their paper and then retrieve a book (that they left up the front of the room) to read until the exam is finished. If you can finalize the submission of an electronic exam before the exam time is up, perhaps you can do a similar thing.

In the first case: Did most of the disturbance happen shortly before the end of the exam? It’s due to this that we do not allow students to leave early in the last 15 minutes of the exam time at my university.

Also: How long do your exams take and how many of your students finish how early?

Exams are 90 minutes. The fastest 5% of students finished in 30 minutes. Most finished at 60-70 minutes. The slowest students 5% used the entire time.

I'm surprised your institution discourages leaving early. Students that are finished don't need to be monitored for cheating as opposed to keeping them there and watching them fidget for another hour. In all of my grad and undergrad classes we were allowed to leave as soon as we were done. Hopefully, by this level of education, the students will understand that reasonable time spent on the exam does not relate to grade on the exam.

Out of the box answer: do exams 4 times longer than now (that should take them 120 minutes). Correct exams. Normalize so that the best one has 100% of the qualification. Alternatively (if there are several exams per course) allow qualifications over 100% (e.g. up to 150%, never 400%), so they can get the (ponderated) average. 120% in one exam and 80% in another could imply a 100% in the final score. If there are special mentions for some students (there are in some cases) then a draw is less likely in this scenario.

@Trylks: I strenuously disagree.  I think exams that are designed to take longer than the time available are a terrible idea pedagogically, regardless of how they are scored.

@NateEldredge I completely agree. In my studies I had an exam done with that reasoning: complete a project in groups (of ~3) in X hours (basically write a compiler's front-end, from lexer & parser, typechecker and intermediate code generator, in about 160 hours). The end result is: 1) students wont go to lessons during such a project 2) Students get really tired 3) Students get really angry because, even without sleeping, you simply cannot produce what most people would consider a good result. 4) The professor is hated forever afterwards. Although written exams are probably less stressful.

@Bakuriu I did that, for a subset of Pascal, and it wasn't that bad. Besides, that is not an exam, that's homework that may count for the final qualification. An exam (according to my personal and very casual definition) has questions, answers, and at most as many time intervals exclusively for it as questions are in the exam.

@Trylks The reasoning is the same and the disadvantages are the same: what do you earn by stressing students so bad? You are just evaluating how much the students handle stress, not how much knowledge do they have on the actual course subject.  Besides I wouldn't call "homework" something worth 85% of the course mark, that you have to complete in 6 days straight during lesson period.

One thing you could do is encourage your students to double-check their test results if they finish early. Whenever I finish a test well within the time limit, I always go through the entire test again and make sure my work was correct, which usually saves me a few points of credit and takes up the entire remaining test time. I suspect this would give many of your students who leave early something productive to do with the extra time that also improves their performance.

@NateEldredge At what level? I was raised on exams like these. I think it depends on the maturity of the class. I've made tests where there are more points possible than the value of the test so students can choose to "punt" on some questions. When I was a student, I regarded tests like these as an opportunity for some friendly competition with my peers. The only problem is if you don't make the situation clear to your students beforehand.

@DavidHill: At every level.  Just because they are often used doesn't mean they are a good idea!  I understand that others may feel differently, however.

Jeez. Just let them leave.

@Bakuriu why do they have to complete in 6 days straight something that takes 160 hours? That's insane. We had about the full semester to find that time and do that work (which was 60% of the mark). How would you call that?

All my university exams in the UK were done under proper exam conditions, meaning you weren't allowed to take anything into the exam other than your pens/pencils, a calculator if allowed, and a clear bottle of water with no label. We were not allowed to leave in the first 30 minutes or the last 15 minutes. If we did leave, we had to do so quietly and weren't allowed to return. All our bags were left outside the exam hall, so we weren't rummaging around in the exam hall.

One way to minimize disruptions would be to make sure that the students leave their coats, backpacks, etc. in a designated area (near the door), such that they aren't opening zippers next to a test-taker or walking past someone with a bulky coat or bag. I think that particular aspect of what is disturbing the others could be clarified.

Why were the students leaving early disruptive? Under the rules of every institution that I've been at you are under exam conditions until you leave the room. They should be utterly silent and respectful to other students or face the usual penalties for breaking the rules of exam conditions.
If you're not happy with them leaving early I'm not sure why you would do anything to occupy them. Make them sit in silence and wait for the the exam to finish. Boredom never killed anyone.

+1. If students are sitting in the rows of a lecture hall, having to get up to let someone else out can be a nuisance. But I can't imagine a computer room layout that would be like that.

@Moriarty We have had decent experience with using only every other row (if you have the space). But then, few students leave our exams early.

The problem with leaving them quietly is that essentially you end up punishing your more capable students for finishing early. In school I finished almost every exam in <50% of the allotted time, that's a lot of hours of sitting quietly wasting life.

@JonStory: Believe me, I am familiar with those wasted hours. However, I think it's entirely reasonable to inconvenience those few for the benefit of a better running exam. Compared to the amount of your life you will waste commuting, say, it's not a big deal.

@JonStory  By allowing students to leave, you also encourage the less capable students to give up and leave early. But which of the two evils is the lesser, is a tricky question.

@Moriarty  I've never seen that in practice,either as a student or a TA.  Both the weaker students and the pre-meds (snark) stay to the end in hopes of either finishing more or triple-checking their work.  Some of my spouse's top medschool classmates left exams very early and nobody gave a [redacted]

Are these adults? If so, isn't forcing them to stay in a classroom false imprisonment? If not, would this question be off topic?

@AaronHall: Don't be stupid, of course it's not false imprisonment! They can make the adult decision to fail the test by breaking the rules.

I don't think I've ever seen such an attitude towards grades, and I've been in quite paternalistic settings.

By allowing students to leave, you also encourage the less capable students to give up and leave early. — [citation needed]  Allow, yes.  But encourage?

-1 for the last part. Boredom never killed anyone, but you waste the time of the students which put effort into learning the stuff. As a Student I hate to sit and wait for an hour or so ...

@AaronHall: In that case, wouldn't forcing them to stay in the classroom, or in a restricted area around the classroom (if you include the bathrooms, for example) during the time of the exam, be "false imprisonment", as well? Of course it wouldn't; by signing up for studying, they agreed with a certain set of rules (and usually had to sign for it!), which include the acceptance of minor inconveniences during certain exam-related situations. Someone at university age is usually mature enough to cope with those restrictions on the comparably few occasions they ever matter.

@AndroidRookie: I slightly question whether someone who gets seriously bored while sitting around for just an hour in a silent environment is actually fit for studying. Aren't there always so many things to ponder about, to plan, to think over, in one's mind? I see it like this: That time is only wasted if the students do waste rather than use it.

-1 for the attitude "Boredom never killed anyone.".

@PiotrMigdal: It's an exam. It's a short, one-off, period of time which serves the primary function of successfully examining the students above all other purposes. If that requires the students to sit quietly with nothing to do for a while, so be it. It really is no big deal.

@JackAidley OK, for you it is not, but other may dislike (or consider it a waste of time) to sit hours doing nothing (it wasn't said if the exam is 1h or 5h). For (almost?) all exams I attended I was able to leave at any moment.

@JackAidley I wouldn't consider it a big deal, but I would definitely consider it an impractical use of time. If there's 15 students in a class and 14 are done at 45 minutes and the last student plans to use all 2 hours, is holding the 14 students there for "benefit" of the 1 student really necessary?

@Compass: if they cannot leave early for whatever reason then it is a perfectly decent alternative. As you should note from the first part of the answer, I don't advocate it as a first course of action.

@PiotrMigdal: Our OP has already called that out as a problem.

@O.R.Mapper Not everyone works in the same way you do. Some people need (or prefer) external stimuli and will grow extremely agitated just sitting there quiet. And some people just have other exams they could be studying for, except they didn't bring study materials because they assumed you'd treat them like an adult and let them go home when they were done.

@ChrisHayes: I'd argue one can expect that an adult can sit quietly without growing extremely agitated. That's how the students are treated, when the situation (e.g. an exam with the conditions alluded to in various of the answers here) calls for it.

@O.R.Mapper Being an adult means that when you grow agitated you won't act out in some way. It does not mean that your students aren't in some form of distress from the experience. Personally I find it ludicrous to even consider telling students they aren't allowed to leave when they've finished their work. Penalize the ones who leave loudly and let the good, quiet students go.

@ChrisHayes: They shouldn't act out, by, say, refusing to remain seated until the exam is over. I think the reality is even the quietest student will make some noticeable noise when packing their bag and leaving (let alone force others to temporarily get up from their seats, if the exam is done in a "lecture hall" type of room). And even if they could be entirely quiet, that would just start the issue of someone outside of the room knowing the exam questions, and thus possibly colluding by leaving messages in the bathroom etc. Unless that can be prevented, fairness requires that everyone stays

@O.R.Mapper If they want to cheat by leaving messages in the bathroom, they will, unless you're barring them from bathroom breaks (and I hope you're not). I don't think we should punish everybody for that remote scenario which isn't even prevented here. I've never sat in an exam where students weren't allowed to leave at the end, and I would massively lose respect for the professor if I were in such a position, that their best solution to cheating was to force me to waste my time.

@O.R.Mapper In any case this is getting rather too long for comments and I expect the whole thread to be deleted soon anyway, so let's agree to disagree.

I would suggest perhaps letting them leave in a more constructive way - when finished, the student raises their hand and waits for a tutor to come to them. They state they've finished and are escorted, quietly, from the room. Add a minimum time at the start and end during which they can't leave, to avoid disruption at the important settling in and final rush times, but during the bulk of the exam, people will barely notice.
Alternately if they aren't allowed to leave by the faculty, I'd arrange something whereby the student has a marker (eg a red cone of paper) on their monitor during the exam. When they finish, they raise their hand again and you come to remove the cone, at which point they're allowed to browse and do homework etc. that way you can differentiate between those finished and those attempting to cheat.

I like both options you've presented, but the first is probably simpler in practice to monitor. Proctoring an exam is complicated with the distractions that the second option you've indicated provides.

Well, it depends to an extent on staff ratio, but I agree that it's better to avoid if you can't do it well. My preference would be my first option too - allow them to leave without disrupting others

That 'red cone' strategy needs to be the other way around - that is, holding it gives permission to browse - otherwise you can just take it off yourself mid-test.

That probably depends on your room setup: I imagine it should be fairly obvious in most if someone is reaching up on top of their monitor. Either method should work, though, and can be chosen as to suit your environment

I had one teacher, that put crossword or sudoku at the end of each test sheet. I think it is much better to give students something to do in the spare time. You should choose something that you can easily differentiate from cheating.

I would worry that the students that get stuck or lost would get distracted with such items and lose focus, potentially causing lower grades than they would get normally.

Well, If someone loose focus on test because of sudoku, they would probably lose it otherwise.

This is from personal experience as a student and proctor.

A student's "stuff" --
The best rule to have is that students are not permitted to bring anything to the testing site.  If this is just not possible, you can ask students to put what they bring along a wall or up front, on the floor, near the teacher's desk.  This rule alone can result in students bringing less stuff to the exam.
Phones and electronic devices --
Ask all students to pull their phones out before the exam starts and ask them to either put them on silent or simply turn them off.  At this time, tell them to put them away and inform them that if a cell phone or other electronic device is seen in the open before the student leaves the exam room, they receive a zero on their exam.  They can wait until they are in the hallway before looking at their missed calls/messages.
Computer monitor --
Tell students that after they have completed their computer-based exam to turn their monitor off.  This can be optional based upon circumstances.  You could also ask the students to close all open programs and return to the desktop.
Permit students to leave upon finishing their exam --
Tell students that they are in "exam mode" until they leave the room.  Any spoken words that are not directed at a proctor or the instructor will result in a zero on their exam.  They can retrieve their belongings from along the wall or from the area near the teacher's desk.  You can even go so far as escorting students to the door and opening and closing it behind them.
DO NOT assume that students who finish last are weak students --
I am a graduate student and am usually one of the last to complete an exam.  I am also one of the highest grades in the class.  Some students have text anxiety or concentration problems that cause them to take longer on their exam.  You can even offer to allow students who know that they take longer on an exam or have concentration issues to sit in desks that are furthest from the door or locations that are prone to disruption from students leaving.

I hope you find this useful.

@nyuszika7h.  Good call, thanks.  I updated my recommendation above.

I simply make an announcement with, say, 30 minutes to go requesting that, to respect the concentration of the students still working, students finishing early remain quietly seated until time is up.
This usually has the intended effect.  The worst thing that's happened is that some students get up and try to leave immediately after that announcement.  I meet them in the aisle and quietly repeat my request.

You did not specify what grade level these students are at.  I tutor third-graders, and if they are let go early, they are definitely disruptive.  But if this is college level, you are nominally dealing with legal adults.  Forcing them to stay in the room until class is over sounds like unlawful restraint.  I didn't even require that my students show up, except for the major exams.  But they would lose all possible points for class participation, and hand-in assignments were always due.  But, I explained, their chances of passing one of the exams were very close to zero unless they managed good study habits.  A couple students tried this each term.  Some got A+ and some got F. I finished one of my 3-hour PhD exams in 20 minutes, handed it in, and walked out of the room.  It never occurred to me to ask permission, and if I had, the exam proctor would have thought it bizarre. (I passed with a perfect score).
If you have college students who are "disruptive" if they are not in class, your school has deeper problems than whether or not students can leave exams early, and they are not your problem.  Even as an undergrad, we were treated as responsible adults, and such grade-school silliness as I've been reading here would never have happened.
I'm amazed that college-age students even tolerate such treatment.  Or need it.
Note: if the students are disruptive, one way to control them is have them line up, buddy-to-buddy, and hold hands while walking in the halls.  Like I did in pre-K.

The context of academia.SE is higher education, so grade level is implicit in the question.

I would have thought that from the "charter" of the group, but the question and many of the answers made it sound like the context was Pre-K-5, like the school I tutor in.  The whole discussion seems remarkably silly in the context of higher education.

I reread your answer and now I understand your point that students at a certain level should be treated like adults.  But in large classes (e.g., hundreds), or with only a few minutes to go, a stream of leaving students (who are also packing up their belongings and excusing their way to the lecture hall's aisles) can be disruptive to those trying to finish.

I do see the general concerns of allowing students to leave early; after all, they might meet up with whoever leaves for the bathroom while still taking the exam and provide them with information specific to the exam.
Now, of course that could happen as long as no-one has left, too. Someone could hide information in the bathrooms, or someone could meet up with someone not taking part in the exam at all, or that latter person could hide some information in the bathrooms after the exam has started. These issues could be mitigated in the following cases:

Only students taking part in the exam can enter the bathrooms. This depends a lot on the architecture; unless the lecture hall is extremely large (> 500 seats maybe, from what I could observe so far in universities?), the average number of required toilets at any moment during lectures generally does not warrant an extra set of bathrooms reserved for a single lecture hall. And even then, those bathrooms seem to be more often than not accessible in a way that one does not have to cross the lecture hall (and thus can enter and leave the bathrooms without anyone in the lecture hall noticing), for the very purpose of allowing outside people to use the bathrooms without disturbing whatever is going on in the lecture hall.
Students need to be accompanied to the bathroom door. Depending on how many proctors were assigned, and the size of the room/number of students, this may or may not be feasible. And even then, it would not totally prevent the exchange of information to take place in the bathrooms themselves.
The inside of the bathrooms needs to be checked whenever a student is brought there. This would require to have at least one male and one female proctor around. Highly unlikely to happen in gender-unequal disciplines such as computer science.

Those cases of cheating would be somewhat undirected, anyway. It may depend on the exams, but we generally try to not ask for any knowledge that needs to be memorized (in some cases, notes are even allowed during the exam). Our exams are usually designed in a way to test whether some knowledge can be applied in scenarios that are described on the exam sheets. Hence, the major concern is not cheating by accessing the course material or other references; the major concern is having someone else specifically solve one's particular tasks from the exam at hand. Therefore, what needs to be prevented is the contact between someone who is still taking the exam with someone who also knows the exam tasks. The straightforward solution to this is asking students to wait until everyone has finished.
Another point is that no matter how quiet students try to be, when they get up, they will make at least some noise:

Walking around creates some noises on non-carpeted floor, so that should generally be minimized. Leaving for the bathroom is allowed as there are medical reasons for that, but there are usually no such reasons that would warrant the impatience of having to leave right away.
Students who leave need to pack their stuff (writing utensils, drinks/food, other objects they needed to have around such as watches and their student IDs), which again will create some (more than just from writing) sounds.
In case of "lecture hall" type rooms, that do not have single chairs, but folding seats mounted to the next row of tables, students who do not sit right next to an aisle can only leave by making everyone else between themselves and the aisle get up. It is annoying when that happens in a cinema, and it is downright antisocial to disrupt someone's concentration like that who is taking an exam.

Some students may complain that they are wasting time, and - from an egoistical point of view - they may be right. However, unless we can provide a single room and a single proctor for every single student, that is not how exams realistically work:

They do not have to stay for an unexpected amount of time. If the exam was scheduled to take place between 2 PM and 4:30 PM, they can expect to leave by 4:30 PM. The time was known beforehand, and they will have arranged their schedule accordingly.
They are not the only ones taking the exam. Indeed, they (think they) have finished their own exam. But that doesn't mean they can stop caring about their environment at that moment, as the world still isn't centered around them; once they have stopped writing, it is their obligation to allow the other students to finish the exam without any further disruptions. I do not believe in punishing students who take longer by giving them an even harder time. Proctors need to guarantee avoidable disruptions are avoided, and giving in to someone's impatience is definitely avoidable.
The time is only wasted if they decide to waste it. There are plenty of things to do while waiting in a silent environment; from thinking - to get one's thoughts away from the exam topic - to sleeping. All of those are much less counterproductive with respect to the other students than insisting on creating more noise by leaving immediately.
Some overhead is to be expected. Reading out the exam rules and checking attendance in the beginning takes quite some time (in large exams, often more than 20 minutes). That is expected when taking an exam, and likewise, students should expect that there will be some time after they have finished writing that they still need to spend in the exam room.

Therefore, my general preference is to simply ask them to wait till the time is over. They can use some of that time to make sure they completed everything correctly (when do you ever get the chance/time to check what you wrote in an exam? You should use that opportunity!), and other than that, they are adults. They should be able to show a little patience on a few occasions.
EDITED to further address some more specific points that were brought up in various of the comments in this thread.

We always had exams in rooms with directly connected washrooms...

@gerrit: Sure, some rooms are built like that, but most exams I have seen were conducted in normal lecture rooms with the closest washrooms being the those for the whole floor (which tends to be more than enough, in terms of free seats per interested person on the floor).

In that case, I would anyway expect anyone going to the toilet to be accompanied.

@gerrit: With only two proctors available, that would leave only one person in the room, so that's not always possible. And if you add the possibility that someone who finished early might hide something (or themselves) in the washrooms to exchange some information there, the proctors would have to enter the washrooms with the students, which would consistently require at least one male and one female proctor - extremely unlikely to happen in "gender-unequal" disciplines such as CS ;-)

@gerrit: For smaller classes there is a good chance that there is only one proctor.  Certainly that's the case for the classes I currently teach: it's me.   Don't ask what happens if I have to go to the washroom...

@NateEldredge I've seen situations where the single proctor left for 5 minutes.  Formally that would have invalidated the entire exam, so understandably, nobody reported this incident.

It depends a lot on the class and on the age of the students, which the original poster did not share. University students are expected to behave like adults, secondary school students may need a bit more supervision, especially if the school is not set up to accommodate students with no particular assigned place to be.
If a lot of students are going to be finishing early it may not be possible to give them a place where they can sit separately and use materials that contain possible exam help (even their class textbook may be inappropriate to read within view of students who are still working on the exam); in this case it may be best to provide reading material on the exam computer that will allow them to get an early start on future work, such as a reading assignment from a later chapter of the coursework.
Extra credit questions on the exam may also provide a way to keep all students occupied for the full exam time.
(As an illustrative example, I finished my final exam in a university economics class in 20 minutes, checked my work three times, and still walked out in less than a third of the allotted time.  Telling students to spend the extra time to check their work may not be helpful.)

Based on your question, it sounds like several students may be asked to sit quietly for 30+ minutes.  I don't know the specific policies of your university, but here are a few suggestions.

If you have an empty back row, allow any student who has turned in his exam to take a seat behind currently working students and use phone, laptop etc.
Have a TA offer student escorts outside the building every 10-15 minutes.  This will be the way students can leave before the exam is up.
Double check policies.  Universities in America (and likely Europe) generally count the exam done when the student gets up from the chair.  The students are not generally disruptive as they leave the building, and the no in and out policy prevents students from colluding in the bathroom.

Checking all entrances of a whole building sounds like quite a huge task.

I'm going to agree with Jack on a couple points (why is students stepping out disruptive?) - with few exceptions and on major difference I think:
Test is over, class is not over
The main difference I'd like to point out is that in my limited experience (as a student) the test being over doesn't mean the class is over - normally that means you start lecture on the next chapter/section/subject... After a test students are free too:

sit at their computer - visible to staff and clearly not involved in test taking activities
run to the store (nearby, in building, snack shack)
just meander outside in the hallways until everyone is done - being respectful of our class AND neighboring classes.
etc
be back in your seat at X-time for further lecture

Either at a predetermined time (Test is 45 minutes) or as soon as all students are done taking the test, a 15 minute break commences that is then followed by further class time.
Test starts at 11 in the computer lab. You have until 12:15 to be finished (60 minutes for test, 15 minutes for break) and seated in the normal class room. Be respectful of those taking test, and those in nearby class rooms is more than reasonable if you are doing it outside of the normal classroom (computer lab).
What do you consider disruptive?
I think all of these suggestions to "Raise your hand to get permission" reeks of grade school and isn't something you do with responsible adults. I would find THAT more disruptive than Be quiet, respect others and wait or step outside until the test+break is over
But I think a major unanswered part is what are YOU calling disruptive? Students weaving through packed seats? Students saying "I'm done"? Students simply moving? Doing cartwheels through the isles due to the joy of finishing a test?
If you need to put up a guide, and treat it like every other disclaimer -

Don't eat the Chiclets included with your hard drive.
don't use hair dryer while in the shower.
Don't do cartwheels after the test.

... because SOME idiot had to eat the Chiclets or use a hair dryer in the shower, then do so. But I think those kinds of situations are probably covered in the generic "Don't do that" information you get when you start college.

This will depend on your local norms for class scheduling and exam length.  Around here classes are usually 50 minutes, and an exam is scheduled to take the entire 50 minutes - there isn't time for lecture or other activities afterwards.

Raising your hand in an exam room is fairly common, due to the restrictions on non-verbal communication. Having class after exams is fairly uncommon - especially after a 90 minute exam (2 hours plus, including the time taken to seat everyone, read regulations, collect papers etc)

Among the three policies you suggested, only Policy 1 - Let them leave when they're done is good - or rather, it's the only morally acceptable one.
The reason is that you have no right to keep people in their seats doing nothing when they're done with their exam. That would be treating them like prisoners. You can and should make an effort to arrange things so that their leaving will be less disruptive (e.g. chairs which don't make a scratching noise when pulled...) - but nothing beyond that.

There's nothing immoral about making students stay; they freely agree to that condition when they sign up to the course.  By this logic, it's immoral not to let them talk to each other...

@sapi: They did not "freely agree" to let you lock them up doing nothing.

That would be treating them like prisoners No.

@einpoklum: They freely agreed to obey by exam conditions (restrictions), and in turn be guaranteed exam conditions (warranties). Among those warranties is the one that they do not get disturbed by other students walking around, unless inevitable (such as by going to the bathroom), and among those restrictions is that they do not disturb other students by walking around, unless inevitable (such as by going to the bathroom). I do not see a reason why it would be inevitable that they show their immature side by becoming impatient rather than wait until the exam is over.

@O.R.Mapper: I believe you're fetishizing contractual consent. You do realize there's not much that's 'free' about it, right? Anyway, leaving a room after you're done with the exam rather than being forced to remain in your seat is not acceptable, and if students have any bit of backbone, they just won't do it, so it's inevitable.

@O.R.Mapper: Oh, by the way - my university decided a few years back to prevent exam-takers from going to the bathroom through most of the exam. So the inevitable is also sometimes evitable, if the establishment is presumptuous enough...


text:
Can non-peer-reviewed work be recognized by the community?
A number of years ago I was enrolled in a university as a PhD student and was working on a specific thesis topic. My intent was to earn my PhD with this work. Due to lack of funding, I was unable to continue, but I have continued this work independently for a number of years.
I have managed to complete this work now but finding the right place to publish it seems ever more causa perduta for me. I am thinking about simply publishing it for free and trying to distribute it on the Internet. I am also thinking of making translations into several languages myself. Besides publishing in my mother tongue I am thinking to translate it into English and Russian and possibly German, too. I hope this can make it available for as wide range of specialists as possible. But yet the problem of peer review remains. Even if accessible to everyone on the Internet it isn't going to be something gone through the peer review process and it is likely my thesis itself would cause controversy.
As such, my question is, can research be accepted "in any form" by the scientific community? Does anybody know of cases within the past two decades when such a thing (research first published "free on the internet" being eventually accepted as mainstream) has happened ? I just want to know is this even possible in modern science or would it just "disappear" in the huge noise of the net.

Comments are not for extended discussion; this conversation has been moved to chat.

It sounds like there are several questions going on here, so I'm going to try and break them down one at a time.

As such, my question is, can research be accepted "in any form" by the
  scientific community?

Yes. While the scientific community at large is much more likely to accept research that has been peer-reviewed and published, within a community of researchers things are a bit more loose. Some ideas get passed around on list servers and through emails via pre-prints long before they are published.

Does anybody know of cases within the past two decades when such a
  thing (research first published "free on the internet" being
  eventually accepted as mainstream) has happened ?

As noted by zibadawa timmy in the comments to the question, the work of Grigori Perelman appeared on arXiv, was well received and is highly cited. However, as the commenter also pointed out, Perelman is a well established mathematician so their work gets much more notice when something is published to arXiv.

I just want to know is this even possible in modern science or would
  it just "disappear" in the huge noise of the net.

There is no guarantee that this will happen, nor is this a modern phenomenon due to the internet. Always remember the history of science. Gregor Mendel's "Experiments on Plant Hybridization" is considered to be a seminal paper, but was published in the fairly obscure Proceedings of the Natural History Society of Brünn and it wasn't until after his death that his work was rediscovered and he was properly credited.
To address your work directly based upon the comments to your answer. I would suggest that your best approach would be to get back into university for your PhD so you can receive mentorship and guidance. If that is not possible, your second best bet might be put together a manuscript and send it off for peer review in various journals. If it get bench rejected by the editor then they may provide feedback on better venues for publication. Likewise, reviewers who think the work is worthy of publication in a different venue will also do the same. This could help to find a home for your work.
Also, based upon some of the things you have mentioned with regards to complexity, I would highly recommend looking into the work of Stuart Kauffman and their book "The Origins of Order: Self-Organization and Selection in Evolution" in particular. Some of what you are describing sounds like work that is actively being done and published by scholars that investigate complexity in the context of complex adaptive systems. If that work is indeed similar to what you are doing, then that may point you in the direction of venues for your own publications.

As far as I know (and believe me I follow the field closely) Kauffman himself has a tons of problems finding funds and places to publish (and this is really HUGE name by all standards there) so what about a little researcher from an unknown country having ideas outside the mainstream? Can you see now why I am so desperate?

@YordanYordanov Maybe when Kauffman was first getting started, but as a MacArthur Fellow he's very well established at this point and a lot of people want to hear what he has to say. The hardest time in an academic career is going to be when they first start establishing themselves.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Is it normal that a peer reviewer would ask for an executable file in order to check my results?
I have just received a decision letter for my submitted manuscript to an Elsevier journal. It was a revise and resubmit. However one of the reviewer asked for an executable file in order to check my results. (I felt distrust from his comment..)
This is regarding a computer science paper on testing the efficiency of an algorithm on a set of instances from the literature. I compared the results of the algorithm with those of other authors.

To clarify: the paper talks about some software, and the reviewer wants to be able to run the software?
I don't know how common that is (I'm not a computer scientist), but so long as any license on the software permits you to give it to the reviewer it does sound like a reasonable request.

If the algorithm is ganna be published anyway, I don't see "why not"!

@DSVA: "If you publish an algorithm you need to somehow publish the actual code"... what percentage of published algorithms do you think come with source code?

@Mehrdad "what percentage of published algorithms do you think come with source code" a lot less than the percentage that should come with source code. Imho if you can't verify the claim without implementing the algorithm, then it should have source code attached, and the source code should be reviewable, or else it's not good science. No reviewer in their right mind would accept a paper with "magic happens here" in the middle of the proof so "closed source software happens here" shouldn't be allowed either.

A distrustful reviewer is a good thing, it means they are going to give your paper a stern test, and if there are problems with the paper, they are more likely to find them, which is to your long term benefit.  I very much doubt they distrust your honesty, just the result.

As someone with little-to-no experience publishing software (or algorithms), wouldn't providing them with a compiled executable only provide them with a way to run your code, not view it?

@Sumyrda "Imho if you can't verify the claim without implementing the algorithm, ..., it's not good science" Unfortunately, things are often not that simple for algorithms. Many algorithms can be proven correct, yet still be entirely non-trivial to implement. This is the reason why algorithm engineering is a serious field. One such case is Chazelle's triangulation algorithm. His proof is most likely correct, but there is no implementation for a while (as far as I know), even though it is an often 'used' algorithm!

In general Science terms, only a genuine peer-reviewer is liable to do such a thing. An excessive percentage are liable to flag it away as too hard. Their was a day (I'm told) when peer-review meant REVIEW!. Someone else was confident that you knew your stuff and would put their name (albeit anonymously) to the conclusion. Such people are of course very annoying, but they are part of the foundation on which real science is built. Or used to be.

What other way do you suggest should he go to verify your results?

@Discretelizard "Many algorithms can be proven correct, yet still be entirely non-trivial to implement." But that's exactly what I said. You either show pseudocode and prove that it is correct, or if you can't do that for some reason, especially when you claim that your algorithm is faster than some other algorithm, then you should have to show your code - or provide some other way to verify your claim if you can think of something.

Please clarify: Is the paper about an algorithm? Or a program? Or the results a specific implementation produced on specific input?

You should never distribute executables, but always the source code. Nobody, wants to execute an binary program without compiling it from source. It could do anything on your system. One can try to workaround with an sandbox, but why not distribute the source, if it is part of your scientific work?

@JonasStein I think the only reason I would want the source code would be so I could examine it for errors, not because I'm worried another researcher is going to bug my system. Especially one who didn't volunteer such executable but instead is producing it on request.

@PhdStudent "The good thing about science is that it's true whether or not you believe in it." I certainly have trust in our scientists, but that trust is earned through verification and peer review. When we publish science without verification, we open ourselves to pesudo-science at best and willful deception at worst.

I observe the use of male pronouns to refer to a presumably anonymous reviewer. I think this is our implicit biases working against us.

@Sumyrda There's a difference between source code and pseudo-code. Source code IS an implementation, just not in executable form yet. Pseudo-code is not source code, it is only a method to describe algorithms. But if you meant that at least some sort of verifiable (by the reader) argument for the results of an algorithm must be present, then I of course agree.

@Discretelizard Exactly, it has to be verifiable. Whether it's verifiable by going through the pseudo code alongside your reasoning or by reviewing your source code doesn't matter, but one or the other has to be possible. I'm torn about verifying the claim by running tests against a closed source executable - there's still too much "magic happens here" in there for my taste - but it's better than nothing.

I don't know if it's normal, but it should be normal for all reviewers to make reasonable efforts to verify that the claims authors make are correct, so to the extent that it's not normal, I can only commend the reviewer for being willing to make an effort that other reviewers don't make. What you sense as "distrust" is the reviewer doing their job, nothing more or less (and it is probably somewhat accurate to say that a reviewer's job is to distrust the author's claims, so I don't see the idea of being distrusted by a reviewer as something to be ashamed of or offended by).
By the way, it should also be normal for authors to make available any software (including source code whenever possible) needed to replicate and verify their results. So if you are unhappy with the reviewer coming back to you with annoying requests that delay the decision on your paper, next time around you can preempt such issues by releasing your source code (or at least submitting it to the journal) alongside your manuscript. I am sure the reviewer would be much happier and ultimately everyone would benefit, including you.

Thank you for your comment, perhaps I felt offensive because it is my first journal paper. However, in my research field it is not common that authors make available their code..
Anyway, i will send the executable to the reviewer because he stated that without this check the paper cannot be accepted . Thank you again.

As someone who works in the industry and reads papers as "an outsider", it has always struck me as very odd that there's a tendency to "hide the code". My feeling is that code, just like a paper, should be out there and available. I'm absolutely with Dan here, it should be done more. To me at least, not publishing code always has a bitter aftertaste of there being something dodgy going on.

Unfortunately, most of the algorithms and numerical methods described in journals like Journal of Computational Physics (a very reputable journal!) are implemented in in-house closed source codes. I did some review there and I just had to trust the results.

It would be nice for authors to make available software even when it isn't needed to replicate or verify their results. I can think of one algorithm paper which I once tried to implement, and I gave up because it didn't seem to type. A reference implementation would have helped understand the paper.

Agree about the proper framing of the feeling of "distrust" - better to interpret it as healthy scientific skepticism, which every reviewer should have. I put a level of trust in peer-reviewed papers (especially for topics I'm not intimately familiar with) precisely because they have satisfied a panel of reviewers' skepticism.

@SBI It's not odd at all. If you invest 4 years of work into a code, you need it to pay off in enough papers/citations/attention to get you a promotion. One paper will not do that, and if you release your code, you'll need to spend another lengthy period of time coding up a new publishable result while your rivals use your code to scoop you. It's not so much "publish or perish" as "publish more than alternative hires or perish". (Personally, I did release my code; I also perished.)

The answer has assumes that the reviewer has only the best intentions at heart along with a idealistic tendency to improve the scientific culture. While I'd commend those things instantly, unfortunately we live in the real world. One needs to consider that the reviewer is an anonymous person asking for the core part of a research process with a clear expertise in the field and the consequent potential interest in it. I'd advise carefully considering such requests also from the perspective that there are individuals who could try to exploit their reviewer anonymity and privilege.

Another red flag that catches my attention is that they request an "executable file". The user has no way to verify that the executable doesn't just print the results with no algorithm behind, aside from reverse-engineering it (without going into too much detail here). Such request can possibly mean that the reviewer tries to lull the author into a false sense of security by intentionally not requesting the source code.

@user3209815 : They may be catering toXerxes point: releasing the source code may not be acceptable because other authors can then scoop the OP.  Making the file executable means they'll have to decompile it... which may not be viable.  Although, as you say there's no way to verify that such an executable doesn't just print out the results required.

@Xerxes How can your idea be publishable in a paper though, if it depends so much on closed source code? I mean, you lay open your entire idea anyway. I've written code basing on papers countless times. Usually, code is more a proof of concept rather than holding more information than what you publish anyway. The other way around though, it becomes trickier. Making claims without showing they actually work is... difficult.

@SBI Of course, the paper must be such anyone can implement the algorithm indeed! But that can be a lot of hard boring software engineering work.

@Xerxes Ideally your work would be adequately supported by some funding agency (usually public).  In return, you open source your code.  Practically, I understand that you do what you have to do.

@Xerxes that is why code should be released with licenses that requires attributions. If people are going to take your code and publish a paper based on it, they would have to attribute you as the original author of the code, just like they'd have to cite you if they used results from your published papers.

@VladimirF: "I just had to trust the results". No, you didn't. If results are produced by software, then you should be able to verify the results, and if you can't verify them, then you don't accept it. There's the saying "pictures, or it hasn't happened".

I'd add that the executable (compiled code) by itself, while it would allow replication of effort, wouldn't be particularly useful in the grand scheme of things. The source code, as others have said, is the more important thing. Either it's commonly available software (in which case, why couldn't the reviewer(s) download it themselves?), or it's a rare or proprietary code, in which case the source code is the more important detail. It's the method that needs to be replicated, and the binary is a black box that hides the code, which is the method.

Reviewers are bound by confidentiality—you can provide code to reviewers without releasing it. Yes, reviewers cheating happens sometimes—though seldom.
There are multiple movements towards reviewers trying out code; see http://www.artifact-eval.org/ for one approach.

@gnasher729 Do you trust experiments even when the authors did not make their facility available to you to re-do their experiments? What if that numerical method was implemented in a CFD code to which the authors don't have full redistribution rights? The computation may take many a lot of cpu time on a supercomputer and the reviewers will not re-compute them anyway. I wouldn't for that paper I reviewed. The analysis of the results might also be time consuming.

@Xerxes No, you did not spend 4 years on your code. You spent four years on your algorithm, which you are already letting the world know about through your paper. The code is simply an implementation of it that you could have written in a fraction of the time, had you started on it after you had settled on the algorithm.

I come from a different field, in which the code we use isn't a major output.  But if a referee asked for the code, we would provide it and happily.  Most of our work is done in python so an executable wouldn't be usual, the source would (also true for matlab).
In fact the only thing I find slightly odd here is the use of executable rather than source.
Don't be offended by the request for a couple of reasons: It's not the reviewers' job to trust you; it's their job to check your paper.  If a reviewer takes enough interest in your work to want to run your code, they haven't dismissed your paper out of hand.

I like the final point. It would even seem a bit honouring to me if the referee would be willing to try out my software :)

@Džuris indeed, it means they are taking your paper very seriously.

Thank you for your second paragraph! No one here seems to see this. When I read the title of the question, I assumed OP found it odd that someone would request an executable rather than the source code because that indicates they don't care enough to check the source code for errors / mistakes / deliberate mistakes and in fact are too stupid or lazy to compile the source code (which I assumed was provided) themself.

@UTF-8 It's also possible they don't have a compiler (maybe can't because of licensing/platform issues) or don't have the skill in that language to make sense of the source.  Even then I'd still want the code

Upvoted and a ++ for the fact that it is weird they want an executable and not the source.  Heck when I grade homework, I want the source and anything else required to get it to compile, including compiler commands/statements/options/arguments if it is anything more complex than "g++ foo.cpp -o foo"

To summarise the situation with your data:-
1) You came up with an algorithm on paper/Matlab/whatever.
2) You implemented that algorithm in some programming language.
3) You built a set of test data to exercise your algorithm, and came up with some results for what it should do in theory.
4) You put that test data through the code and came out with some results for what it does in practise.
In this process there are various places where things can go wrong with your methodology.  Your code may not correctly reflect your algorithm.  Your test data may have been worked backwards from the code instead of forwards from the algorithm.  Your test data for your algorithm and your test data for your code may not be the same.
Unless the reviewer has the algorithm and the source code and all the test data for both and all the output data for both, they cannot verify that your work is sound and your conclusions are valid.  This is not subject to dispute - it is logically impossible, if they want to properly review your work.  Anything else is making assumptions which may not be valid.
I have personally been affected by this situation, when my company bought some control theory IP from a researcher.  He'd written papers on how this was supposed to work and the theory behind it, and then he'd built some electronics to implement his theory.  His papers covered the theory, and also included schematics for the electronics.  When I read this to work out how to implement his theory in software, I found that the schematic had an extra filter in it.  The action of this filter turned out to be critical to the system being stable or even effective, but it was not documented at any point anywhere in his work.  It wasn't until we had a phone call with him that we found out what the purpose of the filter was, and how we were supposed to tune it.
This was in a paper which theoretically had been peer reviewed when it was published.  Clearly it hadn't been peer reviewed thoroughly enough!  His results showed that given the same data, the implementation output was pretty close to the theoretical expected output, and the effect of the filter was at a different place in the response.  Still though, the implementation flatly would never have worked without this filter present, and it wouldn't have been at all hard to include this in the theoretical model.  He could even have said "this filter is required for these reasons, but can be ignored in this area of the response we're looking at for these reasons" and he would have been covered.  What is not acceptable is what he did, which is to fail to mention it at all, because the end result of that is that someone trying to implement his work would be unable to.
Like I said, he still got his paper published, and no-one complained at the time.  It should have been spotted by his original reviewers though.  In your case, your reviewer should be looking for discrepancies like this - it's the whole point of peer review.  So if people are asking you for things you haven't made available, (a) it's a good sign they're checking thoroughly, and (b) you should have made it available in the first place as best practise.

Artifact submissions are a thing in CS. What I've seen is that you'd prepare a virtual machine, where your software is already set up and ready for making experiments. So, the reviewer may be referring to that the journal has some official procedure for artifact submissions. Alternatively, some authors just make the source code of their tools and benchmarks available via services like github, and the reviewer may be suggesting you should also do this. Regarding the distrust, computing people are naturally wary about benchmarks and tool comparisons, as the final figures may depend a lot on how your experiment is set up (e.g., if you compare to your own implementation of an existing algorithm, did you implement it correctly). It could also be that the numbers that you give in the paper seem a bit odd, but then the reviewer would have pointed to what exactly doesn't look right to them.

Submitting an executable isn't the same as submitting source code. An executable doesn't really give the recipient any access to your original code (as a computer science student should already know, of course). I don't see a problem with this request.

I see a major problem: the executable run on the same test cases is going to produce the same results. even if those results are wrong due to an error in the program.

I'm curious how you would send an "executable" of, say, Python code, without giving access to your original code? Are you expected to obfuscate it?

@jamesqf They could try other cases, not covered by the authors.

Any executable can be decompiled to produce functionally the same code. Any code that compiles to JVM or .NET can be decompiled to something relatively close to the original, and even machine code can be torn apart with enough work. While I deeply believe source code should be published with publications like this, if you refuse to do so, you can't offer an executable as if it will hide what you wanted to hide from your source code.

@CaptainEmacs but they can't do that if they can't see what the program actually does. It might as well have data embedded in the executable or do something else than what is claimed.

@mathreadler Of course, it could all be unlucky. However, I have had exactly this situation during my PhD, and the authors sent me an executable and I could try my examples with it. If there is bad coding (hardcoded constants), sometimes it is still possible to edit the binary to fix that (I have done that, too, once).

Source code can have bugs, and to truly effectively review an algorithm, a prose description of the method alone may well be insufficient. Sharing something beyond the text is beneficial; a good paper with the actual source code (+sample inputs) is the gold standard for reproducibility.
One fun wrinkle: depending on where your reviewer is, you might not be allowed to give them a binary. Eg, some code uses proprietary libraries that are licensed freely in academia, but someone in industry might require a separate license to even use an existing binary, much less compile it. (this happened to me once, though not as part of peer review)

Given my personal experience with open source communities and the assumption that the paper includes the entirety of the algorithm in question, then sending the source code or related compilation of said software wouldn't produce many negative effects.
This would allow the reviewer to verify results and claims made by the paper's author. The key issue the reviewer might be looking for is that you correctly implemented algorithm in source code an are not mistakenly relying on a feature of the programming language, OS, or hardware to make claims about its running time or other features.
Off the top of my head I would relate that in I/O bound cases its easy to mistake efficient algorithms for, as an example, Javascript's ability to make almost every function call asynchronous. Of course this is mostly seen in I/O bound operations rather than proliferative computational loops. Then the efficiency measured is not that of the algorithm as a formal proof but; instead it relies on a language specific feature.
The salient point is that there are many cases in which the formal algorithm and the implementation can diverge from representing each other faithfully and in doing so the conclusion, if based on empirical metrics such as running time, can run into many issues where an improper implementation can attest to an incorrect conclusion.

It is an idiotic request on his part.

He could catch a virus.
There is no realistic way he can check that the executable implements what is described in your paper, ergo no way the request has any scientific value whatsoever.

He should be asking for the source code, and that is all you should agree to give him.

#2 is often false.  A significant number of problems have the characteristic that checking a solution is much easier than finding a solution.  In such a case, a reviewer can verify that a black box does indeed produce correct solutions and measure the runtime complexity.  This would be particularly valuable if, for example, the reviewer noticed that all the examples used in the paper had particular characteristics that made them easier to solve than the general case, as he could formulate his own test cases.

@BenVoigt But what black box? How can the reviewer know he has the black box implementing what the paper claims?

There is a valid point that use of a black box doesn't assure that paper contains an accurate description and explanation of the method used -- but the reviewer may be much less worried about the risk of having faked the description given that a novel method is proven to exist, at least compared to the risk of faking both method and description.

@BenVoigt I cannot make head or tail of that after the word 'but'. A black box does not prove anything about the claims in the paper. It only proves that a black box exists that produces the results claimed, somehow.

@Magicsowon I would like to sell you the Brooklyn Bridge. I have the title deeds. I won't show them to you but I can send you an .exe that will print 'yes' every time you ask it whether I own it.

@Magicsowon “a Linux user might feel safer than a Win user” Linux user here. I don't agree at all. You could easily write a program that damages personal documents or sends them through a network connection on both platforms, if the recipient is kind enough to run it for you. Linux is safer for "widespread" malware that is not targeting you specifically, also because there are more which are made for Windows.

authors: Vladimir F Героям слава

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
When should I start applying for a PhD program in the UK?
I am starting my MSc. this September (UK), and am strongly planning ahead and setting myself goals. Simply, when would be the appropriate time to submit my applications?
My MSc is 12 months long and will go from Sept 2018 to Sept 2019. I was thinking of applying 6-8 months into my MSc.
What do you think? When is the most logical time to submit my applications?

Just before the application deadline?

This kind of depends on what sort of PhD you wish to study.
For the most part, you don't get accepted for a PhD position in the UK by applying to a graduate program. Most universities/departments simply do have a formal "program". There are acceptations to this of course. Several of the larger funders run formal programs, look out for the Wellcome Trust PhD 4-year programs for example. Some of the research council's Doctoral Training Partnerships (DTPs) operate on this basis as well.  These are rare, and tend to be highly prestigious and often associated with research institutes rather than university departments  (the Wellcome trust ones are also very well paid compared to other ways of doing a PhD). They are recruiting earlier and earlier in order to try to grab the best students. For example, the Sanger Institute 4-year program's 2017 deadline with 2nd of December.
For the majority of PhDs in the UK though, the supervisor will apply for funding from a funding body for a particular project. If they are successful they will advertise individually for applicants, interview on their own schedule, and the successful candidate will start at the earliest convenience of the both the supervisor and the candidate. This often ends up being the start of the next academic year as the candidate generally has to finish their Undergrad/Masters degree first. When these positions are advertised/when the closing dates are will depend most of all on the funders grant deadlines. Our DTP (where we get most of our students from) has a dealine just before christmas and we will generally be advertising in Feburary. A good place to look for adverts for studentships like this is FindAPhD.com. Another place to look is jobs.ac.uk. I would have alerts set up on these sites for the whole year, but expect the busiest time to be the first quarter of the year.
Finally, if you will fund your PhD yourself somehow (either though your own funds, or that of a company or from a foriegn government), its generally up to you. The application process may take a couple of months and if you are not native the visa process can take months as well. If you want to aim to start at the beginning of the new academic year (which is not a terrible idea, since you'll be starting with all the RCUK funded students, making things socially easier), I'd probably start this process in the early spring: give you plenty of time, and it you are sorted early, waiting to start is not going to be a problem.

Minor counterpoint: this could be discipline-dependent, even within STEM. It seems currently the case in most UK (pure) maths departments that one should apply directly to the university, perhaps after unofficially contacting some people in the maths department, stating explicitly that one is seeking funding. Departments will then have their own mechanisms for allocating the limited amount of funding they have, from various sources; this can include RCUK studentships that have been allocated to the department from a more central pool

I guess this is more or less what I meant by the third route I talked about. Except that we would never consider anyone unless they knew exactly what funding it wast they were planning to apply for and what their chances of being successful were. The EPSRC, like other councils in RCUK, now longer awards all departments block grants for students, but operates through DTPs. Some depts will choose to operate a program, others will fund projects. I don't know of any that operate as a scholarship scheme that is only available if you apply by other means, but I guess that probably exists somewhere.

Funded PhDs in the UK usually follow the academic calendar and start in late September or early October. Accordingly, application deadlines tend to be in February or March (often set by the funding councils and hence will be the same for every university).
The exception to this is if you are planning to self-fund your PhD, in which case you will have more flexibility on starting dates (but check with the universities you are going to apply to if you are planning to do this).
The best time for you to submit your applications is probably as close to the deadline as possible, especially if you want to use your supervisor/ tutor/ other lecturer from your MSc as a reference. The longer you wait to submit the application, the better they will get to know you and hence be able to write a better reference.
However, don't wait until the last minute to ask for a reference! Ask well ahead of time (>1 month) and let them know when the application deadline is so they have time to prepare.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Website management for group seminar
I am planning to start running a group seminar, with talks scheduled regularly, at my institute.
The seminars might be of interest for people in neighbouring areas, too, so I would like to have an "archive" website with all the abstracts and a calendar of the upcoming seminars, and of course I'd like to send out e-mail notifications (and optionally also a RSS feed/calendar widget for the more tech-savvy users).
Is there any software or service that can help me automate some of this setup? I thought of opening a blog-type site on some hosting site, probably either Wordpress or github/Jekyll.
Do you have experience working with similar tools? Do you think they would really save me some time? Or maybe is it better if I just add a page to my academic website, send the mails manually and forget about the other fancy addons?

Having once been the guy making a site for a lab only to have it abandoned because I made it too complicated, I would strongly recommend that, whatever you do, you make it simple to maintain; unless you're the lab PI, the site you build will likely outlast you.
Wordpress is very easy to use, with lots of built-in functionality, freely available themes, and tons of tutorials online describing how to use it. Other CMS packages have similar benefits. Unless it's strictly necessary, I would avoid "rolling your own" software; almost all lab websites are the same few pages, and you don't need something complex for that.
I agree with @seismail that you should check whether your department will make the page for you or at least agree to host it. It will definitely improve branding.

A site hosted by your university, on your university website, will probably be a better choice than an externally hosted website. This is because you'll be able to immediately identify the seminar series with your university, and that will help to improve its branding. (It also looks a lot more professional!)
As for software, there are a lot of different options. I can't really offer a lot of guidance on this, as we have staff whose job it is to maintain our websites. Which one you pick will depend a lot upon the kinds of features you want, and how steep a learning curve you're willing to negotiate.

You might consider using google groups combined with a google calendar. It can be set up to provide email alerts and a calendar, and you should be able to extract an RSS feed as well.

+1, Google Apps are the easiest solution for this setup. You have a Calendar, Group Discussion, and also Drive where you can upload abstracts, papers, whatever you like. Drawback is, every participant needs a Google Account.

Another option could perhaps be using Google Sites (https://sites.google.com) rather than Google Groups (and again use integration with the Google Calendar).

I have always hosted reading groups and seminars using a page within a wiki. There are a bunch of firms that will provide you with a wiki for free or for a small price and many that specialize in doing it for Academia (e.g., PBWiki and WikiSpaces and I'm sure there are many others).
You might have to send out your own email announcements but that burden is pretty minor.


text:
Are PhD scholarships and assistantships taxable?
I'm currently in the process of applying for various scholarships to fund my PhD.
Many scholarships mention explicitly the approximate amount per year of the scholarship. This is usually the amount excluding tuitions, so the amount mentioned is intended to be a stipend to cover living expenses.
I've never seen it mentioned anywhere, nor could I find a definite answer of this online: will this stipend be taxed? E.g., should I subtract a certain percentage off the scholarship amount that's mentioned, to calculate my real monthly income? Does this depend on the country the scholarship is given in, or are there international agreements on this?
I'm in the initial stages of setting up my PhD programme, therefore I'm in contact with professors in New Zealand, Australia, the United States and Canada, so ideally, answers to my question apply to any of these countries. If it's relevant: I'm a dual citizen (European/American Citizen).

Depends on country, undoubtedly.

have you found out what is the tax percentage in New Zealand? I cannot find a way to send you a private message

Helpful discussion at http://sbseminar.wordpress.com/2010/03/23/wanted-page-of-tax-info-for-nsf-fellows/

@user4050: No, I had to postpone the start of my PhD till next year, so I stopped exploring all this some time ago. But next year I'll probably have to dig into it again, so if you find the answer....please share :)

If you are a US citizen, you have to file taxes every year, regardless of where you live. Whether you owe money depends on tax treaties between the US and your country of residence.
If you are a US citizen: GET AN ACCOUNTANT

Taxation does depend very much on the country, as well as the type of award you receive. The amounts similarly vary from nation to nation. To give two examples:

In the US, scholarships and fellowship stipends are taxed as ordinary income. However, in some cases, for externally funded awards, you need to be careful, as the university might not withhold taxes. In that case, however, you would need to make estimated tax payments, as you are still responsible for paying the tax "on time!"
Don't forget that your tax burden may also include state and local income taxes, depending on where you live. Making matters even more complicated, cost of living fluctuates wildly: you're probably better off with a $25,000 award in the midwest than a $35,000 or $40,000 award in New York City.
In other countries, the system can vary. In Germany, for instance, graduate fellowships are not taxed, while "standard" graduate positions, which are considered employees of the state, are taxed. At the same time, however, people receiving the taxed positions receive health benefits and pay into the social security system. Stipend recipients are responsible for their own health insurance, and do not accrue time in the social security system.

Wow...here I was, half-expecting most scholarships (except a few privately funded ones) were exempt from taxes. Seems I've opened up a stinking tar pit of doom...

My graduate stipend was taxable in the US at the federal level. However, there were exceptions at the state (Pennsylvania) and local level. If I was being paid out of grant money to perform "duties necessary for the completion of my degree", then it was not taxable at the state or local level. Those taxes were not taken out of the paycheck in the first place. If I was being paid to teach more than the two semesters it was required, then I got state and local taxes taken out.

@DaveClarke Although Dave's answer provides useful info, this answer provides more general, top-level information. In any case, this answer makes obvious that it varies so greatly from country to country, that my question has no clear answer. Therefore, I'll accept this answer as the most satisfying one.

The stipend is taxable at the federal, and possibly the state level depending on where you live.  The tuition waiver however is non taxable.  Any grants and loans you may receive are also taxable as income.

@JohnB: It depends on how the tuition waiver is handled. A direct payment to the school would not be taxable. A payment to the student which has to be redirected to the school may be taxable.

@aeismail, while the payment to the student may be taxable it will become a wash when you file your taxes, in the US at least. In the instance of a payment to the student the school will report it as billed tuition on the 1098-t at which point the student will be able to claim that payment was made when the file taxes and it will wash out and tax obligations for the payment provided it does not exceed the amount of tuition and fees billed.

@JohnnyB: It probably comes out as a wash in the end—but students might need to watch out for estimated tax payments.

When I was in grad school (1998-2004), my stipend was subject to US federal and CA state income taxes, but not to payroll taxes (Medicare & Social Security). It's complicated.

@coneslayer: Not really. Income and salary are two different concepts in most tax systems. A stipend is definitely income, but not normally considered a salary, therefore it's not subject to payroll taxes.

@aeismail If your 23-year-old grad students are showing up with an understanding of that distinction in the tax system, they're way ahead of where I was at that point. All I knew is that I was getting a paycheck that looked like every other job I worked, but not all the same taxes were being taken out.

OK—I was filing the federal 1040 plus tax forms in three different states while I was still a teenager, so it was obvious to me. I can't vouch for my colleagues.

I don't know if you're interested in building a single and comprehensive answer, since you're already upvoted, but: in the UK, students do not pay income tax (as they receive a studentship/scholarship) and are exempted from paying local taxes (Council Tax). They have access to the national healthcare system. Students who TA may get access to national insurance and may have to pay tax on their TA revenue. In France, students are treated as normal workers and pay income and local tax, but the level of tax is relatively low for the income of a PhD student. Healthcare and pension are included.

Adding a data point for France: it is taxed, and it is in the lowest income bracket.

In Belgium, we have two kinds of ways of paying PhD students. One is a bursary, which is untaxed. The other is a salary, which is taxed. The amount the student gets in the hand is roughly the same, though there are factors such as amount of experience, whether there's a family and/or children, etc, that affect the value.
Whether a bursary or salary was offered depends upon where the funding comes from. In practice, the tasks of the students in each case are the same. No additional money is provided for tutoring, though it is expected that students help out with tutoring and other activities.
Tuition fees (less that 1000 euro per year) are not covered by the scholarship. Money for books is not provided, though in our department students can order books for the library and keep them on their desk for as long as they want to. Sufficient money for conference attendance is generally available, independent of the scholarship, as far as the student is concerned (which means, managed collectively by the supervisor).
Belgian PhD students earn a comparatively good amount of money, I think almost the highest among PhD students in Europe. (I can't find a reference for that at the moment.)

Would the salary come out of tutoring (or similar), so "non-PhD related labour"? Or is that the part of the scholarship that's left over after tuition fees, books, expenses for conferences, etc. (PhD related labour)? Or both? :)

I've updated my answer. Let me know if it doesn't answer your questions. And note that it applies only to Belgium, but perhaps only to Flanders or even just KU Leuven or our department.

I agree with most of what has been said. To add up to it, in UCLouvain grad students are typically either salaried teaching assistants (which takes around 1/3 of total time or less) or untaxed bursaries who are forbidden from teaching or any other work, at least for a FNRS/FRIA scholarship.

When thinking about taxation you need to consider how tuition and fees are handled. Under some circumstances, you could be responsible for paying tax on the money used to pay your tuition.
You also need to think about minimum earnings. If your only income is your scholarship, the tax burden will be small. If you have additional income, then the additional tax burden from the scholarship could be very large.

So in other words, before starting to work for extra income, I should make a few calls and check if the "extra income" will result in a net loss of income?

@RodyOldenhuis yes, but in most countries negative income cannot happen. What I mean is that the increase in tax liability associated with a scholarship (or extra work) can be very different between two people. The tax liability of someone with no other income might increase by less than 10% of the value of the fellowship, while the tax liability for someone with a lot of other income might increase by 50% of the value of the fellowship.

Is this a theoretical or real-world issue?  I don't think I've heard of anyone being taxed on tuition coverage, but I'm certainly not an expert.

@AnonymousMathematician real-world (at least to the extent that the US is real-world): See example 2 at http://www.irs.gov/publications/p970/ch01.html#en_US_2011_publink1000178003. I have been told that in the humanities example 1 also is relevant since some scholarships are all "teaching income".

I think it's important to point out that an important criterion for taxing tuition benefits is that you need to get the money. If the scholarship pays the money for tuition directly to your university (as many do), then you're not normally responsible for the taxes on it (unless you get a form declaring it as income).

I am in a similar situation. Based on my research on departments in Canada, Australia, and the US,  as already been mentioned the stipend is always taxed. Sometimes though you are in a lower tax bracket.
EDIT: I found this information about the UK
Income Tax
58.  Payments  made  as  part  of  a  NERC  studentship  are  not  regarded  as  income  for  income  tax  purposes.
Students should note, however, that earnings received during the final year from sources such as teaching
and demonstrating should be aggregated with income from post-award employment when assessing income
tax liability for the tax year in which the award ends.

I think your UK research is mistaken. Training awards such as PhD stipends are exempt from tax.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What does a research proposal for PhD applications consist of?
Am presently trying to get PhD admission in somewhat reputed universities in India. However most of their application ask for a research proposal about which I have no idea. What does that mean and how should I write one in about 500 words?
Any guidance will be very helpful! Thanks in advance!

Welcome to Academia.SE. You may want to take a look in this close related question. In addition, where the restriction of 500 words comes from?

"research proposal" aka the question you want to investigate / research .,..

@TheDoctor The application limits the number of words in research proposal.

Propose a research topic, maybe using a hypothesis (it depends on you subject area): clearly identify what you want to find out.
Show why this a relevant piece of research for the rest of us.
Explain how you would attack the topic, what is your intended approach.
Show related research where others, if possible well known researchers, have done their research.

During your research, all these elements could change, however, if you lack of an idea before, it will be hard.


text:
Is copying methods from a paper from the same group plagiarism?
I just wanted to see what people think about a possible plagiarism issue. I found a new article which, as far as I can tell, uses data first shown in an older publication from the same group but with a different first author. This may be irrelevant but I thought I'd mention it anyways; this new article's only novelty is to analyze the old data with a model developed in another group.
To the point: this new article has many (more than 8) sentences taken from the older article from the same research group. These sentences describe the experimental setup. It's clearly the same setup as the older article.
Is this plagiarism? I've been taught that it is but I've talked to another academic who says it's not since there are only a limited number of ways that methods can be explained.

Do they cite the old paper where these methods were first discussed?

Yes, they cite that old paper.

Suggesting quotation marks is a job for some people.

Experimental and methodology sections are increasingly being recognized as "exceptions" from plagiarism, for reasons of practical necessity. For instance, I use the same basic technique to study different problems in engineering and material science. One of the sentences in my methodology section is:

We use the SHAKE algorithm to constrain the bonds in water molecules [citation].

There are only so many ways to write this statement, and at the same time, I can't ignore this information, because it would imply a change in method. Is it plagiarism for me to reuse the same wording as in an earlier paper? If it is, I'm in big trouble. Similarly, if I were an experimental group, how many ways are there to say:

We used chemical X from vendor Y as received.

For this reason—because often methodologies are very similar to previous methodologies within a research group–many journals now do not consider such so-called "self-plagiarism" of the methodology section to be a problem.

If a methodology is common to many papers, then reusing a description of it (with citations to previous use) is not plagiarism: the ideas are manifestly not original, and the language is viewed by those in the field as being routine rather than distinctive.  Still, I think one should keep verbatim copying below a certain threshold....

....I don't read papers with methodology sections, so let me explain it in terms of my own field.  If you are quoting a theorem, no one would ever say "Hey, your statement of it is exactly the same as X's; why didn't you change it?"  That would sound nuts.  On the other hand, if I state a sequence of theorems in a manner identical to a previous paper, then after a certain point one wonders why the reader is not just referred to the previous paper.

I was thinking along the same lines as @aiesmail . Especially in the biomedical sciences one lab may produce a series of methodologically near identical papers where I'd consider it highly desirable to not try to gratuitously vary the description. It's hard enough to follow the exact steps, and any variation might induce the question whether the description means a variation to the process. In the literature I see a lot of hand-wringing about the "problem" of how to avoid self-plagiarism, while offering consistent, standardized methods, and I think that's a pendulum swinging too far.

@DSVA: I didn't say that there has to be a citation; my comment was of the form "under these conditions there is no plagiarism".  That does not imply "under other conditions, there is".  I am working in a very different academic field from you (no experiments!) so in many cases I simply don't have an informed opinion.  I'm curious though: how long are the repetitious parts of your papers?   Under a page?  Over a page?

Plagiarism is presented to students as a binary concept: either you're doing it or you're not, and you'd better not, because if you do it there are clearly defined repercussions meted out by a clearly defined group of people, the academic honesty personnel at the university.
In post-student academia, the truth is that plagiarism functions more as a continuum.  Instead of one set of rules that all the world's academic members agree upon, there are less precise (though no less deeply held) cultural norms that most of the world's academic members mostly agree upon.  There is also no good analogue in the "real" academic world of "academic honesty personnel": instead, academia is largely self-policing.
So in your case, instead of asking "Is it plagiarism?" you should be asking "Is this an academic best practice?  If not, how bad is it?  Is it actionably bad?"
In the case at hand:

[T]his new article has many (more than 8) sentences taken from the older article from the same research group. These sentences describe the experimental setup. It's clearly the same setup as the older article.

My view on this is: it's far from being bad enough to merit any outside corrective action.  Because the word "plagiarism" sounds very serious and we in academia have a stake in keeping it that way, I would not use that word in this instance.  (And if you do, your colleagues / superiors / affiliates may think you're overreacting.)
Three key points:

Plagiarism is defined as using the ideas and/or distinctive language of someone else.  I gather from your description that the two papers were written by the same group.  (You say "with a different first author".  I don't see the relevance of that -- all the authors of the paper are all the authors of the paper when it comes to issues of academic integrity and citation.)  Recycling your own work is a different academic crime, often called "self-plagiarism," but I discourage people from using the term "plagiarism" to describe it: it gives the wrong idea.
You say that they cite their older paper.  That's a key point in their favor -- it means they are not trying to fraudulently pass off old work as being new.  One can even argue that the paper went through the refereeing process and the referees and the editors apparently had no issue with the repetition.  (That doesn't necessarily make it okay, but it provides some useful perspective.)
The essential currency of academic work is intellectual novelty, but that does not mean that everything that appears in an academic work is or should be intellectually novel.  There is a certain amount of routine, procedural stuff that needs to be there, but that most expert readers will quickly pass their eyes over.

I hope you notice that at no point did I claim that the authors have followed best practices here.  I do think it's lazy to lift multiple sentences and whole paragraphs from a previous work.  In my opinion, even if something is completely routine and what you write is going to read the same way as what many other people have read before: okay, then you can type up a new, uncopied version quickly and easily.  Also, as a reader, when you catch someone copying anything, your opinion of them and the novelty of their work goes down a bit.
I will end with this: I remember once, some years ago, reading a paper of a certain student of one Professor A and having the feeling that the paper was morally similar to a paper of Professor A's that I had glanced at before.  When I compared the two papers side by side, I found that not only was the intellectual content closely analogous, but the student had evidently written the introduction to his paper by starting with the introduction to Professor A's paper and making close to the minimum possible amount of change necessary for the same text to introduce his own paper.  I did not seriously consider pursuing a plagiarism charge on the student...but it left me with a negative impression of his creativity, independence and work ethic.  From what you say about the two papers, the main issue is not that some routine procedural passages were copied, but really that the second paper is rather derivative on the first -- not fraudulently so, but in a way that makes the second paper not that interesting to you.  That you think this about the authors' work is, I think, the most appropriate negative consequence of their actions.

I would argue that in some cases citing the older paper can actually be less 'ethical'. If you consider the situation where the authors use essentially the same experimental technique in two different papers and describe it with the same wording, but that the technique itself is not  novel and they were not the first to do it, then citing their previous paper could seem like an unjustified/frivilous self citation. Of course it is always possible to reword sentences to satisfy some arbitrary rules about 'self citation' (or copyright), but in many cases this serves no sensible purpose.

@Bobgom: I agree that a more primary citation is to be preferred.  In terms of reusing common, routine text: some positive amount of this is certainly okay.  On the other hand if, say, three pages of text were copied verbatim from the previous paper, that would not be okay -- no academic paper should contain this amount of derivative content -- and doing this without a self-citation would make it much worse, since then the derivative-ness is being hidden from the referees and readers.

Does the new article cite the older article, or otherwise address the older article? If not, I would say it's almost certainly plagiarism. If the author's of the new paper did, in fact, quote the older paper, and not give it an in text citation, they could be considered plagiarists, but are more likely just sloppy writers.
I think that the 'other academic', you referenced, is probably incorrect. Their statement is true to some degree, but I cannot think of a single instance, except for a definition or law, where there is only a single way to phrase a statement like you described.
If you don't think my answer was clear enough, I'd recommend running the paper through a plagiarism checker. Many schools do this, and they are often the final word in discussions like this.

There may not be a single unique way to say something, but if I use the same methodology in forty different papers, do I need to find forty different ways to say it?

@aeismail Plagarisim is copying someone else's work without due acknowledgement.  Even if you copied yourself, you would have a defense against plagiarism because it would be yourself.  If you really want to make it clear it is work done based on prior work, you can always reference yourself.

Every journal has their own rules regarding this scenario. Most commonly, if the method is identical to one published and you can be sure that no changes at all were made, you would be asked to make an very brief description of the method (few sentences) and cite the published paper. Other editors may ask you to rephrase the methods section, which should not be too difficult. I would still cite the original publication.
In short, ask the editor what their policy is.