text:
Why are papers without code but with results accepted?
I just started reading some papers (Computer Science, specifically Computer Vision) and thought "now let's look at the source code" and was quite astonished that most of the papers don't have any source code implementing the described methods to look at, while claiming some performance or being better than other papers.
How do these papers get accepted in journals / conferences? Do people have to submit their source code privately to the reviewers at least, so that they can reproduce the experiment if possible.
Do most journals / conferences just "trust" that people who submit the paper really implemented the theory and got those exact results?
I always had this idea that any experiment should be reproducible by others else it's not scientific justified. Been wondering about this the last few days.

Folks in my research area do not really care so much about the actual code used to obtain the results; the concepts behind a coded implementation are more important. It is assumed that anybody with knowledge in the field can produce code in a language of their choosing so long as the concepts (algorithm descriptions, etc.) are explained very explicitly and clearly.

Related: How do Academic Journals protect against empirical results given by bugs?

Note that the problem is not specific to computer code, but follows analogously to labwork as well. Even if the procedures/protocols are described in detail, there is no way of knowing whether or not the person that did the work followed the protocols meticulously.

This is really too vague to allow for a possible answer. What kind of "code" are you talking about?

"Do most journals / conferences just "trust" that people who submit the paper really implemented the theory and got those exact results?" This is my impression and I don't like it at all, specially when the funding to produce that implementation comes from public money. If the funding is public, the code should be public. If you want to publish a paper about a system, the code should be published. Very unfortunately this is rarely the case.

BTW: see also this related answer: http://academia.stackexchange.com/questions/13726/should-i-supply-code-as-supplemental-material/13730#13730

@user11192 Hmm, well in practice that may sometimes be problematic. Especially as time is always short. Here is an example http://iopscience.iop.org/1367-2630/7/1/034 :)

@posdef: Labwork is not analogous. You can easily put code available online for anyone to download, read, test, and use. You cannot do the same with labwork. This is why we have to trust your word on your labwork, but there is no need to trust your word on your code.

@JukkaSuomela I see your point, but one could videotape yourself doing the labwork and put it online. I am not saying that it makes sense, merely saying that it's possible.

@JukkaSuomela The point is that the whole academic publication system is based on (relatively) short summaries and a large amount of trust. Lab work is a little more opaque but researchers typically don't publish raw data or logbooks either. With electronic publishing, supplementary material, etc. it could be different but the analogy is still valid.

I'm not aware of any sciences where pre-publication review involves reproducing the experiments - experiments should be reproducible in a reasonable way, but it's not expected for the reviewers to actually do it before publication, that's for other research groups to do afterwards. Still, what is needed is details on measurement process and the test set - so that I can get comparable results from clean-room reimplementations of your methodology. And that is the only way to call the experiment reproducable - using your code isn't, as it may do things a bit differently than described.

Re-running the program wouldn't be a proper verification anyway.  If the implementation is buggy, re-running it still gives wrong results.  Re-implementing is a proper verification.  Reproducibility means that it's explained clearly enough to allow reimplementation. A clear description is much more helpful for reproducing results than code. Code is meant for computers to run, not for people to read. (Remember: in the vast majority of cases this code is implemented by a single person and it's not meant to be maintained, this is not "software engineering"). The paper is meant for people to read.

I'm a geophysicist who writes a lot of modeling code. But the code itself is frankly irrelevant. It's the idea behind the code that matters. On the other hand, if you want your model (not code) to be accepted, you'd better show verification that it worked against something known or previously verified. No reviewer will let a modeling paper go without some verification step.

@posdef There's actually a journal devoted to 'video yourself doing lab work' - JoVE. http://www.jove.com

@Szabolcs I'm with you - in my field, "reproducibility" in its proper form means being able to conduct a comparable study on an entirely different population. "I can click run and get the same answer" is a low and base form of reproducibility. And that ignores the prospect of data I cannot share with you that the code requires to run.

@Trylks, I agree with you 100%, if it was up to me... no code, no public funds... for anything CS. I also strongly believe many students take that publicly funded secret research, and jump directly in the private market with their 'goods'.  I've got no problem with somone commercializing public work, that's why we fund it to begin with... but I'm not ok with the publicly funded head start provided by code.  I don't care if it's usable, or practical... or completely useless. we bought it, it should be up to us to decide if its of value.

It all depends how provable the results are. If they are mathematical theorems, what is required is mathematical proof, not running code (unless it's code for a theorem prover).

@Szabolcs "Reproducibility means that it's explained clearly enough to allow reimplementation". The best explanation is the code itself, the code should be meant to be read. If you bet on quantity (two explanations, one in English, one in code) then the quality is very likely to fall short due to tight deadlines. If you want to prioritize one quality explanation it should be code. In theory, theory and practice are the same, but in practice they are very different. Disregarding practice leads to a confinement in the "ivory tower".

@Trylks But what if you are genuinely interested in the theory and that's what you are good at? You imply that “ivory tower” is bad and practice is good but should every computer scientist in the world devote time and resources to producing production-grade software or spinning off businesses?

@Relaxed I'd say the work of a computer scientist is to produce software. I've seen papers that propose something in theory and the same author struggling to get an working implementation (and I haven't seen the implementation yet). What would we say if this happened in other areas? Think about a person that claims to have cured cancer (in theory) but when trying to take that to practice it turns out that empirically it doesn't work (and kills people with great pain). It's not rare to see researchers that cannot "produce" their own research, "reproducibility" should be stressed and enforced.

@Trylks I think your analogy is very telling. There is plenty of research in cancer biology that's not aimed at an immediate cure of anything. Even narrowly defined medical research is not limited to clinical trials of ready-to-test remedies and different people do different kinds of research (i.e. some people specialize in clinical trials, others in various sub-areas of biology). If it's empirical, it should be as reproducible as possible but not everything is or should be about practical applications (medical or otherwise) and I don't think CS is only about producing software.

They have to put it more tentatively than “we cured cancer” but people are indeed publishing papers all the time without going all the way to a clinical trial of an actual molecule (the implementation if you will).

@Relaxed I would summarize it in: "code or it didn't happen".

@Trylks That much I guessed ;-) But that's not how research works (or should work, IMO).

"Re-running the program wouldn't be a proper verification anyway. If the implementation is buggy, re-running it still gives wrong results" If the implementation is "buggy" then the results may be based on a bug therefore the results shouldn't be trusted.

For me it seems that the reasons are two:

the belief that code is only a tool, a particular implementation being secondary to the idea or algorithm,
the historical residue (it was unpractical to print a lot of pages; especially as no-one could copy-paste it).

Additionally:

many scientist seems to be afraid to show their code in public, as they are aware of its poor quality (see also Why do many talented scientists write horrible software?), so they don't want to risk reputation (for both quality and catching possible errors).

Moreover, things related to current incentives in academia (where publications, not code, are related to one's career possibilities):

sharing code may mean risk of being scooped (instead of milking the same code for years),
cleaning up code takes time, which can be used for writing another publications.

Do people have to submit their code privately to the reviewers at least, so that they can reproduce the experiment if possible.

Typically - not. If the code is not made public, almost for sure no reviewer have checked its existence; much less - correctness.
However, many scientists are starting to notice the problem (and they see how open source culture flourishes). So, there are new initiatives addressing such issue, like Science Code Manifesto:

Software is a cornerstone of science. Without software, twenty-first century science would be impossible. Without better software, science cannot progress.

Or e.g. this manifesto. Try to search for reproducible research or look at things such as knitr for R and this intro to IPython Notebook, or read about using GitHub for science. And it seems it is taking off.

+1. For me, particularly the time consumed for cleaning up is a major point. Even though I already do most analyses as knitr .Rnw files. But there still is a major difference between basically a script with few headlines and something that can go to the supplementary material.

@Zindarod They don't, unfortunately.

@Zindarod: They rewrite it.  If they get the same results, that's evidence that both of them had no (or identical) buts.  If the source was provided, a verification that the source provided the results would be easy, but that does not show that the theory produces those results, because there could be a bug in the code.

@Zindarod this seems like an interesting new question.

The code cleanup can be literally intractable too. At my undergrad, the CV group had an internal library that was at least several tens of thousand lines that severely needed cleaning up. But cleaning it up would have taken months, and steamrolled any untold number of underway projects in the process of refactoring it. Since any publishable research coming from them almost certainly included parts of that library code, well...

@Jsor In any case, dirty code is infinitely better than no code. "Since any publishable research coming from them almost certainly included parts of that library code..." then taking a few month to clean it up may be worth it.

@PiotrMigdal Well, I agree with you on principle, but the library had a fully connected dependency graph and a home-rolled cmake-like build system (yes, it took forever to build anything), so publishing any snippet required also dealing with the hacked together build tools and the fact that releasing any portion entailed releasing EVERY portion so it could even compile.

i think code dumps should be the norm in academia. Purposefully witholding code because there may be bugs and to prevent other people from advancing on your work is understandable if code dumps are not a requirement, but they should be a requirement

I think an issue that is related to that raised by Piotr (+1) which is that research funding is not generally available to cover the costs of producing highly reliable portable code or the costs of maintaining/supporting code produced to "research quality".  I have found this to be a significant issue when trying to use code released by other researchers in my field; all too often I can't get their code to work because it uses some third party library that is no longer available, or that only works on a Windows PC, or which no longer works on my version of the software because it uses some deprecated feature of the language/environment.  The only way to get around this is to re-implement the routines from the third party library so that all of the code is provided as a single monolithic program.  But who has the time to do that in an underfunded "publish or perish" environment?
If society wants high quality code to accompany every paper, then society needs to make funds available so that good software engineers can write it and maintain it.  I agree this would be a good thing, but it doesn't come at zero cost.

What you say actually speaks against the publishing of code, since it's likely to become obsolete or to face portability issues. In my opinion, what matters is the theory behind the implementation and I don't think research funding should go into developing robust, platform-independent, frequently updated software products. That's the software industry's job.

I don't fully agree with that.  The reason we publish papers is so that other researchers take up our ideas and run with them.  A good way of making sure that happens is by making sure that the tools required are available.  Thus there is often a good reason to provide tools that are adequate for that purpose, but that still has a cost that is not currently met.  We don't need to produce production quality code, but we do need funding to produce code of adequate quality (from the perspective of portability and reasonable longevity).

@DikranMarsupial: "so that other researchers take up our ideas and run with them" - that sounds very easy, as if it were enough to publish one's code and that would directly allow other researchers to build on top of it. However, that is linked to many ifs in reality - it only works if the other researchers know the technologies used for the code, if anything else they want to combine with that code is compatible, if the code runs at all on their platform, etc.

@O.R.Mapper you are missing the point, if you provide code it is easier for others to build on your work.  How much easier depends on a number of factors, but if you read my answer then you will find I had raised those same points already!  I also did not say that it was enough to publish the code - it isn't, the code supports the publication, nothing more.  The more you can do to alleviate the "if"s, the better, but at the end of the day you are working to a time & effort budget, so there will always be "if"s left, but that doesn't mean you shouldn't give away research code.

The code does NOT need to meet any code quality standards. As long as the code produces the results as promised by the paper. Let software engineers optimize or rewrite the code.

Because some researchers do not like to think about the real world and reviewers do not want the hassle.
(What's next is a bit of a rant)
I've recently done a survey of a specific type of geometry related algorithms. In all the papers the program was described as working perfectly but once I requested the source code from about a dozen authors things became ugly.
50% of the software was missing important advertised features. For example the software would only work in 2D while the paper showed 3D examples. (Which in my case really makes things a lot more difficult). After inquiring why these features were missing they had usually never been implemented or had been implemented but proved unstable/non-working. To be clear: it was 100% impossible to generate the results shown in the paper with software that was often even improved after the paper was released.
75% of the software wouldn't work perfectly in normal cases. In my case this usually was because the algorithm was designed with 'perfect numbers' in mind but the algorithm was implemented using normal floating point numbers and thus had representation problems which resulted in large errors. Only a few papers mentioned these problems and only two tried to (partially) address them.
85% of the software wouldn't work in scenarios specifically designed to find problem cases. Let's be honest; if a 'mere' student can find a scenario in a few weeks that totally breaks your algorithm you probably already know about this.
Not supplying code makes it possible to lie and to my disgust (I'm new to the academic world) this is done extremely often. My supervisor wasn't even surprised. However testing code is a lot of work so this behavior will probably go unchecked for a while longer.

You seem to think that we should request code, because without code, any crazy result, be it fraud or honest mistake, can be slipped into the journal. But this is not so. Including code is a nice-to-have feature, not a must-have feature. The other answers silently assume this and explain the (good and not-so-good) reasons which lead to the current situation of uncommonly included code. I think I can complement them by explaining why it is not a must-have feature.
For theoretical results, you don't need any empirical tools like code to reproduce them, as others mentioned (e.g. proving that an algorithm has a better big O behavior than another). Of course, there are also empirical results, which cannot be replicated that way.
But your reviewers will have an expectation of what your idea will result in. If the current best performance for wugging zums is 3 zums/s, and you add a minor tweak and report 300 zums/s, your reviewers are supposed to notice that your claim is unusual, and do something (possibly demand to resubmit with the code). This is not foolproof, but with multiple reviewers per paper, it is effective, because the magnitude of most empirical results is predictable once the reviewer sees the idea and understands how it works.
For this class of paper, both honest and dishonest mistakes have a good chance of being caught, with bad results for honest scientists (reputation loss, especially if caught after publication) and worse results for dishonest scientists (end of career if proven!). Moreover, the graver the mistake (as measured in the size of error), the higher the chance of being caught. It is less likely that you will get caught if your algorithm manages 4 zums/s and you report 5 zums/second, than if you report 300 zums/s. So, scientists are disincentivized from submitting incorrect papers, leaving less incorrect ones in the submitted pool, and the reviewers catch lots of the remainder.
There are cases where it is totally unknown why an observation is the way it is, and in these cases, it is very important to describe the exact test setup perfectly. But I have never seen this kind of paper in computer science, it is associated with natural sciences. So no code there. Even if you got such results in computer science (e.g. you observed that users are capable of reading a 12000 word EULA in less than 30 seconds, which contradicts common reading speed observations, and you have no explanation for it), it is unlikely that including the code you used to obtain the result will be pertinent to replication.
To put it together, among a large mass of computer science papers, the theoretical ones and the natural-phenomenon-observation ones don't need code inclusion for replication, and the remaining ones will contain only a low percentage of incorrect-but-uncaught papers. Aggregated, this leads to an acceptably low level of incorrect papers being submitted. Requesting the code to go with them will increase quality for one class of paper, but it will be an increase of an already high quality level. It is not that not having this feature makes the current quality too low.

Very good point about the difference between nice-to-have and must-have features. I recall some paper (possibly an editorial?; Unfortunately right now I cannot find it) discussing a similar problematic for standards of reporting statistical analyses. The bottomline was that basically a check list for good practice is known, and it was suggested that everyone who knows and obeys it  includes a single sentence stating this (or a more minute description). The idea is that this nice-to-have strategy will push quality without the need of consensus about must-have policies.

+1 for essentially casting this in signal detection terms. Nice!

As an associate editor of a journal (bridging statistics and psychology), I requested the authors to submit the code when they proposed new algorithms and procedures, and then sent the code to the experts in the statistical package to check that (a) the code does what the paper describes, and (b, secondary) that it is a good code (robust to bad inputs, computationally effective, etc.). I was also asked to review some papers for Stata Journal whose focus is the code, and did the same. There were times when (a) failed, so I had to return the paper and say that the authors had to align the methodology and the code. There were times when (b) would fail, and in case of Stata Journal, this would also mean returning the paper. There were times when the code wouldn't come.
Most of the time, I would be happy to share my code, but it is complicated enough (with internal meta-data-based checks, customized output, etc.) that a researcher less proficient with the packages I use won't be able to edit it to make it work on their computer.
Going back to your main question -- reviewers are lazy pressed for time, and have their own research to push to their journals, so few of them go into the effort of fully verifying the results. This is just how the world is. May be these full professors could request the code and give it to their graduate students to play with, break and debug, as this would be a good educational opportunity for the latter. But again this does not happen very often, as the confidentiality clauses for accepting the reviewer role usually preclude one from sharing the paper with anybody else.

I'd like to add a slightly different point of view from an experimental field (chemistry/ spectroscopy/ chemometric data analysis).
Here, the study starts in the lab (or maybe in the field), with an old-fashioned type of notebook. Old-fashioned typically still being paper + pen. Data analysis is often done with GUI programs in an interactive fashion. Records are kept just like in the lab: paper + pen. Maybe with some saved and/or printed figures. As already the part in the lab was recorded this way, not having a log-file or even a script of the data analysis is not seen as a problem. Anyways, asking for the code to be published is only one part of what you need to re-run the analysis: you'd also need the actual data.
Already the suggestion to type in the data analysis and at least save either a log of the matlab/R session or type it in script form is still kind of new (though people love the knitr generated reports I produce...). But IMHO things are moving quite fast now. I'd say that with tools like git and knitr the largest practical obstacles are almost solved at least for the type of person that prefers code over clicking. However, it is not that all already works smoothly (consider large binary raw data and git, and I frankly admit that I have no idea how to practically set up a "real" database server in an efficient way that it keeps track of changes). This is from my perspective as a scientist that just needs tools for reproducibility as a user - and thus I understand my non-programming colleagues who nevertheless need to analyse their data: they just do not have (or know of) the tools that would enable them to log their analyes with reasonable effort.
The traditional estimation of where the big difficulties lurk also focuses on the lab part. I think many researchers are just not aware of the reproducibility issues with the calculations/data analysis. To be honest, I usually share that point of view: IMHO in biospectroscopy one of the big important problems is the far too low number of individuals in the studies.
If you have only 4 mice in the study,  the precise handling of the data will cannot affect the practical conclusions too much. There is a gray zone where not doing a proper validation may affect the conclusions, but again: everyone I know who does the validation according to the best-known practice does spell this out very explicitly - so again (and accepting some risk of falsely discarding few papers as "probably not reproducible") I tend to think that the practical conclusions are hardly affected.

On the other hand, looking at the requirements that e.g. the  Chemmical Communications put up if a new chemical substance is to be published I don't see why there cannot be computer science journals that require the code in a similar fashion.
Like e.g. the Journal of Statistical Software does. (I'm quite sure other such journals exist, but I don't know them.)
To me this falls into a much larger field of reproducibility issues. Of course, if the data analysis cannot be reproduced there's big trouble.

Yet another point: although publications about software are still very rare in my field, I recently had such a paper for review. Unfortunately, the software is proposed to be distributed by contacting one of the authors - which, as an anonymous reviewer I obviously could not do.
Thus, the actual software may be even less accessible for reviewers than for normal readers!

For your last paragraph: surely you can ask the editor to ask the authors for the code?

@NateEldredge: sure. I merely put it as example that attaching the code is still really far from arriving in the defaults-things-to-do when submitting a manuscript if it is forgotten even for a manuscript that explicitly deals with releasing that code.

I would think that most readers/reviewers would find a sufficiently detailed algorithm enough. You write your paper showing, oh, C++ code, and I use SPSS in my shop -- your code is useless to me. Not that most readers would enjoy reimplementing the code (especially for non-CS papers), but with a specific code that runs on a specific platform, there's bound to be a lot of clutter to wade through. An algorithm reduces it to its bare essentials.
If my paper is showing the improvement in speed of my new Quicksort method over the standard Bubble Sort, showing algorithms for the two methods would make it easier to support my claims for O (n log n) vs O (n^2) speedup. If my paper is on population age distributions in wealthy vs developing economies, unless there's a really neat trick I used to process the data, most readers probably wouldn't even care about the algorithm, except in very broad brushstrokes.
It's going to depend on the subject area (general, say, Computer Science) and the specific subarea (say, sorting methods), as well as how heavy an impact the algorithm used has on the results, as to whether the algorithm is necessary. If I'm showing compiler differences in Fortran code, then it would be good to include actual code. Otherwise, code itself is rarely of interest.

"I would think that most readers/reviewers would find a sufficiently detailed algorithm enough." For an algorithm of sufficient complexity, a description is not enough. "your code is useless to me". Provided it is working code (and maybe even if it isn't) code by definition is a implementation of an algorithm, and therefore useful.

This is a view from Computer Science/Theoretical Computer Science/Mathematics.
Ask yourself: who is the target audience of an academic paper?
It is not end-users. It is reviewers!
Do reviewers want code? Depends on the situation. Sometimes they do. Often they don't.
Think about this: why mathematicians don't provide formal proofs but use informal arguments?
It is costs vs. benefits. Providing a formally verified proof is possible but usually needs too much work, work that authors are not trained for and don't have much experience with. On the other hand, what do authors gain from it? Does it help convince the reviewers about the correctness of the results? No, usually reviewers prefer short informal explanation that allows them to understand and see why the result is true. A formal proof usually will not help much. There are people who don't like computer-assisted proofs which cannot be verified and understood directly by humans.
The same costs vs. benefits thinking applies to programs. If providing code will not help convince the reviewer about the correctness of the paper, then why waste resources (time/money/pages/...) to do so? Do reviewers have time to read codes with thousands of lines to check there is no bug in them?
On the other hand, sometimes the software resulting from the paper is of primary interest. Having the code is helpful in verifying the claims. E.g. you claim you have a faster algorithm for SAT. Then it is helpful to provide the code. In such cases authors provide their code. This is mainly in more experimental parts. We don't care about the correctness of the code but obtaining results better than existing algorithms. In such situations there are typically standard benchmarks to compare algorithms. (See for example SAT competitions.) If there aren't established benchmarks then why publish code? If it is a theoretical result where the asymptotic benefits take place over instances which are too large to test what is the benefit of have the code? More so considering the fact that large code developed by non-professional programmers is highly likely to be buggy? Employing professional software developers to develop quality code is costly (the median annual income for a person with a bachelors in CS is around 100K in the US) (except possibly as graduate students ;) and doesn't typically have any profits afterwards.
But does code need to be included in papers? Of course not! There are better ways to publish code, e.g. having a link in the paper to an online copy (on their website or a public repository like github). Why would one prefer to include a code with thousands of lines inside a document which is supposed to be read by humans?

Possibly the target audience of an academic submission is the reviewer, but the target audience of a paper is definitely the community at large.

@Suresh, obviously papers are written for the readers for the academic community at large. However I think authors pay more attention to what reviewers expect. Of course reviewers are intended to represent the community, but in their role as reviewers they often have priorities which can be a bit different from the community at large.

E.g. they might want more details than other readers to make sure the results are correct, they may need less details because they are experts in the topic, or sometimes they may object to a paper because they think the presentation contradicts their view of the topic, etc. It seems to me we typically consider the community at large much less than the potential reviewers as our audiences when writing papers in practice. Anyway, I don't think the issue has a big effect on what I am trying to say in the answer:

code is published when the target audience wants it from the authors, and end-users who want ready-to-use code are not the target audience of academic papers.

this last point I agree with.

Of course the code shouldn't be in the paper, but instead a link to a repo. Lol can't even imagine looking at software code of 1000+ lines in a single pdf file.

Because in truth most academic researchers in technical fields can't code to save their lives. In computer vision, they just hack something together in a single Matlab file. They like to think that coding is for undergrads. They believe it's beneath them to waste their time on such trivialities. Most of them never learned good software engineering practices and don't have an understanding of the complexity and skill it takes to write good code.
The major problem is that this is supported by researchers because all academia cares about is publishing, quantity over quality. Success is measured by how many citations you have, not by how good your contributions are. At the end of the day, when the only people citing you themselves don't produce anything useful, it doesn't matter. It's not science anymore as much as it is "research for the sake of doing research".

Taken from the book SKIENA, S. The Algorithm Design Manual: Springer. 2008:

After a long search, Chazelle [Cha91] discovered a linear-time
  algorithm for triangulating a simple polygon. This algorithm is
  sufficiently hopeless to implement that it qualifies more as an
  existence proof.
[Cha91] CHAZELLE, Bernard. Triangulating a simple polygon in linear time. Discrete & Computational Geometry, 1991, 6.3: 485-524.

I think that the authors consider costs and benefits and avoid the submission of code when the costs outweigh the benefits.
According to David W. Hogg costs are:

you might get scooped
you can be embarrassed by your ugly code, or by your failure to comment or document
you will have to answer questions (and often stupid or irrelevant ones) about the code, and spend time documenting
you will have to consider pull requests or otherwise maintain the code, possibly far into the future
you might have to beat down bad and incorrect results generated with your code
your reputation could suffer if the code is used wrongly
there could be legal (including military) and license issues

Benefits are:

more science gets done, more papers get written; all measures of impact increase
you get motivation to clean and document the code
results become (more) reproducible and (more) trustworthy
outsiders find bugs or make improvements to your code and deliver pull requests
you get cred and visibility and build community
citation rates might go up
code is preserved
code becomes searchable (including by you!) and backed up
there are good sites for long-term archiving and interface
can establish priority on an idea or prior art on a method

Usually "performance" of code relates to "how well it scales with the size of the problem". This means, in my opinion, that a paper claiming "algorithm A is faster than algorithm B" needs to show timing for both A and B _ for different size problems_. While there may still be inefficiencies (and errors) in the implementations, this will at least demonstrate whether the underlying algorithm is more efficient (lower big-oh).
Where software is they key to the paper (the product, not the tool) I would expect it to be available (github or other). So a paper that says "I can sort in order 1/n by kerfugling the dargibold number" needs to show how that was done; the paper that claims "when I sorted these two data sets, the one from Whoville had bigger flubbits" does not need to show how the code was sorted - they need to focus on explaining what is significant about the flubbits from Whoville.


text:
Path for American to become professor in Europe
I'm an American, and I’m considering – sometime in the future – getting a PhD in computer science. My impression is that Europe is a lovely place, both in general and because of its sane work culture. If I decide I would like to become a professor in Europe, what steps should I take?
For example:

Does one need to get a PhD from Europe, rather than from the U.S., to be a professor there?
If not, does a PhD from the UK qualify one to be a professor in continental Europe?
Are classes in continental Europe generally taught in English?
If not, does one need to be a native speaker to teach them?

Too many questions, you need to edit your post. But generally: 1) No. 2)Yes. 3) Some, especially in western Europe. 4) No.

Please also remember that Europe is not one homogeneous mass and practices vary, often significantly, between individual countries.

I don't know how much saner Europe is than the US when you restrict your sample to academics. The few academics I know in Germany work insane hours and go through their emails every single day - including weekends and holidays.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Online Access of Journals
Recently I got myself enrolled in a PhD programme.
I want to know whether there is any website where one can get registered and get access to various different sites for journals like Jstor, Science Direct etc. I can’t access these sites outside of my university, although my college ID lets me access them from the campus. I am doing this course part-time, so I can’t always be inside the campus.
If I can get a common ID by paying some fee, I am ready for that. But that common login ID should work on different sites. Please let me know if there is any method.

Many uni libraries allow you to do it online. Have you checked?

Yes, I checked it but on the uni libraries all i get is the list of journals. On clicking it opens their website which requires login to access

If I understand you correctly that you want to access journals through your university, then you have to ask your university. Otherwise please [edit] your question to clarify.

@Wrzlprmft actually I asked my university but they said I can access journals using my college ID only from inside the campus and that too when I'm physically in the library or computer lab

Well, then [edit] your question to clarify what you want to ask instead.

I vote to close this question because the described problem is of infrastructural nature and does not belong on academia.SE

I don't know if it is the case at all universities, but at least at mine, journal access is tied to a specific IP range - as long as one connects to the internet within the campus, they can freely access the journals. However, the only way to gain access from elsewhere is through our university's VPN.
You should try asking at your university if they have something like that set up and available for students, but it's possible that they don't.

That's the right way. Greetings from a librarian.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Marking a journal paper as “Minor revisions needed” for minor typos?
I’m doing my first review of a journal paper.
The paper looks good other than a few spelling/grammar typos and one formatting issue where a line has a single letter extend into the center margin between the two columns. These issues are not ones that confuse the content of the work in anyway.
My decision options include accept, minor revisions needed, major revisions needed, and reject. Should I be choosing accept or minor revisions needed? Do these small types warrant a revision for journal papers? Or is it ok to accept these issues?

Unless you are reviewing for a truly lazy editor, it shouldn't matter.  No matter which selection you make, the editor will see from your report that you are asking for extremely minor revisions and act accordingly.
Generally, I recommend minor revisions in such a case, because authors might not be given the chance to correct typos in a straight acceptance, and they might appreciate a second chance to do what they should have before submitting (but almost no one does), which is to find more typos and correct them.  For every one that you found, they will probably find another one.

If authors "might not be given the chance to correct typos in a straight acceptance", then "minor revisions" makes sense, otherwise, "accept" makes sense, as a reviewer you can check which case applies.

"what they should have before submitting (but almost no one does)" - Speaking for myself, I always check for typos several times before submitting, and yet, I continue to find typos in my own papers that have been accepted. Sometimes when you've read something so many times, your brain sees what it expects to see, not what's there. I suspect that's the case for others as well...

@darthbith - I do the same as you, but there are a few people in my field who have a genuine and accurate reputation for producing perfect typo-free papers.  I suspect they may actually sit on their papers for six months before rereading them another dozen times for typos, or something.  In any case, I take this as an indication that we should do a better job of eliminating typos from our papers.  I realize that our time might be more efficiently spent writing more papers, but it is a symptom of our fallen world that we have to choose between doing one thing poorly and another thing poorly.

Most journals provide copyediting service to correct spelling.  If this journal will copyedit accepted papers, then I would recommend the editor accept the paper.  If the journal does not provide copyediting service and requires the authors to provide a "camera ready" version before acceptance, I would recommend "minor revisions" to the editor.

You should recommend to accept the paper. As a reviewer you are asked to judge upon the scientific content of a paper. As long as the typos do not change the paper's meaning or render it ununderstandable, it's the journal's responsibility to deal with them. The format of the printed final version is likely to change anyways, so you couldn't even forsee if those formating issues would still be present.
You can easily write in the report that the list of typos/grammar mistakes and the formatting need to be corrected for before publication in order to make your opinion clear on that point. It will then be the editor's decision on whether to send the paper back to the authors or to rely on the journal's copyediting service.
Minor revisions are meant to be chosen in case the paper in general can be recommended for publication, but there are some flaws that need to be revised. Examples could be

missing factors in a formula, which once inserted do not contradict the statements made,
left-out definitions,
unsufficient explanations of a generally sound method
too strong or too weak statements, which could easily be corrected etc.


text:
Should I apologize to my coauthors for being slow to finish a paper draft?
I am first author on a paper about my research from last summer.  Due to a busy school year and a bout of severe depression, I have taken FAR longer to finish this paper than originally planned.
When I send the draft to my coauthors, should I apologize?  Thank them for their patience?  Promise to be better in the future?
As a very young researcher, my primary concern is seeming professional and reliable to my coauthors.

Maybe thanks them for their patience but apologizing ? Not really, a paper is something you must think carefully about. Taking time is not really a problem.

http://brightside.me/article/stop-saying-sorry-if-you-want-to-say-thank-you-a-seriously-insightful-cartoon-57255/

Related, but not duplicate: http://academia.stackexchange.com/q/17988/22733

The question is interesting in both cases, as a resource for future visitors, but is your primary contact with your coauthors via email or in person?

Don't worry, if they had a problem with your slowness, they would have put pressure on your long ago.

In general, you should.
Especially if you are decidedly the main author of the paper, the delay itself should not be a major issue for the others (unless you have caused a "hard" problem, such as missing a deadline), but you may still have caused some inconvenience. For instance, your co-authors may have allocated some time for the paper that they couldn't work on due to your delay, and they might not have much time left now and have to squeeze it into their schedules again.
Whether the apology as such will be expected depends on the culture you're working in. Different cultures have vastly different stances on what "on time" means and whether it is expected in the first place.

apologizing when an apology is unnecessary makes one look like they have weak character. this is not a good strategy. honestly, it's better in my view to err on the side of maybe not apologizing a couple times when it would be otherwise appropriate to, rather than being too apologetic and going around feeling sorry for yourself/inviting other people to feel sorry for you.

@sig_seg_v: Clearly, this is very culture-specific. Personally, I think apologizing is not about the one who apologizes (such as to bolster a weak character, if I understood you correctly), but about the other person(s). The apology is a way to make them feel respected and let them know the issues that may have been created for them by the delay are acknowledged,  not ignored. I do not see what would be won by not apologizing (especially when it would have been appropriate to), but as I said, this is very culture-dependant.

In addition to the answer's by dan1111 and O.R. Mapper (which basically say something like "an apology is a good idea and will not hurt") I'd like to add the point that it is good working practice to inform co-authors if

you can foresee that you won't be able to work on the paper in the near future (if can say how long, even better) and also if
you couldn't work on the paper in the past (if you think that they expected that you do something) and also indicate when you can pick up the work again.

Related to

my primary concern is seeming professional and reliable to my coauthors.

is the following:
Communicating a schedule and also goals/milestones can be really helpful for a collaboration, but it's even more important to communicate if can't keep up with the schedule and think that rescheduling is necessary.

Important information would be if someone ever complained to you, asked you for a draft, something like that. In this case, you should apologize.
If you did not hear from your coauthors during the last year, I think it would be polite, when you send your draft, to include a short statement acknowledging your delay. Do only say sorry if you mean it and if there is a reason for it. Your coauthors will have been busy with other things as well, so there is no reason to thank them for their patience.

"Your coauthors will have been busy with other things as well, so there is no reason to thank them for their patience." - careful, people can be very busy and very patient at a time. Patiently waiting for something (and possibly even delaying some other tasks for it) does not imply inactivity.


text:
How to understand this schedule of a special issue?
I need to publish a paper before the end of this year and found an announcement with these details:

Submission deadline: 14 July 2022
Final manuscript due: 1 November 2022
Tentative publication date: 20 January 2023

Does that mean that the accepted paper should be in November or after that? I sent an email to the lead editor but without a response.
I got that it is supposed that all final versions send before the first of  November, but when will authors be notified of acceptance?

The schedule seems clear, but I can't understand the sentence "the accepted paper should be in November month" in your question. Please rewrite it with different words. I voted to close as "needs detail" for now, but I am happy to vote to reopen it when your question is clear.

You missed the submission deadline of the 14th of July and are out of luck. On that date, you needed to have sent your manuscript to the journal for review. If the paper would have been accepted, you would have until November 1st to submit the final version. By January 20, the journal would have done the final editing / compilation work and published the issue with your paper.
If the journal special issue has too few submissions, it is possible that they extend the submission deadline. You should check whether this has happened.

14 July: you must submit your manuscript by this date.
1 November: By this date a final decision will have been made for all manuscripts. By implication, you will have received a decision before this date, and if the decision was revise, you need to submit your revision before this date.
20 January: all articles published online, assigned DOIs, etc.

thanks but excuse me, when the accepted paper will be notified ?

Sometime before 1 November.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How much should reviewers care about other things than an article's content?
This is a follow-up question to a previous one asking for the potential impact of the typesetting tool (at the review level) on the chances of acceptance for a given paper.
I'm interested in the general case where reviewers will get a "feeling" of the way the article is presented, be it the writing style, British vs. American English, English from non-native speakers, choice of font, or like in the linked question, typesetting and any other criterion that is not related to the content of the work presented.
To be clear, I'm not asking about the more obvious or well-documented sources of bias, like the sex, name or geographic location of the authors. This can possibly be addressed by double-blind review.
As an example, I'm used to journal submissions in the form of a PDF with the least possible amount of formatting: plain text, typically not justified on both sides, lines numbered, usually double-spaced, figure captions at the end, figures concatenated at the end by the electronic submission system. Where I publish, typesetting is a journal's thing and as an author I'm happy to delegate that hurdle.
I once reviewed conference proceedings where the submission was very informal and couldn't help but having a "feeling" from the looks of the submissions. Authors who used LaTeX, which is unusual in my field, didn't look like they had a lot of experience publishing in this field.
I think that ideally this feeling should be silenced and I tried to do so as much as I could. However, reading comments and answers on this other thread made me wonder if that was a sentiment that others shared. My question then boils down to:
Do reviewers have a duty to try to ignore their biases or is the "gut feeling" an important part of the review?
As a final note, I'm not looking for arguments in favor or against the use of LaTeX in general, or in the specific case I mentioned. I find LaTeX to be very well suited for the applications it's intended for and I simply used that example to illustrate something that is not content but could (and apparently does) induce a bias.

Comments are not for extended discussion; this conversation has been moved to chat.

I think the gut feeling is helpful during the evaluation process - one can probably not fight it anyway, so I think one should just accept that it is there. However, the gut feeling must not hinder a review based on the content of the paper.
So, what should not happen is, e.g.:

Stop reading after a few pages because you don't like the look, typesetting, language, variable names… (you may, however, stop reading if the style hinders your understanding, which is something completely different). Even if you hate Times New Roman or Arial, be professional and keep reading.
Keep worrying about issues as above when it comes to write the comments (with the exception of the language, but I am always very careful about comments on language). Comments should mostly address the content of the paper, especially your recommendation should not be influenced by gut feeling. So once you read everything carefully, form an opinion not based on the look of the paper, but on the content (which you have digested at that point and hence, you should be able to evaluate it without taking the look into account).

One exception is figures: If figures are poorly formatted and don't allow to extract the information, this is a serious issue (too small font, pixel graphics with poor resolution, colors that are indistinguishable for common color blindness…). But also here it is more about "can I extract the information" and not about "do I like the look".
Be extra careful with gut feeling if you review something which is borderline for you field. As can be seen in the above comments, the styles of submission vary greatly between fields (extreme example: Some field have all floats, i.e. tables and figures and such, on extra pages at the end of the documents, other field have submission that basically look like a final article, some fields use double spacing, some don't…).

We're supposed to review papers, not authors.  But, even in fields with blind review, things like formatting can reveal (or just suggest) a lot of information about the author, which we should try to ignore.
If it's a feature of the formatting which is actually going to disappear in publication (like font choice or number of columns) it's genuinely irrelevant: it tells you nothing about what the published article would be like, so any reaction is personal judgement about the author rather than the work.
Formatting decisions that will remain, like some features of figures and tables, are harder. Those, I think, should be treated like any other expository feature: if you don't like it, you make yourself focus on the question of whether it actually makes the paper less clear or harder to read, or is merely unpleasant. In the former case, you hold it against the paper, in the latter you don't.
One thing to keep in mind is that we don't want to reject people from nearby fields with different writing conventions who have something good to say.  Many of the signals your seeing are mostly indications that the person is an outsider to the area, which really shouldn't be held against them. (Note that an outsider is already at a disadvantage, because conventions that matter to the refereeing, like how to organize a paper; how to handle that is, I think, a more difficult question.)

Most people I know use a "fail-fast" strategy when reviewing papers- depending on the size of the conference/journal and the field you may be asked to review many papers, and you might be asked to review multiple times a year. It takes hours (for argument, let's say four hours) to really deeply read and understand a paper. Let's do some math: if I'm asked to do 10 reviews and I spend four hours on each paper that's 40 hours of work, or an entire work week on top of other obligations. That's never going to happen. If I find a good reason to exclude a paper early, I'm going to stop and reject it. If nothing is obviously wrong then I'm going to settle in and read the paper more closely.
Common reasons to exclude a paper are:

Paper's language is so poor that I have to work really hard to just understand what is being said
Paper is not in scope of journal/conference
Paper is incremental
Major technical flaws or incorrect statements

To be clear- nobody has a checklist of reasons that they use to immediately reject papers they don't want to read. These are basic problems that just jump out at reviewers and cause them to not want to read the rest of the paper.
To answer your question- bias can definitely contribute to a feeling of "I'm busy and I just don't want to review this paper." This sounds harsh, but I guarantee you that the odds are very low that someone manages to produce quality technical content but cannot master basic things like typesetting. If you combine that bias with a good fail-fast reason it can absolutely kill your paper before it's really considered. Tracking down all the typos and typesetting errors is tedious, but it's the easiest and simplest part of writing a paper. If they can't get that right, I don't have high hopes for the rest of the content.

Right! It should not be the job of a referee/reviewer to decipher hard-to-read stuff. If nothing else, that'd make it take twice as long, or worse, to figure out what the actual scientific content might be...

Much of this is down to the journal not the authors.  The authors' usual favourite journals will affect what they submit elsewhere.
Taking a journal in my field as an example: the combination of a tight page limit and the submission system means that all authors use the journal's template (LaTeX or word).  The LaTeX template makes both a journal-article-style version and a double-spaced, single column, figures at the end (but still fully justified) version available to reviewers (authors can also download these).  My understanding is that starting from word you get something close to the latter.  In this case, while the journal does some formatting, the authors are expected to provide something quite formatted.  In particular the figures should be publication-ready at the first submission.
An author coming from that background to a conference with a much less formal style may well take the same approach -- I would. And yes, I prefer LaTeX even for writing letters -- for me it's quicker and easier to use, even more so if there's an equation or two, or even a single citation (as I use a master bibtex file).  Other people prefer other tools, and much more of the variation in the aesthetic quality of the work is down to the author than the tools they use.
Many people struggle to hit "informal" without feeling like their submission is sloppy, so will try harder than is necessary, or they go for quite a formal style by default.  Even quite closely related disciplines vary significantly in their levels of formality in style (at an engineering conference I saw suits and ties, but equations made by abusing normal text formatting in the abstract book; an applied physics conference was quite a contrast on both fronts).  The most intersting talks at a conference are often those from the edges of the discipline -- just the people who might be expected to miss the house style a little.
Now as a reviewer, obviously failure to follow simple instructions is hard to ignore, as is general sloppiness (e.g. if figures are expected to be publication-ready and they look like a school project drawn in paint).  It can also be hard to draw conclusions as to the quality of the content if it's presented really badly.  But the specific tool used are irrelevant.

Reviewers should care about matter of text, not aesthetics. Most things you describe belong to the editorial board and editorial staff competences.
That said, I believe that, most of the times, there are some guidelines that a submission must fulfill in order to be passed for review. I find it a basic decency towards reviewers to hand out a properly formatted document. And usually it's not the author who does so.
I would say it's not a matter of pedantry but rather hygiene. It's simply easier and more pleasant to read a document with basic consistent type setting. You can focus better.

"Authors who used LaTeX to write a 5-pages article with 6-7 equations and 10 citations felt like pedantic undergrads with too much time on their hands. Or at least it didn't look like they had a lot of experience publishing in this field."

I don't know, but my personal experience is that it's easier to write in latex anything more than 3 pages, or with at least 1 equation, especially if you have some experience. You do not care about anything else than the matter. You get free bibliography and cross-references. Very easy. And it looks half decent, you get basic type setting for free too. I guess it's just a matter of workflow.
Your sentiment can be turned around very easily. Would you be happy if students handed in assignments written on scrap paper? Probably not. Would you say, with out reading, the content is flawed. Also probably not.

What exactly is the editorial board's competence? Typesetting? I never heard of that.

I should have probably used the term editorial staff. I've made an edit. Though, I'd say that setting the standards including aesthetics, and processes still lies within the editorial board competences. Someone has to execute them, and do the work. Those are often the same people.

What I usually tell to my students is the following.
Let's suppose that you have written a weak paper. And you will write a weak paper one day or another. If, in addition to this, your paper is also full of grammatical errors, badly formatted, with an incoherent structure and poorly drawn figures, your chances of being published will be next to zero. Because not only will you likely annoy the reviewers, but also because the few positive parts of your work will remain hidden.
If, instead, you do your homework and do your best to prepare a well-written and formatted manuscript, the reviewers will be probably more willing to give your manuscript a second chance. Even if your work is weak, its positive sides will be more evident. Of course you will likely have to do a couple of rounds of major revisions, but that's far better than being rejected right from the start.
With a bit of practice, writing a well-formatted paper takes roughly the same time than writing a badly-formatted one: why taking the risk of spoiling months, possibly years, of work?

Everyone in academia has found at some point a paper that is fantastically well written and it turns out to be an incredibly minor contribution to the field. However, so we written that it gets accepted!

My perspective on many of these questions: The goal should be to evaluate the science behind the paper, not the authors of the paper so long as these things do not interfere with the clarity of the paper overall.

British vs. American English

Don't care. The biggest issue for me is that British vs. American English starts me guessing where the paper came from in double-blind review, which I have to avoid.

English from non-native speakers

I only comment on this when it's sufficiently off from native English that it's actively hampering the comprehension of the paper.

Your LaTeX example

I'm actually in the same boat you are - for me, LaTeX or other typesetting tools are very much not the standard in the field. The appearance of that, or some other things that aren't "normal" for the field (for example, reporting regression coefficients rather than effect measures) are indicators that someone is coming in to the field from the outside.
That doesn't necessarily mean I'll judge their paper more harshly, but I may end up suggesting it come back to field standards for reporting results, make sure the results are applicable to readers of the journals, etc. To use a common example, a proof, no matter how correct, is almost useless to a clinical audience.

Do reviewers have a duty to try to ignore their biases or is the "gut
  feeling" an important part of the review?

I think this is a little bit of a false dichotomy. Some of these biases can serve as useful heuristics in evaluating a paper, but yes, you need to be vigilant that "Clearly not native speaker, therefore bad paper" doesn't start creeping into your reviews.

Research is a human activity, and humans always have biases. While we should be aware of our biases and try to minimize their impact, we can never avoid them entirely. Trying to consicously compensate against biases ofter leads to overcompensation or undercompensation.
If you fail to adhere to the social norms in your field, some people will be biased against you. Writing papers that look different is one way to break the norms. If the paper is a clear accept, the biases will probably not matter. The effect will be more significant for borderline papers, which are accepted and rejected based on the gut feeling anyway.

This answer deviates from the typesetting aspects and deals with the more general question in the title.
There were two cases, where I accepted an article for other reasons than its content. The first was a long article, where my impression was that it is correct and difficult, but the question was absurdly artificial, in the sense that it appeared to be invented for the sole reason to get another publication. As the journal was pretty good, for me this was a reason to reject the article. Then I noticed that the article was written by some demi-god, which had for some reason escaped me up to then. A quick literature search showed that he had not that many publications, and all of them had a great impact (not in the bibliometric sense). So my impression changed from "artificial" to "too deep for me", and I accepted.
The second case was an article which was full of errors, but this article was an invited article for a special issue celebrating a person A, and the author B was a good friend and collaborator of A. Usually I would have rejected the article, but out of respect for A I sat down and pointed out the flaws, received a revision with lots of new errors, wrote another report, and finally the article got published.
In both cases I think the deviations from the "pure" procedure were justified.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can American students do research in Europe with only a bachelor's degree?
It's not clear to me, looking over all the various regulations and requirements, how (or even if) it is possible for students holding a four-year or five-year bachelor's degree from the US can work at a university as a part-time or full-time researcher. Is this possible? If so, are there specific programs to help set this up? Does it matter which EU country the institution is located? Is there a specific type of visa involved?
(Effectively, I guess this question could be boiled down to: what is the equivalent EU mechanism for a J-1 visa, when the student will be engaged in research rather than coursework?)

Not US, but Canada (hopefully, close enough). The UBC in Vancouver maintains an exchange program with the MPI-FKF in Germany, where they send some of the BSc candidates for an 8 month internship to Germany. There they do some of the research work (not course work), that a PhD student typically does. I can say from our experience, that we would be very happy to see many of them return for a master's or a PhD, once they acquire their bachelor's.


text:
How to identify A/A* Journal
The academic promotions procedure at my Australian university includes specifically publications in A/A* journals. What defines A/A* and where can I find out which journals are in this category?

http://www.core.edu.au/conference-portal

Any official promotion criteria should be explicitly spelled out by your university.  Only will know the answer to your question.

You can look at CORE if you're in computer science, like a commenter mentioned: conferences, journals.
There's also the ERA outlet ranking lists (unofficial website): conferences, journals, but it is old.
Though you're best asking your institution as to where they are getting the rankings from.

A and A* labels on journals are obsolete.  The current custom is to refer to impact factor.  Consult your head of school.

Impact factor doesn't mean much in some fields, such as mathematics.

Could you add a reference or indicate your experience to add to the credibility of the answer?

@TommiBrander No, I never do that on stack exchange.


text:
Visiting Student for Graduate Studies in Mathematics
All,
I am currently a student at a small liberal arts college which does not offer the courses that would make me more competitive as an applicant to Master's or PhD programs in Mathematics.
Since I am ready to move on, my first thought was to continue as a visiting (non-matriculated) student after graduation at a different university for one or two semesters before applying to programs.
It seems however, that few of the top universities have visiting students programs and most of the ones I've seen are geared toward undergraduates looking to go on exchange.
Does anyone know of programs aimed for people in my situation? Is it common? On NYU courant's site, for instance, they offer the non-degree option, but the major disadvantage is cost of living and lack of campus.
Thanks!

Voting to reopen.  OP does not need a recommendation of a specific program.  See my next comment.

You can take courses as a non-matriculated student at a wide variety of U.S. schools. New York City has a particularly high cost of living. You'll need to live off campus, and you may wish to consider working part time.  While working part time, two challenging courses is often a good fit, although of course this is very individual. If you take courses in a state university in the state where you have residency, the tuition will likely be lower than out-of-state tuition. Do look at a variety of states to compare tuition rates (make sure to consider Texas).

This really duplicates much of the question: http://academia.stackexchange.com/questions/8026/how-can-i-enter-into-the-mathematics-field-at-the-graduate-level-when-i-only-too


text:
Is it possible to submit a MathJobs application before a reference letter arrives?
If I'm awaiting a reference letter, is it possible to send a MathJob application before the reference letter arrives? If I do this, will the employer be able get that reference letter whenever it is uploaded? Or should I not do this because the application would be submitted without the required letter?

If you expect the reference letter to be available soon then you might put off submitting your application through mathjobs.org until the application deadline.

@BrianBorchers I personally wouldn't recommend this.  Many faculty look at applications early, so those applications have a chance to build up some momentum.  Most people I've talked to about this agree it's better to apply early.  Though perhaps this depends some on how the department does things.

Having chaired four search committees in the last 10 years I've had experience of this.  In my experience, it doesn't matter when you apply, since committees typically don't look at applicants much until after the deadline.  In any case, there's a trade-off between having your application in early without a reference letter (which might influence how the application is read) or waiting and having a more complete application.

Yes, this is possible. (At any rate, it was a few years ago -- I assume the software has not changed since then.) Hiring committees will be able to see whatever you have submitted, and once your reference writer submits your letter it will be visible with the rest of your application.

It is still possible now.

Note however that if the committee downloads (or even prints out!) the applications then the late-arriving reference letter may not be seen by committee members.  This is one reason that it is best for committee members to work with applications in the mathjobs system rather than downloading them.


text:
How normal is it to have published as an undergraduate?
I’m currently in the first semester of MSc of psychology in a university in Germany and we got a lecturer from a university abroad who wanted to know what we had published. To his surprise, none of us had any publications. For us at least, not having any publications when entering the graduate program is normal.
I know of a couple of peers who were able to publish their bachelor’s thesis and one person who had several publications even before he started his masters, but at our university those are the exception, rather than the rule.
How is the situation in other countries and disciplines? Do undergraduates generally have any publications?

What field are you in?

I'm doing clinical psychology and he gave a course on general psychology.

What country is your lecturer from? And you are doing your MSc in Germany, but what country did most of you do your BSc in?

Throughout every country and every field that I am familiar with, it is fairly unusual to be part of a publication as an undergraduate.
It is also the case, however, that those who did publish are more likely to go on to graduate school, and that it would be unusual to have a class of any significant size with nobody who had published in it.  I don't know the actual statistics, but let's assume only 1 in 10 undergraduates who go on to grad school have published something as an undergraduate, in which case in a class of 20 students, there is a ~88% chance that at least one person would have published.
Thus, it might well be surprising to the lecturer that nobody had published, even though publishing as an undergraduate is fairly rare.
Note: the numbers that I gave were notional, just in order to illustrate the way likelihood is non-intuitive for a whole class.  As a point of reference on the high extreme, however, consider that 85% of MIT undergraduates participate in on-campus research programs.  A lecturer coming from a high-density research environment like that may well have biased expectations about research frequency.

I agree with @Roland, probably all around Europe is very rare for an undergraduate student to publish, and certainly not alone. 1 in 100 is probably a good guess for a co-authored publication. Note also that historically, at least in my country, undergraduate students and Master's students were frequently excluded from group publications, especially by older professors. Things are now slowly changing.

@Roland Two notes: 1) I am not claiming any justification for these particular numbers; since the OP didn't say anything about class size, I just picked two numbers that balanced, and 2) the number is more consistent with what you are saying than it appears, since only a small fraction (in the US presently ~1/4) go on to graduate school and those are the most likely to have published.

Jakebeal, I don't know the numbers for Germany, but in my country I'd say that 1 in 100 of those who go to graduate school are likely to have published as undergraduates.

Are MSc studies called graduate school now?

@CarstenS Of course; it is one of many degrees offered in various graduate programs. https://en.wikipedia.org/wiki/Graduate_school

I did not know that this is how the term is understood in the US. If you look at the descriptions for different European countries in the Wikipedia article, you may come to the conclusion that it is misleading in this context.

@CarstenS The concept of undergraduate/graduate education is somewhat different between Europe and the US. I've exemplified a bit the difference in this meta question. In some sense, many in Europe, both in industry and academia, consider the BSc degree just as a transitory degree toward the MSc (probably, many would like to go back to the pre-Bologna process era ;-) ).

@MassimoOrtolano, thanks, I am generally aware of that, I was only surprised that "grad school" was not necessarily supposed to result in some kind of doctorate degree (e.g. a PhD). I have a pre-Bologna "Diplom" (and then some), but I am not in academia any more.

@CarstenS An MSc is a graduate degree, i.e., a degree taken by somebody who has already graduated and possesses a bachelor's degree.

Jake, your profile says you're in the US and my impression is that undergrads in the US are significantly more likely to have published than in any other country I'm familiar with. I've worked and studied in several British universities (including Oxford and Cambridge) and I don't recall any faculty I knew doing research with undergrads. There just isn't a culture of doing that, here. Undergrads don't have time to research during term time, and they don't have funding to research outside term time.

@DavidRicherby Perhaps it varies field to field as well? I have interacted with a number of UK undergraduates who have published, and indeed co-authored papers with some.

It depends what field, and also depends on the culture.
Sometimes students only respond to what is available to them. I remember the first article that I wrote as an Architecture Undergrad, we were told that the best articles would have a shot at publication. In this field, obviously research and publication will not be in the minds of young undergrads, therefore, it only became an option because the academic staff had encouraged it.
but it is rare, I would imagine. Especially as even if the students had the motivation, the quality of the work might be lacking.

This quite depends on field - my friends from experimental physics were more likely to get a publication as a member of some team then me doing mathematical physics. (I eventually did get a publication before finishing MSc as a solo author but the results are nothing outstanding or special in the big picture.)
Also some universities (such as the one I attended) provide the opportunity to participate in a students "competition". It goes as this: you get an advisor to oversee you, you get some results and you prepare a poster to present it. Few people are selected (best work/presentation in the respective field) are selected and awarded (they get some extra money, quite a nice amount for the local costs). The results are not published in a journal but I think they are kept somewhere in a database of the university. This system is good because nobody expects the students to produce any significant results - compared to journals but they still get to do some scientific work. But the downside is it is not recognized as a publication.


text:
Can my UK PhD permit me to use the title "Dr.-Ing." in Germany?
I'm moving to Germany in a couple of weeks and I'll be setting up as a freelancer. I'm just curious about my title though.
I got my PhD in the UK in 2015, it states "Doctor of Philosophy" on the certificate, as usual, but my research was in Computer Science, specifically experimental application of Computer Science techniques to Engineering problems (in fact, my work was actually funded by an automobile company based in Germany whom I visited quite regularly).
I'm no longer in academia, but last summer I was briefly at Hannover University, where somebody suggested that I would be eligible for the Dr.-Ing. title instead of the plain Dr. title.
Fact is, I would like the Dr.-Ing. given the technical nature of my freelance work, but I am not sure whether I am legally allowed to use it.
Can anybody advise me?

You probably have to fill out some paperwork with some official place, but it might be possible. The main obstacle is the difference in naming. You having an equivalent degree of education should be rather undisputed.

You may want to look at the Ingenieurgesetz of the state you move to, to see how to hold your title and freelance (which you likely have done allready, just in case)

Yep - If I am not mistaken Dr. and Ing. are actually two protected titles with different regulations behind them...

Fascinating.   I did some high level consulting (non technical) for a German company, even through an agency.  They asked for photocopies of my diplomas.  I found the request very strange (for a temp worker).  But I sent the stuff.

One important thing you need to know about the German Doktorgrad is (alongside other non-academic titles) that its use is restricted by federal law. Adding a Doktortitel to your name in public without being entitled to do so is a punishable offence. This means that you should not trust any answer that you get from the Internet on this topic. Make sure that you verify whatever answer you get with an official side. One way of doing so is pointed out in @Cachapa's useful answer from the point of view of someone who already went through this.
Having said that: First of all, a British Doctor of Philosophy is indeed considered equivalent to a German Doktorgrad. You can verify that using this database (unfortunately only available in German). This means that you're fully qualified for whatever position or task that requires a Dr.
However, while you're entitled to use your PhD in public as an academic title, it is not possible to convert your PhD into a German Doktorgrad, and doing so might be an offense according to German law. This is based on a statement issued by the Kultusministerkonferenz (the "Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany"). These are the crucial points:

Akademische Grade sind in Deutschland rechtlich geschützt.
Der ausländische Grad muss ordnungsgemäß verliehen sein.
Die Umwandlung eines ausländischen Grades in einen deutschen Grad ist nicht möglich (…).

My translation:

Academic titles are protected by the law in Germany.
A foreign title has to have been granted lawfully.
It is not possible to convert a foreign title into a German title.

This statement is very explicit in my opinion, and it means that you cannot just add Dr.-Ing. to your name if you haven't been granted the title by a German university. It also means that no German university can grant you a Dr.-Ing. simply because you have a foreign equivalent title.
Yet, you can use your PhD in public, because it's acknowledged as an academic title. Plus, the Kultusministerkonferenz has decided that EU members can also choose to use Dr. instead of the designation used in their country of origin, this is regulated in (2) in the "Vereinbarung
über begünstigende Regelungen der 'Grundsätze für die Regelung der Führung ausländischer Hochschulgrade'" (thanks to @Pont for pointing that out), but this does not include the "-Ing." part. To quote:

Hochschulgrade aus Mitgliedstaaten der Europäischen Union (EU) oder des Europäischen Wirtschaftsraumes (EWR) […] können in der Originalform ohne Herkunftsbezeichnung geführt werden.

This means that academic titles from the EU, the EEA may be used as part of the name in their original form without stating their origin.

Inhaber von in einem wissenschaftlichen Promotionsverfahren erworbenen Doktorgraden, die in den in Ziff. 1 bezeichneten Staaten oder Institutionen erworben wurden, können anstelle der im Herkunftsland zugelassenen oder nachweislich allgemein üblichen Abkürzung […] wahlweise die Abkürzung „Dr.“ ohne fachlichen Zusatz und ohne Herkunftsbezeichnung führen. […] Führung beider Abkürzungen ist nicht zulässig.

This means that a holder of a doctorate from one of the countries described in (1) may decide to use the abbreviation Dr. instead of the abbreviation that is usually used in their country of origin, but without the addition of an abbreviation indicating the title-granting discipline – that's the part referring to the -Ing. suffix. Also, it's not possible to combine both abbreviations, i.e. "Dr. Schmuddi, PhD" is not allowed.
Edit (March 2021)
The answer above reflects the state of affairs from 2017, a time at which the UK had already voted in favor of Brexit but was still a EU member. Now that the UK is not a member state anymore, paragraphs (1) and (2) don't apply to UK PhD holders. Consequently, the "Vereinbarung" was revised in 2019 by adding the UK to the list of exceptions in paragraph (4) (alongside Australia, Israel, Japan, Canada, and the US):

Inhaber von folgenden Doktorgraden [...] Vereinigtes Königreich: „Doctor of ...“ mit jeweils unterschiedlicher Abkürzung [...] können anstelle der im Herkunftsland zugelassenen oder nachweislich allgemein üblichen Abkürzungen die Abkürzung „Dr.“ jeweils ohne fachlichen Zusatz und Herkunftsbezeichnung führen.

According to this revision, any scholar from the UK who holds a PhD (which is an instance of a „Doctor of ...“ degree) may still use Dr. in their name, but as before, you may not add the degree-granting discipline. So, at least as far as the regulations concerning using an UK PhD in Germany is concerned, Brexit didn't cause any substantial change after all.

Certainly it is possible, in some cases, to be "Dr." in Germany on the basis of a foreign doctorate -- see here: ‘Inhaber von ... Doktorgraden, die in den in Ziff. 1 bezeichneten Staaten oder Institutionen erworben wurden, können ... wahlweise die Abkürzung „Dr.“ ... führen.’ I don't think this conflicts with the "Umwandlung" rule: your doctorate is not transformed into a German one, it is simply recognized as granting you the right to use the "Dr." title.

However: (1) I don't know of any provision of this sort which would allow a "Dr.-Ing." title rather than "Dr." (2) Each Bundesland has its own rules in addition to the KMK ones and (3) UK PhDs are currently recognized because any doctorate from the EU or EEA is automatically recognized; this may of course change after Brexit.

@Pont: Thanks for the comment, I've added that information.

The UK application to leave the EU is irrelevant. First, is it not in force - they started he 2 year negotiation phase but are still in the EU.Second, mostlikely once this is over it may apply to NEW titles, but not to old ones granted while the UK was part of the glorious wonderfull and perfect sarcastic EU ;)

@TomTom: I agree with your first point, but I wouldn't bet on the second. The linked PDF by the Kultusministerkonferenz regulates which current holders of a PhD are officially allowed to use Dr. as a part of their name in Germany. As there is no precedent for a country leaving the EU, it's totally unclear whether UK PhD holders are still automatically allowed the Dr. after Brexit, because this regulation doesn't apply to them anymore. But it certainly doesn't apply to new titles only, otherwise the OP would not be entitled to use Dr. in the first place because it's not a new title.

@TomTom I didn't claim that Brexit currently has an effect: I wrote that "this may change after Brexit". As to grandfathering old titles: the regulation as currently written could be interpreted either way IMO. This is not a case of converting your UK "Dr." to a German "Dr.", as one might exchange a driver licence: it's an agreement to recognize the UK qualification, which can be withdrawn. In practice, even if the UK leaves the EEA, I suspect a new bilateral agreement for recognition will be drawn up -- but as with most aspects of Brexit, we simply can't know at present.

I'm getting a rough feeling of "you can probably use Dr. but be careful" with a very clear sense of "you cannot use Dr.-Ing.". Thanks for your help Schmuddi, Pont, TomTom.

@Matt even if it were legal (which it isn't), calling yourself Dr.-Ing with a British education would look really, really weird to any German who's has contact with the world beyond their borders (i.e. basically every German worth talking to about doctory stuff).

There is no problem if the UK were to leave - think about non-EU citizens obtaining their a PhD/Doctorate in Europe which they will use back home. What may be needed in this case though is something called an Apostille.

@DetlevCM: You are wrong that there is no problem. Currently Germany recognises UK PhDs as equivalent because that is the requirement of EU law. It also recognises various other country's PhDs under different laws but note two things: the UK does not currently appear under those laws, and the absolute requirement of equivalence imposed by EU law is not necessarily reproduced. These are legal issues which will likely be resolved quickly but they still exist and their resolution will depend on the Brexit negotiations.

@JackAidley If EU law ceases to apply, the international agreement with the Apostilles would take over. All that means, would be getting an Apostille for the UK PhD and that's it.

@DetlevCM: An Apostille only validates a PhD - or anything else - is a validly issued document, it does nothing to require that it is recognised as equal to the equivalent qualification in another country.

"Also, it's not possible to combine both abbreviations, i.e. "Dr. Schmuddi, PhD" is not allowed." - since Dr. Dr. ... is used for someone holding two doctorates, wouldn't that actually be allowed for a EU citizen with two PhDs?

Schmuddi already gave a very complete answer above, but I can add my personal experience since I am in the exact same situation: I completed my PhD in Engineering at a British university and am currently residing in Germany.
When you arrive in Germany you will have to do your Anmeldung - register yourself with your city's local authority. You can officially declare your PhD/Dr. title in that process and it will appear in the document you will get from them. I feel that's the safest way to use the title since it's been accepted by the state. As stated above, Germans take their titles very seriously and you do not want to use it incorrectly.
In order to register the title officially you'll need to bring proof of enrolment and completion of the PhD program. In my case I had to ask for a letter from my university with my full name and birthdate confirming my enrolment. For proof of completion I simply brought my diploma. Both documents were in English and were accepted without issue.
Finally, I prefer to sign as "Dr." rather than "PhD" since most Germans aren't familiar with the latter and even many of those who heard of it think they're not equivalent. I've also had a lot of people ask me if I studied Philosophy - consider this if you decide to hang your diploma where others can see it.

Thanks for the answer. I assume therefore, that you don't feel comfortable extending your title to Dr.-Ing. ?

@Matt: Allow me to chime in: Using "Dr.-Ing." instead of "Dr." is not an option. The Kultusministerkonferenz is explicit on that.

Apart from the legal issue mentioned by Schmuddi, "Dr.-Ing." is only recognised in the German-speaking countries, whereas "Dr." is international, this is particularly important in modern fields such as software development. Most of my German colleagues forgo the "-Ing." part in their own signatures.
In any case you typically write your occupation near your title and name, so the fact that your doctorate is in engineering is implied for anyone who cares.

"Germans take their titles very seriously" that has a bad aftertaste. It is not about taking our titles seriously, but about avoiding abuse of the system. I.e., we want to make sure that if someone reaps the benefits of wielding a title (like Dipl.-..., Dr., "Doctor" (as in medical doctor) etc.), then it has been earned.  Compare https://de.wikipedia.org/wiki/Missbrauch_von_Titeln,_Berufsbezeichnungen_und_Abzeichen . Interestingly, it is even illegal to use very similar titles, unless you earned them. Interestingly², the "Professor" (which is more like a job description) is not protected.

@AnoE It shouldn't have a bad aftertaste. Claiming an unearned title is a form of fraud, and undermines the web of trust upon which society is built. When people claim qualifications they do not have, it can erode trust in those who worked for years to earn the privilege to use a particular title, and continue to uphold high standards every day to maintain that privilege,  Some people take that seriously.

@MontyHarder, that is what I said...

@AnoE I think you may have misinterpreted my intention. I fully agree with you and am happy that titles are taken seriously in Germany. The phrase "Germans take their titles very seriously" was meant as a matter of fact and not intended to carry any emotion (positive or negative) behind it.

Yeah, no offence meant or taken, @Cachapa, I guess we're all in the same boat on this topic. It's more of a linguistic issue than one related to titles, I believe, but then I'm obviously not an english native speaker. :)

I had my BA and my MA registered in Baden-Württemberg (each land does it separately). It only means I can put MA after my name, not Dipl. Ing.

Yes, using Dr. when your title is not equivalent is a felony, but in general is not really a career-threatening one: https://de.wikipedia.org/wiki/Andreas_Scheuer#Schule_und_Studium

I would have commented to @Cachapa's answer, but I don't have the reputation...
I'd like to relate my story: I attempted to claim the Dr. title from Finland in the local Einwohnermeldeamt. I was asked for a German translation of the diploma that was already written in English. This probably depends on the state you are in, if not even on the language skills of the officials. In any case, having the diploma translated by a certified translator would have been ridiculously expensive, so I gave up. Therefore I do not dare to include the title at the moment. Perhaps I'll try the Amt again later.
Here are some relevant pieces of information I found:
"Foreign Academic Degrees
Questions regarding the use of foreign academic degrees and titles should be addressed to the ministry of science of the respective Land. As a rule, it is possible to use the academic degree or title in the original language – this is also stipulated in the Länder higher education laws. It is not possible to convert a foreign academic degree or title into a German academic degree or title. The only exception is in the case of ethnic German resettlers. The anabin database provides further information on the recognition of foreign higher education qualifications. " [1]
There is actually a whole database, called "ANABIN", of degrees that are already recognised. [2]
[1] https://www.anerkennung-in-deutschland.de/html/en/academic_recognition.php
[2] http://anabin.kmk.org/no_cache/filter/institutionen.html

My wife went through the same ordeal with a PhD degree from Australia. Getting your diploma translated should not cost you more than 30 Euros, which was about what she had to pay (certified translator accepted by German courts).

Since my degree certificates were in English I had to pay a certified translator to put them into German. Strangely enough, my Irish colleague did not have to get them translated because they were in Latin.

Did you translate the whole thing with supplements and all?

I'm also quite surprised at the "ridiculously expensive" bit - I had to translate my diploma as well (I think it was from French to English), and I think it was about 50 euros tops (maybe even cheaper). Even allowing for wild price differences for the service between the countries, I can't imagine it costing more than double of what I paid (and even that would be surprisingly high - but not sometime I'd consider too ridiculous to get done).

If you belong to a recognised Engineering Institution you may be entitled to apply to FEANI for the prenominals EUR ING in addidion to Dr.

You can apply but not to FEANI (they govern professional engineering registration, but a PhD in engineering is not a PE//CEng//EUR ING), and according to the rule Schmuddi quoted, the body that does have jurisdiction over this will refuse.


text:
Which authority establishes the equivalence of degrees from different European countries?
I hold a degree from a European country, and at that time we didn't have a system similar to the common "Bachelor +  Master". My degree has been considered as equivalent to a Master by the University of another European country, and in this other University I obtained a Doctorate.
Now, I have been offered a teaching position in Russia, and they want a statement of my undergraduate University, saying that that degree is equivalent to a Master. But this University (where I graduated) tells me that they can't make a statement of equivalence, they can only certify that I have that degree.
So, who should be the authority that officially takes care of confirming the equivalence?
To complicate things, I graduated before the Bologna Process came into effect.

I don't understand. If you have a PhD, why would one care if you have a Master degree?

It's a touchy subject. When I got a job in Europe, there was a bit of a hassle explaining why I didn't have a master's degree.

One alternative possibility, although you'd have to consult both of the other parties: would your undergrad university certify that your degree is considered sufficient for admission to a doctorate, and would your destination country consider that acceptable? (This idea comes from observing the admission requirements for a doctorate in Spain, where such a certificate can substitute for a recognised Master's degree).

@PeterTaylor, similarly, would the university that granted the doctorate certify that the undergraduate degree is equivalent to a Master's degree.

After some email exchanges, I can say that the University where I graduated insists that it is not up to them to say to which degree is equivalent the one they grant. They will only provide for details about the degree itself, such as its duration, the exams I passed, etc.
I have also contacted the office in charge of the Lisbon Treaty, and they gave some advice. But I don't have a final answer yet.
What I guess is that the solution is that the non-European University needs to asses and evaluate the degree, and decide.

Each country should have a government office (or equivalent organization) that makes such determinations. You should ask the university to which you're applying for the appropriate office to consult in their country. If you want one from the country where you obtained your undergraduate degree, ask their registrar (or equivalent officer) who is responsible for such certifications.

This is a good answer, but it’s not always governments who enforce these things. In the US there are non-governmental agencies known as regional accreditation agencies on whose authority the degree contains value. The university registrar’s office will know who to contact, be they a government agency or independent group.

This is a bug topic of discussion indeed! I mean: in some countries is the government which gives validity yo degrees, and those are usually countries with "civil law" systems (but not necessarily); on the other hand, in other countries (usually in countries with common law system) the validity is a matter of consensus, or is regulated by local, or independent agencies.

From my experience in European countries, the equivalence is established in the destination country, that is, the country in which you want to use the degree. Basically, the university where you graduated is not habilitated to certify that you have the requirement for a Russian degree. (This would be almost equivalent to delivering Russian degrees.)
Since you have obtained your degree before the Bologna process, you might be able to obtain from the university where you graduated from a statement that your degree is equivalent to a master degree at their university after Bologna. (I know that universities routinely give attestations that one of their master degree grants access to their doctoral programme.) This might help the Russian administration to navigate in the pre-Bologna higher education landscape.

I think that you should ask to the university from which you graduated a Diploma Supplement for your degree. This should help the non-European country in recognizing it.
In many universities Diploma Supplements are nowadays routinely delivered to the new graduates, but maybe they are able to prepare one also for those who graduated before the Bologna process.

In my university (in Spain) the diploma supplement has still not been implemented, but in any case the relevant university official can write a letter detailing the credits/hours (and other relevant information) required to complete a given degree, and this is routinely done for students seeking admission to programs in other (particularly non European) countries.