text:
Are journal papers really viewed as strictly inferior to conference papers in Theoretical Computer Science?
To give a bit of context, I graduated 1 year ago from an undergraduate where I double majored in Math and Computer Science and I am planning to apply for a CS PHD this coming year with a focus on theory.
During my undergraduate studies, I worked on a project with a Professor that lead to a publication in a top Information theory conference (although the paper was about an information theory problem, it was essentially a theoretical cs paper). In addition, we added a new results and submitted the result to a top information theory Journal and we got the paper accepted. The last "breakthrough" came when I was able to come up with a new optimization technique that was interesting from a CS theory point of view, which led to a CS theory paper.
However, after writing the paper, my advisor told me he would prefer to submit the paper to a journal ("Theoretical Computer Science Journal") right away, rather than to a conference.
I understand that in CS, conferences are generally preferred to journals, but my question is, assuming that the CS theory paper gets in, would I be disadvantaged during the grad school application process against someone who has, say a SODA, paper? Are journal papers really viewed as strictly inferior to conference papers?

Depends on who is looking. I judge quality by reading the papers themselves as opposed to where they appear.  However, some of my colleagues just go with 'brand name' conferences/journals.

Does this answer your question? Journal vs conference publications when looking for a job in in computer science

I understand that in CS, conferences are generally preferred to journals

I disagree: Conferences are preferred for earlier, smaller results, journals are preferred for more mature results. Journals can be considered more prestigious, given that journal articles are typically derived from earlier conference papers and are subjected to more scrutiny, e.g., multiple rounds of review (conference and journal reviewers) and discussion (after talks, for instance), and they've survived some test of time.

my advisor told me he would prefer to submit the paper to a journal...right away, rather than to a conference.

Your advisor is confident that the paper is already sufficiently mature and the typical conference-journal route isn't appropriate.

assuming that the CS theory paper gets in, would
I be disadvantaged during the grad school application process against
someone who has, say a SODA, paper?

I'd argue a journal article is superior. However, the turnaround time for a journal article is longer than a conference paper, so you might be in the position that your paper is under review, whereas a competing grad school applicant may have had their conference paper accepted. An article under review at a top journal is a good sign, but an accepted paper at a top conference is perhaps better for an evaluation committee - a bird in the hand is worth two in the bush.

Are journal papers really viewed as strictly inferior to conference papers?

I don't believe they are, but I don't think there's an absolute consensus.

"subjected to more scrutiny, e.g., multiple rounds of review (conference and journal reviewers)" Depends on whether the journal article indeed extends a conference paper, and what conference it was. Top conferences may recruit their reviewers only from leading experts in the field, whereas journals may also invite less-than-stellar reviewers.

@lighthousekeeper That's why I wrote typically - it does indeed [depend] on whether the journal article indeed extends a conference paper. I agree that the pedigree of venue matters. Authors matter too - some will expose their work to comment far more than others. Although journals may...invite less-than-stellar reviewers, that's also true of conferences. Ultimately, the quality of reviewers matters.


text:
Is it appropriate to mention that authors refused to provide code samples when asked?
I work in a sub field of computer science. There was a recent paper (2020) that was published in a reputable conference that claimed to achieve state of the art performance on a specific data set. This data set is a hot dataset, and my research group has many papers on that specific data set (improving state of the art). I asked the authors very politely if they can share their code samples for their fairly complicated algorithm, to which they rejected and told me that it is not available.
One of my colleagues took 2 weeks to code up the algorithm, and verified its performance, as well as the claimed performance. Meanwhile, we reached a new breakthrough on the dataset and achieved a new state of the art. We are currently writing a draft.
Now, I can't help but be left with a bad taste from this whole experience. I don't see why they wouldn't share their code especially if they claim a state of the art on a very important data set. Most papers have to be compared with the state of the art to some extent, and so essentially, they're slowing down all researchers in this specific field to recreate their own work prior to publishing their own research.
Now my question is, would be advisable to add a note that the implementation of THEIR algorithm is our implementation, and that we reached out to them for their code and they declined?

It is called replication crisis. Nothing to do with you particularly. Everyone is facing it

@SSimon: since the colleague was able to code the algorithm from the description in the paper and to verify its performance, this is not an example of the replication crisis. (That it is still not nice(TM) is another question IMHO)

So why he asked ?

@SSimon: if I'm also not nice (to OP's colleague): because OP's colleage wanted to skip the work replicate the paper. He only wanted to repeat the calculation. Thus, one may say that OP's colleague would have contributed to the replication crisis (the part where replications aren't done) but fortunately circumstances prevented them.

@cbeleitesunhappywithSX: I don't think it's fortunate when someone is forced to spend 2 weeks to code up an algorithm with could simply have been in the paper. "skipping the work" is not appropriate here. Are you saying that it is bad when papers contain code or proofs? Should reviewer always have to recreate the details of papers instead of checking them?

@user111388: I don't say it's bad if a paper contains code. I appreciate it. But at the same time I realize that in practice it will further reduce the amount of verification of said papers. The 2nd comment is still very much in the context of the replication crises: knowing that there is a replication crisis I think it fortunate that a paper was thoroughly verified - even if that was not the original intention of OP's group. One of the main reasons behind the replication crisis is that far too few replication is done. Those 2 weeks of work need to be properly appreciated! They are far more

... than something that could have been achieved by using the other group's code. You are right, though, that having the code available should not deter one from properly verifying the paper. I.e., it's even better if the code is available and 2 weeks are spent on verifying it. But: OP's group now has been doing something that we may call a blinded verification, i.e. without access to the code. Verification with the code available would already be a whole lot of an advancement over what is usual in my field, but we wouldn't consider it as independent a verification.

@user111388: to perhaps summarise what @ cbleites says: It’s good when a paper makes its code available.  It’s also valuable, in a very different way, to have independent re-implementations of an algorithm.

I once asked some authors of a paper for source code because their paper claimed they'd implemented a functional version of their novel algorithm.  After a lot of back and forth they reluctantly shared their code and it didn't work at all. Their algorithm was sound though, and after implementing it from scratch my code worked.  So it's quite possible their code has problems they'd rather not share.

Whatever happens here, put your implementation on github (or similar repo, e.g. zenodo) with an appropriate license and let everyone else see it and use it.

If you did get the code,  you'd have to   verify the code anyway, so not a big deal.

When undertaking any activity, it's often useful to ask yourself the question, "What do I want to accomplish by doing this?"  In this case, it sounds like you are annoyed at the other research team and you'd like to inflict some social pain on them for not being "team players," and you want to use your paper to accomplish this goal.  I believe that by casting them in a negative light, you take the risk of making yourself look petty, not because you criticize them, but because you use your paper to do so.  I think that the potential for self-harm here is to great to undertake this path.

Not an answer to your question, but I will state that there are reasons beyond the authors' control for such things. I work in the national labs. We often have government sponsors who want us to publish papers (sharing work is good), yet have a multi-year review process for any code we release to the public (government policies aren't always precise). It's odd, but we do our best to properly characterize our algorithms in our papers!

would be advisable to add a note that the implementation of THEIR algorithm is our implementation

Absolutely yes. This provides important context for your experimentation and as such, is valuable information for the reader.
Even better, you could make your implementation of their algorithm publicly available, so that future groups of authors will not encounter the same situation as you did (and will cite your paper).

and that we reached out to them for their code and they declined?

That is the more delicate question to ask. It would surely be legitimate to do so, as it would represent the situation accurately. But such a statement has a social implication, which is that the authors appear in a slightly negative light (perhaps rightfully so). Proceed with caution.

Maybe one could phrase it in the passive voice to ease the accusatory tone: "The code could not be obtained."

@henning--reinstateMonica and that is how things become vague and useless. Not everything needs saying, but when you want to say it, say how it is in a polite manner. I can also see it creating doubt with the reader ("couldn't they simply send an email? How hard is this") similar to how I will prioritize papers without typos because at a glance they seem like having a higher chance of not being mistaken or trivial.

I have seen this before, especially in hot topics. The paper said something along the lines that they use their own implementation to provide a benchmark for method X, as the code was not available publicly or upon request. I think just something along the lines of "The code was not available" is not vague at all. They also made sure to point out any implementation or parameter details they had to set, and whether there is given value for those parameters in the original publication, or how they determined the parameter values to use.

Make your implementation of their algorithm publicly available. Thus, the other group can check your implementation against their code. If you made error, which makes their algorithm look bad (things can happen), then they can notify you and highlight the issue. The reason why the other group doesn't make their code available can be plenty, Hanlons razer applies. The readers of the paper can draw their own conclusions from you re-implementing the algo of the other group.

@Luc The passive voice is often used to avoid accusations. It is omitting a fact, but that fact is arguably not relevant while the fact the could code not be obtained is. There are many, many reasons getting the code may not be as simple as sending an email and highlighting that it was an outright refusal probably doesn't help anyone while casting the other researchers in a somewhat negative light.

I think you can be clear without being too pointed. "The original code is not yet available", "We were unable to obtain the original code.". Particularly helpful if you leave open the door to them later publishing the code, assume good faith etc.

Just say it as it is: "The code for paper X is not available and was not provided by the authors upon request." This is not accusatory, it's valuable information to the reader (which may also be interested in getting the code), and does not suppose anything about why the authors declined, whether they had a good reason or not to decline, etc. If your community thinks that it's unacceptable to write this, they are the problem.

@Luc The relevant meaning is "the bugs are ours". The omitted part is "The author's refused to share their code". You can discuss if it should be mentioned they did not share the code, but you should at least state that you used your own code, especially when it may contain bugs (or bugfixes) that the original code did not have.

@Luc "Vague and useless" is how people word things when they want to avoid casting blame. Saying the original authors refused to provide code is more accurate, but it is also a publicly statement that those authors refused to comply with one of the base rules of academia (i.e., cooperate with other researchers unless otherwise stated). That would be taken as an insult. I see your point but I can also see why the passive voice might be preferred.

As a software engineer, I'll give the dissenting opinion.
Source code is not an algorithm.  It's a "dusty mirror" version of something which hopefully is the algorithm they intended and which hopefully performs correctly.  Software being software, and coders being human, there are many ways in which those "hopefully" parts may not be as expected.  It is even possible (albeit unlikely) that the perceived improvement may come from an unintended feature of the implementation, and/or that perceived improvement may be as a result of making an implementation trade-off which has deleterious effects elsewhere.
In general science terms, the algorithm is the hypothesis you're testing; and the source code is your laboratory glassware, circuit boards, interferometer, or whatever other equipment you'd use to test the hypothesis.  A general principle of science is that reproduction of results must be independent. That means you don't just get a second person to blindly run the same tests on the same glassware with the same bottles of reagents - instead an entirely separate team reproduces those results from first principles to confirm them.  Independent testing of a hypothesis is what moves this from merely a hypothesis to a solid theory.
By implementing the algorithm independently, you have performed a vital scientific role in reproducing the original results.  If the original authors had simply given you the source code, there would have been no reproduction and test of the algorithm.  If they'd given you the source code, you might as well count photocopying the paper as "reproducing their results". :)
You would like to push ahead with the state of the art, and that's great - it shows that you're ambitious in your field.  But independently verifying a hypothesis to be able to call this a "theory" is not a waste of time, and slowing down to get this groundwork in place is not something which should leave you with a bad taste if the "science" part of "computer science" is for real.  Otherwise you're not doing science, you're doing hacking; which is valuable in its own way, but it isn't science.

I do think it is best to share the source code to aid in a constructive research community, but I agree with you. For every well written paper it must be possible to derive an implementation that obtains the same results. If this is conceivable then the source code is not mandotary, but a nice and helpful addition.

In at least some sub-communities of CS, where algorithms are compared in practice on some relevant benchmark sets, it is well-understood that the results from papers describing new algorithms are effectively not reproducible whenever computation times are reported (which is often the main argument). Hence, there is a focus on ensuring the repeatability of the experiments with repeatability evaluations of papers by independent committees using the author-provided implementations. So while reproducibility would be great to have, grad-student-built implementations made with 100s of ...

...design decision that are too nitty-gritty to put them in the paper will probably be never quite comparable. While from a scientific point of view, this is of course not a good situation, such communities carry on with their work due to lack of alternatives.

Retroactively, after the reproduction came out positive in this case (phew!), it's nice to have it. The bigger hassle is the case where the results cannot be reproduced, and without the original implementation it's hard or even impossible to pin down what happened.

@lighthousekeeper I don't disagree when that happens.  (That's "when", not "if" - we're all fallible!)  But unless you start your reproduction from first principles, you risk your results being biased by what you get from the other team.

Offtopic, but: what could be the value of their papers if they have no proof? The working source code is also a proof of the correctness of their algorithm.

Source code is usually the only complete description of the algorithm. Papers give an outline of the algorithm and discuss some key details, but they rarely give enough details for full reproductibility. After all, who wants to read tens of pages of prose, when the same information is available in a more readable form in the source code. Also, the published algorithm may not even be the one you should implement. It's quite common to have one set of details that makes the algorithm better in theoretical analysis and another set that makes it better in the real world.

@peterh-ReinstateMonica No, the results from your source code are evidence that your algorithm may be correct.  "Proof" is the wrong word.  As for the value of their papers, Alan Turing never had a Universal Turing Machine on which to run his work, and Rudolf Kalman didn't have a computer to run Kalman filters in real time.  On a more basic level, you can perfectly well demonstrate the algorithms for linked lists or bubble sorts on paper or on a whiteboard  - I know I have done for interviews.  That's why this is computer science and not just hacking.

@JouniSirén Anyone who thinks source code is a complete description of anything should not describe themselves as a scientist or a coder, because it demonstrates a lack of competence in both science and coding.  The fact you mention implementation differences should be a very clear pointer for why the paper needs to exist, and why the algorithm needs to be made available to be reproduced, because those implementation issues are what will kill you in corner cases.  (In real-time control, that could mean literally kill, by the way.)

If the program provides correct solutions for new arbitrary inputs, that's very significant. So disagree.

@einpoklum The point is that the algorithm provides correct solutions.  The program is (hopefully!) an implementation of that algorithm.  If the program is doing anything which the paper has not described, then the author is (almost by definition) either working in bad faith when it comes to sharing with the community or has not written their paper correctly, and the results and paper should be rejected for publication.

@Graham I disagree very strongly with this answer. An attempt to replicate someone else's results must cast doubt on their results if it fails. This vital aspect of replication is severely harmed by having the code they used to produce their results kept secret and it seriously degrades the value of a failed replication. Worse, it discourages replication attempts by raising the effort required to replicate the results needlessly. This makes the original results much less useful and reliable because they are deliberately made harder to falsify.

@DavidSchwartz Meanwhile, back on planet Earth, computer science is about the algorithm, not the code.  Peer review, although fallible, is designed to catch errors.  One doesn't complain that the Higgs boson discoverers are harming attempts to replicate the results because they didn't provide large hadron colliders to those wishing to do so.  An implementation that is of very poor quality but validly proves a novel concept is still good (computer) science.

There are plenty of legitimate reasons why source code might not be available: the coder may have graduated and is unavailable to clean the code up for distribution, the code may have been lost, the researcher's funding source may restrict the publication of the code, the authors may not have time to support others in trying to use their code, or they may simply have a different world view that they are not obliged to provide code to help other researchers compete with them, etc, etc.

@Graham The source code is the only complete description of the algorithm used for obtaining the results. The algorithm published in the paper is a related one, but it is often a different algorithm that exists for a different purpose. For example, the purpose of the published algorithm may be proving asymptotic bounds. In such cases, changes that make the algorithm 1000x slower but log log n times faster are certainly a good trade-off.

@JouniSiren I'm afraid there isn't any part of that which is correct. If results have been obtained with a different algorithm, then the paper as an absolute minimum must explain, in detail, the differences between the published algorithm and the practical implementation, and the trade-offs inherent in those differences. Those trade-offs are a key part in their own right. I don't dispute that it's useful to have the source code, but if the paper has any value at all then the source code can be written from the paper. If it can't, the paper's author is an incompetent charlatan.

@Graham You have unrealistic expectations. The purpose of a paper is to describe the key results and the key ideas behind them to the target audience of the journal/conference. There is often a strict page limit to keep the papers readable by avoiding unnecessary details. There may be supplements or external resources that describe the details. For algorithms, source code is the preferred format, because it is shorter and more readable and contains fewer errors than a prose description of the same details.

@Eric if one could include a copy of the large hadron collider in an quick email, it would be polite to offer to do so.

@Eric: the code should have been made available. No point about cleaning the code for distribution, not doing it so you have time to lose it later, no implied obligation to provide help. As for funding sources, I'm not aware of any source restricting the distribution of code. Just dump the damn code on some public hosting platform and link to it from the platform. Not doing this is simply malpractice.

@a3nm You're welcome to your opinions, but please recognize that that is what they are.  No requirement exists to publish source code.  I have certainly encountered situations where results were kept from public release (and posted a question here about them).

You can, and maybe you should in order to make your position clear that you think that academic codes should be shared if that is feasible. Many scientists will agree that sharing codes leads to better science because it makes it easier to replicate research, and also to find mistakes.
At the same time, not everyone feels like that, and you happen to have found someone of that opinion. To publicly out them as "backward" and "unwilling to make it easier for others to replicate their findings" may be ethically the right thing to do if you believe in open software, but it may also rub them the wrong way and create "a bad taste" in their mouths to use your phrase.
So the question is: How much do you want to insist on principle, and how much do you potentially want to alienate one of your colleagues? That question likely comes down to where you and where they are in your respective careers.

I agree. I think in many circles, it's also possible that the code was not shared due to internal regulations. For example, the authors may be working at a company that allows them to publish general advances in knowledge (e.g. an improved algorithm) but more strictly controls the source code (or the data sets that were used). Publishing is not purely academic. Only 6% of conference presenters at the top two AI conferences share code. https://www.sciencemag.org/news/2018/02/missing-data-hinder-replication-artificial-intelligence-studies

@TomAnderson: Yes, excellent point.

My advise is to get the most out of the situation as it is: you did a proper replication from the description in the paper (as opposed to "just" running their code) and verified their claims. Publish this as such: this is proper scientific
work.
In your own interest, I'd not mention that you did not do this replication entirely of your own free will.

TLDR:
I fully agree that it is not nice(TM) to not share code (see also below).
According to what you describe, however, this inavailability of the code has lead to further scientific advance than had the code been available to you:
You verfied their claims by implementing from scratch the algorithm described in their paper without any access to their code.
It is scientifically relevant to describe this in your paper since it means that you provide the next higher level of reproduction compared to "only" running their code on your computers: you also verified that the description in the paper is sufficient to reproduce the claims.
IMHO this is particularly important since we do have a reproducibility crisis in many fields.
Sharing code helps with what would be called repeatability in my field (analytical chemistry), but reprodcution or replication of a study/paper is more than that.
Yes, code sharing is nice since it saves a whole lot of work for those who don't want to replicate, it is often the easiest way to unambiguously describe what computations were done, and is also very helpful in tracking down misconceptions by the reader. Still, iff code sharing leads to everyone (or too many) skipping the effort put into proper reproduction/replication it may be counterproductive wrt. the reproducibility crisis.
Thus, I'm a bit torn:

Their not sharing code is not nice.
Saying that you did the implementation from scratch only because the code was not available also means admitting on your side that you'd skip a more thorough verification of that paper if you have a choice. Also not nice.

The ideal situation would have been had OP's group done a full verification of the paper and have gotten access to the paper authors' code.
Personally, I'd not allude to any non-scientific reason for tackling the reproduction of that paper.
I use "not nice" here to indicate behaviour that is less than ideal (for scientific advancement) but is also clearly within the allowed range of behaviour.
To be clear, I don't want to force anyone into reproducing other groups' studies. At the same time, if all groups deny to do this kind of "homework", the risk for replication crises increases.

There are levels of not sharing that I consider definitively problematic from a scientific point of view:

Your proper efforts cannot reproduce the claims and the authors refuse to guide you (whether by sharing code or by looking at your code and telling you what needs to be done differently)
They claim in the paper that they will share the code, or the journal guidelines stipulate that they'll share but they refuse.

So far, the question doesn't indicate any of these points apply.

To chime in with Anonymous Physicist's answer: I have been working at a research institute where the very official line was that neither code nor data are published with the paper ("are available on reasonable request" was the official wording) - that I would have liked to not only show my research code but put it under an open license did not matter at all.
Thus, had I been author of said paper, you'd have gotten a polite reply that I cannot give you the code without official administrative permission, and would you please contact the director about it. However, I can easily imagine institutes specifying that whoever is contacted about sharing the code has to reply in the negative without creating further work to upper management...
From this experience: if you critizise the authors for not sending the code please make sure that you attach the blame to the correct persons.

"At the same time, if all groups deny to do this kind of "homework", the risk for replication crises increases." An important difference in computer science, compared to other disciplines, is that the effort for replication is minimal if the original authors provide their code and data (with proper documentation). In good conferences and journals, it's becoming increasingly common practice that the authors are required to do so.

@lighthousekeeper: But that's not a full replication: the effort is minimal iff they use that code without checking whether the paper contains a sufficient description of that code.  Yes, there may be field-specific differences here (my field, analytical chemistry looks at repeatability, reproducibility and replicability in levels of increasing difficulty and independence. For us, it is very important to not call a replication what is only a repetition.) I fully agree with you a) that if the authors were required to share, it is a serious breach if they don't and b) that one should work...

... towards requiring this even more. But unless the publication was under any such requirement, it's not nice, but not an academic misconduct and OP cannot criticize that other than as a political opinion (which is fine and laudable, but not scientific content of their paper).

Without knowing why the authors declined, it's a bit hard to tell if the other authors were "nice" or "naughty".
There are some legal and institutional pitfalls when it comes to sharing code. As an example, the university may have the right to check if the concrete research output contains patentable material. The authors may be forbidden from sharing their code if this check is ongoing.
So I suggest avoiding putting a negative light on this interaction if possible. Yet, it is important for the reader to know that you wrote your own implementation. And it's also important for the reviewer as well, as the reviewer may think that the other approach looks unnecessarily bad because you didn't implement it well.
Many papers have an appendix that will not be published with the rest of the paper. You could add a note for the reviewers that sharing the code with you has been refused for unknown reason, so you had to implement your own version. If you want to mention in the main part of the paper why you didn't use the original implementation of the other approach, you could mention that it was unavailable, hence you implemented your own version and made sure that you took similar care in both implementations not to miss important optimizations.

With the information given, it is not possible to assess the motivation of the original authors. As a frequent publisher of open source scientific code, I can think of several.

They did not want anyone to replicate their results using their code, as they wanted to make more publications from it (certainly not nice, but seen before).
Their code was not in a state to be used be anyone else (PhD-ware), and they don't have the man-power to clean it up.
They make use of proprietary libraries, or have internal licensing issues, not allowing them to share the code.
They simply don't want to be bothered by the inevitable tech support issues, that comes with supplying ones code to other people.

And the list could be continued.
You can only act given the information at hand, and that is, that at the moment the code does not exist in an open form. Fortunately, you were able to reproduce the results, and you are now free to publish your own implementation along with your article, prompting users to cite your paper, as well as the original one. After all, you have made a significant piece of scientific work, which should now be credited.
You don't need to be snarky in your paper. Write about what you did, and not what other people did not. You can note that:

The implementation outlined in this paper reproduces the performance by author A, quoted in ref. [A]. The present implementation can be obtained from XX, allowing for further independent validation.

I don't see why they wouldn't share their code especially if they
  claim a state of the art on a very important data set.

Hypothetically, one can make software that can't legally be distributed under copyright law.
If I own a book I can write notes on it and cross out sections all I want - but I can't distribute copies of my modified book. Likewise, if I want to extend a copyrighted Matlab library in a way the authors didn't anticipate, I can copy-and-paste from it and modify it as much as I like for my own use - but I can't distribute the resulting derivative work.
If you want to assume good faith, you could assume some similar benign reason on their part.
Of course, a cynic would say authors are rewarded for wringing as many papers as possible out of a piece of work, and if there are obvious follow-on papers (like obvious tweaks, or tests on other datasets) not releasing the code until you've written those papers is the behaviour their paymasters reward.

It is quite easy to make a software that cannot be legally redistributed: link together open source libraries which have incompatible licenses. See https://opensource.org/licenses; it is widely known that GPLv2 is incompatible with GPLv3

This answer rightly addresses a minor question contained in the OP. However, the main question from the last line is "would be advisable to add a note that the implementation of THEIR algorithm is our implementation, and that we reached out to them for their code and they declined?". It would be helpful if you could address also this last point.

Unless the topic of your research is open source software, your journal article is not the right place to criticise your colleagues for not releasing their source code.  Journal articles and conference papers should be about the knowledge you have created, not your opinion of your colleagues.
Before criticising your colleagues in any venue, you should consider if releasing the source is a reasonable option for them.  There may be legal restrictions you do not know about, or a very inconvenient administrative process.
Peer review is a great time to encourage transparency.

A statement like "Despite our request, the original implementation of the algorithm was not made available to us." is not an opinion, but a fact, and might be relevant to explain why it was necessary to re-implement the algorithm (possibly with different results). People might be disgruntled anyways.

@lighthousekeeper Your fact is still criticism, and clearly implies an opinion.  I would suggest "We reimplemented the algorithm" as a more appropriate wording.  The request is irrelevant.

"We reimplemented the algorithm" might raise the question why a reimplementation was done. OP seems to be in the comfortable situation that both implementations produce the same results. If the results would have been different, one would need to justify what one did to figure out the cause of the differences. If the original implementation was not available, one would need to state precisely this. The request would be relevant, because it signifies more initiative on the authors' side than just saying "it wasn't available".

Personally, as far as I am concerned, if your code cannot be realeased for legal purposes, then your paper cannot be published. I think a passive statement that just states the facts "We reimplemented the algorithm as the orginal code could not be obtained", as suggested by @henning above is the way to go.

@lighthousekeeper: it is a fact, but not a scientific fact. My political opinion is that code produced by public money should be available to the public. But unless there is a hard reason why they should have shown their code (e.g. the journal guidelines saying so, or even the paper saying so) I would refrain from judging them against my higher standards.

@cbeleites Part of the scientific methodology is to explain why you did the things you did. In that light, the point is not judge, but to explain why a reimplementation was necessary -- and in the case of a failed reproduction, explain why it was not possible to get further insights into the reasons for the reproduction failure.

@lighthousekeeper You do not need to explain why you did something if it's not relevant to the conclusions.  Which in this is case is highly probable.  The question says nothing about reproduction.

@AnonymousPhysicist That might be a field-specific difference then. In computer science, rationale is an important part of scientific writing. OP mentions that they "verified the claimed performance". Apparently they came up with a different algorithm that yields better performance than the reimplemented one, so it's key that the performance of the original algorithm was properly reported.

@lighthousekeeper What you are suggesting is not a rationale for the conclusions.  I tire of this discussion.

In computer science, you would need to explain the rationale of your methodology.

@lighthousekeeper: we may be reading the described situation differently (and only OP will be able to say who read it correctly): I understand from the question that OP's group has a new, better algorithm.  Better than said paper which constitutes state of the art. In order to show this, it turned out that they had not only to repeat but to replicate that paper's algorithm. Which they successfully did. I.e. we have 2 scientific advances 1. their new algorithm and 2. a verification of the previous paper.

@cbeleites The verification angle is a nice way to positively spin the need for a reimplementation (+1 for your answer). But unfortunately, in many subfields of CS, a replication will not be considered a big advancement by itself.

@lighthousekeeper: :-) Then let's hope that OP's new paper with a "verification of state of the art" section will help to get verification perceived as as important as it is. (There is the side of me that thinks it is a bad state if verification is an advancement rather than a chore that should not be skipped) (FYI, I also upvoted your answer)

Is it appropriate to mention that authors refused to provide code samples when asked?
I work in a sub field of computer science. There was a recent paper (2020) that was published in a reputable conference that claimed to achieve state of the art performance on a specific data set.

There are several issues.
FWIW, I am a semi-academic research engineer in France, working at CEA LIST in a cybersecurity lab, where I develop Bismon (a GPLv3+ software funded by two H2020 projects: CHARIOT and DECODER and described in this draft report)

how was the work described in the paper funded? if the funding agency is military (e.g. DARPA) the funding contracts may require academic publications but disallow any publication of the code or of the data set. Even H2020 funding may forbid publishing of data, or simply getting permission to publish the data or the code may require a lot of red tape work that usually is boring for any academics.
who owns that dataset and how was it funded? If the dataset was difficult to collect (think of biological data, e.g. Covid19 related, or of nuclear industry related data) you might not be able to access it
are there any privacy issues or personal data issues? Regulations like GDPR are then highly relevant
what legal system and copyright applies to both the data and the code samples?
Read also Jean Tirole's work, in particular his paper with Josh Lerner on simple economics of open source. Notice that Shoshana Zuboff books and papers (e.g. Big Other: Surveillance Capitalism and the Prospects of an Information Civilization for example) are also relevant.
the dataset or the code may be an asset for future research grants.

Notice that academia is a small world (read also about Ershov number and more importantly Erdös number). Mentioning in a formal written paper that some authors refused to provide code samples may backfire, and could perhaps impact negatively your academic career in the next few years. The common practice is dual: mention in acknowledgment sections the people and their institution who provided you with code and data.
If publishing the code or the dataset was not previously and explicitly mentioned in the research grants funding the work, it is likely that you might not get it easily. I got the habit of mentioning (at least in footnotes) such things in most research grant proposals I contributed writing.
By personal experience, obtaining permission -within professional context- to publish some code as open source or some dataset as open data is very boring and stressful (and nearly impossible after the work has started). You should mention these things in writing in research proposals and grants.
there is no such thing as a free lunch
See also these slides, and the RefPerSys project. So far, funding is the main problem in it.
In Europe, ITEA3 and BDVA and HorizonEurope could give interesting contacts. You should find the equivalent in your place.

Ershov number? Or Erdos number?

One would hope that authors would write into the paper "this dataset is not avaible because of miltary funding"..

No, writing that requires approval by military, and getting such an approval require days of full time work

So what? Then the authors should consider it part of their work to improve that paper by getting the approval for this sentence. What is the point of this rule (one is not allowed to write in a paper for which reasons the data is not avaiable) anyway?

Some ramblings -just to provide another perspective- before the answer.
I can share my experience as a "user" for similar occasions. CS is not my field, and I can not rewrite each and every algorithm that claims to be a "drastic improvement" (Spoiler alert: Most are not). I simply do not have time, and brute force algorithms usually work for me. Usually. But if I had access to said algorithms (read: actual code), I would have used them and that would be a huge enabler for my future research.
Some answers/comments claim, roughly stating, using the same code is not reproduction. For CS, this might be true to some extent -I won't even argue that. But reproduction is the real concern here? I value usability more than pure algorithm reproduction.
There are other fields (such as mine, sports sciences) which would use some "help". Don't take it wrong way, I can most probably code said state of the art code myself, given the algorithm. But it simply is not worth my time. Because a potential (and a very little one in my past experiences) improvement in execution speed/memory footprint does not justify working more on my already-working proof-of-concept code than tackling the next problem on my list.
There are lots of articles proposing various algorithms, but not bother to even explain them properly.
Back to your question:

Now my question is, would be advisable to add a note that the implementation of THEIR algorithm is our implementation, and that we reached out to them for their code and they declined?

Definitely advisable for the first part, since that is pure blind reproduction of their algorithm, and it adds value to both parties' research.**
For their refusal: How can you be sure that their proposed algorithm was not a product of a lunch-time conversation, and never before realized in actual code until you did it? How can you be sure their claimed performance figures were not fabricated? You simply trust the researchers? What if your implementation did not verify their algorithm's superiority? Would you still publish it, or give in to appeal to authority? Would you check -rigoriously- if their code match up with their explanation of the algorithm: i.e. there is nothing up their sleeves?
I believe these questions are the reason their refusal leaves a bad taste; not your coworker's 2 weeks.
Whether in a publication/presentation or in your circle, I think you owe science to make this info public. The latter seems less risky if you are not "well established yet" though.

The whole providing source code of the paper thing has been discussed to death on this site and elsewhere, so I won't get into it much. I agree with you that they should have given source, and it should be required by journals anyhow. That said, currently it is not. It would be nice if they helped you reproduce their work, but they don't have to.
If you had discovered that when you take their instructions at face value the results are not reproducible, you would have cause to write to the editor of the journal and the onus would be on them to help demonstrate their own findings as legitimate. However, it sounds like you[r friend] were able to reproduce it without their help, you just wish it was easier. Fair enough, but you can hardly demand they go out of their way to make your life easier.
If you intend to distribute the code you wrote, there is nothing wrong with that and it's a very good idea. I wouldn't attach a little manifesto saying the authors were unhelpful jerks and you wrote this prove your point. While it is true and you are within your rights, it will make you look bad. It's worth explaining that your implementation is independent and not based on the original code, so as to not confuse any readers, but there is no need to go into why they didn't share or who is at fault -- these are not matters of fact pertaining to the research itself. You can say something like "this code is an independent implementation of the algorithm in Smith et al. 2020, written without access to the authors' own implementation". That last bit you can fine tune depending on how fine a point you feel like putting on it. But, once again the less you say about how much you dislike that the authors wouldn't share the code with you, the better (in the long run).

I don't see why they wouldn't share their code especially if they claim a state of the art on a very important data set. Most papers have to be compared with the state of the art to some extent, and so essentially, they're slowing down all researchers in this specific field to recreate their own work prior to publishing their own research.

There can be many reasons, not all of them nefarious.

If the journal doesn't require code, providing it is extra work and inefficient.
You would have to document how other people can build and run it on their own computer - something which can take extra work and maybe is not a concern on your own system.
Generally the code is not that helpful if not portable - while the algorithm is already portable.
Just because they provide the code doesn't mean it will be useful to you. It could be in a language you don't know.
The code may be messy and perhaps they don't feel like inviting pointless questions. This may seem dubious but imagine you were asked to provide every single draft and napkin drawing, and then people made you explain random thoughts that were jotted in the margin and were abandoned later on anyhow.
Whenever you provide code, all sorts of people start asking you for help with matters that are sometimes barely connected to the work itself. While you should be available to answer questions about your work, answering questions takes time. If the journal didn't require it, why create additional work for yourself?

To be sure, I support journals requiring source code disclosure for the same reason that experimentalists are expected not to just say the name of the chemical they used, but give the supplier, catalog no and lot number (and just like companies in ancient papers that no longer exist, code from old papers can often be un-runnable). But at the end of the day, it's only a problem if your results happen to be an artifact of the specific reagent/implementation you used. If the protocol works well with any reagent, and if the algorithm is not hard to implement properly, it's not really a big deal.

This might have to do with their lab policy or partners. I am a computer scientist working on problems with a medical focus, none of my clinical partners are okay with releasing code. This is a real problem with medical communities, they even asked me to wipe my computer when my contract was up so that I wouldn't have a copy of my own code. Of course like many, the real reason I suspect is that they think sharing the code puts them at a disadvantage, they simply don't want others to use their benchmark. This is just my hunch although it sounds counter-intuitive. But many people had no problem replicating my work and my experimental results - so the fact that you were able to implement the work in 2 weeks and achieve their results, I would say they did a decent work (plus you gained more insight and outdid their work). I had trouble getting the same results using my labmates' code at times.
On a personal level, I am thinking that if the authors didn't make their code publicly available, they would be hesitant to share because that requires cleaning the code etc. which might be time consuming for a single individual, which to be honest I can relate. Most people work in groups though so I would expect their code to be more modular and cleaner, or simply they could have someone work on it.

I appreciate the bad taste in your mouth, but please put yourself in the shoes of a reader who is interested in research as a distilled outcome of a methodology in the net of predicaments and accidents.
On a note that the implementation of their algorithm is your implementation
I'd expect yes.
You ought to specify to which extent your work is original and in which way it differs/draws from any things you know. This is about doing justice to the work done by any other scholar, obnoxious to the bone as they may be.
To each his own.
Furthermore, shifting the focus onto the topic, it can be useful to have in mind the (attempts to draw) distinctions between

replication
repetition
reproduction

as discussed in https://plato.stanford.edu/entries/scientific-reproducibility/#ReplRepeReprScieResu borrowing from reflections in social sciences.
On a note that you reached out to them for their code and they declined
Here it is time to do justice to your own effort and working situation.
If anything at all, I would state no more than something like

That source code is not available [to us] (that gentleman, personal communication)

No adjectives, no adverbs in any case. No allocation of blame, just an event, documentable if ever needed. If the code ends up to be available later, it was miscommunication or bad timing.
The reader will appreciate the efforts of advancing the state of the play on their part and yours alike.
At least, as a referee I would be happy like this.
My two cents.
Happy to be corrected.

Acknowledge the source of the algorithm, gratefully, state that the code is not based on the earlier code and do not complain that the earlier code was unavailable. As others have pointed out, they may lack the legal authority to give you the original code, and in any case the new code serves as independent confirmation of the algorithm.

It would be very useful for the community if you clarify that you have implemented your version of the algorithm and provide the details of parameters (if any) in the supplementary material or your experimental setting. You don't probably need to elaborate on why you implemented other algorithms, the readers can guess what has happened! It would be also very useful for the community if you could also publish your implementation so others can save time for their comparison, and obviously your paper becomes more visible as a result. I know this is quite a struggle for researchers to implement the work of others and conferences (and reviewers) are becoming more vigilant to make sure authors provide not only code but also any other info such as data and parameters for ensuring the replicability of the work. Hopefully, someday, we reach a point, where our publications serve as a justification for the key outcome of our work, which is the code we write.

No it is not appropriate and such statement or “accusation” should not be part of a scientific publication. None of the readers would be interested in that detail and it also does not serve the purpose of your paper.
Instead of pointing of that the authors refused to hand out their source code, you can simply mention that the source code is not available and the implementation is of your own. I understand that you felt offended by the author and wanted to retaliate. After all, you could verify their result with your implementation, so no doubt is left. Why should you then bring up  an personal issue in a scientific publication, whose sole purpose is to damage the reputation of the other authors?
@Graham has already provided an excellent answer, which explains why it is necessary and desirable to have independent validation. I was in this “business” before and agree on all the points he made. My field was bioinformatics and we not only published new algorithms but also produce software based on that for the biologists to use. It is a common practice in our field that the source code is published because we all want to have users. Therefore, I did not have your problem when doing performance comparison. However, I had the other problem that @Graham pointed out. Many of the real performance improvements lie in the implementation, which is simply not mentioned in the publication. This could be intentional or unintentional. Some of the implementation details don’t sound like a scientific contribution and are left out. However, their impact could be huge. We were once puzzled by the performance of a software we wanted to compare with. Therefore we decide to analyze its source code. We then figured out the author has a very smart way of implementing a function, which saves a huge amount of time. This is a common mathematical function in our field and it seems quite clear how this function should be implemented until we saw his approach. We reimplemented his techniques in our software and saw wonders happened.

Have you been able to reproduce the results or not, from the description of the algorithm in they article? This is not clearly seen in your question.
If yes, then the article is complete and there is nothing to complain about.
If not, the, yes, there is something to say about just the wording should be different:
"We have repeated these experiments but were not able to reproduce the results"
In order to be a scientific publication, the article must include enough information to repeat the experiment. If multiple independent researchers fail to reproduce, the author gets into trouble.

I have rewritten this question and think the current version is correct.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How should I state my masters degree in my CV if I finished the program, but the ceremony won't be held for several months?
I'm currently updating my CV because recently I finished a master program. although the ceremony is going to be on March, I don't have the diploma yet. How should I state this in my CV?

Related: What date should I use for graduation?. Has your degree been conferred yet?

Yes, the university just confirmed me that I fulfilled all requirements and I'm listed for the ceremony

Wonderful, thanks, I think that is the answer I was looking for

Find out from the university when the degree is conferred. This may be before or after the ceremony (it could be already conferred). (Also see: What date should I use for graduation?)

If the degree has already been conferred, you can write that you have a masters degree (the ceremony is irrelevant).
If it hasn't, write that you have completed the degree requirements, and expect the degree to be conferred on X date.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
To work in statistics for industry and research centers, is a masters sufficient or is there a major advantage to having a PhD?
So I am inquiring general tips from experienced individuals of academia and industry.  I am a recent graduate from a top university with B.S. in math.  I had goals of doing PhD in stats, but recently changed mind.  I have a bit of family pressure to get a job and work to build income.  I am from lower middle class, and am 30 years old.  I've worked at a leading hospital and my boss encourages me to do a PhD.
However, it seems better to work full time at a company which will pay for the masters or some of it. My strategy is that at 33-35 years old, I would be working full time and working on a masters instead. Thus at this age I could have work experience, income, and a graduate degree.
It seems like an unwise choice to pursue a PhD and graduate at age 35-36 without work experience and having to start earning income.
My interests are to work in industry, and research centers. I do not believe PhDs would guarantee job security, higher wages, or happier careers.  A masters would suffice.  I argue that a PhD is not necessary.
Is my logic sensible?  On a personal note, I am quite disenchanted/burned out with higher education.

Welcome to Academia. SE! Nobody here can tell you what your decision should be... so I've edited your question title to focus on the objective, answerable part of your post. Please feel free to [edit] yourself if you feel I've misrepresented your question.

Related (possible duplicate?): What are the benefits of getting a PhD in statistics over a MS in statistics?

Background: I have a math Ph.D. and am working as a statistician in industry.
I'll try to address both the general question you ask and your specific situation.

M.Sc. vs. Ph.D. in stats in general
This quote of yours is key:

I do not believe PhDs would guarantee job security, higher wages, or happier careers.

And you are right about this. A Ph.D. won't guarantee any of these.
If you do a Ph.D., you will end up with a different qualification than with "only" an M.Sc. In doing a Ph.D., you will be spending multiple years thinking up new methods or algorithms. You will become an expert on an (admittedly small) topic, which you will know more about than your advisor. In contrast, in doing an M.Sc., you will learn a lot of different techniques and apply them, but you won't develop new ones. (Some M.Sc. students do just that - exceptions exist.)
What does that mean for your job prospects? If you are looking for a job as a "consulting statistician", "data analyst" or similar, an M.Sc. will usually be enough. Employers may balk at hiring a Ph.D. for such positions for multiple reasons:

Ph.D.s usually ask for higher salaries
Ph.D.s are older, possibly less flexible (families) than entry-level M.Sc.s
Ph.D.s may feel underchallenged and try to leave for more interesting work elsewhere
Ph.D.s may be specialized in one field - but an M.Sc.'s knowledge of multiple methods is more recent

So with a Ph.D., you would be applying for more "conceptual" positions, where you don't only apply your knowledge, but actively create new statistical methods to solve problems.
There are more jobs out there that require applying known methods than creating new ones.
You may be lucky and find a job opening that requires a Ph.D., or (even better) one that matches your Ph.D. research interest. (I wouldn't count on this last possibility - research is so specialized these days that it is rare to find a position in industry that closely matches what you did in your Ph.D. career.) If so, your Ph.D. pays off. If not, you may be in for a long search, or you may need to work for lower wages, and still need to convince an employers that he is better off hiring you than a new M.Sc. graduate - at the same wage. If this happens to you, you will definitely feel like the Ph.D. was a waste of time.
Overall, I would only recommend doing a Ph.D. if you are passionate about it, if you genuinely want to devote three to five years of your life to research. Don't do a Ph.D. for the career value if you plan on leaving academia. It likely will be a step backwards in terms of lifetime earnings or your career progression. (Around me, I see no correlation between having a Ph.D. and job security or higher wages - I can't judge the happiness of my colleagues' careers.)

Your specific situation

I have a bit of family pressure to get a job and work to build income.
  ...
It seems like an unwise choice to pursue a PhD and graduate at age
  35-36 without work experience and having to start earning income.
...
... On a personal note, I am quite disenchanted/burned out with higher
  education.

I see a lot of skepticism about doing a Ph.D., and much focus on your alternatives. I don't see anything indicating you would love to do research for research's sake. (Please don't misunderstand me: I'm not saying you are lazy. I am pointing out what your priorities seem to be, based on your question.) Compare this to my recommendation above.
It does not seem to me that doing a Ph.D. would be a wise move for you.
Nevertheless, we can't usefully help you a lot with your decision. I'd recommend you talk to people who know you and your specific situation.
This and this earlier answer of mine my be helpful.

Be honest to yourself. Are the arguments against truly yours or are you echoing your family? If they are yours, you have your answer.
If you are echoing your family, do your homework on job security, salary, etc. There are salary surveys. PhD does bring you a higher salary and very different job. Job security should be higher with PhD since you are hired for your uniqueness - hence you are irreplaceable, or least it is much more difficult to replace you.
With Master in stats you help others make sense out of data with the existing toolset. Most of your project will be successful. They will be relatively low risk too. If you prefer to feel mastery of the subject, go for MS.
With PhD in stats you invent new tools. Your day to day life is uncharted waters. The risk is higher, and you never feel you have mastery. There will be lots of unknown, dead-ends, and fails. There will be occasional successes, which will make it all worthwhile.
Lastly, I have a feeling that age is the unspoken factor. You have a 30+ career ahead of you. Few years don't matter much. Choose what fits you better, because it will be 30 long, miserable years ahead otherwise.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is there some quantifiable difference between those who seek master's degrees and those who seek doctorates?
Is there some quantifiable difference between those who seek master's degrees and those who seek doctorates? If there is, does the difference justify the current distinction between the two?
The distinction seems like a product of institutional culture rather than anything practical. But that's just my view as an outsider. Regardless, I'd love to get y'all's insight in this matter.

I edited the title so as to avoid having this question closed as too "opinion based" (see the [help/dont-ask]). Can you [edit] the post to explain what you mean by a "quantifiable difference"? Examples might help. Right now, this question is totally unclear to me.

Your question is worded in terms of whether  there are differences between "the people". But the body of your question seems more concerned with the nature of the degree or skills learnt or the work required. If you mean the latter, then perhaps reword the question.

By quantifiable difference I mean mostly intellectual and technical capability. But it can be any metric really.

Well, one can try to quantify just about anything.  I think there should be some data about GRE/GPA averages, but I couldn't find anything after a quick search.  (The Chronicle for Higher Education would be a natural place to check, but their webpage is currently down.)  However some program webpages have data about students in their programs.  Here is one with data about admitted students.  The biggest difference is undergrad GPA (well, that and percent with masters degree).

You refer to students who "seek" a degree, suggesting that you are asking about the inherent qualities of such students before they begin their degree. I don't think this is what you actually mean. Please [edit] your question to clarify.

Your question may become clearer if you specify what you mean with "the current distinction between the two": is it differences in pay, status, or something else? Are you talking about inside or outside academia?

The distinction seems like a product of institutional culture rather than anything practical.

The distinction between masters and PhD is very practical: a PhD is in essence vocational training for future researchers, while a master is intended for those who want to know more than they learned in their bachelor but do not want to become a researcher.
So a meaningful dimension on which you could expect Master and PhD students to differ would be their intended future career. Notice that this is not a hierarchical dimension like intellectual and technical capability, it is just a difference in kind.

In some systems, it really is hierarchical, since a master's degree is a prerequisite for starting a PhD.

I don't think you understood the question. What is the practical difference between the training someone receives in a PhD program verses a masters? And does the difference justify the distinction placed between the two?
Asked another way, why should I be willing to rank a PhD higher given the same amount of time in field?

@DharmaBum What do you mean by "the same time in the field"? That seems to assume they are now both in industry (as continuing in academia is usually not an option without a PhD).

If you want a researcher, you should rank a PhD higher as they received specific training to do that task. If you want other tasks performed, then you should rank someone specifically trained for those tasks higher. So the ranking of grades is not absolute but depends on the task that needs to be performed.

What is the practical difference between the training someone
  receives in a PhD program verses a masters?

In one sentence, experience with planning and conducting scientific research in a particular subject domain.
A Master's degree may be viewed as an fast-track apprenticeship program. If done right, it allows the apprentice to get their feet wet in a knowledge domain, try out a relatively small-scale, "practice" project of independent research, and get oriented in the general space and culture of graduate-level academic (or "serious" professional, as in applied fields like business) life and work through a 1-2 year period of focused study.
In comparison, a PhD is a substantial, systematic program of study with a duration, structure, and checks and balances intended to make it difficult to cut corners, and instead to fully invest one's intellect in training of research in a particular knowledge domain.
This is a big area though, encompassing a fairly long laundry list of competencies, from subject matter knowledge (including past research, currest state the art, and future prospects), analytical thinking (both in the intricacies of the subject matter as well as practical issues of how to formulate, address, and communicate findings related to questions answering which is of value to the field and hopefully society at large).
At the end of the day, as has been pointed out, both are types of vocational training intended to provide a foundation upon which one might build themselves to become a professional in a particular area.
In some sense, both Masters and PhD are indicators of commitment to a particular occupation. It is relatively easy to try something out for a year or two, requiring some but not a very large amount of commitment, time, and effort.
It is relatively more difficult to do this for an extended period of time (typically 5-6 years) and invest oneself fully into a large, long-term research project where much time-on-task is expected and personal discipline, perseverance, and abilities to overcome challenges and make a recognized contribution to a field are put to the test.
The practical difference is that a person with a PhD would be expected, ideally, to be that much more effective in doing these kinds of things and exhibiting these types of qualifies as applied to a research-intensive occupation, all other things being equal. That is the working assumption, and the 'why' in 'why do a PhD'.

Why should I be willing to rank a PhD higher given the same amount of
  time in field?

Whether you are willing to rank a PhD above MA/MS depends on the value you attribute to the practical difference between the two (previous question). If these differences are relevant to your concerns, then you will probably interpret a PhD as carrying more 'weight' in terms of these relevant characteristics. So to some extent it is a matter of personal perspective. Also to some extent, you may want to defer to the "institutional culture" in ranking a PhD higher if your personal perspective aligns with that culture (stated differently, if you subscribe to the same premises).
One example of this is if you consider yourself sharing the cultural norms which exist in certain settings, of valuing greater ability in academic writing, experience conducting scholarly / scientific research (e.g. selecting appropriate methodology, planning/conducting the study, acknowledging methodological limitations, and clearly writing up the background/method/results/discussion).
What it boils down to, in my opinion, is the assumption that experience equals ability. The practical purpose for earning a PhD is formally engaging in a multi-year program of professional development with the assumption that it will yield greater skills/knowledge (ability) in a particular subject domain, as well as in some transferable skill areas such as research competency, as in the example  above.
Some resources:
What's the difference between a masters and a doctoral degree? (see paragraphs "Purpose and uses of a doctorate" and "Purposes and uses of a masters degree".
More focused discussion in reference to computer science
A related question on Academia
Hope this helps. Good luck!


text:
Seeking TT position with criminal record (DUI)
I am a few years away from being done my PhD in philosophy, and plan to seek work in the field. I have a DUI on my record from when I was 18 years old. Will anyone care?

If it's just a DUI, then you're right, nobody will care. Everyone already know teenagers are non compos mentis.

As long as you don't try to hide it, no one will care.

Nobody will care.
A drug-related crime (of which a DUI is one example) from your teenage years is just not at all relevant to your professional qualifications and your ability to do your job.  If it were more recent, or if it were a more relevant crime (e.g., a violent offense), then it might be a different story.
As noted in the comments, though, don't try to hide it if somebody asks about it.  You don't need to bring it up, but if somebody else does, just tell the honest truth and you won't raise any red flags.
Oh, and in the future make sure you have some sympathy for the students who are making the same mistakes that you did.

Violent crime records affect the actual education process? Not just potential of finding a job once you're done?

@jaska Violent crime could have a serious effect on whether potential co-workers would feel safe around a person, which would effect that person's ability to do their job.


text:
Should I accept suggestion to move my name to the front in a field where authorship is usually alphabetical?
I work in an area where most of the time authorship is ordered alphabetically, but sometimes when some authors contribute more than others, ordering by contribution is possible. I'm writing a paper with a former professor and another student. My name is often last when ordered alphabetically, but the professor suggests that I move my name to the front because I contributed more.
Should I accept this suggestion? Of course, moving my name to the front would emphasize my contribution for this paper. But when people see my CV as a whole, might they think that for other papers where my name is last (because the names are ordered alphabetically), it is last because I contributed least?

This is the first time I've ever heard of 'alphabetical ordering of authors' in paper publications. The order usually always depends on the level of contribution.

@ÉbeIsaac It depends on the field. Being in theoretical physics myself, I can't remember the last time I read a paper whose authors weren't ordered alphabetically.

TBH, you should be concerned about the order. Because nearly everyone I know in my field, CS, disregard the last author, especially papers with more than four authors.

@ÉbeIsaac There is a considerable variation within the sub-disciplines of CS. In the more theoretical sub-disciplines, ordering by name is the norm (see, e.g., http://www.fsttcs.org/archives/2014/papers.php for a theoretical CS conference accepted paper list), whereas in the more practical ones, it is the exception

@DCTLib: Intruiging. I would like to know some of such disciplines, and specifically publishers who excercise alphabetic ordering of authors within their publications.

@ÉbeIsaac I just updated my comment above to provide a pointer to a such a conference. Only 3/48 papers do not have an alphabetic author order.

@DCTLib: Thank you for the list. It seems pretty odd for someone like myself who isn't aware of this practice. Your list is a conference proceeding. Do you happen to know of any journal publication which is also does alphabetical ording of authors?

One more thing: is this practice excercised by authors or publishers (or both)?

Poor @DavidZ invariably ends up as the last author in his papers because of his surname, and this weird convention! However, in addition to being field dependent, it is also sub-field dependent. e.g. it happens most of the times in High Energy Physics in general, but perhaps not so much in the nuclear corner of  HEP, which amounts to even more confusion!

@TheDarkSide on the bright side nobody cares that I'm the last author (at least not consciously), because they know author order is alphabetical. Though it is comically frustrating to be on a four-author paper which is cited "W, X, Y, et al." :-P Oddly enough I also work in nuclear HEP. I'm not sure why we seem to know different conventions for author ordering despite being in the same field - I guess it shows just how isolated communities in different niche topics can be from each other.

@ÉbeIsaac in my circles, journals don't enforce or even recommend any particular convention for author ordering. They will list authors in whatever order is on the submitted manuscript. The alphabetical order convention is entirely up to the community of authors.

@ÉbeIsaac: the questioner has explicitly stated that alphabetical order is the standard in their field. It seems that even the existence of such a convention is new to you, which is fine. But it's probably a reason to refrain from giving advice on a situation you're so unfamiliar with, rather than assuming that everyone does or should operate the way you're used to.

I think you should be honored and flattered by your professor's suggestion. He/she thinks you deserve a larger share of the credit for the paper and is trying to help you get that credit.
Your concern about the CV/publication list is somewhat valid, but there is a simple way to address that issue. In your publication list, put a small asterisk next to the paper in question, accompanied by a footnote saying: "For this publication my coauthors agreed to name me first author as credit for my contributions. In all other publications my name appears in alphabetical order, as is the custom in my discipline and indicating roughly equal contributions from all coauthors."
Of course, adding such a footnote is a bit tedious and awkward, and the benefit you will get from being first author may not be very large. Perhaps you may conclude that it's not worth the hassle. Or you may feel it could distract people looking at your CV from other things about you that you feel are more important. Or you may think this is just too trivial an issue to care about. All of those would be completely reasonable reasons not to accept the offer. But there would certainly be nothing wrong with accepting it.

A less awkward implementation: you could have two footnotes, "†Authors ordered alphabetically" and "*Authors ordered by contribution", and for each author list, append either † or * as appropriate.

Or simply, at the top of your list of publications, have a note that states "In my field, the convention is to list authors in alphabetic order. Deviations from this happen only in exceptional cases."


text:
How can someone with a large and unique data set, but limited time to write articles, find experienced researchers to co-author papers with?
I have a very significant number of publishable projects and have a  very large data set that can be used to publish dozens of articles. I have published a number of top peer reviewed journal papers but do not have the time and resources to do this myself and am looking for research collaborators who are experienced in writing Computer Science or Social Science journal articles to co-author these research papers with me.
How can I find such researchers? Are there any online resources that might guide me in this process?

Good researchers typically have their own projects and ideas to work on. Why would they want to spend time writing up your projects, rather than doing research on things that interest them?

To add to the comment by @ff524: You want to find researchers to do this and that for you. What do you give these researchers? What makes your data/projects publishable? It seems that you haven't published (or you would know how to transform data into publications; and from there, to recruit collaborators), so you should ask yourself how you can correctly estimate the quality of the contribution you have to offer in this "cooperation". Not saying it is not there, but it clearly needs a reality check. Businesspeople, for instance, often misjudge the motivations of scientists to write papers.

@ff524 because if they saw the scope of my data set which is huge and unique with many areas of application they would literally run to have access to it.

And yet... you aren't being bombarded with requests to work with your data, so maybe it's not as interesting to other researchers as you think.

@ff524 because I haven't reached out to potentially interested researchers so they don't know what I have..which is the whole point of my question.

@ff524 fwiw the two academic colleagues who know about my data set are extremely interested in accessing it. But I'm looking for researchers who have the time and resources and experience to publish papers in short order. Many academics are stretched too thin to devote the time needed.

That was exactly my point: researchers are typically very busy working on their own projects, and are not interested in devoting time and resources to working on other people's projects. I'm afraid the thing you are looking for - an experienced and talented researcher with lots of free time to work on other people's projects - doesn't really exist.

@ff524 so you think that no matter how significant the data set and publishable results no researcher would give it a second thought? Hmm

There is a difference between "finding good people to collaborate with on a project of mutual interest, with members of the collaboration having the opportunity to contribute to all parts of the research, including: conception, execution, and writing", and "finding people to write up my data in exchange for co-authorship." The former is plausible, and having a good data set to offer will help. But what you're asking sounds an awful lot like the latter, which you are much less likely to find.

@ff524 so you're saying it had to be presented the right way. Makes sense. Thanks

Regardless of how you present it, if you are just looking for someone to write up your ideas and your work, it's not likely to attract anybody. I'm a researcher because I like having interesting ideas, doing the research behind them, then writing it up. If I just wanted to write about things other people came up with, I'd be a science reporter instead.

We have had a very similar question before, can't find it right now. The answer was that there are no Tinder-like services to match eager researchers with lonely datasets.

I'm looking for researchers who have the time You will find academics, particularly in some corners of the world, who are desperate to publish anything because that's a prerequisite to obtain government jobs and privileges. These people couldn't care less about the content of the paper and will welcome any collaboration. Don't expect brilliant results though.

Possible duplicate of How do researchers from different institutions get to know each other to begin a collaboration?

How good is your data?  If you truly have amazing unpublished data that could readily form the basis for multiple top tier publications, then with some persistence you should be able to find academics who are interested in writing publications based on that data.
However, are you able to truly judge the suitability of your data for academic publications? In general, if you are not in academia, you are likely to have a poor to modest understanding of what makes a great dataset. For example, I sometimes work with industry datasets that look at the characteristics of workers (traits, job performance and so on). There are all sorts of academic reasons why a given dataset (that the consultant may think is interesting) will not yield a good publication: e.g., the sample size is too small; there were issues with data collection; the measures used are insufficiently reliable or valid; the measures used do not map onto the theoretical constructs in the literature; the questions that can answered by the dataset have been examined many times already in the literature; the meta data is incomplete.
Even if you have a dataset that could be used for a publication, there is still the issue of whether the dataset is good enough to persuade an academic to work on it. In general, more established academics have a large collection of studies and datasets sitting around waiting to be potentially written up. In this case, the academic is likely to strategically prioritise their publications in terms of some sort of effort-reward trade-off. So, your dataset needs to not only be publishable but sufficiently aligned and interesting to an academic to persuade them to work on it.
Thus, my advice would be to start to connect with a few academics in order to assess how suitable the datasets are for publication.
How to find a suitable academic? There are many strategies for finding a  academic who might be interested in analysing your data for publication. Ideally, you'd have a sense of the kinds of publications that could be obtained from the data, and therefore which academics are working on these topics. A targeted email to some of these people explaining the data you have and your interests in sharing for publication should help to start a conversation. You can then have skype chat or face to face chats (with people in your city). If you're lucky you might have existing social networks with existing academics (e.g., where you did your studies, or in your town).
As you interact with these academics, you can get a better understanding of whether the data is suitable for publication. Also, if you have good data, but the first few academics are not interested in it, such academics may be able to refer you to other academics who are interested.
You also mention that you want "experienced researchers" to write the papers. In this regard, there is a trade-off. More high-profile researchers are more capable of getting quality work published. However, they also have many more opportunities. So there is a trade-off. For example, early career researchers may be more interested in your data, if they've had less time to obtain a broad range of research options. Your dataset might also be useful to form part of a PhD project.
How to negotiate co-authorship? Based on respected criteria for co-authorship (E.g., here), merely providing data is not sufficient to warrant co-authorship. You generally need to contribute to some degree to the writing and conception of the paper. That said, if you make co-authorship a condition of sharing your data, then academics are free to take or leave your proposal. If collecting the data in itself involved some degree of academic contribution and you also agree to contribute to the writing and analysis process (even if not leading), then such a proposal is likely to be more palatable to academics.

I work in DC, where there's a large concentration of both researchers and places (government offices, think tanks, foundations and such) that generate data.  One thing I've noted repeatedly is that, outside of the obvious places like the Fed and World Bank, where everyone goes for data, every office in the city has data they wish people would work with.
So your condition isn't particularly unique.
The strategy I see most often used is to try and contact academic departments directly, with the goal of reaching PhD students.  The (econ) departments I'm familiar with are often happy to forward opportunities to their researchers, and PhD students as a rule are out hunting for research projects.  My department regularly has guest speakers that come to talk about data and projects that need researchers.  This would be a particularly fruitful strategy if you're offering to work with them on the project, but it should work even if not - PhD students can still take projects based on your data to their advisers to work with.

I like this idea of providing data and topics to students.  We could go further, and suggest that the OP get some funding and some PhD students and postdocs of his own.

That's true. Everybody has tons of data these days, many people think that their data is unique and interesting. I, for one, do not need any more data…

Publish a detailed description of your dataset, say, on your website, along with your offer+terms to share it for people to work with you on it. You don't need to disclose the data for that.
If it's as good as you're telling us, people will spread the word to the relevant mailing lists, and run to you. You can start by telling the few people you already worked with, and ask them to spread the word. If the data is that good, they'll share the memo until someone wants to work with you on it.
Then also, as comments point out, maybe the dataset it not interesting enough to hijack the attention of busy people.

Excellent idea. Open declaration of the expected conditions makes sense; however, keep in mind that, unless the data acquisition had considerable contribution by you, people will consider authorship as not warranted.

Indeed, if OP intends to have so little part in the work as not to deserve co-authorship, expectations shall be discussed beforehand. I assume he's genuinely looking for collaborators, as he states, rather than ghostwriters, as we all seem to understand.

authors: Диана Багаева


text:
What can be done about a terrible galley proof of my accepted paper, which would be a lot of effort to correct?
Recently my paper was accepted in one of the best computer science journals in Elsevier. In less than 12 hours, I received the galley proof of the paper. Initially, I was surprised at the speed of conversion of the paper into the final format, but when I saw the paper, it turned out to be in a terrible condition.
I wrote my paper with LaTeX; it appears to have been converted to Word, and the typesetting of all of my equations is ruined. The arrangement of figures, algorithms, and tables was OK in pre-print, in a one-column format; now, it has become a terrible mess.
What can I do now?

Email to the publishing editor to reconsider the final preparation of
the article?
Prepare the paper in a two-column format myself and resend it to the journal's office?
Ask the publisher to publish the paper typeset in LaTeX?

@NateEldredge All accepted papers are cute and fluffy (to their parents... er.. authors at the very least), but unfortunately I have still to find out how to make them reproduce rapidly.

@NateEldredge In my view just transformed to word in two-column format without any other modification:)

I'd be inclined to send an email to the editor who accepted the paper, explaining the problem and asking what I should do.  If (as often happens) the production staff gave me a tight deadline for correcting the proofs, then I'd also ask for more time if whatever the editor suggests would take too long.

In such extreme case (which, by the way, are disturbingly frequent at larger, very expensive commercial publishers) I recommend sending an e-mail to the editor, including the proofs to show that you are not talking about a misplaced comma, and simultaneously answering the production that you do not allow them to proceed. Beware not to sign or click anything that might make them able to pretend you allowed them to send the paper into production (as it would be much more difficult to get things right then). Ideally (i.e. if you are not forced through a web platform) you would e-mail the production team with copy to the Editor in Chief or handling editor for the record, or the other way round.
I don't recommend trying to fix manually an absurd amount of problems in Galley proofs: 1. Many libraries put a lot of money to pay for this work, you and your institution should not pay with your time; 2. you are bound to miss some of the bad stuff, half your corrections might be ignored, and new problems could be added after galley proofs (I would not believe that if I had not seen it myself -- my first paper actually).
I don't recommend sending a new version and suggesting they start back from that. It is pretty unpredictable what they could imagine doing, and if they use the same process again it will not be pretty.
Any diligent Editor in chief, when receiving your e-mail, should reach out to the publisher's management to have things sorted out. Without management involvement, the production team will probably do what they are probably asked to by default: treat things quick and dirty.
Last, I hope you put your preprint on the arXiv. If you did, you can at least have this as a version of record (and Elsevier allows you to update it to a postprint, including referees' remarks). If you did not already put the preprint on the arXiv, then Elsevier's policies are too complicated for me to understand and remember.

They sent you a proof, so review it and respond with the things you would like to change. Don't send a "camera ready" version with how you wished it would look like. The paper was most likely re-typed in whichever typesetting software they use so that would be useless.
Your question seem to indicate you are mostly dissatisfied with the cosmetics of the paper, that's the journal's responsibility. Simply focus on issues where the content is altered in a way that might change the meaning of what you wrote.

The issue is that there are huge amounts of changes. Marking up huge amounts of changes takes a huge amount of time, which is not time well spent.

@DavidRichery And in particular, if the time has already been invested.