text:
Are datasets really random for Bioinformatics tools
For my masters thesis, I needed a to run a tool to filter out RNA sequences and then research on the filtered data. I researched online reading papers to find an appropriate tool and I found one that really fitted my needs. The tool was published by a renowned group in a good journal (IF around 7). The tool claimed to have been run on a 30 random datasets and claimed to have around 90% specificity and sensitivity. I was very happy.
When I ran it on my dataset, the tool produced so many false positives and false negetives. Its accuracy was less than 10%. I ran on other datasets too and never got an accuracy of more than 20% in any one of them. To my surprise, the tool only worked well on the dataset that was published with the tool, with a good accuracy and no other input. I cannot believe that the dataset was randomly chosen.
I spoke to my PI about this and he said that he understands. He allowed me to choose another topic for my thesis and helped me a lot to complete. In the end I could successfully defend my thesis.
My questions are:

Is this common for groups (esp in Bioinformatics) to select a biased dataset as input and claim it to be random?
Does reviewers really run the tools (or read the source code) that are accompanied by the paper in order to test it and see whether it really does what it claims or they only read the paper?
Can an end user later write to the editor and let him know that the tool published in their journal isn't what it was suppose to be?

Please note that on StackExchange sites it's common to not "chain" multiple questions into one super-question, but ask them separately (see: http://meta.stackexchange.com/questions/39223/one-post-with-multiple-questions-or-multiple-posts)

Can an end user later write to the editor and let him know that the tool published in their journal isn't what it was suppose to be? yes, and you should if you suspect foul play (cherry picking). Talk this over with your advisor, first.

Have you asked the authors of original paper for more details (regarding how their dataset was chosen, how to apply their tool, and what its limitations are)?  It sounds like you suspect them of something between incompetence and fraud, and it would be best to investigate their side before accusing them of anything.

The tool may have worked well on that dataset because it only works well on datasets with certain characteristics. Did the paper in question make no reference to what qualities a dataset would have to possess for the tool to perform well? Unfortunately, such tools can have a lot of specificity - so one has to know when it is appropriate to use a certain tool.

You do not mention the tool or anything about your data, but in case the tool was built to work on prokaryotic data and you have eukaryotic or the other way round this might be a problem.

What you are describing here is an in silico form of Cherry Picking Fallacy. Before answering your questions specifically, I would like to point out that before assuming bad intent or foul play, consider the biological variability in the datasets, and difficulties writing generalised solutions to biological problems. Take it from a 3rd year grad student and a bioinformatician in training, it's not straight-forward to develop tools for biological datasets. There are both biological and technical challenges in writing "foolproof" software in bioinformatics.
Consider for instance, that there are multiple ways to estimate error and false discovery rate (FDR). There are also varying levels of thresholds used in different labs. Any heuristic value, if hardcoded, might alter the results.
Similarly the biological diversity, as well as technical variability introduced by wetlab benchwork will likely have serious influence on the datasets. Once everything boils down to a large table of numbers, all of that variability is implicit and thus often forgotten.
That said, my answers to your question(s):

No it's not common to pick datasets just to show that your model/software works, at least not publicly. That would go against scientific rigour and ethics. At the same time, you start from what you have available. I typically start with datasets we have in the lab, when I start developing a new tool. But in order to publish your method, you should typically show that it works on data from independent sources, or alternative data types etc. All in all, your results might not be representative of all datasets out there.
It is impossible to answer this question, factually. I am sure there are reviewers that take a look at raw data and/or source code, but it's unlikely that any reviewer actually digs into the source code, line by line, to see if everything checks out. But I am also certain that there are some who just see if you have actually made the source code or test datasets available or not, without actually look at them.
You can always do that, end-user or not. But beware, before you go blowing the whistle on someone else's handiwork (which probably took them a significant amount of time and money) you better get your facts straight. Claiming that they are selling false results or simply misinterpreted their results is a very serious accusation, especially if the authors are renown and respected in the field.

Prior to writing a letter to the editor (who AFAIK have a duty of publishing scrutinising critique on something that has been published in that journal) you should check, and double check, your results. Make sure you are not misunderstanding the statements made in the original work. Make sure you are not misunderstanding your own results. Make sure what you have observed is not a side-effect of your datasets or your own use of the software, try to talk to other people who have used the software. And make sure, your supervisor (and others in the group) are behind you, get your own results investigated by your superiors and colleagues.
If everything checks out you can, and should, inform the journal that the results in the original article might not reflect a general reality, and that you could not replicate the results on independent datasets, then show your results.

As well, I would suggest it's probably worth while to get in touch with the original authors if you believe their system is faulty after more investigation.  It could simply be a misunderstanding of the properties the data needs for the tool to be applicable -- bad writing/editing can really destroy the meaning of technical terms sometimes!


text:
Bulk download of arXiv (or other publication data set) with metadata AND citations
This is to extend the question:
Getting a dump of arXiv metadata
I would like to download arXiv in bulk, and found a useful answer above for using this syntax:
http://export.arxiv.org/oai2?verb=ListRecords&set=math&from=2015-01-01&until=2015-01-31&metadataPrefix=arXiv

However, in data there is not a field for citations.
I wonder if there is another dataset.
In
https://arxiv.org/help/bulk_data
there is no description about citations, apart of an old dataset (KDD) of 2003.
Is there any parameter to extend the above query and also fetch citations?
Is there any reference to see how arXiv metadata are classified?
Any other dataset of articles possible to freely download?
I would like to use a dump for testing visualizations for knowledge discovery. If there is already a sample DB standardized / polished, it will be really helpful not to have to polish / curate data.

This answer might be related to the second part of your question (getting citation counts to articles), but it does not allow for bulk downloads of citations.

What do you mean by "see how arxiv metadata are classified"?

arXiv's metadata does not include any citation information.
If you are interested in citations in arXiv documents, your best bet is probably to extract them from the PDF files using a dedicated tool (Cermine, pdfx or pdfextract for instance), or from the LaTeX sources, by inspecting the .aux files.
You can download PDF or source files using the dedicated bulk data interface. Using the OAI interface for this purpose is not recommended as it generates a lot of traffic on the main arXiv site.
You could extract a similar dataset from PubMed Central using their API. If you are looking for something broader (not restricted to a particular topic), you could use COnnecting REpositories (they have full texts from a variety of sources, with an API).
You can also use a pre-curated dataset such as the Microsoft Academic Graph.

Thank you for CORE - did not know that. Are you saying that to extract metadata of citations one need to parse pdf by pdf ? It would be quite of a work for standardizing all the elements and typos and such. I thought citations may have been already cleaned by publishers. Is it not available publicly in bulk (e.g. at least, one can download patents corpus from google in xML, but at least XML not pdf!)

If you are looking for a clean dataset of citations, have a look at the last sentence of my answer :-)

Since recently, arXiv citation information is available from Semantic Scholar -- it is the one that is displayed on the arXiv website.
Semantic Scholar apparently provides a dump of their data.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Is it professional for a professor to ask "surprise" questions on a test?
To clarify, I am asking if it's professional to ask questions that while relevant to the subject/course, and are related to the topic, but have not been discussed in class, assigned as homework, reading, etc. and are also not related to any prerequisite class that the students should already know. I am also not asking about "gotcha" questions where it's a quick "know it or not" fact, but rather an entire procedure, proof, or concept of some sort.
For example in class, using a formula from the textbook to solve problems, but on the exam asking for a proof of the formula that has been used.
Another example - in a foreign language class, asking about a never-before-seen word that may be related to some other, studied words or that has a similar sound/meaning in the native language.
Do such questions "make students think out of the box", or are they more likely to discourage students when they struggle with them? While in the real world there may be many problems similar to this where the problem is completely new, is this something that should be taught in an unrelated class?
Does it matter if the test in question will be graded on a curve where even if a student doesn't answer the question correctly at all, they could still get a good grade?
P.S. - this question was prompted by the comments in this answer

Since I'm the one who prompted the question, I'll surely give a detailed answer later on, but let me firstly comment on the adjective "professional". The professionality is not really related to administering "surprise" questions, it's only related to the way you ask them and with which aim. It's unprofessional to ask "surprise" questions with the only purpose of failing as many students as possible. Of course, I consider the way I ask surprise questions as professional, and I'll try to expand on this with an answer.

At Edinburgh University Physics, at least 20% of the exam has to be unseen material; but the pass grade is 40%. Some put higher percentages, though.

I consider all exams that do not have this property -- i.e. that they require the examinee to display independent, creative thought based on the course material -- broken. Checking only memorisation and calculation skills is beneath what (technical) university degrees (should) stand for. That is, a professional professor should ask such questions (in order to save the world from rote-learning "engineers")!

Just a thought on the meaning of the word surprise: I'd distinguish between testing insight and ability to transfer and apply the knowledge and methods learned in the course (which covers your example, but which I'd expect in an exam) and suprises like asking about biology on the mars in beginner level biology course (assuming astro biology wasn't in teh curriculum, would require detail knowledge about totally other subjects) or even "give an alternative to the proof we had in the last lecture" (= unprofessional attempt to punish students for absence)

@MassimoOrtolano - I am not sure if the words professional and surprise are the correct ones, but I hope I explained what I was asking for better in the question that the title. If not, feel free to edit it as appropriate, and thanks for clarifying what you meant. I think I also have a bias in that I believe that the homework problems should always be harder than those on an exam, as the students have more time/resources to work on them and as such can solve harder problems than during the limited, stressful exam time, but that may be yet another question.

When I started teaching, I thought these sorts of questions would be beneficial to separate those students that had truly understood the material from those that could merely reproduce solutions similar to homework problems. I quickly found that the student that could encounter a new application of the material and discover a correct solution within the time constraints of a typical undergraduate exam were far to rare for this to be useful, given how demoralizing such questions were to most students.

In general, students strongly dislike getting test problems on topics they (believe) have not been exposed to. They focus on getting good grades. Learning is a secondary goal in most cases.

@Davidmh The physics department of Harvey Mudd College in California has the same policy (most likely adapted from Edinburgh, I now imagine), and it has long been my favorite grading scheme of all those I have run into, and that's as a (former) student.

Just don't be this professor: http://s2.quickmeme.com/img/a1/a1c68c93a41960bcf8646bac8b46c81cdfa695a4221b960958801eb14f827dd3.jpg

For comparison, even if it is not exactly within the scope of this site: In Austrian high schools, the grades "A" and "B" are defined as being able to answer such questions. Only being able to do what was explicitly covered in coursework would be a "C" or less.

This issue is in the news today: http://www.bbc.co.uk/news/education-31057005

Among the answers given thus far, the one which is closest to my thinking is that of xLeitix, and the side note about the context applies too.
Now,

Is it professional for a professor to ask “surprise” questions on a test?

It can be professional. As I said in a comment, it's unprofessional to ask such questions with the only purpose of failing as many students as possible.
Moreover, I don't like very much the adjective surprise in the title: a surprise is something unexpected, but if a professor clearly warns the students that at the exam they will find problems which have not been solved during the course, there is no surprise. So, in the following, I will talk about "new" problems.
Exams and tests have, as their main goal, that of assessing how well students master/understand a (small portion of a) certain subject. As others have well explained, new questions or problems can give a hint on how deep this understanding is.
But apart from the above stated main goal, exams and tests might also have secondary goals:

An exam can be an occasion for learning new stuff. The well defined separation between learning and verification, which typically happens in a course, is something that drastically comes to an end when one starts working, even in academia. Learning and verification in everyday life are really interleaved, and many times learning has to be done along a stressful verification. So, a new problem during an exam can be an occasion to continue learning in a more "unprotected" way.
An exam can be a hint, one of the many, that what has been taught during the lessons is not the whole story, and that beyond the lessons there is much more: new problems surely deliver this message.
For a professor, an exam is an occasion to fish for good students to whom propose a thesis. Giving new problems can be a way to find students who are capable of independent thinking.

So, I tend to give new problems at the exams keeping in mind the above points.
To avoid being too general, let's make an example related to my experience. A few years ago I taught a course about sensors, transducers and signal conditioning circuits for graduate electronic engineers. The written part of the exam consisted in one problems about designing or analyzing a signal conditioning circuit or about evaluating the measurement uncertainty of certain transducer. Due to the vastness of the topic, the course could neither describe all kind of sensors and transducers, nor all possible signal conditioning circuits. So, I decided that every exam would have been made of a new problem, where "new" meant:

A problem about a transducer not described in the course. Indeed the exam text contained a short description of this kind of transducer.
A problem about the analysis and/or design of signal conditioning circuit not described in the course. The students, being electronic engineers, were expected to know how to analyze electronic circuits, even of moderate complexity. In more difficult cases, hints were provided.
A problem about a known transducer applied in an unknown way.

Exams were open books and students could bring the solutions of all previous exams and all the class notes. After the written part, if successful, there was an oral examination which was more about class material.
What was the outcome of this kind of exam? The course was in general very well received by the students, even if the exam was considered hard: the percentage of success was around 30% (the pass grade is 60%). The major complaint was about the number of exercises solved during the classes, but this happens in all kind of courses. My answer to this complaint was that there were, indeed, time constraints that prevented us to solve more problems but, anyway, whatever the number of problems solved during the course, at the exam they would have found a new one (sometimes students ask for more solved problems in the hope that these will exhaust all possible cases).
From this and other experiences along 15 years, I think that students can withstand new problems at the exams as long as the motivations are well explained and, especially, as long as the course is worth of it.

I don't see a reasonable answer beyond "it depends."  It depends on the question: some surprise questions are not actually so difficult and could reasonably be asked on a test with no special preparation, while others are very difficult indeed.  It also depends on the students: you can demand more of experienced and talented students than you can of typical beginners.
There's certainly no rule that says you can only ask test questions that are similar to previous questions the students have seen.  Sometimes asking unusual questions can be an excellent way to judge how well students have mastered the material.  At the same time, test questions that are too unfamiliar or difficult can be unproductive.  This is a balancing act that can be solved in many different ways, depending on the style of the person writing the test.

What would you suggest as the boundary of "too difficult" - would it be when over half the class does not get the concept? how would you handle a situation where a student who got it wrong comes up to you after the test and says you never taught it (well enough)?

@user2813274: both "too difficult" and "never taught it" are very relative concepts. As an extextreme example,  I've had a student complain that my midterm was nothing like the assignments,  when that particular midterm had 3 out of 5 questions taken verbatim from the assignments.

Some of the best questions in my undergrad math courses were 'surprise' questions in that they could be answered in a few lines by people who truly understood the concepts and the theorems, or in a few pages by those who didn't.  I was usually in the latter category, but I think this is a good way to go.

If a professor chooses to do so, it can be helpful to explain before the test that a question or two will go beyond the mere facts discussed in class and expect an application.  Good tests have question(s) to distinguish A students from A+ students.  However these questions need to be prepared very carefully.  Has the student seen all the relevant theorems they will need?  Consider trying it out in quizzes or as a bonus question first.

Every question on a test should be about the material in the course.  Many times, however, the professor may be trying to teach a deeper concept than some of the students have learned.  This is what creates a "surprise" question: the professor asks something that requires mastery of the material or insight into its deeper meaning, and the student has only learned material to a relatively shallow level.
For example, when I was a TA for a large undergraduate artificial intelligence class, the class taught two things simultaneously. The underlying concept threading through the whole class was how to think about data representation and problem decomposition.  As part of teaching this, the students were also taught a number of standard AI algorithms.  The tests then typically involved variant algorithms the students had never seen before.  Weak students, who had learned the standard algorithms but not the underlying concept would often do badly and complain about the "surprise" questions, since they were being asked about an algorithm that they had never seen before.  Strong students, who were learning the underlying concept, had no problems.
In general, then, encountering a "surprise" question means that the student is failing to learn the deeper concepts that the professor is trying to convey. Where the pedagogical problems lies, the professor or the students, is a completely different question...

So in short, you're saying "learn the subject, not the material" - which I hope most people in education and academia would agree with!

@G.Bach Also, "teach the subject, not the material": pedagogy can fail from the professor's side as well...

Looking at it from a student's point of view: The homework is the student's opportunity to verify that they have a sufficiently deep understanding of the material. If the exam contains "surprise" questions that require a deeper understanding than the homework did, then how is the student supposed to realise that they need additional study, or at least more focussed study?

@mhwombat The homework may well have pushed for deeper understanding without this fact ever being noticed by a student who is middling along on partial credit and help from friends...

@mhwombat: I think the point is that the best students studied everything they could in the first place, rather than doing the bare minimum just to pass some test.

I almost failed my first degree by arrogantly "learning the subject" only.    During my masters, I thought "to hell with it" and gamed the system as much as possibly, getting >90% in several assignments and finishing considerably better than my first degree.    There was one professor who I really hated (and still do) but I have a lot of respect for him as 60% of his exam was split between two questions which were each basically "here's an equation, solve it".  No amount of studying past papers and memorizing assignment formats would help gamers with that question!

I guess it depends on what you call a "surprise question". Usually, when you design tests, you don't want all questions to be the same difficulty. Rather, you would want to have a number of basic questions to find out who actually did not "get" the fundamental messages (and should hence fail), some intermediary questions which the majority of students will be able to do if they studied, and a small number of challenging problems, which are there to separate the excellent from the good students.
In my tests, "surprise" questions often form the "challenging" part of the test. I write them in the full expectation that only 10% to 20% of the class will be able to do them, but that is ok - not the entire class should have the best grade anyway. This way, me and others know after the course who the students that really understood the material were, and who just studied a lot.
Sidenote: I teach in a european country where it is usual to have a Gaussian distribution over the entire grading spectrum - it is not like in the US where having a "B" is often already considered a bad grade. Also, at least in  undergrad courses, it is not uncommon that more students fail than have the best grade.
What makes "surprise" questions difficult for some students and attractive for many teachers is that they actually test understanding, transfer skills, and the ability to apply knowledge as opposed to mechanical memorisation of pre-learned procedures. This is easy to see in your "formula" example. A student that studied can apply the formula (he knows how it works, and how to apply it, and under which conditions), but only a student who really grasps the math behind it can do a proof that they had not covered before.

This is roughly what I felt about the example given: that a mathematician by definition can construct proofs without seeing them first, and so it's legitimate in a mathematics course to use that ability to distinguish the top. If all you do is repeat proofs you've seen that's very modest mathematics. That said, I did one first-year course where we explicitly used the Divergence Theorem without proof on the basis that the proof was too difficult for the scope of that course, and I would have been most perturbed in that situation to find the exam asking for a proof ;-)

FWIW it's not true everywhere in the US that a B is considered a bad grade. In many of the (undergraduate) physics courses I've taken and TA'd, it was standard that grades would form a distribution centered on B or C. Doesn't stop the students from complaining about it though.

I have a similar method (in math at a U.S. school).  For an undergraduate exam, many questions are relatively routine, but 10 to 20 percent may require deeper mastery of the material. In my mind, this is what distinguishes an 'A' from lower grades: demonstrating an ability to go slightly beyond what they have already seen.  Of course, for freshman I don't expect as much as for senior math majors.

(https://en.wiktionary.org/wiki/European#Adjective)

In my view, a fair examination question draws on any or all of the following:

Material discussed or presented during class contact time.
Material from any of the items on the course reading list.
Core material from any prerequisite courses.
Material that might properly be regarded as common knowledge for students at this stage in their education (basic mathematics, basic use of the English Language, etc.).
Knowledge that can reasonably be derived as a logical consequence of numbers 1.–4. Here, 'reasonable' is calibrated to the level of the course. Much more should be expected of graduate students who are essentially training to do point 5. for the rest of their professional lives.

In my experience, "surprise" questions usually come about because (a) students have not fulfilled their obligation to apprise themselves of the material in 1. and 2., or because (b) students are not sufficiently capable/comfortable with the subject matter to conduct the logical deductions in 5.
In either case, my view is that it is professional to ask questions that draw on all of 1.—5. In their capacity as educators, the main professional responsibilities of university teachers are to decide upon and deliver the appropriate material clearly, and to administer assessments capable of identifying students' success in mastering this material. A question that does not 'surprise' the majority of students can only test this mastery to a limited extent because it leaves little way to differentiate students who have truly mastered the subject from those who have merely done a good job of rote memorization. I would therefore view 'surprise questions' as an essential tool in professors' fulfillment of their professional responsibility as teachers.

+1 for point 5. Questions that require the student to combine concepts that they should know in order to solve a problem are great. However, questions that require previous knowledge that has not been covered in class to answer correctly are a no-no. Making students do critical thinking is great. Testing whether they've memorized things that you haven't asked them to memorize, not so much.

My field is mathematics. I always tried to ask at least one question that looked quite different from anything that the students were certain to have seen before (though of course it relied on the relevant material), or that required them to combine several ideas that they might not previously have had to combine. On an exam in first-year calculus I’d have had at most a couple of questions of this type; on an exam in the more theoretical courses and in liberal arts mathematics courses I generally had quite a few such questions alongside the more routine ones, covering a range of difficulty. All questions, of course, required the students to write something, be it a proof, an explanation, or merely a routine calculation, and partial credit was always available.
I should point out that I was not grading to any pre-set scale. I have always preferred to construct the exam that I wanted and then interpret the results. Indeed, I refused to assign letter grades to individual exams, preferring to reserve that painful chore for the end of the course when I had as much data as I was going to get. Needless to say, I always explained all of this at the beginning of the course and again before the first exam. I also made it clear that I did not have the expectations to which most American students are used to being held: it generally worked out that the A students (apart from the rare curve-breaker) averaged 80-85% over the entire term — and I was not especially generous with A grades. A 50% average was generally a solid C.

All my real analysis tests (me being a student) were 50%+ completely new theorems to prove. It was to be expected that one would have to think outside the box to even pass the test. And I would say I learned an order of magnitude more in that class than I have in any with more standard tests. But that's a senior level course and being good at writing proofs from scratch was a skill that had been taught gradually over many lower level courses.
One could argue that being able to pass general tests with questions that require uncovered to vaguely-touched-on material or methods that involve combining techniques in ways unseen in class is the culmination of education. If that's in fact the case, it would make sense to introduce it early to cultivate the skill of being able synthesize new answers from covered material.
But it is important that it be reasonable for the student to know the prerequisite material to synthesize the answer. Don't ask a measure theory question on the first test in real analysis. Ask a question that requires use of the least upper bound property of the reals in a tricky way, for example.

This is very common for senior-level or graduate-level mathematics. It would be silly (and perhaps insulting) to ask students at this level to simply memorize proofs and replicate them on exams. The real goal in math education at that level is to learn background material while also developing the problem solving ability that is vital for research.

Teachers have been asking questions that are not know to students for as long as teaching has been around.  How else can a teacher make the student step out of their comfort zone where the answers have been clearly laid out and instead help them broaden their mental abilities.  It is important that the questions is related to the subject being taught, but if I ask you a question about the really world cases rather than the hypothetical it will require some out of the box thinking.  Yes it is professional.  It is not if they are questions that the class has not prepared the student to answer.

I prefer a system where the homework problems are the most challenging. Also, I'm  not in favor of a system where the homework is graded, because this makes it more difficult to choose good homework problems. The idea is that students learn best when they struggle a lot to solve difficult problems. One then has to accept that students may not have been able to do well on a particular problem, even if they are one of the best. Graded or not, homework should still be submitted and records should be kept about the student's performance.
The exam should serve only as a basic test that all the students who have seriously followed the course should easily pass. There is no way you can challenge the students in an exam that only lasts for a few hours as you can challenge them with homework that they would need to work on for several days.
The exam should be judged in combination with the homework. Each student's homework record (graded or not) should be taken into account when judging the exam. If it is found that the homework record is inconsistent with the exam performance, then the student should be invited to speak to the Prof. about the subject. It can be the case that the student was nervous and didn't see the solution to simple problems, such issues can be corrected in an oral examination especially if the student does not know that the meeting is in fact a secret oral examination.
It can also be the case that the student did not know much about the subject and just copied the homework assignments from other students.That will then become clear after speaking to the student, the student will then be given a failing grade for the subject.

The real test, your first assignment at your first job, in my experience is often much harder than any homework you're likely to have done. So why shouldn't a test that decides weather someone gets an engineering degree simulate at least part of this?

Yes, but this mainly happens in the usual setting where the homework exercises are not that challenging and you have a normal exam. If you consider giving a course where students have to master doing the really technical research level stuff, then the usual system doesn't work well. In fact, you should expose the students to something similar as that "assignment at your first job" that you mention. That's best done with elaborate homework assignments and note that you don't need to give each student the same homework assignment.

There's no problem with Surprise questions as long as they're a small part of the marks and you tell students that there may be such questions.
Some of my professors would structure tests along the lines of 40% basics, 40% intermediate, 10% hard, 10% Surprise.
if you covered and understood all the basics you could reliably scrape a pass.
If you'd covered and understood all the material well you could get a good mark.
If you'd gone above and beyond and mastered the material on the course and had good general knowledge in the area you could get an excellent or perfect mark.
I liked the system as it meant in later years I could generally drag students I was tutoring through the exam based on the predictable stuff and you were also rewarded for independent study.

I don't think this has anything to do with being professional, but to what extent it is fair and/or desirable so ask such questions.
To me, the utility of unexpected questions that completely surprise students  also depends on what kind of grading system they are used under, and this has not been addressed in the other answers. Under curve-grading/norm-referenced tests, one of the points of exams is to differentiate between students, so difficult surprise questions can be useful to e.g. test if students have gained a deeper understanding of a topic. Under such a system, it is reasonable (and to some extend desirable) that only a small proportion of students can answer some questions.
Under a criterion-referenced/goal oriented grading system, students are ideally supposed to know exactly what knowledge is needed to achieve a particular grade. Totally unexpected questions might be more problematic here under such grading criteria. However, what is an unexpected question is also subjective to some extent, and the learning criteria could also specifically mention fundamental understanding and the ability to apply the material to new situations. Even so, if a large proportion of students fail to understand or answer harder surprise questions this can partially be seen as a failure of the teacher/course (which not necessarily the case under curve grading), since this could indicate that the teachers have failed to convey either the required knowledge or the learning-goals needed to achive a particular grade (alternative explanations would be e.g. high goal standards or uninterested, lazy or weak students).

I see this problem spanning several levels of information:
On the bottom, we have course information that is memorized. Students should not be asked to reproduce definitions, facts, axioms, and other types of foundation information that have not already appeared in course material.
Next, we have "techniques."  Especially in mathematics, we learn techniques for approaching hard problems. In other fields, techniques manifest as the methods we use to make inference, the types of reasoning that we use to interpret new situations. These are very general (like integration by parts, the epsilon over n trick from early analysis courses, or less rigid logic like the the broad historical notion that starvation corresponds with instability which makes revolution more likely) and can be combined in many interesting ways. Plenty of good, unfamiliar problems can be written based on familiar techniques. It can be very appropriate to ask students to develop a new technique to solve a problem on a test, as long as the intuitive leap is somewhat reasonable. This is a judgement call on the part of the professor that can reflect professionalism.
The ability of students to adapt new techniques generally depends on their grasp of broad overarching concepts. Development of entirely new concepts probably does not belong on tests because students are not likely to retain them very well under exam pressure. In general, exams are a method for gauging students' current knowledge/understanding of course material. I would say that it is more equitable and appropriate to introduce entirely new concepts on homework and in lecture, so that students have a greater opportunity to internalize them.

What is the problem with introducing a new definition in an exam? This can be a great way to test a students understanding of the material.

I should clarify. I mean that students should not be asked to define a term that is new to the course. It does make perfect sense to introduce a new definition and then use that definition as part of a problem.

My answer is a bit biased in favour of students since I had a bad experience with such questions.
TL;DR; 'Surprise' questions are good and necessary in some cases but don't make them weight 50% of exam.
While I was a student we had a couple of professors who would start giving you surprise questions when you do relatively good in your examination (oral). Then they could find something you don't know and seriously decrease your overall result or even fail you.
There were some professors who have made surprise questions in written exams grading them as 40-50% of the test itself.
I personally hate this. In my case it had led to the situation when you try not to learn general course material but to anticipate surprise questions. Student have more than one course at the same time and sometimes there is not enough time (or interest) to have deep(out of program) understanding of all courses and you just want to pass the course with 75% grade or whatsoever.
It of course depends on the field of study. If it is something like theoretical physics it is necessary to have skills to think outside of the box.
I liked an approach of one of my school teachers. You could get 110-115% on the exam and the grade you receive is based on 100%. 90% you could get by regular questions and 20-25% by 'surprise' questions. So if you studied diligently you could have a good grade and if you have spent more time on the subject you could even cover minor issues with 'surprise' questions.

I guess it depends on the details, the country (different education culture) and subject (some of them are fact heavy, some of them are more problem solving focused).
I happened to come from a place where a student who can solve only problems in STEM field that explicitly showed in lecture or assigned as reading, considered to be rather mediocre (C level). I assume there are different educational approaches, too.
On the other hand, giving a question way out of blue can be off limit and just mean.

"New" material can be tested for on an exam if the students have access to or have been taught materials that are sufficient to make sense of the "new" material. This can apply to the following scenarios:

Open-book tests. In an open-book test, it is fair to test students on material that they are capable of looking up on the spot and synthesizing with the knowledge they have gained in the course or were required to have in order to meet prerequisites. For example, in a test of French grammar where the students are provided access to a basic dictionary, it is fair to expect the students to look up a word and identify its grammatical and morphological categories despite the fact that the word never appeared in any of the lectures or required reading. Make it clear to your students which books/materials they should bring to the exam. Similarly, if your course is "Open Google", you can include a question that requires the student to figure out "Will a Hyperbarkonian Dehydrogenated Redonkulator output data that meets the following specification...?" as long as your course has taught the students how to read a machine data output specification and compare it to a requirements specification.
Material that can be deduced from context. These are things where students can apply context clues in connection with course material to identify the likely meaning of the new material. An obvious, well-known example of this is reading comprehension through context clues, a technique commonly taught in lower grades. For example, consider the following question:

After Mary found out about the death of her father, she blargragathed for three hours until she fell asleep from exhaustion. Which of the following counseling interventions is most likely to assist her?...

If your reading comprehension skills are sufficient, you can probably figure out that blargragathing is a kind of negative emotional or behavioral reaction despite the fact that I made up the word on the spot for this answer. You can now apply the counseling interventions you learned in my course to answer the question.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Are there online resources for finding information on M.S. programs that fit my criteria, without having to go through every school's website?
I'm looking for an M.S. in Computer Science and I found it difficult to properly navigate all the available degrees. I'm interested in a particular focus (computer graphics, not modelling and that kind of stuff but how to do use algorithms in order to create graphics). I also have some constraints on language: I currently speak both Dutch and English, so no German or French master's for me.
There are other questions on this site about finding graduate programs and their curriculums, but I'm mainly looking for quick preselection (like a top 30 of options I have). I'm looking for a overview of courses as it enables quick elimination which if I have to wait for each uni to answer will take quite a long time
Is there any website or other online resource that will allow me to filter all M.S. programs by my requirements (language or country, research focus) and then find out information (such as the list of courses that comprise the degree) about the programs that meet my criteria, all in one place?

(Edited based on extended discussion in comments, which I have since cleaned up)

The most useful I found (which also helped me finding one for me) is Find a masters.
It does let you select a country of your choice, provides a summary of program where courses list, research focus and other things important are mentioned.
It also lets you filter by program type (Dip, Cert or degree), by study plan (part time, full time etc) and more.
Lastly, it also provides a link to university website so you can go there easily for complete details if you like the description.

Have a look at this site study in us
They are still in early stages but have most of the information you need.
You can filter the colleges based on your requirements and find more details about each college.


text:
I have to leave my PhD program with a Masters degree. What are my options?
I started out in one of the best PhD programs in Ecology and Evolutionary Biology (at an American university) two-years ago. I took my qualifying exams last month, which I could not clear (twice) and will have to leave with a Masters degree instead. I do not have a good relationship with my advisor and was having a hard time coming up with a convincing "story-line" for my thesis, so the outcome is not all that surprising... but I am certainly finding it incredibly difficult to look for non-academic opportunities for myself.
I don't want to try getting into another PhD program because my experience here was incredibly stressful, and took a toll on my mental health. And because I spent the last 8.5 years following my research interests, my profile isn't well suited for industrial jobs either. I do have a knack for coding though, but I don't have necessary experience/ coursework to account for it.
I have been thinking about my options a lot lately, and am hoping to get advice from anyone who has gone through/ seen someone get through a similar situation as mine.

If your institution subscribes to it, you may find interesting info at versitilephd.com. You should also consult the career services office in your school or college.

Well what can you do? There's only one option and that is to find a job, right? Use your university's career center if they have one, use your network, use your local job portals. See my answer to a similar question here: https://academia.stackexchange.com/questions/101858/what-to-do-after-failing-out-of-grad-school/101861#101861

I think this question is not really well versed for academia.SE, because it is more of a "What kind of non-academic jobs for a Masters degree holder in Ecology and Evolutionary Biology are there?" Correct me if I am wrong. From a bioinformaticians perspective I have to say though: There are quite a bit of people in your and related fields with a "knack for coding" now, so to make this a competitive feature I would definitely try to get something to account for it. Put up your projects on GitHub, at least.

@Dawn I wasn't aware of that website - while my school doesn't subscribe to it, it does look like a good place to start.

@Allure, Thanks for taking the time. I have set up an appointment with the career centre

@skymningen Thanks for your insight. I'll probably get a few certifications and increase my job prospects. I have projects on GitHub but don't know if those would be sufficient to land a job.


text:
How does one cite a book series/sequence?
I have a need to cite a certain scifi book series (i.e. a collection of books functioning as elements in a single larger story, not a series like the Springer Graduate Texts in Math) as inspiring a thought experiment in a philosophy paper, but I’m having trouble finding advice on how to cite a book series rather than a particular book in the series.
At least in this case it doesn’t seem appropriate to pick any particular book in the series (the relevant conceit is present across the whole series, but it’s quite possible some books in the series don’t mention it at all and I don’t even know which of the books I read).
Also, any suggestions for representing a series in BibTeX?

It seems natural to question whether you can cite a book series. That really depends on your definition of a citation. Some definitions might include book series, others might not. If a definition permits book series, then I'm unsure how to cite (citing a particular book is easy, just use BibTeX book entry with field series), otherwise, such citations aren't permitted, but I nevertheless have a workaround: refer to the book series in the main body, omit a bibtex entry.

In what way could you not cite a book series?  You can cite collected works or other groupings of independent sub-works joined by some common organizational structure.   One can also cite multi-volume books even though they span multiple physical chunks so why wouldn't you.  Regardless of semantics you understand what I wish to communicate to the reader:  this information came from this series but doesn't belong particularly to any one instance more than the others.

Have you considered not citing the series but explaining it in a footnote? E.g. "This idea was inspired by the Xeelee Sequence, a series of science fiction novels and short stories by Stephen Baxter. For the complete list of works see http://www.stephen-baxter.com/articles.html#xeelee" It's a workaround, not a full answer.

@PeterGerdes You can only cite something that's citable. Is a book series citable? You can cite collected works (incollection). Whether you can cite "other groupings of independent sub-works joined by some common organizational structure" depends on the structure. I'd argue that you cannot cite multi-volume books, you can only cite each volume individually, because they are distinct. I'm not questioning your question... I'm merely trying to provide an answer. I only know how to provide an answer, well workaround, if book series aren't citable. Henning has seconded my workaround

The Purdue MLA site provides a recommendation for how to cite an entire television series, which is essentially identical to citing a unified work: instead of naming an episode, one names the series as a whole, and the date shifts from a single point to a range.
By analogy, then, to cite a book series, one should simply use the series name as the book title, and list the range of publication dates, e.g.,
@book{LOTR,
 author = {J.R.R. Tolkien},
 title = {The Lord of the Rings},
 year {1954--1956},
 ...


text:
How much unfinished work and unexplored ideas should one share in a presentation?
Background: I am a fourth year math graduate student working in differential geometry in the US.
I'm currently preparing a presentation for a graduate seminar at my university, and today I met with my research advisor to go through my outline.  Shortly into our discussion, he became rather serious and expressed concern about my inclusion of unfinished work and unexplored ideas.  While they only make up about 5–10 minutes of my hour-long talk, he was worried that I'm not sufficiently protecting my IP from theft.  He went on to say that this is a much more significant concern at conferences where I would know very little about my audience members, but he still wanted to exercise caution.
I don't disagree with him, but that led me with the following questions:

In my personal opinion, I work in a rather dark corner of my discipline.  I don't intend to contradict my advisor, but is this a legitimate concern?
Obviously the risk is venue dependent, but would there be any forum in which I would be completely safe?
How nervous should I be to discuss ideas with people who ask questions at the conclusion of my presentation?  Do people ever go "fishing" for ideas?
Should I be more careful speaking about "unfinished work" or "unexplored ideas"?  Does one carry more danger than the other?
Since publishing is such a drawn out process, is there some point before the release of one's results that they could be given as part of a presentation?
Finally: Is there any way to say how much should I feel safe sharing?

Any responses to these questions or related comments are much appreciated.

Why do you need include "unfinished work and unexplored ideas" on your talk. Don't you have enough to talk about your already published material?

I only have one publication, and it's from my REU.  To be honest, I feel that this is quite common among graduate students in math, at least among those outside the very best schools.  I'm at a school I'd consider to be pretty good, and there are only a handful of students here with any publications.

tl;dr: Light is the best disinfection.
If you work in a relatively small sub-area, and know all your potential colleagues and collaborators, you can decide for yourself how likely is that some of them can "borrow" an idea from you and publish it as your own. Your personal intuition about your the situation in your particular case is much more relevant than a general discussion.
However, you can also consider the following facts to help you make your decision:

Publishing process in Maths usually takes much longer than in Physics, Chemistry, Engineering, or related "hard" areas. It is not uncommon for an article to be in first review for 6-9 months, and it is not uncommon to have 2 or more review cycles. Are you really ready to keep your findings secret until the paper is published?
If some ideas were shared among a small circle of colleagues, and two of them claim that they "own" the result, it is actually more difficult to find out, who was the original author, and who pretends to be one. On the contrary, if the pre-print is made public, and the ideas are presented on a large international conference (in a room with more than 10 people), than there are more witnesses who connect the idea with your name, and it is less likely that someone will dare to stole the authorship (but still possible, of course).
An important task of your supervisor (as any supervisor) is to ensure that the work will be eventually published; and as you know publishing is time- and effort-demanding. By expressing their concern, your adviser in some form encourages you to publish these ideas as soon as possible, which is a completely legitimate request. In fact, if one shares a good idea on a conference, but then never publishes it, someone else definitely will, sooner or later.

Better safe than sorry -- think about how devastating it could be, if someone from that audience decides to explore with your ideas and publish before you can get your thesis out.
No. Maybe in a group meeting with the other students of your advisor.
You don't need to be too nervous. But do say clearly that you are working on a problem for your thesis, if someone comes too close to asking questions that you're exploring in your thesis. Again, I'm not suggesting that most mathematicians are jerks, but it's better to be safe than sorry.
Yes. For two reasons. One, your ideas might get stolen (it's rare, but it does happen, and especially if you don't know who is in your audience, it's a gamble). Two, especially as a graduate student, people don't really believe you if you have all these grand ideas that you have not explored. Talking about unfinished work lowers your credibility.
Only if you have a preprint on hand (ideally on arXiv). If the audience asks for a preprint, you should be ready to hand it to them. As you become more senior, you might be able to get away with not having a preprint, but as a graduate student, you should really have a preprint.
Uh, only the stuff that you feel absolutely sure about. Don't spew out nonsense, because people can (sometimes) tell that you're bluffing. It's better not to take any chances when you're so early on in your career.


text:
What GPA is considered as Competitive for top five schools in Engineering in US
Some prestigious universities like Caltech, Stanford, MIT and Berkeley never disclose their information regarding their cut off major and cumulative GPA for M.Sc and PhD programs in engineering. I also aware of the fact that graduate schools will look at the whole package and make their decision based on sets of factors like letter of recommendation, statement of purpose as well as GPA.
However, since application fee is pretty high it is not worth it to apply to schools that you know you have a slight chance of getting into it so, I would like to know what would be the acceptable GPA (Both major and Cumulative) for applicants to be competitive for top five schools in the world?

You said it yourself. It is about the whole package.

I know but still GPA is one of the most important factor so I need to know what GPA is considered as really competitive GPA or what GPA is usually cutoff GPA for the top schools.

GPA is meaningless without specifying the school.  A 3.0 at some schools is better than a 4.0 at others.

@DavidKetcheson and the major and the classes and the ...

still GPA is one of the most important factor — [citation needed] No, really, at the top schools, it isn't. To first approximation, after the applicants with low GPAs are culled, GPAs don't matter.

As noted in the comments, there is no single answer, because when you are applying for a highly competitive graduate program you need to be both an "excellent student" and have additional strong qualifications that set you apart from all of the other excellent students.
That said, GPA is definitely a negative filter, and one can still extract some guesses. Here are MIT's requirements for entry of its own students into the internal EECS 5th year M.Eng. program: 4.25 on its 5-point scale (equivalent to 3.25 on the usual 4-point scale), plus at least 4.0 (3.0) in the most recent term.  Since these are internal requirements, you can assume it is likely that external applicants for graduate programs will be measured a bit more harshly since they will be less well known.
Assuming a 4-point scale, I think that a safe rule of thumb for Ph.D. admissions is that if you've got at least a 3.75 both overall and in your field, you can stop thinking about GPA and start worrying about the rest of the package.  If you're below 3.0, it will be extremely hard (though not necessarily impossible) to convince people that you are a strong enough student, even if you are very good in non-GPA aspects of admission.  In between, the lower your GPA, the more you have to shine in other aspects in order to overcome GPA-based concerns: e.g., 3.6, not so hard, 3.25 very hard.  This are rough heuristics, of course, and every case may differ...


text:
Will my Professor tolerate this medical excuse again?
A week ago In one of my classes I e-mail the professor and asked to reschedule one of my quizzes due to illness generated from sleep deprivation, she accepted and was highly considerate of my situation.
Now, due to my intense anxiety & stress of exams (to perform excellently) and lack of sleep for the last couple of days as well, my body is failing me and now I woke up to dizziness (fainting/falling sensation), fluctuating irregular heartbeat, and confusing feeling I can't focus on my work at all...
I'm going to the clinic as soon as it opens in a few hours and the exam is also in a few hours, I feel as if the professor will not believe me but as I visit the clinic to be sure this is not serious, I will ask for a medical note. Will my professor tolerate this again? In our syllabus an exam can be re-scheduled for medical emergencies. I believe this is a medical concern.
I maxed out my body for school too much, trying to do well and sleeping less (creating "more time").
Bear with the writing, I'm not in a perfect mental state...

See http://academia.stackexchange.com/questions/6144/student-missing-exam-for-medical-reasons

I have, I believe I'd have to wait and see how things go, honestly I am worried now about my health than an exam, I highly doubt my Professor would refuse a reschedule, this is a medical issue, I can't process the material due to this dizzies and irregular heart beat, the only reason I haven't been to the ER is I don't feel it's life threatening, but certainly disabling at the moment. I barely can type things out what takes me a few minutes take much much longer.  Taking this exam would mean a huge reduction of my grade due to a medical disability. I cannot ignore this and "take" the exam.

what do you think @StrongBad?

@Adam99: I think you should focus on your health now. Even if it's not a life threatening emergency, you clearly do have health issues. You can send your professor a short email if possible, but it's really not that important especially since you yourself think that she won't refuse rescheduling the exam.

There are three components to this question. 1) Medical advice, 2) Lifestyle advice and 3) the question about administrative consequences of missing an exam due to illness. 1) and 2) are offtopic here, and 3) is subjective (who'll know what this professor will do?) and localized in that it depends on the regulations at your school. So all in all, I think this question should be closed. (That is not to say that your problem is not real, just that this is not the venue to get help. Talk to your doctors and a professional who has experience with stress/burnout situations.)

While I wish that your condition gets better soon, I agree with @Raphael that this question is not exactly suitable for this site. Thanks Raphael for your very clear explanation!

Go to the hospital NOW and worry about the exam and your professor later.
If you tell your professor that you had to go to the hospital, she'll most likely just accept that. Of course, you can ask for that note nevertheless, just in case.
Once you are ok again, think about how you can reduce stress and get enough sleep regularly. On the long run, constant lack of sleep and stress will hurt you (even more than it already did so far).

There are extenuating circumstances that prevent you from preparing for an exam and there are extenuating circumstances that prevent you from taking an exam. Those that prevent you from preparing for the exam, but still allow you to take the exam, should be discussed with the instructor as soon as possible. Circumstances that prevent you from taking the exam should also be discussed with the instructor as soon as possible. In some cases the discussion could happen prior to the exam, while in others, the discussion will obviously have to happen after the exam.
What the instructor is willing to "tolerate" is irrelevant. Students need to do what they can do. If you cannot prepare, but can take the exam, then you take the exam unprepared and then follow the procedure for extenuating circumstances that do not allow you to prepare. If you cannot take the exam, then you follow the procedure for extenuating circumstances that prevent you from taking the exam.
The outcome of having an extenuating circumstance that results in you missing an exam might mean you get a zero on the exam, but there is nothing you can do about it. There are many situations where a relatively healthy individual is experiencing symptoms that the recommend course of action is a visit to the ER. The decision on extenuating circumstances will hopefully not depend on what was actually wrong with you, but rather the symptoms that lead you to miss the exam. If you do not feel the condition would be exacerbated by a few hours delay, then it is not clear how you can justify not taking the exam.

This sounds like a potentially long-term problem that could affect all your courses. You should probably bring this up with the people in charge of your degree programme, rather than trying to arrange things each professor individually. Your university's disability office should also be able to help and give advice.

Any academic institution worth bothering with will have a procedure in place to take this sort of thing in their stride. Hand them any sort of legitimate document from someone with medical qualification, and the system should take care of you. Whether or not your professor is prepared to "tolerate" it is irrelevant; the procedure defines what sort of considerations you'll get, and the professor is required to abide by that.
The point of a system like this is to prevent the sort of worry you seem to be suffering from. It's not in anyone's interests for students to be ignoring potential health concerns for fear of academic reprisals, so the system is designed to protect you academically as well as medically. The rule is simple: health comes first, and the academia will make reasonable effort to fit around that.
I developed migraines, chest pains and spiking blood pressure in the middle of my final year of university. I ended up seeing multiple doctors and getting prescribed meds that could leave me unable to focus on anything, let alone my dissertation or exam revision. My advice to you is a combination of what I did, and what I wish I'd done:

Work only when you feel able to work effectively.
 This goes for any scenario, even just minor sleep deprivation. You'll get further by not having to go back and fix mistakes you made while tired, ill, or otherwise not thinking clearly.
Get to a doctor as soon as possible.
Two reasons: it's on record that you're seeking medical advice so that you can't be accused of making it up later (it's rare, but better to be covered) and it stops the problem going on any longer than it has to. Many health things are easier to fix if you catch them early.
Let the faculty know.
Tell them that you feel unable to work because of health concerns, that you have a doctor's appointment, and you'll keep them posted. Again, it's easier if you don't pull it out of nowhere right before a deadline. Sometimes that's unavoidable, but if you can avoid it, it helps. If they understand why you're absent and they see you being conscientious about it, they're more likely to overlook a brief absence if the doctors tell you it's nothing. They tend to be much less accepting of unexplained disappearances.
Follow medical advice to the letter.
Don't worry about the work you may be missing: if a doctor says you shouldn't, then don't. The qualification is less important than your health.
Keep the faculty advised.
You don't have to be too specific; feel free to say generic things like "taking medication" rather than "antidepressants", for example. They only need to know the bits that directly impact them; you don't need to tell them anything you'd rather keep confidential as long as they know whether you're able to attend lectures, do your coursework, sit exams etc. Phrases like "unpleasant side effects" and "not feeling up to it" can cover a lot without giving them details.
Make reasonable effort to meet them halfway once you're well again.
A little goodwill goes a long way, and once you're healthy again it doesn't hurt to offer things like catch-up work, meeting with professors to discuss your options, that sort of thing.

In the end, this all boils down to one thing: health first, then work.


text:
How to search for graduate schools that have Masters in Complexity Science/Complex Systems?
I'm an undergrad student who is interested in pursuing a Masters degree in complexity science/complex systems in the U.S. I know some schools put this program under physics, math or computer science departments.
I'd like to know which schools that provide such a program. Unlike physics, maths or cs programs, it is not straight-forward to find out such a list. Any recommendations on how to get a complete list of schools that provide a masters degree in complexity science/complex systems?

I empathize with as I had the same issue (and ended up doing PhD in something else, but trying to hijack it to do things I want).

BTW: What do you exactly mean by "complex systems" it is a wide term, covering a lot of very different fields. Statistical physics, mathematical modeling in biology, applied mathematics, complex networks, data-mining and machine learning, data science, etc...

@PiotrMigdal "complex systems" is my edit there, i was checking what the OP meant by complexity science, and a quick google search gave me the impression that the terms are closely related. For instance, wikipedia redirects complexity science to complex systems. Feel free to rollback or re-edit the question, if you feel that it creates ambiguity

@PiotrMigdal That is the other problem. I think it is a safe a way to narrow down the list after I carefully check their curriculum.

@PiotrMigdal Another problem is some school may name their program as "non-linear science","system science concentrate on xx", or other term I do not know yet.

@Joe Also some more good keywords for the search: "collective behavio(u)r", "systems biology".

@Joe as PiotrMigdal mentions, the main problem is that this definition includes too many fields. It is a bit like saying you are looking for a program in "molecular science". Perhaps if you give us a narrower definition or clearer idea of what you are looking for, we could help you better. For example, what kinds of courses would you like to take? What subfields are you interested in?

@Bitwise I'm interested in chaos and non-linear phenomena. And I do not want to give up other sub fields unless I have a better understanding of what question they care:) Thanks.

have you read papers in this subject? if yes, look at the authors affiliations and references.

Try looking at speakers from conferences dealing with complex systems, e.g.:

European Conference on Complex Systems
NetSci

You can also search for other conferences (e.g. at http://www.conference-service.com) and then check the speakers. Furthermore, looking at affiliations from recent (say - last few years) papers you like my lead to some good trails.
When it comes to webpages being hubs from complex systems, try looking at:

http://www.complexssociety.eu
http://www.network-science.org

Some positions (including doctoral programs) and other resources are at http://www.barabasilab.com.
Also, some group websites are in my collections of links (Delicious: complexity and networks or search at my Pinboard).

Though Santa Fe Institute doesn't have a Master program, it has a Complex System Course.

...and you can look at the second (or the first) affiliation of Santa Fe researchers. Usually they have one.


text:
Thesis submitted, but not published – can I put the title on my CV?
My thesis was submitted, but not published by the library or online yet.
I would like to put its title on my CV. Is this OK? Or is it kind of risky?

Has it been accepted/signed off by your committee?  If so, and the only reason it isn't in the library is due to embargo (to allow time for filing patents, submitting journal articles, etc) then you can say "To appear (date)".

It's only submitted, not accepted yet

What kind of risks in particular are you worried about? The only generally important thing is that the information in your C.V. must be accurate. If your thesis has been submitted, but not accepted, then say "(submitted)".

So i can add the title of thesis?

I don't see any reason not to. But as has been pointed out, you need to make clear the status of your thesis.

There is no obvious reason why including the title of your thesis in your CV might be risky. Just make sure that all information is presented accurately. If your thesis has been submitted, but not accepted, then say "(submitted)".

You wrote the paper, and know the material, so yes, you can cite your paper. Just make sure that its (submitted/unpublished) status is disclosed correctly.

Yes, just ask your supervisor.  The risk is to your university if it turns out totally bogus.

It's not Martin's responsibility to ask his supervisor. If the title is bogus, then it's the university's fault for setting him up with a bogus thesis title.

What is this "setting him up with a [...] thesis title" of which you speak?  A PhD thesis is the work of the student; that includes the title.  (Of course, every PhD student should get preliminary feedback from their advisor before submitting their thesis, including feedback on the title, not to mention regular feedback on the developing content over the preceding several months/years.)

@JeffE: My thoughts went into the same direction as the part in brackets. In addition, the direction of the PhD thesis is often heavily influenced by the advisor, even if the phrase "set him up with" might actually be too strong in general.


text:
Is it possible to cancel paper submission after one week?
I submitted my paper one week ago to a journal and I want to cancel my submission. Is it possible?

I don't see any reason why it shouldn't be possible.  The usual terminology is to withdraw the paper (rather than cancel the submission), but it amounts to the same thing.

Without knowing the specifics of the journal and the submission system, no useful answer is possible. And with all those specifics, this would be far too narrow a question.

@EnergyNumbers: Why would the answer crucially depend on the specifics of the journal/submission system? There are only two possibilities to consider: Withdrawal is integrated into the submission system or it isn’t.

@Wrzlprmft or it's a negotiation with the editor. And co-authors. And possibly the funding body. Fees may or may not have been paid. etc.

@EnergyNumbers: or it's a negotiation with the editor – I do not want to exclude this possibility but that does not inflate the possible turns of events. — and co-authors – I would assume by default that the OP is acting on their behalf — Fees may or may not have been paid. – After one week? That’s extremely unlikely, unless there are any pay-to-publish journals where you pay before the review process. — Note that while the lack of details makes this question bad, I cannot see that it does not result in any problem with possible answers and thus I see no reason for closing this question.

@Wrzlprmft  asking for publication fees prior to acceptance sounds like a disaster waiting to happen. Though definitely something a predatory journal might try. I really hope that no reputable journal does this.

It's certainly possible.  You can withdraw a submission at any point until it has been accepted and you have signed the publishing agreement.  You might upset the editor, but there's absolutely no way they will try to forbid you from withdrawing the paper.  The worst case scenario is that they will remember you as a flake who wasted their time with an inappropriate submission.  (Of course, that's not a good outcome.)
You should not plan on resubmitting the paper to the same journal, at least not if the social conventions in your field are like the ones I'm familiar with from mathematics.  There might not be a formal rule against it, but it would look really strange, and you'd get reactions along the lines of "What are you doing?  You had your shot at publishing the paper in this journal, you deliberately withdrew it, and you wasted everyone's time in the process.  Now you want to start again?"  (Only it would probably be stated less confrontationally.)  I doubt they would accept a resubmission unless it was a really important paper.
The disadvantage of withdrawing a submission after just a week is that it looks flaky, like you couldn't be bothered to check the paper carefully enough last week to detect the problem.  On the other hand, the longer you wait, the more time may be wasted by reviewers.  If you know you need to withdraw the paper, it's better to do it early than to delay.
Note that everyone will assume you withdrew the submission because of a serious problem with the paper.  If there's a minor problem, correctable without important consequences, then you could submit a revised version instead of withdrawing the paper.  (That would still look a little flaky, but less so.)
If there's no problem with the paper itself, but you just changed your mind about where to submit it, then you'll really annoy the editors and reviewers.  You shouldn't do it without a compelling reason, and I'd recommend not pointing out that you merely changed your mind (since there's no sense in adding insult to injury).  This would make sense under some circumstances, for example if you discovered you accidentally chose a predatory journal, but under ordinary circumstances it would really look bad.  You could still do it, but I wouldn't advise it unless you feel you really made a dreadful mistake in choosing the journal.

+1. "The disadvantage of withdrawing a submission after just a week is that it looks flaky" - personal anecdote here: recently, I circulated a manuscript for comments, did get some, incorporated these, submitted... and one day after submission, I got comments from one guru that pretty much blew the paper away, but suggested a way forward. I immediately withdrew the submission, spent a few more weeks on the paper, submitted a new version (to the same journal), getting a major revision and finally acceptance two days ago. (I'm pretty certain the guru was one reviewer, which made perfect sense.)

Last night at dinner a friend told me that in his youth he submitted a paper to Math. Ann. because he thought it was the same journal as Ann. of Math.  Indeed he withdrew the paper upon learning of his mistake.

Any reputable journal will provide some way for authors to withdraw their papers.

For journals with an online submission system, there will typically be an option within the same online system: if you click on "current submissions" there will be an option to withdraw it.
If there is no direct online method, then you can email the editors.

While these options are rarely exercised, every reputable journal will have such an option in order to allow for the exceptional cases, such as a group discovering a major error after they have submitted their manuscript.


text:
Is overlapping of two dissertations in TCS acceptable?
Me and my friend in initial stages of our PhD worked on same research problems in theoretical computer science (TCS) under the same supervisor. Is it ethically acceptable in TCS community to have almost identical (50%) dissertations containing identical chapters?
Note: This question was originally asked here.

Unrelated to the field but I think it's important to emphasize that, unless really well explained in both your theses, one could see a case of plagiarism without having full knowledge of the context. If your university has an anti plagiarism tool, then you might want to agree with your friend to (1) write your own dissertations and (2) make sure that both your theses are checked for plagiarism to prevent for honest mistakes.

I do not understand. You published some common papers. How many? One, two, more than two? Then you probably have to decide (with your advisor) which of those papers go to the PHD thesis of whom. You cannot put all the same papers (when they are many) on both theses.

A side note: it would have been better if you had flagged the original question for migration.

@Alexandros, why not?

@usul Your thesis should demonstrate that you have made an original contribution to your subject. Consider the extreme case where they two theses end up identical. Who did the work? At the very least I would expect an explanation of what your contribution was. But if you really are doing everything equally, you should between you still produce enough for two theses.

The correct answer is always to check you local (faculty) rules for theses.
I'm not from TCS, but here are my 2 ct:
If the same chapters are actually papers in a staple (cumulative) thesis, that is acceptable over here (the papers will appear as their print version in cumulative theses). We have rules that establish a point system depending on who is first and who is second author.
Here's an example: student T focuses more on theory and student A on an application (or maybe for TCS, on simulation?). Then 2 papers (not LPUs) are written, one focusing on theory with short application example with T 1st author and A 2nd author; the other paper discussing the application in detail and just briefly the theory. Here authorship is the other way round. Both students can take both papers into their theses, and each gains 1.5 publication points.  If, say, the application example isn't worth a full paper, then only 1 paper is written and T gains 1 point, while S gains only 0.5 points (not being the first author). Both put the complete paper into their thesis.
In any case, the non-stapled parts of the theses need to be different in order to establish who did what part of the work, because the degree is given for the own work of each student. So if all work was done as team work, there will probably be trouble because the committee cannot know how to assign grades to whom.

TCS doesn't have "first" authors, and our papers tend to be a mixture of theory, theory, and theory.

@JeffE The separation between theory and application was just "an example". There is presumably some difference in emphasis between the two students, and some portions of the combined work that each considers to be their contribution.


text:
Do famous works of literature need citations?
Specifically, in a scholarly paper on philosophy of math I want to mention Mo Yan’s novel Red Sorghum Clan.  It is just as an example of literature, not quoted and not used for any specific purpose.
Suppose I were to say:

Think of any famous play, say Shakespeare’s Macbeth.

would I need a citation to some edition of that play?
Of course if I wanted to refer to a particular edition of any novel or play I would have to cite it. And if I were offering a statistical analysis of language in the novel I would have to cite a particular edition.  But I am not doing those things.
When I look at the text, saying “Mo Yan’s Red Sorghum Clan (1986)” looks ridiculously stuffy to me.  This is no journal article or even an academic book.  And as far as the practical issue of readers who want to find it, the title by itself will be far more helpful than any reference to a particular edition which the reader might or might not have any way of finding.
Or am I being ridiculous?  Should I just make the citation?

If you aren't using it for any specific purpose, why are you mentioning it? If you are using it for a purpose, yes you should cite it properly according to the relevant journal format for a book.

Just FYI, "red sor" on Google gives me mostly "red sores on [body part],"  not your book. Google is informed by your search history, after all.

Also, before anyone thinks I'm covered in red sores, I did that in an incognito window on a computer I don't use Google on.

Do you just want to casually refer to the work itself, or do you want to discuss its contents?

To the revised and clarified question, I think the answer is clearly no.  We give citations so that readers can, in principle, compare what we saying about a text to what's actually written there, or to send them somewhere for more information.  In the kind of situation you describe, the mentioned text plays a fundamentally different role.  It's more closely analogous to saying, "Think of any famous building, say the cathedral of Notre Dame." You wouldn't give a citation to the building.

I think this is probably the correct way to think about reference to a "thing", if the thing is not itself part of the scholarly literature, nor even popular exposition of something that does admit scholarly discussion. That is, unless a novel is the object of the scholarly discussion, it can reasonably be "just a thing", like the cathedral of Notre Dame.

Anyone care to explain the downvote?  Does someone think you actually do need to give a citation to a building?

@paulgarrett I seem to recall a Graham Greene novel cited in a representation theory paper (I think to quote a phrase, but now I've forgotten the details---if only my memory came with citations so I could check...).

But I would never give a citation to a building, so the comparison is not really useful. Or is there some mechanism to cite architecture that I missed? The book is something that could be cited and it seems from the clarified question that it is this specific book (not "a" book) that OP wants to mention. So there is probably a reason or a connection between the book and the publication which would be a reason to cite it. (What if another book or publication with the same title exists?)

But which Cathedral of Notre Dame? (And that's just a few in France...)

@skymningen: The point is that in this situation you wouldn't even think something was missing if there were no citation for the building.  There's simply no reason for it.  And similarly, even though the book could be cited, there's no reason for it in the situation described by the OP.

@AlexanderKosubek: Obviously I meant the famous one (and obviously I was translating the familiar German name into French).

Cathedral of Notre Dame, 48.853011, 2.349720. Accessed 22 Nov. 2017

When I published a paper on statistical analysis of various texts, which included really famous works, I gave full references. Why I think they are deserved:

what you consider a famous work may be in fact unknown to a reader and she may become genuinely interested in the work - in that case the more bibliographical data, the better (e.g. call me simple-minded, but I have never heard of Red Sorghum Clan);
you may be implicitly referring to a passage that was present in particular edition, but was not present in others;
where do you draw a line between a scientific work and a work of art that does not require a citation?
in your example you are not citing an original work (which, according to wikipedia is titled 红高粱家族), but a certain derivative work, a translation, done by a certain translator, who - inter alia - came with the English version of the title. I think one should then give precise pointers to this translation.

Also, check whether your manual of style says anything on the matter.

Another possible good reason: you used a translation of the text.

Another reason: some classic works of literature are published with interpretative annotations from the editor, which could differ from one edition to another.

To me, the second bullet point is enough, really: what edition did you looked at?. That's critical to know.

@lukeg Comments exist primarily for the purpose of improving posts. If you think a comment's content should be part of your answer, add it. That is usually the outcome a commenter is hoping for. (Feel free to give credit to the user if you feel it is needed or appropriate.)

Your situation sounds a bit different from the OP’s.  In your case, performing a scientific analysis of the texts, the precise full text is scientifically relevant to your work.  In the OP’s, the book seems to be used more generically as an illustration of some point — as suggested in @MarkMeckes’ answer.

@PLL I understand the point.  However, I still think citation is in place. Let's change OP's example to: "Think of any famous scientific paper, say Watson&Crick's Molecular structure of Nucleic Acids". You wouldn't add citation in this case as well?

If I refer to a specific work, I would cite it.
However, in some fields is common to read articles that name work without citing the original paper. For example, recent hydraulics papers will refer to "Saint-Venant equations" or "Manning formula" without citing the original publication, because those are very well-known and over a century old.

And, as with @Mark Meckes' answer, "Saint-Venant equations" are "things", not scholarly literature.

+1 No reference is necessary for textbook knowledge.

To improve the chances that your reader will be able to know exactly what you're talking about, I would include as much information as possible: the name of the creator and the year of a creation at a minimum, and a full citation if reading the work is crucial to understanding your argument. It's hard to predict how well an artistic work will be known over time, how easy it will be to find in a different country, or how accurately the name will be translated.
An example of this might be a well-known comic book such as The Dark Knight or The Watchman, both of which would have been unambiguous in comic book circles before movies based on them were created. The same might be true of Red Sorghum -- are you referring to the 1986 novel by Mo Yan, the (much more famous!) 1987 movie by Zhang Yimou, or the 1993 English translation (which I love) by Howard Goldblatt? Even just saying "Red Sorghum (1986)" can help disambiguate this.

If you make a fleeting reference to something, you generally don't need to cite it unless you actually quote from it, or reference an obscure part of it.
In Spanish literature, for instance, we can reference the Quijote without including a bibliographic citation unless we are actually quoting the text or if we reference a particular less well known scene. That is, a fleeting reference or to the the windmill scene need not a citation, but one where Don Quijote calls a given book crap might.
A common occurrence of this might be when we talk about books that an author has written. To continue with my Cervantes example, if I'm studying the Quijote then I'll cite the book beginning generally with my first quotation, but not necessarily the first reference to the book. But if I mentioned that his first novel was a pastoral work called La Galatea, but otherwise didn't cite or talk about the book, I wouldn't expect there to be a citation provided. If I also said that he expected Persiles y Segismundo to be his masterpiece, I would cite an article to that fact (maybe, since that's well known amongst Cervantistas), but I'd definitely not cite the book itself (again, unless I actually quoted the book).


text:
Understanding an advisor’s refusal of an internship
I am graduate research assistant in Computer Science in the US. A while ago, a master student in the research group, who plans on a career in industry after graduate school, received an internship offer from a large, well-known technology company which is one of the most sought after and prestigious opportunities for any computer-science student. An internship at a company such as this would be extremely valuable both in terms of shining on his resume and the experience he would gain there. This internship would be a huge break for him and could really launch his career.
However, when he asked for his advisor’s okay to take the internship for the summer, his advisor told the him that he would have to find someone to replace him to continue work on the research project which would essentially mean the student would lose his funding when he returned from his internship.
My question regarding this is:
What are the pros and cons a professor has to consider when deciding upon allowing his student to do such an internship, particularly regarding that such an internship would be tremendously valuable to the student’s career?

"which would essentially mean the student would lose his funding" - I would strongly encourage you/them to clarify this point until it's crystal clear, because to me that could very well mean the student won't be funded only for the summer, which is both fine and the norm. Ensure there is no "hinting" or reading between the lines going on - come out and ask directly exactly what things mean explicitly, or the risk is everyone is worried about nothing.

oh man I hope he takes the job... being a professor's assistant or having a foot in the door/basically a guaranteed job at what sounds like a top tech company... easy choice?

If you consider the Master's assistant position from the perspective of a paid job, it would be unreasonable to expect an employer to grant what essentially amounts to a sabbatical. Professors and school teachers are often given sabbatical leave, but Master's students are in a very temporary position. It's not a common thing, and there are no long-term benefits to the employer to outweigh the inconvenience.
If it is feasible to do so within the bounds of the research project, then it would certainly be reasonable for the professor to grant the student's request. But I don't see that the professor has an ethical obligation to do so.
Funded research is commonly held to tight deadlines – once the timeline of a project has finished, any unspent money usually returns to the funding organization.
If the professor could do so, without a negative impact on the research, I am sure that any reasonable person would grant the request for leave. The simple answer is that it's quite likely that he or she simply cannot feasibly put the research "on hold".

I feel this would depend on the situation through which they gained their position as a research assistant. As you mentioned "graduate research assistant", I'm assuming this is a position where the masters (or similar) student receives funding for their studies by spending time working in the lab on the advisor's project.
In the UK, a research assistant position could be like this, but would involve a contract of employment. As such, it would be expected that an RA would be present and working on their assigned project, since their pay was coming from the associated grant or project funding.
If that's the case and the student is under contract, then I suspect the situation encountered here is somewhat understandable, if not reasonable.
Depending on the funding for the advisor's research project, they may need to fulfil a certain staffing quota - they have project deadlines and deliverables if it's funded by a grant, and they'll have work needing done to meet these. For that reason I believe it's reasonable for the advisor to want to replace them in the lab. It also might be difficult to find someone suitably experienced, who would be willing to do this for a short period, making it easier to transfer the position to someone else.
To perhaps consider another motivation of the advisor, perhaps they view their RAs as potential PhD candidates - one who is going to move to industry might be less desirable for them. I am merely suggesting this could be a consideration, and I am not suggesting that it would be appropriate.
Regarding question 1, if there is a contract in place, I feel it's reasonable for the advisor to take this position - note they're not preventing the student; simply saying they can't keep the post open indefinitely for them to return to. The funders want results, and replacing the RA is the best way to do that, in my opinion and experience.
Regarding question 2, I think this depends on the student's circumstances - if this will be be beneficial for their career and they can afford to not work as an RA, it seems a good idea to take the opportunity. It's not possible with this information to gauge if there could be any other impact for the student though, so they should consider any potential risks or adverse factors (needing to rely on this advisor as a reference for example).