text:
Is it good practice to let students consult their text materials during exams?
I teach undergraduate level courses in the humanities. Following practice in my department, I have let students take their texts for consultation during their written exams. They can choose a number of questions they want to answer from a set of questions. After a few semesters, I have begun questioning the validity of such an approach.
I usually had one exam in this form and one final essay in the end of the semester. The issue is, although the subject dealt with in the first part of the course is more or less objective, I find students "copying" my classes much more than using the texts to answer the questions. A complicating factor is that most students cannot presumably understand the material, available only in English. To make it clear, most of my students cannot read English (I could mention the material is not available at the library, but that is another matter). And it is a required course.
What bothers me is that with this approach I cannot, as suspected, measure the level of understanding of the students. Some of the questions deal with very basic issues and concepts. Even then, the overall level of reading, understanding, and writing, as evidenced by their exams and final essays, is very low.
I have thought about changing the syllabus next semester, to one exam (without consultation), perhaps another exam and the final written assignment, but I am quite unsure of the results. Perhaps a lot of students will fail.
Am I too concerned, or is this the way to go?

There is something I don't quite understand here: Are you saying that the texts for the course are only in english, and that many of the students are not able to read and understand english?

@TobiasKildetoft, yes, that is exactly what I said.

@Joseph ah, it might be useful to edit the question to make that more clear, and even put it right in the title. Because asking the same question but for English-speaking students would induce a whole different set of issues to be raised in the answers.

My belief has been: if your exam allows people to bring the text, then the questions should never be answerable by just looking at the text. If they actually can just copy their way out of it, it's likely the problem of the questions, not the format.

@Penguin_Knight, you're right. I do include questions which they are supposed to discuss and not just mention facts and definitions. But they do have difficulty answering even the latter, let alone develop an argument.

Along with what @Penguin_Knight is saying, I've always seen that tests get harder and more fluid when notes or open book is allowed. More open ended questions. Fill in the blanks. Essay's.

IMHO a proper test should be one that student who doesn't understand the material can't answer even by using their notes + textbook + google. I.e., it should ask to demonstrate competence in some skill and/or application of that knowledge, instead of simply remembering and repeating what was read or heard. If people are failing fact questions because of a lack of understanding English - then that is a problem already in the learning part, maybe distributing 'cheatsheets' with translations of all the key terminology can help?

Another issue - "They can choose a number of questions they want to answer from a set of questions." doesn't mix well with consulting pre-made material.  I'd suggest to switch to random drawing of those questions, maybe "draw X+1 / discard one".

@Peteris, by "random drawing" do you mean randomizing the questions and handling them to students?

Yes - a practice I've seen is publishing the whole list of questions/topics beforehand; and when arriving for the exam, they just draw X paper slips with those questions.

To quote one of my undergrad profs: "Open book exams are bad, because the answers are never in the book.  Unlimited time exams are bad because you never finish.  Take home exams are bad because you forget where you live."

@Peteris, looks very simple and worth trying. I thought about some "randomizing" software, but your idea sounds nice.

Question: Are you teaching (on average) smart students who just don't know English, or are you teaching not-so-smart Americans/Brits? ;)  (just checking...)

@anorton, I'm teaching not-so-smart non-Anglophone students.

About the English: if the job you are preparing them to do (being a social scientist/historian/whatever is their degree) requires knowledge that is only available in English literature, then it is perfectly reasonable to reward those students that are able to read them; because maybe you can accommodate to it now, but you will not be there the rest of their professional lives. I actually have seen this kind of unreasonable helps... in an English language and literature degree.

-1 because this question is unlikely to be useful to anyone else in its present form. It's much too general in the sense that different subjects are different. It's much too specific in the sense that the use of materials in English for a class where the students don't have sufficient English is goofy and unusual.

@BenCrowell, "Different subjects are different"... up to a point. I specified that I teach humanities (history, social science). People here have answered things that might be useful either to exact sciences or to human sciences or to both.

@BenCrowell, Your usage of "goofy" and "unusual" is contemptuous and passes judgment on a situation you don't know. I wish I had an alternative. I was hired to teach a subject for which there is no translation available in my country. As I said, students were supposed to read in English. Should I just jump off the boat? I don't think so. Neither should I try to reform the university or the educational system.

I am not convinced by your question, because it feels like you are telling us the problem is somewhere when in fact, it lies elsewhere. You tell that you feel that the problem is in the format of the test, while in fact it seems that your test allows you to measure how much your students understood the course material. That the answer to this question is "very little" does not mean that the test is wrong: as long as it provides an answer to the question, the test is good (...)

(...)  What one can deduce from that answer is that the teaching has a problem, and the problem seems obvious: teaching in a language not mastered a bit by the students just cannot work. What you need is not to twist the test, it is a language course prior or parallel to your course.

Any answer that isn't "yes" doesn't make sense for me on the Information Age. Unless it's something that, in real life, the student absolutely objectively demonstrably will need to answer in less than 5 seconds or while being absolutely isolated. (Soldiers and air traffic controllers come to mind, definitely not your average History or Computer Science student)

@xDaizu, I don't know where you study/teach, but if your general premise about education is that, given the existence of the internet (or a book, or written notes on your lap), you can freely consult any material about something you should have known without consulting, then I cannot but strongly disagree. It's like having a foreign language exam and letting students use Google translator.

@Joseph Searching the internet is a skill. You need to be able to get a good response from a reputable source in a reasonable time (hence the time limit on the exam) and for that, you usually need to know what you are looking for. Google translate won't make me produce a perfect translation (idioms included) if I don't know the language. But it will be really helpful (mental blockade breaking) if I don't remember a specific word or the past tense of the verb "to sit" or, in other context, how many bytes a HTTP header is made of.

@Joseph I respect your opinion if you disagree (most people do), but I think no material allowed whatsoever,in most subjects and cases,puts a student in an unreal situation; usually rewards the wrong skills for future practical application and unnecessarily hinders the student's potential, not unlike not allowing ANY kind of calculator in a calculus exam (had a teacher that actually DID this). This is just my opinion, I understand and respect the arguments against this, even if I think they are a bit obsolete. (Sorry to get chatty, maybe this should have been an anwer) PS: I study and teach CS

I am in mathematics, so my experience will be different from yours. What I have found with tests on which I have allowed students to use their text or notes is that the students have not prepared as well as they should have, and waste a lot of time looking for things in their notes. Ultimately, they end up doing worse, as a class, than they usually would. Now, in math we have a lot less material for a test than in, say, history or political science, so there may be some legitimate reasons for allowing the students to consult other sources during an exam, but I do not recall ever being allowed to do so in the humanities courses I took as an undergraduate (mainly in political science and diplomatic history). I think a significant component of a college level education is learning how to absorb, and synthesize, relatively large amounts of material. So, my students have only their own brains to consult during an exam.

I don't recall having ever done an exam like that in my undergraduate years, either; maybe only in exceptional cases. That is what baffles me. And there was a lot of material -- political science, international relations, economy, law, history. Maybe I could just accept that standards are lower now that I work at another institution?

There should be 1 mock exam, before each exam where students are allowed to use their resources, as long as they make no sound(other than opening pages). Might get your students to actually read and not sleep.

Also allowing a cheat sheet(3x5inch card or B6 in metric) to be brought to THE exam; Negates any right for your students to complain. Also, Your student might actually produce content, YOU may learn from your students when reading their cards.

In my undergrad mathematics (40 years ago), any kind of reference, even a cheat sheat was generally forbidden. But at the same time, my engineering colleagues never had an exam that was not "open book". Their principle was that those references exist for a reason, and you had better have them available when you go out into the wide world. That said, "learning how to ... synthesize" is exactly what you need to be able to extract the answers from a textbook. And if your students do worse when it's open book, they haven't learned that very well.

I take a compromise approach to the problem of bringing materials to an exam. I allow students to bring one or two sheets' worth of notes that they have prepared themselves. No magnifying glasses or other "reading aids" other than standard prescription glasses are allowed, so they can't simply photoreduce a whole bunch of pages and then use it—it's something they have to hand write or copy themselves.
Such an approach forces students to prepare the material, but still gives them some flexibility not to have to commit everything to memory (Is that sign positive or negative? Is that denominator regular volume or molar volume?).

In my undergraduate abstract algebra course, we were allowed one sheet of paper, both sides. I was able to copy every definition and theorem, some proofs, and several homework solutions onto my paper, yet still had half of one side unused. I know this because I still have the paper and just looked at it. I am much older now, but could still read it with no problem. As I recall, I rarely referred to it during the exam, but the point remains that I had significant resources available to me. I'm not sure this is a good thing.

This is very much a course-specific issue. In some courses, it's much less useful than others. However, as I mentioned, the point of the exercise is to make the students review the material, as they need to decide what's important for them to have "cold" and what they can look up. I'd rather the students know how to use the formulas, equations, and definitions, rather than just being able to write them down.

I use the same strategy, without the proscription against magnifying glasses.  At least twice, I have seen students bring 50 pages of lecture notes and homework solutions, reduced to two 5x5 grids on either side of their single double-sided "cheat sheet".  Neither student did well.

@Aeismail That's a false dichotomy if ever I saw one. There's no reason that students shouldn't be expected to use a formula they've written down.

@MilesRout: What I meant is it's not important for me that the students have memorized the formula, so long as they use it correctly.

+1, one of my math professors allowed to bring 10 pages of handwritten notes, and actually writing these was one of the best things to do in preparation of the exam...

I would ask that the teachers try to also be understandable for those of us with testing anxiety.  I was always very thankful to have notes or the text with me during tests because it allowed me to relax some and actually be able to focus when answering questions without fear of forgetting some small detail.  I also found that I prepared more when allowed to bring notes because I took time creating them which made my foundation of that knowledge stronger. When I didn't have notes, all bets were off.

This, definitely. Books and notes are so helpful to students who know the material but become anxious in timed exams, or who know the material but don't have a good memory for details. Books and notes don't help students who don't know the material (assuming the exam actually tests mastery of the material). I can't understand why anyone wouldn't want to help the students in the former category do well.

Testing anxiety is, unfortunately, something that people just need to get over.

@ff524 I really agree with this and disagree with Miles Rout. "Test anxiety" doesn't necessarily mimic problems people face in their future careers. "Something they need to get over" I would ask "Why?"

The real question is what you're attempting to test.
If the proposed notes are material which the students are going to need "off the cuff" -- data and operations which are absolutely basic to the discipline they're learning -- then it makes sense to test whether they have memorized it, since consulting reference materials every time will slow them down too drastically for them to work productively at the next level up.
If they're material which a practitioner will generally not have memorized and will look up anyway, then it isn't unreasonable to make references available to the students... while pointing out that being able to work without the hardcopy references will let them solve the problems faster and with more confidence and thus may improve their grades.
Note that these cases presume two very different sets of exam questions.

Usually exams that allow the students to use their text materials is because of the density of the topics. I mean there is a lot of material to be covered and for that reason not easy to memorize, or in some cases it is futile to make the student to remember some specific topics (like in my field of CS).
For what I see the students prefer to paraphrase your material instead of what is covered in the text book; that could be a direct consequence that they find the questions in the exam too easy to answer, and with the answers directly related to your material. In this situation what I do recommend is to twist the difficulty a little bit higher, so you can force that the student has read the material from the book beforehand; and also that can show that he or she knows the basics.
In general it is not bad to let students to use textbooks during the exam, the bad thing is not to adequately tune the difficulty of the exam.
Good luck!
Side note: English is almost a universal language and it must be a prerequisite before following some courses. In any case if they are too lazy to learn a new language then try to get translated copies of the required textbooks (or just prepare one for your own).

“English is almost a universal language” and “they are too lazy to learn a new language” is a bit offensive. Many people don't speak it very well and learning a language is not easy. You might very well decide that it's not your job to accommodate people who are struggling with the language but you can't assume that everybody in the world speaks English and that those who don't are lazy people.

Many people don't speak it very well and learning a language is not easy — With all due respect, so what?  Calculus isn't easy, either, and many people don't know it very well.  But any upper-division engineering course requires fluency in calculus, and it's unreasonable to expect instructors to make significant accommodations for student who lack that fluency.

@JeffE, just a side comment, those students did pass through an English exam to enter university. Maybe it's just such a shock because, believe it or not, there are faculty who do not use or willingly reduce English bibliography to a minimum, some of them because of the inherent nature of the subject and some of them because they don't feel comfortable or don't have the language skills themselves. So naturally I'm the odd guy because my subject is the only one that is 100% English-language.

@Layla, yes, I could spend hours and hours to translate all those textbooks and articles, but even that is no guarantee students will even read them. There isn't even a "thank you" for that kind of work; so I stopped doing that.

@JeffE I think I stated quite clearly that it could be unreasonable to accommodate poor English skills. I just think it's not accurate to claim the language is “universal” or that people who don't know it are lazy. I actually think that your example makes my point quite nicely, as nobody would assume knowledge of calculus to be universal.

@JeffE, it's quite hard to learn subjects in a language that you've just beginning to learn. You need to translate in your mind, and you can't absorb it as fast as other native people, or when you learn it in languages that you're familiar with. just from my experience.

@Joseph you're so kind that you make the translation. if it's me, i wont bother.. it's their job to learn..

I come from an engineering background, and we were allowed to bring our books and notes. This was in the 80s and 90s so there was no Internet to access. I found that if I didn't know the material that the books and note were no help, and that searching the materials just wasted time.
Unless the test asked the exact same question as was in an example covered, the materials were not useful as a resource during the test.
It should take only one experience like that to persuade a student to learn the material rather rely on the books and notes as a crutch. Consider it a bonus life lesson for the student.

I can't tell if your recommendation is to allow books and notes (since students have to learn the material anyways) or not to allow (because you don't think they'll be useful).

My recommendation would be to let the student bring the book, and let them experience failing the test because they didn't learn the material. Let them experience failure so they won't use the book as a crutch in the future.

My answer is based on my own experience as a college graduate, and, in short, is "No, you are not too concerned, and, no books during tests is not the way to go." I feel that a teacher's two responsibilities are to encourage students to learn on their own, and to effectively relay information to them: few things are capable of so thoroughly sabotaging this process than offering an open-book test.
Teachers of mine have tried all kinds of policies during my college career, and, in deed, my entire student life. Open notes, open book, open book and notes, closed everything, study sheets, index cards, you name it.
I found these two strategies in particular to be the most effective:
2nd-most effective:
Usually about a week before each test, our teacher offered for us to turn in a blank blue book with our name on it (standardized bound sheets of paper for essay exams) that she would mark in a supposedly tamper-evident way, gave it back to us the next day, and allowed us to fill it with hand-written notes to our heart's content. We were allowed to use these notes and these notes only during the test, during which she would verify the authenticity of her markings on any study guides in use at her leisure. This was incredibly effective for me because taking the time to write down information from scouring books, class notes, the Internet, and even collaborating with others doing the same thing, was, in effect, actual studying. I usually found that I seldom needed it at all during the test because I'd committed so much of it to memory while creating its content! Additionally, it was a confidence boost to have it available, and that any time I felt like referring to it, I usually knew specifically which page of notes on which to find the answer!
Absolutely the most effective:
The first day of class we were handed a list of 150 questions, and were guaranteed that our entire final exam would be exactly one randomly selected question from the list (possibly different for each student), and alone, worth 100% of our course grade. I had never so diligently studied for a class in my life!

This has been touched upon by the two lowest voted answers, so I am going to give it a go myself to make it clearer.
First of all, except for orientation courses where memorization is the central issue, open book examinations are incredibly good, however they are only as good as the questions you ask. The thing you should be aware of is that with examinations like this you're able to far more complex questions which are not about specific sections of the text, but rather about the course as a whole. For example, rather than asking what the effects were of the actions of a specific individual on his field of study in his time you can suddenly ask students to describe the trend over a far greater time period. Or you can ask for an analysis of the approach of a set of individuals on a very specific detail. What you should be aiming for is comprehension of the matter, questions which can not be paraphrased from the covered materials as they simply were not explicitly covered.
Another existing 'trick' is to make the exams so huge that students never have any chance of finishing the entire exam. If you go down this route you have to make very clear which depth and length you expect for each question, but the nice thing of this technique is that you can see quite clearly how well students know the full width of the subject matter.
Now, I have to be honest that a lot of these kind of examinations - open book exams in general - are often none the less not well done. It becomes more about just learning the index of the books well then understanding the topic matter, but in essence I believe it is crucial for a good modern education to not be based around memorization - we have the internet for that - , but rather comprehension and recombination.

It's the reality of how the world is changing, and the reality of how a lot of traditional academics is outdated. Take a look at how people work in real life and the main problems they encounter once they finish university ;-)

It depends heavily on what you're testing.
Is it a computer science paper, where you want to test your students' knowledge of basic data structures and algorithms? Then you probably don't want to let them have text materials.
Is it a first year linguistics mid-semester test, where you want to test the students' knowledge of the International Phonetic Alphabet and different places and manners of articulation? Then you probably don't want to let them have text materials.
Is it a second year linear algebra exam, where you want to test the students' ability to apply different methods to solve various systems of linear equations, but you don't need  them to remember whether a projection is v.n/n.n * n or v.n/v.v * n? Then you should probably let them handwrite their own notes.
Are you testing the knowledge of something? Or are you testing the ability to apply that knowledge to solve a problem or compare two algorithms, historical events, methods of solving an equation, etc.?

Does your school allow tests to be given and answered in your native language?  If so, then ask questions in your native language and have them answer in your native language.  Those who do not understand the text will fail (they cannot pass by just copying the English text, as you have required their answers to be in the native language), and those who do understand the text will pass.

What do you want "create"? Robots like google? or persons that have resolve problems, analyze and creative responses... let robots memorize and let hummans create and be creatives.


text:
Best practices for keeping a campus computer network from getting brought down by inappropriate use?
I work at a California community college that has chronically underinvested in computing and networking. The school has been gradually increasing wifi coverage, which now includes all but a few areas of campus.
As a result, you can walk through the cafeteria or the halls of the natural science building and see one student after another watching TV shows on a laptop. Yes, it's conceivable that they have an educational reason for doing so, e.g., maybe they're watching "I Love Lucy" in order to research a paper on media stereotypes for their women's studies class. But in reality, I think they just want to watch Spongebob for entertainment.
This causes severe problems, because the network was never designed to be a TV conduit for thousands of students. It apparently was one factor leading to the fact that the entire wifi network went down for the first 6 weeks of this semester.
My school's Acceptable Use Policy forbids various uses such as commercial activity, chat, and playing games, but it does not forbid binge-watching "Breaking Bad" over the school's wifi network. I can understand the concern that once the school starts blocking certain web sites, there is a slippery slope leading toward violations of academic freedom. However, it seems farfetched to say that we have to let students access Netflix while logged in to a student account.
Are technological solutions such as bandwidth caps for student accounts a good option? This would seem to sidestep the issue of academic freedom. Spending our way out of the bandwidth deficit does not seem to be an option; not only do we not have the money, but there is every reason to expect that demand would expand to saturate supply.
Can anyone describe best practices for keeping a campus computer network from getting brought down by inappropriate use?  An ideal answer would describe both a successful policy and how it was successfully enforced without infinging academic freedom.

This is the/an issue of the future... The only truly stable answer is that univ's bandwidths have to be massively increased... To say that it can't happen, or won't, or... is irrelevant. Kids do this stuff. It's not possible to police it, either, in either practical or theoretical (free speech, whatever, ...) terms. Thus, the real point is that unless a Univ arranges sufficient bandwidth to accommodate , it fails in certain important ways. To say "we can't afford..." is to say "we can't afford to avoid failure..." Silly, yes, but, ...

I don't have an answer to the question as written, but want to second Paul. There are plenty of relatively low cost technological solutions, and when you consider how much happier people (especially young people, who are, in a significant sense, your university's customers), are when they have access to proper bandwidth, it's an investment that will pay off. Large caches, and maybe even getting a content distribution node for something like netflix installed, could be cost effective solutions. Of course, convincing the administration to go for that may be another thing entirely...

Might be worth asking on Superuser as well - the people-centred side is perhaps unique to academia, but technical options are relevant to many large networks.

I see this as a resource allocation problem, not an academic freedom problem. Your network just does not have the resources to support many users with high-bandwidth demand (whether the demand is academic in nature or not), and your school does not want to apportion more money to upgrade network resources.
It sounds like you don't really want to prevent a particular use of the network; if you did, it would be in your Acceptable Use Policy. You just want to prevent a user from hogging network capacity, which degrades the experience for everyone else.
Instituting bandwidth caps (or a similar content-agnostic strategy) to prevent excessive resource use by one individual does not violate academic freedom. For example, Amtrak limits use of WiFi on its trains; streaming music, streaming video, and downloading large files are disabled. This is understood to be a reflection of their unwillingness to spend the money that would be required to support high-bandwidth content, not censorship.
If you want to make sure that students still have access to I Love Lucy and other essential academic content, you can offer uncapped network access in specific areas (e.g., in the library).
Of course, if you plan to implement this, you should announce the change, explain the reason for it, and expect students to be very, very unhappy about it. "Network" is simply one of the resources that students expect from a college; from their point of view, you are failing as a college if you do not provide a reasonable amount of this resource.

@scaaahu :) I thought as I was typing it, "This will definitely make the library more popular!"

You read very fast. You seem to forget it's wifi wireless network. Not only the library but also the area around would be crowded. It's was tried in Taiwan and abandoned later. The only way working was to increase the bandwidth. Some dorms in Taiwan restricted the internet access after midnight and it was a very unpopular action and was a headline news.

@scaaahu I agree that the only really "good" solution is to upgrade (which the OP is unwilling to do), and that any other solution is likely to be immensely unpopular. (P.S. The OP could make the uncapped network available through wired connections only, if he wanted)

I don't know about the US anymore. In Taiwan, more and more people keep away from wired connection.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
What are the major informal obstacles students face when they go abroad for a research degree?
I have a completely introverted personality. I am seriously struggling with interpersonal relationships with my colleagues, to a point that I am actually at the brink of quitting my job. If I go abroad for a research degree, would there be any problem that may jeopardize my endeavor? My family is constantly warning me against leaving the current job and going abroad for a higher degree; they say that I will surely return without completing my course as soon as I face any rudeness, animosity, or harshness, be it from the adviser or the environment.
A relative of mine in the USA has told me that she has been experiencing harsh behaviors from her PhD adviser due to her religion. One of my cousins in France has complained that he can never get any attention from his instructors because of his ethnicity. He is in service with UN.

I think this question as currently stated is too broad to get good answers. Can you edit it into a specific, answerable question?

Note that in France it's not uncommon not to get any attention from instructors regardless of ethnicity (not saying it can't be a factor, though).

Your edit did add some context, but "Can you outline issues" is still very broad. See http://academia.stackexchange.com/help/dont-ask for more advice on fixing your question.

Perhaps you could make the question: can an introvert make a good academician?

Maslow's hierarchy of needs to a large extent? Of course, some of the items may have to be switched with others such as language barrier, cultural barriers etc.

I looked at your other questions, which were quite enlightening. Given that you had already publicly revealed other relevant facts about yourself, e.g. How will a "local" master's in CSE look when I apply for a Graduate funding in the USA?, I don't know why you didn't include them in your question. Any such details are highly pertinent, and would help people give better and more "customized" answers.
To summarize from your previous question, you are 32, live in Bangladesh, have an IT background, work in a bank, and are afraid your job is taking you nowhere.
I'm Indian, and left India to study in both the UK and the US, at different times. So my background is not so dissimilar, and I can relate to some extent. However, the fact remains that your question is so broad that it is difficult to give useful information without knowing you.
Given that you are from Bangladesh, I'm guessing that part of your interest in further studies abroad is to get away from Bangladesh. Also, there are presumably not that many routes out of Bangladesh besides being a student. If so, I sympathize. However, from your previous question it sounds like you were planning to get a degree locally before going abroad. If so, it sounds like your question might be premature. If you aren't currently associated with a university or do not already have a relevant higher degree, then going abroad as a student would be very difficult. Are you still planning to enroll in a local Master's program? Or have you decided to go for a Master's degree abroad?
The answer by shane, I think, covers some of the issues you will probably run into as an Asian student in the West. How good or bad a situation you find it depends on a complex set of factors including:

Your area of study
Your university and location
The local community from your area/country. (Assuming you get on well with people from your background)
How successful you are at your subject
What kind of advisor you end up with
How much you dislike your native country. If you really dislike it,
you may have an easier time adjusting to a foreign culture.

If you happen to be from a very "sheltered" background, which is not uncommon in traditional Asian cultures, then living away from your family and culture could be good for you. In many ways the West (which it seems you are contemplating) is much more open culturally then a place like Bangladesh. However, how you respond will be up to you. You may find it frightening rather than empowering.
In my case, I was from a sheltered background, and had a difficult time. Grad education is a rough business, and study abroad is not for the faint-hearted. However, I can say that I do not regret it at all. It was (I think) very good for me. I learned to be much more confident and independent. I am now quite a different person than the person I would have been if I never left India.
Having said that, as shane says, grad school isn't the ideal way to go about self-improvement, if you have a choice. Of course, as I have observed above, you may not have a choice.

It is not going to magically solve your problems. You will undoubtedly face similar issues. People outside often think that Academia is the life of the mind and the ivory tower. It is not. A successful career, including tenure, often depends on how well you interact with your fellow academicians never mind your students.
Being an introvert can be very difficult. You can take this opportunity to re-invent yourself. You'll never not be an introvert but you can work on skills to minimize its impact on your life and career. Search out some books on the subject (there have been some popular ones published recently like Quiet: The Power of Introverts in a World That Can't Stop Talking).
Very carefully consider who your advisor will be. You don't need the prima donna, but someone with whom you can work with successfully.
Join Toastmasters or a similar organization where you are now or where you end up. Look for social, cultural, or technical organizations for your interests--and get involved. That may help you find connections and friends who can help you when the going gets tough.
You might also look into counseling, not to become extroverted, but to learn techniques to mitigate your introversion.
On a personal note: go for it! I'm introvert who's fine with friends. I'm a SME (subject matter expert) so knowing my stuff makes it much easier for me to speak up in a meeting, raise my hand in class (when I was in class), and so on. Don't let imposter syndrome get you either.

I am an american who studied in europe for several years. Here are some things I think you might want to consider.

It is always difficult to move to a new cultural environment. If you
already find social situations difficult, then these difficulties are
likely to intensify in a new setting. You will have to communicate
primarily in a foreign language, you will have to learn a new set of
customs and manners, and you will have to make all new friends in
your host country. This can be hard, depending on the country. In my
experience Americans are usually quicker to befriend foreigners than
Europeans.
Think about where you want to live. If you want to live in america,
then maybe doing a Ph.D. in American isn't a bad idea. In general
though it is hard for people who do their degree in one country to
get a job in another country, unless the degree is from somewhere
really famous like Harvard or Oxford. (See my post here for an
explanation why.
Dealing with academic advisors is really difficult, even for people
with excellent social skills. This is a subject that deserves its own
separate consideration. Your advisor will have a lot of control over
your future career and this fact can sometimes make one be overly
deferential. Some advisors use their power over their students in bad
ways. It is very common for advisors simply not to do their jobs. It
is also possible for your advisor to steal your research, although
this is less common. The proper way to deal with your advisor is as a
senior colleague who is paid to help you. You want to approach him or
her with reasonable questions, and you want to insist on frequent
meetings to discuss your work. You need to be comfortable making
these kind of demands, even if you aren't the person in charge,
otherwise you run the risk of a bad advisor taking advantage of you.
This is true outside the academy too, but it remains a real factor
even in universities.
Do not go into debt to do a graduate degree. It's almost always a
horrible return on investment.
Scrutinize your motives. Why do you really want to do this? Do you
really have a burning research question that you want to answer? It
doesn't sound like it. It sounds like you are in a bad situation and
want a lifeboat to take you somewhere else and try something new. If
that is your mindset, I strongly encourage you to reconsider graduate
education. Graduate school is a very difficult environment, and when
you get to the point of academic hiring it becomes absolutely
viciously competitive. Graduate school isn't a good way to find
yourself, or explore and adventure in a foreign country--it's
professional training for a very difficult, very stressful job that
doesn't tend to pay very well. If you don't love the material itself, you probably won't last.

I wish you the very best!

It is just not true that your problems always return back to you wherever you go. I have seen people who feel lots better after they have changed the position. For instance, if the laboratory used to focus on exhausted topic or gets much less funding than before, unable to continue existing projects, or the new supervisor is trying to push some weird concepts, it may be very difficult climate that is not your fault at all.
From the question looks at least you are self-critical enough. Try to be careful at the new job, remember lessons you think you have learned and do not repeat the mistakes. Take this into consideration and go.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How many graduate-level courses should one take to maximize chances of getting into graduate school? (PhD)
This is somewhat similar in broad topic to an earlier question I've asked, but not the same. I intend on applying to interdisciplinary mathematical/computational science/engineering programs like this: https://icme.stanford.edu/ . As a result of the interdisciplinary nature, there are a lot of junior-and-senior-level courses that are relevant, and I want to take as many of these as possible so I have a good grounding for whatever more specific path I follow, within the field of computational science. However, I also know that graduate classes are looked upon favorably by many admission committees; so, should I drop some of the undergrad courses (that may be more relevant in subject matter) and take some grad ones? Because of scheduling and prerequisite issues, I would otherwise not take any graduate-level courses (or take at most one) until my senior year, by when it may not even matter in terms of admissions because some graduate programs don't look at your senior grades)
[As a follow up to this, would it be considered a bad thing to take a undergraduate course that may be considered important or even "crucial" in the final semester of senior year - when admissions decisions are already coming out - because of taking graduate classes earlier on instead?]

Graduate-level courses are helpful, but there isn't any magic formula that "X courses with a grade of A or A- will guarantee admission." Every case is different, and good grades are by no means sufficient for admission, if your letters of recommendation are weak.
If you're applying for a terminal master's program that is primarily coursework, then the rules are different, but for anything relying on research, your research experience and letters of recommendation will carry significant more weight than your classwork and test scores. What doing well in the graduate courses does is signify that you will be able to handle the coursework in your program, but it does not shed light on the rest of your abilities.
So my recommendation is to take graduate classes because you're interested in the subject and want to learn more about it, rather than just to impress an admissions committee.

I just received an acceptance to a very computational cognitive science program. I had a single grad-level AI course, and I doubt very seriously if they even noticed it. I don't regret taking it, it was probably my favorite course because it was very loosely structured and allowed me to develop research experience, but it was also harder than a typical undergrad class.
Basically, from my (admittedly limited) experience, I would say take as many as you can earn A's in if you want to challenge yourself, but don't do it solely for the grad apps.

You should be thinking about taking the "best" Masters courses, not the "most" courses.
So how does one define "best" courses: 1) They are courses closest your current areas of interest. 2) They are the courses most relevant to your future (projected) areas of interest 3) they are courses in which you can get the best grades.
And I'd certainly take the "crucial" undergraduate course in preference to the graduate course if "timing" is an issue.
Don't take masters courses that are "irrelevant" just for the sake of taking master's course. Take a master's course because it fits your long term needs better than equivalent bachelor's courses. Tim's experience, from another answer, "only" one master's course, just because it was of interest to him, is relevant.

I disagree with (3).  Above a certain threshold, your grades really don't matter as much as other factors.  (Personally, I'd much rather see a B in a post-qual PhD-level course than an A+ in an undergraduate course.)

@JeffE: My concern is for the person who gets a C in a graduate course, and would have gotten an A-minus in the undergraduate course. (That once happened to me when I "overreached."


text:
What citation style is this? It has numeric reference, title in quotes, "Last, F.F." name style
A style I'm required to follow, unfortunately unnamed:

[1]   Last, G.L., Loo, B.F., Mooo, C.C., “My wonderful title” Int. J.
  Heat Mass Transf., 32(19-21), pp. 234-245, (2001).
[2]   Moobar, A.D. and Been, D., Book about pink ponies, New York:
  Wonderful publisher, pp. 34–41, (2003).

Features....

numeric numbering in []s
First, L.L. name style
article title in quotes
year in braces at the end

The closest I could find is MLA Seventh edition, but it never does numeric references.

To answer some suggestions below: I didn't find a style file with the publisher. And no clear style description exists, other than the examples provided. (There is a few more, but the features I listed are the main thing in common.)

Again more information: http://spie.org/x14101.xml#Word is the instructions to follow. They provide an example MS Word file with manually typed citations. While I can use the LATEX file, I already have the paper in .DOC and need a citation style for that. So this is a question of its name.

What journal or publisher is this from? Many of the styles can be found if you know that information. . . .

The system of referencing using numbers is called the Vancouver system. As alluded to by aeismail in his comment, each journal has some form of style based on this format. Hence, you need to know how the specific journal style works in terms of ordering the different bibliographic fields, for example, author, title, publishing year, journal, volume number, page numbers, doi etc. If no clear style description exists you can try to adhere to the specific style by looking at other published articles that use the same style.

Also, in some disciplines there might be a BibTeX style file and all you need to do is load this (and maybe something similar exists for other systems too).

Try looking at: http://editor.citationstyles.org/searchByExample/. This website lets you search for a citation style by example, will provide details of styles that are a close match, and let you know how close they are. Hopefully you'll find the csl style you need. Then you can pop the style into Zotero or another referencing package that supports csl styles.


text:
What is the likelihood of a PhD-holder from the USA remaining unemployed?
In our country (Bangladesh), a PhD from the US is considered as a pinnacle of achievement in one's life. But, in some of my discussions in one of my previous questions I learnt that even a PhD from a US university is not enough to secure a job.
My question is: are there any statistics on the rate of employment amongst PhD graduates trained in the US?

Having a PhD is not a life-long guarantee to employment (and it shouldn't be). At best, a PhD is a key to open some doors in life that would otherwise be locked. It enables you to enter rooms that others cannot. Once inside the room, it's up to you to do good work, as is always required.

@Marc Claesen.....Laughably enough, that is also true for a Bachelor degree.

It's true for any degree, or any other level of certification.  I don't see what's laughable about that point at all.

Tenure. 2. People should be competent at their job or "unassigned" from it, this may imply firing the person. Keeping incompetent people at jobs they don't deserve is nonsense and not laughable, but rather horrible in fact.
A bachelor opens some doors, a PhD opens some other doors. I don't see any "laughability" there either. 4. If you want "success" in life, the place to search in is politics, not academia. 5. If you want to be "unfireable", sole proprietorship is the answer.

Indeed, with a bachelor degree you just have access to different rooms. If you feel the "good work" required in a PhD-locked room fits your ambitions and abilities a lot better than the one required in a bachelor-locked room, then it's worth getting a PhD. But you should not expect to be able to rest your whole life just because you work really hard for a few years.

@Ri49 "But you should not expect to be able to rest your whole life just because you work really hard for a few years". Why? In some countries public servants are chosen with (very competitive) public contests and they are not evaluated ever again, some public servants may claim a "temporal disability" (with full or partial pay and no work) and chain several of them in arbitrarily long periods of time. This is also the expected outcome for many "artists" (actors, singers, etc.), prostitutes, royalties and patents holders, investors and many other people.

Why is this question on hold as opinion based? According to data collected by the NSF in 2008, of 752,000 US-trained PhD graduates in STEM areas at that time, 651,200 were employed, 75,900 were retired, 13,500 were unemployed by choice, and 11,400 were unemployed and seeking work. The unemployment rate was thus 11,400/651,200 = 1.75%. This compares with a general unemployment rate in the US of 7.3%. Thus a PhD certainly helps secure a job (at least in STEM).

^^ Could we open the question so I could post this as an answer? I understand that how the question is phrased makes it appear opinion-based, but at it's core it is a very interesting objective question.

@badroit: Thus a PhD certainly helps secure a job (at least in STEM) It may just be that certain people have personal qualities that let them earn a PhD (intelligence, willingness to work hard), and those personal qualities also make them less likely to be unemployed.

@BenCrowell, more correctly I could say: having a PhD certainly correlates with being able to secure a job (at least in STEM).

Someone that is able to remain motivated (with little supervision) for long enough to get a PhD  is likely to have enough motivation to find some sort of job.

What is the likelihood of a PhD-holder remaining unemployed?
... I learnt that even a PhD from a US university is not enough to secure a job.

The US National Science Foundation (NSF) provides a bunch of statistics on where doctoral graduates in STEM areas (Science/Technology/Engineering/Mathematics) end up working.
This spreadsheet refers to a sampled survey done in 2008, wherein the NSF provide statistics on the employment status of all PhD graduates from STEM areas trained in the U.S.
Across all of their fields, they count 752,000 relevant graduates, of which they estimate that 651,200 were employed at that time (578,700 full-time, 72,400 part-time), 11,400 were unemployed and seeking work, 75,900 were retired, and 13,500 were not seeking work.
An unemployment rate is calculated as:
UR = UE / LF
(UR is Unemployment Rate, UE is number of people UnEmployed, LF is the number of people in the Labour Force [working or seeking work].)
So for the 752,000 U.S. trained PhD graduates considered, the NSF estimated an unemployment rate of 1.7205% in 2008.
This is compared with a U.S. unemployment rate of ~6% in mid-2008.
So to return to your question ...

What is the likelihood of a PhD-holder remaining unemployed?

More than three times lower than the general population (for US-trained STEM PhD holders).
Furthermore, a small percentage of unemployment is inevitable as some percentage of people will always be between jobs for a short time. (Someone more in tune with economics may be able to comment on whether or not 1.72% is close to "full employment", but being unemployed for >1.72% of your career while taking a break between jobs doesn't seem unreasonable.)
A more detailed break-down of the total size of the labour force and the unemployment rate per field follows, so if you're in a STEM area, you can pick whichever field is closest to yours.
#Field                                                    #LF      #UR

All fields                                                662,600  1.7205%

Science                                                   515,200  1.7275%

  Biological/agricultural/environmental life sciences     167,200  1.9139%
  - Agricultural/food sciences                             17,300  1.7341%
  - Biochemistry/biophysics                                25,700  2.3346%
  - Cell/molecular biology                                 19,700  1.0152%
  - Environmental life sciences                             6,500  1.5385%
  - Microbiology                                           12,300  2.4390%
  - Zoology                                                 9,800  3.0612%
  - Other biological sciences                              76,000  1.9737%

Computer/information sciences                              16,400  1.2195%

Mathematics/statistics                                     30,300  0.9901%

Physical sciences                                         118,200  2.3689%
  - Astronomy/astrophysics                                 >5,000  [Supp.]
  - Chemistry, except biochemistry                         59,600  3.0201%
  - Earth/atmospheric/ocean sciences/other phy. sci.       18,500  1.0811%
  - Physics                                                35,600  2.2472%

Psychology                                                100,500  1.2935%

Social sciences                                            82,700  1.3301%
Economics                                                  22,300  0.4484%
Political sciences                                         19,500  1.0256%
Sociology                                                  14,900  1.3423%
Other social sciences                                      26,000  2.3077%

Engineering                                               118,100  1.7782%
  - Aerospace/aeronautical/astronautical engineering       >5,200  [Supp.]
  - Chemical engineering                                   14,700  2.7211%
  - Civil engineering                                      10,500  0.9524%
  - Electrical/computer engineering                        33,800  1.1834%
  - Materials/metallurgical engineering                    12,300  2.4390%
  - Mechanical engineering                                 16,400  1.2195%
  - Other engineering                                      25,200  2.3810%

Health                                                     29,200  1.0274%

[Supp.] means suppressed: numbers are too low to preserve anonymity.

Caveats:

The context is 2008, US-trained STEM PhDs. Unemployment started increasing in 2008 with the onset of the subprime morgage crisis.
A PhD is just paperwork. What you do for your PhD is far more important than the degree itself.
Of course simply being employed is not necessarily the same as having the great job that you always wanted, and certainly we cannot infer from these data that doing a PhD is always the best path for everyone to follow their dreams.


text:
What is a correct way of evaluating the effectiveness of criteria for admissions?
Most colleges and universities in the US base their admissions on high school grades plus a standardized test (SAT or ACT). To decide whether a particular piece of data X is a good admissions criterion, people seem to focus on how well X correlates with college grades. (This may be freshman grades or total college GPA. Some people think  success in getting a degree would be more appropriate. For the purposes of this question, I don't think these distinctions are relevant.)
Isn't this correlation a completely incorrect way of evaluating X as a tool for admissions?
It seems like an apples-to-oranges comparison. A student who has low X is admitted to a nonselective school, where standards are low and they compete with other low-X students. Grades at this school measure how well they did based on the low standards prevailing there. Students who have high X are admitted to a selective school. A "B" grade at Berkeley is not the same as a "B" grade at Cal State Dominguez Hills. A high-X student goes to Berkeley and gets a "B" in calculus. A low-X student goes to CSDH and gets a "B" in calculus. This shows up in the statistics as a lack of correlation between X and grades, since the students differed in X but got the same grade. But isn't that misleading, since the grades mean different things at the two schools?
Is there some more logically justifiable statistic to use in evaluating whether X is a good admissions criterion?

Bingo. This certainly identifies a nonsensical aspect of "higher education". I have no simple solution... but can confirm that the nonsense is real. So whatever decisions we/anyone makes, if we want them to be genuinely sensible, should take this nonsense into account. (Usually this is "impossible", a.k.a. impossible to document un-prejudicially...)

You could measure applicants against how well students matching a similar multi dimensional profile do at your school but that can be very controversial, apparently

Since most schools are ranked, couldn't one create: score = f(grade, school rank) and see how X affects score?

Basic rules of statistics are violated: there is no control group. You'd need a school that manages to admit everybody and provide everybody with high-quality teaching (as opposed to, let's do one year of admitting everybody with our current staff situation so we can figure out good criteria), then correlations would be meaningful.

@Raphael: You'd need a school that manages to admit everybody and provide everybody with high-quality teaching This sounds like a description of a community college. But regardless of the quality of the teaching, community colleges have lower standards.

@BenCrowell: They don't get a representative sample for other reasons, but maybe they have better chances of figuring out some correlations than other schools.

My university also uses the rank of the applicant's high school, at least for in-state students, presumably (hopefully!) as measured by the performance of past admitted students from that school.  So the valedictorian from a low-ranked high school may be rejected, even though middle-of-the-road students from higher-ranked schools are accepted.

I cannot see the relevance of the tag "graduate admissions" to this discussion. As to whether X is a "good" admissions criterion for undergraduate study, it depends mainly on how well the performance of the student meets or exceeds the expectations (or hopes) of the university for its students. Since these expectations are not necessarily quantifiable, statistical analysis is dubious (I believe this was also paul garrett's point).

Define "effectiveness of criteria for admission". Collect data on the potentially relevant independent variables. Do statistical analysis on the resulting data set.
Be careful, you'll probably be considering just the people who were admitted, so the results will be skewed.

A more meaningful study might ignore college grades, which as you point out depend on the type of institution, and focus on downstream accomplishments. It isn't just local and community colleges that distort the picture; several highly ranked Universities have A-no credit policies, so that everyone graduate with a 4.0. Many education schools are effectively A-no credit, with overall GPA's of ~3.9.
I'd like to know how many university professors had low grades or SAT scores, how many NAS members had low grades or SAT scores, and what the profile of CEOs, successful writers, lawyers, engineers etc. was. The purpose of college is not the production of am impressive transcript.

The question asks for a statistic to use in evaluating admissions criteria, and as far as I can tell you do not propose one. This does not seem to answer the question.

Your comment seems quite authoritative, but on examining the question it doesn't just ask for a more appropriate statistical test, but for criteria other than grades or graduation rates. It does not seem that you have read the question.

The actual question, as expressed in the title and the last line of the post, asks for statistics by which to evaluate effectiveness of criteria for admission (grades are an example of a criteria for admissions). The rest of the post is supposed to give context.

Saying the wrong thing repeatedly does not make it so. The title was 'What is a correct way of evaluating the effectiveness of criteria for admissions?' It does not mention statistics, which is your own narrow interpretation. Try thinking outside the box, and improving your reading comprehension.


text:
Downloadable list of all accredited U.S. colleges?
I am looking for a downloadable list of all accredited U.S. colleges. I have found many options to search databases, but I am looking for a downloadable list, preferably in spreadsheet format such as xls or csv.

This question seems to an exact definition of a shopping question: “Shopping” questions, which seek recommendations or lists of individual universities, academic programs, publishers, journals, research topics, or similar as an answer or seek an assessment or comparison of such, are off-topic here

Superficially this might appear to be a shopping question, but I don't think this is a shopping question at all, and I vote to reopen. The distinction here is that the OP is looking for a database that contains a list of accredited colleges. They are not looking for recommendations of  colleges. The question can be answered objectively, and in a single answer. They are not looking for a list of answers, they are looking for an answer that links to a list.

Here's what you're looking for on the U.S. Dept. of Education page in CSV/Excel.
Here's the header of the file:
  Institution_ID    Institution_Name    Institution_Address Institution_City    Institution_State   Institution_Zip Institution_Phone   Institution_OPEID   Institution_IPEDS_UnitID    Institution_Web_Address Campus_ID   Campus_Name Campus_Address  Campus_City Campus_State    Campus_Zip  Campus_IPEDS_UnitID Accreditation_Type  Agency_Name Agency_Status   Program_Name    Accreditation_Status    Accreditation_Date_Type Periods Last Action

Geez, I was all over their site and couldn't find that. Thanks!

No "Date founded" though...


text:
Returning to academia after a year in industry
I need an advice about returning to academia as a visitor/post-doc after working for a year in finance.
I finished my PhD in summer 2013 with a good record of publications (3 papers in high impact journals). Both my bachelor and PhD were in applied mathematics and theoretical physics. Towards the end of my PhD I felt burned out and was unsure if I want to continue the same line of research.
Furthermore, almost all my colleagues who finished their PhDs around the same time struggled to find positions and all moved overseas, which was not an option for me at that time.
During this time a have kept in touch with my supervisor and colleagues. I also kept reading and searching for new interesting ideas. Recently I started to feel enthusiastic again and full of fresh ideas.
My question is do you think it is acceptable to approach my supervisor/colleagues and ask for a position and what is the best way to go about this?

Sure, why not? What you write makes me think that you could tell them exactly the way you tell it here: Being recovered from the PhD and full of fresh ideas.
It is not unheard of that PhD students need to recover from the PhD by doing some different kind of work for a while (such as a couple of years in industry, I've even heard of running a store in a completely different retail sector).
Personally, I think that it should be taken as a warning sign if this happens regularly. However, that consideration doesn't have anything to do with your position, but I think it is worth while remembering when you enter the side of PhD supervisors.
Regardless of how much enthusiasm you've gained back I think it would be a good idea to sit down and recall what exactly drove you away and what caused you feel (close to) burned out - in order to avoid getting there again. I somehow think it unlikely that a postdoc position would be more calm than a PhD.

The last para is particularly good advice.

I appreciate your advice, specially the last part. Indeed, I have thought carefully about it and will make sure to avoid the same pitfalls.


text:
Reviewing test answers with students using a Course Management System
When I give an online test using a Course Management System (aka Learning Management System, LMS), I like to use the "review answers" capability to go over the test with the class and highlight questions that many of them got incorrect. This can reveal common misunderstandings, poorly worded questions, things I didn't teach adequately, etc.
But given that the recommendation is to scramble the order of the questions and even the answers, how on earth can that work? This is why I DON'T scramble questions and answers.
Does anyone have a way of dealing with this? I have spent hours searching the web and found no reference to this issue. But anyone using a LMS must have encountered it? Thank you!

I'm assuming the following:

Students are taking an online test, with questions entered into the LMS
The LMS produces some sort of visual output
You meet with students face-to-face

I've used our in-house LMS, Moodle and Coursera. In all cases the software offers the students randomized versions of the questions, but produces a view for the instructor that shows student responses that are aggregated (sample given at end of post).
If my assumptions are true, you can pick perhaps the three most often missed questions from the test, make a screenshot, and present this to the class.
Or better yet, offer an often-missed question on a slide and have students work in groups to provide a higher-quality group answer.
We might be able to help you more if you correct my assumptions and provide which LMS you are using.
Sample screen shot:

I guess that is really the best solution. It just feels more natural to me to have them looking at it and me looking and then I can scroll down and say, Question 1... Question 2... But I guess that doesn't coincide with scrambling the questions. But focusing on 3 or so that caused trouble is more time-effective.

@user14016, that the questions/answers get scrambled shouldn't be a problem when presenting them to the class for discussion.

Terascore has a built-in way to do this. After the students filled out the test the report shows you question by question how well the whole group did on the question, even showing you which questions are by-design defective - eg. everyone got it wrong, even the top performers.
Full disclosure: I'm one of the founders of Terascore.

Please disclose your affiliation in the answer


text:
When should you publish code on GitHub? Work-in-progress or after publication?
What is policy are you following about publishing data analysis code on GitHub? Do you do it after publishing or as a work-in-progress?
I developed a number of Python algorithms to analyse a large dataset, and I would like to make my work visible.

What's required by: your funding body? your department / university? your PI (if that's not you)? your publisher?

Funding body: I should cite them; department: not clear, probably I should just cite them; PI: I will ask; publisher: no idea (it's still not clear to what journal we will send our paper). By the way, in general what's the policy for natural sciences?

Definitely share it. I don't know of any policies. See also: http://academia.stackexchange.com/q/16785/386

Version control is about work in progress, by definition. If you are worried about people downloading incomplete or buggy code, make releases.

I don't think this post has a clear question - it reads like a polling question to me.

Git is a computer program for managing code. When you say "git", do you perhaps mean "GitHub"? (GitHub is one of many many websites used for sharing code.)

@albus_c You might consider putting it in a private repository until you're ready to publish.

May want to have a look: http://academia.stackexchange.com/q/17740/5851

There is a movement gathering strength lately to encourage publishing the code:
Nature-Publish your computer code: it is good enough
Or, more vehement:
If there is no code, there is no paper
The reasons outlied on the article are very reasonable. If you are expected to publish detailed derivations, experimental methods, and proofs of theorems, why would you be allowed to keep the code? No one will accept a theorem if you claim: "the proof is too messy to show, but hey, here are three cases where it works".
I think the best way is to publish the code used as supplementary material, and include a link to the repository, so people can get the improved versions. If you are concerned about people using too bleeding edge versions, make releases, but leave the development public. This will also help you get bugfixes and contributions.
Thank you for wanting to release your code. I really believe this attitude will help make research better.
Edit:
After some time, I have something to add. Most of the code in an application is there for "administrative purposes": load and write data, massage, check conditions... For publishing, that part can be as hackish as one needs it to be. The real "research" is usually in a small part. That is where one should dedicate one or two hours of adding a few comments and clearing the code.
For the rest, a docstring in the functions or a paragraph explaining the aim, should be fine.

To touch on "when", I generally do it when I submit the paper - your odds of being "scooped" by someone once you've hit the submission process is pretty low.

No question code can be night or day, I've read countless papers where code would have been extremely valuable.  Even if the academic used less common language, it can be of immense value.

the proof is too messy to show, but hey, here are three cases where it works.

Actually, this happens regularly in mathematics. If the examples are representative of the proof, then it can be considered enough to give a sketch and not fill in the messy details.

Styles and technologies come and go. Git and Github are the flavor of the moment. Tomorrow it will be something else.
What is more fundamental is that scientific results are normally expected to be reproducible. If the code has secret details in it that are crucial for producing the result, then the result is not reproducible. If the code is simply the embodiment of the methods described in the paper, then there is no problem with not publishing the code.
As an example, there is a device called the Bodybugg that people buy and strap onto their arms in order to measure (or attempt to measure) how many calories they're burning each day. There is publicly available information on what sensors the device has built into it, but the algorithms used for putting the sensor readings together to get an estimate of energy consumption are proprietary. That means that any scientific research that uses a Bodybugg is basically worthless.
On the other hand, there can be perfectly legitimate reasons for not wanting to release one's code. For example, there could be a concern that people who lack the relevant expertise will play with it and use it to publish their own half-baked "gee-whiz" papers that turn out to be wrong. That then harms the reputation of the original author. The author may also not want to be in the business of answering questions from lots of people using their code, or they may want the freedom to make major changes that would upset users who were counting on the code to remain stable and backward-compatible. Science is not software development. Scientists want to focus on doing science, not on software distribution, licensing, making regular releases, and supporting a user community.
It may not even be legally possible to release a working version of the code. E.g., it may have some old FORTRAN routines in it that calculate Clebsch–Gordan coefficients, and if the author of those routines is dead, then it may not be legally possible to publish them.
I'm also skeptical about the long-term value of releasing code in most cases. Github will be gone in five or ten years, and the vast majority of the software it hosts will then cease to exist, since the vast majority of coders will not bother to migrate their code.

The Nature letter in my answer addresses your concerns about questions. The software is provided "as is", no necessity to support it. And there is no shame in other people doing bad research with your code, as there is no shame on Schrödinger every time someone claims homeopathy works because of quantum stuff.

Github disappearing in ten years means there will be no code available then; not publishing means there will be no code ever. This is why the code should be published as complementary material, then the journal will be responsible for keeping it for the record. And if I am interested in some code on Github, a git clone and I have on my hard drive exactly what Github has: I no longer depend on them, they can disappear. Keeping mirrors in git (or any other DVC) is trivial.

"Github can disapear" is no argument at all. Chemical or material suppliers can disappear too, but it is still unacceptable to not describe in detail how you obtained your materials.

Sf.net (sourceforge), which was arguably never as popular as github, has been around for 15 years. Though it has certainly lost some "market share" to github and others, it doesn't seem to be showing any signs of disappearing.

"Scientists want to focus on doing science, not on software distribution, licensing, making regular releases, and supporting a user community." This is not a legitimate argument. Code on Github doesn't have a support contract, it just makes it easier for people to figure out what you did.


text:
Is it wrong to impose a challenging time constraint on exams?
I've noticed that many professors seem to impose extremely tight time limits on their exams, even though the subject in question would never have such extreme time constraints in real life. For example, I cannot imagine a situation where I have to write 50 lines of code on a piece of paper in less than 15 minutes.
Is it reasonable for me to complain about such constraints or is it normal to expect fast exam performance from students?
Note that I'm not talking about the actual exam difficulty, just the time constraint built around it.

As a professional programmer, I can't imagine a situation where I have to write 50 lines of code on a piece of paper in any time period. A quite bad (but not necessarily atrocious) job interview might call for a lot of code on whiteboard or paper, but it's fairly futile to write that amount of code without the capacity to test it. The time limit is not what makes that task artificial.

@SteveJessop: That may depend on the strictness of how correct the code must be - I have often found myself scribbling down an algorithm on paper while being underway (also on my way to or from work), so I don't forget it by the time I get to continue development of the respective project at a computer, and also to make use of the time where I don't have a computer with a keyboard around. Of course, seeing an analogy to an exam task here requires that the exam task be considered correct when it is conceptually correct, with a rather lax interpretation of the intended syntax.

@O.R.Mapper: fair enough. I don't produce what you'd reasonably call "code" under such circumstances, but maybe that's just because I didn't do a CS degree and hence didn't take these exams :-) Rather, I write enough to remind myself how to write the code later, which normally is a key insight or a picture.

If you need 50 lines of code to answer a 15-minute question, the time constraint could be unfair, the instructor might be looking for a reasonable line of attack instead of a fully polished solution, or you might just be doing it wrong.

As my math teacher once said: "I can run 100 meters, too, but I still don't go to the Olympics".

@SteveJessop: Personal preference may differ here. I usually find myself stumbling over the ninety-ninety rule when I just prepare a conceptual drawing or keywords and assume that was the "hard part", or 90% of the effort involved.

@O.R.Mapper: I don't expect it to be trivial to convert the rough notes into code, I just think the same rule would apply if I assumed that writing code on paper without testing it is the 90% "hard part" :-)

The objective of an assessment can vary from one lecture to another. Quite often, the point is simply to validate whether the expected learning outcomes of the lecture have been met. For instance, if an expected outcome is to know almost by heart how to write an array sorting program, then it's quite reasonable to ask to write a small amount of lines of code in a fixed amount of time.
If, on the other hand, the expected outcome is at a higher level of understanding (in the sense of Bloom's taxonomy), for instance by asking to design and assess a new data structure to handle a new problem, then it could be more reasonable to expect more time.
The problem you are referring to by "would never have such extreme time constraints in real life" is addressed with the notion of authentic learning (Rule, 2006), which identifies the four following themes for a learning to be authentic:

1) the activity involves real-world problems that mimic the work of professionals in the discipline with presentation of findings to audiences beyond the classroom;
2) open-ended inquiry, thinking skills, and metacognition are addressed;
3) students engage in discourse and social learning in a community of learners; and
4) students are empowered through choice to direct their own learning in relevant project work.

Authentic learning comes with its upsides and downsides (the reference linked acts as a survey paper, if you are interested), so it's not necessarily the best approach. In particular, (Lombardi, 2007) note that the reliance on traditional instruction is not simply a choice made by individual faculty—students often prefer it. For instance, not everybody wants to be tested on writing code in a highly complex environment, using bugged code written by other people, implementing specifications that are sub-optimal, but the client want them in this way, which could be a typical real world situation.
I don't think you should complain about time constraints, but if you believe that authentic learning would be more beneficial to you and your fellow students, you should probably discuss with the professor about the objective of the assessment.

Lombardi, M. M. (2007). Authentic learning for the 21st century: An overview. Educause learning initiative, 1(2007), 1-12.
Rule, Audrey C. (2006). The Components of Authentic Learning. Journal of Authentic Learning, Volume 3, Number 1, Pages 1-10, August 2006

A huge +1 for an answer that ends with a bibliography.

I very much feel the pain for "need to work with bugged code, confused specifications, suboptimal tools". But some form of the above is an essential skill for any professional... and better learn it as a student (where not much is lost if the result explodes) than on the job.

This is not really an answer but some educated guess. Most instructors do not plan to impose a challenging time constraint for exams. Usually we tend to overestimate how easy is the exam. The younger is the instructor, the more optimistic he/she is about students and more eager to write interesting questions. For subjects that require lots of drilling, tight time constraints are helpful in the sense one can immediately screen out the students who did lots of homework/problem solving from those who have to think from scratch.

When starting out teaching, I was given the rule of thumb: Take the time you need to solve the exam, multiply by four. Not that I follow it each time rigurously, but it stuck.

In my experience, challenging time constraints are a hindrance because they induce panic, which inhibits the ability to reason through a problem and make a thoughtful answer to it.
I'm currently tutoring two students on grammar for a business English class. Their teacher sets 100 questions for each 50 minute exam. That comes down to 30 seconds for each question. This tactic makes a mockery of all my attempts to teach them the logic and reason behind grammar. The students barely have enough time to figure out what's being asked of them, let alone apply any rules or reasoning principles.
The students feel this is unfair; they learned the rules, did the homework (which average 300 questions per set), and now they're failing the tests because they can't go through the whole process of reading a problem, conjuring up the correct rule, and finding the correct answer in 30 seconds. This causes bitterness and low morale. It makes them feel that all the work they put in to the homework was a waste of time.
It's not feasible in the standard method of testing to give students unlimited time, and most schools don't have the resources to set aside computer labs for exams so CS students can write code on them. But there's no reason to make the time constraints more of a problem than they need to be. Make the questions easier; put fewer questions; and split long questions into multiple parts, which are graded independently. I've found all of these reduce the pressure, and the second two options let you still put challenging problems on the exam.
In short, yes, I do think it's wrong to impose a challenging time constraint on an exam. Not only because it's unrealistic, but also because it puts artificial limits on the students' performance, which the students will recognize as artificial and react badly to. Unless you're trying to teach people to program on the bridge of the Starship Enterprise during a Red Alert while an anomaly is sucking the ship in, lighten up the time limits.

+1, but I'm not so convinced. In the real world, conversation requires a quick and natural stream of thought and ease of usage of language. While a 100 Qs in 50 mins is definitely harsh, I think it's not completely unjustified because your students need to develop the ability to quickly use their grammatical skills in conversation.

@shortstheory: seems like both things are reasonable to test, and the problem here is that the tutor thinks one thing should be tested and the examiner thinks another thing. In conversation you just have to get it right, you don't have to show your working. I sort of wonder whether a typical native speaker of English would have a sporting chance on the test, though, given that we can spot most errors and correct them extremely quickly, but might be totally ignorant of the theory required to justify our answers.

... that is to say, is the correct answer of form, "If I were you" (in which case the exam is testing fluency, which for conversation must be fast), or is the correct answer of form, "one must use the subjunctive mood" (which is testing academic knowledge of grammar, and where the argument that you have to do it instantly in real life business English doesn't really hold)?

@SteveJessop I agree, but I assumed that the test is an MCQ so you would just need to mark the option and not show your working

@shortstheory: sorry, I was getting to that, see my second comment :-)

Ah, I see. I'm convinced :)

@SteveJessop Actually, the students were native speakers of English, albeit low-income ones unfamiliar with formal written English. The class was supposed to teach them about business writing, but it did so by using (from my knowledge of linguistics, confusing and borderline wrong) grammatical explanations like you'd usually see in an ESL class. You didn't need to show any work, but the questions were purposely designed to confuse people with their background, so you did need to apply the theory to find the right answer.

I think what shortstheory says in the first comment is valid if you're testing non-native speakers. There is some justification. But in this context, the knowledge was meant to be applied to business emails and letters, so, as Steve Jessop says, the argument that you have to do this instantly doesn't really hold. You have time to think "do I need 's or s' here?", which you didn't have on the exam. I found this unfair and counterproductive. I told the students to learn the theory, even if they didn't have time to use it, because it would help them in the long run.

It is reasonable to bring up the issue of exam difficulty, although I would try to bring it up in a way that is not accusing the professor of being unfair, perhaps bringing it up as part of a question of how to study next time, or asking if this is going to be typical of his exams.
It may be that the professor sees value in the tight time constraint that you aren’t aware of.  For example, they might be asking something very challenging, knowing that very few students will succeed completely. It is very common to have test questions ranging from fairly easy to very difficult, with the expectation that most students will miss the most difficult questions. Partially correct answers show partial understanding. In this case, a coding question with a time limit will show up problems that might be harder to see in an assignment where a student has unlimited time. One sort of issue that will show up is that there's big difference between the uncompleted code of somebody who starts writing random code in the hopes it will sort itself out as it comes, vs somebody who takes 3 minutes to plan an approach, and the rest of the time implementing that approach. But if all you are looking at is the final result is a problem set, the difference is less obvious, even if the first person takes twice as long to reach the same result. There could be other possible benefits to this type of question as well.
It’s also possible that the professor isn’t aware of just how difficult the test is, and your question will bring that to his attention. Although if nobody finishes that question, that alone will give most instructors a clue that their estimated difficulty was off. But generally, exams have very little to do with career prep, and more to do with assessing student ability.

Some professors seem to be using this paradigm of testing; instead of giving ample time so that it's reasonable to finish the test, they'll intentionally make it so even the best-prepared testee will only have time to finish 75% of the test. Of course it will be curved based on this. It separates those who know the material really well from the people who only know a little, and apparently creates a pretty good bell curve, which seems to be what everyone is after these days.

It will also separate the people who have test anxiety and the people who don't, since anyone with the slightest amount of anxiety will fail.

anyone with the slightest amount of anxiety will fail — [citation needed]

"It separates those who know the material really well from the people who only know a little" -- [citation needed]

It's not uncommon to have challenging time constraints. Some American tests like the ACT and SAT have a tight limit built in to reward students for being fast.
However, many of these tests are scaled, in that no matter how well or how badly the students do, the grades are normally distributed..

Opinions on this vary, but personally I prefer to avoid cases where the major difficulty of the exam is the time limit.  If the time constraints are too challenging then the process becomes a test of who can write the fastest, rather than who knows the material well.  Personally, I prefer to give a time constraint that allows students to think about each question and go at a reasonable pace, with questions that range from easy marks to hard marks, to test their actual understanding of the subject matter.  I have also adopted the following rules for time limits in exams:

Exam length for undergraduate-level courses: As a rule-of-thumb, the teacher should be able to correctly complete the exam in one-third of the time allocated to students, from a position of initial ignorance of the answers. So if you are setting a three-hour exam, you should be able to do the whole thing, including all working out and writing up of answers, in one hour.
Exam length postgraduate-level courses: As a rule-of-thumb, the teacher should be able to correctly complete the exam in one-half of the time allocated to students, from a position of initial ignorance of the answers. So if you are setting a three-hour exam, you should be able to do the whole thing, including all working out and writing up of answers, in one-and-a-half hours.

These time limits have served me well in my courses, and it ensures that I am testing students on their knowledge of the material, rather than their ability to write really fast.

Tests with tight time constraints are probably created by instructors who are not familiar with personality profiles like those revealed in Meyers-Briggs personality tests. People with some personality types tend to test slowly but deliberately, while others will test quickly but haphazardly. Of course, I'm also sure that there are those that test quickly and deliberately, as well as slowly and haphazardly. But, in the end, if instructors thought about it, they would find that they most likely prefer students go slowly but deliberately, thereby raising their percentage of correct answers. I imagine that most employers prefer slow production in order to achieve a high rate of perfection, thereby limiting wasted time and material.

I am an industry retiree. My personal observation is, most employers prefer to quick production in order to achieve a high production rate, thereby limiting short term cost.

It's all relative, though. My employer wants me to get a bit of a move on, sure, but very rarely do I work at the intensity that I achieved for 3-hour blocks during my finals exam. Quite aside from anything else, I was intellectually exhausted and even physically fatigued after two such exams in a day, where in a normal work day I'd still have an hour and a half to do!

Furthermore, I reckon my error rate was higher in exams than in my professional work, and that's fine because the mark scheme in my exams was to do as much correct work as possible in the time allowed, whereas the "mark scheme" in my professional work puts high negative weight on errors. I mean, we don't have an actual mark scheme, but we'd rather put out one page of correct work than three pages of work, one of which is entirely incorrect and we don't know which. So yes, my employer is more error-averse than my examiners were. And a thesis assessor is more error-averse than a finals examiner.


text:
Are open-book exams generally a superior way to test understanding on practical courses?
I understand why it would be a bad idea to allow math students access to paper materials as it  is necessary for students to know the basic math formulas by heart.
But what about advanced practical courses where it is important to "understand" rather than "know"? For example, knowing statistical formulas won't help if you haven't understood how to apply them. Knowing STL commands by heart is also useless for programming, unless you're actually good at programming. Intel's x86 developer manual won't help a complete newbie in Assembly programming.
Therefore the question is whether it is reasonable to believe that good courses should be focused on "understanding" the subject rather than "memorizing" it, meaning that the instructor should have no problems with his students using paper materials on the exam?

Possible duplicate: http://academia.stackexchange.com/questions/17763/is-it-good-practice-to-let-students-consult-their-text-materials-during-exams?rq=1

The accepted answer on that question is about mathematics, which isn't a concern of mine since it's a purely theoretical subject for CS students.

Also, I think it is undisputed that courses should be focused on "understanding" rather than "memorizing". The question should rather be whether open-book exams are generally a superior way to test understanding (which is, imho, not self-evident).

Feel free to edit my question. I wasn't sure how to phrase it correctly in the first place.

I like the idea of a "cheat sheet". Students may bring in one or two sides of A4 paper with handwritten notes. The act of carefully constructing this summary sheet is usually much more useful than the notes on it.

You might find this article on a computer science class valuable (1). The tl;dr is that sections allowed student-prep notes showed no overall improvement, but mostly because many students didn't bother to create good student-prep notes. Good notes = good grades.
A quick browse of a review/ meta-analysis article on the topic (2), indicates that open-note exams can decrease student anxiety, and that student-prepared notes tend to produce larger improvements than open-book exams.
As an instructor, I've found students often spend energy only where they feel it is needed. I would recommend providing sample exam questions (as a group activity in class?) that gives students a good idea of what will be required and what sorts of notes would be helpful. This will help them expend the energy in preparation instead of frantic in-exam page-flipping.
Thanks for wanting to trigger deep learning in students!

Duncan, D.G. (2007). Student performance shows slight improvement when open notes are used during information system exams. Journal of Information Technology Education, 6, 361–370.
Larwin, Karen H., Jennifer Gorman, and David A. Larwin. "Assessing the Impact of Testing Aids on Post-Secondary Student Performance: A Meta-Analytic Investigation." Educational Psychology Review 25.3 (2013): 429-443.

To me, the interesting questions are ones that these studies don't seem to address. I think (but have no evidence to prove) that the standard emphasis on memorization in school is destructive to learning, because it shifts the focus away from critical thinking and higher-level reasoning. Time spent on memorization is also wasted time that could have been better spent on actual learning; the memorized facts are forgotten within a month after the final exam.

I believe there are two main counterarguments to this line of thinking:

Examinations in which students are given unlimited access to material tends to lead to more challenging exams, since everyone in principle has access to all of the available material they can get their hands on.
If unlimited resources are allowed, this potentially gives students who can access more materials an advantage over those who don't.

The second problem is the more serious, in my opinion, but both are major challenges that must be overcome.
Personally, I "split the difference": I allow students to bring in a limited amount of notes to help them. They can decide what they need to include or not, but it's their choice. Also, by placing the restriction, no one is inherently advantaged

In the internet age, what CS student is limited in the materials available to him or her? It is unreasonable to assume that the student won't have access to a computer and an internet connection, especially as a CS student. (And even if they don't have their own, the school will; how would they do any of their CS work without such being available?) And, while there are CS resources behind paywalls (books, etc.), the same information is available (with different presentation) from free sources -- although the student may have to do more hunting to find it.

"In any reasonable open-book exam, the answers are not in the book."

Personally, I'd argue that an exam is not the way to judge a practical course at all. but that is not the question you asked. And changing from open book to open internet is moving the goalposts again.

When I don't want students spending a lot of time memorizing things, I simply provide supplemental information on the exam itself.
For example, imagine a course textbook lists 12 reasons why something might happen. Rather than expecting them to memorize all 12, I might list all 12 on the exam, with an application question such as:

The book lists these 12 causes; which of the 12 is most likely to lead to a catastrophic failure? Explain.

In the case of a statistics class where "it is important to understand rather than know," and, "knowing statistical formulas won't help if you haven't understood how to apply them," you could simply compile a list of formulae from the course textbook and put them on the last page of the exam.
By the way, I agree with Moriarty's comment: "The act of carefully constructing this summary sheet is usually much more useful than the notes on it." My approach doesn't have this benefit, but it does have a few advantages of its own. For one, a student is less likely to get a better score simply because of a better-prepared cheat sheet. For another, the student might focus on understanding the material at a higher level instead of transcribing lower-level details.
My general rule is: If something is hard for me to remember without looking it up, then I don't expect students to memorize it for an exam. I will supply it instead.

Here's another idea to consider. (I haven't used this one, but I had a professor who did, and I admit I liked his reasoning.)
One professor of mine had a custom of administering open book, closed notes exams. (In other words, you could use your book, but not a cheat sheet.) Why this way, instead of the other way around? Because, he said, "Most students won't keep their cheat sheets, but a lot of them will keep their textbooks." There were no restrictions on writing inside the margins or the back cover (in fact, that practice was encouraged). Sure enough, I still have a book from his course, and I just opened the front cover, and found my cheat sheet still intact, with plenty of complex equations and hyroglyphics in my own handwriting.
As more and more campus bookstores offer e-readers, this approach might become outdated. But I still think it's worth sharing; I always appreciated how this particlar professor was always looking years down the road, as opposed to just the end of the term.