text:
What is the most computationally efficient genetic algorithm?
In researching genetic algorithms, it seems that there are various methods of selection and other operator methods that can significantly change the performance. For example, this picture contains some of the methods that could be used:

Presumably, you can mix & match these operators to optimize whatever problem you're trying to solve.
What most people care about is how many iterations it takes to get to the target. This is understandable, but I've seen things that would be inefficient in real systems such as:

using sort on the current population $\mathcal{O}(n \log n)$ and picking the first n members for the mating pool

appending to a constantly resizing slice to create a mating pool instead of rewriting on the current array

What I am looking for is, how can I arrive at the target using the least amount of computation and memory possible. The number of iterations and the time taken to get there is still a secondary priority.
It's possible that it may be the process of picking the right operators, but what I am also considering is how parallelizable the implementation could be as well.

To me, what you are asking is a variation of the traveling salesman problem.  You can only know you have found the best way if you have tried all of the ways, which is the key challenge and why we use heuristics rather than algorithms for this type of problem.  Genetic search algorithms seem to be this type of optimization approach.

First of all, for a lot of realistic problems, the fitness function evaluation is usually orders of magnitude greater in complexity than the rest of the genetic algorithm. This is not always true, but often is true (e.g. imagine trying to optimise a simulation where you need to execute the simulation completely to obtain the fitness). So optimising the GA itself might only be helpful when the fitness function is lightweight (e.g. it's a mathematical function as used in some competitions).
While your diagram shows a lot of operators, it doesn't show variations of the GA itself. Here are two:

generational: selects a new population at each iteration after performing mutation and crossover on the existing population to generate offspring (which are usually more in number than the population size)
steady state: maintains a single population and replaces only individuals during each iteration with their offspring according to selection/replacement rules, so uses less memory than a traditional GA

If you really wanted to maximise efficiency in the GA, a steady state approach would probably work best combined with tournament selection, as tournament selection does not require any sorting. The mutation/crossover operators you list are likely to all be quite efficient, but simpler methods are likely to to be the most efficient (e.g. bit flip mutation+1 point crossover). For a realistic problem, however, problem-specific operators usually work best.


text:
Could humans hurt a conscious or sentient AI?
If a conscious AI is possible, would it also be possible for someone who knows what they are doing to torture (or hurt) the AI? Could this be avoided? How?
This question deals with computer-based AI, not robots, which are as conscious as people (this is an assumption of the question). The question wonders how a crime as hard to trace as illegal downloads, but far worse ethically, could be prevented. Note that despite most people being nice and empathising with the robots, there are always the bad people, and so relying on general conscience will not work.

This question is not fully clear to me. What do you mean by "computer-based AI"? Why wouldn't a robot be a computer-based AI? Do you mean an AI without a body, so only software?

The article Children Beating Up Robot Inspires New Escape Maneuver System is based on two research papers about an experiment in a Japanese mall that led to unsupervised children attacking robots. The research paper you're interested in is Escaping from Children’s Abuse of Social Robots.
In that research paper, researchers were able to program the robots to follow a planning simulation to reduce the probability of abuse by children. If it detects children, the robot is programmed to retreat into a crowd of adults (who can then discipline the children if needed). This happened because the researchers saw that it was only children who were beating up the robots in the mall in question.
They discuss trying out other options though:

In this work the robot’s strategy to prevent abuse was to “escape”, i.e. move to a location where it is less likely abuse will occur. One could ask why the robot cannot overcome the abuse. In our preliminary trials we have tried several approaches, but we found that it is very difficult for the robot to persuade children not to abuse it. For example, we changed the robot’s wordings in many ways, using strong words, emotional or polite expressions, but none of them were successful. One partially successful strategy was the robot ‘physically’ pushing children. When its way was blocked, it would just try to keep going and behave as if it will collide into children and force its way through (under careful monitoring from a human operator). We observed that children at first accepted the robot’s requests and obeyed them; but, very soon they learned that they are stronger than the robot so they can win if they push, and also that they can stop it by pushing the bumper switch (attached on the robot for safety). After realizing that, they just continued with the abusive behavior. Obviously having a stronger robot would present a problem for safety and social acceptance so dealing with such abusive situations remains difficult.

But let's interrogate your question further:

If conscious AI is possible and is wide spread, wouldn't it be easy for someone who knows what they are doing to torture AI?

Why would you consider such torture to be wrong? After all, one could argue that the machine won't really 'experience' pain if you torture it...so it should be morally okay to torture the machine then. It may be respond as if it is in pain, but it's dubious whether the ability to simulate an emotional state such as "being in pain" is equivalent to actually being in that emotional state. See the question Is the simulation of emotional states equivalent to actually experiencing emotion? for more discussion on this topic.
You can make such an argument, but it won't really work on an emotional level because most humans would feel empathy towards the machine. It may be hard to justify it logically (and it may be based on humans' tendencies to engage in anthropomorphism), but we feel this empathy. It's this empathy that caused you to ask this question in the first place, caused researchers to figure out how to protect a robot from being beaten up, enabled police officers to arrest a drunken Japanese man for beating up a SoftBank robot, and made many humans upset over the destruction of hitchBOT. And that's how AI abuse would be avoided - human empathy. If most humans care about the welfare of machines, they'll make it a priority to stop those few humans who are able and willing to abuse the machines.

EDIT: The OP has edited his question to clarify that he is talking about software, and not about robots. For robots, you can rely on anthropomorphism to produce some level of sympathy, but it's hard to sympathize with raw lines of code.
You're not going to stop abuse of algorithms. Put it frankly, since the algorithms aren't like us, we aren't going to extend the same sort of empathy that we would to robots. Even chatbots are kinda iffy. If you could get people to sympathize with lines of code though (possibly by making a convincing simulation of emotion and sapience), then the above answer applies - humans anthropomorphize the machine and will come up with countermeasures. We aren't that level yet, so "stopping AI abuse" will be a low priority.
Still, some failsafes could be programmed in to limit the damage of abuse, as detailed in this thread on chatbot abuse - making the bot respond in a boring manner to make the abuser feel bored and move onto the next target, responding back to the abuser in a "battle of wits", or even just blocking the abusers from using the service.
These failsafes are cold comfort to those that want to prevent abuse, not respond to it.
Also...an abuser can happily learn how to program an AI to then abuse to his/her heart's content. Nothing can be done to stop that, and any possible measures to stop said abuse (such as monitoring every human being to make sure they don't program an AI to abuse) will probably cause more damage than it'd solve.

I suggest you look at all the ways we have tried to stop people from abusing OTHER PEOPLE. There is no ethical grey area here - everyone is clear that this is wrong. And yet people are murdered, raped, and assaulted in their millions every day.
When we solve this problem with regard to human victims, the resulting solution will most likely work just fine for AIs as well.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
What happens if 2 genes have the same connection but a different innovation number?
I have read the Evolving Neural Networks through Augmenting Topologies (NEAT) paper, but some doubts are still bugging me, so I have two questions.

When do mutations occur? Between which nodes?
When mating, what happens if 2 genes have the same connection but a different innovation number? As far as I know, mutations occur randomly and thus it is possible that 2 genomes have the same mutation.

When do mutations occur and between which nodes?

There are two types of mutations in the NEAT model, each of them appears randomly during one epoch on different individuals; the number of structures affected by mutations may vary depending upon the nature of the problem.

A new gene/node is added to the structure and properly linked.
A new connection between two nodes is added.

When mating, what happens if 2 genes have the same connection but a different innovation number?

During a single epoch/generation every mutation is tracked and if the same mutation appears it can't have the same global innovation number. Same mutation, same innovation number. In that way, during the mating phase, there is no decisional problem which leads to prefer one structure to the other.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Has anyone used YodaQA for natural language processing?
Has anyone used YodaQA for natural language processing? How easy is it to link to a document database other than Wikipedia?
We're thinking we can create a bot to use AI to analyze our developer and user documentation and provide a written or spoken answer in reply. YodaQA comes linked to Wikipedia for starters, but we'd need to link to our own source info. I'm trying to get an idea of the development time required to set up the AI and then to link to the database.

i am interested in the exact same topic: creating a question answering system for a programming API. in my case, the API is the Revit API. i have heaps of existing documentation and would like to implement a question answering system based on that. i would be very interested in hearing what you find out your end, and also in any answers to this question anyone can provide. thank you! ontology sources for Revit API: http://thebuildingcoder.typepad.com/blog/2017/01/virtues-of-reproduction-research-mep-settings-ontology.html#8

Look at yodaqa issue #17 Domain Adaptations for a pretty extensive discussion of this and closely related topics.


text:
How does mating take place in NEAT?
In the Evolving Neural Networks through Augmenting Topologies (NEAT) paper it says (p. 110):

The entire population is then replaced by the offspring of the remaining organisms in each species.

But how does it take place? Are they paired and then mated? Could this lead to fast extinction? Or are they pair each with each other? This would lead to overpopulation very fast. How are individuals paired, in NEAT?

NEAT has a constant number of organisms in its population, which prevents overpopulation from happening.
The process of mating includes the following steps.

The worst networks from every species are removed.
All species receive a number of offsprings that they can have. This is calculated by an adjusted neural network fitness.
Offsprings for species are divided among neural networks in those species. Fitter neural networks have more offsprings.
Networks from the same species are combined and create an offspring.

Speciation prevents fast extinction.


text:
How can I design an AI that knows when its being spoken to?
I am trying to make an intelligent agent similar to Jarvis from Iron Man, but much less complex. However, I want my AI to be able to determine if I am talking to it or not. So, I plan on having it always listen to my voice and convert that to text. However, I am not sure how I can train the AI to recognize if it is being spoken to or not?

You may have a look at jarvis for inspiration.

Phrase detection instead of text-to-speech
It's worth noting that detection of particular phrases or commands is considered a distinct problem, different from text to speech / text transcription.
While you can simply convert everything it hears to text and then look up keywords there, a specialized detector that directly tries to match incoming audio to a small subset of commands can be done with better accuracy and less processing power required. For this reason, this would generally be the preferred approach in commercial products.
However, for beginner experiments with home automation, you should probably start with choosing an existing speech analysis API where all the audio and natural language parts are appropriately implemented by someone else. Building a good speech command analysis system from scratch is a major undertaking by itself, and you will have your hands full with developing an "artificial intelligent agent"; as a rule, you don't want a project where you have to tackle two major open-ended problems, pick one of them and then you'll have a chance to achieve something interesting there.

I am making this in python 2.7 what active API's would I be able to use for this type of task @Peteris

Cheep digital assistant "AI" 's have a call word Hey, <AI's NAME>
I assume you want a bit more than that.
You could train it to figure out which words in some context determine if you are engaging with it or not.
If your only question to the network is if you are engaging with it or talking to someone else this is all you'd need.
Index a dictionary or have it build one from collecting words (building a dictionary from scratch is a better solution it saves space in the short term and is more easily expandable in the long term) and score words based on usage in engaging speech and non-engaging speech or what you want it to do.
Build on that with an index of multi word strings.
By the end hopefully you will have a table of contexts when you are engaging with the AI when you definitely are not and some grey area.
The training process is long and tedious but if you have a recording of you talking and not talking to the AI and you feed it with such knowledge and you breed the network you should have it get okay at determining context.
If you have to sit and hold it's hand for 2-72 hours while it grows up it will likely be painful, although you may end up with a better result.


text:
How can we use crowdsourcing for deep learning?
Most companies dealing with deep learning (automotive - Comma.ai, Mobileye, various automakers, etc.) do collect large amounts of data to learn from and then use lots of computational power to train a neural network (NN) from such big data. I guess this model is mainly used because both the big data and the training algorithms should remain secret/proprietary.
If I understand it correctly the problem with deep learning is that one needs to have:

big data to learn from
lots of hardware to train the neural network from this big data

I am trying to think about how crowdsourcing could be used in this scenario. Is it possible to distribute the training of the NN to the crowd? I mean not to collect the big data to a central place but instead to do the training from local data on the user's hardware (in a distributed way). The result of this would be lots of trained NNs that would in the end be merged into one in a Committee of machines (CoM) way. Would such a model be possible?
Of course, the model described above does have a significant drawback - one does not have control over the data that is used for learning (users could intentionally submit wrong/fake data that would lower the quality of the final CoM). This may be dealt with by sending random data samples to the central community server for review, however.
Example: Think of a powerful smartphone using its camera to capture a road from a vehicle's dashboard and using it for training lane detection. Every user would do the training himself/herself (possibly including any manual work like input image classification for supervised learning etc.).
I wonder if the model proposed above may be viable. Or is there a better model of how to use crowdsourcing (user community) to deal with machine learning?

First thing, you need to give more credit to more reliable users. You can establish the credit from amount of data they send, and a feature, where other users can review other's feed and classify it. From there, you will have a measure of certainty to what data is good and what is not.
You will need to implement a centralized server, unless you're trying to do some kind of a peer-to-peer trust systems, but I don't think smartphones are powerful enough to do training themselves.
You will need big machines for training NNets. Don't trust users to have them. You would end up with tons of badly trained NNets, which don't make for a good CoM.

There is already an approach similar to the one you describe: federated learning (FL), where local nodes (e.g. mobile, edge devices but also companies of different sizes) keep the training data locally, so each node might have different (unbalanced and non-i.i.d.) datasets, and models, which then need to be aggregated.
One possible definition of federated learning is

Federated learning is a machine learning setting where multiple entities (clients) collaborate in solving a machine learning problem, under the coordination of a central server or service provider. Each client's raw data is stored locally and not exchanged or transferred; instead, focused updates intended for immediate aggregation are used to achieve the learning objective.

If you are interested in more details, there are already many sources on the topic that you can find on the web, but I would recommend the paper Advances and Open Problems in Federated Learning (2021, by Peter Kairouz et al.) or the Google's article Federated Learning: Collaborative Machine Learning without Centralized Training Data (2017). There are also software libraries for FL, such as TensorFlow Federated (TFF).
However, note that there are other approaches to distributed machine learning/training.


text:
Are there any ongoing projects which use the Stack Exchange for machine learning?
Are there any ongoing AI projects which use the Stack Exchange for machine learning?

There certainly appear to have been research projects involving some form of text mining / information retrieval /etc. and StackExchange sites.
Some examples I was able to find through google/google scholar (unlikely to be anywhere near an exhaustive list):

TACIT: An open-source text analysis, crawling, and interpretation tool describes numerous text-crawlers for a variety of sites (including Stack Exchange sites, but also Twitter, Reddit, etc.). At first glance, this appears to be primarily about crawling, not about doing anything else with the data afterwards. Searching for other papers that cite this one on Google Scholar may yield interesting results though, it may lead to papers that used this for crawling and did more with the data afterwards.
Chaff from the Wheat : Characterization and Modeling of Deleted Questions on Stack Overflow describes research into the quality of Stack Overflow questions in some sense (specifically, predicting whether questions will get deleted for example). I'm not 100% sure if this is also the kind of stuff you're interested in; it is Stack Exchange + Machine Learning as implied by the title of your question, but not necessarily about retaining information from answers as implied by the text in your question.
Text mining stackoverflow: An insight into challenges and subject-related difficulties faced by computer science learners also describes text mining in StackOverflow questions and answers, though at a very quick glance it appears to be primarily about topic detection etc., not necessarily about automated question answering for example.
Different Facets of Text Based Automated Question Answering System appears to be a relatively recent survey on the topic of Automated Question Answering research. Stack Exchange is mentioned a few times as an example of a source of data for such systems, but doesn't appear to be used otherwise.
Extending PythonQA with Knowledge from StackOverflow is specifically about incorporating Questions and Answers from StackOverflow in an automated Question and Answering system for questions about the Python programming language. The paper provides a link to more details (http://pythonqas2.epl.di.uminho.pt), but that link appears to be down. I suppose you could always try contacting the authors directly if you're interested in more information on this.

More generally, Automated Question Answering systems appears to be a rather active area of research still, not a trivial / "solved" problem. StackExchange can be one source of data for such systems, but there are plenty of other sources of data too (Wikipedia, Quora, etc.).

DuckDuckGo learns answers to technical questions from StackExchange. Type a technical question like "ongoing projects use stackexchange" into DuckDuckGo and it will provide a highlighted summary of the answer on the right-hand side.
And the duck has an open API for many (100s) more question answering data sources. Or you can go directly to the stackexchange api.
Projects can use the data from the SE open API as long as they comply with their TOU. Basically just make sure your users can tell that the data came from Stack Exchange. The copyright license may also limit your ability to alter the contents of the text, with say a learned abstractive summarizer. Perhaps that is why the Duck.com just highlights keywords.
Data rights law is in flux, especially when it comes to the data you submitted to a site and the machine learning models derived from that data. New European data and privacy rules empower you to download or delete all data you submit to a site like stack exchange.


text:
How could we define passion in artificial intelligence?
Mankind can create machines to do work. How could we define passion in artificial intelligence? How could we define a passionate AI?
Would passion cause the machine to do a better job? How could we compare the performance of a passionate machine, as opposed to a non-passionate one? How could we measure the level of passion?

You're asking too many questions in this post.

An elementary approach to 'passion' would be to pre-assign different areas for the program to be 'passionate' about and associate different numeric 'drive strengths' with each (perhaps adaptively). Mechanisms of this sort were studied in Toby Tyrell's widely cited PhD thesis on 'Action Selection in Animals'
More recently, some more sophisticated AI architectures have been developed under the heading of 'Intrinsic Motivation'.
Here  is a link to a paper on the subject by Pierre-Yves Oudeyer, a leading expert in the field of Developmental Robotics.
With regard to the question "would this cause the machine to do a better job?", that would very much depend on how open-ended the architecture is:
It's clearly easier if, rather than having to spell everything out in detail to a machine, we can simply specify a problem at a high-level and let its own motivations cause it to explore promising avenues.
Conversely, if motivations are too open ended, it may well spend all its time doing the equivalent of 'doodling on its paper' (Hofstadter).
Hence, like people, the quality of the output will be a function of its internal dispositions and could be measured in the same way for a given task (e.g. quantitatively for scientific activities, qualatatively for the arts).

Interesting question.
Well if you really think about it, what is passion? How does that passion comes to be a passion.
One of the main topics you might want to touch here is conditioning and thus motivation.
Think about the following:
I have a passion for programming

Why do I have a passion for programming?

Because when I wrote my first program I was positively reinforced by the fact that I completed a program, I was negatively reinforced because I removed my frustration of not completing the program

How come that I have gone through that programming frustration and
  stick to it even if I was frustrated?

Because I wanted to learn programming

Why did I wanted to learn programming?

Because I wanted a light on an arduino to turn on (projected reinforcer)

Why did I wanted to turn the arduino light on?

So I could learn programming and because I though it was cool (classical conditioning association that will later be reinforced, projected reinforcement happened right after the classical conditioning association between turn on a led happened)
This can be done through a neural network, where each association is reinforced through a probability of outcome
For example, I did learn arduino, on purpose because it seemed the easiest way to start coding, so the probability of positive outcome was high
This about an opposite situation
Let's say I do not know calculus, and I barely know elementary algebra, if someone started to teach me about integrals saying that this is the only way to start learning more math, I will not be motivate to do so because since I cannot even conceptualize what an integral can be, it will be really hard for me to understand it thus I will not learn calc
Thus we can also discern that motivation is reinforced in small behaviors
Another more practical and realistic example you might use is
If you trow a rat in a cage, and make him lever-press do you think he is going to? No. Although if you reinforce the behavior of going next to the lever slowly and at the end he will lever press and you then reinforce that behavior he will.
Thus, passion is compartmentalized, and that's what you have to do in your NT and make it mathematically

WINK WINK:

Small hint, it's a progressive function


text:
Are there local search algorithms that make use of memory to give better solutions?
I have been studying local search algorithms such as greedy hill-climbing, stochastic hill-climbing, simulated annealing, etc. I have noticed that most of these methods take up very little memory as compared to systematic search techniques.
Are there local search algorithms that make use of memory to give significantly better answers than those algorithms that use little memory (such as crossing local maxima)?
Also, is there a way to combine local search and systematic search algorithms to get the best of both worlds?

You could parallelize the search by dividing the global space in distinct regions/subsets. Then apply in each region a local search. This way you can search the global space systematically, more exhaustively and perhaps in different ways (e.g by applying a different local search method to each region). Finally you can compare the results and choose the best one.

Tabu search uses memory to rule out parts of the neighborhood for local search, allowing the trajectory to typically pass through local optima instead of getting stuck in them.


text:
Are these two versions of back-propagation equivalent?
Just for fun, I am trying to develop a neural network.
Now, for backpropagation I saw two techniques.
The first one is used here and in many other places too.
What it does is:

It computes the error for each output neuron.
It backpropagates it into the network (calculating an error for each inner neuron).
It updates the weights with the formula: $\Delta w_{l, m, n}=k \cdot E_{l+1, n} \cdot N_{l, m}$, where

$\Delta w_{l, m, n}$ is the change in weight,
$k$ is the learning speed,
$E_{l+1, n}$ is the error of the neuron receiving the input from the synapse, and
$ N_{l, m}$ is the output sent on the synapse.

It repeats for each entry of the dataset, as many times as required.

However, the neural network proposed in this tutorial (also available on GitHub) uses a different technique:

It uses an error function (the other method does have an error function, but it does not use it for training).
It has another function which can compute the final error starting from the weights.
It minimizes that function (through gradient descent).

Now, which method should be used?
I think the first one is the most used one (because I saw different examples using it), but does it work as well?
In particular, I don't know:

Isn't it more subject to local minimums (since it doesn't use quadratic functions)?
Since the variation of each weight is influenced by the output value of its output neuron, don't entries of the dataset which just happen to produce higher values in the neurons (not just the output ones) influence the weights more than other entries?

Now, I do prefer the first technique, because I find it simpler to implement and easier to think about.
Though, if it does have the problems I mentioned (which I hope it doesn't), is there any actual reason to use it over the second method?

The two examples present essentially the same operation:

In both cases, the network is trained with gradient descent using the backpropagated squared error computed at the output.
Both examples use the logistic function for node activation (the derivative of the logistic function $s$ is $s(1 - s)$. This derivative is obviously very easy to compute, and this is part of the reason why it was so widely used (these days the ReLU activation function is more popular, especially with convolutional networks).
The first method also uses momentum.

The main difference I can see is that in the first case backpropagation is iterative while in the second example it is performed in batch mode.
The last video in the series by Welch Labs introduces a quasi-Newtonian method which offers the advantage of finding the minimum of the cost function by computing the Hessian (matrix of second-order derivatives of the error with respect to the weights). However, this feels like comparing apples and oranges - the vanilla gradient descent does not use second-order information.


text:
What is the difference between an agent function and an agent program?
In section 2.4 (p. 46) of the book Artificial Intelligence: A modern approach (3rd edition), Russell and Norvig write

The job of AI is to design an agent program that implements the agent function — the mapping from percepts to actions.

After that, in section 2.4.1, they write

Notice the difference between the agent program, which takes the current percept as input, and the agent function, which takes the entire percept history.

Why does the agent program only take current percept? Isn't the agent program just an implementation of the agent function?
So, what is the difference between an agent function and an agent program (with respect to the percept sequence)?

It looks as if 'function' is being used here in the mathematical (or functional programming) sense of 'pure function', i.e. it is without state or side-effects. Hence the function cannot store previous percepts anywhere, so the entire historical percept sequence is considered to be passed to the function each time.
In contrast, the notion of 'program' appears to allow state/side-effects, so it is assumed that earlier percepts are memoized as needed (or that they otherwise updated the variables used within the program).
The 'function' notion is the conceptually cleaner one, in that the 'program' version can always be abstracted to the functional one. Which aspects of percept history happen to be cached by the 'program' version is merely an implementation detail.

An agent function is an abstract mathematical mapping from percept sequences to actions; an agent program is a specific implementation in a system.


text:
What regulations are already in place regarding artificial general intelligences?
What regulations are already in place regarding artificial general intelligences? What reports or recommendations prepared by official government authorities were already published?
So far, I know of Sir David King's report done for UK government.

[law.se] might be a good place for this question, too.

I don't know that it has yielded any actual reports or regulations yet, but in the USA, the White House has been running a series of interagency workshops / working groups dedicated to Preparing for the Future of Artificial Intelligence.
Some of those sessions have been dedicated to legal / governance issues.


text:
Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?
In the 1950s, there were widely-held beliefs that "Artificial Intelligence" will quickly become both self-conscious and smart-enough to win chess with humans. Various people suggested time frames of e.g. 10 years (see Olazaran's "Official History of the Perceptron Controversy", or let say 2001: Space Odyssey).
When did it become clear that devising programs that master games like chess resulted in software designs that only applied to games like the ones for which they were programmed? Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?

Many publications from the middle of the twentieth century prove the questioner's statement that it was a widely held belief during that period that AI would quickly become conscious, self-aware, and smart.
Great Success
Many tasks and forms of expertise once the exclusive domain of human intelligence, after the development of the Von Neumann general purpose computing architecture became, by the end of that century, more or less the exclusive domain of computers.  These are only a few examples.

Scientific and statistical computation
Drafting and manufacturing process automation (CAD and CAM)
Publishing and typesetting
Certain forms of algebraic and calculus reductions (Maxima and its derivatives)
Circuit analysis
Masterful board game playing
Profitable stock speculation
Pattern recognition (OCR, fingerprint, voice recognition, sorting, terrain)
Programming in predicate logic and recursive predicates
Strategy evaluation

Disappointments (thus far)
In contrast to this impressive array of successes, there is an equally long list of failed expectations.

Consumer available bipedal robots
Automated vacuum cleaning (major disappointment for this answer's author)
Autonomous mechanical factory workers
Automated mathematicians (creative hypothesis generation and proof/disproof to extend theory)
Natural language comprehension
Obedience to arbitrary commands
Human-like expression in conversation
Automated technical innovation
Computer morality
Human (or at least mammalian) emotional states
Asimov's three laws operating system
Adaptive strategy development in arbitrary and shifting set of domains

Domain and Domain-free Distinction
When did it become clear that devising programs that master games like chess resulted in software designs that only applied to games like the ones for which they were programmed?
Although the general public may have thought that a cybernetic chess master would also be smarter than people in other ways, those creating those programs were well aware of the distinction between developing software that exhibited excellence in chess play hard coded and developing software that exhibits the ability to learn chess play and develop excellence iteratively from novice.
The end goal had always been high powered general intelligence.  More short term achievable objectives were created to facilitate the demonstration of progress to investors.  It was the only way to maintain a continuous stream of research funding from the military.
The first milestone was to master a single game without machine learning.  Then research turned to the building of domain knowledge so that a class of solutions, adaptations, and forms of planning could be realized in real time during warfare.  As economic domination became more preferable to military domination during the third quarter of the twentieth century, the vision for AI scaled to embrace the domains of economics and natural resource management.
Consider this spectrum of automation maturity.

A program that enumerates current move sequence possibilities at each turn in the play of a chess game, eliminating probable bad moves at each projected move point, and selects the next move most likely to lead to a win
A program that does the above but also skews probability based on pattern recognition of known winning chess strategies
A program that is designed to be a run time optimized rules engine that centralizes and abstracts the redundant operations of the play of an arbitrary game and isolates and aggregates the representation of chess rules, chess strategies, and chess patterns and anti-patterns
A program that, given a set of rules of a game, can generate a next move based on any game state, remembers success and failure results and the sequences that led to those outcomes, and has the ability to assess the probable loss or gain of individual moves and the game patterns in space and time around them based on history, and then leverages these abilities to learn an arbitrary game, reaching the masterful level of play of chess through learning
A program that learns how to learn games such that, after learning several games, it can learn chess faster than an intellectually gifted human can

The first is easy.  The last is extremely challenging.
When the distinctions between these phases of automation maturity became apparent and how clear people became of those distinctions in which research groups is a complex probabilistic function.
Key Contributors
Who was the first person to recognize the distinction between human-like general intelligence and domain specific intelligence?
Norbert Wiener was likely the first to deeply comprehend the distinction between electronic control of relays (investigated theoretically by Claude Shannon) and closed loop control.  In his book, Cybernetics, a primarily mathematical work, he precisely established the foundation for self-correcting and adaptive systems.  John von Neumann had a comprehension of the distinction between programming good game play and the human ability to learn good game play and published much on the topic.
It was Arthur Lee Samuel who actually wrote the first impressive demonstration of the distinction between game playing software and machine learning.  It was he who bridged Wiener's work with the contemporary digital computer and first coined the term Machine Learning.
Distorted Restatements of Authentic Research and Innovation
The categories artificial narrow intelligence (ANI), artificial general intelligence (AGI), and artificial super intelligence (ASI), proposed in The AI Revolution: The Road to Superintelligence by blogger Tim Urban (Huffington Post, THE BLOG, posted 2/10/2015, updated 4/12/2015), is referenced in AI Stack Exchange in multiple places, but the distinctions between these categories are not precisely defined and the ideas contained therein are neither peer review nor validated by other research or statistics.
The work is no less conjecture than mediocre science fiction — entertaining enough to gain some popularity but not rational conclusions drawn from either repeatable experiments or randomized studies.  The trend graphs provided in the article are of invented shape, not graphical representations of actual data.
Some of the material may later be found to have some truth in it, as in the case for many lay interpretations of scientific research or the futuristic thoughts of science fiction authors.  However, much of the material leads to misconception and false assertions.

I expect a very precise answer to this question may be lost to the sands of time, although I hope somebody can given such an answer. In the meantime, here's one clue on the trail...  This anthology of papers from 2007 starts with the following blurb:

Our goal in creating this edited volume has been to fill an apparent gap
  in the scientific literature, by providing a coherent presentation of a body of
  contemporary research that, in spite of its integral importance, has hitherto
  kept a very low profile within the scientific and intellectual community. This
  body of work has not been given a name before; in this book we christen it
  “Artificial General Intelligence” (AGI). What distinguishes AGI work from
  run-of-the-mill “artificial intelligence” research is that it is explicitly focused
  on engineering general intelligence in the short term.

But even if this is the origin of the specific phrase "Artificial General Intelligence", I am pretty sure people were making the distinction between "general intelligence" and "task specific" techniques much earlier.
The Wikipedia article on AGI also has a clue, where it states:

However, in the early 1970s, it became obvious that researchers had grossly underestimated the difficulty of the project. The agencies that funded AI became skeptical of strong AI and put researchers under increasing pressure to produce useful technology, or "applied AI".

That section cites this this book as support for that statement.  And indeed, it contains the following verbiage:

Although most founders of the AI field continued to pursue basic questions of human and machine intelligence, some of their students and other second-generation researchers began to seek ways to use AI methods and approaches to tackle real-world problems. Their initiatives were important, not only in their own right, but also because they were indicative of a gradual but significant change in the funding environment toward more applied realms of research. The development of expert systems, such as DENDRAL at SAIL, provides but one example of this trend.

Given that DENDRAL began around 1965, it appears that some significant body of researchers (or at least funders) became strongly aware of the distinction between research into "general intelligence" and "applied AI" somewhere around the end of the 1960's. If you keep reading, other passages support the notion that DARPA in particular started pushing a more "applied" approach to AI research throughout the 1970's.
So, not a definite answer, but it looks like we can say that the distinction was known and taken into account at least by 1970, although use of the exact term "artificial general intelligence" appears to be of more recent coinage.

In 1973, the British government hired Sir James Lighthill to commission a "general survey" on the state of artificial intelligence. His report was a condemnation of current AI research, leading a wave of pessimism among AI scientists and the First AI Winter. You may view Lighthill's report (and contemporary criticism of his report) here, but I will summarize Lighthill's key points.
Sir James Lighthill divided AI into three categories:

Advanced Automation - task-specific work
Computer-based CNS research - research into the the "central nervous system" of humans
The Bridge between Advanced Automation and Computer-based CNS research. This bridge would generally be seen as "general-purpose" robotics, so Lighthill would also use the term Building Robots.

Advanced Automation (or "applied AI") is obviously useful. Computer-based CNS research is useful because we want to know more about human intelligence. Both fields of AI had some successes, but its practitioners were overly optimistic, leading to disappointment in those fields. Sir James Lighthill was still very supportive of research in these two fields though.
Building Robots, on the other hand? Sir James Lighthill was very hostile to the very idea, probably because it was more overly hyped up than the other two categories and produced the least amount of valuable output.
He mentioned chess in particular as an example where "robotic" research  has failed. At the time the report was published, the chess-playing engines were at the level of "experienced amateur standard characteristic of county club players in England". However, these chess-playing engines relied on heuristics that were made by human beings. The engines weren't intelligent at all...they merely were following the heuristics that were created by intelligent humans. The only advantage the robots bring to the table is "speed, reliability and biddability", and even that wasn't enough to beat the chess grandmasters.
Now, today, we would probably not treat chess as an example of general-purpose problem solving. We would more accurately classify it as "advanced automation", a "narrow AI" problem divorced from broader real-world implications of general problem-solving. But Sir James Lighthill probably would agree with us. He never used the term "narrow AI" and "AGI" (neither of those terms existed yet) but he would write:

To sum up, this evidence and all the rest studied by the present author on AI work within category B during the past twenty-five years is to some extent encouraging about programs written to perform in highly specialised problem domains, when the programming takes very full account of the results of human experience and human intelligence within the relevant domain, but is wholly discouraging about general-purpose programs seeking to mimic the problem-solving aspects of human CNS activity over a rather wide field. Such a general- purpose program, the coveted long-term goal of AI activity, seems as remote as ever.

Sir James Lighthill believed that the only thing that connects Advanced Automation and Computer-based CNS research is the existence of the Building Roobts "bridge" category. But he's very pessimistic about this category actually producing anything worthwhile. So instead, the AI field should instead breakup into its own its constituent parts (automation and research). Any robots that are built could then be specialized within their subfield...either industrial automation or CNS research. Trying to build the holy grail of "general-purpose program" would be worthless...for the time being, at least.


text:
Why can't we apply value iteration when we do not know the reward and transition functions, and how does Q-learning solve this issue?
I don't understand why we can't apply value iteration when don't know the reward and transition probabilities. In this lecture, the lecturer says it has to do with not being able to take max with samples, but what does this mean?
Why does Q-learning not need to know the reward and transition functions? In Q-learning, we also have a max, so I am not understanding.

For normal value iteration, you need to have the model, i.e. the transition probability, denoted by $P(s' \mid s,a)$.
With Q-learning, you use the current reward and the already stored Q value:

The relation between the value function $V(s)$ and the $Q$ function $Q(s, a)$ is that the $V(s)$ function is simply the value of the action $a$, such that $Q(s, a)$ is the highest, that is, $V(s) = \max_a Q(s, a)$.


text:
In 2016, can $1000.00 buy enough operations per second to be approximately equal to the computational power of a human brain?
In The Age of Spiritual Machines (1999), Ray Kurzweil predicted that in 2009, a \$1000 computing device would be able to perform a trillion operations per second. Additionally, he claimed that in 2019, a \$1000 computing device would be approximately equal to the computational ability of the human brain (due to Moore's Law and exponential growth.)
Did Kurzweil's first prediction come true? Are we on pace for his second prediction to come true? If not, how many years off are we?

Edit (12.19.2020)
According to the answer here, estimates of the processing power of the human brain range at least as widely as from $10^{17}$ flops to $10^{28}$ flops.

Who says that the problem is quantitative ?

The development of CPUs didn't quite keep up with Kurzweil's predictions. But if you also allow for GPUs, his prediction for 2009 is pretty accurate.
I think Moore's law slowed down recently and has now been pretty much abandoned by the industry. How much that will affect the 2019 prediction remains to be seen. Maybe the industry will hit its stride again with non-silicon based chips, maybe not.
And, of course, whether hitting Kurzweil's estimate of the computing power of the human brain will make an appreciable difference for the development of Artificial General Intelligence (AGI) is another question altogether.

Yes, we do have computing systems that do fall in the teraFLOPS range (where 1 teraflop = 1 trillion FLOPS = $10^{12}$ FLOPS)

The human brain is a biological system and saying it has some sort of FLOPS ability is just plain dumb because there is no way to take a human brain and measure its FLOPS. You could say "hey, by looking at the neurons activity using fMRI we can reach some sort of approximation", but comparing the result of this approach with the way FLOPS are measured in computers will be comparing apples with oranges, which again is dumb.

Why don't we measure it in energy consumed instead, with some sort of efficiency factor that denotes how much of the heat is being generated by useful computation (as opposed to supportive biological processes?)

Heat is just another factor to optimise. You want to maximise the FLOPS and minimise heat generation (aka energy consumption of the system). People, in high performance computing, first focus to maximise FLOPS generally as they want their algorithms to run fast and later focus on heat depending on the requirements.

@DJG it's not a currently useful measure, because the "heat being generated by useful computation" (e.g. Landauer limit) is many orders of magnitude smaller than the waste heat of even the most efficient computing devices that we can build. For both modern electronic computers and biological neurons, despite their enormous efficiency differences, they still effectively are 100% waste heat, spending millions of times more power than theoretically necessary for that computation.

Perhaps this indicates a flaw in or incompletion of the theory.

@DJG Given a list of 100 numbers, who would be fast to compute their sum, a computer or a human? This simple example shows that what human brain does and what computers does are completely different things. We built computers to perform a given sequence of arithmetic and logic operations as fast as possible because human brains are very very slow to do this task.

The average person would be slower than a computer. Certain autistic individuals might be faster.


text:
Why are autonomous cars categorized as AI?
How are autonomous cars related to artificial intelligence? I would presume that artificial intelligence is when we are able to copy the human state of mind and perform tasks in the same way. But isn't an autonomous car just rule-based machines that operates due to its environment? They are not self-aware, and they cannot choose a good way to act in a never before experienced situation.
I know that many people often mention autonomous cars when speaking about AI, but I am not really convinced that these are related. Either I have a too strict understanding of what AI is or

There is a neat definition of artificial intelligence, which circumvents the problem of defining "intelligence" and which I would ascribe to McCarthy, the founder of the field, although I can only find it now in this book by H. Simon:
"… having to do with finding ways to do intelligent tasks, to do tasks which, if they were done by human beings, would call for our human intelligence."
So, at its core we call the automation of every task AI, that can only be done by the human mind. At the time people thought that a computer able to play chess would also be intelligent in other ways. When this turned out to be false, the term AI was split into "narrow or weak AI", i.e. a program able to do one task of the human mind, and "general or strong AI", a program that can do all the tasks of the human mind.
Self-driving cars are narrow AI.
Note, that all these definitions don't specify whether these programs copy the way the human mind works or whether they come to the same result via completely different algorithms.

It may be that self-driving cars that can handle a variety of changing conditions (broken traffic lights, police directing traffic, poorly marked detours) on not-very-well-mapped roads requires fairly strong AI.  It hasn't been done yet and it's not clear that it can be done with anything like current technology.

@antlersoft Yes, in envs with more mixed use where understanding language and eye contact is a key to driving, it's more or less strong AI.  A researcher from Uber's autonomous lab told me that driving in most other countries is closer to cycling in the US than to driving in the US.

Other answers tell about sets of instructions for the car in certain situations, or a goal seeking machine, while in fact, self-driving cars don't have a specific set of instructions. Most self-driving cars use deep learning to figure out what to do at certain events. We don't tell them what to do. They learn what to do by example.
The neural networks used to automate cars need massive amounts of data to train. Using the data, the car can figure out what the best action is for certain events.
According to this video Tesla's Autopilot had only one casualty in 300.000.000 miles. For human drivers, the number of casualties in 2014 was 32.675. That is per 300.000.000.000 miles. That means 1 in 90 million human drivers cause a fatal accident, compared to 1 in 300 million for automated cars. Deep Learning surpassed our own 'safety-rate', not by instruction, but by learning what to do itself. If that isn't AI, I don't know what is.

Self driving cars exhibit a level of agency and multi-domain resilience. By certain definitions they are self aware and they are definitely designed to fail safely in a large number of potentially unknown circumstances, which is similar to biological agents.
AI really has to do with the study of non-biological agents and their methods of agency. Everything else is just computer science, algorithmic efficiency, biology, art, etc. Eventually the study of biological and non-biological agency will converge, though, and we'll just call it the study of "intelligence."

Others have given very detailed answers, this is my layman view of the problem statement. The self driving car is a 'goal seeking' machine. It has a set of goals with different priorities. Example. Safety of Occupants, Safety of others, Go from Point A to Point B etc. Some are negotiable, other not so.
To satisfy the goals, the system should use the inputs available (radar, GPS, Camera etc) to determine what is the best possible course of action. At times when it doesn't have all the info (a truck which is hiding a speed sign), it still has to take a decision (historic memory or through awareness of its surroundings) to satisfy its design goals. Hence the AI.

Autonomous vehicles are dependent upon AI technology in that, to be autonomous in their driving or piloting, they cannot be controlled by people.  Therefore they must make complex decisions required of drivers and pilots at least as safely and reliably as human drivers or pilots.

They must recognize objects to the degree that both the value and the typical behavior can be assigned to those objects (i.e. people, pets, property, barriers, curbs, grass, trees, bridges)
They must map trajectories of a wide array of object types based on their object type, what is known about that type of object, detectable variations such as age or condition, and what the object appears to be involved in doing at the time.
They must be able to acquire publicly available representations of drive-able roads (route segments, connection points, and other data), match the representation with the current state of the roads, and track their progress along an intended route to the destination.
They must plan their course in lieu of these real time and difficult to predict actions, traffic law, traffic conventions, traffic signs and signals, given destination, known possible routes, discontinuities, and anomalies.
They must be able to alter the plan to reach the destination if at all possible regardless of changes and challenges encountered.

Driving or piloting a vehicle is an intelligence intensive task.  The only reason AVs will likely surpass human driven vehicles on the road in the near future in terms of the distributions of rates of fatalities and injuries per million meter of travel in the near future is because humans have two key handicaps that offset their intelligence potential as drivers.

Carelessness, as defined as multitasking either mentally or physically at a time when hazards might appear
Selfishness, as defined as risking the life, health, or property of others to gain a transportation related or psychologically related advantage

Although the above two appear to be subjective, they can be easily proven empirically by taking a sample of traffic patterns at any point in time in any highly trafficked road in the world.  This is less true of pilots.
We should not presume that artificial intelligence in AVs is achieved when the behavior of the human mind is copied.  That is the criteria for Alan Turing's Imitation Game, a test that was intended to define intelligence in the context of natural language dialog.  But words don't normally kill people directly.  Vehicles often do.
It would be a very limited vision the potential AV design space to consider human minds as the model of driving excellence.  The tasks should not be performed in the same way by the AI system.  The AI design objectives of AVs should be more consistent with these concerns and interests.

Road or sky safety laws
Ethics regarding right of way in normal and emergency situations
Civil rights concerns in terms of equal access to public resources
Balancing of spacial flow details to maximize transportation throughput
Collision aversion when difficult to predict risks emerge

These requirements on the cognitive and adaptive capabilities of the driving or piloting AI are not solely rule-based and mechanical.  The vehicle itself is mostly mechanical in its operation, but it too presents surprises like blowouts or other difficult to predict failures.  Vehicle control is not at all like chess or a game with a fixed rules of play and fixed game-play environment.
Although the intelligence requirements do NOT include self-awareness of itself as an intelligent system, there are forms of self-awareness required.

The relative position of the exterior surface of the vehicle and its projected path relative to that of other objects
The condition of the operational parts of the vehicle
The mass and location of passengers and any other transported objects in the vehicle

The question ended with an interesting and challenging requirement.

Choose a good way to act in a never before experienced situation

That is perhaps the most challenging aspect of AV driving or piloting system design.
Returning to the question of, "Why are autonomous cars categorized as AI?", the meaning of AI is indeed a critical aspect of answering well.  Taken literally, the term artificial intelligence specifies two things.

It is artificial, in that it does not naturally occur in nature
It is intelligent, in that it adapts in ways that, if those ways are mechanical, they are mechanical at a level of detail that is beyond obviousness without considerable study

As year dependent and culturally dependent as that definition of intelligence is, no other definition is quite as sustainable over decades from both scientific and linguistic perspectives.  By narrower definitions, AVs may not require AI, but there is no compelling scientific reason to narrow the definition of AI to a subset of this previous definition.


text:
What are the advantages of having self-driving cars?
What are the advantages of having self-driving cars?
We will be able to have more cars in the traffic at the same time, but won't it also make more people choose to use the cars, so both the traffic and the public health will actually become worse?
Are we really interested in this?

One of the main arguments for self-driving cars is that presumably they'll get better and better at driving as the technology progresses, they have no temporal attention deficits or aggressive urges or drug habits and sense their environment 360°, all the while communicating with the other cars, which all together basically amounts to LESS DEAD PEOPLE.
We are really interested in this.
It is also unclear whether most people will actually own cars in 30 years. Maybe there'll be a net of mini busses with flexible routes which take you from door to door on demand. That would reduce traffic quite a bit and there would also be less incentive to drive 200 m to get cigarettes or something. Self-driving cars would allow us to use the car as a resource a lot more efficiently, because suddenly we can relocate empty cars without paying a driver.

There are multiple motivations for self driving cars.

Self driving cars have the potential to be much safer.

Self driving cars are far more reliable than humans and can learn and have their software improved and upgraded, resulting in safer roads and far fewer accidents.
More on self-driving car safety: http://bigthink.com/ideafeed/googles-self-driving-car-is-ridiculously-safe

Self driving cars can lead to greater road efficiency.

Traffic jams and obstructions occur due to inefficiencies in human driving, see this MIT simulation of a "phantom traffic jam": https://www.youtube.com/watch?v=Q78Kb4uLAdA and self driving cars can be programmed to avoid this.

Greater economic and environmental benefit

Self driving cars can keep driving costs down by conserving fuel and hence lead to a better environmental impact.
More on fuel efficiency: http://movimentogroup.com/blog/how-self-driving-cars-increase-fuel-efficiency-decrease-waste/

Ease of transport

Self driving cars make transport easier and mean that drivers may be unnecessary in the future, resulting in a more pleasurable and easier drive.

In addition, this would make it easier for people with disabilities to travel as well as simplify the travel experience. Children could potentially be driven to school by a car without the supervision of a parent, for instance.

Parking

Self driving cars can be called to pick you up, meaning the need for parking in nearby locations and/or long walks to find your car may become a thing of the past as your car would drive up to you to pick you up.

Things we haven't even thought of yet :)

Why are self-driving cars awesome?

Safety: better awareness (due to more sensors), better reaction time, fewer distracted/injured/drunk/texting drivers on the road, etc
Convenience: pick up my kids from school, park itself at the grocery store, take itself to be serviced, etc
Faster transit: with increased safety, you can increase speed limits, with proper routing algorithms you don't need traffic lights and stop signs any more (when you have dedicated self-driving lanes & intersections)
Comfort: recline, read, game, or snooze while traveling (yay!)
Cost: subsidize the cost of the vehicle using ads (e.g. projected onto the windshield)
etc

If they are able to network, then they can notify the car behind that it is about to break. In this way they can drive closer together at high speeds. As soon as one puts on the breaks, all the cars behind would apply the breaks. They would not require the 2 seconds that it takes for a human to respond.
Children could be dropped at school or the train station automatically.
People would not need to park a car; it could drop them at work and drive away.
Taxis would probably become more viable than private car ownership.
Car theft might be more difficult.
Where I live, public transport is hardly viable because the government struggles to provide enough parking spaces at train stations and bus stops. The closest empty parking spot by 8:30am is 30minuets walk to the platform. Driverless cars would solve this problem, and Traveling by train would actually become viable for me.

Self driving cars are good for the following reasons:

In the case of an emergancy, urgancy, or just someone being unable to drive unexpactedly, the car can go by itself to a designated location - this is useful in so many use cases - kids who need to get somewhere while parents are busy, Parents who drank a little too much and prefer to take 'the cab' home, or while running, you got injured and need a pick-up.
The examples above are for the more obvious things, which we currently have a struggle with. but other than those, Self-driving cars will open a door for a much wider scale of things: safe police chases (just a car without a police officer), taxies, help in the battle field, and much more...
The third and most important benefit, is the safety and economical properties of self driving cars: with a lot of those cars on the road, they can 'understand' each other and nothing will go unpredicted. they have much faster response time then humans, and maybe in the future they will even be able to predict traffic-light changes, and by that save gas and money (even more than what they can save right now by driving economicly)

Safety is often put in focus by journalists. Although there is potential to make the roads safer, I don't think that is the driving force behind the push for self-driving cars. The main advantage of self-driving cars is that this will reduce costs for businesses, while increasing efficiency (both fuel and time). From the perspective of the public, the self-driving cars are attractive, because they will turn the task of driving, into commute. Activity that requires attention will be replaced with somewhat free time.

I think that one very big advantage would be that if the cars could communicate with each other, they could drive synchronously.
For example, if there was a traffic light, and, let's say, 10 cars are waiting for it to change to green (let's just assume that there would still be something similar to traffic lights). Then when it changes to green all cars could accelerate at the same speed (depending on the acceleration of the front car) at the same time.


text:
What are some datasets to train an MLP on simple tasks?
I have implemented an MLP. Now, I want to train it to solve simple tasks.
Are there any data sets to train the MLP on simple tasks, that is, tasks with a small number of inputs and outputs?
I would like to train it to solve problems which are slightly more complex than the XOR problem.

Questions that ask for datasets are generally off-topic here. The most appropriate site to ask this type of question is probably Open Data SE.

There are a ton of sample datasets our there you can play with. A bunch of good ones install with R in the datasets package.  Luckily you can download them independently if you're not an R user.  Try  https://vincentarelbundock.github.io/Rdatasets/datasets.html
You might also be interested in the MNIST database which is one of the canonical databases used in handwriting recognition research.
Beyond that, you can look at / ask on http://datasets.reddit.com and/or http://opendata.reddit.com and you'll find all sorts of useful datasets.
And finally, don't overlook the UCI Machine Learning Repository.

A popular dataset is the fisher iris dataset. It consists of 150 samples each with a dimensionality of 4. You can find it at
http://archive.ics.uci.edu/ml/datasets/Iris

If you want to solve a multi-class classification problem, you could use the famous iris flower dataset, which was introduced by Fisher in 1936. In this dataset, each flower has (only) $4$ features (the inputs), namely

petal length,
petal width,
sepal length, and
sepal width

There are $3$ classes (the outputs)

iris setosa,
iris virginica, and
iris versicolor

And there are a total of $150$ observations (or records).
The iris flower dataset is available in sklearn. See, for example, Iris plants dataset.
To search for other datasets, you can also use https://toolbox.google.com/datasetsearch.

After almost three years the question is still relevant.
Let me add some too:
Deep Learning Datasets
The datasets from the above link can be used for benchmarking deep learning algorithms.
STL-10 dataset
An image dataset which is inspired by CIFAR-10 dataset


text:
How would DeepMind's new differentiable neural computer scale?
DeepMind just published a paper about a differentiable neural computer, which basically combines a neural network with a memory.
The idea is to teach the neural network to create and recall useful explicit memories for a certain task. This complements the abilities of a neural network well, because NNs only store knowledge implicitly in the weights and the information used to work on a single task is only stored in the activation of the network and degrades quickly the more information you add. (LSTMs are one try to slow down this degradation of short term memories, but it still happens.)
Now, instead of keeping the necessary information in the activation, they presumably keep the addresses of memory slots for specific information in the activation, so these should also be subject to degradation.
Why would this approach scale? Shouldn't a somewhat higher number of task-specific information once again overwhelm the network's capability of keeping the addresses of all the appropriate memory slots in its activation?

Examining the architecture of the DNC indeed shows many similarities to the LSTM. Consider the diagram in the DeepMind article that you linked to:

Compare this to the LSTM architecture (credit to ananth on SlideShare):

There are some close analogs here:

Much like the LSTM, the DNC will perform some conversion from input to fixed-size state vectors (h and c in the LSTM)
Likewise, the DNC will perform some conversion from these fixed-size state vectors to potentially arbitrarily-lengthed output (in the LSTM we repeatedly sample from our model until we are satisfied/the model indicates we are done)
The forget and input gates of the LSTM represent the write operation in the DNC ('forgetting' is essentially just zeroing or partially zeroing memory)
The output gate of the LSTM represents the read operation in the DNC

However, the DNC is definitely more than an LSTM. Most obviously, it utilizes a larger state which is discretized (addressable) into chunks; this allows it to make the forget gate of the LSTM more binary. By this I mean that the state is not necessarily eroded by some fraction at every time step, whereas in the LSTM (with the sigmoid activation function) it necessarily is. This might reduce the problem of catastrophic forgetting that you mentioned and thus scale better.
The DNC is also novel in the links that it uses between memory. However, this might be a more marginal improvement on the LSTM than it seems if we re-imagine the LSTM with complete neural networks for each gate instead of just a single layer with an activation function (call this a super-LSTM); in this case, we can actually learn any relationship between two slots in memory with a sufficiently powerful network. While I don't know the specifics of the links that DeepMind is suggesting, they imply in the article that they are learning everything just by backpropagating gradients like a regular neural network. Therefore whatever relationship they are encoding in their links should theoretically be learnable by a neural network, and so a sufficiently powerful 'super-LSTM' should be able to capture it.
With all that being said, it is often the case in deep learning that two models with the same theoretical capability for expressiveness perform vastly different in practice. For example, consider that a recurrent network can be represented as a huge feed-forward network if we just unroll it. Similarly, the convolutional network isn't better than a vanilla neural network because it has some extra capacity for expressiveness; in fact, it is the constraints imposed on its weights that makes it more effective. Thus comparing the expressiveness of two models isn't necessarily a fair comparison of their performance in practice, nor an accurate projection of how well they will scale.
One question I have about the DNC is what happens when it runs out of memory. When a classical computer runs out of memory and another block of memory is requested, programs start crashing (at best). I'm curious to see how DeepMind plans to address this. I assume it will rely on some intelligent cannibalization of memory currently in use. In some sense computers currently do this when an OS requests that applications free up non-critical memory if memory pressure reaches a certain threshold.


text:
How are generative adversarial networks trained?
I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($G$) and the other discriminative ($D$). The generative neural network generates some data which the discriminative neural network judges for correctness.
How do the discriminative ($D$) neural nets initially know whether the data produced by $G$ is correct or not? Do we have to train the $D$ first then add it into the GAN with $G$?
Let's consider my trained $D$ net, which can classify a picture with 90% accuracy. If we add this $D$ to a GAN, there is a 10% probability it will classify an image wrong. If we train a GAN with this $D$, then will it also have the same 10% error in classifying an image? If yes, then why do GANs show promising results?

I changed the title because your question is more about "how are GANs trained" rather than "how do they work" or "what are they". I suggest that you review the answers below and accept of one those, if they are good enough.

Compare generated and real data
All the results produced by G are always considered "wrong" by definition, even for a very good generator.
You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by the generator or not - you're not comparing "good" and "bad" results, you're comparing real versus generated results.
This will result in a "mutual evolution" as $D$ will learn to find features that separate real results from generated ones, and $G$ will learn how to generate results that are hard to distinguish from real data.

A discriminative network ($D$) learns to discriminate by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two.
Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by producing new data that is more similar to "real data". It is not about the accuracy of network $D$ at all. It is not about improving the accuracy, it is about improving the ability of the computer to generate more "believable" data.
That said, using this scenario could be a good "unsupervised" way to improve the classification power of neural networks, as it forces the generator model to learn better features of real data, and to learn how to distinguish between actual features and noise, using much less data that is needed for a traditional supervised learning scheme.


text:
How can you represent the state and action spaces for a card game in the case of a variable number of cards and actions?
I know how a machine can learn to play Atari games (Breakout): Playing Atari with Reinforcement Learning. With the same technique, it is even possible to play FPS games (Doom): Playing FPS Games with Reinforcement Learning. Further studies even investigated multiagent scenarios (Pong): Multiagent Cooperation and Competition with Deep Reinforcement Learning.
And even another awesome article for the interested user in the context of deep reinforcement learning (easy and a must-read for beginners): Demystifying Deep Reinforcement Learning.
I was thrilled by these results and immediately wanted to try them in some simple "board/card game scenarios", i.e. writing AI for some simple games in order to learn more about "deep learning". Of course, thinking that I can apply the techniques above easily in my scenarios was stupid. All examples above are based on convolutional nets (image recognition) and some other assumptions, which might not be applicable in my scenarios.
I have two main questions.

If you have a card game and the AI shall play a card from its hand, you could think about the cards (amongst other stuff) as the current game state. You can easily define some sort of neural net and feed it with the card data. In a trivial case, the cards are just numbered. I do not know the net type, which would be suitable, but I guess deep reinforcement learning strategies could be applied easily then.
However, I can only imagine this, if there is a constant number of hand cards. In the examples above, the number of pixels is also constant, for example. What if a player can have a different number of cards? What to do, if a player can have an infinite number of cards? Of course, this is just a theoretical question as no game has an infinite number of cards.

In the initial examples, the action space is constant. What can you do, if the action space is not? This more or less follows from my previous problem. If you have 3 cards, you can play cards 1, 2, or 3. If you have 5 cards, you can play cards 1, 2, 3, 4 or 5, etc. It is also common in card games, that it is not allowed to play a card. Could this be tackled with a negative reward?

So, which "tricks" can be used, e.g. always assume a constant number of cards with "filling values", which is only applicable in the non-infinite case (anyways unrealistic and even humans could not play well with that)? Are there articles, which examine such things already?

For many cases you dont have to build everything from scratch instead you can use openAI Gym

I see this as a framework for many algorithms. Which to take? How to solve my problem with the action space? What's behind the observation space and how does it provide an infinite state space with deep learning, i.e. with which algorithm? I would take frameworks, of course, however, my question is also about theoretical background. You should understand a little bit about the technology. For example, in order to specify a meaningful action space and more..

Try this tutorial https://medium.com/@awjuliani/super-simple-reinforcement-learning-tutorial-part-2-ded33892c724

Instead of having the AI learn what action to take, you can alternatively train it to judge how "good" a position is. In order to determine what move to make, you don't ask the AI "This is the current state, what move should I make", you iterate through all possible moves, and feed the the resulting state into the AI asking "How good do you think this new state is?". You then chose the move with the resulting state that the AI liked best. (Or you probably even can combine this with a traditional MinMax approach)
I'm new to this area myself, but I'd guess you would use this approach when the action space is large, and in particular when most possible actions are not a legal option in most states.

Filling values is totally fine. In the case of image recognition the filling will be the background of the image (examples). For example in Belot you have total of 32 cards, which can be 32 boolean features. You can set the ones the player has to 1, while the rest are 0. Note that the in most games you'll need more features than the cards in your hand. I.e number of the round, cards that have been played so far, calls that have been made etc.
Defining the scope of the "action space" will be specific to the game. For Belot, it can be number encoding for each of the 32 cards.

You can find articles via Google. Here is a paper about ML for a card game. Instead of articles, I'd recommend checking out a course on ML (i.e. Coursera and Udacity have good free online courses).

Thanks already. You are right concerning (2). At least, it should be the best way for most cases. However, I think that it is probably not the perfect way for some specific scenarios and I guess that I have one when choosing my first example. In my game (as a human) I would say that playing number "33" is best. If I don't have, then number "44". If I don't have etc. This is because of side effects in the game mechanics. For sure, creating heuristics for the current cards in the hand would help a lot, but isn't this like creating part of the AI already? So this would not be a generic solution?

I had something a bit different in mind, but I messed it up trying to generalize it. I'll edit the answer with a suggestion that's fitting to some card games, but not an option for others.

Yes, the action space must be defined game specific. However, I am wondering about a general strategy to handle my question. For Belot 32 actions can be specified. But is that all? Do I have to care, if a specific action can be taken or not (e.g. the card is currently available). I can think about two scenarios: (a) allow this, but use a negative reward and no state change, (b) do not allow this, retry until something useful can be done. There might be more strategies and I don't know which is the best concerning performance, training quality etc. Therefore, my question!

The rules of the game should be enforced by separate mechanism. You have to be optimizing the choice from the available set of actions, not from all encoded actions. So it's option (b), but  instead of "retry", I'd say "filter" the optimum from the available actions.

Considering your use case, I would not use Deep Learning methods... what is the point?
Instead of just winning, good AI is fun to play with. In practice when fine tuning game mechanics, you will want to analyze the game for churning events. Then it would be nice, if you could show the AI that "Hey, this is messed up, could you come up with a nice way of playing, when this situation happens?" and then the AI would be like "Okay, sure, I didn't know, that me winning all the time was not, what humans considered fun... I'll be more fun next time, while also trying to win".
Lately I have been toying around with Computational Creativity and specifically Partial Order Causal Link planners (POCL) and Agents.
POCLs attempt to create plans, which fulfill goals; this makes them computationally effective as they only need to fulfill a flaw in a goal (having best possible cards on the table) and iterate towards the initial condition (specific cards on table and some cards on hands etc.). I believe, that with Conflict driven POCL you could easily introduce bluffing. I have written POCL algorithm in declarative way, so you don't have to code the action space, but instead have them configured by using modal logic.
Then you would have agents, who would use Plan artifacts generated by the POCL algorithm in order to play in a fun way (evaluation function of the Agent), while also trying to win (search towards best odds for winning). The fun thing with Agents is, that you can compose them and discover personalities easily; I have no idea how Deep Learning methods would provide that easily.
So, by using POCL and Agents, you could first teach the Agents to win efficiently or optimize the Plans to provide good "basic moves" by using some heuristic system (like you will do when using Reinforcement Learning). I don't know about the computational complexity issues regarding specific games; however, such POCL algorithms have been implemented, which are context aware (= reduced action space), so if you add a bit more strategic gameplay abstraction, the POCL should be fine (remember to use some kind of damping factors for reducing the path length of plans, in similar way to PageRank).
In all programming a good mental model will make many things a lot easier. With Deep Learning, you will be using image recognition or similar algorithms / methods to solve a different problem, because nobody is preventing you from using wrong tool for the problem at hand. In real games, there are players (Agents), strategies (POCL plans), bluffing (POCL conflicts) and rules (action space of POCL defined by modal logic). Of course some games might have computational complexity issues; however, usually those are solvable by minor optimization to the algorithms, which provide a good mental model of the problem.

Nice point.  I'm looking to use prior AI victory vs. Human as a switch for trying less-optimal seeming in the subsequent game.  It seems to me that for game AI, my ultimate goal is human engagement.  For a small subset of players, analogous to Chess masters, engagement may be a function of an unbeatable AI, but for the vast majority of the human player base, losing every time is no kind of fun!  In Mbrane, we made a point of designing weak, classical AI's that reason like humans.


text:
What is a time-step in a Markov Decision Process?
The "discounted sum of future rewards" (or return) using discount factor $\gamma$ is
$$\gamma^1 r_1 +\gamma^2 r_2 + \gamma^3 r_2 + \dots \tag{1}\label{1}$$
where $r_i$ is the reward received at the $i$th time-step.
I am confused as to what constitutes a time-step. Say, I take an action now, so I will get a reward in 1 time-step. Then, I will take an action again in timestep 2 to get a second reward in time-step 3.
But the formula \ref{1} suggests something else.
How does one define a time-step? Can we take action as well receive a reward in a single step?
Examples are most helpful.

Hello. This is an old question, but you should accept one of the answers below, if they answer your question. Take a look at https://ai.stackexchange.com/help/someone-answers.

In a Markov Decision Process (MDP) model, we define a set of states ($S$), a set of actions ($A$), the rewards ($R$), and the transition probabilities $P(s' \mid s, a)$. The goal is to figure out the best action to take in each of the states, i.e. the policy $\pi$.
Policy
To calculate the policy we make use of the Bellman equation:
$$V_{i+1}(s)=R(s)+\gamma \max _{a \in A}\left(\sum_{s^{\prime} \in S} P\left(s^{\prime} \mid s, a\right) V_{i}\left(s^{\prime}\right)\right)$$
When starting to calculate the values we can simply start with:
$$V_{1}(s)=R(s)$$
To improve this value, we should take into account the next action, which can be taken by the system and will result in a new reward:
$$V_{2}(s)=R(s)+\gamma \max _{a \in A}\left(\sum_{s^{\prime} \in S} P\left(s^{\prime} \mid s, a\right) V_{1}\left(s^{\prime}\right)\right)$$
Here you take into account the reward of the current state $s$: $R(s)$, and the weighted sum of possible future rewards. We use $P(s' \mid s, a)$ to give the probability of reaching state $s'$ from $s$ with action $a$. $\gamma$ is a value between $0$ and $1$ and is called the discount factor because it reduces the importance of future rewards since these are uncertain. An often-used value is $\gamma = 0.95$.
When using value iteration this process is continued until the value function has converged, which means that the value function does not change significantly when doing new iterations:
$$\left\|V_{i+1}(s)-V_{i}(s)\right\|<\epsilon, \; \forall_{s \in S},$$
where $\epsilon$ is a really small value.
Discounted sum of future rewards
If you look at the Bellman equation and execute it iteratively you'll see:
$$ {\scriptstyle  V(s)=R(s) + \gamma \max _{a \in A}\left(\sum_{a \in A} P\left(s^{\prime} \mid s, a\right)\left[R\left(s^{\prime}\right) +  \gamma \max _{a \in A}\left(\sum_{s^{\prime \prime} \in S} P\left(s^{\prime \prime} \mid s^{\prime}, a\right)\left(R\left(s^{\prime \prime}\right) + \gamma \max _{a \in A}\left(\sum_{s^{\prime \prime \prime} \in S} P\left(s^{\prime \prime \prime} \mid s^{\prime \prime}, a\right) V\left(s^{\prime \prime \prime}\right)\right)\right]\right)\right.\right. }$$
This is like (without transition functions):
$$R(s)+\gamma R\left(s^{\prime}\right)+\gamma^{2} R\left(s^{\prime \prime}\right)+\gamma^{3} R\left(s^{\prime \prime \prime}\right)+\ldots$$
To conclude
So when we start in state s we want to take the action that gives us the best total reward taking into account not only the current, or next state, but all possible next states until we reach the goal. These are the time steps you refer to, i.e. each action taken is done in a time step. And when we learn the policy we try to take into account as many time steps as possible to choose the best action.

You can find quite a large number of examples if you search on the internet, for example, in the slides of the CMU, the UC Berkeley or the UW.

While this is nicely detailed, I think you could have answered the question more directly and succinctly

In the reinforcement learning setting, an agent interacts with an environment in (discrete) time steps, which are incremented after the agent takes an action, receives a reward and the "system" (the environment and the agent) moves to a new state.
More precisely, at time step $t=0$ (the first time step), the environment (including the agent) is in some state $s_t = s_0$, takes an action $a_t = a_0$ and receives and reward $r_t = r_0$ and the environment (including the agent) moves to a next state $s_{t+1} = s_{0 + 1} = s_1$, which will also be the state that the environment will be in at the next time step, $t+1$, hence the notation $s_{t+1}$. Here, the subscripts $_t$ refer to the time step associated with those "entities" (state, action and rewards). So, after one time step (or after $t=0$), the agent will be in state $s_{t+1}$ and the new time step will be $t + 1 = 0 + 1 = 1$. So, we are now at time step $t=1$ (because we have just incremented the time step) and the agent is in state $s_{t} = s_1$. The previously described interaction then repeats: the agent takes an action $a_{t} = a_1$, gets the reward $r_t = r_1$ and the environment moves to the state $s_{t+1} = s_{1+1} = s_{2}$, and so on.
In your summation, we are just discounting the rewards using a value denoted by $\gamma$ (which is usually between $0$ and $1$), that is often called the "discount factor". That summation represents the summation of the rewards the agent will received starting (in this case) from time step $t=1$. We could also just have $r_1 + r_2 + r_3 + \dots $, but, for technical or mathematical reasons, we often "discount" the rewards, that is, we multiply them by $\gamma$ (raised to a power associated with the time step that reward will be received).
In the above description, I said that, at some time step $t$, the agent takes an action $a_t$ and receives a reward $r_t$. However, it is often the case that the reward received after taken an action at time step $t$ is denoted by $r_{t+1}$. I think this is a little confusing, but not conceptually "wrong", because one might think that the reward for having performed an action at time step $t$ is only received at the next time step. (You should get used to slightly different notations and terminology. At the beginning, it is not easy to understand, if the notation is not precise and consistent across sources, but you will get used to it, the more you learn about the topic, in the same way that you get used to a new language).


text:
What kind of road and weather conditions can a self-driving car deal with?
Can self-driving cars deal with snow, heavy rain, or other weather conditions like these? Can they deal with unusual events, such as ducks on the road?

This video shows an AI driving at night: https://youtu.be/-96BEoXJMs0

This video shows an AI driving at night in light rain: https://www.youtube.com/watch?v=GMvgtPN2IBU. It was posted on February 14, 2017, so technically out of scope of this question, but close enough.

No, smart cars do not know what to do when surrounded with ducks or flood waters, and it's possible they never will.
As with all machine learning, a computer knows only what it's taught.  If an event arises that's unusual, the AI will have less relevant training on how to respond, so its reaction behavior necessarily will be inferior to its routine "standard operating procedure", for which is has been heavily trained.  (Of course this is true of humans too.)
Due to liability concerns, when encountering an outlier condition, smart cars will almost certainly be designed by their makers to immediately pull off the road and wait to be explicitly told what to do -- by the human in the car or by communicating with a central command office that exists to disambiguate such confusion and resolve cognitive impasses.  When confused, just like a child, a smart car will be designed to seek external assistance -- and is likely to do so indefinitely, I suspect.
That's why, despite Google's recent cars that lack steering wheels, smart cars most certtainly will retain some means of manual control -- be it a wheel and pedals, or at least verbal commands.  Given the many forms of weirdness that are possible on the road, it's possible smart cars will never be fully autonomous.
As for bad weather conditions, how well do smart cars currently perform?  Nobody outside of a car manufacturer can say for certain.  Lidar and radar are superior to the human eye in seeing through fog and snow.  But (competent) humans are likely to remain better than a smart car at dynamically learning the limit of adhesion and compensating (since this is a learned skill few smart cars will already know or can learn quickly -- given this car, these tires, this road surface, this angle of road, etc).
Initially smart cars will turn to the human when the going gets rough, ceding control back to them.  Once smart cars have driven a few million miles in snow, slush, high wind, floods, and ice, and encountered many ducks, angry mooses, and irate pedestrians, they will have been taught to do more for themselves.  Until then, and perhaps for decades yet, I suspect they will turn to mommy and ask for help.

The state of the art AI driving systems utilize stereoscopic/depth cameras for visual perception. Scenarios such as your ducks on the road example would make the system perceive them as obstacles on the road (it doesn't really matter if they are ducks/goats/humans). The base algorithm should be able to circumvent this situation and bring the vehicle to a safe halt avoiding chances of possible disaster. Hence I doubt scenarios such as this would pose much of a problem to today's AI drivers.

Many cars now instead of just cameras, use radars. Snow, heavy rain, and other weather conditions should not affect them at all. Objects like ducks will be detected. The only problem right now is dealing with things like red lights or road signs, as you have to use a camera to see and interpret them.