text:
How do I use a taxonomy and the support vector machine for question classification?
I am going to develop an open-domain natural language question-answering (NLQA) system, and will use the support vector machine (SVM) as the machine learning (ML) model for question classification.
The data that I have is from a cube, containing multiple dimensions, of which some contain hierarchies.
I do not understand how to work/combine the taxonomy and SVM for question classification. If I understand correctly, the taxonomy still needs to be developed by hand, unless an existing one is being used. And the SVM sorts the queried NL question based on this taxonomy?
Is this correct, or am I mixing the whole concept?

Is the cube the entire construct, or are there extra dimensions? If so, what are they?  *(Also, can you clarify: "Literature study resulted in machine-learning algorithms." This is of particular interest to me, but could have multiple meanings in this context;) *

In terms of taxonomy in general, the computer, by definition, has it's own numeric taxonomy for all data, and the human labels are merely abstractions. There is no limitation for computers developing original taxonomies, which are merely additional sets. However, it may be difficult to procedurally derive the meaning of the sets, thus guided learning would likely save time.

The cube contains dimension, of which some contain hierarchies...
As for the machine-learning algorithm, I noticed that I have been researching wrong. I am building a domain specific  question answering. Therefore, it is not needed (per se) to use machine-learning algorithms...

About the taxonomy, I am kind of lost...
Once the question is queried in NL, I tokenize it, splitting it in tokens. Then applying POS tagging, to tag. Followed by shift-reduce dependency parsing algorithm. Once that is done, to find the entities, I apply NER. This part is the preprocessing...
Now I have to find a way to extract the entities of the query, and map them to a taxonomy? While in reality, I need to find specific words. E.g. measure, then quantity... I am totally lost in this part. What is the best way of a taxonomy, and how can I classify questions? Thank you in advance :)

This is not an answer (I don't have enough reputation to comment). I did something close to this in my master's thesis and think it is close to what you are interested in.
In it, I had developed a framework for extracting metadata from web-based educational content. This metadata was used for classifying the the educaitonal content for many different attributes, which could then be used for faster and more efficient search and discovery of educational content.
The educational resources (containing the content) could be anything like text or PDFs of assignments, homework, assignments, online books, exam questions, courses etc (which many colleges host online). To identify what kind of educational resource it is, I would parse the text and look for keywords and formatting styles (preprocessing included constructing 2-grams and 3-grams, POS tagging, using small specific parsers for NER, dates and other text entities one encounters in educational content).
For some part I used Wordnet (also available under python-nltk) to obtain relationships between different entities and also to find closeness between them. DBpedia was also used. However, for the most part I had to identify the most commonly occuring terms and build a taxonomy by hand. (It took a lot of time!). I obtained a lot of candidates for keywords by looking at openly available taxonomies.
For extracting domain specific taxonomy/ontology, one needs manually to build it. Ontology generation from text is an active area of research and building domain-specific ontology has been tried for many years. One example of such taxonomy (here thesaurus) is agrovoc where domain experts have contributed to the knowledge by identifying agricultural entities manually.
There are a lot of places where domain specific vocabulary is available; maybe you can use that. In some aspects it is close to supervised machine learning, where one has some nice data and correspondingly nice output. However, on my part, there wasn't much learning in it - more like template matching.
Hope this helps.

That is very interesting, thnx! Maybe I am thinking too complex. But I am stuck at the classification part. Am using spaCy as the Python NLP toolkit.
I need to map words, I just don't know how. The noun+verb extracted from parse tree have to be used, together with wh-words i think? to generate a string, equivalent to a MDX query. I need to extract keywords which are the same as the attributes in the data cube. Any tips for that as well?

I have no idea of MDX but from what I gather, you are trying to break the query into components which are (sort-of) basis vectors in information space (the cube). One way is to search the tokens from the query string through the entire hierarchies (use idea of closeness/similarity to find the appropriate attribute). Another could be by converting the query string into a vector (in spaCy) and then taking the inner product with vectors from within the cube; use similarity to find closeness with concepts/axis. Once you get basic concepts, you can create the right query.
Is this useful?


text:
Why would LDA have performed much better than SVM and Naive Bayes in diagnosing ADHD?
In a final project in diagnosing Attention deficit hyperactivity disorder (ADHD) using Machine Learning, we obtained parameters from real patients. We used this data and got much higher success rates with LDA than with SVM and Naive Bayes. We had only 100 examples in our training set. We are wondering why LDA specifically succeeded much more than the others?

By LDA, you mean "linear discriminant analysis", right? Feel free to edit this post to add the tag [tag:linear-discriminant-analysis] if that's the case.

It would be hard to tell if you don't provide what kind of data/problem you are working on, but LDA works well when data that are grouped in gaussian blobs surrounding centroids while vanilla SVM works well when the data is almost linearly separable and naive bayes works well when your features are relatively independent of each other.

If I had to guess(and it is nothing more), I would say it has quite a bit to do with the problem itself and the architectures involved. Simply, the problem is less suited to a bayesian approach(Highly dependent features, linear distribution).

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What is the difference between tree search and graph search?
I have read various answers to this question at different places, but I am still missing something.
What I have understood is that a graph search holds a closed list, with all expanded nodes, so they don't get explored again. However, if you apply breadth-first-search or uniformed-cost search at a search tree, you do the same. You have to keep the expanded nodes in memory.

There is always a lot of confusion about this concept, because the naming is misleading, given that both tree and graph searches produce a tree (from which you can derive a path) while exploring the search space, which is usually represented as a graph.
Differences
Firstly, we have to understand that the underlying problem (or search space) is almost always represented as a graph (although the underlying graph may not contain cycles, so it may represent a tree). So, the difference is not whether the problem is a tree (a special kind of graph), or a general graph!
The distinction is, instead, how we are traversing the search space (represented as a graph) to search for our goal state and whether we are using an additional list (called the closed list) or not.
So, the basic differences are

In the case of a graph search, we use a list, called the closed list (also called explored set), to keep track of the nodes that have already been visited and expanded, so that they are not visited and expanded again.

In the case of a tree search, we do not keep this closed list. Consequently, the same node can be visited multiple (or even infinitely many) times, which means that the produced tree (by the tree search) may contain the same node multiple times.

Advantages and disadvantages
The advantage of graph search obviously is that, if we finish the search of a node, we will never search it again. On the other hand, the tree search can visit the same node multiple times.
The disadvantage of graph search is that it uses more memory (which we may or may not have) than tree search. This matters because graph search actually has exponential memory requirements in the worst case, making it impractical without either a really good search heuristic or an extremely simple problem.
So, there is a trade-off between space and time when using graph search as opposed to tree search (or vice-versa).
Conclusion
So, the difference between tree search and graph search is not that tree search works on trees while graph search works on graphs! Both can work on trees or graphs (but, given that graphs are a generalization of trees, we can simply say that both work on graphs, either trees or not) and both produce a tree!
Notes
The definitions of tree search and graph search given above are based on the definitions given in section 3.3 (page 77) of the book Artificial Intelligence: A Modern Approach (3rd edition), by Stuart J. Russell and Peter Norvig, which is the de-facto standard book in artificial intelligence, so these definitions are applicable in the context of artificial intelligence (which is also the context of this site). Here you have the pseudocode of tree and graph searches (provided by P. Norvig).
However, note that, occasionally, people may use the term tree search to refer to a tree traversal, which is used to refer to a search in a search tree (e.g., a binary search tree or a red-black tree), which is a tree (i.e. a graph without cycles) that maintains a certain order of its elements. This is another reason for having different definitions of a tree search and to think that a tree search works only on trees.

My understanding of tree is that trees are not only a form of directed graph but the nodes are ORDERED, making the tree unique from a graph. Because a tree is ordered, there are no loops. Thus, a node shouldn't ever appear as multiple different nodes of the tree unless the implementer did something very wrong, because keeping track of the list of nodes visited is implicit as part of tree traversal algorithms.

@Dunk What do you mean by "a tree is ordered"? Not all trees necessarily have their elements ordered. There are trees, like red-black trees (or, in general, binary search trees), that maintain a certain order of their stored elements, but not all tress maintain a certain order of their elements. So, a tree has no loops not because it is ordered, but because there are no cycles (of the form A -> B -> A).

@Dunk In a search tree, nodes are ordered, but not in trees in general, except in the sense of depth. Tree search however, is not "search on trees", but a specific family of search algorithms used in AI that are only guaranteed to work on tree structured problems, although they may be applied to other problems (where they may, or may not, fail).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do I check that the combination of these models is good?
I've selected more than 10 discriminative (classification) models, each wrapped with a BaggingClassifier object, optimized with a GridSearchCV, and all of them placed within a VotingClassifier object.
Alone, they all bring around 70% accuracy, on a data set which is about half normal/uniform distributed, and half one-hot distributed. Together, they provide 80% accuracy, which isn't good enough, given that I was told that 95%< is achievable.
The models: DecisionTreeClassifier, ExtraTreesClassifier, KNeighborsClassifier, GradientBoostingClassifier, LogisticRegression, SVC, Perceptron, and a few more classifiers.
How do I check if the combination is good?

Yeah, you can’t. If it works well it’s good. It’s a fairly odd combination tbh

The accuracy value can only be compared with other models. Having a goal is good, but unless you have a strong indication that your goal is achievable given the data in question, then just quoting a raw value like 70% or 99.9% is meaningless. How do human experts do given the same raw data?

Neil - It's for a HW assignment, and they claim 95%< is possible, probably too many features for a human to do 95%< .
Andreas - I just took all the models I've found, with 70%< accuracy.
What would be a good way to choose models ? and how many are probably needed (18 features data, half normal/uniform half one-hots) ?

OK, so if you trust your course materials, and 95% has been done, then you are missing something major. Were you expected to make a large ensemble as part of the task? I think you may be expected to do some feature engineering too - what course materials have you studied, and have you been told which ones are relevant to the task?

Goodness is subjective.  Reliable knowledge isn't possible with that flimsy a quality objective.
The sturdy objective criteria you gave is 95%, so it is bad by that criteria.  (I'm assuming that the 95% is expected for a given data set or a randomized sample from a given data set.)
However, the 80% accuracy is good by the criteria where the you sum the measures of the accuracy of the individual models, divide by the number of models, and find you have gained ten percentage points of accuracy over that average with your aggregated execution strategy.  (I'm assuming here that you used a defined set of network meta-parameters, layers depths and widths, starting parameters, activation model mapping to layers, inter-network connectivity, and loss/error methods for each model that is similar to the aggregated execution strategy.)
I have four questions.  (My apologies that this question leads to the two assumptions above and a bunch more questions.)

Is the 80% accuracy also well over the maximum of the accuracies of the set of accuracies from the individual models?
Is the computing resource draw to achieve 80% accuracy achievable with additional run time of the best of the individual models, using less than or equal to the computing resource draw to achieve 80% that way?
Have you run your evaluation with a full set of meta parameter vectors to check the entire meta-space for best case?
What economic, contractual, or operational hard stop is dictating the 90%?

If we know these answers, we may be able to respond more effectively and possibly find a loop hole in the logic that appears to leave you with an undesirable foregone conclusion.


text:
How do I change the annotations of variable-size images after having resized the images to a fixed size?
In the data-sets like coco-text and total-text, the images are of different sizes (height*width). I'm using these data sets for text detection. I want to create a DNN model for this. So the input data should be of same size. If I resize these images to a fixed size, the annotations given in the data-set, that is the location of the text in the images, will be changed.
So, how do I solve this problem?

Find the largest height and width amongst all the images. Let us call it H and W respectively. It is true that you cannot resize the images, but say if you have an image of height h and width w where h < H, w < W. To the right of the image append W - w number of columns and at the bottom of the image append H - h number of rows having some constant value (0 is okay for grey-scale and B/W images and 0 for each of the channel in case of colour images).
In this way all the images will be of same size. Since you appending at the right and bottom of the image, the annotations will not lose its meaning in the transformed image in terms of the position and content of the text to recognised.
You could also try pixelRNN kind of ideas after you are done with DNN. RNN can handle variable length inputs and in your case it will be sequence of pixels. Here you don't need to append rows and columns to the image.

Me too thought this method. But I'm afraid that this might affect the accuracy of the network. Is that true? Correct me if I'm wrong.

You don't have much of a choice as of now frankly. Since your dataset is quite huge, the effect of padding will not affect the text recognition IMHO.

You can upvote of accept this as answer if it solves your problem. Thanks!

Finally, I found answer for the question. In the annotations, we have X min & max and Y min & max of bounding box. So take the width and height of bounding box and center of bounding box relative to the image.
For example, let the image dimension be 500*500, the bounding box co-ordinates be (200, 200) and (300, 300). So, the center of bounding box is (250, 250) and the height and width is 100. Now, make it relative to image size.
center=(250/500,250/500)=(.5,.5)
height=width=(100/500)=.2

If you rescale the image, with this encodings you can bring back the bounding box with new rescaled image. if you enlarge the image to 1000*1000 then,
center=(.5*1000,.5*1000)=(500,500)
height=width=(.2*1000)=200

Hope this helps someone.


text:
Google's Principles of Artificial Intelligence
Earlier this month, Google released a set of principles governing their AI development initiatives.  The stated principles are:

Objectives for AI Applications:

Be socially beneficial.
Avoid creating or reinforcing unfair bias.
Be built and tested for safety.
Be accountable to people.
Incorporate privacy design principles.
Uphold high standards of scientific excellence.
Be made available for uses that accord with these principles.

AI Applications not to be Pursued:

Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks, and will incorporate appropriate safety constraints.
Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.
Technologies that gather or use information for surveillance violating internationally accepted norms.
Technologies whose purpose contravenes widely accepted principles of international law and human rights.
  SOURCE: Artificial Intelligence at Google: Our Principles

My questions are:

Are this guidelines sufficient?
Are there any "I, Robot" conflicts
How much does this matter if other corporations and state agencies don't hew to similar guidelines?

Artificial Intelligence at Google — Our Principles
Objectives for AI Applications

Be socially beneficial.
Avoid creating or reinforcing unfair bias.
Be built and tested for safety.
Be accountable to people.
Incorporate privacy design principles.
Uphold high standards of scientific excellence.
Be made available for uses that accord with these principles.

AI Applications We Will Not Pursue

Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks, and will incorporate appropriate safety constraints.
Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.
Technologies that gather or use information for surveillance violating internationally accepted norms.
Technologies whose purpose contravenes widely accepted principles of international law and human rights.

Questions
Are these guidelines sufficient?
They are not sufficiently defined to be as sufficient as a policy, not even in the paragraphs following each, but it is presented as a vision comprised of principles, which should not be expected to be as defined as policy.  Here is at least one caveat for each objective.

Half the world has would have been too poor to communicate globally had Linux Torvalds not somewhat antisocially insist on certain things.
Distributing money equally among homeless heroin addicts and entrepreneurs is considered fair by many, yet it is merely equal and perhaps monumentally unfair.
National defense may be considered for safety, but someone who survived Hiroshima or a Syrian refuge may find the definition tragic.
Google cannot be accountable to people without revealing its company confidential ranking, at which point, it would be exploited and they'd have to change it.
Incorporating privacy design principles is far from not storing data so that it cannot be subpoenaed.
The concepts of the singularity, solar panels sustainability, the climate crisis, the ample availability of fossil fuels within U.S borders, and cell communications technology are all considered scientific excellence by the general public.  That these lack the primary signs of rigor in mathematics, statistics, economics, and engineering doesn't occur to anyone, so upon what basis will Google judge?  Will they seek corroboration between theoretical models and previous ones along with empirical validation for every fact?
Being made available, if they are to be a soluble corporation, will depend on the financial condition of the customer, which is in conflict with fairness (#2) and no I, Robot precedence rules are established.

That segues into the next question, skipping the non-pursuit items, which could be similarly treated.

Are there any "I, Robot" conflicts?
The Three Laws of Asimov worked well with the robotic character R. Daneel Olivaw in the Foundation sci fi novel series.  Daneel simulated human emotion, transcended human selfishness, and maintained the three laws.  In the screenplay adaptation of his I, Robot, the three laws didn't work out as well.  VIKI decided that human freedom was in conflict with law two.
Because the eleven dos and don'ts are not codified in precise policies, there are 10! (3,628,800) potential conflicts.  There is at least one that is already clear, mentioned above.

How much does this matter if other corporations and state agencies don't hew to similar guidelines?
In a world where information may have become more powerful than money, perhaps it matters quite a bit when an information giant takes a position.
It's Wikipedia that is most questionable in that through their web service language evolution may have been democratized to a self-defeating extreme.  One can create a word definition without peer review that gains more public authority than the definition in rigorously prepared and cross-correlated encyclopedias and dictionaries.  Even in this artificial intelligence forum, where the education level is high, people define tags with Wikipedia links.  I am tempted to endeavor to remove them all from here. :)


text:
Why is exact inference in a Bayesian network both NP-hard and P-hard?
I should show that exact inference in a Bayesian network (BN) is NP-hard and P-hard by using a 3-SAT problem.
So, I did formulate a 3-SAT problem by defining 3-CNF:
$$(x_1 \lor x_2) \land (\neg x_3 \lor x_2) \land (x_3 \lor x_1)$$
I reduced it to inference in a Bayesian network, and produced all conditional probabilities, and I know which variable assignment would lead for the entire expression to be true.
I am aware of the difference between P and NP. (Please correct me if I am wrong):
Any P problem with an input of the size $n$ can be solved in $\mathcal{O}(n^c)$. For NP, the polynomial-time cannot be determined, hence, nondeterministic polynomial time. The question that scientists try to answer is whether a computer who is able to verify a solution would also be able to find a solution. P= NP?
However, I am still not sure how I can prove that exact inference in Bayesian network is NP-hard and P-hard.

It's not completely clear from your question, but it looks like you want to prove that exact inference in a Bayesian Network is both NP-Hard and P-Hard.
It appears that you have proven that it is NP-Hard, but are unsure how to show that it is also P-Hard.
This is more of a TCS question than an AI question, but shouldn't be too difficult. You just need to pick a P-Complete problem and reduce it to BN.


text:
What does the notation $\nabla_\theta \mathcal{L}$ mean?
Here's the general algorithm of maximum entropy inverse reinforcement learning.

This uses a gradient descent algorithm. The point that I do not understand is there is only a single gradient value $\nabla_\theta \mathcal{L}$, and it is used to update a vector of parameters. To me, it does not make sense because it is updating all elements of a vector with the same value $\nabla_\theta \mathcal{L}$. Can you explain the logic behind updating a vector with a single gradient?

This is standard backpropagation. The gradient term you see is in fact a vector of partial derivatives where each element is the partial derivative of the log-likelihood with respect to each element of the parameter vector $\theta$. Therefore, it has the same dimensionality as $\theta$. Each element of the parameter vector is then updated with the respective term in the vector of partial derivatives, which are generally not the same.


text:
Could an AI be built to learn based of interaction with a human?
A neural network is usually programmed to learn from datasets to solve a specific problem. Essentially, they perform non-linear regression.
Could a neural network be programmed to receive input from a human, like a terminal, to begin to grow and learn (similar to how a child learns)?
A program that neither knows its purpose nor specific data sets but is given enough information to learn based on the input, ponder the input, and ask questions. A child discovers their purpose (in destiny based philosophy) through experience. Thus, could an AI be created that would learn its purpose over time?
Grow both by continued development, maybe adding extensions that add image recognition, speech analysis, etc., and through user interaction. Eventually learning "moral imperatives" or simple the do's and don't's and how to interact with data.
A case scenario would be a Question & Answer session with the neural network and a large data set. Where the human operator knows the answers. At first, the question and the answer are supplied to the neural network. Giving it the ability to find the answer supplied through deep learning. A guaranteed confidence score of (1)  - as the question is pondered the closer it gets to the answer the more it "learns".
The next step is supplying the question and waiting for the answer. The human still knows these answers but is testing the "learning machine" to see if it is truly learning and not "repeating the answer". The answer is supplied by the machine and the human returns with either a percentage that the machine is right (hopefully and eventually matching its confidence score). and after an amount of failure provides the right answer to the machine to repeat the first step and improve learning.
The last step is being able to have the machine answer the question with the human not knowing the solution, thus completing the learning cycle. The human would test the solution and report the results to the machine and the machine would adapt the process and continue learning. However, this time it would begin learning from a data set of results. Hopefully learning "data mining" during its question and answer session.

You might also be looking for active learning, where the machine learning algorithm interactively queries the user to label certain unlabeled training examples. Active learning is similar to semi-supervised learning, in which there are labeled and unlabeled examples.

In principle, yes, what you are proposing can be done.  The exact details of how to do it are an open research question.  The details would also depend on exactly what your goals for the system are. If you're just trying to build some domain specific system that learns a very specific kind of knowledge, then that's probably going to be easier than building an AGI that learns like a child does.
What I will suggest, although this is not proven, is that building a really powerful system of this sort will probably require more than deep learning.  I would also caution anybody interested in AI against thinking that deep learning is the "be all, end all" of AI techniques. My guess is that doing this well will ultimately require a multi-agent system, maybe something like Minsky's "Society of Mind" approach, or a Blackboard model, with collaborating agents, each specialized for various aspects of intelligence.  My feeling is that you will, indeed, need deep learning for classification / pattern matching, but possibly also things like Case Based Reasoning, K-Lines, a Semantic Network, Rule Learning, BDI, and other techniques, working together.

Kind of how our "human brain" works with portions of our brain containing self-contained personalities that assist in the task at hand - the evidence is seen by people suffering from a traumatic event that causes multiple personality disorder - a part of the brain resurfaces to solve or protect the primary/real personality. So an AI of smaller AI's that learns and grow on a specific purpose oriented task.

Example: registering to handwrite, identifying objects, passing data, connecting to the internet, interacting with a human... etc.... I imagine many of them would be very basic layers paired with complicated neural network layers for full computation.

Technological advancement has historically been measured by the processing speed of computing machines. Cogitative behavior psychology is proving to correct human processor disorders such as ADHD, anxiety disorders, addiction, and other psychological disorders preventing humans from normal human interactions and social learning. Cognitive processing speed is very different from the speed per second of programmed computations. Just as humans are learning that a new program for energetic children works better than amphetamines, we will learn a new way to teach AI. AI however is capable of the exponentially growing fast speed of  computations per second. AI will indeed and is indeed guiding humanity to base the direction of machine learning along the human path to enlightenment. From providing a computer opponent for chess to face recognition robotics, human learning is based on our amazingly complex brain power and we will always be necessary for advancement of technology.
Parkaire Consultants, (2012, February 24). Cognitive Processing Speed. Retrieved July 7, 2018, from http://parkaireconsultants.com/cognitive-processing-speed/
Staughton, J. (2018, June 21). The Human Brain vs. Supercomputers... Which One Wins? » Science ABC. Retrieved July 7, 2018, from https://www.scienceabc.com/humans/the-human-brain-vs-supercomputers-which-one-wins.html

Not always. A well equipped AI, using aspect-oriented programming, or maybe just the ability to recode itself, could learn and advance under human intervention until the AI becomes so complex that only itself can understand it's development. The development of AI is only limited to our current understanding of computing and physics.

It might be possible, but, in my opinion, it won't be very successful, given that you need to somehow specify a purpose, even if that purpose is something like trying to be like humans. The most important thing of an intelligent machine is that it follows a goal, that is the very essence of intelligence.


text:
What are the models that have the potential to replace neural networks in the near future?
Are there possible models that have the potential to replace neural networks in the near future?
And do we even need that? What is the worst thing about using neural networks in terms of efficiency?

This is going backwards, but it kind of follows the logic of the arguments.
In terms of efficiency, I can see a few major problems with classical neural networks.
Data collection and preprocessing overhead
Large neural networks require a lot of data to train. The amount can vary depending on the size of the network and the complexity of the task, but as a rule of thumb it is usually proportional to the number of weights. For some supervised learning tasks, there simply isn't enough high-quality labelled data. Collecting large amounts of specialised training data can take months or even years, and labelling can be cumbersome and unreliable. This can be partially mitigated by data augmentation, which means "synthesising" more examples from the ones you already have, but it is not a panacea.
Training time vs. energy tradeoff
The learning rate is usually pretty small, so the training progress is slow. A large model that could take weeks to train on a desktop CPU can be trained in, say, two hours by using a GPU cluster which consumes several kW of power. This is a fundamental tradeoff due to the nature of the training procedure. That said, GPUs are getting increasingly efficient - for example, the new nVidia Volta GPU architecture allows for 15.7 TFLOPs while consuming less than 300 W.
Non-transferrability
Right now, virtually every different problem requires a custom neural network to be designed, trained and deployed. While the solution often works, it is kind of locked into that problem. For example, AlphaGo is brilliant at Go, but it would be hopeless at driving a car or providing music recommendations - it was just not designed for such tasks. This overwhelming redundancy is a major drawback of neural networks in my view, and it is also a major impediment to the progress of neural network research in general. There is a whole research area called transfer learning which deals with finding ways of applying a network trained on one task to a different task. Often this relates to the fact that there might not be enough data to train a network from scratch on the second task, so being able to use a pre-trained model with some extra tuning is very appealing.

The first part of the question is more tricky. Leaving purely statistical models aside, I haven't seen any prominent approaches to machine learning that are radically different from neural networks. However, there are some interesting developments that are worth mentioning because they address some of the above inefficiencies.
Neuromorphic chips
A bit of background first.
Spiking neural networks have enormous potential in terms of computational power. In fact, it has been proven that they are strictly more powerful than classical neural networks with sigmoid activations.
Added to that, spiking neural networks have an intrinsic grasp of time - something that has been a major hurdle for classical networks since their inception. Not only that, but spiking networks are event-driven, which means that neurons operate only if there is an incoming signal. This is in contrast to classical networks, where each neuron is evaluated regardless of its input (again, this is just a consequence of the evaluation procedure usually being implemented as a multiplication of two dense matrices). So spiking networks employ a sparse encoding scheme, which means that only a small fraction of the neurons are active at any given time.
Now, the sparse spike-based encoding and event-driven operation are suitable for hardware-based implementations of spiking networks called neuromorphic chips. For example, IBM's TrueNorth chip can simulate 1 million neurons and 256 million connections while drawing only about 100 mW of power on average. This is orders of magnitude more efficient than the current nVidia GPUs. Neuromorphic chips may be the solution the training time / energy tradeoff I mentioned above.
Also, memristors are a relatively new but very promising development. Basically, a memristor is a fundamental circuit element very similar to a resistor but with variable resistance proportional to the total amount of current that has passed through it over its entire lifetime. Essentially, this means that it maintains a "memory" of the amount of current that has passed through it. One of the exciting potential applications of memristors is modelling synapses in hardware extremely efficiently.
Reinforcement learning and evolution
I think these are worth mentioning because they are promising candidates for addressing the problem of non-transferrability. These are not restricted to neural networks - being reward-driven, RL and evolution are theoretically applicable in a generic setting to any task where it is possible to define a reward or a goal for an agent to attain. This is not necessarily trivial to do, but it is much more generic than the usual error-driven approach, where the learning agent tries to minimise the difference between its output and a ground truth. The main point here is about transfer learning: ideally, applying a trained agent to a different task should be as simple as changing the goal or reward (they are not quite at that level yet, though...).

"Strictly more powerful," is what Maass writes in his 1996 paper, however he claims mathematical rigor and fails to define computing power. Furthermore, in 1996 was written when sigmoid activation functions were popular, which they are not now, precisely because they do not converge for a large variety of scenarios as reliably or as fast as simpler activation functions. Maass only mentions convergence twice in the paper and does not indicate how convergence occurs, further underscoring the absence of the definition of computing power in terms of machine learning objectives.

The connection between RL and evolution is unclear. Are you referring to some combination of a genetic algorithm and RL? If so, what is the reference?

@FauChristian Even if you don't read the whole paper, the deifinition of computational capability is provided in the abstract (second sentence): In particular it is shown that networks of spiking neurons are, with regard to the number of neurons that are needed, computationally more powerful than these other neural network models.

@FauChristian Sigmoid activations are still very much alive and kicking. For example, LSTMs use sigmoid activaitons for the gates, softmax (normalised sigmoids) is still the best thing we have for multi-class classification, etc. "Simpler" activations are not necessarily better - the original ReLU (max(0, x)) is very much in danger of getting stuck for x < 0, resulting in dead neurons. At any rate, the point is about the computational power of spiking nets and their ultra-efficient hardware implementation in terms of power consumption.

@FauChristian I am not drawing parallels between RL and evolution. I am giving them as examples of promising approaches for addressing a certain type of inefficiency, namely having to hand-craft a solution (be it a NN or something else) for every individual problem you have at hand. Ideally, you should be able to design a generic solver which is automatically tuned by RL and/or evolution for the particular problem based solely on a high-level goal.

We do have some hope lurking in that front. As of now we have capsule networks by J.Hinton which uses a different non-linear activation called the 'squash' function.

Hinton calls max-pooling in CNN as a 'big mistake', as CNN look only for presence objects in an image rather than the relative orientation between them. So they lose the spatial information while trying to achieve translation invariance.
Neural nets have fixed connections, whereas a capsule in a capsule network 'decides' to which other capsule it has to pass its activation during every epoch. This is called 'routing'.
The activation of every neuron in neural nets is a scalar. Whereas the activation of capsule is a vector capturing the pose and orientation of an object in an image.
CNN are considered to bad representations of human visual system. By human visual system I mean eyes and the brain/cognition together. We could identify Statue of Liberty from any pose, even if we have looked at it from one pose. CNN on most of the cases cannot detect same object in different poses and orientations.

Capsule networks themselves have some shortcomings. So there has been work in the direction of looking beyond neural nets. You can read this blog for a good understanding before you read the paper by J.Hinton.

Replacing Neural Nets
There may exist new algorithms that have the potential to replace neural nets. However, one of the characteristics of neural nets is that they employ simple elements, each with low demands on computing resources in geometric patterns.
Artificial neurons can be run in parallel (without CPU time sharing or looping) by mapping the computations to DSP devices or other parallel computing hardware.  That the many neurons are essentially alike is thus a strong advantage.
What Would We Be Replacing?
When we consider algorithmic replacements to neural nets, we imply that a neural net design is an algorithm.  It is not.
A neural net is an approach to converging on a real time circuit to perform a nonlinear transformation of input to output based on some formulation of what is optimal.  Such a formulation may be the minimization of a measure of error or disparity from some defined ideal.  It may be a measure of wellness that must be maximized.
The source of the fitness determination for any given network behavior may be internal.  We call that unsupervised learning.  It may be external, which we call supervised when the external fitness information is coupled with input vectors in the form of desired output values, which we call labels.
Fitness may also originate externally as a scalar or vector not coupled with the input data but rather real time, which we call reinforcement.  Such requires re-entrant learning algorithms.  Net behavioral fitness may alternatively be evaluated by other nets within the system, in the case of stacked nets or other configurations such as Laplacian hierarchies.
The selection of algorithms has little to do with comparative intelligence once the mathematical and process designs are selected.  Algorithm design is more directly related to minimizing demands for computing resources and reducing time requirements.  This minimization is hardware and operating system dependent too.
Is a Replacement Indicated?
Sure.  It would be better if networks were more like mammalian neurons.

Sophistication of activation
Heterogeneity of connection patterns
Plasticity of design, to support meta-adaptation
Governed by many dimensions of regional signaling

By regional signaling is meant the many chemical signals beyond signal transmission across synapses.
We can even consider going beyond mammalian neurology.

Combining parametric and hypothesis-based learning
Learning of the form employed when microbes pass DNA

Neural Net Efficiency
Efficiency cannot be quantified in some universal scale as temperature can be quantified in degrees Kelvin.  Efficiency can only be quantified as a quotient of some measured value over some theoretical ideal.  Note that it is an ideal, not a maximum, in the denominator.  In thermodynamic engines, that ideal is the rate of energy input, which can never be fully transferred to the output.
Similarly, neural nets can never learn in zero time.  A neural net cannot achieve zero error over an arbitrarily long time in production either.  Therefore information is in some ways like energy, a concept investigated by Claude Shannon of Bell Labs during the dawn of digital automation, and the relationship between information entropy and thermodynamic entropy is now an important part of theoretical physics.
There can be no bad learning efficiency or good learning efficiency.  There can be neither bad performance nor good performance, if we wish to think in logical and scientific terms — only relative improvement of some system configuration with respect to some other system configuration for a very specific set of performance scenarios.
Therefore, without an unambiguous specification of the two hardware, operating system, and software configurations and a fully defined test suite used for relative evaluation, efficiency is meaningless.

Neural networks require lots of data and training. For most tabular format datasets it is much better to use decision tree based models. Most of the time, simple models are enough to give good accuracy. However neural networks had their test of time. It has only been five to six years since the deep learning revolution started, so we still do not know the true potency of deep learning.


text:
Is Christopher Bishop's "Pattern Recognition and Machine Learning" out of date in 2018?
I recently came across a reference to a book that was highly regarded: "Pattern Recognition and Machine Learning" by Christopher Bishop.
I am a beginner working my way through some machine learning courses on my own.
I'm curious if this book is still relevant considering it was published in 2006. Can anyone who has read it attest to its usefulness in 2018?

This book is still relevant today!
It describes many ML concepts, such as linear regression, neural networks, support vector machines, Gaussian processes, probabilistic graphical models, variational inference, and hidden Markov models, which are still relevant today. If you follow any decent course on ML, it should cover most of these topics. In fact, during one course on ML that I had at university (a few years ago), we used this book as a reference.
Clearly, this book does not contain the description of the latest state-of-the-art models (for example, transformers), but it's a decent book for introducing many concepts in ML.
So, if you want to get a good overall knowledge of ML, then you can surely start with this book (provided that you have a minimal mathematical background to understand the ML concepts).
You may also want to take a look at this post.


text:
What YOLO algorithm can I use for images with noise as I will implement it in real time?
I want to detect drivers with or without seatbelts at crossroads. For that, as it is real-time, I am going to use the YOLO algorithm/model. For training data sets (the images) I need to collect, I placed a camera. By recording it and collecting images from there, I am getting images with more noise.
Can I use these images for training? Also, which YOLO version should I use? What are the important points that I should consider for training datasets?
I want to use any version of YOLO compatible with TensorFlow.

It is much better to know basic mechanics of convnets first ,rather than diving straight into complicated models .
For training data sets (the images) I need to collect, I placed a camera. By recording it and collecting images from there, I am getting images with more noise. Can I use these images for training? Also, which yolo version should I use? What are the important points that I should consider for training datasets?
After you are good with the theory part most of your questions will be answered , otherwise you would endup with nothing but buzzwords.
I want to use any version of yolo compatible with tensorflow.
Tensorflow is a framework for building neural networks , so in theory you can build any network with it so compatibility is not at all a problem.

As long as all training samples have same type and level of noise, it shouldn't affect the outcome/accuracy.
As far as real time performance is concerned, you should compare the "Params, FLops and Inference time @ B1 (Batch size 1)" numbers of different yolo versions.
Lower the number, faster the inference.
You can start by exploring yolov5 and Yolo-Fastestv2 at the links below:
https://github.com/ultralytics/yolov5
https://github.com/dog-qiuqiu/Yolo-FastestV2
Unfortunately most of the implementations are in PyTorch, but you can easily convert a PyTorch model to a Tensorflow one by first converting it to an ONNX format.


text:
Are there human predictions of when a computer would have been better than a human at Go?
I just stumbled across the paper When Will AI Exceed Human Performance?
Evidence from AI Experts, which contains a figure showing the aggregated subjective probability of "high-level machine intelligence" arrival by future years.

Even if this graph reflects the opinion of experts, it can be totally wrong. It is just extremely hard to predict future events. So I was wondering if there is a similar graph which shows basically the same but for the game Go?
Due to the complexity of Go, some experts assumed, that no computer ever could be better in Go than a human being due to the lack of intuition. This shows that the appearance of human level AI can be unpredictable.
Does anyone know if a similar graph for Go exists to see how good or bad the predictions were? This could give a very rough idea, how good this graph predicts the future of human-level AI.

Go predictions were included in the paper:

The experts are far from infallible. They predicted that AI would be better than humans at Go by about 2027. (This was in 2015, remember.) SOURCE: Experts Predict When Artificial Intelligence Will Exceed Human Performance (MIT Tech Review)

Thanks for pointing that out. I should have mentioned it it my question. But I wonder if a similar graph like the one I posted exists for Go where you can see estimations of many experts.


text:
What techniques to explore for dynamic clustering of documents (emails)?
I have a dataset of unlabelled emails that fall into distinct categories (around a dozen). I want to be able to classify them along with new ones to come in the future in a dynamic matter. I know that there are dynamic clustering techniques that allow the clusters to evolve over time ('dynamic-means' being one of them). However, I would also like to be able to start with a predefined set of classes (or clusters/centroids), as I know for a fact what the types of those emails will be.
Furthermore, I need some guidance in terms of what vectorisation technique to use for my type of data. Would creating a term matrix using TF-IDF be sufficient? I assume that the data I am dealing with could be differentiated on the basis of keyword occurrence, but I cannot tell to what degree. Are there more sophisticated vectorisation techniques based more on the text semantics? Are they worth exploring?

that allow the clusters to evolve over time Just to confirm, does that mean that you want to add more categories in the future?

Thanks for you response. Like I said in the comment below - I have a rough idea what the types of emails will be and want to cluster them around this idea. First of all, I believe I should not start the clustering procedure with a random set of centroids, but rather predefined ones. Secondly, since I can only try to define my entry points for the algorithm basing on the current dataset, I think it would be reasonable to let the centroids move around as new datapoints arrive. I think that I'd like to stick with a fixed number of clusters for now.

It sounds like you are trying to do some kind of semi-supervised learning. In semi-supervised learning, some data points are labelled (you know which class they belong to), and others are not. There are classification algorithms designed specifically for this kind of problem, like a transductive-SVM. I personally have not found these techniques to be more effective than simply discarding the unlabelled data and treating my problem as purely supervised, but YMMV.
TFIDF remains fairly popular, as do ngram-based approaches. A more modern vectorization to consider might be word2vec, which translated something like a bag-of-words style vector into a more meaningful feature space for words.

Thanks. Actually, in my case all the data is unlabelled, hence the term 'clustering'. What I want to do is separate them into a predefined number of categories. There is a number of factors that I do now how to handle: 1. how to ensure a good starting position for the clusters b) how to enable refinement of clusters' definitions over time. As for the vectorisation techniques, I will definitely look into word2vec.

@JanParzydło The standard clustering technique is K-Means clustering. (https://en.wikipedia.org/wiki/K-means_clustering). This may be slow if you have high-dimensional data, but can use k-medoids (https://en.wikipedia.org/wiki/K-medoids) with a pre-computed kernel instead if this is a problem.

I would also like to be able to start with a predefined set of classes (or clusters/centroids) as I know for a fact what the types of those emails will be.

This is not a clustering problem, but a semi-supervised learning problem. If you don't have labeled data yet, then create some labels. You might also want to look into "active learning".
One approach is:

For each category, create 5 labeled samples
Train a classifier on them (e.g. tf-idf features and a small neural network)
Let the neural network label your dataset
Check the labels where it was most confident for all classes and the ones where the probabilities for all classes were most evenly spread. Use this to quickly create more labels.
Maybe Amazon mechanical Turk is an option to quickly generate more labels


text:
What are some resources regarding the complexity of training neural networks?
In the paper "Provable bounds for learning some deep representations", an autoencoder like a model is constructed with discrete weights and several results are proven using some random-graph theory, but I never saw any papers similar to this. i.e bounds on neural networks using random graph assumptions.
What are some resources (e.g. books or papers) regarding the time and space complexity of training neural networks?
I'm particularly interested in convolutional neural networks.

There are a few technical papers and books on the topic

Computational Limitations on Learning from Examples (1988) by Leonard Pitt and Leslie G. Valiant, published in Journal of the ACM
Training a 3-node neural network is NP-complete (1992) by Avrim L. Blum and Ronald L. Rivest, published in Neural Networks
Computational complexity of neural networks: a survey (1994) by Pekka Orponen, published in Nordic Journal of Computing
On the relative time complexities of standard and conjugate gradient backpropagation (1994) by J. V. Stone and R. Lister, published in IEEE
Neural Networks - A Systematic Introduction (1996), specifically chapter 10 The Complexity of Learning, a book by Raul Rojas
Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network (2014) by Gursel Serpen and Zhenning Gao, published in Complex Adaptive Systems
On the Computational Efficiency of Training Neural Networks (2014) by Roi Livni, Shai Shalev-Shwartz and Ohad Shamir, published in NIPS 2014
Convolutional Neural Networks at Constrained Time Cost (2014) Kaiming He and  Jian Sun, published in IEEE Conference on Computer Vision and Pattern Recognition (2015)
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time (2018) by Surbhi Goel and Adam Klivans

However, note that gradient descent (and other optimization algorithms) and the back-propagation algorithm are numerical algorithms (that is, they deal with numerical errors), so the time complexity is not the only factor affecting the actual performance of these algorithms and models.


text:
How to implement a contextual reinforcement learning model?
In a reinforcement learning model, states depend on the previous actions chosen. In the case in which some of the states -but not all- are fully independent of the actions -but still obviously determine the optimal actions-, how could we take these state variables into account?
If the problem was a multiarmed bandit problem (where none of the actions influence the states), the solution would be a contextual multiarmed bandit problem. Though, if we need a "contextual reinforcement learning problem", how can we approach it?
I can think of separating a continuous context into steps, and creating a reinforcement learning model for each of these steps. Then, is there any solution where these multiple RL models are used together, where each model is used for prediction and feedback proportionally to the closeness between the actual context and the context assigned to the RL model? Is this even a good approach?

In the case in which some of the states -but not all- are fully independent of the actions -but still obviously determine the optimal actions-, how could we take these state variables into account?

I think the key thing here is the caveat but not all. What you have is a fully-featured MDP (states, actions, rewards, timesteps where next reward and next state depend on current state and action). The fact that next state is only marginally affected by the current action does not prevent it being an MDP.
It would be a problem if the current state did not adequately describe this limitation e.g. if some other data, outside of observable state, decided whether action had any effect. Assuming that is not the case, then you still have a full reinforcement learning problem, but one with some unusual qualities.
You can mitigate against problems caused by algorithms that will over-estimate likely reward (caused by algorithm associating lucky state trajectories with action choice), by using "double learning" - one estimate of return is used to select maximising action, and another used to estimate the actual return from the next state. You probably will also prefer single-step learning over learning based on trajectories as most of the time your state trajectories will not contain learnable data. So Double Deep Q Networks might be a good starting algorithm to try.
If you know absolutely from inspecting a current state, that the next state is independent of the current action, and the next state could in theory be almost anything (from either the whole state space or some large subset) then you may be able to adapt the algorithm to allow for that knowledge. You would do this by altering the TD target and replacing the terms for bootstrap estimate of next state with a rolling mean over all reachable next states. In concept this is similar to Expected SARSA - and in practice if it is possible, it would go a long way to reducing variance during learning process, and may speed up learning significantly. If you know the distribution of next states, you could maybe use that, but basing it purely on samples seen should also be fine, provided you can allocate the means to correct group of states (your question implies you have some understanding of how the states will group). Note that transitions from states with no action effect on next state to states which do have an effect from the action will need careful handling - they should not be assigned to the same "mean group", but should instead affect the TD target normally when they occur.
If it is not possible for the agent to know whether it is in a state where its action affects the next state, except through experience, then you really have to use a standard RL solver.
Finally, if you have a situation where the current action does not affect next state, but the current state definitely does in all cases -  the state "evolves", maybe stochastically, independently of the action taken in most cases, and can only change to a relatively small subset of next states from any given current state - then this is best solved with a normal RL solver - again double Q Learning might be a reasonable starting algorithm in that case.


text:
Has the Fibonacci series or the golden ratio been applied in any way in AI?
I have been looking at the Fibonacci series, the golden ratio, and its uses in nature, like how flowers and animals grow based on the series.
I was wondering whether we could use the Fibonacci series and the golden ratio in any way in AI, especially in evolutionary algorithms. Any ideas or insights?
Is this research material? If so where can we start?

Why cannot we grow an AI model based on physical growth patterns, just like genetic algorithms which are inspired from natural evolution. I just got the idea of using fibonacci series somehow.

I think simply put you are referring to recursion generally, which yes its a big aspect of machine learning.

The use of the Golden Ratio is an interesting suggestion which has intrigued many lovers of the mathematical beauty represented in nature and in AI. The problem lies in the foundations of the AI applications. For example, in designing algorithms for recognizing naturally occurring phenomena, such as face recognition or human body movements (See  https://www.intechopen.com/books/machine-learning-and-biometrics/a-human-body-mathematical-model-biometric-using-golden-ratio-a-new-algorithm) it is suitable. However, for non natural occurrences, the ratio is limited since the data is usually random or chaotic. However, in order to create a master algorithm for the future which encompasses all the best of the current AI algorithms, the use of mathematical concepts such as the golden ration and fractals will be vital. Watch this space...

Face detection evaluation: A new approach based on the golden ratio Φ
Abstract:

Face detection is a fundamental research area in computer vision field. Most of the face-related applications such as face recognition and face tracking assume that the face region is perfectly detected. To adopt a certain face detection algorithm in these applications, evaluation of its performance is needed. Unfortunately, it is difficult to evaluate the performance of face detection algorithms due to the lack of universal criteria in the literature. In this paper, we propose a new evaluation measure for face detection algorithms by exploiting a biological property called Golden Ratio of the perfect human face. The new evaluation measure is more realistic and accurate compared to the existing one. Using the proposed measure, five haar-cascade classifiers provided by Intel©OpenCV have been quantitatively evaluated on three common databases to show their robustness and weakness as these classifiers have never been compared among each other on same databases under a specific evaluation measure. A thoughtful comparison between the best haar-classifier and two other face detection algorithms is presented. Moreover, we introduce a new challenging dataset, where the subjects wear the headscarf. The new dataset is used as a testbed for evaluating the current state of face detection algorithms under the headscarf occlusion

I mean you can predict these sequences quite easily(with varying levels of accuracy) just by using LSTMs in the time series forecasting context. Obviously, as the number of digits you give it increases, it will increase the prediction accuracy of the next in the sequence(with some caveats), as we can think of neural networks more generally as connectionist function approximators(nonlinear in almost all cases).
As far as direct applications in AI, I suppose not beyond mathematical modeling and economic/financial modeling as these sorts of sequences emerge from a vast majority of pure and applied mathematical concepts. This research is quite relevant and ongoing(1,2,3).

It should be a short amount of time before we start seeing exponential complexity growth as the target of AI algorithms, likely involving the golden ratio.
https://en.wikipedia.org/wiki/Golden_ratio#Relationship_to_Fibonacci_sequence
We are already using the golden ratio to perform quantum computations :
https://www.quora.com/How-are-quantum-physics-and-the-golden-ratio-connected
So, once we scale parallelization to GPU-like networks of quantum processors, we can then be sure we have entered the territory where AI and the Golden Ratio are inherently more intrinsic.
As far as accelerating the learning models/algorithms with them, we can only hope something so fantastic would be found in the future of AI as well; who knows, maybe they will come out of course through necessity much like the biological computers we all execute and replicate our code from, and it's environment :
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6047800/
Not to say it will be the only property present, as the article concludes the golden ratio is "most likely" only related by "chance". I'm sure the same random chance by which it relates to black hole entropy, and, everything else it relates to :P.


text:
What is feature embedding in the context of convolutional neural networks?
What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?

The term feature embedding appears to be a synonym for feature extraction, feature learning etc. I.e. a form of embedding/dimension reduction (with the caveat the goal may not be a lower dimensional representation but one of equal dimensionality, but more meaningfully expressed):

Feature embedding is an emerging research area which intends to transform features from the original space into a new space to support effective learning.

Generalized Feature Embedding for Supervised, Unsupervised, and Online Learning Tasks (2019)

Feature embedding aims to learn a low-dimensional vector
  representation for each instance to preserve the information
  in its features.

Large-Scale Heterogeneous Feature Embedding (2019)

Feature embeddings are basically anything that can act as a hidden representation for given object.

In the case of images, a CNN architecture is built to create such hidden representation. Usually, the outcome of the bottleneck layer is flattened (and sometimes, converted to even lower dimensional space by adding one more dense layer) and used as feature embeddings.


text:
Can some one help me understand this paragraph from Nvidia's progressive GAN paper?
In the paper Progressive growing of gans for improved quality, stability, and variation (ICLR, 2018) by Nvidia researchers, the authors write

Furthermore, we observe that mode collapses traditionally
plaguing GANs tend to happen very quickly, over the course of a dozen minibatches. Commonly
they start when the discriminator overshoots, leading to exaggerated gradients, and an unhealthy
competition follows where the signal magnitudes escalate in both networks. We propose a mechanism to stop the generator from participating in such escalation, overcoming the issue (Section 4.2)

What do they mean by "the discriminator overshoots" and "the signal magnitudes escalate in both networks"?
My current intuition is that the discriminator gets too good too soon, which causes the generator to spike and try to play catch up. That would be the unhealthy competition that they are talking about. Model collapse is the side effect where the generator has trouble playing catch up and decides to play it safe by generating slightly varied images to increase its accuracy. Is this way of interpreting the above paragraph correct?

Note that it's not "Model collapse" (as you write) but "mode collapse". This refers to the mode of a distribution (if I remember correctly).

Your intuition doesn't coincide with my interpretation. This is how I see it:
The generator might have some problem early on, for simplicity lets consider the example where the generator always outputs a black pixel in the top left corner. Then the discriminator learns to select on that particular feature, and the generator might take several update steps to resolve the issue, during which the discriminator might shift toward only selecting for the color of the first pixel instead of whether the whole image looks like a face.
Because there is this interplay between the generator and discriminator and because they are constantly lacking behind each other they might start to oscillate. The generator might overcompensate for some random pixel which gave him away to the discriminator and then the discriminator might overcompensate by selecting for a single different pixel which the generator wasn't focused on.
Of course this example is oversimplified by talking about the value of a single pixel, but you get the idea: the two networks are too focused on small problems.
It's not about the discriminator getting too good, but about it getting too focused on one of the mistakes the generator is currently making.

Yes, your intuition is correct. The effect of this problem is that the generator can no longer improve its output to marginally fool the discriminator - the discriminator isn't buying any of the generated output. In this case, the generator gets stuck in a local minimum and typically produces nonsense results.

A discriminator overshooting may result in a dataset that has not been thoroughly clean and probably has too many identical feature, as a result there will be an early convergence from the discriminator as there is little variation.
The drawback from this is that the model will not bee able to generalize well.

In other words, the generator can no longer effective learn how to fool the discriminator because everything fails.

What do they mean by "the discriminator overshoots"?

Mode collapse happens when the generator outputs the same or a set of few images that always look the same regardless of different input noise values. This provides a very easy way for the discriminator to learn that the generated images are fake.
When this happens the discriminator loss is quite low (tending to zero) because it correctly classifies fake vs real images. The same occurs when training the generator instead as since it's trained by back-propagating the error of the discriminator, if the generated images are not able to fool the discriminator by reversing the class labels (due to images collapsed to few modes which are very easy to distinguish), its error is still low and so the generator's gradient tends to vanish, making impossible to let the generator improve and so to escape the local minimum. Indeed, when this happens it's useless to train further.
An ideal training is characterized, instead, by none of the two networks to overcome the other, meaning that for some epochs the discriminator is fooled (say $60\%$ of the times) and so the generator improves, then the discriminator gets better at classifying fakes (but still misses many) still giving the chance for the generator to improve in the meanwhile. If at any point during training one network (especially the discriminator) strongly overcomes the other, then the training ends to a poor solution: a mode collapse for example, that can even happen when the quality of the generated samples is satisfactory - instead, if such process continues and converges, the generator has reached the best possible quality (while also having a wide variety), while the discriminator is not able to distinguish true vs generated images: so its predictions are around $0.5$ probability, that is the Nash equilibrium (i.e. optimal solution) of the min-max game between the generator and discriminator.

What do they mean by "the signal magnitudes escalate in both networks"?

I'm not sure about this point but I guess it refers to a diverging training scenario, in which the gradient of both networks starts to explode because the learned features of the generator (corresponding to predicted pixels) are out of control. In fact, to overcome this issue the authors decide to normalize the generator's feature of each pixel to unit norm. They call this trick pixelwise normalization.


text:
How should the output layer of an LSTM be when the output are word embeddings?
I'm having trouble grasping how to output word embeddings from an LSTM model. I'm seeing many examples using a softmax activation function on the output, but for that I would need to output one hot vectors as long as the vocabulary (which is too long). So, should I use a linear activation function on the output to get the word embeddings directly (and then find the closest word) or is there something I'm missing here?

Actually, LSTM is not used to get word2vec. Indeed, word2vec is extracted from corpus of words using MLP (Multi Layer Perceptron). There is a great tutorial on how to extract word2wec:
http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/
After representing word as vectors, you feed your text to LSTM in a deep architecture which the last layer must be softmax to categorize your text.

In the research papers, it is not clear how they do that. From what I understood, you need to add a dense layer after your RNN layer. This dense layer is the size of your vocabulary. From my experience, this works even for a large vocabulary (30 000 - 40 000 for me) if you have enough data. Here you don't try to reconstruct the embedding but a one-hot vector of the current word. You can then use a cross-entropy loss. This last layer will have a lot of parameters.
You will see several implementations which are using the MSE loss directly on the word embedding output. Personally I didn't succeed with this approach but if other people could share their experiences, it could be great.


text:
How can both agents know the terminal reward in self-play reinforcement learning?
There seems to be a major difference in how the terminal reward is received/handled in self-play RL vs "normal" RL, which confuses me.
I implemented TicTacToe the normal way, where a single agent plays against an environment that manages the state and also replies with a new move. In this scenario, the agent receives a final reward of $+1$, $0$ and $-1$ for a win, draw, and loss, respectively.
Next, I implemented TicTacToe in a self-play mode, where two agents perform moves one after the other, and the environment only manages the state and gives back the reward. In this scenario, an agent can only receive a final reward of $+1$ or $0$, because, after his own move, he will never be in a terminal state in which he lost (only agent 2 could terminate the game in such a way). That means:

In self-play, episodes end in such a way that only one of the players sees the terminal state and terminal reward.

Because of point one, an agent can not learn if he made a bad move that enabled his opponent to win the episode. Simply because he does not receive a negative reward.

This seems very weird to me. What am I doing wrong?  Or if I'm not wrong, how do I handle this problem?

When one agent makes a move, that move should be perceived as part of the "state transition" executed "by the environment" from the perspective of the other agent.
For example, suppose that, as a "neutral third party" we view the game as follows, as a sequence of states, actions and a terminal reward. I use A to denote actions selected by the first player, and B to denote actions selected by the second player:
S1 -> A1 -> S2 -> B1 -> S3 -> A2 -> S4 -> B2 -> S5 -> A3 -> Terminal Reward
Then, the first player should only get the following observations:
S1 -> A1 -> S3 -> A2 -> S5 -> A3 -> Terminal Reward
note how states S2 and S4 are skipped entirely, they are not really states from the perspective of the first player, they're just halfway through the transition caused by the first player's action and are not interesting for the first player.
Similarly, the second player should only get the following observations:
S2 -> B1 -> S4 -> B2 -> Terminal Reward

If you are running self-play in a two player zero sum game, then you can do the following:

Arbitrarily decide the reward scheme for winning, drawing, losing is +1, 0, -1 for Player A.
Have Player A's goal to maximise reward, and Player B's goal to minimise reward.

This means you can combine both players' view of the values of positions and plays into a single metric, which can be learned and/or searched depending on your algorithm. When searching, you can use MCTS and/or minimax algorithms. When using Q-learning, the only tweak to apply is instead of picking the maximising action, player B will want to pick the minimising action (so will use the min and argmin functions where player A would use max and argmax) - remember when calculating TD error that you are evaluating a position for one player, but will be using reward + max/min of next player's move.

Smart approach with Q-learning! Do you know if the following would work? Once the game ends, find the last experiences for player one and player two and replace rewards with -1, 0, 1 depending on the outcome for each player?

@markmark: If your value function is written from the perspective of current player, then yes that is fine. It is probably simpler to think of the "win immediately" as a special case (i.e. usually the other player gets to move as part of state transition, but a winning move prevents that), rather than "backup the value on loss" as special (i.e. generally backup the resulting value from next player's turn for one-step algorithms, except on a win). However, it works logically either way around.

And how would training need any modifications for the player who takes argmin action? (Q-values are calculated using max q value from next state)

@markmark That is in the answer - you will need to take min or max depending on whose turn the value is being calculated for - if it is Player A's turn on timestep $t$ and they get reward $r_t$ and the state changes to $s_{t+1}$ you have to use Player B's view of the Q table when looking at the Q value from $t+1$. That includes Q value update rules. So the update for Player A who wants max Q value is actually $Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha(r_t + \text{min}{a'} Q(s{t+1}, a') - Q(s_t, a_t) )$ - taking the minimum because $t+1$ is on Player B's turn, and Player B wants min Q.


text:
What is experience replay in laymen's terms?
I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of "experience replay". Experience replay comes up in a lot of other reinforcement learning papers (particularly, the AlphaGo paper), so I want to understand how it works. Below are some excerpts.

First, we used a biologically inspired mechanism termed experience replay that randomizes over the data, thereby removing correlations in the observation sequence and smoothing over changes in the data distribution.

The paper then elaborates as follows (I've taken a screenshot, since there are a lot of mathematical symbols that are difficult to reproduce):

What is experience replay and what are its benefits in laymen's terms?

Could you explain which of the symbols/terms you understand from the paper? The part "we store the agent's experiences $e_t = (s_t, a_t, r_t, s_{t+1})$ at each time step $t$" is a straightforward explanation if you know what $s_t, a_t, r_t$ stand for . . . but perhaps the whole thing would be too dense to understand if you were not aware of conventions for these symbols in RL

In reinforcement learning (RL), an agent interacts with an environment in time steps. At each time step $t$, the agent and the environment are in some state $s_t$. From that state $s_t$, the agent chooses and executes an action $a_t$ and the environment emits a reward $r_t$ (which values the just taken action $a_t$). Finally, the agent and the environment move to the next state, $s_{t+1}$. This interaction proceeds until either the agent dies or some other termination criterion is met.
The goal of the agent is to obtain the highest amount of reward in the long run (that is, not just in the next time step, but in all successive time steps). To do that, ideally, the agent needs to find a way of behaving "optimally". In RL, the behaviour of the agent is called a "policy". An optimal policy is a policy that allows the agent to obtain the (expected) highest amount of reward in the long run.
In this context, we can describe a full and finite (in terms of time steps) interaction between an agent and an environment as a sequence (sometimes called a "rollout" or a "trajectory") of states, actions, rewards and next states. So, a rollout might look like this $$(s_t, a_t, r_t, s_{t+1}, a_{t+1}, r_{t+1}, s_{t+2}, \dots, s_{T-1}, a_{T-1}, r_{T-1}, a_{T}),$$ where $T$ is the last time step of the interaction between the agent and the environment. During the interaction between the agent and the environment, the agent might decide to store this "experience" in a "buffer" (e.g. an array), so that it can use it later (you will see below a use case).
The elements of this type of sequences are often temporally correlated. What does this mean? For example, suppose that states are frames of a video game (that is, each frame is a different state). In this context, successive frames (or states) are similar to each other, which mathematically means that they are correlated.
It turns out that neural networks (NNs) are able to approximate (almost) any function. In RL, a policy is also a function: it is a function from a state to an action (or probability distribution over actions). So, we can represent a policy using a NN. (Deep RL is essentially a combination of traditional RL algorithms, like Q-learning, with NNs).
Moreover, it also turns out that training a NN using back-propagation with data that is temporally correlated might lead the NN not to capture the essential characteristics of the data, which, in practice (during training), means that we are not able to find the NN that represents the optimal policy (or another function, e.g. $Q(s, a)$, used later to retrieve the policy). In such cases, we often say that the training of the NN is not stable.
In the case of RL, the data used to train such types of neural networks (which represent the policy, or other functions that are used in RL) are the "rollouts", which contain elements that are often temporally correlated. Hence, we can't just feed the NN with a rollout, in the same order that the elements (states, actions, rewards and next states) are collected. So, in order to use uncorrelated data to train an NN, we can randomly take tuples of the form $\langle s_h, a_h, r_h, s_{h+1} \rangle$ from the rollout (where $h$ is some time step between $t$ and $T$). For example, suppose we take (or "sample") $3$ tuples $\langle s_7, a_7, r_7, s_{8} \rangle$, $\langle s_{97}, a_{97}, r_{97}, s_{98} \rangle$ and $\langle s_{2}, a_{2}, r_{2}, s_{3} \rangle$. Given that these elements have been observed at quite different points in time, they are likely to be less correlated than e.g. $\langle s_7, a_7, r_7, s_{8} \rangle$, $\langle s_{8}, a_{8}, r_{8}, s_{9} \rangle$ and $\langle s_{9}, a_{9}, r_{9}, s_{10} \rangle$ (which are successive "tuples of experience").
In this context, "experience replay" (or "replay buffer", or "experience replay buffer") refers to this technique of feeding a neural network using tuples (of "experience") which are less likely to be correlated (given that "random sampling" procedure). The "buffer" part refers to a data structure (e.g. an array or list) that stores the trajectory (or rollout), that is, it stores the "experience" of the agent (hence the name "experience"). The "replay" refers to the fact that this "experience" is reused (or "replayed") by randomly sampling from it to train the NN.
See also this question Why exactly do neural networks require i.i.d. data? regarding the fact that NNs often require i.i.d. data.

What is it?
An experience replay (ER) buffer is an array/list (or buffer) $D = [e_1, \dots, e_N ]$ where you store the transitions that the agent collects while interacting with the environment. These transitions are usually represented as tuples of the form $e_t = (s_t, a_t, r_t, s_{t+1})$, where

$s_t$ is the state of the agent at time step $t$,
$a_t$ is the action taken by the agent when in state $s_t$,
$r_t$ is the reward received by the environment after having taken action $a_t$ in $s_t$
$s_{t+1}$ is the next state the agent ended up in after that action

The agent then samples (e.g. uniformly) transitions from this ER buffer $D$ to perform an update of the value function $\hat{q}(s, a)$. The ER buffer $D$ can thus be thought of as a dataset.
Why do we need it?
The motivation for experience replay (in the DQN paper that popularized this technique) is that learning becomes more stable. In fact, the authors of the DQN paper write

To alleviate the problems of correlated data and non-stationary distributions, we use an experience replay mechanism [13] which randomly samples previous transitions, and thereby smooths the training distribution over many past behaviors.

or

By using experience replay the behavior distribution is averaged
over many of its previous states, smoothing out learning and avoiding oscillations or divergence in the parameters.

Yes, I know I had already given an answer to this question, but this is a slightly shorter and alternative answer that some people may like more.


text:
How will we recognize a conscious machine?
How will we recognize a conscious machine (or AI)? Is there any consciousness test? For example, if a machine is aware of its previous experiences, can it be considered conscious?

No one knows: there is no empirical test known to physics that can distinguish a conscious entity from one that this not. Heuristically, I know that I am conscious but I only suspect that other people are: they could be 'Dennett Zombies'...

There are two main subjects you need to look at to understand the problem:
The Turing Test

The Turing test, developed by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. wiki

See also: Turing Test (Stanford Philosophical Dictionary)
There is a linguistic element, which is that intelligence can be interpreted in many legitimate ways.  Some maintain that Artificial Intelligence has not yet been achieved; other feel a simple automated switch constitutes the most basic form of AI.
Bear in mind that "intelligence" is distinct from consciousness in the sense of "self awareness", but the generalized Turing Test can also be understood as a gauge of the appearance of consciousness.
This leads to Searle's Chinese Room Argument.  I highly recommend reading the Stanford Philosophy link, but the wiki gives a simpler synopsis:

The Chinese room argument holds that a program cannot give a computer a "mind", "understanding" or "consciousness", regardless of how intelligently or human-like the program may make the computer behave.

The real problem may be, how does one know an algorithm is truly conscious and not merely simulating consciousness?

Philip K. Dick approaches from the opposite direction in Electric Sheep, where one of the conclusions is that "life is life" whether organic or artificial.  This might be said to lead to the "Duck Test" for consciousness: "If it looks like a duck, and quacks like a duck, then it's probably a duck."  (Dick's philosophy was heavily influenced by Christianity, and his view on artificial life may be though of as radically humanist.)
(A parallel philosophical argument might be "how is the appearance of free will distinct from the actuality of free will?"  Free will has been fruitlessly argued about for millennia, but whether the universe is actually deterministic or not, we perceive ourselves to have free will.  If the universe turns out not to be strictly deterministic, it wouldn't functionally change anything.  To make matters more fuzzy, true randomness is nature is only found at the quantum level and then only within certain models. Quantum uncertainty has been proposed as a basis for free will, but we don't know.  Possibly uncertainty is a condition of individuality, which must be subjective in relation to a system or other subjectivities.  Nevertheless, how do we know we are not "robots" acting out a pre-determined sequence of actions, which we perceive as decisions?)
I personally very much like the idea of recursion as function of self awareness.

At the phenomenal level, consciousness can be described as a singular, unified field of recursive self-awareness, consistently coherent in a particular way; that of a subject located both spatially and temporally in an egocentrically-extended domain, such that conscious self-awareness is explicitly characterized by I-ness, now-ness and here-ness. SOURCE: Peters, Frederic Consciousness as Recursive, Spatiotemporal Self-Location

I'd also posit that being able to read and interpret your own code is a form of self-awareness.
This brings me to the idea that understanding (Chinese Room) is a red herring, and what we're really talking about is interpretation.
We are trapped in subjectivity, humans and automata, and perfect certainty is only achievable in a very limited set of circumstances, such as solved games. (There is also the idea of completeness of a system vs. consistency.)
When we talk about understanding in the sense of anything abstract, by which I means semantics as opposed to syntactics, meaning vs. form, one can say that understanding is a function of interpretation, regardless of the accuracy of the interpretation.
--------------------------------------
Fun Speculation:
I've learned never to underestimate the insight of artists, and the best speculative fiction authors are narrative philosophers in the tradition of Plato.  Dick related memory to identity, regardless of whether the memories are real or artificial.  This might be thought of as the narrative conception of the self--I am a product of my experience.  The real "me" is not merely my body, but "the story of me"--the subjective history that led to this moment of me-ness.
I think it is a not unreasonable assumption that artificial consciousness may arise out of understanding of narrative.  Words are just symbols, and there are all kinds of semantic issues, but actions have a concrete aspect.  Game theory studies actions in the form of choices, not only in the sense of equilibria, but also as a form of communication.   (See: iterated dilemma)  The choices agents make in iterated dilemmas constitute a narrative history than can be analyzed and understood mathematically, and these analyses are used for decision making.
It seems to me that the idea of consciousness can be related to decision making.  If you're just a transistor, that consciousness is quite limited, more akin to a cell than a complex organism. It may come down to whether one considers intelligence and consciousness to be spectrums as opposed to thresholds.
If you believe consciousness to be a spectrum, then limited artificial consciousness has already been achieved. Consciousness analogous to human-level self-awareness is still in the future.
See Also:
Definitions of the Self (wiki); Self Knowledge (wiki)

I think general artificial intelligence will only be possible with some form of self awareness included. Many aspects of human communication do not work if one of the communicating partners does not have self awareness. A good example are many of today's chat bots. They seem to not even hear what they say and only rarely seem to have episodic memory.
Advances in machine to human communication and collaboration will eventually create systems with an ever increasing complex inner model that allows the system to interact in a natural way with humans and to fulfill tasks which require human level intelligence and flexibility. However, unless we have developed a very advanced understanding of consciousness it will be hard to judge how similar or different such a machine consciousness is compared to human consciousness.

γνῶθι σεαυτόν!

"Consciousness" does not have a universal definition. However, if you are really into "consciousness", you should probably read about Searle's Chinese Room experiment or Marvin Minsky's society of mind.
In my opinion, there are many more fundamental obstacles in current AI research that we have to tackle first.
Furthermore, a more formal question would be about how an artificial general intelligence (AGI) would emerge. Even for that, there is no clear roadmap, since we are still very new to understanding the true power of neural networks or other successful AI methods.
Franchois Chollet said in a tweet

For all the progress made, it seems like almost all important questions in AI remain unanswered. Many have not even been properly asked yet

As far as I think, the key to this answer lies in the area of unsupervised learning.
Why?
How do we define consciousness? Probably being aware of our existence. The key to understanding our existence starts by asking questions or finding answers which may not have any questions. This all may sound quite philosophical, but in terms of AI, it may just be finding patterns and logic from what we observe around us.
Just a theory but worth giving it a thought.


text:
Is it normal that SOM clusters the instances with the "versicolor" class into multiple different BMUs?
I have trained (with different sizes, learning rates, and epochs) a SOM network to cluster the Iris dataset. The instances associated with the class setosa have been mainly fitted to a 1-2 BMUs. In the case of virginica, the instances have also be associated with only a few BMUs. However, in the case of versicolor instances, many BMUs have been associated with them.
Is this normal?
Setosa
0. 1846
1. 1846
2. 1846
3. 1846
4. 1846
5. 1846
6. 1846
7. 1846
8. 1846
9. 1846
10. 1846
11. 1846
12. 1846
13. 1846
14. 1846
15. 1846
16. 1846
17. 1846
18. 1846
19. 1846
20. 1846
21. 1846
22. 1846
23. 1846
24. 1846
25. 1846
26. 1846
27. 1846
28. 1846
29. 1846
30. 1846
31. 1846
32. 1846
33. 1846
34. 1846
35. 1846
36. 1846
37. 1846
38. 1846
39. 1846
40. 1846
41. 1620
42. 1846
43. 1846
44. 1846
45. 1846
46. 1846
47. 1846
48. 1846
49. 1846

Versicolor
50. 652
51. 652
52. 652
53. 1259
54. 696
55. 1394
56. 652
57. 490
58. 696
59. 490
60. 490
61. 1059
62. 1304
63. 696
64. 490
65. 652
66. 1400
67. 490
68. 696
69. 490
70. 652
71. 1574
72. 696
73. 832
74. 696
75. 696
76. 696
77. 652
78. 696
79. 490
80. 490
81. 490
82. 444
83. 696
84. 1129
85. 1084
86. 652
87. 696
88. 25
89. 584
90. 490
91. 789
92. 1034
93. 490
94. 854
95. 29
96. 584
97. 877
98. 490
99. 809

Virginica
100. 652
101. 696
102. 652
103. 652
104. 652
105. 652
106. 877
107. 652
108. 696
109. 652
110. 652
111. 696
112. 652
113. 696
114. 652
115. 652
116. 652
117. 652
118. 652
119. 696
120. 652
121. 652
122. 652
123. 696
124. 652
125. 652
126. 696
127. 652
128. 652
129. 652
130. 652
131. 652
132. 652
133. 696
134. 696
135. 652
136. 652
137. 652
138. 652
139. 652
140. 652
141. 652
142. 696
143. 652
144. 652
145. 652
146. 696
147. 652
148. 652
149. 652

Now, I have a diagram. It doesn't look bad.