text:
Can abstractive summarization be achieved using neural networks?
Text summarization is a long-standing research problem that was "ignited" by Luhn in 1958. However, a half century later, we still came nowhere close  to solving this problem (abstractive summarization). The reason for this might be because researchers are resorting to statistical (and sometimes linguistic) methods to find & extract the most salient parts of the text.
Is summarization problem solvable using AI (neural networks to be precise)?

The ability to re-frame summarization as a problem for ANN is rather dependent on what kind of output you're looking for: you mentioned 'salient parts of the text'.
One possibly is to use a deep learning approach that first chunks together words that belong in the same phrase as a single 'feature'.
Another possibility is to identify both key words and relations between them. Here is some previous work on using neural nets for relational learning.

that's interesting, thanks for the info! Is it theoretically possible to build a knowledge base from the text (using deductive reasoning) and to train NNs to generate text from the knowledge base?

@JamhurMustafayev I would imagine so, although deductive reasoning would not be my first choice for finding relational primitives in text: NLP isn't my area, but my understanding is that it has a lot of specific techniques for entity relationship extraction.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
Must people tell an AI which algorithm it should use? Can an AI learn algorithms by itself?
I'm a freshman to machine learning. We all know that there are 2 kinds of problems in our life: problems that humans can solve and problems we can't solve. For problems humans can solve, we always try our best to write some algorithm and tell machine to follow it step by step, and finally the machine acts like people.
What I'm curious about are these problems humans can't solve. If humans ourselves can't sum up and get an algorithm (which means that we ourselves don't know how to solve the problem), can a machine solve the problem? That is, can the machine sum up and get an algorithm by itself based on a large amount of problem data?

All intelligence, both human and machine, is mechanistic. Thoughts don't appear out of the blue; they're generated through specific processes.
This means that if a machine generates an algorithm to solve a problem, even if the object-level algorithm wasn't generated by humans, the meta-level algorithm by which it generated the object-level algorithm must have come from somewhere, and that somewhere is probably its original creators. (Even if they didn't program the meta-level algorithm, they probably programmed the meta-meta-level algorithm that programmed the meta-level algorithm, and so on.)
How you think about these distinctions depends on how you think about machine learning, but typically they're fairly small. For example, when we train a neural network to classify images, we aren't telling it what pixels to focus on or how to combine them, which is the object-level algorithm that it eventually generates. But we are telling it how to construct that object-level algorithm from training data, what I'm calling the 'meta-level' algorithm.
One of the open problems is how to build the meta-meta-level; that is, an algorithm that will be able to look at a dataset and determine which models to train, and then which model to finally use. This will, ideally, include enough understanding of those meta-level models to construct new ones as needed, but even if it doesn't will reflect a major step forward in the usability of ML.

Thank you! Well, I think I can't say more. All answers now are point back to the source question - What is an intelligence object? How people define
an intelligence object? Anyway, thank you!

There are problems we for which we don't have a known, optimal, deterministic algorithm. By and large we use heuristics to "solve" those problems.  A closely related idea is that of satisficing where we seek out answers that are "good enough" for immediate purposes.
Likewise, machines can also use heuristics, whether they are programmed in explicitly or, presumably, learned.  Within the range of ways that a machine can use heuristics, there are meta heuristics and hyper heuristics.
Going beyond that, there are other ways that machines an learn "algorithms" or "rules" for solving problems.  One are that I'm particularly interested in is known as rule induction.
This is all an area of open and active research BTW... so if you're interested in exploring any of these approaches, you'll probably find a lot of ground to cover.

Thank you so much! It helps me a lot. Yes, it's a lot of ground to cover. My journey has just begin. :-D

I'd like to offer also a slightly different view on the machine cannot better its master.  Consider the very simple case of content classifiers.  It's already to the point where for some areas classification and prediction can be performed way better than a human.  And while a human may have designed the "algorithm",  the algorithm was likely a recurrent neutral network or other form of ML that could well have self trained. In these cases we don't actually understand or need to understand the individual weights in the net,  as we would need to have traditionally understood the imperative programming constructs we used to write. It just works.
So if we get to where we develop a meta-algorithm for classifying problems and building more optimal deep learning solutions than we would by hand, but I think that would pretty much take us out of the picture for quite a lot of problem spaces.  Thoughts?

New guy here, please go easy on me as this answer will come from personal experience, and will probably be a tad philosophical.
Every algorithm I've designed was built to systematically tackle and solve specific problems in specific situations, each with an end goal in mind. Think of algorithms as solutions to a problem. In my career as a programmer, this rule has always stuck with me (it came from my favorite Computer Sciences professor): "If there is no solution, then there is no algorithm. If there is no algorithm, no machine can solve the problem."
Can machines generate their own algorithms? Most likely. But not to the point that it will exceed us (and by exceed, I don't mean just speed). AIs can never solve problems using methods that humans will never be able to come up with, because we programmed AIs to solve problems just like us humans do.

You're welcome! I got your point. You're not wrong. I think even we do have some strong AI machines, do humans really know how to operate them?

@iheshi Well if these machines really are strong AI, then we humans shouldn't even have to operate them. Also, thanks for the welcome! Good luck on your machine learning adventures!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is meant by "reproduction" in the description of this exercise?
In the following exercise, the word reproduction is mentioned.

Your task is to design a simple genetic algorithm, with binary-coded chromosomes, in order to solve a pattern-finding problem in 16-bit strings.
...
You may use the following operators:

Reproduction: You can use either one of the following reproduction types: Proportional, Ranking, Tournament.

Crossing over: In order to perform this operation the individuals must be grouped in pairs (randomly), and, with certain probability p, cross information from their chromosomes must be exchanged.

Mutation: This operator changes the value of each bit in the chromosome to the opposite one with a very small probability

So, apparently, crossover and reproduction do not refer to the same concept. What does the word reproduction refer to in a genetic algorithm and in this exercise?

The terminology of this exercise is not standard. What is referred to as "reproduction" in the exercise is usually referred to as "selection".
The term "reproduction" does indeed seem conceptually closer to the notion of crossover or recombination (these two are the same thing), which is probably where your confusion has arisen.
See the excellent (and freely-downloadable) 'Essentials of Metaheuristics' for an introduction to the usual terminology for evolutionary algorithms.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What could possibly replace artificial intelligence?
There is no doubt as to the fact that AI would be replacing a lot of existing technologies, but is AI the ultimate technology which humankind can develop or is their something else which has the potential to replace artificial intelligence?

By definition, artificial intelligence includes all forms of computer systems capable of completing tasks that would ordinarily warrant human intelligence.
A superintelligent AI would have intelligence far superior to that of any human and therefore would be capable of creating systems beyond our capabilities.
As a consequence, if a technology superior to AI were to be created, it would almost certainly be created by an artificial intelligence.

For the purposes of mankind, however, superintelligent artificial intelligence is the ultimate technology due to the fact that it will be able to surpass humans in every field, and, if anything, replace the need for human intelligence.

In our past experience, intelligence has been the most valuable trait for any entity to manifest - for this reason, in an anthropomorphic context, we can predict that artificial intelligence will be the ultimate achievement.

The main reason why we will certainly not be able to replace superintelligent AI is that it will surpass us in every respect - if there is ever any replacement, it will be created by the AI similarly to the way we may create an AI that replaces us.

Who's gonna invent the test program to test whether any given "AI" is "superintelligent"? :) Who would write that program? quasi-superintelligent AI? humans? how can humans write a program to check if something is smarter than humans?

That is funny @ErikAllik. Unfortunately for us, we will know immediately once an AI becomes superintelligent.

How will we know that? Is there a measure for intelligence?

Consider this story about AI mentioned by Stephen Hawking @ErikAllik:

Humans create superintelligent artificial intelligence. The first question they ask it is whether there is a God. A bolt of lightning strikes the plug so that the AI cannot be disabled and it responds "There is now."

that is funny, @GJZ ;)

Or in other words: Artificial Intelligence seems to have been obsoleted by Transformer architectures. Wait. You say those also count as AI? Dang.

A new physical lifeform could outperform and replace artificial intelligence when it
has feedback from organism (its body) to its design information (replacement of genes).
This evolution is expected because:
Artificial intelligence will redesign its own software very soon in its evolution.
After that, it will be restricted by the performance of the available hardware
and communication speed between parts.
Therefore it will design better processing hardware for itself, to run its next generation.
To squeeze most processing power out of a given amount of resources (matter, energy) and
circumstances (temperature, radiation) the design has to be small (material resources
and delay of interconnections), energy efficient (heat evacuation), and adapted to the
kind of functions used by the software (hardware architecture).
To tackle this profoundly, artificial intelligence will design the new hardware at
atom by atom scale.
This leads to new problems of natural degradation by radiation, atomic decay and other
quantum-mechanical problems and opportunities.
The solution of these new problems is redundancy and the ability to repair
degraded parts by atomic-level machinery.
This atomic level repair machinery is the same machinery which builds and extends the hardware
for new individual artificial intelligence systems.
Since this feature is there, it can, and will, also be used to restructure parts
of the hardware while it runs to integrate (compile) knowledge in hardware (more efficient).
The machinery to build and maintain such hardware could be inspired by the biological machinery
which will be understood by the system by then.
However, when the artificial intelligence refactors these principles with full understanding
and anticipation, the resulting "hardware" will be quite different from the old biological
machinery and very different from the static silicon based processor structures.
The main differences are:

The design information will be available, relating the features of the realization
with design choices.
This provides direct feedback from performance of the realization (the hardware, the body)
to the design information.
That augments the design information for designing new generations.
This feedback channel is the main difference between the new machinery and biological life.
Once that exists, it will be used for everything, not only for processing hardware for
artificial intelligence.
The new design described here is basically processing hardware rather than a body
for fighting and propagating
(although the eternal fight for resources will not end with the dawn of
artificial intelligence).
It will use more compact molecules because it is designed rather than evolved blindly.
(The current biological life uses monster-molecules evolved by random changes until
one or other corner of the molecule has the right shape to catalyze a specific chemical reaction).
Since parts of the hardware can be restructured while the system runs, the distinction between
hardware and software will become very fuzzy (as in biology).

The drastic increase in efficiency and evolution speed let it outperform the old biological
life (which lacks design and feedback) and outperform artificial intelligence (which was not
integrated in matter).
When this stage is reached, the systems will look like a natural, intelligent,
propagating life-form and therefore supersede the stage of artificial intelligence
running on human-made processing hardware.
This answers the last part of the question:
"... or is their something else which has the potential to replace artificial intelligence".

Artificial intelligence can be so intelligent for the very reason that it is not constrained by its biology.

No matter how intelligent the system, it will be constrained by its hardware (biology or other material structure).
Physical restrictions impose small sizes to reduce the propagation time of signals between parts (speed of light).
The original answer is edited to clarify that the new atomic level machinery is the hardware of the artificial intelligence rather than a body and that knowledge gets integrated in the hardware.
Remark that I answered the second part of the question:
".. is their something else which has the potential to replace artificial intelligence".

That entirely fails to clarify why a lifeform based on genes would be any more effective than extremely efficient self-replicating and self-improving (at a much faster rate) hardware.

A superintelligent AI would develop something similar to the concept of computronium, as outlined in The Technological Singularity by Murray Shanahan. This material would be able to concentrate the largest physically possible amount of calculation capability in a given area.

This would be the theoretically most efficient (and therefore completely irreplaceable) configuration of particles possible for the purpose of intelligence. No lifeform of any kind can compete with that.

In order for something to replace AI, it would need to out perform AI. AI currently uses number systems to represent information and logic to perform operations. So the replacement would need to be based off of something more efficient than numbers and logic. Some sort of super-logic. Or something similar to intuition and instinct which do not require linearly figuring things out.


text:
How could human behavior and relationships be implemented?
One of the most compelling issues regarding AI would be in behavior and relationships.
What are some of the methods to address this?  For example, friendship, or laughing at joke?  The concept of humor?

I know they're working on this to some degree in Japan, although I think one of the great insights is that even simulated behavior by an automaton can produce chemical response in humans.  I don't know what's going on re human behavior, but they do have robot animals designed for human interaction: http://www.parorobots.com/

Theory of mind
Relationships and normal social behavior require a human to possess a reasonable "theory of mind", a skill in understanding and modeling the thought processes that happen in the minds of others, and making reasonably accurate predictions on how particular actions will be understood by others.
In general, this might be treated as any other machine learning/prediction task - while this skill is quite complex and too hard for our current systems, there doesn't seem to be any obvious qualitative barrier that needs to be breached. Notably, there's no reason to believe that a mind needs to be able to experience a certain "feeling" in order to model that other minds can have it - an AI agent could form a causal model of how friendship works in human relationships and use that to exhibit behavior that's consistent with friendship in all aspects and/or facilitate friendship behavior towards it by particular humans, if it fits the agent's goals. Whether you'd consider that "real friendship" is pretty much just a matter of how you define the word, with some parallels to the "p-zombie" discussion.

In general, what you are describing implies a hierarchical sequence model, in which mannerisms adapt to the regime or paradigm in effect.  Expressive modalities are how we recognize the operative context from the behaviors of other agents.  For an artificial agent, avoiding un-canniness would involve clustering the factors underlying the classification of discourse contexts, tagging them with corresponding factor models for manner in a way which closely aligned to the classifications which human agents make.  This involves developing an adequate latent representation for both spaces, which is likely to involve quite a lot of training data, or some clever transfer learning in conjunction with one-shot techniques (generally, taking samples as modes).
When the context becomes one of friendship, for example, the style factors of expression should extend to factors of trust, collaboration, disclosure, sympathy.  In order to implement these, layered abstractions in the representation of behaviors will need to be crafted, presumably by training.  In order to align these learned categories to human categories, which I suppose to be essential to emotional fluency, exploiting the structure of corpus semantics - including distributional characteristics of the linguistic labels and inferences - when estimating loss gradients seems a natural strategy, which exploits cultural learning.
These comments are necessarily speculative, as no such systems are current, to my knowledge.  At least, not in any significant degree of maturity or application.  Certainly other approaches are conceivable. I am just extrapolating plausible strategies, in the context of current technologies, strategies for implementing the kinds of behaviors you describe in a useful way.
More, specifically, I am considering the question as a manifold-learning problem, and hence requiring a representation model in which the inputs and outputs vary over state-spaces, with the behavior being learned as a mapping between those spaces.  Each of those components (the input space representation and its natural topology, the output space representation likewise, the mapping between them, and the sequence model hierarchy which extrapolates from the manifold to a control process emsemble) has its own peculiar challenges, and the challenges to creating an adequate implementation are a confound of those.  Still, it seems more a question of time and money than of notional feasibility.

Methodology for a social enabled AI.
Regarding social interactions, I believe that trying to build a copy of human behaviour based on technologies we understand, might not be effective .
Instead, I would start from the roots of how human grows and learn, or better, by what is each human trying to solve from childhood to the end of their life.
In that way we will be able to build artificial beings, able to exploit the best of what technology might provide.
It seems to be based on emotions.
As an example, humour is the kind of social trick a gifted social artificial life could learn and master, first, like a child, then like all of us, in order to empathize with others when this common emotions arise, before to analyse the conditions in witch it arises so that to be able to reproduce it, then eventually if the need become a priority, by theorizing its mechanics, possibly mastering it and using it like a tool.
(I believe that humour is often generated by instigating surprise to others, it requires modelling how others generally think, leading them to be surprised.)
Mastering social interactions require modeling
Humour, friendship as examples of social interaction should be naturally discovered by AI like by Human, and eventually used in order to fit once goal.
On the other hand this require intelligence to be sensible to surprise in a positive way, leading either to laugh, interest, curiosity or fear depending specific conditions. Mastering that process requires to model both the world (how you think) and how others do model it (how they think).
Modeling is greatly accelerated by interactions and communication
Additionaly, Human may trigger others laugh without building a conscious theory,
this help us understand intelligence and build an artificial one:
Language is here used to transmit initially abstract concepts built by intelligence. It helps us accelerate our modeling of the world, that would remain very basic if only based on our own experience.
Most important interdependent skills for Modeling are observation then logic; interactions comes third
Unconscious resolution of problems seems tofirst,  be a characteristic of live-beeings, but also of machine-learning techniques like deep learning that is criticised for that. Because we would like it to be able to explain us the concepts they discovered and used to solve the problems we gave it.
This is because we would like that tools to build abstract and possibly original models, before to give it tha ability

to convert observations into concepts,
to verify concepts by observation or deduction.
to communicate.

A causal and statistical approach backed by hierarchical goals (possibly dynamic) including the hunger for discovery
For this to happen in a complex world made both of physics and though,
a causal statistical approach seems required, backed by, to give the direction to take:
a hierarchical list of goal two of wich

the eager to discover and
the eager for accomplishment itself judged by others against hierarchical list of caracterisitics,

should manage to find their best possible place behind that to survive,
allowed or helped by the conscious to be a finished beeing.
Adaptation strategies
For discovery goal to reach its best level, biological related mechanics involving reproduction, variability, complexity and adaptation to environment, could be involved, but this seems to go beyond the question.
Still some work remains to be done in order to give our reasoning tool the ability to comunicate, that is to master a language, and to use it given the knowledge of what concepts and semantic domains, interlocutors do already master, to then build new concept in a transmissible form like words, images, video or just body-movements, face expression or so.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the difference between a Hopfield Network and a Neural Network or a Perceptron?
Neural Net can be feed-forward or recurrent. Perceptron is only feed forward.
So, what is Hopfield Network then?

Hopfield stores some predefined patterns (lower energy states) and when an non seen pattern is fed to the Hopfield net, it tries to find the closest match among the stored patterns. Hence the output of a Hopfield is always one of the predefined patterns which matches closely to the unseen input pattern.
This is not the case with Feed Forward Neural Nets (where no such predefined patterns are stored) and every input generates a corresponding output. Here (in Feedforward) the output is generated by a predefined function (which is self-adjusted during training session) where as in Hopfield predefined patterns are stored and outputted (no such functions exist).

It is not "always" one of those predefined patterns as the network may converge to one of the spurious patterns which the network was not trained on.

Hopfield Networks are recurrent. However, they are not as general as more modern Recurrent Neural Networks such as Long Short-Term Memory Networks as they cannot process sequential input. I've never worked with a Hopfield Network but I've been told that they are mostly of historical interest today due to their limitations.


text:
What kind of algorithm is the Levenberg–Marquardt algorithm?
Is a Levenberg–Marquardt algorithm a type of back-propagation algorithm or is it a different category of algorithm?
Wikipedia says that it is a curve fitting algorithm. How is a curve fitting algorithm relevant to a neural net?

In the context of Neural Networks, Backpropagation (with Gradient Descent, to use its full name) and Levengerg Marquardt are both members of the broader family of gradient descent algorithms. Backpropagation itself is not gradient descent, but it does the gradient climbing portion of a broader gradient descent algorithm.
You can imagine the function of a Neural Network as a function from its inputs to its outputs. If you were trying to solve, for example, a regression problem you can imagine this problem in multidimensional space with each training point corresponding to the coordinate provided by the values of its inputs and outputs (each of which represent a dimension). Then the entire training set you have for learning this regression problem become a set of points in this multidimensional space, and the function that your neural network performs is a curve in this multidimensional space. The closer this curve is to the points on the training set, the better it performs at the regression problem which essentially means we can generalize the task of training the neural network to a curve fitting problem.


text:
Why do activation functions need to be differentiable in the context of neural networks?
Why should an activation function of a neural network be differentiable? Is it strictly necessary or is it just advantageous?

It is almost mandatory to have a differentiable activation function unless, of course, you have an alternative to training the network by back-propagating the error.

No, it is not necessary that an activation function is differentiable. In fact, one of the most popular activation functions, the rectifier, is non-differentiable at zero!
This can create problems with learning, as numerical gradients calculated near a non-differentiable point can be incorrect. The "kinks" section in these lecture notes discuss the issue.
I don't know the full answer as to whether this is a problem in practice and how people get around them. There are ways to avoid getting these incorrect gradients for some non-differentiable functions, as the lecture notes I discussed above mention.

Training
While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.
This is commonly done by backpropagation and variations of gradient descent, which requires the existence of a gradient - i.e., requires activation function to be differentiable. The adjustment of each parameter is calculated from the gradient of the activation function(s) that this parameter affects, so if you cannot get a gradient, then this approach can't be used.

Lets go through your question point by point.

Should an activation function be differentiable?

No, it is not of any compulsion of it to be differentiable. We use Re-Lu's which have a non-differentiable point at 0. But this is simply a trivial case, since the point 0 will never be reached unless we run out of precision points to denote extemely less numbers.
So lets take another example, the Perceptron learning algorithm. In this algorithm, there is no satisfactory way to evaluate the performance of a particular solution. So we don't have a cost function. But still we are able to reach to a solution, albeit maybe not the best. I'll come to the point why it is not used later.
NN's can be broadly thought of as just function approximators. Normally, you give it some continuous function, the NN adjusts it by elongating, shifting, distorting parts of that function by changing only and only the parameters of the function and not the nature of the function itself i.e. it'll decompose into the same sort of a Fourier series as before, with only phase and amplitude differences. You can also design a NN on lines of a random search puzzle. You give each node of a, say single hidden layer NN a part of the function to be approximated i.e. between some intervals -a < x < b say. Say the connection weights to each output layer is fixed. You can only change the part of the function (i.e.parts broken like a jigsaw from a function to be approximated) you want to give to a node, from your fixed box of parts. So you see, in this case the NN will try almost all combinations of giving different parts to different nodes until the function is perfectly approximated. So you see, this can also be a type of learning for NN without any continuous function.

Why should an activation function of a neural network be differentiable?

The main advantages of differentiable function is the mathematics behind it. You can easily handle huge amounts of data just by simple mathematics. 1000's of years of mathematical theories can be applied to verify the working of your NN, predict how will it work, select best algorithms. In discrete non-differentiable activation functions you have no realistic way of predicting what will be the final result.

Say in a problem like this, if you apply the perceptron learning algo., do you have any realistic way to predict where the final decision boundary will rest? Whereas, if you train it by continuous methods, you can easily plot the cost function versus weights, check the global minima, get the weights which will be the final weights. So we easily solved it by plotting (definite matrix methods also exist).

Is it advantageous to have a differentiable activation function? Why?

So why is it advantegeous?
--> Final answers easily predictable (if relatively small data set).
--> Easy to draw graphs and visualize the working of your NN and adjust your hyper-parameters accordingly.
--> Easy to apply time tested mathematical tools to test/evaluate the effectiveness of your algorithm.
--> No sudden changes in error and so your weights will also not change suddenly due to stray readings.
These are all the advantages I could think of and I am sue there are more. All these tools are not available for discrete algorithms, if you think (can't give the full intuition - too lengthy).
I gave some more intuition here:
What is the purpose of an activation function in Neural Networks?
Feel free to add anything I missed by editing the answer.
Hope this helps!

As already said , Activation function is almost differentiable in every neural net to facillitate Training as well as to calculate tendency towards a certain result when some parameter is changed. But I just wanted to point out that The Output function need not be differentiable in all cases. We can have non-sigmoid (hard-limiter threshold)output nodes but still train them with backpropagation and gradient descent.


text:
How do you explain Hebbian Learning in an intuitive way?
In these lecture slides, it's written

The neuropsychologist Donald Hebb postulated in 1949 how biological  neurons learn:
"When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place on one or both cells such that A's efficiency as one of the cells firing B, is increased."
In more familiar terminology, that can be stated as the Hebbian Learning rule:

If two neurons on either side of a synapse (connection) are activated simultaneously (i.e. synchronously), then the strength of that synapse is selectively increased.

Mathematically, we can describe Hebbian learning as:
$$w_{i j}[n+1]=w_{i j}[n]+\eta x_{i}[n] x_{j}[n]$$
Here, $\eta$ is a learning rate coefficient, and $x$ are the outputs of the ith and jth elements.

So, my main question is: what do all these descriptions mean? Here are a few sub-questions.

Is Hebbian learning applicable for single-neuron networks?
What does "two neurons on either side of a synapse" mean?
Why/when would two neurons activate "simultaneously"?
What are these "elements"?

Fire together, wire together.

Hebbian learning is trying to answer "How the strength of the synapse between 2 neurons evolve over period of time based on the activity of the 2 neurons involved.". You can call it learning if you think learning is just strengthening of synapses.
The connection between 2 neurons are called synapse. A synapse is the point where the axons of a neuron meets with the dendrites of another neuron.
The 2 neurons are connected to each other but they are also connected to other neurons as well. So it may happen that both the neurons gets activated by their other connected neurons at the same time.
The elements refer the 2 neurons. The equation is basically saying that the synapse strength between i and j neuron at time (n+1) depends on its strength at time n plus activations of the i and j neurons at time n.


text:
Understanding the perceptron algorithm in the book "A Course in Machine Learning"
The following text is from Hal Daumé III's "A Course in Machine Learning" online text book (Page-41).

I understand that $D$ is the size of the input vector $x$.

What is $y$? Why is it introduced in the algorithm? How/where/when is the initial value of $y$ given?
What is the rationale of testing $ya \leq 0$ for updating weights?

Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors.
Rationale behind Y.a<=0 :
Prerequisite knowledge :

A=A-B : Moves vector A away from direction of vector B
A=A+B : Moves A in the direction of B
A (.) B >0 ; A vector is directed acutely (<90 deg.) towards B vector
A (.) B <0 ; A vector is directed away from (>90 deg.) from B vector [(.) denotes dot (scalar) product and bold letters indicate vectors]
W is augmented vector (includes threshold as another weight along with normal input weights)
X is augmented input (includes -1 as extra input (corresponding to threshold) along with other normal inputs
a(activation) = W (.) X
a >=0 ; Perceptron output 1
a<0 ; Perceptron output -1 (Not zero as implicit in the given algorithm I think)

Now Rationale :
(Y.a<0)
This means
Either of the following :

Y=-1 and a>0 ; in this case the target output is -1 but as a>0 so the Perceptron outputs 1. So we must move the weight vector away from this set of input vector, so that angle between them increases and the dot product (a) becomes < 0 so that we can get the target output.
Hence : W=W-X

Or ,W=W + (-1)*X
Or, W=W+YX

Y=1 and a<0

This means the target output is 1 but as the activation is <0 so Perceptron is outputting -1. So we must move the weight vector close to this set of input vector so that the activation can become >0 (angle decreases) and the Perceptron can output the desired output.
So :
W = W+X
Or W=W + YX
Again , Y.a=0 is a boundary case.
By now I think you can understand the rationale behind Y.a=0. If any doubt , comment to this answer , I will explain it.
Sorry for so much long answer though. :) :)

Is that $W(.)A = = W^T.A $?

What are A and T in the above comment?

Sorry for that, W(.)X == W^T.X. I.e. dot product of transpose of W with X.

No they won't be same until you perform transpose on both the vectors. [ Although I assume that by transpose you mean reversing the components of the vectors. I.e , xi+yj+zk -> zi+yj+xk ] as transpose is inherent concept with matrix not vectors.


text:
What are the main differences between a perceptron and a naive Bayes classifier?
What are the main differences between a perceptron and a naive Bayes classifier?

Naive Bayes is a generative algorithm while Perceptron is a discriminative algorithm. That is the main difference.

A perceptron is a linear threshold function. That means it has a weight vector $w$, and it outputs $w \cdot x > t$, where $x$ is the input vector and $t$ the threshold.
Naïve Bayes makes the assumption that all features are independent (hence the term naïve). It predicts the most likely class by using Bayesian probability, for each class multiplying the class prior with the the probability of the input given the class. The fact that we are modeling $P(X|Y)$ instead of $P(Y|X)$ makes it a generative model. Since we make the strong independence assumption, $P(Y|X)$ is modeled independently for each $x_i \in X$, usually via some sort of maximum likelihood estimation.


text:
Is it possible to design a neural network from the plot of data points?
Suppose I have been given the following diagram

I need to design a neural network to separate the points or classes. How can I do that?

From the diagram you have given, it's quite clear that you have to design a supervised network. Also, it's clear that you are dealing with a problem that's not linearly separable i.e. you can't separate two classes using a single line. Particularly, it will require three lines to separate these two classes.
From the above observations, what you can do is:

Make training data sets of form [(x,y), o] where (x,y) are the co-ordinates on the plot and o is the class that point belongs to.
Note: You can actually divide your data set into training data and test data. Just, extract the data, randomly shuffle it and take first 80% of data to train and rest 20% of data to test.

Design a 2-2-2-1 neural network with bias units and some random initial weight values. This is because we will require minimum three lines to separate two classes, so 2 hidden layers (each consisting of 2 nodes and bias nodes) and 1 output layer (with bias node). The first layer is input layer with 2 nodes (no bias) as we have x-coordinate and y-coordinate.

Train your training data using this network using the Error Back Propagation algorithm i.e. update the weights between the layers and the bias weights too. Run the EBP algorithm for 100-2000 epochs.

Then, use the test data to see, if you get the desired output.

Note: I am NOT quite sure how to obtain the coordinates from the diagram. You can either manually note all points or research for some methods to get those points automatically from the image (some image processing algorithms).
Edit: I found a good video which explains neural networks. In the video, there is non-linear data like the one in your image which is separable using multiple lines and the network is designed to try to find curves such that data can be separated and the output resembles to that when used straight lines.

can u plz supply me any study material regarding this?

You can see some video lectures of Andrew Ng's lectures (Week 4 and 5). Also, I learnt from this book and this book. Hope this helps! :-)


text:
Does it exist a human-like artificial intelligence?
Does it exist a human-like artificial intelligence?
I define human-like as something that can act like a human in most aspects.

Do chat-bots count, OP? Are you looking for more akin to an android?

This depends on your definition of human-like.
If you mean a robot that looks and acts like a human, arguably, yes. Here's one of many examples: http://www.hansonrobotics.com/robot/sophia/
If you are looking for something that performs work and tasks, or works and thinks and talks like-or better than a human, the answer is mostly no, not yet.
I recommend you look at 'ANI, AGI, ASI"
ANI: artificial narrow intelligence. This is what you see around you right now.
AGI: artificial general intelligence. A theoretic AI that can "think" like a human. It does not yet exist. Estimates are between 20-60 years before we will successfully create AGI.
ASI: artificial super intelligence. In a nutshell, it is theorized to be everything we wish we could be or hope never to be. It does not exist yet. It is generally believed that, IF we create an AGI, ASI will evolve seconds or less than a decade after AGI is created.

Today I learned Sophia's skin is held in place by a zipper. And I really wish I hadn't.

I would say that we're not even close to a "real" human-like AI. For all the wonderful things that applications like Siri, Cortana and the like can do, they're actually really dumb compared to even a child.  Of course part of that, IMO, is that most AI applications are not embodied and don't experience the world the way humans do.  So if you show an AI a video of a dog walking behind a table and briefly disappearing from the frame, it has pretty much no ability to apply "common sense" and know that the dog will reappear in a few seconds, and that if it does't reappear, it's probably because it found its favorite toy on the floor behind the table.
For some examples of the kinds of "easy" questions that computers still don't do well at, check out the Winograd Schema Challenge.  You might also find this page interesting:  http://www-formal.stanford.edu/leora/commonsense/

It might be of note to comment/update that the SuperGLUE benchmark, which is a suite of common sense reasoning tasks, incorporates the aforementioned Winnograd Schema challenge, among other tests that are said to be reflective of natural language understanding (as opposed to simply its processing or the optimal statistical generation of language). The most recent result by Google (T5) has reached parity with the Human Baseline in terms of the average performance across all tests.
The leaderboard is available below, and the site includes subscores for each of the challenges:
[1] https://super.gluebenchmark.com/leaderboard


text:
How can I identify the car (stored in a database) that corresponds to the car in the new image?
I am looking for a solution that I can use to identify cars.
I have a database with images of cars. About 3-4 per car. What I want to do is upload a picture to the web of the car (picture taken with camera/phone), and then let my pc recognize the car.
Example. Let's say I have these 2 pictures in my database (Mazda cx5)(I can only upload 2 links at max. atm. but you get the idea).

Now, I am going to upload this picture of a Mazda cs5 to my web app:

Now, I want an AI to recognize that this picture is of a Mazda CX5 with greyish color.
I have looked on the net and found 2 interesting libraries I can use, TensorFlow and Clarifai, but I don't know if these are going to work.
So, my question to you is: What would be my best bet to go with here?

Please, do not roll back my edit unless it interferes with the meaning of your original post. I edited your post to fix some language issues and to improve the title: I didn't change anything else. Isn't the title that I propose more descriptive of your problem? If you are not comfortable with the idea of your contributions being collaboratively edited by other trusted users, this may not be the site for you.. Note that you still appear as the author of this post.

Since this question has been receiving several flags and had some edit wars going on, I'd just like to note again that I also looked into the edit applied by @nbro and that edit looks perfectly fine and warranted, no reason to continue rolling it back. See the links provided by nbro above for more info. I'm going to try temporarily locking this post once more since the edit war resumed again after the previous lock. Please stop this. If there are specific issues with the edit that change the original intent of the question, please address only those issues specifically and explicitly

Model of the car
What you want to do is close to one-shot image recognition. You have not 1, but 3-4 examples of each car, but that is still a small amount, especially considering the car looks different from different angles (are you supposed to recognize them from any point of view, including sideways, rear, front, and 45 degrees etc.? maybe you also want to recognize them photographed from the top?).
One interesting article I found is: Siamese Neural Networks for One-shot Image Recognition by Koch, Zemel, and Salakhutdinov.
I also found that Caffee supports Siamese networks.
You may want to read other literature about the One-shot learning.
One trick you can do is to utilize the fact that cars are symmetric, so you can double the number of learning examples by reflecting each image.
Color
Determining the color is not as simple as it seems. Your algorithm need to determine where is the car at your picture, and then determine the color, taking into account the lighting conditions, as well as light effects, most notably reflection. For example, consider the following image: .
We see strawberries as red, but there are no red pixels on this picture. The images of strawberries consist on grey pixels.
Maybe you also need a convolutional neural network, or just a neural network, for this task.

If you are going to use a CNN to do the classification you are going to need a large data set to train the network. The Stanford Cars Dataset has 16,185 images and 196 classes of  cars. It can be found and downloaded from Kaggle at https://www.kaggle.com/jessicali9530/stanford-cars-dataset

There are several ways you can do this. One way would involve several steps and would probably work best:

Use a trained Gaussian detector to filter out the car from the rest of the image
Use a convolutional neural network to classify the car
Use a neural network or a simple most common color algorithm to find the color of the car

You would be able to implement this method most easily in MATLAB but you would also be able to do it in python with tensorflow or torch. You would probably be able to implement the trained Gaussian detector in tensorflow.
Method 2:

Use a spatial transformer network to "transform" the image of the car for easy classification
Use the output of the spatial transformer network for classification via a convolutional neural network.
Use another neural network or a most common color algorithm to find the color of the car.

This method would also work pretty well but using a spatial transformer network with a convolutional neural network may be hard to code because it is an area of developing research where there are many problems because the spatial transformer network and the convolutional neural network have to work well together and this is usually hard to get right.
Method 3:

Use a convolutional neural network straight up on the input image maybe with down sampling to classify the car
Use another neural network to find the color of the car

I would personally go with method #1 because it would be fairly simple to implement with existing libraries such as tensorflow and it would most likely provide a high accuracy.
As always I would also recommend that you use LIME during the development process to debug your model and determine what features you could add in or remove to help your model perform better.
**Edit*
Since you need to detect certain patterns on the cars for color classification I would recommend that you use a convolutional neural network to classify these patterns. So your method would now look like this:
1. Use a spatial transformer network or a filtered Gaussian detector to filter out the car
2. Use a convolutional neural network to classify the make and model of the car.
3.  Use another neural network that has either a convolutional or deep architecture to classify patterns and solid colors. So the outputs would contain all of the colors that you want and all of the patterns that you want to detect.

Hi Aiden, thank you for your answer. I will look into all of these. Would option 1 also be viable for recognizing 2 different color, for example zebra stripes?

Using a neural network yes, you would be able to detect two different colors.

Hi, aiden. I have looked into it a bit more, another question I have is would pattern recognition work with these CNN's? when a car has for example the american flag printed on it would I be able to detect that pattern?


text:
What are examples of AI that use philosophy derived ontologies?
I'm looking for AI systems or natural language processors that use, in the classification and interrelation of notions/objects, some philosophical system, like basic laws of logic, Kantian or empiricism.
I have also read about goal-seeking procedures. Are these based on psychology fields and some particular psychology theory or these are ad hoc experiments, with only general terms applied?

Excellent question!  I'm currently working on the pre-Socratics as the most basic philosophies for first principles (These philosophers are intriguing for their simplicity and universality, and the "dawn of consciousness", in some conceptions, may be ascribed to the Classical Era.  Linguistically, ancient Greek is fundamental to meaning in the West.)
However, I was pointed at this very interesting blog post "Algorithms of the Mind" which has some useful links, and discusses Kant, which you may find useful.


text:
Confusing Matlab Artificial Neural Toolbox script
I'm working on a project which uses artificial neural network. I looked up at the Matlab Neural Network toolbox. I got a Generated Script from it. When looking at this script, it is confusing because for both testing and training it seems that the toolbox just uses the same data. Could you explain the reason?
The script is given below:
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

% Train the Network

[net,tr] = train(net,inputs,targets);

% Test the Network
outputs = net(inputs);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)

% Recalculate Training, Validation and Test Performance
trainTargets = targets .* tr.trainMask{1};
valTargets = targets .* tr.valMask{1};
testTargets = targets .* tr.testMask{1};

trainPerformance = perform(net,trainTargets,outputs);
valPerformance = perform(net,valTargets,outputs);
testPerformance = perform(net,testTargets,outputs);

Also is it right to split the data set as below for training and testing?
trainData = inputData(:,1:213);
trainTargetData =  targetData(:,1:213);
validationData = inputData(:,214:258);
testData = inputData(:,259:end);
testTargetData = targetData(:,259:end);
validationTargetData = targetData(:,214:258);

[net,tr] = train(net,trainData,trainTargetData);

% Validation
outputs = net(validationData);
errors = gsubtract(validationTargetData,outputs);
performance = perform(net,validationTargetData,outputs);

% Test the Network
outputs = net(testData);
error = gsubtract(testTargetData,outputs);
performance = perform(net,testTargetData,outputs);

Sorry for the late notice, but just for future readers' sake: programming/implementation questions are much better suited to [stats.se] or [datascience.se], since this site is for social/conceptual/scientific questions about AI (as enumerated in the [help/on-topic]). I'm glad you got a good answer here, though!

The Matlab Answers forum has a lot of answered questions about their Neural Network toolbox too.

I don't see anything wrong in the generated code.
Let me explain:
In the following block, we define that 70% of the data will be used for training the network, 15% for the validation, and 15% for the testing:
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

Then the network is trained from the inputs/targets passed as arguments. Note that here, all inputs are given at once for efficiency. The separation training/validation/testing is done inside the net function from the proportions passed in the net input argument (through the net.divideParam.XRatio properties).
% Train the Network
[net,tr] = train(net,inputs,targets);

Then we test the results of the network on the data without discrimination between training/validation/testing. This performance score should be higher than the "real" performance score of the net as the performance over the training data should be higher than the one over the validation/testing data.
% Test the Network
outputs = net(inputs);  % Compute predictions from the inputs
errors = gsubtract(targets,outputs);  % Compare the predictions to the targets (true values)
performance = perform(net,targets,outputs)  % Built-in Matlab function giving an overall performance score

Now that we know how the net performs overall, we can look at how it performs on the training, validation, and testing sets, respectively. The masks are created as an output of the net function, from the proportions given in the first block of code. The output have been already all computed when the net function has been called.
% Recalculate Training, Validation and Test Performance
trainTargets = targets .* tr.trainMask{1}; % Apply a mask (0's and 1's to select the proper targets)
valTargets = targets .* tr.valMask{1}; % idem
testTargets = targets .* tr.testMask{1}; % idem

trainPerformance = perform(net,trainTargets,outputs); % Compute a score for the training data
valPerformance = perform(net,valTargets,outputs); % same for the validation data
testPerformance = perform(net,testTargets,outputs); % same for the test data

I think your code does the exact same thing but in more steps, with multiple calls to the net function and without using its native way of working (you define the proportions of training/validation/testing by hand for instance without making use of the net.divideParam.Xratio property). So it is probably not optimal from the computational point of view.


text:
Are the shortcomings of neural networks diminishing?
Having worked with neural networks for about half a year, I have experienced first-hand what are often claimed as their main disadvantages, i.e. overfitting and getting stuck in local minima. However, through hyperparameter optimization and some newly invented approaches, these have been overcome for my scenarios. From my own experiments:

Dropout seems to be a very good regularization method.

Batch normalization eases training and keeps signal strength consistent across many layers.

Adadelta consistently reaches very good optima

I have experimented with scikit-learn's implementation of SVM alongside my experiments with neural networks, but I find the performance to be very poor in comparison, even after having done grid-searches for hyperparameters. I realize that there are countless other methods, and that SVM's can be considered a sub-class of NN's, but still.
So, to my question:
With all the newer methods researched for neural networks, have they slowly - or will they - become "superior" to other methods? Neural networks have their disadvantages, as do others, but with all the new methods, have these disadvantages been mitigated to a state of insignificance?
I realize that oftentimes "less is more" in terms of model complexity, but that too can be architected for neural networks. The idea of "no free lunch" forbids us to assume that one approach always will reign superior. It's just that my own experiments - along with countless papers on awesome performances from various NN's - indicate that there might be, at the least, a very cheap lunch.

There are several related questions on this site: this, this, this. There are probably other posts.

Neural Networks have other short comings as well.

It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are resource constrained in anyway, you probably should not look at Neural Networks first. Evaluation of a trained deep NN can be much more expensive than competing techniques too.
The effort involved in learning how to architect and train a NN is still much higher than competing methods, like an SVM. People who are just starting out in Data Science should probably use other techniques to learn about the nuances of fitting data before getting involved in neural networks. And although simple NNs with only one or two hyperparameters are often available in
many data science libraries, they don't perform any better than other techniques so are just another ML black-box technique really.
While we have made a lot of progress in understanding how neural networks do their magic, they are still less accessible and dissectible than most competing methods. So while NNs might solve the problem, they might not give you as many insights as easily as other techniques do.

Looking forward to what other people have to say here.

Good points, although some of the performance issues can be mitigated by GPU's, I think. Additionally, I can warmly recommend Keras, as it enables very easy composition of basic MLPs. Although, as you say, when using a very high level library such as SciKit Learn, it is a bit hard to get good performance, as the hyperparameters are a bit too superficial, and they don't provide some of the newer methodologies.

Yes, but even with massive numbers of GPUs, they are still a lot slower than competing techniques. DNN training is really heavy, I am working on projects at a very big tech company where they routinely talk about days of training of DNNs on very big GPU clusters devoted to these kinds of tasks alone. You don't see that kind of computation devoted to random forest based algorithems AFAIK.

I see. But I guess this is in exchange for a vastly superior performance from DNNs? Or perhaps the margin that makes a difference?

Well, we know DNNs are potentially capable of human level intelligence :) - although we don't know exactly how yet. Concretely, today, we know they give us the best results for tasks like complex image recognition, decoding sound, intent inference in a conversation, and a few others.

Just to add to what has been said in @MikeWise's brilliant answer,

All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:

Like everything, it all boils down to the dataset at hand, neural networks are good on other datasets but at the same time, they will be bad on other datasets. When it comes to unstructured problems(e.g visuals, text, sound), at this time neural networks appear to be the best algorithm. That said, when it comes to structured data, a quick scan at the type of algorithm used to win online data science competitions reveal that, the so called machine learning algorithms such as XGboost rank supreme.
When it comes to other models, feature engineering plays a big role in the algorithm's efficiency. Feature Engineering is generally a tricky thing to do and do right. Deep learning algorithms don't require as much feature engineering(if any at all) compared to other algorithms, in fact they learn features on their own.
If the Google guys say they didn't see deep learning coming who is to rule out the possibility of some so called machine learning algorithm coming out and taking over the world by storm?
Here is a Poll on what data scientist said when asked: if deep learning match the hype in real world application?.
Even some of the popular deep learning applications like Google's AlphaGo aren't 100% deep learning, instead they are part deep learning, part good old "machine learning". My 2 cent is, maybe we shouldn't rule out other machine learning algorithms yet.


text:
Do dreams have a similar role to backpropagation in deep learning?
Humans often dream of random events that occurred during the day. Could the reason for this be that our brains are backpropagating errors while we sleep, and we see the result of these "backpropagations" as dreams?

Neural backpropagation and backpropagation in AI are 2 different things.

This is a very interesting question and also an important one for AI. All the current Deep Learning successes are built on the effectiveness of backprop, so what if it doesn't play a role in the only examples of intelligence currently around?
It makes a lot of sense for the brain to employ some form of backprop, because that would allow it to create low-level features in a way most conductive for the high-level features that finally give rise to intelligent behavior. And if the brain employs backprop, dreams are a logical candidate.
Of course the problem here is that the brain doesn't calculate and propagate gradients, so the question is whether the backpropagation algorithm can somehow be implemented with the biological constraints in mind, such as the locality of neural computation and the non-symmetrical feedback connections.
It turns out this is actually possible, as explained in these talks by Hinton and Bengio. Symmetrical feedback connections (and in fact continually updated feedback connections) aren't necessary for backpropagation, however unintuitive this may seem. And backpropagation can be emulated with a form of temporal difference learning, that is plausible within biological constraints.
Of course this doesn't conclusively answer the question, but it certainly makes the possibility much more plausible than a priori assumed.

There has been a lot of research in cognitive science on the relationship of sleep/dreaming and memory/learning.
I don't know enough about the subject to say if it resembles backprop in spirit, by as the BlindKungFuMaster points out, that may be corollary.
Here's a paper from 2004, Memory Consolidation in Sleep: Dream or Reality?, which concludes that "there is no compelling evidence to support a relationship between sleep and memory consolidation."
However, a more recent article from the Harvard Medical School,  Sleep Helps Learning, Memory (2015), comes to a different conclusion, citing a 2010 study:

A 2010 Harvard study suggested that dreaming may reactivate and reorganize recently learned material, which would help improve memory and boost performance.

The general idea is that sleep/dreaming is a process where neural connections in the brain are reinforced or suppressed.  Following are several papers that touch on that topic.  The first one, in particular may be of interest, and references the work of Hinton, although be aware it's over 20 years old, before recent NN breakthroughs:
Neural Networks: Sleep and Memory (Sejnowski, 1995):

Hinton et al.6 have provided an elegant new theoretical framework for creating efficient memory representations in hierarchical neural network models. In this model (Figure 1), the feedback connections generate patterns on the input layers of the network that correspond to the representations at the higher level, when the external inputs to the cortex and feedforward processing have been suppressed. During this generative sleep stage, the strengths of the feedforward synaptic strengths are altered. Conversely, during the awake stage, the feedback connections are suppressed and the sensory inputs drive the feedforward system, during which the weights on the feedback connections can be altered.

Dreaming of a Learning Task is Associated with Enhanced Sleep-Dependent Memory Consolidation (NIH, 2010):

These observations suggest that sleep-dependent memory consolidation in humans is facilitated by the offline reactivation of recently formed memories, and furthermore, that dream experiences reflect this memory processing. That similar effects were not seen during wakefulness suggests that these mnemonic processes are specific to the sleep state.

Learning while you sleep: Dream or reality? (Harvard Medical School, 2012)
Neuroscientists Reveal How The Brain Can Enhance Connections (MIT Tech Review, 2015)
Memory Consolidation Reconfigures Neural Pathways Involved in the Suppression of Emotional Memories (Nature, 2016)
Sleep and the Price of Plasticity: From Synaptic and Cellular Homeostasis to Memory Consolidation and Integration (National Institutes of Health, 2014)
The brain uses REM sleep to cut unneeded connections (Ars Technica, 2017)
The Brain’s Connections Shrink During Sleep (The Atlantic, 2017)
And, just for fun:
The Link Between Dreaming and Learning Is Stronger Than Ever. How Long Until There’s an ‘Inception’-Style Classroom?


text:
What chatbots can answer this type of simple question that only requires to keep track of the context?
I have tried with two chat-bots Cleverbot and Alan, and I got disappointing results.

me: Socrates is a man
bot: blah blah (common bot nonsense instead of an "ok")
me: Who is a man?

alan1: The people that write my answers haven't provided an answer for
this.

Another example of the mediocre "clever bot"

me: Socrates is the name of my dog.
clever bot: I don't know!
me: What is the name of my dog?
clever bot: That's a nice name.

Another example.

me: Socrates is a man.
clever bot: When does the narwhal bacon?
me: Who is a man?
clever bot: Men are man.

And they dare name this thing "clever"...
So, is there any chatbot that can actually answer this straightforward question?

I think the issue here is that the chatbots you're using aren't very good at "short-term memory". What I mean by is that the bots construct responses that are slowly and incrementally tuned according to the overall usage of the chat bot, from every user. The bots are responding to each message based on how a new user would expect them to. As Alan1 notes, "Men are Man". It's making this response based solely off your single most recent message.
Instead, you are looking for a bot who focuses moreso on persistent memory of the individual conversation. The problem here becomes you're now almost asking for a Natural Language Parser, a big problem many people are working on and something that's years away from existing as robustly as you suggest.
The chat bot not only has to recognize the words 'Socrates', 'Name', and 'Dog'; but that in this sentence, it's the dog's name that is Socrates. That's a lot of information to gain beyond just the words. Which is why from a server / implementation standpoint, the above method is also a lot easier to program (every message just query your server, no need to maintain state - that is, memory of the conversation).
The chat bots can't possibly get enough information from one person to train how to speak and respond, so they 'crowd-source' that information for training. But that means that Clever Bot (or any similar caliber chat bot) won't respond in terms of parsing the meaning of what you're asking.
Taking this even further, one can consider the notion of such a program being Turing-Complete. Supposing we had a chat bot like you're suggesting, we could perhaps show equivalence to a Turing-Machine, or even perhaps show we can do something like decide the halting problem. Off the top of my head I imagine the procedure being basically showing you would be able to decide halting given initial conditions. E.g. Given "Socrates is a man" and "All men die" can we decide if the chat bot will ever be able to deduce if Socrates dies?
I'll work on a formal proof from the latter and post it if it works out.

My opinion is that this type of algorithms can not even be called "artificial stupidity" because stupidity requires a tiny amount of intelligence to be present, and there is none in them.

Um I think I agree but are you sure this is the right thread @johnAm

I'm not sure I understand your last comment. Anyway I'm reading now the essay "Understanding natural language" by Terry Winograd. It's a great work... Thanks for your answer.

Just to lean your notice to specific problem. In my opinion, all your questions "Who is a man" (beside "Socrates dog..." one) are incorrect, and not a bot, but a human shouldn't answer them on first look, in spite of what you told it before ("Socrates is a man")... Yes, it might know that Socrates, John Kennedy or Leonardo DiCaprio is a man, but you just "expected" it to get the answer you wanted, because you mentioned it earlier.
During conversation, humans can catch & understand grammar, semantical or logical (the most important part) mistakes in another person's questions, and in return they can give correct answers, but bots depend mostly on correct grammar and correct semantics of questions.
There might appear some bots that could overcome incorrect grammar slightly, nowadays, even Googles Search's AI hardly struggles to correct a wrong input, other than basic grammar or basic semantics), and you supposed that chat bots could "understand" semantically incorrect question, and you rant them heavily with no reason and call stupidity or whatever.


text:
Does neuroevolution require a labelled dataset?
A neuroevolution algorithm, such as DXNN, can be used to refine the topology and weights of an artificial neural network (ANN). The GA will require a fitness function, which means you need labeled data for comparison.
Can neuro-evolution be used with unlabelled data? I have no labeled datasets.

Could you give a link or an explanation for feed-forward neuro-modulation?

@BlindKungFuMaster https://github.com/CorticalComputer/DXNN2 it is a genetic algorithm that tunes the NN weights and add remove neurons. GA requires to know how each individual performed using their fitness which is established by testing the individual NN solution against a known answer.

The GA will require a fitness function, which means you need labeled data for comparison.

That conclusion is wrong. Yes, sometimes your fitness function will use labeled data. For example, if you want to train an XOR gate or any other known function.
However, there is arguably no advantage of training a function with neuroevolution versus backpropagation, except for the fact that you might discover some new architectures which solve the solution very well.
You don't always need a labelled dataset for neuroevolution.
Take for example IAMDinosaur, which trains neural networks through a genetic algorithm - however, the optimal solution is not known. There is no labelling of input data, all it does is calculate the fitness from the score.

Got it. That gives me three types of problems, 1- labeled data, 2- unlabeled data but a simulator that gives feedback, 3- no label no feedback. What to do with the a problem of the latter type? like here is a log file, detect what is good and what is bad. Would a classification like SOM works for that? I it feasible?

The neural network in IAMDinosaur maps from sensors to actions (if I understand correctly) and its weights are modified to maximize the rewards it receives from its actions. This makes it a reinforcement learning problem (see Wikipedia for a concise definition).


text:
What are examples of techniques to prevent bias in artificial intelligence systems?
I recently read an article about how artificial intelligence replicates human stereotypes when applied to biased datasets.
What are examples of techniques to prevent bias (and stereotypes) in artificial intelligence (in particular, machine learning) systems?

There was also the disaster in which Microsoft made a Twitter bot to emulate millennials and it learned/tweeted things that were, uh, less than desirable.

It's important to note that, ultimately, the statistical methods we currently use in ML research are just that: statistical methods. So, when they show some "bad behaviour", it's not because of problems with the statistical methods, but with the data we give them. But if the data we give them are as "genuine and unfiltered" as it gets, then it probably shows something about us.
From a cognitive science perspective, it's probably the case that the same heuristics and biases that create stereotypes are also the ones that make us powerful agents (note the similarity between categories and stereotypes), so, at least at this moment, it's unclear how we can segregate desired from undesired behaviour.
To combine the points mode above, it seems we can only either:

Remove "bad content" by curating the data by hand or by some metric that we don't know of yet

Accept that our methods will produce AI as "bad as we are", because that's what we are, and let it operate under the knowledge that it might produce undesired behavior sometimes.

Unless we have some crazy new theory of mind that we can begin to analyze this more rigorously, it seems like there is no clear cut solution.

The field of statistical psychology offers many methodologies to remove bias from datasets, or rather gather a dataset with minimal unknown biases. This will be the responsibility of the programmer. An AI that learns from datasets will not be able to find bias in those datasets.

In other words, fudge the data so it gives the results that the researcher wants instead of the "real" results. It seems like this is the where the future of scientific research is heading.

@Dunk Biased data is already fudged: otherwise there'd be no need to debias it.

@JeremyList - Fair enough, the data could be biased and it is reasonable to 'debias' the data, as long as a concrete explanation justifying why the data is biased can be determined first. Otherwise, debiasing data just to give more desirable results is essentially falsifying the data to give the answers the researcher is looking for. All the computer is going to do is find patterns and the most significant relationships in the data. Just because someone doesn't like what the computer finds doesn't automatically mean the data is biased.


text:
What are good alternatives to the expression "Artificial Intelligence"?
I read a really interesting article titled "Stop Calling it Artificial Intelligence" that made a compelling critique of the name "Artificial Intelligence".

The word intelligence is so broad that it's hard to say whether "Artificial Intelligence" is really intelligent. Artificial Intelligence, therefore, tends to be misinterpreted as replicating human intelligence, which isn't actually what Artificial Intelligence is.
Artificial Intelligence isn't really "artificial". Artificial implies a fake imitation of something, which isn't exactly what artificial intelligence is.

What are good alternatives to the expression "Artificial Intelligence"? (Good answers won't list names at random; they'll give a rationale for why their alternative name is a good one.)

With artificial~made by human mind (vs emerging from nature, which kind of excludes human kind from nature, which is contradictory), it seems pretty accurate. Artificial is not isomorph to fake or imitation !

Artificial is said to derive from the Latin word "artificium" which connotes ideas such as crafting.  Thus, artificial is a correct usage, and algorithms can be regarded as "artifacts" in the context of information as opposed to physical manifestation of information (i.e. matter).
However, I agree that the use of artificial is problematic in that, should strong Artificial General Intelligence ever be achieved, there is a stigma to "artificiality" that could have implications regarding personhood.
My personal feeling is that we should be using:

Algorithmic Intelligence

which this is functional definition, and therefore more meaningful than "artificial".  Additionally, "algorithmic" is a neutral term, and provides a very accurate description of what these systems are.

In terms of what is considered "intelligent", you may want to look at the concept of Bounded Rationality.  There is no hard definition of "intelligence", just degrees of optimality in regard to decision making in a condition of uncertainty.
Because this is subjective for any problem that is not solved, modifiers are utilized, and thus we refer to AI as "strong" or "weak". These terms are also used to describe the degree to which certain types of problems (for instance a non-chance, perfect information game like Checkers) has been solved. Complexity theory will shed more light on this concept.
For more insight on "artificial", you might find this question on the philosophical origin of the Turing Test interesting, because it partly involves the meaning of a "thing". (There were multiple words for this in Ancient Greek.)

I don't think that "algorithmic" is a neutral term. Maybe it is a less ambiguous or more specific term than artificial.

@nbro Good point.  "Algorithmic" has been getting scrutiny lately over issues like bias. (If the computationalists turn out to be right, and and human intelligence is algorithmic, the term would apply equally to artificial and biological life.)

Google defines 'artificial' as something created by humans rather than occurring naturally so I wouldn't quite say that it's so bad.
Given the question however, you could perhaps say "smart machines" since that's what they essentially are these days.
Artificial Intelligence is a very broad term, pre-dating modern AI, simple things such as mechanical wooden robots were considered Artificial Intelligence.
https://en.wikipedia.org/wiki/Timeline_of_artificial_intelligence

Why would 'smart machines' be better? Isn't that kind of ambiguous?

I doubt Google has definitions. Please make sure to cite the original source.

There are several expressions that are often used as synonyms for artificial intelligence, but, nowadays, the most common ones are likely machine intelligence and computational intelligence.
However, these expressions are not well defined, so not everyone will agree that they are interchangeable, but we can all agree that these fields (either if we consider them the same or not) are quite related to each other (and they overlap).
Moreover, these fields also evolve over time and they embrace techniques from other fields, which makes it more difficult to define them. More concretely, initially, AI was mainly based on the manipulation of symbols and logic, but nowadays AI is mainly machine learning, statistics and, in particular, deep learning.
Furthermore, the expression artificial intelligence was apparently coined after the term cybernetics, which some people might consider the first serious attempt to building intelligent systems.

These are correct. Artificial implies that it runs on artifically made hardware. There is no reason to distinguish between the natural processes what do the same.
Further, the term intelligence is nor realy precise. What is more/ less intelligent or has or has not intelligence among : mowgli, monkey, crow, common game bot, whatever? MAing thing, some learning on data happens here.
The best alternative would be Mashine learning, but again, that mashine like  "artifically made stuff" does that is irrelevant.
So my definition is:
 Algorithmic Learning.

Machine Intelligence

I believe intelligence is not a proprietary entity meant only living beings.
In fact the very origin of human intelligence is unknown. It is still not known if we can generate a brain just by fixing up the corresponding molecules of the real brain(Even theoretically). Even if we could do that does that constitute a real intelligence or artificial intelligence is still very hazy.
I think the phrase "Machine Intelligence" would sound appropriate.

'It is still not known if we can generate a brain just by fixing up the corresponding molecules of the real brain' where did you get this statement? Like really if something is identical to something its behavior is also identical (identical at fundamentally quantum level)

@DuttaA  True, but let's not going to quantum level as it's too complicated (We cannot have exactly same brain in quantum level due to no-cloning theorem).
The main question is what after that? Do we consider "Real" intelligence or "Artificial" Intelligence is still a question.

@DuttaA Does a artificially created brain has a mind? If it does exactly the same as a normal mind then how do we differ it from a "Real Intelligent Machine"

I don't knoow why is this downvited so badly

Josh Worth provides a critique for the use of the term artificial intelligence in his Stop Calling it Artificial Intelligence, but there are some caveats in the treatment of the term.  (One is a typo in the URL path.)
It is not the term Artificial Intelligence that is the issue.  It is what is collected under it in the media and sometimes in academic literature.
Is Artificial Unclear?
The term artificial is not particularly ambiguous or inaccurate.  Artificial doesn't mean fake as Worth suggests.  It simply means that it did not arise from natural processes.  High end artificial flowers feel and smell like they are grown.  Artificial flight is now called flying.  We no longer see birds as the exclusive pilots of flight, so the word flying has changed to include artificial things called aircraft.
People don't normally make the mistake of placing the adjective artificial along side human capabilities, so there is no tendency to misnomers.  There is a problem with the use of the term artificial intelligence though.  Worth is correct in that overall statement.  There may be two forks to the misuse of the term.
Defining Intelligence
One problem is with the term intelligence.  The definitions we've seen are considerably problematic.  Some are flat out wrong and can be proven so with heaps of counterexamples.  Furthermore, definitions tend to be qualitative and difficult if not impossible to quantify.
Some propose the standardized testing of academia to quantify intelligence. If we use that definition of the word, from the staunch g-factor adherents, artificial intelligence is a target idea and nothing comes close to approximating it. No computer system has yet been admitted into a major university on the basis of high scores in college board testing.
If we use the ability to learn as a standard, then flies are intelligent because they frequently learn from one swat strategy changes to avoid getting squashed by the irritated person.  There are a thousand other reasons why learning, by itself, is not an adequate characterization of human intelligence.  A crack addict can learn how to make a purchase without holding a job. We wouldn't characterize that as intelligence, but rather dysfunction.
Excessively Inclusive Use of the Term
Another problem arises from avarice. To appear as an expert in what is perceived to be a lucrative expansion of technology, some who have no conception of the topic sometimes present conjecture as if it were peer reviewed fact.  This has been typical of popular topics for centuries.  There is often insufficient bandwidth of peer verification available to address even a small portion of publicly available information.  Web publication has only augmented this problem.
Resulting from this coveting of expert reputation is collecting under the name artificial intelligence a number of things that are not intelligent.

Multi-dimensional control systems
Brute force searches of permutations
Decisions made by statistically analyzing sample data
Parameterized functional networks that converge to a defined optimal behavior

Those of the above that have no component of

Cognition,
Comprehension,
Complex modelling and use of such models,
Semantic mapping,
Rational inference,
Or some other clearly distinctive and broad form of adaptation

should not be included under the technology that exhibits authentic artificial intelligence.  However, those that developed the theory of control systems, searches, parameterized function convergence, and statistics or developed working systems that use that theory are intelligent, just not artificial.
The above four might have terms that distinguish them from authentic intelligent systems outside the realm of biology.

MDC — Multi-dimensional Control
BSS — Brute force search
NBL — Network Based Learning
SDS — Statistical decisioning systems

It seems that terms that have three words and form distinct acronyms get further and last longer.


text:
Does Musk know what gradient descent is?
From Futurism.com:

Musk indicates that internet infrastructure is "particularly susceptible" to a method called gradient descent algorithm, a mathematical problem-solving process. Bad news is, AI is excellent at doing gradient descents, which can become devastating digital weaponry.

A screenshot of the tweet is provided below.

He is an intelligent man, so why would he say something like this? Does Musk know what gradient descent is?

I believe Musk would make this comment because he is aware of the valid questions raised by Neo-Luddism in relation to the dangers of technology. Rather than reject technology, Musk is trying to raise awareness at a time when "narrow" Algorithmic Intelligence is exceeding humans at specific tasks, and may advance beyond our capacity to understand what they are doing. My personal feeling is that Musk's industrial endeavors are not primarily motivated by personal profit, but out of a deep concern for the future of our species, and uses his celebrity as a platform to raise awareness of issues.

I think Musk was using the terminology correctly though perhaps with hyperbole.
I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time.  He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a toolbox of targets and hacks and using well-known machine-learning techniques to optimize the effectiveness of using the attacks in combination to take down the internet.
Gradient descent itself isn't harmful; it is using gradient descent to train a botnet that could theoretically result in a really devastating attack.
I also don't think he is necessarily implying that there is an actual machine learning system at work; rather that the hackers were training themselves with a methodology analogous to gradient descent.

Thank you for this excellent context.  This subject may require hyperbole at the present state of AGI, but I think it's important to start attempting to tie moral philosophy to Algorithmic Intelligence, particularly in a game theoretic (economic) context, which has many applications in computing, and may be said to form the basis for Algorithmic Intelligence in terms of optimal decision making in the context of bounded rationality.  As someone engaged in research on general economic automata in partisan contexts, deeply concerned about value alignment, I think this is of paramount importance.


text:
Is it feasible to train a Machine Learning Model (with image inputs) in an average personal computer?
There are lots of examples of machine learning systems that can recognize objects and extract other information from images with very high precision. To train the models of such systems is necessary (I guess) a computer with a lot of computational power.
My question is: For a system with images as inputs, depending on the complexity of the problem, is feasible to train a model in an average laptop? It will take to much time?
I know the time taken to train any machine learning model will be a function of lots of variables. I really don't expect a quantitative answer here, I just want to know whether I will be forced to upgrade my computer to develop and train machine learning models that have images as inputs.

Of course it's possible. We can even do it on the phone. All depend on your model and data.

You may play around on an average laptop but training will be very slow and you will be limited on the size of your model.
Once you try to build something more serious you will run out of memory very fast. A system with a GPU is recommended if you want to really do things like image recognition. If you buy something I would not go for any GPU with less than 8 GB memory and for an X99 motherboard to keep the flexibility to add a second GPU on full speed later if needed.
Don't mix up image recognition running on phones etc. Those are trained models and actually just being applied that is much easier process. Training is way much more expensive.


text:
How can one find / collect data for, and come up with ideas for, using Deep Learning / AI to improve one's everyday life?
I can see a lot of tutorials and examples about using TensorFlow and other free, open-source AI/ML/DL frameworks on enterprise level where enough data was collected for such AI solutions.
How can one can collect enough data in normal everyday life to practically and effectively make use of such freely available AI/ML/DL technologies to improve one's life and security?

I edited the question title to try and turn it into more of a "proper" question.  I think this actually is a really good question in the general sense, but if anybody has suggestions for edits to make it better, fire away.

Speaking to the "collecting data" part of the question, I'll say this:
Keep in mind that not everything requires massive amounts of data. Consider also that large amounts of data about all sorts of things are available in the OpenData / LinkedData realms. Governments in particular are sources of massive amounts of data.  See, for example, data.gov or Google around and see if your (state|city|county|whatever) has an "open data" portal. You might be surprised.
Finally consider that inexpensive sensor systems and distributed sensor networks can be constructed fairly easily / inexpensively using things like Arduino / Raspberry Pi and communications protocols and software often associated with the term Internet of Things.  You may find that you can actually collect your own data, especially if you crowd-source the effort and get a bunch of other people involved.
See also:
http://opendata.reddit.com
http://datasets.reddit.com
http://www.wikidata.org
http://www.dbpedia.org
etc.


text:
Can AI write good jokes yet?
Just watched a recent WIRED video on virtual assistants' performance on telling jokes. They're composed by humans, but I'd like to know if AI has gotten good enough to write some.

I don't think the AI has gotten to that point yet. Here are some of the interesting papers on the subject:

A paper was recently written that attempted to generate jokes using unsupervised learning. The jokes are formulaic: they're all of the form "I like my X like I like my Y: Z" where X and Y are nouns, and Z is an adjective that can describe both X and Y. Here are some of the jokes generated in this paper:
I like my relationships like I like my source, open
I like my coffee like I like my war, cold
I like my boys like I like my sectors, bad

How funny these jokes are is a matter of personal taste I guess.
Another paper by Dario Bertero and Pascale Fung makes use of an LSTM to predict humor from a dataset of the Big Bang theory shows. This is not generating jokes but finding out where the jokes are said in this dataset (so theoretically, the resulting labelled dataset can hopefully be used to train a model to create jokes).
Yet another paper is that by He Ren, Quan Yang. Unlike the first paper mentioned above which was unsupervised, this is a supervised learning model. Their neural network model, generates jokes such as:
Apple is teaming up with Playboy Magazine in the self driving office.
One of the top economy in China , Lady Gaga says today that Obama is legal.
Google Plus has introduced the remains that lowers the age of coffee.
According to a new study , the governor of film welcome the leading actor of Los Angeles area , Donald Trump .

My two cents:
As of this writing, it appears that Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models are by far the most promising way to go about it. Maybe if you find some really cool data you can come up with some funny jokes, similar to how Janelle Shane was able to generate what I find to be really funny pickup lines such as:
Are you a 4loce? Because you’re so hot!
I want to get my heart with you.
You are so beautiful that you know what I mean.
I have a cenver? Because I just stowe must your worms.
Hey baby, I’m swirked to gave ever to say it for drive.
If I were to ask you out?
You must be a tringle? Cause you’re the only thing here.
I’m not on your wears, but I want to see your start.
You are so beautiful that you make me feel better to see you.
Hey baby, you’re to be a key? Because I can bear your toot?
I don’t know you.
I have to give you a book, because you’re the only thing in your eyes.
Are you a candle? Because you’re so hot of the looks with you.
I want to see you to my heart.
If I had a rose for every time I thought of you, I have a price tighting.
I have a really falling for you.
Your beauty have a fine to me.
Are you a camera? Because I want to see the most beautiful than you.
I had a come to got your heart.
You’re so beautiful that you say a bat on me and baby.
You look like a thing and I love you.
Hello.

I found those pretty hilarious tbh

Those are a selection of "entirely new ones" - and they are a bit wierd, as the person saying the joke claims to like something that could easily offend a lot of other people. I am guessing that is one reason they are "entirely new", i.e. it is not that they are so new, it is more that people who think them have probably learnt toto keep them to themselves. Interesting post though.

The last "hello" line reminds me of this.

As of now we don't have a satisfying cognitive theory of humor (or at least, one that can evaluate the hilarity of a joke), so a quick survey of the literature seems shows that we don't have much of a clue on how to build a model.
Because of that, and the fact that existing methods don't seem to reliably produce good jokes free form, there seems to be little reason to believe that ML methods can produce good jokes.
But of course this is all normative.

Part of the model would need to be real-time continuously dynamic because jokes are dependent on current/recent events. And if it's a standup AI comedian, then the audience's mood changes and reactions determine its timing. What a great idea, you know how Jeopardy had "Watson"? Well, maybe SNL or CC could have "Chaplin," the first standup AI comedian. ^_^

yeah the point you raise is a very good one. it doesn't seem that our ML methods can encode things like that (timing, historical context, audience reaction) and stuff in a meaningful fashion. more worryingly is that we also don't know ow these things affect humor, more less how to quantize its effect. or to put it more succinctly, we don't know how we even or how to even

Amazingly, I just found a claim about that. I just read it in twitter and the model is simply a GPT-2 WITH 355M params trained with 200,000 raw title and body-based jokes. what is amazing is that GPT-2 is the most advanced text generating model it even can translate or answer math questions if trained well.
Let's see example output from twitter.

"I asked my girlfriend if she knew what sex was like || she said that
you can kiss her and she'll think you're a queer."
Why does the teacher have her own car? || She's a car company for Santa.

https://twitter.com/lgbtinethiopia/status/1294644776772472834?s=20

Personally I have seen the google colab notebook for that, that's why i trusted it.


text:
What algorithms are used for image segmentation of images where objects are not composed of pixels that are similar in value?
In the process of segmentation, pixels are assigned to regions based on features that distinguish them from the rest of the image. Value Similarity and Spatial Proximity, for example, are two important principles that assume that points in the same region will have pixels that are spatially close and have similar values.
In lots of situations, this is true, but what about regions composed of pixels that are not similar in value?
Consider the image below.

The same "logical" region is composed of different elements that together represent something meaningful. In the same region, there are trees of different sizes and shapes, with shadows over some of them, etc. There are different things, with pixels that differ a lot in value, but I still need to group them together in the same region. From the image, you can see that I don't care so much about differences in color. In this case, the texture is the most important attribute.
What algorithms are used to do the segmentation and classification in problems like that?
I'm already looking for some algorithms and techniques that focus on texture, but some opinions from the experts will help me a lot. I think I need some orientation.

If you have enough data, you can train a segmentation net example for a specific group of data (i.e. trees). Single DNN with multiple output branches can easily segment & classify the data that you are searching for.