text:
What is chaotic behavior and how it is achieved in non-linear regression and artificial networks?
I'm finding it hard to understand the relationship between chaotic behavior, the human brain, and artificial networks.  There are a number of explanations on the web, but it would be very helpful if I get a very simple explanation or any references providing such simplifications.

Comments are not for extended discussion; this conversation has been moved to chat.

Regression for models more complex than $y = a x + b$ is a convergence strategy. Surface fitting algorithms, such as Levenberg–Marquardt, are often successful at achieving regression using a damped version of least squares as an optimization criterion. The marriage of regression and the multilayer perceptron, an early model artificial network, led to the use of a back propagation strategy to distribute corrective signals that drive regression.
Back propagation using gradient descent is now used in artificial networks with a variety of cell and connection designs, such as LSTM and CNN networks as a convergence strategy. Both surface fitting and artificial network convergence share the method of successive approximation. With each successive application of some test, the result is used to attempt to improve the next iteration. Proofs have developed around convergence for many algorithms. Actual successive approximation runs have five possible outcomes.

Convergence within the time allotted and within the accuracy required
Convergence within the time allotted but not within the accuracy required
Convergence appears that it would have occurred but time allotted was exceeded
Oscillation appeared by the end of time allotted
Chaos appeared by the end of time allotted

The following illustration from Chaos Theory Tamed (Garnett P. Williams, 1997, p 164) modified slightly for easy viewing can explain how chaos arises when the learning rate or some other factor is set too aggressively.  The graphs are of the behavior of the logistic equation $x_{i+1} = k x_i (1 - x)$ which plots as an inverted parabola in phase space.  The one dimensional maps on the right of each of five cases show the relationship between adjacent values in the time series on the left of each of the five.  Although the logistic equation is quite simple compared to regression algorithms and artificial nets, the principles involved are the same.

The right hand cases, with $k = 3.4$ and $k = 3.75$ correspond to the last two possible outcomes in the list above, oscillation and chaos respectively.
Care in Drawing Parallels
Care should be taken in drawing parallels between distinct things.

Surface fitting algorithms, like Levenberg–Marquardt
Algorithms that realize back propagation with gradient descent
Logical inference AI, such as production systems and fuzzy logic
Real time learning, such as Q-learning
Devotion of the human brain to a problem

Regression and artificial networks can be compared meaningfully because the math for each is fully defined and easy for those with the mathematical skill to analyze them for the comparison.
Comparing known mathematical systems with unknown biological ones is interesting, but to a large degree, grossly premature.  The perceptron, on which MLPs (multilayer perceptrons) and their deep learning derivatives are based, are simplified and flattened models of what was once thought to be how neurons in the brain work.  By flattened is meant that they are placed in the time domain where they are convenient for looping in software and do not take into consideration these complexities.

Neuron behavior is sensitive to the timing of incoming signals — Incoming signals may overlap but not precisely align in time.
Neuron behavior is sensitive to the history of incoming signals (because of cell body and axon thermodynamics, synaptic chemistry, and other neuro-chemical and structural functions not yet understood)
Neuron structure changes in terms of its connectivity
New neurons appear
Neurons may die due to cell apoptosis

In summary, multilayer perceptrons are not a model of neural networks in the human brain.  They are merely roughly inspired by obsolete knowledge of them.
Chaos in the Human Brain
Whether there is chaotic behavior in the brain is known.  It has been observed in real time.  How coupled it is with human intelligence is a matter of conjecture, but it is already clear that it may appear to contribute to function in some cases and contribute to dysfunction in others.  This is also true in artificial systems.

When used to deliberately interfere with a stable condition that may not be the optimum state of stability to find a better one, chaos may be a source of noise that benefits learning.  This is related to the difference between local minima and global minimum.  The good is sometimes the enemy of the best.  Improved learning speed has been documented for artificial network algorithms with deliberate injection of pseudo-random noise into the back propagation.
When appearing not to deliberately inject noise in a portion of the system that can benefit from the noise but out of a basic instability in the system, the chaos can be detrimental to overall system function.  Chaotic behavior in the human brain is a likely cause for various disorders.  There is much supporting data but not yet a proof.

In summary, chaos in a system is neither productive nor counterproductive in every case.  It depends on where it is in the system (in detail) and what the system design is expected to perform.

It looks like you have some common misconceptions about AI and neural networks.
First, AI programs generally do not try to imitate the human behaviour of a human brain. Instead, they try to imitate some higher-level behaviour. For example, they might imitate the reasoning process that you go through when you make a plan. In this context, the building-blocks (silicon or flesh) don't matter too much.
Second, artificial neural networks are also (mostly) not intended to imitate the human brain. Although they are inspired by the arrangement of neurons in a human brain, the networks used in most ANN systems have very little to do with real brains. The main similarity is that both systems have a lot of simple little computational units connected in patterns such that signals passed from one to another lead to interesting computations. However, real neurons produce lots of different kinds of signals, are connected in arbitrary ways, and randomization and transmission times play a significant role. Artificial neurons generally are deterministic, produce only one kind of signal (or sometimes a couple different kinds), are connected in extremely regular ways, and usually simulate instantaneous transmissions between neurons.


text:
How to use a Generative Adversarial Network to generate images for developmental analysis?
I want to generate images of childrens' drawings consistent with the developmental state of children of a given age.  The training data set will include drawings made by real children in a school setting.  The generated images will be used for developmental analysis.
I have heard that Generative Adversarial Networks are a good tool for this kind of problem. If this is true, how would I go about applying a GAN to this challenge?

So, you can find interesting stuff at this github repo, or this berkeley course... I have to add that none of this is my work, and I'm not an expert of adversarial nets so I can't say that here lies your answer. Moreover, please developp your question, expose your project with more details, because I doubt anyone can really help you with so little info...

Welcome to AI.SE @SidneyGuaro. As the other commenters note, it is difficult to understand what you are asking here. In particular, what do you mean by "combine"? GANs are a useful tool for generating new images in certain style. Are you hoping to use a GAN to generate images in the style of children at different developmental ages?

@JohnDoucette yes, exactly

@SidneyGuaro Ok, I edited your question to reflect that. Hopefully it is clearer now, while still matching your intent?

Do you have a big data set of children's drawings? Could you point me to it, if it is publicly available?

Data set is not available for the meantime but it will be around 200 images of children's drawing. The reason to why it is small its because data set that will be collected will be based on a school setting. For example, children with ages 4 produced 190 drawings. It will be used to train the GAN to generate an image for analysis. Another set will be produced by children with ages 5 and so forth.

You won't be able to get anything meaningful out of a GAN trained on 200 images. The data set will be too disjoint for the NN to model features at the semantic/stylistic level you are interested in. If all the images are coloured pencils on white paper, you might expect the GAN to roughly produce lines and blobs of the right colour on a white background. Such output might bring to mind "a bit like a child's drawing", but it will not be convincing or realistic, and you won't be able to analyse what is represented.

. . . alternatively, if the GAN could over-fit, and you end up with slightly distorted versions of the original inputs for most samples, with occasional weird and obviously wrong results. This answer on SO might help a little: https://stackoverflow.com/questions/48601991/training-gan-on-small-dataset-of-images

A generative adversarial network is probably not the best approach for generating the images desired.  We can assume from the comments that the data is not collected.  That's a good thing, because a set of rasterized images, labeled with student age or grade is an inferior input form.
It appears that access to a student population is planned or already negotiated, which is also good.
Although the drawing, as it is being drawn, is seen through each student's eyes, the primary features correlated with drawing skill development is motor control, shape formation, and color choice.  If the sheet of paper is placed over a drawing tablet, the tablet's incoming USB stream events are captured to a file, and the color selection is somehow recorded or automatically determined by having students hold the pencil or crayon up to the computer's camera before using it, a much better in natura input stream can be developed.
Pre-processing can lead to an expression of each drawing experience as a sequence of events arranged in temporal order with the following dimensions for each event.

Relative time from the instruction to draw in seconds
Color
Nearest x grid
Nearest y grid
Pressure

Determining color from camera input may be developed using LSTM approaches.
The dimensions of the label for each of these sequences would be those demographics and rankings that would most closely correlated with developmental stages.

Student age
Student gender
Curriculum grade (-1, 0, 1, 2, ... 12, where -1 is preschool and 0 is kindergarten)
Identifier of the drawing instructions given to the class
Grade ranking of the student in the class

The micro-analysis attached to each ELEMENT in the sequence includes these additional dimensions.

Drawing rate of the utensil given by $r = \frac {\sqrt{(x - x_p)^2 + (y - y_p)^2}} {t - t_p}$ where the subscript p indicates the values are drawn from the previous event in the sequence.
Drawing direction given by $\theta = \arctan (x - x_p, \; y - y_p)$
Curvature $\kappa$ calculated using cubic splines or some other data fitting approach
FFT spectrum $\vec{a}$ and Lyapunov exponent $\lambda$ applied to auto-correlation results

This is a modification of the system Google uses to synthesize speech, based on the WaveNet design. In the diagram, the residual function is defined as follows.
$z = \tanh \, (W_{f,k} x + V_{f,k} y) \, \odot \, \sigma \, (W_{g,k} x + V_{g,k} y)$
The development required is that the $\vec{a}$ must now be accompanied with scalars $r, \theta, \kappa, and \lambda$, but the resulting drawings are likely to have many of the hand-eye developmental features of the examples.

Sir @Douglas, can I ask for a translation of your answer in simple terms or maybe Python code? I'm having trouble understanding the text since I'm neophyte on the field.

I think your task is as follows.
$\ $
Let's assume 5-year-old children.
You have numbers of pictures that drew by them.
(Let, these pictures are your [training set].)
And, you want to synthesis similar pictures with the training set.
Because you need more pictures for your study.
Am I right?
$\ $
OK.
From the pictures, you want to extract some meaningful information about a real child who drew them, right?
Then, I think GAN is not faithful for your study.
Of course, GAN can make very similar pictures with your training set.
However, it does not mean the synthesized images can contain the things what you want!
GAN just synthesizes "fake pictures" that cannot be distinguished from your training set.
The synthesized pictures may do not have any meaningful thing.
Because it is not drawn by the real child.
$\ $
But it is worth to do.
May GAN captures some features of "children-like". (But I think it is too hard.)
You can find lots of GAN for your research, especially DCGAN.

I want to use GAN to generate a generalized image for a certain age (4-12). Just one per age. That image will be used for manual analysis.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Constructing a dataset that scores well only for a specific set of hyper parameter values
When designing a machine-learning system, there are various parameters that have to be determined. I am interested in the following general question: is it possible to construct a dataset on which the system will have good performance with some specific set of parameters, but not with other paramteres?
To be more concrete, let's focus on neural networks. Suppose we have a simple neural network: a multilayer perceptron with a single hidden layer. The size of the input is fixed, the activation function is fixed (e.g. tanh), and the output is binary. The only parameter that has to be determined is the size of the hidden layer.
My question is: given a number $n$, is it possible to construct a dataset $D_n$ such that:

The MLP with $n$ hidden nodes has good performance on $D_n$ (e.g. in 10-fold cross validation);
The MLP with $n-1$  hidden nodes has bad performance on $D_n$

?
Note: I asked in CS theory but got no reply.

I'm not familiar with any existing, robust methods to generate such a dataset. Here are some thoughts though.
You propose using an MLP with a single hidden layer. That means we have two weight matrices, and two activation functions (one for hidden layer, one for output layer). Some notation:

$d$: dimensionality of input vectors
$n$: dimensionality of hidden layer
You mentioned binary output, so I'll assume that dimensionality is $2$
$W_n^{(1)} \in \mathbb{R}^{d \times n}$: the first weight matrix in the scenario where we're using $n$ hidden nodes
$W_n^{(2)} \in \mathbb{R}^{n \times 2}$: the second weight matrix in the scenario where we're using $n$ hidden nodes
$g$: first activation function
$h$: second activation function

Then, given an input vector $x \in \mathbb{R}^d$, our neural network will generate output $f_n(x) = h(W_n^{(2)} (g(W_n^{(1)}x)))$.
Now, in general we of course expect Neural Networks to only find a local optimum, but if you want a robust solution you'll want it to be able to handle the worst case, and the worst case for your "adversarial" task is when the Neural Network manages to find the global optimum. So, we'll assume it can find the global optimum.
Essentially, what you're looking for is a dataset $D_n$ containing a number of input vectors $x$, such that:

$f_n(x) = h(W_n^{(2)} (g(W_n^{(1)}x)))$ provides good / optimal results (after training to the global optimum)
$f_{n - 1}(x) = h(W_{n - 1}^{(2)} (g(W_{n - 1}^{(1)}x)))$ provides poor results (even after training to the global optimum of this setup).

In other words, you want to find a collection of vectors $x$ such that it becomes impossible to find a collection of weights in the $n - 1$ case where $f_{n-1}(x)$ is a good approximation of $f_n(x)$. You want to make it impossible that $f_{n - 1}(x) \approx f_n(x)$.

Now this has turned into a clear mathematical problem. I'm not familiar with any established methods in mathematics to solve a problem like this, maybe there are though. My best guess at this point in time would be a procedure like the following:

Generate random "ground truth" versions of the weight matrices $W^{(1)*}_{n}$ and $W^{(2)*}_{n}$ (just completely random matrices).
Generate completely random input vectors $x$. Compute the corresponding ground truth labels as $f^*_n(x) = h(W_n^{(2)*} (g(W_n^{(1)*}x)))$.
Hope that the neural network with $n$ hidden nodes can recover the ground truth weight matrices that were previously generated randomly.
Hope that the neural network with $n - 1$ hidden nodes cannot find an accurate approximation.

In theory, the MLP with $n$ hidden nodes should be able to learn the exact ground truth function. In theory, under certain conditions, the MLP with $n$ hidden nodes should not be able to learn the exact ground truth function. I suspect those "certain conditions" would be that the rows/columns of the weight matrices should be linearly independent, which is likely with randomly generated matrices, but I'm not 100% sure on this. Even if it can't learn the exact ground truth, it may still be capable of learning an approximation... there may be ways to find upper bounds on how close such an approximation could get, but I'm not sure.

Thanks, these are very interesting insights that may lead to a solution.

Hidden layer number is an indicator of the dimension of the weight matrix , so once you train a network with certain number of  hidden layer neurons  , the weights get fixed at the point you stop training. so changing the number hidden layer neurons makes the previous weights incompatible with it due to change in weight-matrix dimenstion . even if you alter the weight matrix to work with the new network it is again equivalent to a network with randomly assigned weights. so a fully trained networks becomes a new network once you change its hidden layer dimension and gives bad performance. This hase a very little to do with the dataset.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How can AI be used to more reliably analyze and plan around the tie between climate and emissions?
Note to the Duplicate Police
This question is not a duplicate of the Q&A thread referenced in the close request.  The only text even remotely related in that other thread is the brief mention of climate change in the Q and two sentences in the sole answer: "Identify deforestation and the rate at which it's happening using computer vision and help in fighting back based on how critical the rate is. The World Resources Institute had entered into a partnership with Orbital Insight on this."
If you look at the four bullet items below, you will find that this question asks a very specific thing about the relationship between climate and emissions.  Neither that question nor that answer overlaps with the content of this question in any meaningful way.  For instance, it is well known that CO2 is NOT causing deforestation.  The additional carbon dioxide in the atmosphere causes faster regrowth.  This is because plants need CO2 to grow.  Hydroponic containers deliberately boost it to improve growth rates.  Plants manufacture their own oxygen from the CO2 via chlorophyll.
If you recall from fifth grade biology, that's why they are plants.

Now Back to the Question
Several climate models have been proposed and used to model the relationship between human carbon emissions, added to the natural carbon emissions of life forms on earth, and features of climate that could damage the biosphere.
Population growth and industrialization have many impacts on the biosphere, including loss of terrain and pollution. Negative oceanic effects, including unpredictable changes in plankton and cyanobacteria are under study.  Carbon emissions from combustion has received attention in recent decades just as sulfur emissions were central to concerns a century or more ago.
Predicting weather and climate is certainly difficult because it is complex and chaotic, as typical inaccuracies in forecasts clearly demonstrate, but that is looking forward.  Looking backward, analyses of data already collected have shown a high probability that ocean and surface temperature rises followed increases in industrial and transportation related combustion of fuels.
How might AI be used to produce some of the key models humans need to protect the biosphere from severe damage.

A more reliable analysis of what has already occurred, since there is some legitimacy to the differing views as to how gross the effect of carbon emissions has been on extinctions of species in the biosphere and on arctic and antarctic melting
A better understanding as to whether the climate of the biosphere behaves as a buffer of climate, always tending to re-balance after a volcanic eruption, meteor stroke, or other event, or whether the runaway scenario described by some climatologist, where there is a point of no return, is realistic
A better model to use in trying out scenarios so that solutions can be applied in the order that makes sense from both environmental and economic perspectives
Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated

Can pattern recognition, feature extraction, the learned functionality of deep networks, or generative techniques be used to accomplish these things?  Can rules of climate be learned?  Are there discrete or graph based tools that should be used?

Can AI provide a more reliable analysis of the gross effects of carbon emissions on extinctions of species ice-cap melting, and other effects?
Yes. The work of Judea Pearl and others over the last 20 years began out of a desire to address uncertainty within AI. Eventually, this led Pearl to become fascinated by the need to quantifiably determine when one event has caused another, the problem at the root of "correlation is not causation". He substantively succeeded with the combination of causal modeling and the do-calculus. The do-calculus allows you to formulate queries of the form "To what degree did X cause Y?", and to automatically determine what measurements are needed to determine the answer in a statistically meaningful way. The algorithms used under the hood are closely related to those used in AI and robotics systems to reason about uncertainty (i.e. Bayesian Inference). Causal modeling is still relatively new, and is not yet as widely used as it could be. An open problem is how to specify what the model of the world looks like from data (rather than just reasoning over a model that is given). If this problem is solved, we could see major improvements in the ability to provide analyses like those you ask about.
Can AI provide a better understanding about whether the runaway scenario described by some climatologists is realistic?
Probably not. Predicting future events without any observable precedents is not something that anyone can be sure about. The questions raised in those kinds of scenarios aren't about calculation, but about modeling. For example, if you believe that there is a large amount of methane trapped in the arctic permafrost, and you believe that temperatures above a certain range release this much faster than in the past, and you believe that methane warms the climate rapidly, then you would tend to believe that the runaway scenarios are plausible. AI can't tell us whether the methane is there (we have to go measure it, though maybe AI can make the measurement more accurate), and can't tell us how temperature will affect the release rate (again, we have models of this, but they rely on different assumptions, and we have to go measure to find out which are right).
Can AI provide better simulations of the impacts of interventions on both climate and the economy, to inform decision making?
Maybe. Agent-based modeling can help with this to some degree, and is arguably part of AI. In fact, it already has been to some degree, by Beeger & Troost in J. Agriculture Economics, in 2014 and again in 2017, though interest in this kind of modeling looks like a pretty new development in this area. Although ABM can give us reasonable models and help simulate the impact of interventions, ultimately they are just one modeling tool among many. Their potency may improve if more realistic agent models are used, but it is not clear that AI is going to provide advances in this area in the near future.
Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated
Probably not. Although AI techniques have made some kinds of economic planning problems a lot easier, the main barrier to the effects you describe is a social/political one: countries are sovereign, and the world operates as a de facto anarchy (i.e. the UN is impotent). Your AI model can tell, say, India not to industrialize, but Indians want to enjoy the same kind of lifestyle improvements that Americans do, and would rather enjoy them sooner than later. India would collectively rather that Americans put an enormous tax on their carbon emissions, drive less, eat much, much less beef, and stop flying everywhere, than that Indians continue living on the equivalent of $7,000 each per year. In contrast, Americans would rather that Indians just wait a few decades while the developed world decarbonizes, and only industrialize once we have enough solar panels to replace all our current needs within current industrial economies.
Basically this is a resource allocation problem within an anarchy: we can only burn X carbon within Y time, everyone wants to burn some, and the only way to enforce contracts is with the threat of massive violence (or massive economic sanctions, which in turn, are imposed through the threat of violence against other actors if they trade with the sanctioned country). AI can help us answer questions like "How much should the USA pay India in exchange for India not industrializing this year?", see, e.g. work in Auction Theory, but AI can't actually make nations do those things if they can't reach a diplomatic compromise.


text:
How can I build an AI with NLP that reads and understands documents?
I have to read a lot of papers, and I thought that I can use an A.I. to read them and summarize them. Maybe find one that can understand what the papers are talking about it seems a lot to ask.
I think I can use natural language processing. Is it the right choice?
I'm sorry, but I'm new in A.I. and I don't know much about it.

I don’t hunk there exists such an AI that is open. Yet. Might be wrong though, so just hang in there!

Besides searching to something done, I'm also asking, how can I do it? Maybe, I can use natural language processing.

I am also interested to know if there is any online service or API that does this. For example, it scans documents and answers the questions as per the facts it learnt from the docs. From my understanding, Google Cloud NL can analyse documents to extract information but I don't know if it can also answer your questions. With Google you can have docs in Q/A and it parses intents from the docs but again not sure if it is answering your questions? Do we need another service to feed the analysed docs to chat with AI?

This field of AI is called 'automatic text summarisation'.  An Internet search of that term will get you started.

When automating tasks involving text, NLP techniques are definitely the way to go! Let me be frank, though, starting from scratch and "read and understand" will be harder with today's tools than directly reading the papers.
Some typical required skills are below, and the bar height will depend on your mileage:

Knowledge of NLP (e.g. terminology like embeddings, bag of words, even perhaps tf-idf).
Knowledge of NLP libraries and implementation frameworks (e.g. Stanford's or TensorFlow).
Knowledge of a programming language (e.g. Python or Java).

In short, the goal you set will probably require many more papers than expected. But if you succeed only the automated reading / summary part, you have a bright future ahead (let aside understanding, a way more contentious problem).
Just for the sake of putting an example, companies like Iris.ai and others try to solve related problems---great potential, but definitely challenging.

I do not usually understand the papers I read, so I'm looking for some help (AI). Thanks for your answer.

Sorry just for my own understanding, are you saying with the frameworks like Stanford's and programming language we can write an application that reads and analyses documents and answers questions or this is still a challenging issue? How does these chat bots like Oracle's do this?

@VansFannel Understanding the papers takes time and work. AI is a broad topic, even if we are just talking about NLP. Perhaps you can select something that motivates you, and start from there. That should lead you to more understanding toward next steps, and more questions to ask here. Narrow question are easier to get attention and answers.

@xbmono The existing tools and techniques have qualities and limitations, yet they can be used for many problems that have not been tried yet. For example, one of my previous employers used the Stanford NLP library to perform automated comment moderation (similar to spam filtering). Now the tools have limitations, and there are a lot of challenges. NLP libraries usually break facing indirect speech, some forms of humour, etc. And NLP libraries are usually ignorant of context, e.g. in a discussion thread. A lot possible, and a lot yet to do!

@EricPlaton Thanks, but it was only a joke. Thanks again.

I have received this on email a few minutes ago, coincidence?: https://software.seek.intel.com/DLNaturalLanguageWebinar_Reg


text:
What functionality, does control look like in autonomous vehicles levels 4 and 5?
We are doing a research design project on autonomous vehicles and have some questions on AV Levels 4/5; specifically on the roles, impacts and consequences of AV on society, government, users and other stakeholders.
We're currently stuck on this main question:
Q: What functionally, does control look like in AV levels 4 and 5?
For example, is the whole purpose of a level 4/5 that a user has no input into the control?
Could a driver in AV (level 5) stop in an emergency, or say they want to "take corners harder, speed up, slow down"?
Could I choose to change the equi-distance between my AV and the others around me because I like space?
We're wondering about what functionally, does AV level 4/5 offer a user; and what it looks like?

Context:
Our remit is within the world of design (design thinking), not specifically technology, or expert system functionality. We're looking at the issue from a design perspective; who does it impact, who are the stakeholders, what are the consequences and impacts. What role does a driver have an in level 5? Could an auto-manufacturer want to give drivers control in level 5? How do emergency services act in these situations? What are the touchpoints to society and whom does it impact and what does it say about the design of AV for the future of society.

Automation Levels
Most cars have some Level 1 automation, such as cruise control and various skid/flip probability reduction systems. Most high volume passenger vehicles have higher levels. Some military and private air, land, and sea equipment are already at Level 5.
Level 4 requires that driving be automated during normal driving conditions, with manual override.  However, to my knowledge, no one has published a mathematically terse and comprehensive distinction between normal and abnormal to aid in testing Levels 4 vehicles, so testing to a standard is probably not yet possible.
For legal and political reasons, Level 5 is essentially a statistical criteria. For fully automatic to be viable as a market product for general public use, the safety data for passengers and pedestrians must indicate at least the level of safety for manually driven vehicles. Although this will likely suffice from a law and public relations purpose, it is inadequate as a quality standard for automation engineering and testing.  The ambiguities are numerous.

Statistical criteria required to pass the test (i.e. sample size, duration, randomization, and double or single blindness)
Mathematically terse and comprehensive scenarios for the test
Allowable proportion of level 2, 3, or 4 vehicles in the control group.
Probably others

There will be no vehicle driver in what they are calling Level 5 — only passengers.  The idea is to give no power to the occupants of the AV other than destination or change in destination.
This has been the safety norm in other sub-sectors of transportation for a century.  Passengers cannot talk to the pilot of a jet or the engineer of a train.  In the majority of cases, safety is compromised whenever a person that has not undergone the discipline of intense safety training has control over any aspect of the vehicle's operation.
That is the primary impetus behind AV from the forward thinkers in government and academia.
Specificity and Insight
It is of paramount importance that researchers define system criteria more specifically and scientifically.  Systems architecture, software engineering, safety evaluation, and quality control policies and procedures of the automated system driving the AV requires such.
With a billion plus lives at stake, the design should progress with the diligence and care as if designing a human-occupied drone aircraft or a civilian Mars lander from the ground up, even if your first phase is to only achieve what is being called Level 4 for basic passenger cars.
Target Reliability and Safety
Humans eat food, have emotional conversations, text each other, ingest mind altering substances, and fall asleep while driving.  Judging the safety statistics of an AV by comparing with those of humans driving may sound practical, but it is absurd.  It will become clear just how absurd the popularized Level 5 criteria is as the parameters of design are enumerated.
Design should instead minimize the possibility of any accident ever.  The goal should be zero mistakes, both at point of sale (the dealership) and at later points in the product life-cycle as the AV learns.
Defining a Mistake
A mistake should be defined as follows:

Any less than optimal state indicated by the correction signal used to direct reinforced learning in any of the system's re-entrant or coincident training mechanisms

The adaptive (i.e. machine learning) portion of the system must permit re-entrant or coincident training (reinforced learning) because there is no possible way to predict the common routes of the buyer at the vehicle's point of sale.
To comprehend the complexity of the problem space for AVs and begin to simplify it, consider the dimension of conditions, controls, and priorities (embodied in feedback signals) related to driving on roads for any vehicles that use roads.
Control Channels

Starters (there are two in the case of most hybrids)
Engine stop
Breaking controls (there are two in the case of regenerative breaking)
Steering shaft or hydraulic control position
Break control position (three or four depending on emergency/parking break design)
Transmission planetary gear clutch state, traditional automatic transmission control state, or clutch and traditional manual transmission control state
External lighting switch positions (Widely variable, but at least six for headlight, high beam, break light, left signal, right signal, and tail light)
The content of any messages broadcasted, multicasted, or sent specifically to any other vehicles with compatible reception (if birds do it, so can electronic systems) either via light, sound, or RF (this will require the development of layers of inter-vehicle communications protocols)
Horn
Probably others

Data Acquisition Channels

Wheel positions (there may be 2 or 4 positions to read with an encoder since one cannot assume perfect alignment)
Break torques (4 of them, which can be read by 16 redundant strain gauges)
Break metal temperatures (4)
Torques and temperatures for any independent emergency breaks
Accelerometers (two devices x three dimensions per device to detect acceleration/deceleration, centripetal force, and, with some math, tire skid velocity for all four tires in two dimensions)
Tachometers (one before the transmission and one on each wheel)
Engine and coolant temperature detectors
Cameras (must be high resolution to recognize animals, humans, shopping carts, curbs, road signs, train signals, speed bumps/humps, and hazards, which can be IR or visible and the more angles covered the better)
Wind turbulence resistant external microphones to detect horns and sirens (at least four to detect likely orientation of audio source)
Suspension strain gauges (to detect vertical road force on each tire)
The content of any incoming messages from compatible systems
Battery voltages and currents (two batteries for regenerative breaking or hybrid startup and motive assist, and possibly several currents)
Fluid levels, pressures, viscosity, and transparency (fuel, oil, steering, transmission, break hydraulic, and possibly others)
Probably others

A system can be operational and possibly reach what most would consider Level 5 with fewer channels of acquisition and control than above, but it would be poor technology planning to start designing systems with unnecessary limitations.  Such limitations will also very likely increase the cost of engineering and the effectiveness of training while saving nothing.
Why Vehicles Have Very Little Instrumentation Today
A human cannot make use of all of all the information above.  Nor could a human control all the channels listed above without a high frequency of mistakes.  A properly designed electromechanical learning system can.
It would be lazy systems architecture not to capitalize on the positive impact the additional instrumentation would have on safety, total cost of ownership, and other quality criteria for the AV buyer who can afford the extra sensors and computing power.  Furthermore, after a few years of manufacturing for a mass market, the cost of the additional components will become small in comparison with the cost of metal and plastic.
Operational Criteria and the Formalization of a Mistake
The problem space contains at least nineteen dependent variables (output channels) and forty-six independent variables (input channels).  Some are binary, some are floating point, some are streamed data, some are streamed audio, and some are streamed video.
Together they form a space in sixty-five dimensions.  That is what must be optimized according to some predetermined and possibly re-programmable formalization of what is optimal.
Let's consider this idea of optimum safety, thrift, and comfort as quality control criteria.  Real time quality control should follow TQM ideals, continuously ensuring quality in multiple dimensions and at multiple points in the total system.

Maximal distance from other vehicles
Maximal distance from stationary objects (bridge abutments, buildings)
Maximal distance from pedestrians on foot, bike, wheelchair, ...
Maximal distance from edges of pavement
Minimal lane switching
Minimal loads and torques on wheels
Minimal fuel consumption
Within operation parameters for tires, breaks, engine RPM, and dozens of other parts and subsystems
Shortest distance for route to destination
Shortest time distribution mean for route to destination
Safest route to destination
Least stops on route to destination
Probably others

Most of this must be balanced, so optimization criteria must be aggregated.  Such aggregation must go beyond the simplicity of a loss function.  Summing squares will not work at all, so give up aggregating in such a simplistic way.  A multivariate extension of the Pythagorean Theorem is find for calculating distance in linear space, but driving cars is very non-linear.  This kind of robotics system will require more thought in the formulation of balances, priorities, and the concept of emergency.
Further expanding on the above definition of a mistake, any real time control that does not optimize the sixty-five-plus dimensional surface is faulty.  Now what is optimal must be defined.  Consider the following quality control criteria, roughly in correct order.

Pedestrian safety
Passenger safety
Mechanical system integrity
Fuel conservation
Vehicle external coating integrity
Mechanical system wear
Passenger comfort
Time conservation in reaching destinations
Probably others

Applying Optimization in This Context
Aggregation of the incoming signals acquired are not only based on multiple criteria but the prioritization is not always constant, implying the need for a vector of correction signals rather than a single floating point number.
A single dimension of signaling to feed the disparity between ideal operation and the current system behavior (called a loss function in gradient descent) will not suffice.  There will, out of necessity, be a need for training and reinforcement with a complexity that involves the idea of preemption.  Evolution has declared preemption the design of choice for nervous systems with brains.
For instance, the pedestrian safety feedback signaling must always preempt the fuel conservation feedback no matter how much fuel would be consumed in staying clear from pedestrians, in planning a route where pedestrian density is lower, steering the vehicle clear of people, choosing speed, and applying breaking.
All biological systems have these preemption mechanisms — even bacteria.  A turtle doesn't balance the transportation aspect with safety when it retracts under its shell.  The behavioral interest in the turtle's destination is shelved (stored and temporarily forgotten) until the preemptive system that detected danger indicates the danger has passed.
Humans Should be Passengers on the Streets of the Future
The reason humans are generally driving in a continuous state of mistaken control is because the priorities that maximize the parameters of transportation for society (above) are inconsistently followed by humans.  Birds fly smarter than humans drive.  The priorities of an emotional being that wants to get somewhere fast and while talking, texting, eating, and possibly getting high will often be mistaken.
Future people may look back at the period between the advent of Model-T market penetration and the complete transition to AV as a period of strange inequality.  Stepping back, the worldwide interest in domestic security, airline safety, train and subway safety, and building code contrasts strongly against the cultural insistence of every household have instant access to drive anywhere, any time, and in any mental condition.

this question is a softie, but could probably benefit from your rigorous approach and highly organized thinking: https://ai.stackexchange.com/q/6643/1671 :)

The idea of what role of design thinking plays in AI is an interesting concept we should certainly consider

+1. I would include stochastic control theory and cybernetics topics, fuzzy versions of OODA loops, the use of mathematical theories of evidence/beliefs due to epistemic uncertainties (Dempster-Shafer theory is already being used in AV research), and the like.

I like the sentence, "Birds fly smarter than humans drive," best.

Control should look like low numbers in highway and city accident report statistics.
There will be no drivers in Level 5 AV.  In fact, there may be no driver's position in the vehicle as with train passenger cars and dining cars.  This is quite different than Level 4.  In fact, more levels will probably emerge because of the huge jump from 4 to 5 in the current understanding.
The AV will stop in an emergency because the AV technology to do so will have proven its ability to determine and react to an emergency condition more quickly and reliably than a human driver.  The passengers will still be under control, just not THEIR control, except for their determining the destination address.  Control should be a communal function, since the danger when control fails is communal.
This is the benefit of Level 5 over Level 4.  Highway pileups and many other accident scenarios are caused by human inability to react to risks.  Pileups are usually caused by human perception of a normal driving condition as an emergency and jamming on the breaks unnecessarily.  Most drivers will not steer into a skid even though they've heard to do so.  Rather than controlling the risk with skill, most people react in a way that reduces their tentative control of the vehicle when a skid begins.
Regarding the passengers liking space between cars, the AV will prefer space too because proximity creates risk, so that question is moot.
The primary impact of AV on society will be the gradual demolition of cultural values about engine power, which is reaching its time anyway because of the reduction of the burden of transportation on energy consumption that removing adrenaline and testosterone from the roads and highways will bring forth.
Government is pro AV simply because many humans like to "drive through" when traveling far, drive when on prescription medications that warn against operating heavy machinery (which cars are), and text in heavy traffic and in school zones.
The automotive, aeronautical, and transportation industry is not a primary driver of the AV technology.  Their profit is maximized by occasional (but not frequent) accidents, stress on engine and body components, sales pitches about safety and power unsubstantiated with actual data, and marketing that equates freedom with control.  With rare accidents, minimized wear, and accountability created by deeper scrutiny there may be significant profit lost.
Insurers will gain.  Ticket and accident attorneys will lose.  Parents of children in school will win.  Body shops will lose.  The electronics industry will win.  Tire sales will drop.  Government will be able to pocket more tax revenue.  Gear heads may eventually be thought of like KKK members are in today's postmodernism.  Custom wheel manufacturers will win.  Towing services will lose.  Bicyclers will win.  Petroleum extractors will lose.
Just as the personal automobile changed the horse industry and the Internet changed publishing, AVs will lead to adjustments throughout government and the economy.

@zardon, what was most assistive in my answer? Knowing is helpful for future answers of questions like yours.

I found it very useful in helping to frame further research as part of my design-led innovation project on autonomous vehicles.  Some of the insights have been useful in guiding our small team in understanding the interplay between users and autonomy, and how future journey experiences might be framed.


text:
In speech recognition, what kind of signal is used?
Speech is a major primary mechanism of communication between humans. With respect to artificial intelligence, which signal is used to identify the sequence of words in speech?

The signal path in speech recognition as one travels further from the basic representations of sound depends increasingly on the role the recognition plays. Consider these roles.

Translation to another language
Comprehension as in listening to a lecture
Comprehension as in a bunch of people out for food, drink, and laughs
Transcription into text

There are similar differences in other natural language facilities of reading, writing, typing, speaking, interpreting body language, and expressing with body language. With auditory language, the stages of sensory processing proceed in a particular order of signal types, each representing the dynamics of language at successively abstract levels, essentially reversing the process of speaking.

Physical dynamics of air
Frequency dependent vibrations of cochlear hair
Spectra, transients
Pitch, tone/timbre, impacts, loudness
Phonetic elements
Linguistic elements distinct from other noise
Beginnings of attention based functionality

The first six elements are common between all language oriented hearing. The seventh is where the above four roles diverge. In translation and comprehension, comprehension of linguistic nuance is required. Colloquialisms, euphemisms, culturally dependent references and analogies and other higher level linguistic and social constructs is required for full proficiency. For transcription, comprehension is required only to improve transcription reliability and to distinguish between phononyms like to, too, and two.
Notice that the sixth level is not words or sentences. Words and sentences are later developments in human language and occur later in an individual's childhood. Our educational systems are largely word oriented, but people do not talk or listen in words. There is no signal for "way" in the brain when someone hears, "No way!" which, in the current semantic state of U.S. English is a single linguistic element that means, "What you just said is very surprising to me." Two written words represent this one linguistic element.
Conversely, there are two distinct signal representations for, "wanted", specifically, "want," and "-ed." The second of the two is reused for, "planted." The -ed ending is not relearned for every verb.
Consider these lines out of a dialog.

Jenna: Cats run.
Chelsea: That's ridiculous.

In transcription, the voices must be distinguished and the words must be written, including the contraction of, "That is," whereas the plural of cats could be possessive, so the computer attempting this transcription must somehow chose the plural over the possessive in this case. Out of context, the pronoun, "That," does not refer to the cat or a person named Cat but the idea of the running. Consider this conversation between people out for food, drink, and laughs.

Catherine: He's so sexy.
Jenna: Cat, you always pine over the really toxic guys.
Catherine: No I don't.
Jenna: What about that cute (and well dressed) guy that wanted to meet you for lunch last week?
Chelsea: She gave him her number.
Julia: Yea, with the last two digits transposed.
Chelsea: All the Jennas I've ever met are first to run when a nice guy comes around.

Catherine nods. (beat) Patrick comes to the table.

Patrick: Hey, you're Catherine, from Central High, right? Didn't we have study hall together in junior year?

Catherine puts some cash on the table for the drinks.

Catherine: I didn't go to Central. I went to East Barnard. (beat) Girls, I'll see you later. Gotta go so I can get up for work tomorrow.

Catherine exits. Patrick wanders off.

Jenna: Cats run.
Chelsea: That's ridiculous.
Julia: What, you think she left because of work?
Chelsea: Why not?
Jenna: She works at a restaurant closed on Mondays.

This dialog is a series of signals relying heavily on understandings that are not in the language. There is no formal grammar that could faithfully guide parsing by words. The signals after all seven levels of processing still fall far short of complete descriptions of the scenarios or the ideas of the individuals involved. Yet a series of themes and triggers to existing information shared in common between those at the table communicate volumes of information.
Level six from the layers listed above is important here because people are talking and possibly loudly at other tables. Level seven of the listening is entirely different at this girl's night out and may not share any significant functionality with the listening process when Jenna, Chelsea, Catherine, and Julia are in class taking notes.
For computers to partake in a conversation like this one and actually socialize, many more functions beyond the seven layers of abstraction above must be assembled and must learn all the nuance and information brought in as cultural reference. For example, there are several cultural ideas involved.

Some people tend to pick those that are not particularly healthy for mates.
Getting a phone number is a first step toward talking in private.
Cat is Catherine.
The conversation is occurring on Sunday, thus the next morning is Monday, the day when the restaurant at which Cat works is closed.
A man producing some connection from the past is a segue into further conversation based on some relational interest.

Not sure I understand what you mean by signal. However, incoming audio must be cleaned and processed for a machine to understand. Some people speak faster and others slower. First thing is to sample in the incoming audio. In other words use analog electic signals and store them as binary values at a set rate of sampling frequency.
See Nyquist Shannon theorem.
Now that it is sampled we need to clean it up and extract only the voice data. Effectively the sample is a set of values (frequencies) recorded over time. Or a signal :).
We can cut up this signal into many smaller components of a few milliseconds say 10ms. Each of the millisecond samples is further processed
A Fourier transform is generally used to break a signal into components, or frequencies that make up the signal. This is basic signal processing and we have not even entered machine learning yet.
Now we have a fingerprint of each 10ms sample these samples are then fed into a neural network. Samples are broken between frequency drops to create blocks of sound which may or may not be a word. These blocks are passed to the neural network.
A traditional feed forward neural network will not work as it forgets the data. So a recurrent neural network is used.
To train the neural network known words are tagged and encoded as a number. The neural network will output a number and the difference between this number and the expected number is the error. The error is then used to update the weights
Once we have the words, those can be further processed by other networks that specialise in natural language understanding


text:
Will we be able to build an artificial intelligence that feels empathy?
Nowadays, robots or artificial agents often only perform the specific task they have been programmed to do.
Will we be able to build an artificial intelligence that feels empathy, that understands the emotions and feelings of humans, and, based on that, act accordingly?

A very related question (if not a duplicate): Could an AI feel emotions?.

This could maybe be considered an extreme version of the value alignment problem.

Let us describe a very simple system that does something we could label as empathyc.
A chatbot answers "I am sorry to hear that. What happened?" when we type "I feel bad", and it replies "I am glad to hear that. Fancy some music?" when we type "I feel good".
Somehow, it perceives a human emotion, and acts accordingly.
Planes fly but they do not fly as birds. Similarly, we can expect artificial empathy from an AI, not necessarily natural empathy as we feel it.
Here is a related paper, Shared Laughter Generation for Empathetic Spoken Language
And here is an actual example:

The system correctly maps tears to sadness and outputs a proper answer, while also explaining its non-sentience. Not bad!

Comments are not for extended discussion; this conversation has been moved to chat.

It seems to me that empathy is based on understanding the experience of another entity:

originally Psychology. The ability to understand and appreciate another person's feelings, experience, etc. SOURCE: OED

Using this definition, the AI would have to understand human experience.  (There may be a "Chinese Room" issue in terms of whether one considers the algorithm to truly "understand".  But, if it can classify the input sufficiently to produce an appropriate response, that can constitute understanding.)
The underlying problem is that the algorithm likely doesn't "feel" in the same way humans experience emotions, in that the human experience is colored by chemical response.  So while the algorithm might be able to demonstrate sufficient "understanding" of a human's experience, and act in an empathetic manner, the degree of understanding may always be limited.

Would you agree that the example I just provided classifies the input sufficiently and produces an appropriately aligned response?

Empathy is about living and values, just as someone can be a good person, or a bad person, based on their education, culture, past experience and values, just as I who recently became a father became much more empathic about my parents.
If it's really a semi-programmed artificial intelligence that captures around you the experience with your programmers, with people, with the environment, etc., and somehow is set to learn from it, I believe you can develop some kind of “ empathy ”, but as Jaume said, it won't be“ natural ”
We have to remember that real artificial intelligence is the one that absorbs and learns, not a set of IF's and ELSE's.

I don't see a contradiction between learning and using a set of IF-ELSE to "classify the input properly" (from DukeZhou's answer), especially if conditions can change as more information is available. Were you thinking about predefined, static conditions?

I also see no problem, after all for presets would have to have some IF-ELSE as input, just as you said.
But I was thinking of setting empathy as something "fixed" to having a predefined "forced" empathy


text:
Is it ethical to create a chatbot to answer questions on Stack Overflow?
I was wondering if I should do this, because 2 out of 5 questions on Stack Overflow don't ever get answered, or if they do get (an) answer (s), most of the time they're not helpful.
So I was thinking -- why not create a chat bot to answer Stack Overflow's questions & provide necessary information to the general public?
I mean why not? I've always been interested in AI, and all I'd need to do is create a basic logic database and a context system, pack an artificial personality with (partial) human instincts, and bam I'm done.
But then again, would it be ethical?

"all I'd need to do". You seem to me to be rather optimistic about the ability of known AI techniques to emulate even the poorest of answers. You've also been extremely sketchy on detail. In any event, the burden of proof is on you.

This is a really interesting question! Ethics, though, is very broad. If you could narrow down the ethical concern you have, I think we could give more helpful answers.

Yes, it is possible, and has actually been done in the past.
The University of Antwerp created a bot to answer questions (this is the technical report). It focused on the git tag only though (even though it did answer one mysql question).
Its accuracy was pretty good, and the bots in the tests did earn some reputation. So I assume it is possible.
But do note that the last bot in the tests revealed that it was a bot, and thus got banned. So if you reveal that the account you are running the bot on is a bot, there is a high chance that it will get banned.

Technically, creating a non-human account on Stack Exchange would violate the Terms of Service. You would have to find some way to keep it from getting banned.
That having been said, creating, and learning are always good things. It would be a somewhat complex task, but I'm sure you would learn a lot from it. There are plenty of bots out that use the questions and answers from Stack Exchange already, but none directly on the site.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What artificial intelligence strategies are useful for summarization?
If I have a paragraph I want to summarize, for example:

Ponzo and Fila went to the mall during the day. They walked for a long while, stopping at shops. They went to many shops. At first, they didn't buy anything. After going to a number of shops, they eventually bought a shirt, and a pair of pants.

Better summarized as:

They shopped at the mall today and bought some clothes.

What is the best AI strategy to automate this process, if there is one? If there isn't, is it because it would be dependent on first having an external information resource that would inform any algorithm? Or is it because the problem is inherently contextual?

The following post has a bit of math, which I hope helps to explain the problem better. Unfortunately it seems, this SE site does not support LaTex:
Document summarization is very much an open problem in AI research. One way this task is currently handled is called "extractive summarization". The basic strategy is as follows: Split this document into sentences and we will present as a summary a subset of sentences which together cover all the important details in the post. Assign sentence $i$, $1 \leq i \leq n$, a variable $z_i \in \{ 0, 1 \}$, where $z_i = 1$ indicates the sentence was selected and $z_i = 0$ means the sentence was left out. Then, $z_i z_j = 1$ if and only if both sentences were chosen. We will also define the importance of each sentence $w_i$ for the sentence $i$ and interaction terms $w_{i,j}$ between sentences $i$ and $j$.
Let $x_i$ be the feature vectors for sentence $i$. $w_i = w(x_i)$ captures how important it is to include this sentence (or the topics covered by it) while $w_{i,j} = w(x_i,x_j)$ indicates the amount of overlap between sentences in our summary. Finally we put all this in a minimization problem:
\begin{aligned}
\underset{z_i}{\text{maximize }} & \sum_{i} w_i z_i - w_{i,j} z_i z_j \\
\text{s.t. } & z_i = 0 \text{ or } 1
\end{aligned}
This tries to maximize the total weight of the sentences covered and tries to minimize the amount of overlap. This is an integer programming problem similar to finding the lowest weight independent set in a graph and many techniques exist to solve such problems.
This design, in my opinion, captures the fundamental problems in text summarization and can be extended in many ways. We will discuss those in a bit, but first, we need to completely specify the features $w$. $w_i = w(x_i)$ could be a function only of the sentence $i$, but it could also depends on the place of the sentence in the document or its context (Is the sentence at the beginning of a paragraph? Does it share common words with the title? What is its length? Does it mention any proper nouns? etc)
$w_{i,j} = w(x_i,x_j)$ is a similarity measure. It measures how many repetitions there will be if we include both words in the sentence. It can be defined by looking at common words between sentences. We can also extract topics or concepts from each sentence and see how many are common among them, and use language features like pronouns to see if one sentence expands on another.
To improve the design, first, we could do keyphrase extraction, i.e. identify key phrases in the text and choose to define the above problem in terms of those instead of trying to pick sentences. That is a similar problem to what Google does to summarize news articles in their search results, but I am not aware of the details of their approach. We could also break the sentences up further into concepts and try to establish the semantic meaning of the sentences ( Ponzo and Fila are people P1 and P2, a mall is a place P, P1 and P2 went to the place P at time T (day). Mode of transport walking.... and so on). To do this, we would need to use a semantic ontology or other common-sense knowledge databases. However, all the parts of this last semantic classification problem are open and I have not seen anyone make satisfactory progress on it yet.
We could also tweak the loss function above so that instead of the setting the tradeoff between the sentence importance $w_i$ and the diversity score $w_{i,j}$ by hand, we could learn it from data. One way to do this is to use Conditional Random Fields to model the data, but many others surely exist.
I hope this answer explained the basic problems that need to be solved to make progress towards good summarization systems. This is an active field of research and you will find the most recent papers via Google Scholar, but first read the Wikipedia page to learn the relevant terms

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What was the first machine that was able to carry on a conversation?
What was the first AI that was able to carry on a conversation, with real responses, such as in the famous 'I am not a robot. I am a unicorn' case?
A 'real response' constitutes a sort-of personalized answer to a specific input by a user.

The well-known 'Eliza' program (Weizenbaum, ~1964) would appear to be the first.
Eliza was designed to model the emotionally-neutral response of a psychotherapist and this masks some of the weaknesses of its limited underlying pattern-matching mechanisms.

In 1986, the first PC therapist program was written by Joseph Weintraub. This program won the first Loebner Prize in 1991, and then again in 1992, 1993 and 1995.
In 1981 or 1982, Jabberwacky was founded, which is the foundation of the current Cleverbot. Jabberwacky  appeared on the internet in 1997, reaching the third place for the Loebner Prize in 2003, the second place in 2004, and won in 2005 and 2006. In 2008, Cleverbot was launched as an variant of Jabberwacky.
I'm not sure these are really the earliest, but that also depends on what you want earliest (programming started, first conversation,  first decent conversation, etc.). Also, it depends on what you call a "real response".


text:
Training network to detect spelling mistakes
I would like to know what kind of dataset I need (to prepare) for training the network to recognize the spelling mistakes in individual words for English text.
Given the large database of words, having correct one for each incorrect. What kind of input is more efficient for that tasks? Is it using one input per each letter, syllable, whole word or I should use different pattern syllable?
Then the input should be incorrect word, output correct, and if the word doesn't need correction, then both input and output should be the same. Is that the right approach?

I don't think the english language with it's long and complicated history and sometimes convoluted etymology (affecting spelling rules) is a good target for that,

Have you considered using decision trees or fuzzy logic?

I haven't decided yet which one would be more suitable, but I'm open to suggestions. First I'd like to decide on pattern boundaries.

I'd personally be more inclined to try longstanding deterministic methods such as Damerau (for typing errors) or Soundex (for homonyms arising from transcribed speech). At the very least, I'd use those as a baseline for any more 'AI-based' approach.

I would also look at Minimum Edit Distances such as the Levenshtein distance.
You could use a dynamic programming technique such as the Viterbi Algorithm.
If you don't have a dictionary to work against, you may want to train with a Markov Chain model using a known "good" text. The Viterbi Algorithm could be used again to solve the model for the text being considered.


text:
Could we code in Natural language once we pass the Turing test?
Technically speaking, could we code in natural language once we pass the Turing test? Would passing the Turing test at least simplify programming languages' syntax?

Looking for a quantitative answer..

Yes, we can program in natural language.
Specifications in natural language are normally neither precise nor complete. But when using a large language model like GPT-4, that is not a problem:
The model makes assumptions for parts you did not specify.
Sometimes that is good enough, but in general, you need to refine your specification until it is complete and precise enough to fit the requirements you try to express in natural language.
Note that you do not need to fully understand your actual requirements from the start, you can incrementally refine them too.
To illustrate it, I will give a example using GPT-4:
(All prompts are shown, there is no further prompt engineering, it is zero-shot. Prompt: is what the programmer writes, GPT-4:, Output: is the output from the current version of the program executed as python.)

Prompt:
Write a python function that can concatenate some strings, and print the result with 10 characters per line.

GPT-4:
def concat_and_print(strings):
    concatenated = "".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
strings = ["hello", "world", "this", "is", "a", "test"]
concat_and_print(strings)

Output:
helloworld
thisisates
t

Prompt:
I want to give the strings as args.

GPT-4:
def concat_and_print(*strings):
    concatenated = "".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
concat_and_print("hello", "world", "this", "is", "a", "test")

Output:
helloworld
thisisates
t

Prompt:
add spaces

GPT-4:
def concat_and_print(*strings):
    concatenated = " ".join(strings)
    for i in range(0, len(concatenated), 10):
        print(concatenated[i:i+10])

# Example usage
concat_and_print("hello", "world", "this", "is", "a", "test")

Output:
hello worl
d this is
a test

Note how the specification is not only incomplete, but also imprecise. Much less exact than natural langue could be. The specification add spaces does not say where the spaces should be added. Not even that they should be added to the output. It is not specified that they should be between the words, but the interpretation of it is what I meant.
(This is a real chat session with GPT-4, not cherry picked or edited.)

Controlled Natural Language (of the second type) can be used for programming, with no need to pass the Turing test.

I think this is a very interesting question.  It's highly hypothetical, so my simple answer will also, necessarily, be hypothetical.

If the AI can pass the Turing Test, it can be assumed it has a command of the human language(s) used in that process

Self-awareness in this context would seem to be irrelevant, as would the question of whether the AI actually "understands" the content, or is merely imitating natural language.

If the AI has a command of the human language(s), it would be a reasonable assumption that it could translate natural language instructions into machine code

It might be best to think of the problem in the simplest terms.  If you asked this hypothetical AI to "draw a red square on the screen", it's hard to see that task as impossible, or even difficult.
Obviously, as the instructions became more complicated, there would be a greater margin for error, and it would be useful to integrate precise, formal terms related to functions, as Randy points out.

@Bobs it depends on what the AI is being asked to do.  It might not necessarily have to code in machine language proper, for instance Python or JS or any computer code would work.  But unless it's using a computer language based on natural language, it would still have to code in code.

@Bobs I suppose you could have the low-level function be autonomic, even where they had the intelligence to translate natural language into machine code, but classical computing will always be based on 1's and 0's, and even quantum computing on qbits, so there's no way to get around the the fact that any instructions are ultimately reduced to a sting, mostly likely in base-2.  On top of that, natural language is inefficient for coding, but even if it were used, it's getting translated into machine language and a binary string.

@Bobs haha.  that would be awesome if someone could find a way to harness it!  Wouldn't hold my breath though

No.  Humans pass the Turing Test but cannot express themselves with enough precision and little enough ambiguity to code well in English (natural language).  No machine will improve on that until it can greatly improve on human-level clarity in expressing their purpose when programming.
Is it possible to program in natural language?  Yes, I think so.  But it would require superhuman intelligence to anticipate all the possible confusions that might arise when a word is not sufficiently precise or accurate to represent an software activity or mechanism.  This probably would require adopting a rigorous convention in choosing a subset of unambiguous words to form a "natural" programming language (a kind of creole).  Coding consistently using only those words would require discipline that exceeds the ability of most humans, methinks.

Follow up: Given its precise syntactic grammar, is Sanskrit the best candidate for natural language coding\ passing the Turing test? If not, which  language is?

I like this answer, but I think the the final sentence needs some clarification.  "Most humans" may be correct, but is ambiguous and hard to substantiate.  It's probably a safe assumption that programmers, technicians, engineers, scientists, etc., would be able to master this "nlp programming language".

The kind of help development environments provide, showing alternatives at the current context, might be a great help to guide humans, in a way that extreme discipline would not be required.


text:
Can Convolutional Neural Networks be applied in domains other than image recognition?
I'm new in this argument, my question is:
Can convolution be applied in other contexts different from image recognition?
Is there a good source to learn from?

Yes, Convolutional Neural Networks(CNNs) can and have been applied to non-image problems. Arguably, any problem in which the location of a feature(s) is relevant can be attempted via CNNs. CNNs works under the assumption that points close to each other in the data share some correlations/relationship whist points further apart don't share as much information. So, theoretically, if you can phrase your problem so that it meets this requirement(s), it can be attempted by a convolutional neural network. Here are few applications of CNNss that are dont involve images:

Speech recognition
Character-level Convolutional Networks for Text Classification
Text classification using CNNs
Natural Language Processing
Time Series Forecasting
Sentence modeling
I've used 1-D CNNs on spectral data. Here are some examples of CNNs applied to spectral data :

Spectral Convolutional Neural Network for music classification
Convolutional neural network based classification for hyperspectral data

CNNs are generally designed for 2D (generally image) data. Hence, other usages are most likely a "hack" to CNN logic.
You can check how to extract features and classify texts with CNN through here.


text:
What AI service can define personality portfolio based on text?
Imagine this theoretical situation:

A group of people are asked to provide a solution for an imaginary
  problem via email. Then an AI service runs through their written
  solution, analyzing the kind of words and sentences they use,
  essentially exploring how they faced the problem. Then the service
  finds a personality portfolio of each person.

Is there such an AI service which can find such personality portfolios by analyzing the text they use describing their solution to a problem? If there is not a current service, how might it be approached?
Regards,
Koppany

https://datascience.stackexchange.com/q/662/23402

My first recommendation would be before you create an AI or ML based solution. Kindly consider using a business Q&A Software such as Questions for Confluence by Atlassian among others. An enterprise multiple choice solution could be a simple and elegant fit for this problem.
However, if one was to design a solution in accordance to your specification the first step to resolve the problem would definitely be one of applying NLP. In my answer I utilize the open source procedure laid out by EASE (Enhanced AI Scoring Engine) by Edx and a combined initiative of MIT and Harvard towards improving automated essay scoring (Parachuri V 2013).
Computers cannot directly understand text the way people can. People naturally break sentences into units of meaning, however when using a computer we will first have to instruct the computer how to do it. This process is called tokenization. Tokenization is the task of chopping a phrase into pieces called tokens at the same time discarding some characters i.e. punctuation.
After we tokenize the essay answers, we then convert the tokens into a matrix (bag of words model also known as vector space model). This model is a very convenient way to represent text in matrix form. Here we disregard grammar and even the word order but the frequency of words is retained.
Now we can proceed to the machine learning part!
Here I will recommend that we either use regression or classification. The difference between these two algorithms is that regression assumes you are using a continuous scale while classification is discrete.
In statistical modeling, regression is a set of statistical processes for estimating the relationships in variables.

A simple linear equation is $y=m*x+b$ , where y is the target value(score), m is a coefficient, and b is a constant.
  In linear regression, we would do something like $y=m{1}*x{1}+m{2}*x{2}+\dots+m{n}*x{n}+b$.

Classification (also sometimes called a decision tree) on the other hand is a data mining method for assigning categories to a collection of data which is useful since we want to classify essays on the basis of their scores.
The next step will be to measure the error in the scores that are generated by the algorithm on the previous step.
In order to measure accuracy, lets use cross-validation. Here we split the training dataset randomly into n parts each segment is a 'fold'. This is called n-fold cross validation. We then iterate from 1 to n predicting the scores of the parts [n] from all the data in parts [!n].
Hope this helps.


text:
Can deep learning be used to help mathematical research?
I am currently learning about deep learning and artificial intelligence and exploring his possibilities, and, as a mathematician at heart, I am inquisitive about how it can be used to solve problems in mathematics.
Seeing how well recurrent neural networks can understand human language, I suppose that they could also be used to follow some simple mathematical statements and maybe even come up with some proofs. I know that computer-assisted proofs are more and more frequent and that some software can now understand simple mathematical language and verify proofs (e.g. Coq). Still, I've never heard of deep learning applied to mathematical research.
Can deep learning be used to help mathematical research? So, I am curious about whether systems like Coq could be combined with deep learning systems to help mathematical research. Are there some exciting results?

See https://ai.stackexchange.com/q/7416/2444, especially, this answer (and also this one).

I think your question isn't an exact duplicate, because your question is slightly more general, but if you think it's a duplicate, I will close it as such.

Yeah we could keep it open since it is slightly more general but the answers to the other question are already very interesting, thanks!

This is difficult but not impossible. Deep Learning is a powerful tool, but it will depend on how the problems are formed. I mean it's not easy to make your model do researches like this, it will depend more on how these mathematical researches can be turned to formal problems.

Look for research published by FAIR (Facebook Artificial Intelligence Research) on application of language (NLP) and deep learning to mathematics. But yes, every different class of research problem is different and hence most likely need to be formulated specially so you could use deep learning. In other words, given enough example mapping between question (X) and expected answer (Y), you are more likely to successfully build a deep learning model.

Paper on Facebook AI research: "Symbolic Mathematics Finally Yields to Neural Networks
After translating some of math’s complicated equations, researchers have created an AI system that they hope will answer even bigger questions.  "
https://www.quantamagazine.org/symbolic-mathematics-finally-yields-to-neural-networks-20200520/?utm_source=Nature+Briefing&utm_campaign=607b58d6d1-briefing-dy-20200522&utm_medium=email&utm_term=0_c9dfd39373-607b58d6d1-44895933

An automated prover based on Transformer architecture found new short proofs that were accepted into the main Metamath library: https://arxiv.org/abs/2009.03393

Mathematical equations are generally expressed in a sequential form known as 'infix notation'.  It is characterised by the placement of operators between operands. To make the order of the operations in the Infix notation unambiguous, a lot of parenthesis are needed. Infix notation is more difficult to parse by computers than prefix notation (e.g. + 2 2) or postfix notation (e.g. 2 2 +).
There is a deep learning approach to symbolic mathematics recommended in the research paper by Guillaume Lample and François Charton. They have found an interesting approach to use deep neural networks for symbolic integration and differentiation equations. This paper proposes a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models.
Deep Learning for Symbolic Mathematics

This approach is essentially representing mathematical problems in prefix notation. First a symbolic syntax tree is constructed that captures the order and values of the operations in the expression. Second, the tree is traversed from top to bottom and from left to right. If the current node is a primitive value (a number), add it to the sequence string. If the current node is a binary operation, add the operations symbol to the sequence string. Then, add the representation of the left child node (could be recursive). Then, add the representation of the right child node. This procedure resulted in the following expression.

We can expect further more advances in this area with the emergence of better symbolic learning models leveraging attention based transformers and other neural symbolic learning models. Recent work by MIT, DeepMind and IBM has shown the power of combining connectionist techniques like deep neural networks with symbolic reasoning. Please find the details in the following article.
The Neuro-Symbolic Concept Learner


text:
How to understand the GCN equation?
I understand GCN does message passing with its neighbours to learn the node embedding.
But I don't understand the following equation.
What "tilda" is referring to equation 1?

Can you please use latex to transcribe the equation? You can use a tool like Mathpix Snipping tool to quickly convert text to mathjax.

$\tilde{A}$ is related to normalized Laplacian matrix that "shows many useful properties" of matrix $A$. Note that:

Since the degree matrix $D$ is diagonal, its reciprocal square root $D^{-{\frac {1}{2}}}$ is just the diagonal matrix whose diagonal entries are the reciprocals of the square roots of the diagonal entries of $D$. If all the edge weights are nonnegative then all the degree values are automatically also nonnegative and so every degree value has a unique positive square root. To avoid the division by zero, vertices with zero degrees are excluded from the process of the normalization.

Hence, if $A$ represents the adjacency matrix, first normalize it by degree matrix and get $\tilde{A}$. Then, times the result into the feature matrix $X$ (in the first layer) and get the output times the weight of the network, denoted by $W_0$. In the end, apply the softmax function on the the obtained result to get the final output of this layer: $\rho(\tilde{A}XW_0)$.


text:
What's the threshold to call something 'machine learning'?
For example, if I use some iterative solvers to find a solution to a non-linear least squares problem, is that already considered machine learning?

This question seems to be a duplicate of https://ai.stackexchange.com/q/12558/2444. Can you clarify how your question is different from that other one?

@nbro: I think your potential duplicate asks specifically for a definition, whilst this question is asking for a way of classifying a problem as ML or not, when it might seem similar. This one goes into detail where the linked one stops.

@NeilSlater Well, a problem is just a problem. You can apply different techniques to solve it, so asking if a problem is machine learning does not make much sense, to be honest, but I didn't think much about it too. Asking which kind of problems can be solved with machine learning techniques makes more sense. This question seems to be about "what kind of techniques are considered machine learning techniques", which, I agree with you, is slightly different from the more general question "what is machine learning".

T. Mitchell defines machine learning in "Machine Learning" book as

a computer program is said to learn from experience  concerning some class of tasks  and performance measure , if its performance at tasks in , as measured by , improves with experience

Hence, based on the above definition, we can't say a machine learning method to every iterative method.
In your specific example, it is just a non-linear solver such as the Newton method to finding roots.
However, you should notice that a non-specific machine learning method can be used in the learning process. For example, you might need some numerical methods to compute the measure $P$ (in the above definition). But, we can't say that the specified method is a machine learning method.

but wouldn't then for example an Autoencoder also not fall into the class of machine learning?

@user1282931 Sorry, but It's not a reasonable conclusion. Because, in autoencoder, the collected data for auto-encoding is E, and we have a loss function P (as a performance) as well. Hence, it is a machine learning method.


text:
How does t-SNE preserves embedding orders?
According to the triplet loss Wikipedia page:

t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on embedded distances.

I don't understand how does t-SNE preserves embedding order from the description given by its Wikipedia page.
I am trying to understand this claim in order to translate the page into other languages. I don't have a very quick understanding of maths, don't be afraid to explain it as if I was a teenager.

You just to look at the method of T-SNE. It "computes probabilities $p_{ij}$ that are proportional to the similarity of objects ${x} _{i}$ and ${x} _{j}$". This means the same as the sentence that you have already quoted:

t-SNE (t-distributed Stochastic Neighbor Embedding) preserves embedding orders via probability distributions, whereas triplet loss works directly on embedded distances.


text:
Why are Siamese Neural Networks used instead of a single neural network?
Siamese Neural Networks are a type of neural network used to compare two instances and infer if they belong to the same object. They are composed by two parallel identical neural networks, whose output is a vector of features. This vector of features is then used to infer the similarity between the two instances by measuring a distance metric.
I was wondering, why not using instead a single neural network that receives as input the two objects that are being compared (e.g. two images) and directly outputs the similarity score? Wouldn't it be better to let the model compare some features of the intermediate layers? Why the Siamese Neural Networks are used for this task and what are the benefits of a Siamese Neural Network over a single neural network that receives as input two instances (e.g. two images) and directly outputs the distance score?

This is a very good question. What are the alternatives to the Siamese network if we want to measure the similarity between 2 inputs, please?

I come up with multiple advantages for siamese against a single neural network for similarity measuring:
Training Phase. If using a single network to replace Siamese, it might be required a double number of parameters (weights) for learning. Hence, training the network will likely converge slower and the network will be more volatile to noise.
Testing Phase. Note that these similarity measurements are used in the applications like face recognition. Now, suppose we are going to use the model in such a system. If we have implemented the model by the Siamese, we would only need to compute the output of the model for the input once, and then use the cached results for the existing images in the database, and eventually fasly compute the similarity measures. On the other hand, if we have implemented the measurement by a single neural network, we should compute per query the result for all combination of the input and images in the background. Hence, in the latter, we cannot cache the results for the existing data in the database. Therefore, single neural network implementation will have much more intensive query time for massin dataset than Siamese implementation.

Why using a single NN to replace siamese neural networks, which the OP describes as being composed of 2  neural networks, would necessarily double the number of parameters? I am not currently familiar with siamese neural networks, so maybe that's why I have this question. Moreover, regarding your second point, I think I understand what you mean, but note that you could also cache the results of the comparison itself. Of course, for objects that you haven't yet compared, you wouldn't have anything in the database.

@nbro Not necessarily, double. So, I updated the post to might be.  For the second part, you're right. We can cache all queries; however, the space complexity of the cache will be the query size that can rapidly grow. But for the Siamese network, it's constant to the number of training data and does depend on the query size. So, for the latter, the cache size is manageable.

Thank you. What are the alternatives to the Siamese network if we want to measure the similarity between 2 inputs, please?

In addition to @Omg's answer note that Siamese networks are typically used in situations where applying (A,B) to the inputs must generate the same output as applying (B,A) (i.e. the similarity measure of A to B is the same as the similarity of B to A).
With a network with separate weights, this is not guaranteed. One way to get close to this is to not only use samples (A,B) as training input but also (equally often) (B,A). Effectively this doubles the number of training steps (and therefore training time) and the network output is still not guaranteed to be symmetric.
By sharing weights, the symmetry of the response of the network ((A,B) gives the same output as (B,A)) is guaranteed by design.

You say "With a network with separate weights, this is not guaranteed". But what do you mean by "separate weights"? It seems that the OP is suggesting to use a neural network that receives as input the 2 objects that need to be compared, so the weights of this neural network would, in a way, be shared among these 2 objects, so not separate. So, can you clarify this?

Good point -- OP mentions "why not using instead a single neural network that receives as input the two objects". I implicitly assumed that this would be two subnetworks of identical structure (at least for the first few layers). So effectively I had 'two subnetworks with separate weights' in mind.

Indeed, I was thinking of a single NN in which the input are the two images concatenated. Anyway, at least in the first layer the weights will be different and what Andre Holzner has answered applies.

Fantastic answer. What do you mean by the symmetry of the network and how Siamese offers that mathematically please (source if possible)?

@nbro. Are there other networks for measuring similarity between a pair rather than Siamese architecture pleae?

Assuming you want to train an object detector to detect people in an image.
The detector's neural networks learn to identify features that help them detect people. These features range from low-level aspects like edges to high-level characteristics such as the presence of feet, hands, and more.
If you use the features from this network for person re-identification, you won't obtain good results because the learned features do not focus on extracting differences between individuals.
However, if you employ a Siamese neural network with a loss function like Hinge, triplet,..., which involves comparing predictions between positive and negative matches, the network learns to extract features that highlight the differences between individuals, such as clothing color and more.


text:
Can an Markov decision process be dependent on the past?
As far as I know MDP are independent from the past. But the definition says that the same policy should always take the same action depending on the state.
What if I define my state as the current "main" state + previous decisions?
For Example in Poker the "main" state would be my cards and the pot + all previous information about the game.
Would this still be a MDP or not?

Yeah, it's possible. If we can abstract past state into current state then MDP will still holds good.

But the definition says that the same policy should always take the same action depending on the state. --> this seems not right, considering stochastic policies, such as epsilon-greedy.

MPD are not independent from the past, but future actions starting from current state are independent from the past i.e. the probability of the next state given all previous states is the same as the probability of the next state given the previous state.
Any state representation which is composed by the full history is a MDP, because looking at the history (coded in your state) is not the same as looking back previous states, so Markov property holds. The problem here is that you will have an explosion of states, since you need to code in the state any possible trajectory, and it is unfeasible most of the times.

What if I define my state as the current "main" state + previous decisions?
For Example in Poker the "main" state would be my cards and the pot + all previous information about the game.

Yes it is a Markov Decision Problem.

It's not totally clear from your description, but it sounds like you may be onto something like an Additive Markov Chain.