text:
Are there tabular datasets where deep neural networks outperform traditional methods?
Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin?
I'd prefer tabular datasets rather than image datasets, since most image dataset are either too simple that even XGBoost can perform well (e.g. MNIST), or too difficult for XGBoost that its performance is too low (e.g. almost any dataset that is more complex than CIFAR10; please correct me if I'm wrong).

I just remembered a good example... http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/  but that was in 2012 and didn't outperform "by a large margin" and was before the xgboost mania.

I'll try out the dataset used in this post: it's great that the columns are uncleaned, since I'd expect the advantage of DNN to be better pronounced when the features are less structured. Thank you @user1269942!

In my opinion, no.
Also images could be interpreted as tabular dataset as well, where certain columns represent different rgb codes of pixels. If you seek to use neural nets opt for image datasets, with large sample size.
Neural networks generally require large sample sizes to perform, and huge dimension inputs to not be outperformed by boosting.

That makes a lot of sense, thank you Matthew! May I ask if you know any complex / large tabular dataset (e.g. perhaps with low-level features as columns) that could be difficult for boosting? Thank you!

Unfortunately, I don't. But you can always make one for yourself, out of image data sets,

I can confirm this anecdotally.  I have tried many times to apply NNs to tabular and/or non-complex data and I cannot recall a NN ever doing better than more "traditional" models.  Ha ha, of course, we all know that the plural of anecdote is not data!  So exceptions may exist.  I do, however, disagree with trying to represent an image as column data.

Disagree on what ground? Possibility or usefulness?

@MatthewTuby  on the grounds that it will likely make the typical convolution operators ineffective as "columnified" image data will likely lose the benefit that pixels near-by in the actual image are near-by in the new representation(column data) like they are in the standard RGB layered matrix representation. A pedant could assert that the normal matrix representation is constructed from columns...but that is not what is being implied here. One example of image data in column format, used in GIS, is the .csv format with encodes a single channel image into "x,y,value" rows.

@MatthewTuby  a quick note to say that current image modeling success is highly dependent on convolution operations.

@user1269942 Do we require convolution operator on tabular data set? Since the question originally focused on tabular data, I somehow implied that the "deep neural network" term refers to multilayer perceptrons and not convolutional networks.

@MatthewTuby we're going to get shut down for too many comments!  "are convolution operators required?"  --no.  But in some cases they work(some tabular time series data for example).


text:
What would be the implications of mistakenly adding bias after the activation function?
I was looking at the source code for a personal project neural network implementation, and the bias for each node was mistakenly applied after the activation function. The output of each node was therefore $\sigma\big(\sum_{i=1}^n w_i x_i\big)+b$ instead of $\sigma\big(\sum_{i=1}^n w_i x_i + b\big)$. Assuming standard back-propagation algorithms are used for training, what (if any) impact would this have?

Let $L(\mathbf{w}, b) = \sigma \left(\sum_{i=1}^n w_i x_i \right)+b$, then the partial derivative of $L$ with respect to $b$, in Leibniz's notation, is $\frac{\partial L}{\partial b} = 1$. Let $L(\mathbf{w}, b) = \sigma \left(\sum_{i=1}^n w_i x_i + b \right)$, then the partial derivative of $L$ with respect to $b$ is $\frac{\partial L}{\partial b} = \frac{\partial L}{\partial \sigma} \frac{\partial \sigma}{\partial b}$.
So, in general, the partial derivatives with respect to $b$ would be different in these two cases, thus, in the gradient descent step, $b$ would be updated differently in both cases. In practice, it may or not affect the performance of the model, depending on the importance of the bias with respect to the given problem. Read this answer to understand the role of the bias.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Why AI is (or not) a good option for the generation of random numbers?
Why AI is (or not) a good option for the generation of random numbers? Would GANs be suited for this purpose?

Why AI is (or not) a good option for the generation of random numbers?

AI approaches are generally not good for generating random numbers, for these reasons:

Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and "true" random sources, possible without using any AI approach, and demonstrably good enough for purposes such as simulations and cryptography. These existing algorithms and devices perform much more efficiently at the task than any AI could.
Generating random numbers with good statistical properties is a very precise task, that the approximations typically used in machine learning cannot cope with (a pseudo-random function is inherently noisy and difficult to learn). AI search techniques are not helpful either, because the amount of output that needs to be generated in order to validate a random data source is very large, making things like tree searches for solutions intractable.

One possible relationship between AI and random number generation that could be explored is in code generation and code analysis for random number generators. In other words, an AI that is sophisticated enough to understand the complexity of code used in a typical random number generator (RNG) might help with making random number generation more efficient, or less easy to hack, or expose flaws in existing implementations. This is beyond the capabilities of AI techniques at the moment - or put another way, human intelligence has taken RNG technology quite far already, so there is very little practical that current AIs can do to improve on it.

Would GANs be suited for this purpose?

No. The input to a GAN is typically a vector of random numbers that have been generated, by an already-existing RNG. The purpose the the GAN is convert this simply-distributed random vector input $\mathbf{x}$ - which might be suitable for many other purposes, such as running simulations of stochastic processes - into a $\mathbf{y}$ sampled from a distribution of some target population $\mathbb{Y}$, which is typically being expressed as a subset of a large space $\mathbb{R}^d$. The values that $\mathbf{y}$ will take are biased to a subspace (or manifold) of all possible space within the representations possible in that vector space. For instance, you might input a random vector with 100 variables and the GAN will generate a picture of a dog. It cannot generate anything other than attempts at pictures of dogs, whilst the space of all possible images that it might create if completely random is far larger (and usually not interesting to look at).
So a GAN would take an already-good random source and turn it into a biased set of numbers that would be generally unusable as a RNG without further processing. The kind of processing you would do (e.g. "noise whitening") is something that some current "secure" random generators do - meaning to use the GAN to generate quality random numbers you would literally put existing RNG code on its input and output layers. The GAN would add nothing but wasted computation.
You cannot even train a GAN to do this task, since a truly random space contains all possible outputs, there will be no negative examples for the discriminator. Although a random picture most often looks like static noise, and a picture of the Mona Lisa is clearly considered "not random" by most people, in fact it is just as likely to be seen as any other image.

any article that proves it? Could you have any me?

@Santos: There might be a blog or something by someone stating the same things, but I don't expect any academic article. This is not an area of speculation or missing knowledge where academic research is necessary. If you can express your question in a way that makes it mathematical in nature, I could probably match that with a suitable proof.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How can we make sure how well the reinforcement learning agent works on a stock dataset?
I read a paper, which is about Deep Reinforcement Learning and it tries to use this method on stock data set. It has been shown that it reaches the maximum return (profit). It has been implemented in Tensorflow.
My question is, how we can make sure that we achieve the maximum value? I mean, is there a parameter or value that can show us how well the RL did its job?


text:
How would the "best function" been constructed if there are no computationally limitations?
I am reading the Wikipedia article on gradient boosting. There is written:

Unfortunately, choosing the best function $h$ at each step for an arbitrary loss function $L$ is a computationally infeasible optimization problem in general. Therefore, we  restrict our approach to a simplified version of the problem.

How would the "best function" been constructed if there are no computationally limitations?
Because the wiki gives me the idea that the model could been better, but that compromises have been made.


text:
Are there any easy ways to create annotated training images for object detection?
For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have $10,000$ images and want to draw bounding boxes on 2 objects for each image, do we have to physically draw those boxes? Is that what most people do these days to create training data?

I would suggest you to get a look at what openCV can do. I don't think you'll need to draw boxes but you may have to answer manually for each picture if it's object A or B.

Possible duplicate of Are there tools to help labelling images?, but maybe this is not an exact duplicate.

None of these comments are in the direction of the question. James is asking whether there is a smarter way to automatically annotate large amount of training images. Here is my input. The short answer is: it depends. If objects are certain know objects that you may be able to use a trained model, you may have some decent annotations, but I am afraid that is likely. Google now does it with AutoML. Besides, you do not need more than 300-400 annotations per each object. Why you think you need to train 10,000 images in the first place?

I once had a fresh dataset and needed to draw bboxes. I made my own little python+opencv program. I always assumed that people in similar circumstances would do the same...but perhaps a wee bit different depending on the needs of the problem. So, the answer is yes, the boxes need to be draw by somebody and if it's your task, then you're the best person to do it. When I did it, I learned a lot about my images as well.

If you don't manually enter in the information, you expect a system to. Which becomes circular since you're trying to teach a system to detect images....


text:
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times $a$ has been picked in $s$.
However, I found that this does not work well in some cases depending on the magnitude of $Q(s,a)$. For example, if $Q(s_i,a_1) = 100$, and $C(s_i,a_1) = 1$, then then this gives a standard deviation of 0.5, which is extremely confident even though the action has only been picked once. Compare that to $a_2$ which may be the optimal action but has never been picked, so $Q(s_i, a_2) = 0$ and $C(s_i,a_2) = 0$. It is unlikely that TS will ever pick $a_2$.
So, how do I solve this problem?
I tried normalizing the Q-values such that they range from 0 to 1, but the algorithm returns much lower total returns. I think I have to adapt the magnitude of the standard deviations relative to the Q-values as well. Doing it for 1 normal distribution is pretty straightforward, but I can't figure out how to do it for multiple distributions which have to take into consideration of the other distributions.
Edit: Counts should be $C(s,a)$ instead of $C(s)$ as Neil pointed out

"if $Q(s_i,a_1) = 100$, and $C(a_1) = 1$, then..." in your imagined scenario, you have made a very large update to $Q(s_i,a_1)$ with either 0 or 1 samples from it - is this actually possible with the learning mechanism you are using, e.g. are you processing rewards such that $\alpha \times r_t$ could make a $+100$ value update in a single step? This is salient, because there is no need to worry about and attempt to fix numerical problems that don't occur in practice.

Counts should be s,a. I have edited the question. I am selecting the action with the highest sampled Q-value.

I have a learning rate of 0.1, I didn't include it, perhaps I should have. even then, it doesn't change the issue where $\mathcal{N}(10, 0.5)$ for $a_1$ is still going to be picked over $a_2$ with distribution $\mathcal{N}(0, 1)$.

What about using the advantage function $A(s,a)=Q(s,a) - V(s)$ instead of $Q(s,a)$? It should have a much lower variance, and, indeed, you should be able to compute the value function to do so.


text:
How does the neural-network know how to tweak weights for a specific neuron?
I know backpropagation uses cost and gradient descent to tweak the weights to minimize the cost. But how does it know which weights to give more weight to in the first place? Is there something inside each neuron in the hidden layers that defines how this is an important neuron for the correct result in some way? How does the network know how to tweak those weights for that specific neuron?

tl;dr
The whole point of gradient descent is to assess the contribution of each parameter towards the loss. This information is uncovered through the gradient of the loss w.r.t each parameter.
A deeper look...
Suppose we have a NN with parameters $w_{i}, \; i={1, 2, ...}$. This NN makes some predictions, which we compare to the actual targets and compute a loss $J$. The loss (or cost) function tells us how far off we are from the target. This is what we want to reduce, so that the predictions fall closer to the target.
By computing the partial derivative of the loss function $J$ w.r.t a parameter $w_i$ (so this is just one partial derivative and not the full gradient vector)
$$
\frac{\partial J}{\partial w_i}
$$
the NN uncovers two pieces of information:

The slope of $J$ w.r.t $w_i$, which tells the NN how much $w_i$ affects $J$.
Its sign, which tells the NN which way to tweak $w_i$ to decrease (or increase) the value of $J$.

By making the parameter updates depend on the derivative
$$
w_i^{new} \leftarrow w_i^{old} - \lambda \frac{\partial J}{\partial w_i}
$$
the NN causes parameters that affect the loss the most, to be updated the most.
You can think of this as: parameters that are more to blame for the network's mistakes (i.e. contribute more towards the loss) are forced to change the most, in the direction that will decrease the loss.


text:
Why do Bayesian algorithms work well with small datasets?
I read very often that Bayesian algorithms work well on small datasets. Why is that? I think it is because they might generalize more, but why is that?
See also Investigating the use of Bayesian networks for small dataset problems.

The main reason should be that Bayesian algorithms naturally incorporate a form of regularisation (the prior), so they should be less prone to over-fitting the small dataset. Of course, the choice of the prior can affect your estimates.
You can view certain training regimes in machine learning and deep learning as an application of Bayesian statistics: for example, if you train a neural network with weight decay, this is equivalent to a MAP estimate.
I guess it might also be computationally easier to perform Bayesian inference with a smaller dataset (although I have never tried to show that this is true: my practical experience in this context is limited to Bayesian neural networks). Of course, the more data you have, in theory, the closer your estimates should be to the actual true values.
Here you have a similar question.


text:
Is artificial intelligence really just human intelligence?
Essentially, AI is created by human minds, so is the intelligence & creativity of algorithms properly an extension of human intelligence & creativity, rather than something independent?
I assume that intelligence does not necessarily require creativity, however, creativity can result from machine learning.  (A simple example is AlphaGo discovering novel strategies.)

This is an old question, going back at least to 1950. It is one of the original objections to AI that Turing considers and attempts to refute in his seminal 1950 paper Computing Machinery and Intelligence.
Turing actually attributes this objection to Lady Lovelace, apparently quoted by another author. In Turing's paper, this is objection #6: Lady Lovelace's Objection, in section 6 of the paper. The objection is concisely stated as

The Analytical Engine has no pretensions to originate anything. It can
  do whatever we know how to order it to perform.

where "The Analytical Engine" was an early design for an all-mechanical general purpose computer.
Turing offers two replies to this objection. First, he reminds us that computer programs have bugs. That is, they often do things their creators did not intend. This is unsatisfying to many readers, but it does address the objection: programs may act in ways that are unrelated to our intelligence, and in doing so, might display unexpected intelligent behaviors. In this sense, their intelligence would not be an intentional product of human intelligence.
Turing's stronger objection comes from an anticipation that learning would eventually move to the center of AI research (keep in mind again, this is written in 1950, well before any reasonable learning algorithms had been proposed!). Turing uses the example of a robotic child in Section 7 of the paper (Learning Machines) to elaborate on his point. A child is created by its parents, but, endowed with the ability to learn, quickly begins to display behaviors its parents do not anticipate or intend. No one would suggest that a person's intelligence is "really just" the intelligence of their parents, even though their parents created them, and are partially responsible for that intelligence.
Likewise, Turing's proposed robotic child is created by a parent, but, endowed with learning, quickly begins to engage in behaviors the parent does not anticipate or intend. Therefore, machine intelligence need not be reduced to just human intelligence.
I think that if Turing were alive today, he would agree that we are now beginning to move into the era of learning machines he anticipated. Some of our programs now engage in intelligent behaviors that we do not anticipate or understand. For example, self-driving cars now kill or maim people, because they have learned behaviors their creators did not intend or anticipate, perhaps not unlike a reckless teenage driver.

Thanks for treating the historical dimension!  The bugs, or errors, Turing references may have more weight in the context of random mutation in nature, where some increase fitness...

This is definitely the best answer so far, especially because you cite one of the most influential papers in artificial intelligence. However, I need to agree with Ada Lovelace. Machines can only do what we originally program them to do, either directly or indirectly, with different levels of detailed commands. If I say to a machine, "Optimize! I don't care about the solution (numbers), just do what I said machine!", it will optimize. I command the machine to optimize, with respect to a metric that I (a human) chose.

Turing's argument relies on the idea that random or expected operations can lead to "intelligent behavior", but can intelligence be measured only by observing the behavior? I don't think so. The other argument is also flawed, in my opinion. The fact that you didn't predict something, it doesn't mean that you didn't indirectly cause it.

@nbro yeah, Turing won't have a lot to say to you if you don't buy his earlier argument that intelligence must be determined by observing behavior. I think that you're right, without that piece, Lovelace's objection is really strong.

I think no, it isn't. The reason I would say no, is that in order for it to be an extension of our intelligence & creativity, it must be limited by it. This, I believe, isn't the case however. We are capable of creating an AI that is smarter than ourselves (say at Go or Chess, without cheating and checking every possible move), and so it is not bound by our own intelligence.
I would liken it to creating a child. Just because you gave birth to Einstein, doesn't mean he's an extension of your intelligence. (This is of course pretty rudimentary, as it's very debatable as to whether it's reasonable to liken humans to AI).
Of course, this is a philosophical question, so it's hard to really answer yes or no.

Your analogy of the child is not a good one, in my opinion. When you develop an algorithm, you know exactly what it is doing. In principle, you can exactly compute what the computer computes. For example, when you're training a neural network with gradient descent and back-propagation, you know exactly what's happening, even though you do not necessarily memorize the numerical or symbolic representation of the neural network (the combination of all the weights) or the gradients, etc. In the case of a child, you can't really be certain of the outcome, a stochastic combination of genes.

The confusion arises because apparently the AI can do more things than us, but this is not true. In principle, we can do everything a computer can do. In fact, computers were invented exactly to mimic our methodological calculations. Computers simply do it a lot faster than us. AI is really limited by our limits, in terms of computable functions that we can compute (but not in terms of speed). We could in principle perform as well as AlphaGo, if we just had enough time and memory.

@nbro I agree that that AI is limited by our limits given enough time and memory. The reason for this, is I think a human has absolutely no limits with infinite time and memory. However, in reality this is never the case. So for all practical purposes, I would say an AI takes an intractable problem (which could be solved with infinite time) for a human, and makes it possible, making it unbounded by our limits for normal cases.

I would say: no, it's not just an extension of human intelligence.
Actually, I would argue there's nothing like human intelligence. At least it's not clearly distinguishable from intelligence in general.
If you say AI is just a set of instructions that are made by humans, you might be right. But what if this set of instructions contains instructions on how to change instructions? That would mean that the AI knows how to learn. What if you include instructions on how to learn to learn to learn to learn (...) to change instructions?
At what point would you say that this intelligence is still an extension of human intelligence? If you argue like this then you must also put "human intelligence" in a set altogether with every animal intelligence because it all originates from some sort of intelligence that is based on physical brain activity.
In fact, when a child is born, it is not more intelligent than most of the animal species. The only thing that enhances its intelligence from time to time (and do stuff like speaking or using its hands like tools) is the ability to learn.
I don't see why an AI hasn't got the potential to increase its intelligence to level where one would say: "This is not an extension of human intelligence anymore, this is something independent".

No, the way human minds think is in no way related to the way an AI thinks. Although you could say that AI is a much simpler form that represents how the brain processes information. For the human brain to think, sense, and act there are billions of connections is various cortex's of the brain that process information in different ways. If talking about brain information as electrical signals you could say that different cortex's of the brain have change in power of specific frequency bands of the brain signal which can be decoded as planning, preparation, thoughts, visual, movement, creativity, attentiveness and much more.
So, to answer your question AI could be considered as an extremely minute extension of human intelligence. It's like comparing our solar system to the Milky Way, although the comparison maybe a bit too large as we are slowly becoming able to understand the underlying processes and build fast processors mimicking brain processing and efficient power consuming hardware tech to run humongous neural nets. In the soon future your statement may hold true.

Right, AI is an extension of human creativity and the implied limitation is that it inherits bias through the specific choice of which features to consider. Given a set of features it is then far more able at calculating which combination of features best helps explain the relationship being considered than is the human mind. Humans are too distracted to think to the depth that AI and machine learning can. But that extreme focus is not intelligence.
One of the issues that prevents the human mind from thinking at comparable depth is the need to massage the set of features that might apply; we are constantly reviewing features, adding in new and eliminating those that do not contribute. Creativity is openness to admitting other seemingly unrelated features and hoping for emergence, and managing to persist in being creative when emergence is delayed.

I believe AI is, at least in certain ways, both an extension of human intelligence & creativity, and something independent as well. Note people didn't design airplanes to try to fly like birds do. Although planes use the same principles of aerodynamics that birds use to fly, we've adapted how those physics principles are applied to accommodate what we have to work with, i.e., metal, by having things like propellers, jet engines, fixed wings (initially, although later we also had helicopter rotor blades), etc.
In a similar fashion, we have adapted a few things we've learned about how human minds & intelligence work, with artificial neural networks being a prime example. However, even with just our fairly limited understanding, we've implemented neural networks differently, e.g., by which activation functions are used.  Although we are learning more about how our brains work through neuroscience research, there's still so much we don't yet know. Nonetheless, I believe one of the biggest differences overall between our minds & AI is that our general intelligence comes from mostly massive parallel processing, to a much greater extent than even higher end GPUs can deliver, or even at least most supercomputers, while artificial intelligence generally depends instead a lot more on the massive speed of calculations available on our modern computer chips.
It's this learning, adapting & extending what we know about how we think & create, in combination with the mostly independent boost of using the advantages of computer chips (mostly their ability to do very fast computations), that has allowed AI to advance as far as it has so far. Nobody, including myself, can be sure of where & how the next major advances in AI will occur, but I believe it'll likely be a combination of learning & using what we learn about how we mentally operate, along with advances in computer related knowledge & technology (e.g., new algorithm techniques, more & better parallel processing, quantum computers with many simultaneous qubits operating, etc.).

No it isn't.
AI is essentially human intelligence with a combination of computing power to achieve tasks that a human alone cannot achieve in the time period that a programmed machine can.
To give an example. A human can identify a pattern in a data set of say 1000 records. However if that same logic needs applied to a data set of a billion records, a human would take ages to do it. But a machine can do that in seconds if the human gives the right instructions to the machine on how to do it.
Hope that helps.

It's true that computers are a lot more efficient at certain tasks, but giving a machine precise instructions to perform some task would not fall under the definition of AI and this doesn't say much about whether AI is truly intelligent or simply an extension of human intelligence.

The answer in part seems to depend on what you mean by "human intelligence". If you mean behavior that would usually be regarded as requiring intelligence were a human to produce it, then various types machines can be intelligent.
Such "intelligent" machines presumably include player pianos. Playing the piano and producing a melody is widely regarded as requiring human intelligence when humans do it. Player pianos produce the same sort of behavior, but without a human touching a key. Hence (so the argument goes) player pianos are intelligent.
But if "intelligence" includes having the inner process of understanding, say understanding the meanings of symbols of written language, then at least according to philosopher John Searle, purely symbol manipulating devices such as digital computers could never be intelligent.  This is because symbols in themselves don't contain or indicate their meanings, and all the computing machine gets and manipulates is symbols in themselves.
However, there does seem to be a sense in which the question "Is artificial intelligence really just human intelligence" is true of computers. This is when the behavior of the machine is caused by human intelligence. A human writes a program that defines, mandates, the behavior of the machine (just like a human designs the mechanism and paper roll of a player piano). This design takes human intelligence. The machine has no intrinsic, or innate, intelligence. It's just an automaton mindlessly following the causal sequence created by the intelligent human designer.
Now if computers are purely symbol-manipulating devices, and if Searle is right, AI is doomed, at lest as long as its development platform is the digital computer (and no other machine is available or seems on the horizon).
However, are computers purely symbol-manipulating devices? If not, there may be a way they can acquire meanings, or knowledge, and, for instance, learn languages. If computers can receive (including from digital sensors) and manipulate more than just symbols, they may be able to acquire the inner structures and execute the inner processes needed for human-like understanding. That is, they might be able to acquire knowledge by way of sensing the environment (as humans do). A human might write the program that facilitates acquisition of such knowledge, but what the knowledge is about would be derived from the sensed environment not from a human mind.
But here we're talking about "intelligence" defined over inner processes and structures, not or not just external behavior. If you define human intelligence as external behavior, as the Turing test does and as AI researchers often do, then music boxes with pirouetting figurines, player pianos, and programmed computers all have human-like intelligence, and artificial intelligence as it exists today is really just the same sort of thing as human intelligence.


text:
When are compiled vs. interpreted languages more optimal in AI?
When are interpreted languages more optimal?  When are compiled languages more optimal?  What are the qualities and functions that render the so in relation to various AI methods?

For AI in the broadest sense, I don't think the comparison is any different to general programming. If you could narrow this down to a specific task or problem, it might be possible to answer. Otherwise it seems rather broad and not well defined. See https://stackoverflow.com/questions/3265357/compiled-vs-interpreted-languages

Depends.
If it's just a local home project, only you will access, then any language is good. If you are working in a team, then decide together. In general, compiled languages are good because they proof the code before execution, but scripted languages don't take the time to build your project. Most optimal would be python, since it's the language with the most AI libraries. Read More

@NeilSlater fair point—possibly I should reference in general the "Python vs. Java/C" debate.  (The intent here was not to be entirely specific, but address a fundamental aspect of that debate.)

Interpreted languages allow for a faster development cycle, as they don't require time for compilation, and fragments can often be run without having a complete program. They often also have fewer constraints for variable declaration or typing. That means they can be used to quickly scope out a problem and try different solutions.
The drawback is the slower execution speed. But during development this is not a big factor; it only becomes important in a production environment. So one option would be to use an interpreted language during the R&D phase, and then re-implement the algorithm in a compiled language for performance improvements.
Since ML and NNs have become more prevalent in AI, numerical computing has become more important. This is an area where interpreted languages traditionally don't perform too well, so one would use a (compiled) library for, say neural networks, or genetic algorithms, and use 'glue code' to integrate this into a bigger system. The glue code would transform/prepare data and convert this between different formats required by libraries. This is often done in interpreted scripting languages, as they might have to be changed more frequently and are not performance critical.
Apart from development, the type of computation is also key: as mentioned, numerical computing generally works better with compiled code, but interpreted languages often have advantages in symbolic programming. This is why Lisp and Prolog have become popular AI languages, as opposed to Fortran or C.
In an ideal world you would use an interpreted language for development, and then compile this once you're done. However, due to the way these languages work, compilation is often non-trivial.

I think this answer is pretty good. It may be worth also mentioning that some interpreted languages (e.g. Python) have the ability to run high-performance compiled code directly for portions of the program where speed really matters. For example, something like PyTorch allows one to write high performance ML code in Python, by invoking lower-level libraries written in C in a way that looks like one is writing regular Python code.


text:
What could be the cause of the drop of the total reward when using DQN to solve the cart-pole environment?
I'm trying to use DQN to solve the cart-pole environment. I have 2 networks (target and behavior). Both of them have 3 hidden layers with 24 neurons, using the ReLU activation. The loss is MSE and the optimizer is Adam. I copy the weights of the behavior network to the target network every 15 backpropagation steps.
My agent learns. Below you can see the total reward and running average plots.

However, it has a lot of "drops". Usually, after a couple of perfect sequences, it just "kills" the running average with a couple of very short episodes. What may be the reason for this behavior?


text:
In lemma 1 of the TRPO paper, why isn't the expectation over $s'∼P(s'|s,a)$?
In the Trust Region Policy Optimization paper, in Lemma 1 of Appendix A, I didn't quite understand the transition from (21) from (20). In going from (20) to (21), $A^\pi(s_t, a_t)$ is substituted with its value. The value of $A^\pi(s_t, a_t)$ is given as $\mathbb{E}_{s'∼P(s'|s,a)}[r(s) + \gamma V_\pi(s') − V_\pi(s)]$ at the very beginning of the proof. But when $A^\pi(s_t, a_t)$ gets substituted, I don't see the expectation (over $s'∼P(s'|s,a)$) appearing anywhere. It will be of great help if somebody lends some light on this.

Let's assume $\gamma = 1$ to simplify things
\begin{align}
\mathbb E_{\tau|\pi} [\sum_{t = 0}^{\infty} A_\pi(s_t, a_t)] &= \mathbb E_{\tau|\pi}[A_\pi(s_0, a_0) + \ldots A_\pi(s_i, a_i) + \ldots]\\
&= \mathbb E_{a_0 \sim \pi,s_1 \sim P(s_1|s_0, a_0)}[A_\pi(s_0, a_0)] + \ldots + \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] + \ldots
\end{align}
if we observe only $i$-th timestep
\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a')]) \pi(a'|s_i)\\
&= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + \\ &V_\pi(s_{i+1}) - V_\pi(s)]]) \pi(a'|s_i)
\end{align}
\begin{equation}
\mathbb E[\mathbb E[f]] = \mathbb E[f]
\end{equation}
\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]) \pi(a'|s_i)\\
&= \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]
\end{align}
now for all timesteps sum everything up.


text:
How would we define a set that contains itself within a knowledge ontology?
How would we define a set that contains itself within a knowledge ontology?
I am thinking that set membership would probably inherit from a generic base class of total containment from which both physical containment and conceptual containment are derived.

total containment

physical containment
conceptual containment

set containment

Hi Polcott! Your question isn't completely clear to me. You say "I am thinking that set membership would probably inherit from a generic base class of total containment", but then you show that "set containment" (which I think is a synonym for "set membership") derives from "conceptual containment". Can you clarify this? Furthermore, have a look at Russel's paradox, which is at least remotely related to this question.

@nbro I am trying to piece together all humans concepts exactly how they already naturally fit together. I used the term [conceptual containment] as a more generic concept than [set containment].  [Members of a family] may be construed as an example of conceptual containment.


text:
How could an AI detect whether an enemy in a game can be blocked off/trapped?
Imagine a game played on a 10x10 grid system where a player can move up down left or right and imagine there are two players on this grid: An enemy and you. In this game, there are walls on the grid which you can't go through. The objective of this game is to block the enemy in so he can't move around the rest of the board and is effectively "trapped".
I want to write an algorithm that detects which nodes on the board I as a player need to put blocks in, in order to trap the enemy. There are also some other considerations to think about. You have to be able to place the blocks before the enemy place can get out of the box. Also note more thing: You can move AND place a block in the position that you're moving to at the same time.
Here's a picture as an example of the game.

EDIT: note that the board in the picture is 5x5, but that's okay for the purposes of the example
In this example, I could go up, then right and place a block, then right and place a block, then up and place a block. If there's more than one way of blocking off the enemy, then I should use the way that's going to give my enemy the least amount of space.
Researching on google couldn't find me anything relevant, although it may have been because I wasn't using relevant search terms. I also thought about using a monte Carlo search tree algorithm for simultaneous games, but I would need to research into that more.

Any progress on this problem statement as I'm looking for an algorithm to do the same? I've already implemented A* algorithm to find the shortest path to the enemy but not sure how to go about trapping the enemy in its own space.

@SakthiSomaskandan Nope, I ended up giving up on it. It seemed like any solution I would end up with was either a heuristic or its runtime was unacceptable for my time constraints. Good luck with your own work. Please let me know if you find something promising in your research. I'm still curious about this.

If I've understood the logic correctly, the player tries to build a wall such that the enemy cannot reach the player in any way. This can normally be determined by some path-finding algorithms, Dijkstra Shortest Path algorithm being a reasonable choice for the setting and grid-size.
This algorithm explores the possible paths from a starting point to one or multiple end points, and usually returns the shortest path to the point(s). If there is no path to the mentioned point, it will not return anything, and you will know the two points are blocked from reaching each other.
Of course, if I've truly understood the rules correctly, the bigger question is how to avoid having the player just build a wall around themselves to block of the enemy instead, which is likely going to be trivially easy. And additionally rule of requiring the player to be in the "room" with the bigger area after a room has been blocked off (Something which coincidentally also could be done by extending the Dijkstra's algorithm a bit with custom logic)

Dijkstra's would give you the shortest path between the starting point and the end point, but the problem is that you don't know what the end point is. You are actually searching for the end point.
Also, OP mentions that the algorithm should block off the enemy by allowing him as little space as possible. If you just build a wall around yourself the you allow the enemy the largest space possible.


text:
Should I train different models for detecting subsets of objects?
Suppose we have $1000$ products that we want to detect. For each of these products, we have $500$ training images/annotations. Thus we have $500,000$ training images/associated annotations. If we want to train a good object detection algorithm to recognize these objects (e.g. YOLO) would it be better to have multiple detection models? In other words, should we have 10 different YOLO models where each YOLO model is responsible for detecting 100 products? Or is it good enough to have one YOLO model that can detect all 1000 products? Which would be better in terms of mAP/recall/precision?

If you have 10 YOLO models that detect 100 different products and you need to detect a product how will you know which model was trained for that specific product ? You would have to wastefully process that product with 9 YOLO models. In that case you might as well make an ensemble of object detectors instead of training more of them on different objects. If we talk about only 1 YOLO model, authors in their YOLO v2 paper claim that they can detect over 9000 object categories so it might not be unreasonable to expect that YOLO v3 could learn 1000 different objects on its own.

Added another correct answer @Prime Number

This is called decomposition of multi-class classifier. Your proposed method is called one vs all.

One vs. all provides a way to leverage binary classification. Given a classification problem with $N$ possible solutions, a one-vs.-all solution consists of $N$ separate binary classifiers—one binary classifier for each possible outcome. During training, the model runs through a sequence of binary classifiers, training each to answer a separate classification question.

Source: https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/one-vs-all.
According to this article. The author of the article did experiments on SVM on 8 different benchmark problems. According to the results, this method is sometimes as good as others, but usually not the best. It is also never substantially better than any other method. The article also stated that the best method is usually problem dependent.
Also, this method will decrease inference speed a lot, and used substantial amount of GPU memory. According to the source, it does not improve performance a lot, so you best bet for getting a higher performance is probably to use a different model architecture, for example the FPN FRCN, which is stated in the YOLO v3 paper having the best performance, but not fast inference speed. YOLOv3 is designed to have a fast inference speed, to provide real time object detection system, so for performance you should probably use other model architecture instead.


text:
What is the difference between evolutionary game theory and meta-heuristics?
Here is a list of meta-heuristic algorithms

Ant colony optimization,
Ant lion optimizer,
Artificial bee colony algorithm,
Bat algorithm,
Cat swarm optimization,
Crow search algorithm,
Cuckoo optimization algorithm,
Cuckoo search algorithm,
Differential evolution,
Firefly algorithm,
Genetic algorithm,
Glowworm swarm optimization,
Gravitational search algorithm,
Grey wolf optimizer,
Harmony search,
Multi-verse optimizer,
Particle swarm optimization,
Shuffled complex evolution,
Simulated annealing,
Tabu search,
Teaching-learning-based optimization

Can anyone explain the similarities and dissimilarities of evolutionary game theory and the meta-heuristic approach?


text:
Can supervised learning be used to solve the inverted pendulum problem?
I know that reinforcement learning has been used to solve the inverted pendulum problem.
Can supervised learning be used to solve the inverted pendulum problem?
For example, there could be an interface (e.g. a joystick) with the cart-pole system, which the human can use to balance the pole and, at the same time, collect a dataset for supervised learning. Has this been done before?

Yes, you can do it, for example this paper uses YouTube videos to learn how to play difficult game Montezuma's Revenge

You may also be interested in this post.

Of course, you can, and yes this has been researched and done before. By using supervised learning, you give the machine some data, and it will try to figure the best way to analyze and predict the next movement in case of an inverted pendulum problem.
I found a complete paper for this problem: Neural network control of an inverted pendulum on a cart by Valeri Mladenov et al.


text:
Where should we place layer normalization in a transformer model?
In Attention Is All You Need paper:

That is, the output of each sub-layer is $LayerNorm(x+Sublayer(x))$, where $Sublayer(x)$ is the function implemented by the sub-layer itself. We apply dropout to the output of each sub-layer, before it is added to the sub-layer input and normalized.

which makes the final formula $LayerNorm(x+Dropout(Sublayer(x)))$. However, in https://github.com/tensorflow/models/blob/0effd158ae1e6403c6048410f79b779bdf344d7d/official/transformer/model/transformer.py#L278-L288, I see
def __call__(self, x, *args, **kwargs):
  # Preprocessing: apply layer normalization
  y = self.layer_norm(x)

  # Get layer output
  y = self.layer(y, *args, **kwargs)

  # Postprocessing: apply dropout and residual connection
  if self.train:
    y = tf.nn.dropout(y, 1 - self.postprocess_dropout)
  return x + y

which ends up as $x+Dropout(Sublayer(LayerNorm(x)))$. Plus there are extra LayerNorms as final layers in both encoder and decoder stacks.
In a quick test, the performance of this model seems to be better than if I change back to the paper's order of operations. My question is: why? And could it be predicted in advance?
I note that Generating Long Sequences with Sparse Transformers uses the $x+Dropout(Sublayer(LayerNorm(x)))$ order, but doesn't discuss it, unlike the other changes it makes to Transformer.


text:
What are the main AI technologies behind the Watson platform?
What AI techniques does IBM use for its Watson platform (specifically, its natural language processing part)?

As a person who works with people who work on Watson, perhaps I can give some insight.
The name Watson is casually thrown around a lot whilst many people aren't aware of its evolution into a larger suite of systems and services. We now have Chef Watson, Watson Health, and many other developing projects along the "cognitive" route. Watson is really an amalgamation and varied application of the different cognitive computing routes IBM is pursuing.
So, there are many forms of NLP that Watson conducts and has conducted, developed by different teams to fit different processes, interconnected in different ways. Additionally, much (probably all) of it is proprietary/classified since, as one would imagine, ongoing research is constantly being conducted and added to Watson. This is likely your largest obstacle. The precise workings of the NLP of the Jeopardy flavor of Watson are probably themselves still classified (I can't find anything in the time I've just spent looking myself)
There are, thus, many answers to this question; many outdated, and others not always applicable. The full answer is very complicated and, by the time you find out what the answer is today, it's probably already been advanced. The researches I know are always working on new, cutting-edge algorithms and processes for text classification and the related NLP topics. See, for example, this article http://www.aaai.org/Magazine/Watson/watson.php, where it is stated

For the Jeopardy Challenge, we use more than 100 different techniques for analyzing natural language, identifying sources, finding and generating hypotheses, finding and scoring evidence, and merging and ranking hypotheses.

and

Watson's metalearner uses multiple trained models to handle different question classes as, for instance, certain scores that may be crucial to identifying the correct answer for a factoid question may not be as useful on puzzle questions

Most of these techniques have likely already been modified, adapted, or dropped altogether for other methods.
To point you to more information, take a look at the links below.
General

The DeepQA Research Team, specifically, have a look at their publications
IBM Research AI

Articles

Building Watson: An Overview of the DeepQA Project (2010) written by the IBM Watson Research Team, led by David Ferucci, and published in AI Magazine Fall, 2010.

Videos

How does IBM Watson work? (2018), by IBM Watson, which states that Watson uses deep learning and transfer learning

Other

Natural language processing: an introduction (2011), a good but technical paper on natural language understanding (that acknowledges Watson)


text:
Are there any DeepQA-based computers other than Watson?
My understanding is that Watson is the name of the computer, and DeepQA is the name of the software or technology. They are both correlated.
Are there any computers/technologies other than Watson which are using DeepQA? Or is Watson the only computer which implements that software/technology?

I cannot say for certain, but I know of no such other uses (I work at the building where Watson is developing but do not directly work with it).
The DeepQA team's page (https://www.research.ibm.com/deepqa/deepqa.shtml) only ever references Watson as the implementation, and based on the structure of the FAQ there I would imagine they'd be eager to list any interesting other uses it would have, but no such entries exist there.
I would, however, also note that while DeepQA is IBM's proprietary implementation of QA using deep-learning methods, that approach is becoming more popular and is almost certainly being worked on by other companies and could soon be on other computers, if not already so in some form.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is the difference between machine learning and deep learning?
Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?

The difference is that deep learning includes implicitly representation learning in their models.

Deep learning is a specific variety of a specific type of machine learning. So it's possible to learn about deep learning without learning all of machine learning, but it requires learning some machine learning (because it is some machine learning).
Machine learning refers to any technique that focuses on teaching the machine how it can learn statistical parameters from a large amount of training data. One particular type of machine learning is artificial neural networks, which learn a network of nonlinear transformations that can approximate very complicated functions of wide arrays of input variables. Recent advances in artificial neural networks have to do with how to train deep neural networks, which have more layers than normal and also special structure to deal with the challenges of learning more layers.

Deep learning is one form of machine learning.
Deep learning refers to learning with deep neural networks, essentially networks with many layers.
Neural networks are one group of many forms of machine learning:

Neural Networks
Decision Trees and Random Forests
Support Vector Machines
Bayesian Approaches
k-nearest neighbors

First, in most condition machine learning actually refers traditional/classical machine learning, and deep learning is specifically referring multi-layered neural network, and neural network is one of the machine learning approach.
Second, Machine learning especially supervised machine learning requires engineers to design and predefine features manually, which are used to represent the data in numerical way. Such as we can represent animals with three features such as the number of eyes, the number of legs and the number of heads. The data [2,4,1] representing an animal with 2 eyes, 4 legs and 1 head. In this scenario, the feature is extracted by us, because we have knowledge on animals, and we think these features can represent animals.  However, instead of hand-crafting features the deep learning learn the features automatically.
Third, when someone say machine learning he is saying algorithm, such as naive bayes, decision tree, linear regression etc. However, the deep learning is more related to the framework and architecture such as RNN, CNN, Transformer etc.
Fourth, it is possible to start deep learning without knowing machine learning, sources from internet like Andrew Ng's course usually covers most topic you should know in deep learing. Try search Andrew Ng, I think he is really good!

This is a good answer that emphasizes that in traditional ML approaches we usually need some form of "feature engineering" (which is a term that you may add to your answer). However, note that when we say deep learning we are not just referring to the architecture but also the algorithms, in particular, gradient descent and back-propagation, which are typically the algorithms used. See also this answer.

Deep Learning is subset of Machine Learning.
Machine learning and Deep learning both are not two different things. Deep learning is one of the form of machine learning.
The level of layers in Neural network are more and more in depth learning is part of Deep learning.

“Deep learning is a particular kind of machine learning that achieves
  great power and flexibility by learning to represent the world as
  nested hierarchy of concepts, with each concept defined in relation to
  simpler concepts, and more abstract representations computed in terms
  of less abstract ones.”

When I started Machine Leraning chapters in book used to look like this

I) Supervised:

Regression

Linear models

Classification

Logestic Regression
Neural Network
Decision Tress and Random Forest
Boosting and Bagging
SVD and SVM

II) UnSupervised Learning:

Clustering

K-Means
Hierarchical
Gaussian Mixture Model
DB Scan

Association Learning.

III) ReInforment Learning:

All of a sudden chapter I>2>b created a sub-field of its own . Well to know why, let me tell you a bit of history. Machine learning word was coined in 1959 by Arthur Samuel to signify that machines were able to learn from data than explicit instruction. Initally it was broken into two groups based on if th approach required label data or not(ie regression, classification), then they realised we can cassify by clustering too which gave birth to unsupervised. And word reinforment learning was born inspired by areas of game theory. Lets keep those details aside for later.
Coming to deep learnign, the word deep learning came very recently, as recent as 2008 from a Geoff Hinton conference. There people started using it to indicate a very deep neural network architecture used in a paper presented by Geoff Hinton and from then onwards it kind of became as a new way of classifying machine learning besides supervised, unsupervised or reinforcement.(Disc: There may be odd reference of calling NN as DL before this but not so popular and acceptable prior to this)
Well I sometimes feel the name deep learning is somewhat misnomer, it would have been better of if it was named as neural learning or to stress on depth maybe deep neural learning. If you are new you might be wondering what depth I am talking about, the entire word deep came from the fact that neural network (thanks the availability of high processing abilities of GPUs) were now able to train successfully on multiple layers. The word deep can also be loosely used to include other non-neural network areas of machine learning which requires lots of computation like deep belief net or recurrent net. To be precise the units of the networks today are no longer a mere neuron or a perceptron, it can be LSTM, GRU or a capsule, so I guess word deep now makes more sense than before.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What are some resources on continuous state and action spaces MDPs for reinforcement learning?
Most introductions to the field of MDPs and Reinforcement learning focus exclusively on domains where space and action variables are integers (and finite). This way we are introduced quickly to Value Iteration, Q-Learning, and the like.
However, the most interesting applications (say, flying helicopters) of RL and MDPs involve continuous state space and action spaces. I'd like to go beyond basic introductions and focus on these cases but I am not sure how to get there.
What are some resources on continuous state and action spaces MDPs for reinforcement learning? And what areas do I need to know or study to understand these cases in-depth?

There is a small survey of continuous states, actions and time in reinforcement learning in my thesis proposal.
Regarding books, Reinforcement Learning: State-of-the-Art seems to be pretty up-to-date from the excerpts I've read.


text:
Why is cross-over a part of genetic algorithms?
Genetic Algorithms has come to my attention recently when trying to correct/improve computer opponents for turn-based strategy computer games.
I implemented a simple Genetic Algorithm that didn't use any cross-over, just some random mutation. It seemed to work in this case, and so I started thinking:
Why is cross-over a part of genetic algorithms? Wouldn't mutation be enough?
This is from a data dump on an old AI site. The asker had the UID of 7.

The user that asked the original question was probably hwlau according to this, so that, if you really want to give credit, we are giving credit to an actual person/account.

Mutation is usually defined to be a global operator, i.e. iterated mutation is (eventually) capable of reaching every point in the vector space defined by the geneome. In that sense, mutation alone is certainly 'enough'.
Regarding the motivation for crossover - from Essentials of Metaheuristics, p42:

Crossover was originally based on the premise that highly fit individuals often share certain traits, called building blocks, in common.
  For example, in the boolean individual 10110101, perhaps
  ***101*1 might be a building block

(where * means "either 0 or 1")
So the idea is that crossover works by spreading building blocks quickly throughout the population.

Crossover methods also assume that there is some degree of linkage between genes on the chromosome: that is, settings for certain genes in groups are strongly correlated to fitness improvement. For example, genes A and B might contribute to fitness only when they’re both set to 1: if either is set to 0, then the fact that the other is set to 1 doesn’t do anything.

Also note that crossover is not a global operator. If the only operator is crossover then (also from p42):

Eventually the population will converge, and often (unfortunately) prematurely
  converge, to copies of the same individual. At this stage there’s no escape: when an individual crosses over with itself, nothing new is generated.

For this reason, crossover is generally used together with some global mutation operator.

Crossover allows to combine two parents (vs. mutation, which only uses one parent). In some cases, it is useful (e.g., if you train a FPS bot, if one parent is good at shooting and another parent is good at moving, it makes sense to combine them). In some other cases, it is not.

When thinking about crossover its important to think about the fitness landscape.
Consider a hypothetical scenario where we are applying a genetic algorithm to find a solution that performs well at 2 tasks. This could be from Franck's example (moving and shooting) for an AI, or perhaps it could be predicted 2 outputs in a genetic machine learning scenario, but really most scenarios where GAs are applied are synonymous (even at solving a single task, there may be different aspects of the task to be addressed).
Suppose we had an individual, 1, that was performing reasonably well at both tasks, and we found a series of mutations which produced 2 new individuals, 2 and 3, which performed better than Individual 1 at tasks 1 and 2 respectively. Now while both of these are improvements, ideally we want to find a generally good solution, so we want to combine the features that we have been found to be beneficial.
This is where crossover comes in; by combining the genomes of Individuals 2 and 3, we may find some new individual which produces a mixture of their performances. While it is possible that such an individual could be produced by a series of mutations applied to Individual 2 or Individual 3, the landscape may simply not suit this (there may be no favorable mutations in that direction, for example).

You are partially right therefore; it may sometimes be the case that the benefits of crossover could be replicated with a series of mutations. Sometimes this may not be the case and crossover may smooth the fitness landscape of your GA, speeding up optimization and helping your GA escape local optima.

If (as should be the case) the mutation operator is global (i.e. capable of expressing all points in the search space) it is always possible to express the results of crossover via (some sequence of) mutations. However (as per my answer) the converse is not the case.

That is true, I just wished to illustrate the point made by you and Franck with an example :)


text:
Are there any machine learning techniques to detect coding standard violations?
Are there any machine learning techniques (such as deep learning or evolutionary algorithms) to detect coding standard violations? Which one would be more suitable?
I don't have any specific programming language in mind, but something similar to PHP_CodeSniffer (following these standards), but instead of using hardcoded rules, the algorithm should learn good techniques, but I'm not sure based on what training data. How would you approach the training session, any suggestions?

If the system claims that a piece of code has violated standards, then to be useful to the programmer, it really needs to provide more information than just a 'yes/no' classifier: you need some form of explanation about why it is claimed to be wrong.
Clearly ANNs aren't much use for that.
If I were tackling such a problem (and my suspicion is that a lot of effort could be spent trying and failing to reproduce coding standards which are already well-understood), then my inclination would be to use a more explicitly rule-based representation.
Possibilities include:

Genetic Programming
Learning Classifier Systems
Decision Trees

The ever-useful "Essentials of Metaheuristics" has a whole section on the evolution of rulesets. Obviously, nothing prevents you from initializing the evolutionary process with rules known to be useful.
As I point out here, with our current AI algorithms, the success of an  approach is very sensitive to human expertise/effort in feature selection/preprocessing, choice of training set etc, so creative experiment with this is vital.
Training set: how about two sets of negative and positive examples, consisting of (features extracted from) bad code and from a refactored version (respectively)?
One elementary choice of features would be to apply a bunch of code complexity metrics and have the learning algorithm combine those. The plus side of working in such a numeric domain is that the learning algorithm might readily find a gradient to exploit. The downside is that the rules (which are then likely of the form if mcabe > 2.8 etc) are still not as informative as might be desired.
For more complex rules (e.g. requiring if elseif else) you may want to extract your features from the abstract syntax tree. You could in principle use the entire tree but to my knowledge ML on graph and tree structures is still in relative infancy.

I don't think that providing more info other than an answer "yes" or "no" is necessarily useless. It can be useful because it might point out which parts of the code are potentially not following the standards.


text:
How does Google's self-driving car identify pedestrians?
Based on the article Google's self-driving cars can now spot cyclists: Sensors can read hand signals and predict rider's behaviour, Google's self-driving cars can spot cyclists, cars, road signs, markings, traffic lights, and pedestrians.
How exactly does it identify pedestrians? Is it based on face recognition, shape, size, distance, infrared signature?

The AI of the car uses sensor data to process all the data and classifies objects based on the size, shape and movement patterns. It can recognize surroundings from a 360 degree perspective by making predictions about vehicles, people and objects around it will move.
It can detect pedestrians, but as moving, column-shaped blurs of pixels, so it really cannot tell whether it's a rock or a crumpled piece of paper.

However it is programmed to determine certain patterns when a police officer has halted traffic or the car is being signaled to move forward.

It also recognizes cyclists as objects outlined in red and can slow down to let the cyclist enter into a lane.

Above images are provided by Chris Urmson who heads up Google's driverless car program.
Sources:

How Google's self-driving cars see the world
Hidden Obstacles for Google’s Self-Driving Cars
(video) Chris Urmson: How a driverless car sees the road

I see colorful pictures here, but nothing about 1) how they are generated and 2) how the car classifies what it sees. The "sources" are not suited to answer such questions, either.

@Raphael The question is how car is recognizing the people, if you're interested how they're generated or how it classifies, it's a topic for another question.

@Raphael These pictures are provided by Chris Urmson who's the project director.


text:
How should I encode the structure of a neural network into a genome?
For a deterministic problem space, I need to find a neural network with the optimal node and link structure. I want to use a genetic algorithm to simulate many neural networks to find the best network structure for the problem domain.
I've never used genetic algorithms for a task like this before. What are the practical considerations? Specifically, how should I encode the structure of a neural network into a genome?

Section 4.2 of "Essentials of Metaheuristics" has a wealth of information on alternative ways of encoding graph structures via Genetic Algorithms.
With particular regard to evolving ANNs, I would personally not be inclined to implement this sort of thing 'from scratch':
The field of neuroevolution has been around for some time, and the implementation some of the methods, such as Neuroevolution of Augmenting Topologies (NEAT) now incorporate the results of much practical experience.
According to the above link:

We also developed an extension to NEAT called HyperNEAT that can evolve neural networks with millions of connections and exploit geometric regularities in the task domain. The HyperNEAT Page includes links to publications and a general explanation of the approach.

That "Essentials of Metaheuristics" looks very interesting!  This is something that's actually on the roadmap for the M-automata, as pure MCTS is never optimal in M games.  From the metaheuristic wiki: "In computer science and mathematical optimization, a metaheuristic is a higher-level procedure or heuristic designed to find, generate, or select a heuristic (partial search algorithm) that may provide a sufficiently good solution to an optimization problem, especially with incomplete or imperfect information or limited computation capacity."

Using evolutionary algorithms to evolve neural networks is called neuroevolution.
Some neuroevolution algorithms optimize only the weights of a neural network with fixed topology. That sounds not like what you want. Other neuroevolution algorithms optimize both the weights and topology of a neural net. These kinds of algorithms seem more appropriate for your aims, and are sometimes called TWEANNs (Topology and Weight Evolving Neural Networks).
One popular algorithm is called NEAT, and is probably a good place to start, if only because there are a multitude of implementations, one of which hopefully is written in your favorite language. That would at least give you a baseline to work with.
NEAT encodes a neural network genome directly as a graph structure. Mutations can operate on the structure of the network by adding new links (by connecting two nodes not previously connected) or new nodes (by splitting an existing connection), or can operate only on changing the weights associated with edges in the graphs (called mutating the weights).
To give you an idea of the order of magnitude of the sizes of ANNs this particular algorithm works with, it would likely struggle with more than 100 or 200 nodes.
There are more scalable TWEANNs, but they're more complex and make assumptions about the kinds of structures they generate that may not always be productive in practice. For example, another way to encode the structure of a neural network, is as the product of a seed pattern that is repeatedly expanded by a grammar (e.g. an L-system). You can much more easily explore larger structures, but because they're generated by a grammar they'll have a characteristic self-repeating sort of feel. HyperNEAT is a popular extension of NEAT that makes a different sort of assumption (that patterns of weights can be easily expressed as a function of geometry), and can scale to ANNs with millions of connections when that assumption well-fits a particular domain.
There are a few survey papers linked in the top link if you want to observe a greater variety of techniques.


text:
Why would someone want to simulate emotional desicion-making within an AI?
The Mario Lives! video (and its follow-up video, Mario Becomes Social!) showcases an AI unit that is able to simulate emotional decision-making within a virtual world, and can enter into "emotional states" such as curiosity, hunger, happiness, and fear. While this seems cool and exciting (especially for video game AI), I am confused about how this would be useful in real-world scenarios.
What would be the point of building autonomous actors that would behave based on these emotional states, instead of simply knowing what they should do (either by hardcoding in the rules, or learning the rules through machine learning)?

Unless you provide some more information on the actual mechanisms used to model emotion, it's hard to say how this actually differs from the learned or hard-coded approach: for example, it could be that, despite the presence of the evocative labels 'happiness' and 'fear', the approach is essentially hardcoded.

Humans have poor understanding of emotional rules. Probably every poster on here has experienced greatly misreading another individual emotionally. Further, people often don't act emotionally how they would expect themselves to act, for example we have all experienced frustration at someone else's irrational concerns and yet we are all guilty of holding irrational concerns of our own. This is the crux of why hard-coded emotional rules do not work - we do not have an understanding of what emotional 'rules' make someone feel real.
By moving towards autonomous state-based actors we move away from this issue. The actor's state transitions will of course be defined rules (hard-coded or learnt) but by abstracting the actor's emotional state from the specific context (e.g. specific actions trigger emotional state transitions which trigger responses, instead of a direct response to a specific action), the programmer prevents them-self from projecting their own beliefs/emotions/logic onto the actor.
Further, autonomous actors are more extendable. Consider an autonomous actor that is hard-coded to move towards 'upset' and 'angry' emotional states when experiencing 'pain'. Simply by associating a new world action with 'pain', one can trigger an emotional response from an autonomous actor that has not experienced that action before. When working in hard-coded emotional rules, this would not be possible.


text:
Is there any board game where a human can still beat an AI?
Significant AI vs human board game matches include:

chess: Deep Blue vs Kasparov in 1996,
go: DeepMind AlphaGo vs Lee Sedol in 2016,

which demonstrated that AI challenged and defeated professional players.
Are there known board games left where a human can still win against an AI? I mean based on the final outcome of authoritative famous matches, where there is still the same board game where AI cannot beat a world champion of that game.

Well, games like Ludo are almost entirely based on luck. So there's a change of about 50% that a human would win from an "AI" in this case.

Can't a human + computer still beat any AI system in these games?  Keeping overall computational power the same in both cases, of course.  Kasparov said he staged human vs. AI chess competitions, and amateurs + low powered computers beat experts and supercomputers.

For many years, the focus has been on games with perfect information. That is, in Chess and Go both of us are looking at the same board. In something like Poker, you have information that I don't have and I have information that you don't have, and so for either of us to make sense of each other's actions we need to model what hidden information the other player has, and also manage how we leak our hidden information. (A poker bot whose hand strength could be trivially determined from its bets will be easier to beat than a poker bot that doesn't.)
Current research is switching to tackling games with imperfect information. Deepmind, for example, has said they might approach Starcraft next.
I don't see too much difference between video games and board games, and there are several good reasons to switch to video games for games with imperfect information.
One is that if you want to beat the best human to be a major victory, there needs to be a pyramid of skill that human is atop of--it'll be harder to unseat the top Starcraft champion than the top Warcraft champion, even though the bots might be comparably difficult to code, just because humans have tried harder at Starcraft.
Another is that many games with imperfect information deal with reading faces and concealing information, which an AI would have an unnatural advantage at; for multiplayer video games, players normally interact with each other through a server as an intermediary and so the competition will be more normal.

Not all games (or even board games) are computationally algorithmic. Even the least skilled player is likely to trounce the hottest pattern-matching algorithm in a game of Pictionary (for example).

If you want to say that the movement of pieces upon successful completion of a task is only ancelary to the object of the game, than your answer will be largely self-selecting. A sufficiently sophisticated algorithm will brute force a computational problem better than human intuition… eventually.

Citation needed.

Yeah, I think Pictionary might be pretty easy to beat these days. Since it's team-based it's not clear what a human vs. computer matchup would look like, though.

Artificially intelligent computer programs should be able to be at the same level or beat humans at every game that we play.  This is because games follow rules that are scriptable, and artificial intelligence is designed to focus on one specific game and learn from its failures.  The difference between humans and artificial intelligence is that artificial intelligence focuses on one specific task like learning to master Go while our brain is dedicated to mastering multiple tasks like...living.  Even Arimaa, a game designed to be difficult for artificially intelligent systems was beaten by a bot called Sharp: https://en.wikipedia.org/wiki/Arimaa.

Many games have social elements. How do you script those?

This is a good point, but I am not sure I understand.  Many of the games I have played or seen artificial intelligence play have clearly defined rules.  By social, do you mean that the rules can change based on the people playing?  For example, my  cousins play the card game "War" a bit differently as they use the "Joker" card as the highest card while my brother and I don't use that card.  Can you clarify what you mean by social?

In Werewolf, the whole gameplay revolves around people talking and deciding whom to take out; there are barely any rules beyond that. But even if you have rules, think of cooperative games like Pandemic; how does the AI discuss? What about games that involve moral decisions like e.g. This War of Mine? In summary, I think your claim "computer programs [...] beat humans at every game that we play" is wrong -- precisely because not every game follows scriptable rules.

In some sense, Go returned to this category in 2023 with the discovery of an inherent weakness in superhuman Go-playing AIs. There is a particular pattern that the AI consistently misinterprets, so forcing that kind of position leads the AI to play blunders that are clear even to a Go novice. Figure J.2 in the paper shows an example. The idea is to create a dead group and let the AI enclose it, but not capture it (which is perfectly correct behaviour). The AI them mistakenly considers the enclosing group unconditionally alive, even though for that it would have to actually capture the dead group. It then lets the enclosing group be captured. The loss involves a misread capture race; the particular blunder move is clear even to a Go novice.
Importantly, though the strategy was discovered through training an adversarial AI, it can be played by an unaided human. Also importantly, this is not a trivially fixable bug. The main author of KataGo confirms this here, pointing out also that KataGo, a commonly-used superhuman Go AI, sometimes misreads specific positions that arise within regular gameplay when they get out of the space explored in self-play. This means that superhuman Go AIs require various ad hoc additions to their training. The original paper discvered two exploits, one that relied on a particular ruleset and the more legitimate one described above, but it's possible that there are yet more exploits that would be found next if this got patched.
Since there aren't big human-versus-AI events in Go these days, it's hard to evaluate how (un)fixable this problem is, but professional Go players take it seriously. Here is a dan-level professional playing the strategy out and reflecting on what it means for the game as a piece of culture.
As an aside, I find this fascinating; it's what got me into Go, through this essay (which I also recommend, even if you don't know Go at all). I learned the game to understand what this kind of exploit means, and I think I've figured it out. It shows one component of human intelligence that we have not replicated so far in AI: humans have a kind of "constant vigilance". A human encountering this strategy would think "this is weird, my enemy is up to something". They would spot where the trap seemed to lie, and play the obvious safeguarding move to block it, even if they never saw anything like it before. I have no idea how humans do this. I guess it shows that intelligence is much more of an ill-defined, ad-hoc, improvisational kind of thing than generally assumed.


text:
Is reinforcement learning needed to create Strong AI?
By reinforcement learning, I don't mean the class of machine learning algorithms such as DeepQ, etc. I have in mind the general concept of learning based on rewards and punishment.
Is it possible to create a Strong AI that does not rely on learning by reinforcement, or is reinforcement learning a requirement for artificial intelligence?
The existence of rewards and punishment implies the existence of favorable and unfavorable world-states. Must intelligence in general and artificial intelligence, in particular, have a way of classifying world-states as favorable or unfavorable?

Your last question asks a different question, "... have a value system?" which is a lot different from reinforcement learning.

Here is a related/similar question.

It's impossible to give a definitive 'yes' answer to your question, since that would require proving that alternatives cannot exist.
More philosophically, it depends on what you mean by "preference over world states":
However counter-intuitive it might seem, it is conceivably possible to create Strong AI purely from local condition-action rules, in which there is no global concept of 'preference value' and/or no integrated notion of 'world state'.

Simply put, we don't know how to create Strong Artificial Intelligence yet, so we don't know what is or isn't required to create it.  At best we can engage in "informed speculation", in which case I'd say that the answer is more likely "yes" than "no".  But that's basically just a hunch.
If you're interested in a pretty good overview of what "pieces" might be required to create Strong AI, and if you haven't read it yet, Pedro Domingos' book The Master Algorithm might be of interest.

Introspect! Do you need to know what's good/bad, pleasurable/painful or so, in order to understand and/or learn?
I am a human, hence a general intelligence, and so are you. So know thyself! I can tell for myself that I have different ways to understand and learn; some may be similar to reinforcement learning. Esp. the ~automatic ~innate ~unconcious ones, like motor movement, remembering tasty food and many other primitive functions.
But I can also understand things through ~intentional ~analytical ~logical thought; which some may call pure reason (Immanuel Kant).
Yet, you don't need to hear all that, since have it already in your own mind.

Simply yes, but it can lead to over fixing of the NN.
Humans favour not dying, which is only realised once a consequence is defined for the system to realise that death is an unfavorable result. Which can be train vai observation. Allow your system to observe between 2 or more separate people/systems. Then allow opportunity to test in a safe environment with the pre  existing info of the consequences that may follow, provind that if the system makes a mistake in the test/safe environment it will be saved  unknownly and then informed that it made a mistake, the place system in an unsafe world in same conditions, informing it that if something happens it will die. That is the way humans grow up, and we've lasted very long with this technic.
I'm an AI Researcher and Software Engineer for the past 7 years.

Thanks for your answer. I don't doubt the power of reinforcement learning. Could you elaborate on AI that doesn't use reinforcement learning?

Learning is a trial and error application. Unless you provide an acurate dataset for the AI, and the applicable environment for it to apply this information. This would be a closed system and would allow the AI not to in anyway harm itself, a good example is a user tracking AI or a chat  bot, neither can be damaged, if provided with enough information that learning would not be needed anymore, it could just understand and implement/act to this new data but not learn from it as it has enough to exist as a closed system


text:
Is there a way to define the boundaries of the optimal size of a training set?
At a related question in Computer Science SE, a user told:

Neural networks typically require a large training set.

Is there a way to define the boundaries of the "optimal" size of a training set in the general case?
When I was learning about fuzzy logic, I've heard some rules of thumb that involved examining the mathematical composition of the problem and using that to define the number of fuzzy sets.
Is there such a method that can be applicable for an already defined neural network architecture?

For a finite value to be 'optimal,' typically you need some benefit from more paired up with some cost for more, and eventually the lines cross because the benefit decreases and the cost increases.
Most models will have a reduction in error with more training data, that asymptotically approaches the best the model can do. See this image (from here) as an example:

The costs of training data are also somewhat obvious; data is costly to obtain, to store, and to move. (Assuming model complexity stays constant, the actual cost of storing, moving, and using the model remains the same, since the weights in the model are just being tuned.)
So at some point the slope of the error-reduction curve becomes horizontal enough that more data points are costlier than they're worth, and that's the optimal amount of training data.

In general, the larger the training set, the better. See The Unreasonable effectiveness of Data, though this article is quite dated (written in 2009). Xavier Amatriain, a researcher at Netflix has a Quora answer where he discusses that more data can sometimes hurt algorithms.
For deep neural networks in particular, it does not seem that we have hit these limits yet.


text:
How do capsule neural networks work?
Geoffrey Hinton has been researching something he calls "capsules theory" in neural networks. What is it? How do capsule neural networks work?

Now the paper can be viewed at: Dynamic Routing Between Capsules by Sara Sabour, Nicholas Frosst and Geoffrey E. Hinton.

It appears to not be published yet; the best available online are these slides for this talk. (Several people reference an earlier talk with this link, but sadly it's broken at time of writing this answer.)
My impression is that it's an attempt to formalize and abstract the creation of subnetworks inside a neural network. That is, if you look at a standard neural network, layers are fully connected (that is, every neuron in layer 1 has access to every neuron in layer 0, and is itself accessed by every neuron in layer 2). But this isn't obviously useful; one might instead have, say, n parallel stacks of layers (the 'capsules') that each specializes on some separate task (which may itself require more than one layer to complete successfully).
If I'm imagining its results correctly, this more sophisticated graph topology seems like something that could easily increase both the effectiveness and the interpretability of the resulting network.

The paper is now (November 7, 2017) published: Dynamic Routing Between Capsules.

To supplement the previous answer: there is a paper on this that is mostly about learning low-level capsules from raw data, but explains Hinton's conception of a capsule in its introductory section: http://www.cs.toronto.edu/~fritz/absps/transauto6.pdf
It's also worth noting that the link to the MIT talk in the answer above seems to be working again.
According to Hinton, a "capsule" is a subset of neurons within a layer that outputs both an "instantiation parameter" indicating whether an entity is present within a limited domain and a vector of "pose parameters" specifying the pose of the entity relative to a canonical version.
The parameters output by low-level capsules are converted into predictions for the pose of the entities represented by higher-level capsules, which are activated if the predictions agree and output their own parameters (the higher-level pose parameters being averages of the predictions received).
Hinton speculates that this high-dimensional coincidence detection is what mini-column organization in the brain is for. His main goal seems to be replacing the max pooling used in convolutional networks, in which deeper layers lose information about pose.

Capsule networks try to mimic Hinton's observations of the human brain on the machine. The motivation stems from the fact that neural networks needed better modeling of the spatial relationships of the parts. Instead of modeling the co-existence, disregarding the relative positioning, capsule-nets try to model the global relative transformations of different sub-parts along a hierarchy. This is the eqivariance vs. invariance trade-off, as explained above by others.
These networks therefore include somewhat a viewpoint / orientation awareness and respond differently to different orientations. This property makes them more discriminative, while potentially introducing the capability to perform pose estimation as the latent-space features contain interpretable, pose specific details.
All this is accomplished by including a nested layer called capsules within the layer, instead of concatenating yet another layer in the network. These capsules can provide vector output instead of a scalar one per node.
The crucial contribution of the paper is the dynamic routing which replaces the standard max-pooling by a smart strategy. This algorithm applies a mean-shift clustering on the capsule outputs to ensure that the output gets sent only to the appropriate parent in the layer above.
Authors also couple the contributions with a margin loss and reconstruction loss, which simultaneously help in learning the task better and show state of the art results on MNIST.
The recent-paper is named Dynamic Routing Between Capsules
 and is available on Arxiv: https://arxiv.org/pdf/1710.09829.pdf .

One of the major advantages of convolutional neural networks is their invariance to translation. However, this invariance comes with a price, that is, it does not consider how different features are related to each other. For example, if we have a picture of a face, a CNN will have difficulties distinguishing the relationship between the "mouth" feature and "nose" feature. The max-pooling layers are the main reason for this effect, because, when we use max-pooling layers, we lose the precise locations of the mouth and nose, and we cannot say how they are related to each other.
Capsules try to keep the advantage of CNN and fix this drawback in two ways

Invariance: quoting from this paper

When the capsule is working properly, the probability of the visual entity being present is locally invariant – it does not change as the entity moves over the manifold of possible appearances within the limited domain covered by the capsule.

In other words, capsule takes into account the existence of the specific feature that we are looking for, like the mouth or nose. This property makes sure that capsules are translation invariant the same that CNNs are.

Equivariance: instead of making the feature translation invariance, the capsule will make it translation-equivariant or viewpoint-equivariant. In other words, as the feature moves and changes its position in the image, the feature vector representation will also change in the same way which makes it equivariant. This property of capsules tries to solve the drawback of max-pooling layers that I mentioned at the beginning.

In the abstract of the paper Dynamic Routing between Capsules (November 7, 2017) that formally introduced capsule neural networks, the authors write

A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule
  becomes active. We show that a discriminatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher-level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.

Capsule Networks have two key ideas:

the first idea is how to represent multi-dimensional entities. Capsule Networks does this by grouping these properties of a feature together ("capsules").
the second is that you activate higher-level features by agreement between lower-level features ("routing by agreement").

First, Capsule Networks partition the image into regions subsets.
For each of these regions, it assumes that there is at most one instance of a single feature, called a Capsule.
A Capsule is able to represent an instance of a feature (but only one) and is able to represent all the different properties of that feature, e.g., its (x,y) coordinates, its colour, its movement etc.
The difference from Convolutional Neural Networks (CNNs) is that the Capsules bundle the neurons into groups with multi-dimensional properties, whereas in CNNs the neurons represent single, unrelated scalar properties.
This structured Capsule representation allows you to do "routing by agreement".
To understand this, lets look at the example of a face detector. Here, you could have capsules representing "mouth", "eye", "nose" etc.
Since the Capsules are multi-dimensional you also train them to predict the parameters for the entire face.
Now, if the "mouth", "nose" and "eye" Capsules agree about the parameters of the face we have a very strong signal that this is a good prediction since accidental agreement in a high-dimensional space like a neural network is very unlikely.
You use this to stack the Capsules into deep networks where the activation of higher-level Capsules are conditioned on agreement between the lower-level Capsules (e.g. the Face capsule being activated by agreement on the face position between the Nose, Mouth, Eye Capsules in the earlier, lower-level layer).
In contrast to regular feed-forward nets this requires a bit of iteration in the forward pass through the network, but you can still use back-propagation train it.
It is an interesting way to add a bit of structure to the data. So far, it looks like they are able to provide better generalization from limited training data.

A few sketches on paper could be helpful

The paper, Dynamic Routing Between Capsules by Sabour, Frosst and Hinton has some illustrations.