text:
Is there a naming convention for network weights for multilayer networks?
In the diagram below, although the flow of information happens from the input to output layer, the labeling of weights appears reverse. Eg: For the arrow flowing from X3 to the fourth hidden layer node has the weight labeled as W(1,0) and W(4,3) instead of W(0,1) and W(3,4) which would indicate data flowing from the 3rd node of the 0'th layer to the 4th node of the 1st layer.

One of my neural networks teachers did not emphasize on this convention at all. Another teacher made it a point to emphasize on it.
Is there a reason there is such an un-intuitive convention and is there really a convention?

@DuttaA: it should be always welcome student who thinks about things and arise questions to himself or publicly.

@pasabaporaqui Humans,lets get serious here.someone in Microsoft is working on ai project,however if it requires him/her to query through SE knowledge base,then we shouldn't bring here students course work but rather world real problems to solve, out of school.Humans,hope this can save your planetary civilisation.

@quintumnia: If I must choice help students or Mycrosoft ... . But, being serious, taken into account that first comment has been similar to "it is a personal choice" ', I think the question is on-topic and  deserves an answer explaining the motivation under usual notation.

@pasabaporaqui If you analyze and comprehend this question very well,then it's right fit in cross validated community for effective feedback.Try to analyze it !

@DuttaA: What is, according to you, a personal preference in my answer? Say that matrix notation is used in NN design ? The math notation for matrix elements ?

Guys, students are the ones going to spread false info in the professional world if concepts and standards are not clear. I am a seasoned StackExchange user and a working professional doing a part time course. A proper answer to this question will ensure that anyone Googling this question in future wont end up with a nonsensical concept. I wanted to know if it is a standard notation uniformly used and recognized across the world.

@Nav: yes, there are a standard/common/usual/recommended notation, the one that your teachers are using, for the reason I've explain in my answer.

When the system grows matrix notation is used, as a=Wx, being a (input to activation function in hidden layer) and x (values from input layer) column vectors, transpose of (a1,a2,...a_m) and (x1,x2,...,x_n), and W a m-by-n matrix of dimensions m rows and n columns. The standard way to denote matrix elements is w(i,j) where "i" is the row number and "j" column number:

(from wiki)
For this reason, the weight that applies to h4 from x3 is element in row 4 column 3 of the matrix W, that is, W(4,3) ( as your teachers advocates but with a sad lack of ability to explain ).
In your example:

Note: things are a few more complex when x1, x2, ... are itself vectors, but final conclusion is the same.
( PS: URGENT to allow latex notation on this stack exchange ! )

Indeed there seems to be a naming convention but a more elaborate and clear explanation would be appreciated. Even the author here has not explained the reason properly http://neuralnetworksanddeeplearning.com/chap2.html

Which part of current explanation do you think must be expanded?

Depicting the weights of the ANN in my question as matrices could make it more intuitive. Thanks.

@Nav: answer edited

Lovely! That's a lot clearer now. Thanks Pasaba.


text:
How are connection weights "perturbed"?
I am reading through the NEAT paper here. On page 14 of the PDF, there is this quote about mutation:

There was an 80% chance of a genome having its connection weights mutated, in which case each weight had a 90% chance of being uniformly perturbed and a 10% chance of being assigned a new random value.

What exactly does it mean to perturb weights? What is uniform vs. nonuniform perturbation?
Is there an established method to do this? I am imagining the process as multiplying each connection weight by a random number, but I'm unfamiliar with the term.

Perturbed here means adding a small random value to the weight. That random value comes from a uniform distribution or from a gaussian (or any distribution really).
Imagine just nudging the weight by a little.
It’s done to overcome the problem of local minima where models can get stuck with a good set of weights but not the best set of weights. By perturbing the weights a little, the model has a chance of finding a better set of weights. Searching for methods that deal with local minima like stochastic gradient descent will give you a better intuition.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
A way to give more weight to particular data?
Let us for these purposes say with are working with any feed forward neural network.
Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or important to our underlying representation. Is there anyway to add that “weighting” to our data?

Please clarify what algorithm you are using, the answer will vary significantly.

You can add a bias to your step function

You could add some sort of "Experience replay" algorithm to pass the same data several times.

Yes, You just weight the entries of loss function like sum( Wi(li-t2)**2) for MSE. Most frameworks provides this ability. For example in keras fit method has a parameter - sample_weight: Optional Numpy array of weights for the training samples.

I think you are looking for "Class Balancing" if I understand your question correctly. In most frameworks you can pass an additional weights tensor/array to the loss function/criterion. This is essentially just a weighting factor for each loss term in the sum of the total loss. Through that you achieve that the error/loss for some classes are weighted higher than other classes. Usually you weight classes higher that are less present in your dataset to emphasize that these are not "ignored" by your network over other classes.
Second what you can do if it's your data samples that are imbalanced and not the trained classes you can of course weight the data sampling of your dataloader. Instead of using a uniform distribution over all data samples you can weight specific samples higher or lower so that they are shown more/less often to the network.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is the difference between ConvLSTM and CNN LSTM?
What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?

After doing a bit of research I found that the LSTM whose gates perform convolutions is called ConvLSTM.
The term CNN LSTM is loose and may mean stacking up LSTM on top of CNN for tasks like video classification
Reddit thread discussing this


text:
Which is best: evaluation of states or probability of moves?
If you have a game and you are training an AI there seems to be two ways to do it.
First you take the game-state and a possible move and evaluate whether this move would be good or bad:
(1) GAME_STATE + POSSIBLE_MOVE  --> Good or bad?
The second is to take the game state and get probabilities of every conceivable move:
(2) GAME_STATE ---> Probabilities for each move
It seems that both models are used. I.e. in language and RNN might use (2) to find the probabilities for each next word or letter. But AlphaZero might use (1). Noting also that in a game like chess GAME_STATE + POSSIBLE_MOVE = NEW_GAME_STATE. Whereas in some games you might not know the result of your move.
Which do you think is the best method? Which is the best way to do AI? Or some combination of the two?

Actually AlphaGo uses 1 + 2 (plus a look-ahead search to help refine them), and this is relatively common too (search "Actor-Critic").

Ah, I see! I don't think AlphaZero uses Actor-Critic though?

Yes you are right, it is not Actor-Critic (as far as I understand it). It does however both generate policies directly and evaluate board positions. The tree search (MCTS) is what links the two.

Which do you think is the best method?

As with most machine learning, each approach has its strengths and weaknesses, and other than a little bit of intuition:

Policy-based methods are strong in large or continuous action spaces, and/or where there is a simple relationship between state and optimal action. E.g. controlling a robotic arm with continuous action space.
Value-based methods are strong where there is a simple relationship between state and value under an optimal policy. E.g. in a maze game.

It may not always be clear which is best, in which case experimentation is required. If using neural networks, there will then be a large number of hyper-parameters on each approach, so it may be hard to come to a strict conclusion about which is better. Although you can include "easy to find a working model" or "robust for a range of hyper-parameter values" as benefits of any type of model if you wish - these are important practical benefits of any approach, developers rarely want to do 100s of experiments to tune a learning rate parameter for example.

Which is the best way to do AI? Or some combination of the two?

Actor-Critic, as seen in Asynchronous Advantage Actor Critic. A3C and A2C (deterministic variant of A3C) is producing current state-of-the-art results in video games. This is a combination of both approaches, where the agent maintains two related models - one, the Actor, generates a policy directly by looking at the state, and the second, the Critic, tracks the estimated value of each state. Often, these two models share some parameters - e.g. using neural networks, the initial layers may be the same for both.


text:
Is the following neural network architecture considered deep learning?
I am working in the following neural network architecture, I am using keras and TensorFlow as a back-end.
It is composed by the following, embedding of words, then I added a layer of
Long Short-Term Memory (LSTM) neural networks, one layer of output and finally. I am using the softmax activation function.
model = Sequential()
model.add(Embedding(MAX_NB_WORDS, 64, dropout=0.2))
model.add(LSTM(64, dropout_W=0.2, dropout_U=0.2))
model.add(Dense(8))
model.add(Activation('softmax'))

I have the following question, if I am getting a model through this code, could the final product be called a deep learning model?,
I know that this code is very small however there is a lot of computations that the machine is making on the background.

"Deep learning" is not formally defined. However, typically even simple RNNs are taught as advanced neural network subject alongside other topics labelled "deep learning".
Technically, given the time dimension, the depth of a RNN can include many layers of processing (as opposed to many layers of parameters). As such, some of the knowledge and experience used to help with deep feed-forward networks also applies to RNNs. You could consider the LSTM architecture one such thing, because it is designed to address the vanishing gradient problem that plagues simpler RNN architectures.
So, yes you can call your model a "deep learning model" and have that generally accepted.
I'd be slightly concerned if anyone important to the success of a project thought that label was a big deal - either placed on your CV as or as a buzzword on a resulting product. However, it is not unrealistic marketing  because it is essentially true.


text:
How to evaluate a NEAT neural network?
I'm trying to write my own implementation of NEAT and I'm stuck on the network evaluate function, which calculates the output of the network.
NEAT as you may know contains a group of neural networks with continuously evolving topologies by the addition of new nodes and new connections. But with the addition of new connections between previously unconnected nodes, I see a problem that will occur when I go to evaluate, let me explain with an example:

INPUTS = 2 yellow nodes
HIDDEN = 3 blue nodes
OUTPUT = 1 red node

In the image a new connection has been added connecting node3 to node5, how can I calculate the output for node5 if I have not yet calculated the output for node3, which depends on the output from node5?
(not considering activation functions)
node5 output =  (1 * 0.5) + (1 * 0.2) + (node3 output * 0.8)
node3 output =  ((node5 output * 0.7) * 0.4)

Consider the execution order, 5 will have an invalid value because it hasn't been set form 3 yet. However the second time around it should have a value set. The invalid value should falloff after sufficient training.
0 -> 5
1 -> 5
5 -> 2
2 -> 3
3 -> 4
3 -> 5
RESTART
0 -> 5
1 -> 5

Thanks for the reply, okay, I'm not sure if that answers my question, node3 cannot be calculated and node5 cannot be calculated, because they both depend on each other, how can you evaluate either node? Am I missing something?

I don't see how it can't be calculated. The first unresolved value you get is when calculating 5 (3 -> 5).
Considering 0 has a value of .5, 1 has a value of .8, and 3 is unresolved, you may consider the following
v5 = .5 * w0 + .8 * w1 + (3?) * w3
Where the 3? is an unresolved value. I think the hack here is to initialize it to zero so your equation becomes
v5 = .5 * w0 + .8 * w1 + 0 * w3
Reduces to
v5 = .5 * w0 + .8 * w1
This can calculate a value for 5 that can be used to calculate 2. You can then calculate 3 from 2. Then the next time 5 is calculated the value wouldn't be zero

That makes sense, ty.

@Chris: NEAT does not usually allow backwards connections and loops AFAIK, connections are normally feed-forward. So you are creating more work for yourself here. In principle it can be made to work, and for all I know there are some notes on implementing recurrent NNs in NEAT somewhere. But you may want to consider starting with feed-forward only.

@NeilSlater, So you're saying hidden nodes should not connect with other hidden nodes? Just inputs to hidden, inputs to output and hidden to output?

@Chris - no, I am saying that nodes should not link in a way that causes loops.

@Chris: Actually, from re-reading the documentation, recurrent connections can be quite common and are not necessarily defended against in original NEAT implementations. It seems that the algorithm relies on weeding them out if they are not useful.

Following is the pseudo code of the NEAT's network evaluation (converted from original source code),
Until all the outputs are active
    for all non-sensor nodes
        activate node
        sum the input
    for all non-sensor and active nodes
        calculate the output

Note that there is no recursion for feed forwarding concepts according to the original author.

I can think of two possible ways of enforcing NEAT to create a feed forward network. One elegant one and one a little more cumbersome one;

Only allow the "add connection" mutation to connect a node with another node that have a higher maximum distance from an input node. This should result in feed forward network, without much extra work. (Emergent properties are great!)
Run as you did and create a fully connected network with NEAT and then prune it during a forward pass. After creating the network, run through it and remove connections that try to connect to a node already used in the forward pass (example 3->5). Alternatively just remove unused input connections to nodes during the forward pass. Given how NEAT mutates, it should not be possible that you remove a vital connection and cut the the network in two. This property of NEAT make sure your signal will always be able to reach the output, even if you remove those "backwards pointing" connections.

I believe these should work, however i have not tested them.
The original NEAT paper assumed a feed forward ANN, even though its implementation as described would result in a fully connected network. I think it was just an assumption of the paradigm they worked in. The confusion is fully understandable.

Hello chris i am also implementing this algorithm from scratch and the way i go about activating my mlp net is as follows:
I instantiate a list of nodes(actives), this is set to all input nodes initially, i then pass that to a function that initializes an empty list(next actives) and proceeds to loop through each set of conns for each node in the actives list, it adds each "to" node from those connections to the "next actives) list unless its an output node or has already been activated, once all the "actives" list is looped through, i call the function again this time passing "next actives" as the actives list unless "next actives" is still empty after then i know the net has been fully activated.
in this scenario the connection from node three to node five would be evaluated but it would not be added to the next actives list because it had already been activated, preventing an infinite loop.

In my implementation, I used a recursion system to calculate the output nodes. It works as follows:

Assume a feed-forward network

Only allow the "add connection" mutation to connect a node with another node >that have a higher maximum distance from an input node. This should result in >feed forward network, without much extra work. (Emergent properties are great!)

Define function x, a recursive function that takes in a node number
Define function y, a second function that takes in a node and returns all the connections with that node as an output

In the recursive function:

Call function y
Call function x on function y outputs
If the parameter for x is any input node, return the node value.

This was the most elegant way of implementing I could think of, and its a lot simpler than explicitly tracking all of the connections.

Okay, so instead of telling you to just not have recurrent connections, i'm actually going to tell you how to identify them.
First thing you need to know is that recurrent connections are calculated after all other connections and neurons. So which connection is recurrent and which is not depends on the order of calculation of your NN.
Also, the first time when you put data into the system, we'll just assume that every connection is zero, otherwise some or all neurons can't be calculated.
Lets say we have this neural network:
Neural Network
We devide this network into 3 layers (even though conceptually it has 4 layers):
Input Layer  [1, 2]
Hidden Layer [5, 6, 7]
Output Layer [3, 4]

First rule: All outputs from the output layer are recurrent connections.
Second rule: All outputs from the input layer may be calculated first.
We create two arrays. One containing the order of calculation of all neurons and connections and one containing all the (potentially) recurrent connections.
Right now these arrays look somewhat like this:
Order of
calculation: [1->5, 2->7 ]

Recurrent:   [ ]

Now we begin by looking at the output layer. Can we calculate Neuron 3? No? Because 6 is missing. Can we calculate 6? No? Because 5 is missing. And so on. It looks somewhat like this:
3, 6, 5, 7

The problem is that we are now stuck in a loop. So we introduce a temporary array storing all the neuron id's that we already visited:
[3, 6, 5, 7]

Now we ask: Can we calculate 7? No, because 6 is missing. But we already visited 6...
[3, 6, 5, 7,] <- 6

Third rule is: When you visit a neuron that has already been visited before, set the connection that you followed to this neuron as a recurrent connection.
Now your arrays look like this:
Order of
calculation: [1->5, 2->7 ]

Recurrent:   [6->7 ]

Now you finish the process and in the end join the order of calculation array with your recurrent array so, that the recurrent array follows after the other array.
It looks somethat like this:
[1->5, 2->7, 7, 7->4, 7->5, 5, 5->6, 6, 6->3, 3, 4, 6->7]

Let's assume we have [x->y, y]
Where x->y is the calculation of x*weight(x->y)
And
Where y is the calculation of Sum(of inputs to y). So in this case Sum(x->y) or just x->y.
There are still some problems to solve here. For example: What if the only input of a neuron is a recurrent connection? But i guess you'll be able to solve this problem on your own...

It's preferable that answers are self-contained, so, rather than linking to another post (in another website), I suggest that you provide the necessary info to answer the question here, in this answer.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Where do 'random seeds' get used in deep neural networks?
I know that when creating neural networks it's standard practice to create a 'random seed' so that you can get producible results in your models. I have a couple of questions regarding this:

Is the seed just something that is used in the 'learning' phase of the network or does it get saved? i.e. is it saved into the model itself and used by others if they decide to implement a model you created?
Does it matter what you choose to be the seed? Should the number have a certain length?
At what step of the creation of a model does this seed get used and how does it get used?

Other information about 'random seeds' would be welcomed! But these are my general questions.

I suppose the most common part where it will be used is in the initialization of weights before training; the best ways currently known to do that involve randomness.
If you use Dropout during training (randomly setting some activation levels to zero to combat overfitting), that also involves randomness, so your seed could also have influence there. Dropout should not be used anymore after training, although it could if you accidentally implement it to be used there. If you don't make that mistake though, your seed here should also only matter during training.
Depending on implementation, I suppose the seed could also have influence on random ordering of input data between epochs / random selection of minibatches from the training dataset during training. Of course, this is very much implementation-dependent. If you end up implement that kind of data processing yourself and don't do it through some framework, you'll also be the one who determines what random seed has influence on that process.
In general (barring any special cases that I'm unaware of), a Neural Network should behave deterministically after training; if you give it the same input, it should provide the same output, your random seed should no longer have influence after training.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is it possible to classify songs by genres based on spectrograms?
Is it possible to categories songs based on their spectrograms using image recognition or would there need to  more features? I was thinking that the spectrograms might also run into problems with EDM songs. Such as House music being closely related to their sounds. Would there have to be immense amount of data? I was thinking of using a CNN.

It is possible to use spectrograms for genres classification. See relevant articles Music genre recognition using spectrograms or blog post here. The later link did use CNN for this task.
If you need training data, GTZAN or FMA could be your starting point.


text:
Can TensorFlow minimize "symbolically"
From https://stackoverflow.com/questions/36370129/does-tensorflow-use-automatic-or-symbolic-gradients, I understood TensorFlow requires all the operations in the Graph to be explicit formulas (instead of black-boxes, such as raw python functions) to do Automatic Differentiation. Then it will do some kind of Gradient Descent based on that to minimization.
I'm wondering, since it already know all the explicit formulas, can it directly find out the minimum by examining the equation itself? Like computing the points where gradient is zero or do not exist, then do some kind of processing to find out the minimum.
I found it is simple to do this "symbolic minimization" above with few variables such as minimizing Σ(a_i - v)^2 where v is the trainable variable an a_i are all the training samples. I'm not sure is there a general way though.

If by "symbolic" you mean finding an analytical solution, that is, an equation for each weight, then the answer is no. The example you chose results in a system linear equations, which can be solved analytically. However once you introduce non linearities (by using activation functions with more than one layer), most non trivial cases will have no analytical solution and will need to be solved numerically. This is not a problem specific to tensorflow, it is  a mathematical issue, it will not be possible on any language, current or future. Unless there is some revolution in math first.


text:
What are good parameters of an encoder?
I am trying to assess an encoder in my autoencoder. I can not seem to grasp which specs make an encoder better than other one in, lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats, so that when I provide a picture of a bird, my autoencoder would tell me that it is not a picture of a cat. I am trying to understand what exact specs make my encoder (and decoder) better? I understand it is all about chosen weights but is it possible to be more specific?

I can not seem to grasp which specs make an encoder better than another one

In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent variables that explain the variations observed in the training set.
The autoencoder family (Variational, Denoising, Contrastive, Sparse) try to approximate p(x) so we have a performance metric to tell us how our model is doing.  e.g. (negative log likelihood of p(x))

lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats,

If you use some autoencoder model to learn the distribution of cats, you could use the encoder part to further augmented with a linear classifier to discriminate between cats and other categories Therefore you have an intrinsic task (learn a good representation of the data distribution) and an extrinsic task (learn to classify cat vs not a cat). So you could do a hyper-parameter search for the model that best suits your problem by measuring its accuracy on the extrinsic task.
Side note:
GAN (Generative Adversarial Network) is a generative model, it provides some way of interacting less directly with this p(x) by drawing samples from it starting without any input. thus the situation here is different.

the link mentioned above is simply explaining the naive autoencoder, the bottom line, data in high dimensional space always lie in a lower dimensional space called manifold, where the probability concentrates and decrease abruptly  as you go far away from the manifold, of course, the manifold could be composed of a set of non-connected manifolds (modes), for more explanation see ch14 of http://www.deeplearningbook.org/

So if I understood correctly, for an autoencoder, Id calculate the probability of a function h given data x that best describes x? Lets say, my function that describes x is x^3 + 1, so that means I would go through each vector of x and calculate its probability falling under that function?

If the data distribution is concentrated on x^3+1 with some variance, the model will try to put more probability on points that are similar to the training data and puts a lower probability on points that don't resemble the data distribution based on the maximum likelihood principle.

What if I do not know the probability? What if I start with a function x +1 even though the real h is x^3 + 1

You don't need to know the probability, you just have training data, where they have some internal structure, you just exploit it by letting the model learns P(x)

Can you give an example to me?

Let us continue this discussion in chat.


text:
Is this overfitting avoidable?
I am trying a modification of Mobilenet in which I add feedback from the softmax layer into the early layers (to implement this I put a second net after the first, which receives connections from the softmax layer of the first, the pretrained weights being non trainable). The idea was to mimic the massive feedback projections in the brain, which presumably could help object recognition by enhancing specific filters and inhibiting others.
I took the pretrained network from Keras and started to retrain it on Imagenet. I noticed that the training accuraccy increased right in the first epoch. My computer is very slow thus I cannot train for too long, an epoch takes 3.5 days. So after an epoch I tried the validation set, but instead the accuracy went down to almost half that of the pretrained values.
My question is if this is and obvious case of overfitting. That is, will continued training increase the accuracy of the training set at the expense of the validation set, or is this a normal behavior expected at the initial stages of training, so that if I keep training for a few more epochs I could expect the validation set accuracy to go eventually up? Any ideas that could help are welcomed.

Hi could you take a look here? https://ai.stackexchange.com/questions/27687/visualisation-for-mobilenet-feature-extraction

an epoch takes 3.5 days

First of all, use colab to iterate quickly, it offers unlimited 12 hours of free GPU.

to retrain it on Imagenet.

That said, we could use complex models without being afraid of overfitting due to large training size.

will continued training increases the accuracy of the training set at the expense of the validation set

In many cases that's the case.

The idea was to mimic the massive feedback projections in the brain.

My suggestion is to read the third part of the deeplearning book which includes Representation Learning, Structured Probabilistic Models for Deep Learning, Monte Carlo Methods, Confronting the Partition Function, Approximate Inference, Deep Generative Models.

Part III is the most important for a researcher—someone who wants to understand
  the breadth of perspectives that have been brought to the field of deep
  learning, and push the field forward towards true artificial intelligence.

thanks for the book suggestion, I will read it. I was not sure to understand your answer. Large data should not result on over fitting, but that is what I get. So could continued training improve the validation accuracy or not (it is lower after training for an epoch?). Also I have a 1080, it should be about the same speed as the k80 used in colab, right?

The 1050ti should be the same speed as the k80, Large data should not result on overfitting BUT what if you have a very very very large model? "with a second net after the first" even with non trainable parameters. by the way, your approach is somehow ill-defined. if you insist to work on some crazy idea, consider toy problems with small (synthetic) dataset and iterate from there. All researcher try their experiments with a suitable computing/time budget.

Hi could you take a look here? https://ai.stackexchange.com/questions/27687/visualisation-for-mobilenet-feature-extraction


text:
Is it common in RL research with Atari/ALE to automatically press FIRE to start games?
In some Atari games in the Arcade Learning Environment (ALE), it is necessary to press FIRE once to start a game. Because it may be difficult for a Reinforcement Learning (RL) agent to learn this, they may often waste a lot of time executing actions that do nothing. Therefore, I get the impression that some people hardcode their agent to press that FIRE button once when necessary.
For example, in OpenAI's baselines repository, this is implemented using the FireResetEnv wrapper. Further down, in their wrap_deepmind (which applies that wrapper among others), it is implied that DeepMind tends to use this functionality in all of their publications. I have not been able to find a reference for this claim though.

My question is: is it common in published research (by DeepMind or others) to use the functionality described above? I'd say that, if this is the case, it should be explicitly mentioned in these papers (because it's important to know if hardcoded domain knowledge was added to a learning agent), but I have been unable to explicitly find this after looking through a wide variety of papers. So, based on this, I'd be inclined to believe the answer is "no". The main thing that confuses me then is the implication (without reference) in the OpenAI baselines repository that the answer would be "yes".

Based on this comment in the Issue I created about this question on github, it looks like there is confirmation that at least DeepMind does not use this kind of functionality in their Atari experiments, contrary to what is implied by the comments in the OpenAI baselines code.


text:
How to manage high numbers of input layer data points
Not sure if this is the correct forum, but I have been working with a large (non-image) dataset that will eventually be used to train a neural network.  I have been puzzling over how to manage wide data sets.  For this application "wide" is maybe 10,000 or 20,000 points wide.  It is not really possible to store this as a row in a conventional RDMS (which are usually limited to several hundred columns).  Is it better to use a huge CSV file or maybe a no-sql technology like Cassandra (the data is originally in JSON format)?

Training with images makes use of pictures that are very large, like 7360×4912 (36 mega pixels). For image recognition at any angle and size, an image is rotated (about 60 times) and resized (depends on the original size, possibly 20 times) many times. This means you end up with a ton of data. In my example here, you would get some 60 × 36 × 2 = 4Gb of data for that one image. Although you really only save the data of interest (noise in the light signal) so in the end the AI does not use as much, but it would still be a quite large data set... especially when you know that a well trained image recognition system uses a good 2,000 images.
So 10 to 20 thousands data points isn't much at all. I guess it depends how many of those you have (one per second would be 86,400 a day...)
I would consider using a system like Cassandra because the database can grow as big as you'd like, if that is one of the problem you're facing. Also you can save the data in binary and the number of columns can vary on each row. However, Cassandra expects a key for each row. This is an important point as it makes use of that key to distribute the data over multiple computers (for faster access later and distribution of the data set over all computers.) It also includes replication "for free" so if a computer fails, you can just replace it (no need to restore it).
What is not a very good idea with Cassandra is creating many indexes. Those use rows (i.e. one index = one row) and thus the few computers handling that one row will work extra-hard on queries on that index... But I would imagine that for AI you would not really need such.
A really large CSV will be difficult to work with because you'll be responsible for accessing the rows (unless you have to access all the rows each time anyway?) Also the file format is Text and the file size will be limited by your OS (which should be really large, but be careful as many I/O calls in higher level libraries miss the fact that the size could be 64 bits.)
If you are looking at a longer/more permanent solution, I would definitely look into Cassandra, especially if you're using Java as a language (also I use it with C++ and it works just fine with C++ too.) If you're just writing a throw away app. anyway, then CSV is probably much easier than handling a whole Cassandra cluster! That's a learning curve...


text:
What are good action outputs for reinforcement learning agents acting in a trading environment?
I am trying to build an agent that trades commodities in a exchange setting. What are good ways to map the action output to real world actions? If the last layer is a tanh activation function, outputs range between [-1,+1]. How do I map these values to real actions? Or should I change the output activation to linear and then directly apply the output as an action?
So let's say the output is tanh activated and it's -0.4, 5. I could map this to:
- -0.4 --> sell 40% of my holdings for 5$ per unit
- -0.4 --> sell 40% for 5$ in total
if it was linear, I could expect larger outputs (e.g -100, 5). Then the action would be mapped to:
- sell 100 units for 5$ each
- sell 100 units for 5$ total

Working Backward
Working backward from the trading interface available to you Note 1, you will need two things for each Exchange-Traded Fund (ETF) or other tradable commodity of another class.

An operation to perform
An associated monetary amount

The system can have an output structure Note 2 for each ETF like this (depending of course on the trading interface available to you to you and your banks primary monetary system).

Ternary operation indicator (buy, sell, hold)
Trade amount in USD

Why Not Just One Number per ETF?
A few corroborating reasons exist for why a single positive, zero, or negative trade amount is not likely the optimal architectural choice.

The difference between holding and trading an amount of 1.0 monetary unit is not equivalent to the difference between trading 1.0 and 2.0 monetary units.  Stated mathematically, the function of profitability to trade amount is not smooth and probably not even continuous.
When you implement the ternary output as two binary outputs {buy, sell}, training against the Boolean expression (buy AND sell) is likely to improve your initial performance and possibly your ongoing performance Note 3.

Limitations on Real Trades to Consider
Because you have the limit of the assets in the liquid account from which you can buy, you will need to train against breaking the bank or using a stage after the NN outputs imposing rules or a formula based on gain and loss probabilities.  This financial constraint muddies your question because there are several ways to ensure you do not break the bank (get an insufficient funds response from your trade operation).
Optimizing a Deeper Architecture
Let's first assume you use probabilistic calculus to produce a closed form (formula) for how to trade based on predictions from the NN architecture you design.  Then the NN outputs might be continuous values representing the distribution of outcomes for each ETF.  Projections will almost always be dependent on investment duration.
In such an architecture the NN output activation scheme would be a continuous function (not necessarily linear) producing something like this Note 4.

Mean expected delta value in one day
Std deviation in one day expectation
Mean expected delta four weeks
Std deviation in two week expectation
Mean expected delta two years
Std deviation in two years expectation

Any NN optimization of an investment portfolio that does not inherently deal with probability is nonsense.  Optimizing for maximum gain will introduce great risk.  Optimizing for minimum risk could result in losses.  The goal of optimization must be some representation of the balance between the desire to win and the fear of loosing, to put it in anthropological terms.
Mean and standard deviation are obvious starting choices, and the traditional categories of short, medium, and longer term investment is also reasonable to begin with in the temporal domain Note 5.
Pure NN
Now let's assume you replace the calculus with another NN scheme trained to maximize portfolio total assets Note 6.  Such a replacement NN scheme must also take as an input your available liquid asset amount along with the above probabilistic projections.  You must also train to ensure the aggregation of buys and sells do not break your bank, that your liquid asset account never drops below zero.
The trade amount activation should be a continuous function, but not necessarily linear and probably not tanh either because the asymptotes of that function would be counterproductive unless it is made proportional to your available liquid assets to train by aggregating your options in your architecture.  However, that's not optimal because you may find a better deal to use those assets a minute later.
The odd roots (third root or fifth root or both with coefficients), when used along with training to not break the bank and to maximize the rate of portfolio growth will produce a better environment for learning in the earlier layers because of the probabilistic and aggregation realities of liquid asset limitations.

NOTES
Note 1 — Preferably a secure RESTful API, however experimentation can employ a web browser the HTTPS transactions you can control using https://github.com/SeleniumHQ/selenium or https://github.com/watir/watir.
Note 2 — Input would be things like number of companies, exposure ratings provided by investment firm(s), short options, inception date, and a sequence of events, each event containing fixed point numbers and flags like these.

Price per share
Closing price flag
Expense ratio
Dividend per share

Note 3 — The binary output vector {buy, sell} value of {1, 0} causes a buy.  The value of {0, 1} causes a sell.  The value {0, 0} is ignored, and {1, 1} may trigger another training operation to correct this illegal output, which is likely to be indicative of the staleness of the last round of training.  If the NN is re-entrant (reinforced), the feedback vector could include this anomalous flag or weight it heavily in an aggregation of feedback sources.  In summary, the ternary scheme can be expected to augment the training speed and resulting accuracy.  More importantly, it opens additional options for continuous optimization.
Note 4 — Delta is an aggregation of price increase or loss, dividends, and holding and trading expenses because that is the proper metric related to profitability of the portfolio.
Note 5 — Four weeks and two years have been chosen for the middle and longer range projections so that the time ratios are 26.09 and 28 between the three durations.  Common choices are temporally skewed.  If 1 day, 1 week, and 1 year had been chosen, the ratios would have been 7.0 and 52.18.  If 1 day, 1 month, and 1 year had been chosen, they would have been 30.44 and 12.
Note 6 — Do not assume that even a well trained NN will ever outperform formulae properly derived from probabilistic calculus for the last stage in a trading profitability architecture.


text:
How to apply a kernel to an image?
As I know, if we consider a 3*3 kernel, we should add a padding of 1px to the source image(if we want to have effect on whole of the image), then we start to put the kernel in upper-left side of the image and multiplying each element of kernel to corresponding pixel on image. Then we sum all the results and put it on the anchor point of kernel(usually center element). Then we should shift the kernel one step to right side and do these things again.
If I am right till here, I have a question about the summation results. I want to know: should we consider the replaced value of image in previously calculated summation and replaced in anchor point in new step of calculation or not?
I mean we must put the anchor point's result in source image and consider it in calculations of shifted kernel? Or we must put it in distance image and we don't consider these results when we shift the kernel on source image(It means don't replace the results on source image for next steps calculations)?

Assuming that you are performing a normal discrete convolution, then you would use the original source image for all calculations, and only replace at the end. As you put it, like this:

put it in distance image and we don't consider these results when we shift the kernel on source image(It means don't replace the results on source image for next steps calculations)?

The other option, changing in-place during the operation, does not relate to any common use that I know of, but might have some interesting behaviour in studies of cellular automata or similar.


text:
How to make a fair comparison of a convolutional neural network (cNN) vs a mutlilayer perceptron (MLP)?
I'm working with deep learning on some EEG data for classification, and I was wondering if there's any systematic/mathematical way to define the architecture of the networks, in order to compare their performance fairly.
Should the comparison be at the level of neurons (e.g. number of neurons in each layer), or at the level of weights (e.g. number of parameters for training in each type of network), or maybe something else?
One idea that emerged was to construct one layer for the MLP for each corresponding convolutional layer, based on the number of neurons after the pooling and dropout layers.
Any ideas? If there's any relative work or paper regarding this problem I would be very grateful to know.
Thank you for your time
Konstantinos

Konstantine, I assume you refer to plain MLP and CNN, without any modifications.
I believe what you ask is how to set both of them up, in order to have the fairest comparison possible.
The way I would do it, is to use their plain implementations but both tuned as much as possible, in every hyperparameter. Both should work as black boxes that accept the same inputs and give the same outputs.
This will give you insight on the true raw performance of both algorithms.
Hope it helps :)

I'm not sure what plain implies. My models are sequential models with two hidden layers (for the convolutional network the first hidden layer is composed by two convolutional layers and a pooling layer). The idea of comparing the best out of the two is appealing. Although i don't have a lot of hyperparameters to play with, so I was thinking more of a size comparisons between the two (size in number of neurons, weights, parameters to learn, etc)

By "plain" I refer to the most "basic" implementation of both algorithms. I don't see direct evaluation from just comparing the e.g. weights of both. This is because the architecture is different so you are comparing "apples and oranges". What would make sense is to compare the overall performance, not the inner characteristics. This means, don't focus on comparing hyperparameters, just make sure you fine-tune both networks. You should compare their results for the same inputs. Results like accuracy and training/inference time. Hope it makes sense. Kalh tyxh :)

The best best way to monitor an architectures performance would be comparing the resource utilization, model accuracy, value loss and confusion matrix.

e.g VGG16 consumes less system resources in comparison to inception V3

There is an article that goes in depth.

The article talks more about various metrics of performance, which is not really my concern. The idea of resource utilization is interesting. Any ideas how to compare these in my models? (I use the keras library). Perhaps check the number of parameters to train for each network? (keras outputs the total number of trainable parameters)


text:
Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?
I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.
The central service dispatches some information at regular intervals to all the agents (Lets call this information as energy). The information can be very different for all the agents.
The agents on reception of this information select a particular action. The execution of the action should leave the agent as well as the environment in a positive state. The action requires a limited amount of energy which might change on every timestep. If a agent does not have sufficient energy to it may request for energy from other agents. The other agents may grant or deny this request.
If all the agents are able to successfully perform their actions and leave the environment in a positive state they get a positive reward.
As the environment is stochastic, where a agent's behavior is dependent on another agent can approximate Q Learning be used here?

Not particularly sure what you are asking, so the question that I will be answering is this:

Can Q learning be used to estimate a value that depends on another value in the Q Learning Matrix even though there is a certain amount of unpredictability involved?

The answer is yes!
I will use the example of a robotic arm trying to reach a point in space since that is what I am most familiar with.
Imagine a robotic arm with a shoulder, elbow, and wrist joint. The desired elbow value depends very much on the shoulder value which is also being learned. Given enough iterations, the Q learning algorithm will come up with a solution (out of possibly many) for the elbow joint based on where the shoulder joint is at that time.
The intrinsic unpredictability (air drag, motor power, etc in this case) is countered intuitively by the Q-learning algorithm iteratively learning what works best.

I know the question is difficult to understand with less context. But the way you have framed the question is exactly what I was trying to ask "unpredictability". Thanks! Could you add a few more comments as to where or in which environment will Q-learning fail?

I am not an expert in the limits of reinforcement learning, but I know that time can be limiting factor. It takes a long time to train and will go through many iterations before the output becomes useful.


text:
Question classification according to chapters
I have a corpus, say an instruction manual. The text in this manual is grouped into chapters and each chapter is split up into sections. For example, Chapter 1/Section 1, Chapter 1/Section 2 and so on.
Assume the corpus has C chapters and each chapter has S sections. My goal is, given a sentence or question, to classify this sentence/question. In other words I want to compute three most probable chapters to which this sentence or question belongs to.
I tried MultinomialNB model using sklealrn, but it did not give me the desired result. I want to try another approach, for example using a Neural Network and compare it with the MultinomialNB model. I have Googled and found Doc2Vec but haven't tried yet.
Can anyone suggest a better or another possible approach so that I could try and compare? What is the standard approach to such kind of problem?

You have got text which have words. Now each of the words have dependency with words that are occurring previously or/and with the ones occurring later. So to capture the "context" in which a particular word occurs is done by sequence models in deep learning. Sequence models that are quite used often are Recurrent Neural Networks (RNN), Gated Recurrent Units (GRU) and Long Short Term Memory (LSTM) listed in the order of their functionality and complexity. To know more about them visit Jonathan Hui's blog.
Your problem is a classification problem i.e. given a sentence the classifier should tell which chapter and section it belongs to. You can use one of the sequence models to "encode" the sentence as a vector (based on the context of words in it and meaning of sentence) and pass that vector to a fully-connected neural network that has a softmax layer at the end to tell which chapter and section the text belongs to. The number of classes you are having are C * S (one for each chapter-section pair). From the output of softmax layer you can pick up the top 3 classes that got the highest probability for a sentence.
Doc2Vec converts a paragraph (can also be a sentence) into a vector representation. The Doc2Vec model has been trained over a huge corpus of text to be able to capture the context of a text and represent it as a high dimensional vector.
The corpus of text that you have, is from a probability distribution is quite different from that used for training Doc2Vec model. Sequence models (mentioned above) can learn that probability distribution for your text corpus exclusively. In other words, you have a tailor-made representation for your text so that the context can be understood much better.
That is why I suggested you to use sequence models. You can also compare the performance obtained by using pretrained Doc2Vec as text representation with that obtained by using one of the sequence models.
To know about the effectiveness of RNN visit Andrej Karpathy's blog.

I have another question. As far as I know there are pretrained models for Word2Vec, e.g., Google's Word2Vec or Standford Univ's GloVe. We can download those models and use it. However, libraries such as Keras/Gensim provide with  means for Word2Vec using which we can train our own model by feeding with labeled sentences from our corpus. Also, Word2Vec (shallow) neural network is well known and proved to be efficient (in the scientific circles). The same for Doc2Vec.

So, wouldn't it be better to exploit the underlying Doc2Vec neural network to train my model using my domain specific corpus? Or should I try different NN architectures of RNN, LSTM and etc and later compare them? I mean should I spend my time on what people have already proved?

Its worth doing both so that you can compare the extent to which neural nets in Doc2Vec and RNNs can model dependencies between words.

The easiest way would probably be not to use machine learning at all. Create an inverted index of words (ie for each word occurrence record the chapter and section), and then use an information retrieval algorithm (eg TF-IDF) to find the section that best matches your question.
This will be more efficient (no huge models to train) and more transparent (you can easily see why a particular section has been selected).
Additional steps you can take to adjust performance is stemming/lemmatising of words, and adding synonym lists (eg in a car manual application you might want to treat trunk and boot as the same word).
I have used that approach recently in a commercial project, and it performed well.

Thank you very much for your suggestion. I will try it. Unfortunately cannot upvote since I do not have enough reputation.

I create tf-idf scores for each word for each document. Sklearn's TfidfVectorizer allows to do it in a couple of lines. But it creates scores for words, not for sentences. As I described, my task is classify a particular sentence. So, I need to come up with a method that computes similar score for sentence. For example given a sentence S, I want to compute C scores, one score for each class (chapter in my case). Do you know any approach how to go about it? Any reference? A naive approach is to sum up scores of each word or compute average over all words in the sentence.

@B.K. Yes, I'd sum scores or average, depending on what works better in that particular application. Adding favours longer sequences, so if that is a problem you could average instead.


text:
Structured lighting basic principles for depth mapping
I've been wondering, how, in the most simple-to-implement basic principle, does the light projection to depth map technique described here https://www.lightform.com/how-it-works actually functions? Is it some kind of an average based on the color of x pixel over all the patterns or what? How difficult would it be to code something that could do this, up?

how, in the most simple-to-implement basic principle, does the light projection to depth map technique described here https://www.lightform.com/how-it-works actually functions?Is it some kind of an average based on the color of x pixel over all the patterns or what?
No it is not that simple , more details on that are described in https://kevinkarsch.com/publications/sa11-lowres.pdf
P.S the author of this paper went on to become CTO of lightform.
Is it some kind of an average based on the color of x pixel over all the patterns or what? How difficult would it be to code something that could do this, up?
As mentioned above it is not simple average based on color , coding would be a tough task , you could implement model if you are capable enough to code a ray-tracer from scratch. But replicating the exact results would require a lot of low-level knowledge and certain domain expertise as not much of the implementation details and hyperparameters are described.

Cool, although this seems to do 3d based on color image input, What about the structured lighting approach Lightform uses?


text:
Using CNN to identify buildings from aerial images
I want to train a CNN (Vggnet) to identify different types of buildings from aerial images.
However seeing that a CNN "ignores" size, e.g. the same type of dog in one image can be large and small in another image but will still be classified as a dog.
My issue is that non-residential buildings are mostly larger than residential houses, now I want to use this property to distinguish between residential and non residential. Is this even possible?

Thanks

CNN doesn't  ignore size at all,who told you.Can you give us a hint!

You need to know the basic mechanics of CNN , you can't simply finetune a pretrained net (VGGNET) everytime. VGGnet is trained on imagenet dataset so most of the features in it are not so relevant for your task (recognising buildings from aerial view). so it is much better to train a cnn from scratch. You can't simply say that it fails to classify residential , non-residential . It is best to find what could be done by trying,
My issue is that non-residential buildings are mostly larger than residential houses, now I want to use this property to distinguish between residential and non residential.
Most of the times it is better to observe the features recognised by the net and then come with an approach , rather than enforcing your views.
You may find this helpful https://www.cs.toronto.edu/~vmnih/docs/Mnih_Volodymyr_PhD_Thesis.pdf


text:
Convergence in multi-agent environment
I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy include - request, deny request, grant), which they have produced from renewable sources and is stored in their individual batteries. The overall goal is to reduce the energy used from non-renewable sources.
All agents have been built using DQN. All (S,A) pairs are stored in a replay memory which are extracted when updating the weights.
The reward function is modelled as such — if at the end of the episode the aggregate consumption of the agent group from non-renewable sources is lesser than the previous episode, all agents are rewarded with +1. If not, then -1. An episode (iteration) consists of 100 timesteps after which the reward is calculated. I update the weights after each episode.
The reward obtained at the end of the episode is used to calculate the error for ALL (S,A) pairs in the episode i.e. I am rewarding all (S,A) in that episode with the same reward.
My problem is that agents are unable to learn the optimal behavior to reduce the overall energy consumption from non-renewable sources. The overall consumption of the group is oscillating i.e. sometimes increasing and sometimes decreasing. Does it have to do with the reward function? Or Q learning as the environment is dynamic?

Does it have to do with the reward function?

This seems likely to me. You have chosen a reward that is unusual in that it cross-links episodes. It is not really a reinforcement problem to optimise behaviour with respect to results of previous episode behaviour in this way. This might be an option for an evolutionary fitness context, if you have competing teams against the same environment in a tournament style selection.
Reinforcement learning should really take as direct measure of your goal as you can construct. In this case you want to minimise some scalar quantity, and that is an obvious candidate as a negative reward. So the reward should be the negative of the total non-renewable energy consumption. The maximum possible value in theory for any single episode would be zero.
You may still have problems with a multi-agent setup oscillating using Q-learning, it will depend on how much each agent views the full, relevant state, and whether you are training several distinct agents at the same time (more likely to oscillate), or a single type of agent with multiple instances in each environment (less likely to oscillate, but still can if it has too blinkered a view of the environment as experienced by the other agents). But having a single shared goal with shared reward like you do here should in theory help with stability.

I see what you mean. Two questions:
First - If I reward the agent negatively every time wouldn't this be a problem? I do understand that the value will be smaller or greater everytime.
Second - Is it okay if I distribute (apply) the same reward across all (S,A) pairs in the episode or should I be rewarding only in the final (S,A) pair at the end of the episode?
Thanks.

@Neil Slater It is possible that the agent might never get a positive reward as the negative of total non-renewable energy consumption can only be zero in an ideal scenario. Additionally, I did not understand the third paragraph in the answer, by "several distinct agents" do you mean with different implementations? I have multiple agents running as separate processes with the same implementation.

@amitection: The absolute value of the reward should not matter in this case. There is no special need for positive rewards in general in RL. Although you do have to take care choosing some absolute values, in this case re-framing a positive cost as a negative reward is just fine. By "distinct" I mean different implementation and/or different learning parameters. If the agents are separated and learn in isolation, this leaves you more open to co-evolutions appearing as local minima. It may still be OK though for you, since the only criterion for success is the shared group goal.

@amitection: Your Second question in the comment above: It is not OK to re-distribute reward directly like this in general. One trait of TD learning (of which Q learning is an example) is that it will, eventually, figure out long-term consequences from sparse rewards - e.g. a single reward value granted at the end. In your case, declaring the same reward for all actions is probably counter-productive. If possible, the actual reward at any time step should be the negative amount of non-renewable energy used at that time step. If that is not possible, use one large (-ve) reward at the end.


text:
Can DQN announce it has things in its hand in a card game?
More informations on the card game I'm talking about are in my last question here: DQN input representation for a card game
So I was thinking about the output of the q neural network and, aside from which card to play, I was wondering if the agent can announce things.
Imagine you have the current hand: 2, 4, 11, 2 (The twos are different card type).
When you're playing the game and you get dealt a hand like this, you have to announce that you have the same number twice (called Ronda) or thrice (called Tringa) before anyone plays a card on the table. Lying about it gets you a penalty.
Could a DQN handle this? I don't know if adding "Announcing a Ronda/Tringa" as an action would actually help. I mean, can this be modeled for the  NN or should I just automate this and spare the agent having to announce it everytime.

Is there any possible tactical advantage to lying and accepting the penalty, in the normal game?

Thing is, if both players have a Ronda, the one with the highest number wins 2 points so u could lie and he gets only 1 point but thats too human i believe, plus if they catch u lying u get the penalty either way so im starting to belive that i shouldnt include it?

If lying/concealing the Ronda, plus detecting or calling out the lies is part of the game, then I think it is up to you when implementing the bot. Perhaps save it for v2 if you are just starting out with first implementation. How hard in reality is it to detect the lie? If it is something that will always be noticed by a perceptive player with a good memory, then perhaps it is not so good in the automated game, because it may be possible to fully automate it without any real use of AI in the first place.

It's not that hard to detect that a player lied about a Ronda, since you only have to remember the 4 cards he played and you'll find out.

In that case it seems to be a minor tactic in any game with experienced/good players, and you could maybe skip it. However, it is an interesting question in general - other card games may not have such a clear decision about when to include this kind of tactic.

The thing is, in order to catch your opponent lying you have to wait for him to play all his 4 cards, and I cannot think about how would the NN understand that...

Let us continue this discussion in chat.

The simplest thing to do when you make you first implementation of the agent, is to automate decisions like this, in order to keep representations and decisions simple.
However, if you want to explore tactics surrounding declaration, then I think the following applies:

There should be an initial round of actions where the agent may get to decide whether or not to declare a Ronda, based on the cards it holds. These will be different action choices to playing cards, so you would need to alter your action representation to include those choices. Only allow action choices which are valid, so if it is not valid to declare a Ronda or a Tringa when a player does not have one, then the player does not get to make that choice.
You may want to add a state feature "has a Ronda" and "has a Tringa" for the agent's player, to help with the action decision.
You should also add a state feature for each player according to whether they declared a Ronda or a Tringa.
Rather than have the agent learn to detect a lie, given your comment that all cards are played so it is easy to tell (there are only 8 cards total in play by the end), then I would just assume lies are automatically found out and include that in the game engine. In other words, the penalty is always paid.
The interesting question is then whether withholding the declaration can give a tactical advantage when the round is being played (because your opponent knows less about your hand), and whether that advantage offsets the inevitable penalty. This might not be true in your card game, but could be true in games with similar choices.

Could a DQN handle this?

A DQN is maybe going to struggle with partial information in this game. The opponent's cards are hidden from the network, but could have a non-random influence on the opponent's choices of action. It is possible that you will need to investigate agents that can solve POMDPs to get the best player.
I don't know for certain though. It depends on how much tactical advantage there is in having concealed cards, or how much that is just luck that plays out much the same whether you know the opponent's cards or not. If there is strategy, and ways to determine/guess what the opponent holds based on your cards and their actions so far, this is more like a POMDP.

Thanks for the detailed answer, I appreciate the informations you provided! I guess as a first step I will try to make the agent know when he has a Ronda (given the cards in his hand as features) and to announce it as an action, I will leave detecting opponent lies or lying itself to a "version 2" of the side project. Thank you.


text:
Does machine learning continue to learn?
If I do supervised learning the model learns from the labeled input data. This seems to be quite often a small set of human annotated data.
Is it true to say this is the only 'learning' the model does?
It seems like the small data set has a huge influence on the model. Can it be made better using future unlabeled data?

Could you please explain what you mean saying that the labeled data is quite often a small set of human annotated data? Do you mean that the inputs are manually labeled?

Also, without labels, how could you expect to train a NN? You cannot provide a cost function for these data.

@gvgramazio yes manual labeling of a sample.

Then if you use supervised learning you cannot use those labeled data. Instead, if you use unsupervised learning you could use unlabeled data.

This may not seem trivial but yes, the models we train can potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.
It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, which may not even be specific to the task at hand. This is the reason transfer learning works very well even on a host of different datasets.
This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.
P.S.: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.

Good answer, thanks. But for simple regression, say, there is nothing new unless we label new data, is that right?

Yeah @schoon, I can't think of any way a simple regression model can learn something interesting. But, since the structure of data in classification and regression problems is more or less the same, it may still learn something which might not be perceivable by us. This gave me an idea though, we can try to do transfer learning in case of regression tasks and see if it yields any useful results. If it does, we can say that the model has indeed learned something else other than the task at hand!


text:
Why was Go a harder game for an AI to master than Chess?
AI became superior to the best human players in chess around 20 years ago (when the 2nd Deep Blue match concluded). However, it took until 2016 for an AI to beat the Go world chess champion, and this feat required heavy machine learning.
My question is why was/is Go a harder game for AIs to master than Chess? I assume it has to do with Go's enormous branching factor; on a 13x13 board it is 169, while on a 19x19 board it is 361. Meanwhile, Chess typically has a branching factor of around 30.

The branching factor is important, as it limits the effectiveness of search.
However, the branching factor in chess is already too high to effectively search without techniques that reduce the size of the search space. Even with millions of tests per second, a computer can only check a small fraction of the possible future games in order to find results in its favour.
One key factor is heuristics - approximate measures of the value of each game state. A good heuristic can guide and improve search by orders of magnitude. There are some effective heuristics possible in chess, from weighted values of the pieces in play, scores for areas of the board that a position controls etc.
Heuristics for Go are much harder to find. Here's a sample paper from a few years ago that makes an attempt, there are several similar ones available online. Although plenty of options have been tried, and many were partially successful, none managed to bring the quality of computer play up to the standard of best human players.
One of the major achievements of AlphaGo was training a neural network that had good position evaluation - the "value network". The technology that made generating this approximate function of board game positions possible was deep learning, which has been developed very strongly since about 2010.
It is still possible a more analytical heuristic approach could be found that challenges deep learning models driven by self-play reinforcement learning on more raw board data. However, in some regards the reverse has been shown, with AlphaZero taking the same learning technique into chess and demonstrating its effectiveness against "old school" tuned expert heuristics.

Here's an interesting blog about Chess and Nim from 2011.  Also an interesting 2005 paper Positions of Value *2 in Generalized Domineering and Chess.


text:
Are any organisations using AI to predict weather?
This seems like a natural fit, though I've not heard of any, yet.
I would love to know if any MET office, government, military or academic institution has taken all (or sizeable portion of) recorded global weather data for, say, the last 50 years (or since we, as a race, have been using weather satellites) and used it in an AI system to predict future weather.

People have used machine learning models on aspects of weather forecasting, as here: Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting or here Predicting Solar Generation from Weather Forecasts using Machine Learning.  I've been loosely associated with an effort to use ML techniques to predict utility demand from weather data.  But note that these are looking to predict the implications of the weather forecast, not the forecast itself.  Lots of effort goes into refining the physics-based models that underlie our everyday weather forecasts, and AFAIK machine learning hasn't resulted in any "secret sauce" that gives better results than these models provide.

A recent development is Google's GenCast.
See GenCast: Diffusion-based ensemble forecasting for medium-range weather
It is a machine learning based weather prediction model based on empirical fits. It currently outperforms a physics based extrapolation model.
It is not unexpected that such a model can work better. You can compare it to a scale up of a farmer's gut feeling, intuition and common sense. No underlying mechanistic computations (which can be wrong) but instead experience based predictions.


text:
Why do we need floats for using neural networks?
Is it possible to make a neural network that uses only integers by scaling input and output of each function to [-INT_MAX, INT_MAX]? Is there any drawbacks?

TL;DR
Not only it's possible, it even gets done and is commercially available. It's just impractical on a commodity HW which is pretty good at FP arithmetic.
Details
It is definitely possible, and you might get some speed for a lot of troubles.
The nice thing about floating point is that it works without you knowing the exact range. You can scale your inputs to the range (-1, +1) (such a scaling is pretty commonplace as it speeds up the convergence) and multiply them by 2**31, so they used the range of signed 32-bit integers. That's fine.
You can't do the same to your weights as there's no limit on them. You can assume them to lie in the interval (-128, +128) and scale them accordingly.
If your assumption was wrong, you get an overflow and a huge negative weight where a huge positive weight should be or the other way round. In any case, a disaster.
You could check for overflow, but this is too expensive. Your arithmetic gets slower than FP.
You could check for possible overflow from time to time and take a corrective action. The details may get complicated.
You could use saturation arithmetic, but it's implemented only in some specialized CPUs, not in your PC.
Now, there's a multiplication. With use of 64-bit integers, it goes well, but you need to compute a sum (with a possible overflow) and scale the output back to same sane range (another problem).
All in all, with fast FP arithmetic available, it's not worth the hassle.
It might be a good idea for a custom chip, which could do saturation integer arithmetic with much less hardware and much faster then FP.

Depending on what integer types you use, there may be a precision loss when compared to the floating point, which may or may not matter. Note that TPU (used in AlphaZero) has 8-bit precision only.

Floating Point Hardware
There are three common floating point formats used to approximate real numbers used in digital arithmetic circuitry.  These are defined in IEEE 754, a standard that was adopted in 1985 with a revision in 2008.  These mappings of bit-wise layouts real number representations are designed into CPUs, FPUs, DSPs, and GPUs either in gate level hardware, firmware, or libraries1.

binary 32 has a 24 bit mantissa and an 8 bit exponent
binary 64 has a 53 bit mantissa and an 11 bit exponent
binary 128 has a 113 bit mantissa and a 15 bit exponent2

Factors in Choosing Numerical Representations
Any of these can represent signals in signal processing, and all have been experimented with in AI for various purposes related to three things:

Value range — not a concern in ML applications where the signal is properly normalized
Averting saturation of the signal with rounding noise — a key issue in parameter adjustment
Time required to execute an algorithm on a given target architecture

The balance in the best designed AI is between these last two items.  In the case of back-propagation in neural nets, the gradient-based signal that approximates the desired corrective action to apply to the attenuation parameters of each layer must not become saturated with rounding noise.3
Hardware Trends Favor Floating-point
Because of the demand of certain markets and common uses, scalar, vector, or matrix operations using these standards may, in certain cases, be faster than integer arithmetic.  These markets include ...

Closed loop control (piloting, targeting, countermeasures)
Code breaking (Fourier, finite, convergence, fractal)
Video rendering (movie watching, animation, gaming, VR)
Scientific computing (particle physics, astrodynamics)

First Degree Transforms to Integers
On the opposing end of numerical range, one can represent signals as integers (signed) or non-negative integers (unsigned).
In this case, the transformation between the set of real numbers, vectors, matrices, cubes, and hyper-cubes in the world of calculus4 and the integers that approximate them is a first degree polynomial.
The polynomial can be represented as $n = r(as + b)$, where $a \ne 0$, $n$ is the binary number approximation, $s$ is the scalar real, and $r$ is the function that rounds real numbers to the nearest integer.  This defines a super-set of the concept of fixed point arithmetic because of $b$.
Integer based calculations have also been examined experimentally for many AI applications.  This gives more options:

two's complement 16 bit integer
16 bit non-negative integer
two's complement 32 bit integer
32 bit non-negative integer
two's complement 64 bit integer
64 bit non-negative integer
two's complement 128 bit integer
128 bit non-negative integer

Example Case
For instance, if your theory indicates the need to represent the real numbers in the range $[-\pi, \pi]$ in some algorithm, then you might represent this range as a 64 bit non-negative integer (if that works to the advantage of speed optimization for some reason that is algorithm and possibly hardware specific).
You know that $[-\pi, \pi]$ in the closed form (algebraic relation) developed from the calculus needs to be represented in the range $[0, 2^64 - 1]$, so in $n = r(as + b)$, $a = 2^61$ and $b = 2^60$. Choosing $a = \frac {2^64} {\pi}$ would likely create the need for more lost cycles in multiplication when a simple manipulation of the base two exponent is much more efficient.
The range of values for that real number would then be [0, 1100,1001,0000,1111,1101,1010,1010,0010,0010,0001,0110,1000,1100,0010,0011,0101] and the number of bits wasted by keeping the relationship based on powers of two will be $\log_2 4 - log_2 \pi$, which is approximately 0.3485 bits.  That's better than 99% conservation of information.
Back to the Question
The question is a good one, and is hardware and domain dependent.
As mentioned above hardware is continuously developing in the direction to work with IEEE floating point vector and matrix arithmetic, particularly in 32 and 64 bit floating point multiplication.  For some domains and execution targets (hardware, bus architecture, firmware, and kernel), the floating point arithmetic may grossly out perform what performance gains can be obtained in 20th century CPUs by applying the above first degree polynomial transformation.
Why the Question is Relevant
In contract, if the product manufacturing price, the power consumption, and PC board size and weight must be kept low to enter certain consumer, aeronautic, and industrial markets, the low cost CPUs may be demanded. By design, these smaller CPU architectures do not have DSPs and the FPU capabilities don't usually have hardware realization of 64 bit floating point multiplication.5
Handling Number Ranges
Care in normalizing signals and picking the right values for a and b are essential, as mentioned, more so than with floating point, where the diminution of the exponent can eliminate many cases where saturation would be an issue with integers6.  Augmentation of the exponent can avert overflow automatically too, up to a point, of course.
In either type of numeric representation, normalizing is part of what improves convergence rate and reliability anyway, so it should always be addressed.
The only way to deal with saturation in applications requiring small signals (such as with gradient descent in back-propagation, especially when the data set ranges are over an order of magnitude) is to carefully work out the mathematics to avoid it.
This is a science by itself, and only a few people have the scope of knowledge to handle hardware manipulation at the circuitry and assembly language level along with the linear algebra, calculus, and machine learning comprehension.  The interdisciplinary skill set is rare and valuable.

Notes
[1] Libraries for low level hardware operations such as 128 bit multiplication are written in cross assembly language or in C with the -S option turned on so the developer can check the machine instructions.
[2] Unless you are calculating the number of atoms in the universe, the number of permutations in possible game-play for the game Go, the course to a landing pad in a crater on Mars, or the distance in meters to reach a potentially habitable planet revolving around Proxima Centauri, you will likely not need larger representations than the three common IEEE floating point representations.
[3] Rounding noise naturally arises when digital signals approach zero and the rounding of the least significant bit in the digital representation begins to produce chaotic noise of a magnitude that approaches that of the signal.  When this happens, the signal is saturated in that noise and cannot be used to reliably communicate the signal.
[4] The closed forms (algebraic formulae) to be realized in software driven algorithms arise out of the solution of equations, usually involving differentials.
[5] Daughter boards with GPUs are often too pricey, power hungry, hot, heavy, and/or packaging unfriendly in non-terrestrial and consumer markets.
[6] The zeroing of feedback is skipped in this answer because it points to either one of two things: (A) Perfect convergence or (B) Poorly resolved mathematics.

There are a few typos in your post, such as [0, 2^64 - 1] which renders as $[0, 2^64 - 1]$. From context, I think you mean $[0, 2^{64} - 1]$.

It is possible in principle, but you will end up emulating floating point arithmetic using integers in multiple places, so it is unlikely to be efficient for general use. Training is likely to be an issue.
If the output of a layer is scaled to [-INT_MAX, INT_MAX], then you need to multiply those values by weights (which you will also need to be integers with a large enough range that learning can progress smoothly) and sum them up, and then feed them into a non-linear function.
If you are restricting yourself to integer operations only, this will involve handling multiple integers to represent high/low words in a larger int type, that you then must scale (which introduces a multi-word integer division). By the time you have done this, it is unlikely there will be much benefit to using integer arithmetic. Although perhaps it is still possible, depending on the problem, network architecture and your target CPU/GPU etc.
There are instances of working neural networks used in computer vision with only 8-bit floating point (downsampled after training). So it is definitely possible to simplify and approximate some NNs. An integer-based NN derived from a pre-trained 32-bit fp one could potentially offer good performance in certain embedded environments. I found this experiment on a PC AMD chip which showed a marginal improvement even on PC architecture. On devices without dedicated fp processing, the relative improvement should be even better.

A multi-word integer division would surely kill performance, but nobody should ever thing about it. Such a scaling gets simply done by bit shifting, which is maybe two orders of magnitude faster.

Hello Neil! Do you have time to give an answer to this question https://ai.stackexchange.com/questions/20871/what-is-the-advantage-of-using-more-than-one-environment-with-the-advantage-acto ?


text:
What type of AI would you recommend for this complex problem?
I'm trying to develop a kind of AI that will assist in debugging a large software system while we run test cases on it. The AI can get access to anything that the developers of the system can, namely log files, and execution data from trace points. It also has access to the structure of the system, and all of the source code.
The end goal of this AI is to be able to detect runtime errors during execution, and locate the source of these errors.
I was considering making use of a deep neural network, where the input would be the execution data and log output. Using this input it would be able to verify whether the current version of the system we are running is functional, or non-functional. The problems with this approach is that the system it would be evaluating would be constantly changing as it gets developed, so the only training material the NN would have is from the last stable version of the system (and even that could have some errors). Additionally, producing test cases for the system off of which we could train the NN would be very time consuming, and would defeat the purpose of using the NN in the first place.
I would like to know what AI design you think would be suitable for this task. Please let me know if you would like any other information relevant to the problem. As far as I can tell, nothing quite like this has been done before.
It's probably worth mentioning that my team has a some extremely powerful machines on which we can run the AI.

I dunno neural networks very well. For what you are trying to achieve, I guess you need to first freeze the state and do the execution. If your task is telling if the system is functional or non functional, it can be thought of as a classification task. You can start off looking at using DNN for classification. I know this is a broad overview, however, should give you some lead.

Could you please try to re-phrase your question.ideally,we don't have types of AIs but specific fields where your concept idea could fall given the problem is solved  ie machine learning,In this case a piece program once coded effectively, can learn from previous errors or data logs and keeps on advancing thus helping you at the end;debugging your code.You need to be kind specific.However,CybergrandChallenge one team did the same program and i think it's being worked on there! Ain't asleep!!.

Let me comment on assist in debugging a large software system : I've worked on a subclass of that problem with great success using linear regression on code features.  Rephrasing the question, or changing the requirements slightly could make it a lot easier.  For example, if your specific objective was assign a probability to each function that a defect will be discovered in our next round of regression tests and you had data of which functions had defects from the last regression, then that is very doable!

you are not clear on what is it exactly that you want,
you want an AI that can run your program for different inputs and log errors if they exists?

Is the programming language you are using Turing Complete?

I think you would find this link helpful. It demonstrates how to identify patterns in large arbitrary byte data.
https://devblogs.nvidia.com/malware-detection-neural-networks/

You are trying to solve a variant of The Halting Problem, which is the problem of detecting whether a computer program is going to stop, or run forever.
The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:

run(X)
Error("X finished running").

and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.
That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for every program though.

Debug and Validation of large software systems like video software stack
If you take example as validation and debug of video software stack, It is very difficult for naked eye to identify failures on the display. In this case you can use DNN based image classifier to identify functional failure.

If you consider also 3rd party options instead of building your own system, you may want to check this solution https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html


text:
Target values of 0.1 for 0 and 0.9 for 1 for sigmoid
I recently read an article about neural networks saying that, when using sigmoid as activation function, it's advised to use 0.1 as target value instead of 0, and 0.9 instead of 1. This was to avoid "saturation effects". I only understood is halfway, and was hoping someone could clarify a few things for me:

Is this only the case when the output is boolean (0 or 1), or will it also be the case for continual values in the range between 0 and 1. If so, should all values be scaled to the interval [0.1, 0.9]?
What exactly is the problem of output 0 or 1? Does it have something to do with the derivative of sigmoid being 0 when it's value is 0 or 1? As I understood it weights could end up approaching infinity, but I didn't understand why.
Is this the case only when sigmoid is used in the output layer (which it rarely is, I believe), or is it also the case when sigmoid is used in hidden layers only?

Can you link to the article you mention? Out of context, it's not completely clear what is meant.

Derivative of the sigmoid curve is 0 when the output is 0 or 1 as you can see from the image above. The technique you are referring to is called label-smoothing which is used in various applications (e.g. GANs) but I can see how it would be applicable here also, by helping to avoid 0-gradients saturating learning.
To answer your third question, sigmoids are rarely used in the intermediate layers of networks these days. They have been replaced by ReLUs (or variants of ReLU) for this exact reason - sigmoids are prone to causing 'vanishing gradients' where the gradient becomes 0 and therefore does not get backpropagated any further. ReLUs alleviate this problem by always providing a gradient of 1 for positive input values.


text:
Can you analyse a neural network to determine good states?
I've developed a neural network that can play a card game.  I now want to use it to create decks for the game.  My first thought would be to run a lot of games with random decks and use some approximation (maybe just a linear approximation with a feature for each card in your hand) to learn the value function for each state.
However, this will probably take a while, so in the mean time is there any way I could get this information directly from the neural network?

What kind of neural network do you have / how did you train it? What outputs does it have (did you train a network to output values with an algorithm like DQN, or did you train it to output a policy using policy gradient methods, or something else?)

This may or may not be relevant, but I'm personally interested in what type of card game it is.  Deck building implies some form of post-Garfield CCG.  Algorithmic deck building is an exciting prospect!  Can I ask, how many cards in the card "library" which a deck can be built from?

@DennisSoemers There is 1 output for every action that could ever be played (almost all of this is playing card X from your hand), the highest output that is a valid move is selected to play.  I used PPO to train the network.

@DukeZhou There are about 500 cards in the library right now.

I don't think your network, trained using PPO to play a card game, already contains sufficient information to also use for drafting. I'm not saying this with 100% certainty, maybe there's something I'm overlooking, but I can't think of anything right now.
A small adaptation to the network might be sufficient (though it would also involve re-training again). Recently, OpenAI has been writing about their attempts to train agents to play the game DOTA 2. Now, this isn't a card game, it doesn't require deckbuilding, but there is an aspect to the game that is somewhat similar to deckbuilding: drafting. In DOTA 2, there are two teams of 5 players each. Before a game start, each team selects 5 heroes (one per player) to play in that game. This is very similar to deckbuilding, except that it's likely a much smaller problem; there's only a "deck" (team composition) of 5 "cards" (heroes).
Anyway, they also trained agents to play the game (controlling one hero per agent) using PPO. In a blog post, they write the following about how they managed to add drafting capabilities relatively easily:

In late June we added a win probability output to our neural network to introspect what OpenAI Five is predicting. When later considering drafting, we realized we could use this to evaluate the win probability of any draft: just look at the prediction on the first frame of a game with that lineup. In one week of implementation, we crafted a fake frame for each of the 11 million possible team matchups and wrote a tree search to find OpenAI Five’s optimal draft.

So, if you want to try a similar technique, you'd have to adapt your network such that it also learns to generate a prediction of the win probability as output. I imagine that it'd be much less effective for deckbuilding, because win probabilities may all be very close to 50% in card games where luck (when drawing cards for example) can be a significant factor, but it might be worth a try.

Alternatively, instead of generating lots of random decks and playing with them all, you could view the problem of deckbuilding as an additional separate "game" or Markov Decision Process; adding a specific card to the deck can be an action, and this MDP terminates once you have a complete deck. Then you can try to do that better than random using search algorithms (like Monte-Carlo Tree Search) or, again, a Reinforcement Learning approach like PPO. Again, I imagine it will be a very difficult problem though, likely requiring lots of time before it will be capable of doing better than random.

I also know of some research related to deckbuilding in the collectible card game Hearthstone, which may be relevant for you. Unfortunately I did not yet get to read through any of this in detail, so I don't know for sure if you'll find a solution here, but it may be worth a try:

https://dockhorn.antares.uberspace.de/wordpress/additional-material/: At the recent IEEE CIG 2018 conference, there were two competitions involving Hearthstone: one for game-playing AI with pre-built decks, and for generating new decks (and then playing with them). The page I linked to here contains references to related work from earlier years.
Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games: This is a new paper that was published in the proceedings of this recent IEEE CIG 2018 conference.

authors: xCovelus - Слава Україні


text:
Figuring out mapping between two matrices
Imagine I have a 2D matrix, A. I apply some transformation to it, for example:
B = A_shifted + A.
Would it be possible to train a CNN to learn back the mapping from B to A? Giving B as example and A as target?
Thanks!

Yes, with some limitations.
CNNs can be used to map images to related images, and that should include many simple matrix transformations. For instance, here is an example of de-blurring OCRed text using a CNN.
Basically, you would train your network with lots of A, B examples, with the input as B and desired output as A.
The limitation is that where you have transformations that are technically irreversible, then the CNN may learn to produce a best "mean" output. The symptom of this will be fuzzy images lacking high frequency components, and matrices which are not representative of the target distribution, but that do solve the transformation (within limits of training accuracy). If you want to improve on that, and produce a more realistic/precise original, then you will probably want to look at adding a generative component - a GAN, VAE/GAN or RBM etc. Note this would not accurately produce the original matrix, but would generate one that both transformed into your given transformed matrix (within some level of accuracy) and was sampled from your input distribution. That is, it could be more of a feasible original than one generated using a simpler CNN architecture.


text:
Use cross-validation to train after model selection
I have been recently reading about model selection algorithms (for example to decide which value of the regularisation parameter or what size of a neural network to use, broadly hyper-parameters). This is done by dividing the examples into three sets (training 60%, cross-validation 20%, test 20%) and training is done on the data with the first set for all parameters, and then choose the best parameter based on the result in the cross-validation and finally estimate the performance using the test set.
I understand the need for a different data-set compared to training and test for select the model, however, once the model is selected, why not using the cross-validation examples to improve the hypothesis before estimating the performance?
The only reason I could see is that this could cause the hypothesis to worsen and we wouldn't be able to detect it, but, is it really possible that by adding much more examples (60% -> 80%) the hypothesis gets worse?