text:
How to split data for meta-learning?
I've been trying to understand the meta-learning paradigm, more precisely, the optimization-based models, such as MAML, but I have a hard time understanding how I should correctly split my data to train such models.
For example, let's consider we have 4 traffic datasets, and we would like to use 3 of them as source datasets (to train the model) and the remaining one as target (to fine-tune on it and then test the model performance). As far as I understood, for each source dataset, I need to split it into train and validation. During training, I would randomly select 2 batches of samples from the training datasets, use one batch to compute the task-specific parameters and the other one to compute the loss. Then repeat the same process with the validation dataset, such that I can select the best candidate model. After the training is done, I need to fine-tune the model on the target dataset. I assume the process is the same, but I need to use the target dataset instead.
During testing (after the model is fully learned and fine-tuned), how exactly do I test it? It is the same procedure as if I was training a supervised model? I would like to know if the setup I described is correct and it fits the meta-learning paradigm.

I assume in your case what you need to be doing is to collate your 3 datasets together - these would form the training dataset, and then leave the testing dataset aside.
During Meta-Training, the code will sample a batch of tasks in each iteration. This batch of tasks will be split into support and query, the algorithm will train on the support and update model parameters, then it will test on the query and then re-update initial theta.
This Meta-Training is a form of 'cross validation'. Now during meta testing, you must ensure that it samples from the testing dataset.
You can use this code as a start. In data_generator.py it has a place where it splits the images into training and validation. What you need to be doing is to do the same thing but for your structured data, there, you will specify which datasets to sample for training and which dataset to sample for testing.


text:
How to deal with losses on different scales in multi-task learning?
Say I'm training a model for multiple tasks by trying to minimize sum of losses $L_1 + L_2$ via gradient descent.
If these losses are on a different scale, the one whose range is greater will dominate the optimization. I'm currently trying to fix this problem by introducing a hyperparameter $\lambda$, and trying to bring these losses to the same scale by tuning it, i.e., I try to minimize $L_1 +\lambda \cdot L_2$ where $\lambda > 0 $.
However, I'm not sure if this is a good approach. In short, what are some strategies to deal with losses having different scales when doing multi-task learning? I'm particularly interested in deep learning scenarios.

I've seen multiple times the approach that you're currently using in different contexts, such as Bayesian neural networks and multi-objective optimization with evolutionary algorithms, so your approach is at least not new and may work for your purposes. Off the top of my head, no other completely different approach comes to my mind, but it's possible there are others. So, I think the main question/problem is not whether your approach is good or not, but how to scale the losses. Maybe this Wikipedia article: https://en.wikipedia.org/wiki/Multi-objective_optimization can be useful.

I am currently working on a similar problem. I think your approach is good. As for setting the parameter lambda, since you are using deep neural networks, you can make it a learnable parameter, instead of a hyperparameter you set. This way, as the two losses fluctuate over your training iterations/epochs, the model will be able to adjust the lambda parameter accordingly.

This will not work as a parameter. The network will just set the harder task to zero. Must be a hyper param

You can add (1/lambda) to penalize your model for making the weight zero.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What to do with a GAN that trained well but got worse over time?
I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss going down, but I also made sure to regularly check the quality of the images.
Unfortunately, what I am seeing now (for the last $20$ epochs) is that the generator is getting worse and worse, the images don't look that good anymore. I save checkpoints every epochs, so in principle, I could stop training and get myself a state of the network from where it was still performing quite okay.
However, my question would be: How can I improve the training of the GAN? Would you decrease the learning rate?
I use a batch size of 124 and a learning rate of 1e-3. Maybe I could/should continue training (with a checkpoint that was still quite okay) with a learning rate of 5e-4?
Any other hints would be appreciated!

This is from my own experience with (Vanilla) GANs, so it might not translate exactly to your application, but maybe it gives some orientation.

your learning rate seems quite high. I've quite frequently found that 1e-5 is a good value for me. The training might take longer but will probably be more stable.
have you tried using dropout? It's a good regularisation mechanism that can prevent from overfitting and also make training more stable. I've made good experience with a dropout rate of ~0.4 and keeping it in both training and testing stages.
what is your discriminator vs generator update ratio? In my experiments, ~5 discriminator updates per generator update has often shown to improve things.
how many training samples do you have? If they are limited, it might be helpful to add a bit of noise on them. This manipulates the data a bit, but might make training more stable.
I have often not seen a major change when varying the batch size.
what are your network architectures? I have the feeling that small networks are often sufficient...

This is entirely based on my own experience, but maybe there are some interesting directions for you to explore. Happy to hear back from you about the success of these tests.

Hi Mafu, thanks! (i) You're right, the LR of $1e-3$ seems to high.. Maybe I will restart training from scratch and see what it does. (ii) Interesting, I didn't know one could also use Dropout with GANs.. (iii) My critic (as for WGANs, we don't strictly have a discriminator) updates itself $5$-times before a generator update. (iv) I have enough training samles (more than $200$k images). (v) My network architectures are heavy, I won't deny that.. I oriented myself at this: https://github.com/aladdinpersson/Machine-Learning-Collection/blob/master/ML/Pytorch/GANs/4.%20WGAN-GP/model.py

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the name of this algorithm that estimates the gradient with an average by sampling from a distribution?
Consider maximizing the function $R(w)$ with parameter $w$ using gradient ascent. However, we don't know the gradient $\nabla_wR(w)$ formula. Now suppose $w$ is sampled from a probability distribution $\pi(w,\theta)$ parameterized by $\theta$. Then we can define
$$J(\theta)=E[R(w)]=\int R(w)\pi(w,\theta)dw.$$
And we have
$$\nabla_\theta J(\theta)=E[R(w)\nabla_\theta \log \pi(w,\theta)]$$.
Then, if we sample $w_1,\ldots,w_N$, we can estimate the gradient as $$\nabla_\theta J(\theta)\approx \frac{1}{N}\sum_{i=1}^N R(w_i) \nabla_\theta \log \pi(w_i, \theta).$$
It looks like REINFORCE algorithm in Deep Reinforcement Learning. Does this algorithm have a name? Is the above derivation correct?
I wonder if it is useful in optimizing $R(w)$ function.


text:
What is the effect of K in K-NN on the VC dimension?
What is the effect of K in K-NN on the VC dimension? When K increases, is the VC dimension decreased or increased, or we can't say anything about this? Is there a reference book that discusses this?


text:
How do I get started with multi-agent reinforcement learning?
Is there any tutorial that walks through a multi-agent reinforcement learning implementation (in Python) using libraries such as OpenAI's Gym (for the environment), TF-agents, and stable-baselines-3?
I searched a lot, but I was not able to find any tutorial, mostly because Gym environments and most RL libraries are not for multi-agent RL.

That question was focusing more on specific setting of multi agent learning explained in three bullet points. This question focuses more on the specific libraries, namely three: OpenAI gym, tf-agents and stable-baseline-3. The two are definitely not the same. Unfortunately, there is no answer on both questions.

Cross-posted: https://ai.stackexchange.com/q/26806/1794, https://datascience.stackexchange.com/q/90626/8560.  Please do not post the same question on multiple sites.

After checking the Internet, you will probably find several resources such as

https://github.com/mohammadasghari/dqn-multi-agent-rl
https://rlss.inria.fr/files/2019/07/RLSS_Multiagent.pdf
https://arxiv.org/abs/2011.00583
and others.

Try to understand the principles first (see above). After some reasonable amount of coding you can adapt OpenAI gym.
Good luck!
Update 17 March 2022:
You may want to check this popular repository as well https://github.com/Farama-Foundation/PettingZoo
Update 06 August 2023:
The best multi-agent tutorial I have seen so far comes from RLlib documentation. See RLlib for muti-agent RL.

Multi-Agent Deep Reinforcement Learning in 13 Lines of Code Using PettingZoo
A tutorial on multi-agent deep reinforcement learning for beginners.
This tutorial provides a simple introduction to using multi-agent reinforcement learning, assuming a little experience in machine learning and knowledge of Python.
https://towardsdatascience.com/multi-agent-deep-reinforcement-learning-in-15-lines-of-code-using-pettingzoo-e0b963c0820b

Hugging Face RL Tutorial has a practical tutorial on Multi-agent reinforcement learning.
For those who want to dive deeper, it has a reading list at the end that covers its theory and more advance techniques.


text:
Can I use the SIFT feature detector on data other than images?
I know how to use SIFT algorithm for images but I never use it for other kinds of data. I have tabular data (x, y, z, time) where x,y,z is the joint position along x, y, z coordinates. Now, can I apply the SIFT algorithm to this data to find features that will act as input to traditional machine learning algorithms, like SVM, DT, etc.?

Check out this paper: Dense Bag-of-Temporal-SIFT-Words for Time Series Classification (2016). After reading it, maybe you can provide an answer to your own question ;)


text:
It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?
It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?
If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say I have a bunch of smaller examples, where I compute the optimal values by checking all possibilities, can this be then used for bigger problems?
In particular, let's say I take a large graph and just optimize subgraphs of this large graph. This is perhaps a more general question: My experience with deep learning (TensorFlow/Keras) is to predict values. How can I get graph isomorphism and/or a list of local moves on the graph, to obtain a better solution? Can ML/DL give you a list of moves or local changes to get closed to an optimal value, or does it just return the predicted optimal value?

Maybe this article solves your question: Universal Function Approximation on Graphs.

Yes, it is indeed possible to use deep learning to provide approximate solutions to NP-hard graph theory problems. While solving NP-hard problems exactly in polynomial time is generally considered computationally infeasible, deep learning techniques can be employed to approximate solutions with impressive results in some cases.
Here's how it can work:

Encoding Graphs: Deep learning models can take graph structures as inputs. Graph convolutional networks (GCNs), graph attention networks (GATs), and graph neural networks (GNNs) are architectures specifically designed to work with graph-structured data.

Learning from Data: The models are trained on a dataset that includes graph instances and their corresponding optimal or near-optimal solutions. These solutions could be obtained through heuristics, metaheuristics, or other optimization methods.

Loss Function: The loss function used during training compares the model's predictions to the known solutions. The aim is to minimize the discrepancy between the predicted and actual solutions.

Approximate Solution: After training, the model can be used to predict solutions for new graph instances. These solutions might not be optimal, but they can be very close to optimal and obtained much faster than traditional exact methods.

Trade-off Between Accuracy and Efficiency: The balance between solution quality and computational efficiency can be adjusted by controlling the complexity of the model, the amount of training data, and the training process itself.

Transfer Learning: Pretrained models on one problem can be fine-tuned on related problems. This transfer learning can help speed up training and improve solution quality.

Hyperparameter Tuning: Like in other deep learning tasks, the performance of the model can be sensitive to hyperparameters. Finding the right combination can significantly affect the quality of the approximated solutions.

Evaluation: The quality of the approximate solutions produced by the model can be evaluated against known optimal solutions or against the solutions produced by other approximation algorithms.

It's important to note that the success of using deep learning for NP-hard graph problems can vary widely depending on factors such as the problem itself, the availability of suitable training data, the architecture of the neural network, and the efficiency of the training process. Some problems might benefit more from these techniques than others.
Examples of NP-hard graph theory problems that have been tackled with deep learning include the Traveling Salesman Problem (TSP), Minimum Vertex Cover, Maximum Independent Set, and more. While the solutions provided by deep learning might not be guaranteed to be optimal, they often strike a good balance between solution quality and computational efficiency, making them valuable in practice.
Thanks!!!


text:
How to normalize images before training?
I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers there were not enough enlightening for me.

tl;dr subtracting the mean and dividing by the standard deviation is theoretically more sound, but is impractical compared to dividing by $255$.

Explanation
As you know neural networks perform better when their input is scaled. The 2 most common ways to perform scaling are:

normalization, where you want to scale the image to the $[0, 1]$ range.
standardization, where you want to bring the mean to $0$ and the standard deviation to $1$.

Theoretically, I think, the latter has some advantages, however in practice there is not a significant difference, especially if the network has normalization layers.
Images are usually stored in 8-bit color mode, meaning that they take integer values from 0 to 255. Because you know the minimum, and maximum values that each pixel can take, it is extremely easy to normalize an image (i.e. you simply have to divide each pixel value by $255$).
On the other hand it is much less practical to compute the mean and standard deviation of the pixel intensities of the training set. Keep in mind that in most cases the training set doesn't even fit into memory, and you have to approximate the true mean/std by moving averages (i.e. load a few images, compute their mean/std, close them, load some more, add their mean to the MA, ...). But why bother if you can similar results by simply dividing each image by $255$.

I guess you mean Batch Nornalization by normalization layers right?

Yes, batch norm, layer norm, group norm, instance norm, etc.


text:
Did the Hutter Prize help research in artificial intelligence in any way?
Wikipedia states:

The Hutter Prize is a cash prize funded by Marcus Hutter which rewards data compression improvements on a specific 1 GB English text file.
The goal of the Hutter Prize is to encourage research in artificial intelligence (AI). The organizers believe that text compression and AI are equivalent problems.

Did the Hutter Prize help research in artificial intelligence in any way?


text:
Given the word embeddings, how do I create the sentence composed of the corresponding words?
I have done some reading. I want to implement an LSTM with pre-trained word embeddings (I also have plans to create my word embeddings, but let's cross that bridge when we come to it).
In any given sentence, you don't usually need to have all the words as most of them do not contribute to the sentiment, such as the stop words and noise. So, let's say there is a sentence. I remove the stop words and anything else that I deem unnecessary for the project. Then I run the remaining words through the word embedding algorithm to get the word vectors.
Then what? How does it represent the sequence or the sentence 'cause it's just vector for a word.
For example, take the sentence:

The burger does not taste good.

I could remove certain words and still retain the same sentiment like so:

Burger not good.

Let's assume some arbitrary vectors for those three words:

Burger: $[0.45, -0.78, .., 1.2]$

not: $[9.6, 4.0, .., 5.6]$

good: $[3.5, 0.51, 0.8]$

So, those vectors represent the individual words. How do I make a sentence out of them? Just concatenate them?


text:
Why would my neural network have either an accuracy of 90% or 10% on the validation data, given a random initialization?
I'm making a custom neural network framework (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or get stuck at 10-9% (on validation set).
I shuffle all my data before feeding it to the neural net.
Is there a better randomizer I should be using, or maybe I am not initializing my weights properly (Using srand to generate values between +/-0.1). Did I somehow hit a saddle point?
My network consists of 784 size input layer, 256, 64, 32, 16 neuron hidden layers, all with RELU, and 10 output with SMAX
Where should I start investigating based on this kind of behavior, when I can't even replicate what is going on?

This could be due to exploding gradients maybe. I'd suggest you to train with gradient clipping and see how the model learns.

@SpiderRico wouldn't exploding gradients produce nan outputs (and also cause nan loss)? My loss and acc are both normal looking values, but I have not checked what the gradient looks like so I will look into it, thank you

Correct, loss should be NaN but you mentioned accuracy getting stuck at 10%. This is equivalent to random guessing for MNIST (as there are 10 classes), and can be caused by exploding grads.

How are you optimising your network? If you're using some form of gradient descent, is the learning rate you're using reasonable?

If you're using srand for a number between +-0.1, you should verify that none of your weights are initialized to zero.  If they are, they will stay there.


text:
How to train a neural network with few weights and biases held constant?
I am a beginner in neural networks. I am building a neural network with 3 layers. The input $X$ has 7 features and the output $Y$ is a real number. In the hidden layer, there are two nodes. The bottom node contains weights and biases which should be hard set.

Now, I want to train this neural network with the training data $X$ and $Y$, such that the red weights are held constant while all other weights are learnable.
Is there a way of doing this during the training of the neural network? I'm using TensorFlow and Keras, so, if you could provide also the code necessary to do this, that would be very useful.

maybe set weights for b2 or train it first, and use it as a function

The strategy is turning b2 into a separate model, initialise b2 the way it should be, and train your network without b2 as usual.
In the middle of the main network, combine the output of b1 layer and b2 network using concatenation function, for example in TensorFlow:
# Axis 0 is the batch dimension, axis 1 for the dimension of value in every sample
tf.concat([b1out,b2out],axis=1)

Example source code (paste to Google Colab to test):
%tensorflow_version 2.x
%reset -f

import tensorflow                   as     tf
from   tensorflow.keras             import *
from   tensorflow.keras.layers      import *
from   tensorflow.keras.activations import *
from   tensorflow.keras.models      import *
from   tensorflow.keras.callbacks   import *

class MyModel(Model):
    def __init__(self):
        super(MyModel,self).__init__()
        self.dense1 = Dense(200, activation=relu)
        self.dense2 = Dense(1,   activation=tf.identity)

    @tf.function
    def call(self,x):
        # MIND THE CONCATENATION IN THIS PART:
        h1a = self.dense1(x)
        h1b = b2(x) # b2 won't get trained, so it stays fixed

        # CONCATENATION AT AXIS 1, COZ AXIS 0 IS BATCH DIMENSION
        h1  = tf.concat([h1a,h1b],axis=1)
        u   = self.dense2(h1)
        return u

class ModelB2(Model):
    def __init__(self):
        super(ModelB2,self).__init__()
        self.dense1 = Dense(200)
        # self.dense2 = ...
        # Init weights of B here or B is pre-trained model

    @tf.function
    def call(self,x):
        u = self.dense1(x)
        return u

# PROGRAMME ENTRY POINT ========================================================
if __name__=="__main__":
    inp = [[1,2,3,4,5,6,7],[7,6,5,4,3,2,1]] # Example values
    exp = [[0],            [1]            ]

    mm = MyModel()
    b2 = ModelB2()

    mm.compile(loss=tf.losses.MeanSquaredError(), optimizer=tf.optimizers.Adam(1e-3))
    b2.compile(loss=tf.losses.MeanSquaredError(), optimizer=tf.optimizers.Adam(1e-3))
    mm_loss = mm.evaluate(x=inp,y=exp, batch_size=len(inp), steps=1) # Init weights in here
    b2_loss = b2.evaluate(x=inp,y=exp, batch_size=len(inp), steps=1) # Init weights in here

    print("\nbefore training:")
    print("mm weights:",mm.get_weights()[0][0][:3],"...")
    print("b2 weights:",b2.get_weights()[0][0][:3],"...")
    print("mm loss:",mm_loss)
    print("b2 loss:",b2_loss)

    mm.fit(x=inp,y=exp, batch_size=len(inp), epochs=500, verbose=0)
    mm_loss = mm.evaluate(x=inp,y=exp, batch_size=len(inp), steps=1)
    b2_loss = b2.evaluate(x=inp,y=exp, batch_size=len(inp), steps=1)

    print("\nafter training:")
    print("mm weights:",mm.get_weights()[0][0][:3],"...")
    print("b2 weights:",b2.get_weights()[0][0][:3],"... <-- UNCHANGED AS WANTED")
    print("mm loss:",mm_loss)
    print("b2 loss:",b2_loss,"<-- UNCHANGED AS WANTED")
# EOF

Google colab: https://colab.research.google.com


text:
Is there a way, while training (with contrastive learning) the embedding network, to find the test accuracy?
I aim to do action recognition in videos on a private dataset.
To compare with the existing state-of-the-art implementations, other guys published their code on Github, like the one here (for the paper Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework). Here, the author first trains the embedding network (3D ResNet without final classification layer) with contrastive learning. Finally, he adds a final layer and finetunes the weights, training the whole network again for some epochs.
Now, here is my doubt: Is there a way, while training just the embedding network, to find the test accuracy?
One way to tell if the final accuracy after finetuning would be good is to see if the training loss is decreasing or not. If the training loss decreases, that certainly builds up the hope that the test accuracy would be improving during the training, but in no way gives an idea about how much the test accuracy would be.
Another way is to plot the t-SNE, see if, on the test data, the data points from the same class are close together, thus forming a cluster. Then it could be said that the test accuracy would also be good. But it's not quantifiable, and hence it would be hard to compare the t-SNE plots obtained from two different models.
I was also suggested to add a final layer to my embedding network and just test it on the test data, without training or fine-tuning again. The reason for that is that the embedding network should have learned the weights reasonably by now; even if I finetune the model, the test dataset's test accuracy won't vary a lot. I need some advice here. Is that suggestion good? Are there any potential pitfalls with this suggestion?
Or do you have any other possible suggestions I could try?


text:
Where does the hierarchical reinforcement learning framework name "MAXQ" come from?
I've been researching different frameworks for hierarchical RL (mainly options, HAMs, and MAXQ) and noticed that both options and HAMs have names that relate to how they function. I can't seem to find anything stating how MAXQ got its name and I was wondering if anyone knew what the name was referencing.

Hello. Have you already looked at the MAXQ paper? The answer probably lies there. If you find the answer to your own question, feel free to write a formal answer below, so that future readers can also see it.


text:
How can I select features for a symbolic regression problem to be solved with genetic programming?
I want to solve a symbolic regression problem with genetic programming. My dataset is similar to this one, but I have 30 features, and I want to use only the most sensitive features. I found this library interesting for Symbolic Regression, but could not find the right approach for feature selection.

Have you tried to look at the existing feature selection techniques typically used in machine learning? If yes, why aren't they good for you? If not, it may be a good idea to have a look at them. Once you know the answer to your question, feel free to write it below (if it's good ehough) ;)

You should probably read this: Feature Selection to Improve Generalization of
Genetic Programming for High-Dimensional Symbolic Regression.

I do not have full access to that paper, but it claims as follows, In this paper, we propose a new feature selection method based on permutation to select features for high-dimensional SR using GP. So, permutation using GP. what's it really?

You can find a freely available version of this paper (or a draft) on the web.

Is there a python library for GP?

Yes, you can use DEAP. Anyway, if you find an answer to your own question, feel free to write it below, with some details and maybe a link to the references that you used.


text:
Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?
In a linear regression problem, a line can divide a data set into two categories. So, basically, points above the line belong to category 1, and points below the line belong to category -1.
However, my professor has asked me to write a C++ program in which the program will classify whether the data points lie above or below a sine function.
Let me explain a bit more. So, first, we will generate a data set $$D = \{(x_i, y_i) \} \label{0}\tag{0} $$ with random $x$ and $y$ coordinates, for example, according to this equation
$$y(x) = A + B \sin(Cx)\label{1}\tag{1},$$
where $A$, $B$, and $C$ are known.
The data points above the sine function will have a label 1 on them, and the points below the function will have -1.
Now, this data set $D$ in \ref{0} has to be fed to a C++ program. This C++ program has to somehow learn the curve separating the two data point categories. After training, the program will then classify some new query data points.
The key difficulty is that the program does not know in advance that the points were scattered around a sine curve. It does not know the values of $A$, $B$, or $C$ in equation \ref{1}. It also does not know that the curve is a sine curve.
Now, this is where I am stuck. I do not know if I need to use a neural network to solve this problem. If a neural network is to be used, then I presume that backpropagation will have to be used in some way. I can generate the data set and I can feed the data into the program.
Which approach (algorithm and model) should I use to solve this problem?
I have studied linear classification with the perceptron learning algorithm, but this sine-classifier stuff is a huge step-up for me. Another important thing is that I am not allowed to use any ready-made C++ libraries for Machine Learning. If a neural network solution is needed, then I will have to design the neural network from scratch. Note that I don't need any C++ code, but I am just looking for some guidance on how to approach this problem.

You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y_i, x_i)$ where $y_i$ is $i$-th output, and $x_i$ is $i$-th input then you can transform your input variable into vector
\begin{equation}
\phi = [1, \sin(x), \cos(x), \sin(2x), \cos(2x), \ldots, \sin(nx), \cos(nx)]
\end{equation}
where $n$ is some arbitrary number.
Then you will have a standard linear regression model
\begin{equation}
Y = A\Phi
\end{equation}
where $Y = [y_1, \ldots, y_n]$, $\Phi = [ \phi_1, \ldots, \phi_n]$ and $A$ is matrix of unknown parameters that you need to learn.

Comments are not for extended discussion; this conversation has been moved to chat.


text:
How to find good features for a linear function approximation in RL with large discrete state set?
I've recently read much about feature engineering in continuous (uncountable) feature spaces. Now I am interested what methods exist in the setting of large discrete state spaces. For example consider a board game with grid as a basic layout. Each position on a grid can contain exactly one of multiple elements and the agent makes decisions according to the current board position. If the grid is large enough, say 30x30, and there are only two different elements we could model the states as a linear model with $2*30*30 = 1800$ variables (using dummy variables) and this model can't even distinguish relationships between positions. For this we would need to use $\binom{90}{2}$ or even $\binom{90}{k}$, $k = 2, 3, 4$ more features.
How would one approach this problem? Are the methods for feature selection for linear approximations, which even automatically find/learn non-linear combinations? What was the approach to solving these problems when NN where not around?

@DuttaA So it's almost necessary to rely on hand crafted features?

@DuttaA Ok, of course NN do feature engineering themselves, but for linear models the features have to be selected separately (and we can't use all features).

@DuttaA Thank you really much for the insight, I'll check out the lecture as well as deep blue.


text:
Is there a full and precise formulation of Theorem 1 in the Integrated Gradients paper?
Theorem 1 (page 5)  in the paper about Integrated Gradients states that

Integrated gradients is the unique path method that is symmetry-preserving.

What I miss is

A precise formulation of the theorem: in particular, the exact properties that must be satisfied by the function $f$ used in the proof (continuity, differentiability, etc.). Also, should the paths be assumed to be monotonic?

A consistent definition of function $f$ in the proof - note that $f$ is defined inconsistently, e.g. in the region where $x_i<a$ and $x_j>b$, where it is not clear whether its value should be $0$ or $(b-a)^2$.

Point 2 is easy to fix with an appropriate redefinition (e.g. replacing "if $\text{max}(x_i,x_j)\geq 0$" with "else if $\text{max}(x_i,x_j)\geq 0$"). What it is not clear if whether there is a redefinition that:

preserves the properties that have been assumed in the rest of the paper, in particular in Proposition 1 (proving completeness), where the function is assumed to be continuous everywhere, and the set of discontinuous points of each of its partial derivatives along each input dimension has measure zero, and

the function is a constant for $t \notin [t_1,t_2]$.

Does anybody have a precise formulation and full proof of the theorem?

Well, I just realized that Proposition 1 in the paper cannot be correct as stated either. In one dimension that proposition is just the (second) Fundamental Theorem of Calculus stated for functions that verify Lebesgue’s integrability criterion, but Cantor's staircase function is a well known counter-example showing that such condition is not enough to ensure the conclusion of the FTC. The premises of the proposition would need to be relaxed somehow, but still keeping them strong enough to be applied to deep networks with functions such as ReLu and max pooling.

Back to Proposition 1, for univariate functions the condition should be "absolutely continuous" (rather than differentiable almost everywhere) - this leads to a suitable generalization of the Fundamental Theorem of Calculus that can be applied to functions occurring in deep networks, but I am still wondering how to generalize the gradient theorem in a similar way so it can be applied to functions implemented by deep networks. That generalization would contain premises to be used in the formulation of Theorem 1.

After working on it for a while this is what I got.
Concerning proposition 1 in the paper, a rigorous statement could be the following version of the Gradient Theorem for line integrals:
Proposition 1. (Gradient Theorem for Lipschitz Continuous Functions).
Let $U$ be an open subset of $\mathbb{R}^n$.  If
$F : U \to \mathbb{R}$ is Lipschitz continuous, and
$\gamma : [0,1] \to U$ is a smooth path such that $F$ is
differentiable at $\gamma(t)$ for almost every $t\in [0,1]$, then
$$
    \int_{\gamma} \nabla F(\mathbf{x}) \cdot d\mathbf{x} = F(\gamma(1)) - F(\gamma(0))
    \,.
  $$
I found theorem 1 about symmetry-preserving harder to state rigurously while still fully capturing its intended meaning - this is the best I got:
Theorem 1.
Given $i,j\in \{1,\dots,n\}$, $i\neq j$, real numbers $a<b$, and a
strictly monotonic smooth path $\gamma : [0,1] \to (a,b)^n$
such that $\gamma_i(0)=\gamma_j(0)$ and $\gamma_i(1)=\gamma_j(1)$,
then the following statements are equivalent:
(1) For every $t\in [0,1]$, $\gamma_i(t) = \gamma_j(t)$.
(2) For every function $F : [a,b]^n \to \mathbb{R}$ symmetric
in $x_i$ and $x_j$ and verifying the premises of
proposition 1 with $U=(a,b)^n$ we have
$\int_{\gamma} \frac{\partial F(\mathbf{x})}{\partial x_i} \, dx_i
= \int_{\gamma} \frac{\partial F(\mathbf{x})}{\partial x_j} \,
dx_j$.
Further details and proofs: Symmetry-Preserving Paths in Integrated Gradients


text:
If I want to predict two unrelated values given the same sequence of data points, should I have a model with two outputs or two models?
I want to predict two separate y-values (not really logically connected) based on an input sequence of data (values x). Using LSTM cells.
Should I train two models separately or should I just increase the dimension of the last layer to 2 instead of 1 and feed the fitting algorithm y-values as 2D pairs? In other words, will there be a lot of interference or can I expect on average similar results with one or two models?

Just to be clear, there is only one x sequence for each two different y values? The two y values are therefore two different kinds of thing associated with the same type of x sequence?

yes. For example, if I am trying to predict the next value of the sequence m step ahead and n steps ahead, n > m.
Another example is I am observing a multi-valued function $f:R \rightarrow R^2$ and the values are somewhat related and I'm trying to predict a new pair.


text:
What are examples of good free books that cover the back-propagation algorithm?
What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that cover the theoretical basics of back-propagation.

this is not a book, but if you want to try understand backprop calculation step-by-step, this blog is amazing: https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

@malioboro this blog falls into the same pitfall that most do - it doesn't cover the case of multiple hidden layers. While the task isn't actually that much more complicated, the notation used quickly falls apart because it becomes way too long to write out. This made learning back propagation initially for me extremely difficult, because it seems no one actually covered the case of multiple hidden layers. Until I found this: https://youtu.be/d14TUNcbn1k?t=354

@Recessive Oh I see, multiple hidden layers means more than one hidden layer, that's true the notation will become complicated

@malioboro Thank you. I actually was looking for different sources as books, videos, papers, etc. but moderator nbro asked me to modify my post and to focus only on 1 source.

Deep Learning by Goodfellow et. al is a good book for anything related neural networks, and it's freely available online. Backpropagation is covered in Chapter 6.5.

Thanks. Can one go straight to Chapter 6.5. to read the 21 pages or is it required to read previous text?

@veronika Well I guess it really depends on your background. You can try reading it and see if you can follow the material.


text:
Setting up a deep learning architecture for multi-dimensional data
The input data is thousands, millions of 4x1000 matrices. Each row consists of 3 small natural numbers (1000 combinations) and a corresponding real number between 0 and 1.
The output is a 1x1000 vector for each of the matrices. Each output vector value [1 or 0] is not defined by the corresponding 4-argument entry, but the whole 4x1000 matrix. Each input matrix defines a few valid output 1x1000 vectors that cannot be computed analytically from the 4x1000 matrices.
What would be the options for setting up a deep learning architecture to try to tackle this challenge?

Maybe you should describe the nature of your data and not just their dimensions and what task are you trying to solve.


text:
Interesting examples of discrete stochastic games
SGs are a generalization of MDPs to multiple agents. Like this previous question on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'm hoping to use such examples as benchmarks, but couldn't find much in the literature. One example I can think of is a pursuit-evasion game on a graph.

Some of the domains in the International Probabilistic Planning Competition, such as the Wildlife Preserve benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, RDDL. This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.
There's also a quite useful project that allows to instance OpenAI gym environments from the declarative description of the environment, states and actions.

Thanks for your answer. I looked at the two papers listed under Wildlife Preserve in the page you linked to, but it seems neither is a multi-state game?

From the definition of the instance (see this gist, file wildlife-preserve_inst_mdp__01.rddl), and the definition of the state variables (see section state-fluents in the file wildlife-preserve_01_mdp.rddl posted in the gist above) the smallest instance in the suite has 2^4 x 4 x 4 x 2 = 512 states.
I recommend you too to play around with the rddlgym tool I linked in my answer, will make easier to come to grips with the syntax of RDDL.


text:
Benchmarks for reinforcement learning in discrete MDPs
To compare the performance of various algorithms for perfect information games, reasonable benchmarks include reversi and m,n,k-games (generalized tic-tac-toe). For imperfect information games, something like simplified poker is a reasonable benchmark.
What are some reasonable benchmarks to compare the performance of various algorithms for reinforcement learning in discrete MDPs? Instead of using a random environment from the space of all possible discrete MDPs on $n$ states and $k$ actions, are there subsets of such a space with more structure that are more reflective of "real-world" environments? An example of this might be so-called gridworld (i.e. maze-like) environments.
This is a related question, though I'm looking for specific examples of MDPs (with specified transitions and rewards) rather than general areas where MDPs can be applied.
Edit: Some example MDPs are found in section 5.1 (Standard Domains) of Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search (2012) by Guez et al.:

The Double-loop domain is a 9-state deterministic MDP with 2actions,
  1000 steps are executed in this domain. Grid5 is a 5×5 grid with no
  reward anywhere except for a reward state opposite to the reset state.
  Actions with cardinal directions are executed with small probability
  of failure for 1000 steps. Grid10 is a 10×10 grid designed like Grid5.
  We collect 2000 steps in this domain. Dearden’s Maze is a 264-states
  maze with 3 flags to collect. A special reward state gives the number
  of flags collected since the last visit as reward, 20000 steps are
  executed in this domain.

Although I am not aware of any "benchmark problems" for (discrete) MDPs, I'll comment a bit on possible benchmarks and I will show some benchmarks used to test POMDP algorithms.
MDP vs POMDP
In Markovian Decision Processes (MDPs) the whole state space is known, this means you know all the information for your problem; therefore, you can use them to find solutions for perfect information problems or games.
Many of these games could use an MDP, some examples: 2048 and chess. Note that you must take into mind that the computational complexity grows with the number of states.
Although I could not find any benchmarks for MDPs, games with perfect information can be used to compare MDP solvers.
When a problem or game has imperfect information, you should use a  Partially Observable Markovian Decision Processes (POMDPs); in which you do not need to know the current state, but you keep track of the probabilities of being in any of the (discrete) states.
POMDP Benchmarks
Since I worked with POMDPs, I will comment some of the benchmarks researches used for discrete POMDPs (Pineau et al. (2003), Spaan and Vlassis (2004), Kurniawati et al. (2008), Ong et al. (2010), ArayaLopez et al. (2010)):

Tag: a robot and target move in a grid environment and can move one step at a time, moving has a cost, and a reward is gained if the robot is at the same position as the target (i.e. tagged it).
Two-Robot Tag: two robots attempt to catch a target, thereby sharing their observations and actions; the target tries to get away from them.
Mazes (Littman et al. (1995), Kaelbling et al. (1998), Spaan and Vlassis (2004)):

Hallway and Hallway2 are robot navigation tasks in a hallway, where the robot has only local noisy sensor information. The difficulty of hallways is it being long areas which look alike, which causes ambiguity in the localization.
Tiger-grid a two world state with a tiger being either behind the left or right door. The actions are listen, open the right or left door and there is a positive reward when opening the door without the tiger, otherwise a large negative reward.

Rock Sample: a rover explores a grid area, it knows its own position and the position of the rocks, however, it does not know which rocks are valuable. The rover can sense how valuable they are, but this sensor is less reliable when it is used farther away.

The tag game: the robot (blue) and target on a map with 29 positions and 870 states (29 for the robot, 29 + 1 (tagged) for the target).
These problems tend to be of the same size (number of states and actions) such that the results of different algorithms can be compared easily.
References:

Araya-Lopez, M., Thomas, V., Buffet, O., and Charpillet, F. (2010). A closer look at MOMDPs. In 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, volume 2, pages 197–204.
Kaelbling, L.P., Littman, M.L., Cassandra, A.R. (1998).
Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2): 99-134
Kurniawati, H., Hsu, D., and Lee, W. (2008). SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. In Proceedings of Robotics: Science and Systems IV, Zurich, Switzerland.
Littman, M.L., Cassandra, A.R. and Kaelbling, L.P. (1995). Learning policies for partially observable environments: Scaling up. in Proc. 12th Int. Conf. on Machine Learning, San Francisco, CA.
Ong, S. C. W., Png, S. W., Hsu, D., and Lee, W. S. (2010). Planning under Uncertainty for Robotic Tasks with Mixed Observability. The International Journal of Robotics Research, 29(8):1053–1068.
Pineau, J., Gordon, G., and Thrun, S. (2003). Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI),
pages 477–484.
Spaan, M. T. J. and Vlassis, N. (2004). A point-based POMDP algorithm for robot planning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 2399–2404, New Orleans, Louisiana.


text:
What are the top artificial intelligence journals?
What are the top artificial intelligence journals?
I am looking for general artificial intelligence research, not necessarily machine learning.

A couple of others:

Journal of Artificial Intelligence Research (JAIR)
IEEE Transactions on Knowledge and Data Engineering
IEEE Computational Intelligence Magazine

This link includes various journals for artificial intelligence applied to various domains.
Some of those are:

IEEE Transactions on Human-Machine Systems
Journal of the ACM
Knowledge-based systems
IEEE Transactions on Pattern Analysis and Machine Intelligence
Journal of Memory and Language.

There are many more. You can refer to any of those journals and explore the research done by AI enthusiasts and researchers.

I most often reference:
http://dblp.uni-trier.de/
It's not a journal but it gets me where I need to go.

The journal "Artificial Intelligence (AI)" (https://www.journals.elsevier.com/artificial-intelligence/) was not listed, yet, although being considered the top-level journal on AI. Although this is a journal for AI (just being named "Artificial Intelligence"), it is not to be confused with another top-level AI journal, called "Journal on Artificial Intelligence Research (JAIR)" (http://www.jair.org/), which was already listed in one of the other answers.
Further, there is a German Journal on AI, called "KI - Künstliche Intelligenz" (German for AI), but almost always the articles are in English as well (http://www.kuenstliche-intelligenz.de/en/ki-journal/). While being internationally recognized, it is not regarded a top-level journal. A nice feature of that journal is that every special issue has an editorial (a special "article" at the beginning of each journal), in which there is a section called "service". This service section lists publication media (like journals) and conferences etc. that are related to the given special issue. So, in case you are interested in journals of a special field of AI (like human-computer interaction), just search for a special issue that is related to that topic and read the editorial's service part.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Knapsack of mixture with constraints
I'm trying to find the optimized mixture for a specific set of substances. Each of those substances have characteristics that I want to optimize in the mixture (some characteristics I want to minimize and others I want to maximize). But I can't have more than 50% (random value that will be set on running time) of one of those substances in the mixture.
I thought about using Genetic Algorithm, but I'm not sure it's the best approach for this problem. Do you have any suggestions?
Edit: it doesn't need to be a evolutionary algorithm.

Can you calculate a single value that reflects the optimization? If so some sort of simplex would work.

@MaxW I'll try the simplex. Thanks, I think that will do.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How does an svm work? How does it perform comparisons between malignant and benign tumor
How do Support Vector Machines (SVMs) differentiate between a glass and a bottle or between a malignant and a benign tumor when it dealing with it for the first time?
What will be the analysis mechanism involved in this?

https://en.wikipedia.org/wiki/Support_vector_machine

How does a Support Vector Machine (SVM) work?

I will try to give you a simplified explanation of how SVMs work.
The data one works with can be of two types. Either it is very easily separable and there is a clear straight line boundary between data points of different classes. We call such data linearly separable(image) or the data points of the classes are mixed in a way that there is no clear straight line boundary that separates them. In other words the data is not linearly separable.(image).
One way to make data points linearly separable is to map them into a higher dimension where they would become linearly. If we have a set of data points which are not linearly separable in 2 dimensions, we could map them into 3 dimensions where the data becomes linearly separable(like so).
SVMs work by converting data from lower to higher dimensions where it is linearly separable and then tries to find the boundary that separates the data.
The same concept could be applied to images, where the images are converted into a very high dimension in which images of different types can be separated linearly.
This could mean that the SVM model trained is able to easily distinguish between a bottle and a glass or different tumour types but that would depend entirely on the data used to train the model. The problem arises when the distribution of the training data and testing data is completely different. So for example, you trained the SVM to distinguish between glasses and cups of a certain shape and colour and while testing you show it glasses of completely different shapes and colours, the SVM will not be able to distinguish between them.


text:
Would an AI with human intelligence have the same rights as a human under current legal frameworks?
For example, would an AI be able to own property, evict tenants, acquire debt, employ, vote, or marry? What are the legal structures in place to implement a strong AI into society?

As this question is currently worded, it is asking for an opinion. You could try asking "what legal structures or processes would be required to enable an AI to ..." or "How could an AI legally ..."

@Jnani, okay thanks for your suggestion. I've re-worded.

Yes, to some of what you propose.  No to some.
Today corporations are granted rights: to own property, earn income, pay taxes, contribute to political campaigns, offer opinion in public, ad more.  Even now I see no reason why an AI should not be eligible to incorporate itself, thereby inheriting all these rights.  Conversely, any corporation already in existence could become fully automated at any time (and some plausibly will).  In doing so, they should not lose any of the rights and duties they currently employ.
However I suspect certain rights would be unavailable to an AI just as they are unavailable to a corporation now: marriage, draft or voluntary service in the military, rights due a parent or child or spouse, estate inheritance, etc.
Could this schizoid sense of human identity be resolved at some point?  Sure.  Already there have been numerous laws introduced and some passed elevating various nonhuman species to higher levels of civil rights that only humans heretofore enjoyed: chimpanzees, cetaceans, parrots and others have been identified as 'higher functioning' and longer lived, and so, are now protected from abuse in ways that food animals, pets, and lab animals are not.
Once AI 'beings' arise that operate for years and express intelligence and emotions that approach human-level and lifetime, I would expect a political will to arise to define, establish, and defend their civil rights.  And as humans become more cybernetically augmented, especially cognitively, the line that separates us from creatures of pure silicon will begin to blur.  In time it will become unconscionable to overlook the rights of beings simply because they contain 'too little flesh'.

Props for bringing up the idea of AI person-hood in the form of a corporate structure.  The "Citizens United" decision is a precedent for extending rights originally restricted to humans to corporate entities.

I can see clear room on the side of fiction novels for portraying this circumstance well in advance of its eventual evolution.

Murray Shanahan, in his book The Technological Singularity, makes the case that the rights of any being are determined by its intelligence.
For instance, we value the life of a dog above that of an ant and likewise value human life above that of other animals.

From here one could argue that a general artificial intelligence of equal intelligence to a human should have equal rights to a human and a superior artificial intelligence should have more rights.

The question, of course, is whether our anthropocentric society would be willing to accept this fundamental shift in human rights and this idea of removing humanity from its pedestal of importance.
When it comes to legal frameworks, we really are entering into uncharted territory as AI is going to have to revolutionise the way we define many of the terms we take for granted today and question many of our usual assumptions.

AI is going to drive an important shift in our mindset well before it exceeds human intelligence.

Not only wouldn't a strong AI which came into existence today have the rights a human has, or any rights (see these discussions of the implementation of regulation for weak AIs at: The White House and The American Bar Association),  but it seems unlikely the first one will.
Observing that:

Having rights implies that there are restrictions, which means there would have to be a system of control. However the control problem in AI is still unsolved.
Even assuming that problem is solvable, an AGI would then have to appear equivalent to natural humans. They don't yet (see Turing Test Passed?), and even after passing equivalence tests, are unlikely to remain that way, per the Singularity Hypothesis.
Further, if one or more AGIs were to be human-equivalent long enough to desire rights, lawmakers (in the US) would have to re-interpret the definition of personhood and grant them rights, as they did for corporations in 1886.

A sufficiently clever AGI, if self-interested, would pre-empt or co-opt existing legal structures, to seize whatever juridical rights it desired, as the opportunity arose.  Thus it would render my opinions on the subject entirely moot.
Another way of putting this point:  While current legal frameworks would not provide any rights to an artificial agent, current legal frameworks foreseeably will no longer be current, once an AI exists having attributes which imply the transformative change of those frameworks.

There is a legal difference between a "person" (which includes bodies corporate - corporations, incorporated associations, etc - and actual people) vs "natural person" (which is specifically a human being).
For an AI to marry, it would need to get the legal definition of "natural person" changed, and depending on the jurisdiction possibly also the definition of "man" or "woman".
For other things, such as owning property, evicting tenants, entering into contracts, etc, an AI would simply use a corporation. It may be that the corporation might need to have a minimum number of directors who are natural persons, but they could just be paid professionals, so no issue there.
With credit cards, it would depend on the policy of the issuing bank. There is no legal impediment to corporations having credit cards in their own right, but in practice banks often require a director's guarantee from a natural person that they can sue if the bill is not paid. They want to be sure they will get their money, even if the corporation is wound up.


text:
Classifying non-labeled data with high dimensionality
Disclaimer: I am a novice in the world of machine learning, so please excuse my ignorance.
My dataset consists of things like age, days since last visit, etc. This information is medical related. None of which is geometrical, just data pertaining to particular clients.
The goal is to classify my dataset into three labels. The dataset is not labeled, meaning I'm dealing with an unsupervised learning problem. My dataset consists of ~20,000 records, but this will linearly increase overtime. The data is nearly all floats, with some being strings that can easily be converted into a float. Using this cheat sheet for selecting a solution from the scikit site, a KMeans Cluster seems like potential solution, but I've been reading that having high dimensionality can render the KMeans Cluster unhelpful. I'm not married to a particular implementation either. I've currently got a KMeans Cluster implementation using TensorFlow in Python, but am open for alternatives.
My question is: what would be some solutions for me to further explore that might be more optimal for my particular situation?

can you describe more about the dataset rather than saying that is just "floats" , i mean do they have any geometrical significance like 3d points?, it would be helpful to give a description of what the data is about and what the sample data would look like rather than just mentioning "records"

@thecomplexitytheorist I've updated the question, I hope that helps.

you can start by finding correlation between the features , this really helps in understanding the data then you can use a set of those features based on their correlation properties , based on your description i would try DBSCAN first

I would recommend to have a look at Finding Groups in Data, which is a very readable introduction to clustering methods. It gives a good overview over a number of different algorithms, both agglomerative and hierarchical. As far as I remember, source code for the various algorithms is available on the web somewhere.
I am sure you will find a fitting algorithm for your problem in there.

This is supposed to be a comment but I haven't got enough reputation to do that.
In addition to what @the complexitytheorist has said, I recommend you to have a deeper look at your data first, using dimension reduction and visualisation methods such as PCA and t-SNE. A better understanding of data may always save you a lot of work.
Then you can choose which clustering algo to use. For example, KMeans or DBSCAN as a start.


text:
CNN attention maps on non-images
My datasets are not actual images, so using methods with ImageDataGenerator or pre-trained networks might not apply in this case.
Data Structure: Each "image" is a 2048-long vector that has float values between 0 and 1.

Each "image" was associated with a label (multi-label classifcation) and the goal is to perform classification via Keras 2D CNN's.
What are common techniques for finding which parts of the "images" contribute most to classification via convolutional neural nets?
I already implemented the CNNs in keras and have already successfully trained on my images.
*No my data is not time series; however, my model works with either the keras Conv1D and Conv2D layers.

What is the premise to consider the data as image? The input vector is essentially a feature vector of size 2048.

I do not explicitly consider it as an image (hence using the quotes around image); however, I used a CNN as it has image-like qualities. The data has spatial features that determine its label. Regardless, is it possible to get any insight to what the CNN learns on? (whether it may be a CAM or something else)

if your data has signal property then CNN would work. What you are looking for is Saliency Map, you need to do guided back prop to see what your CNN is focusing on. Check out raghakot/keras-vis Github repo.


text:
Identifying cars using deep learning
I would like to use deep leaning for identifying cars; I want the system to predict wether an object is a car or not. How can I do that knowing that im still a beginner in the Deep Learning field ?
I am considering visual recognition. The system must recognize the car from anything else on the road.

The first thing you should consider is about the recognition -- are you considering visual, audio, olfactory, ..l recognition? And, assuming audio, are you recognizing cars vs. what, everything? Recognizing cars from e.g. trucks is much different than recognizing cars from silence. And this is just the beginning. The point is that the question has to be better defined if you want a meaningful answer

@Teusz That doesn't make any sense. The point of deep-learning is to provide a huge data set for the model to figure out how to classify a car vs a truck. There is no need to consider audio from a car image. Your comment is wrong and misleading.

Identification need not be performed only with basis on image data. Possible also to recognize from audio (or audio-visual) data.

@Teusz Well... The question is about "an object is a car or not". OP didn't ask if the sound come from a car or a truck. I still believe your comment is wrong.

Sound can be used to classify objects -- that's the entire field of acoustic event detection. Check it out!

@SmallChess you have been lost in small wires of the internet.hahaha....However,the. OP is kinda a beginner..And one shouldn't complex concepts here,because computer vision is wide.

That's definitely possible with Convolutional Neural Networks. You should follow Google's tutorial at:

https://www.tensorflow.org/tutorials/deep_cnn#cifar-10_model

The tutorial will teach you how to build a deep learning model for classifying cars, trucks etc. It's an easy tutorial to follow.
Your model will build on images pixels. The classifier will know what a car looks like.


text:
Building an AI that generates text by itself
Now I know this might break some StackExchange rules and I am definitely open for taking the thread down if it does!
I am trying to build an AI that can write it's own book and I have no idea where to start or what are the appropriate algorithms and approaches to go with.
How should I start and what do I exactly need for such a project?

Text generation has a long history in computing, and there are plenty of AI areas that might help. This question might be on topic if you explain more about your project goals and where you are stuck. I thnk some details on the nature of the book and its purpose are important, because state of the art language algorithms can generate text with certain stylistic forms, but which is usually vacuous nonsense. Take a look at https://talktotransformer.com/ perhaps - is that the sort of text generation you had in mind?

something like that but not quite exactly, I want to do something like feed the AI some history books about a country (this is just an example because I can't disclose the purpose of the AI) and it generates an alternate history for that country

OK, make up some alternative but similar purpose like the one you suggest and add it to your question. I think that what you want is beyond current AI techniques, especially if you expect anything the AI produces to many any kind of sense. So it may help to understand whether your purpose is in the question to really make such an AI (an impossible task for 2020) or to better understand the technologies that could be involved in such a project and why it is hard to do

There have been many methods proposed for text generating, but recurrent network dominates natural language processing with a key component: the perception of time.
Many networks have been tried for text generation, with notable examples such as Markov chain. However RNN have been proven to work the best and is dominating the field of language modelling (text generation).
How text generation works
A neural network that generates text is commonly called a language model. It is trained on large amount of text with labels being the next token. The text generation process uses several random token as the starting phrase and then the network predicts the rest. However the network does not just predicts the most probable word, instead it randomly chooses one of the few most probable token, hence the generating part.
Why RNN work best on language modelling
RNN have a perception of time built into the architecture of teh network. LSTM, a popular RNN variant used, is composed of "memory units" that "remembers" past text, thus the "time" part. RNN process input according to the sequence of time, so the network can naturally understand time, thus the superior performance compared to other networks.
Architecture of language model
A language model consists of the encoder and the decoder. The encoder compresses word one-hot representation to a smaller sized vector representation. The smaller sized representation is then passed through the decoder, which maps the encoding to the words one hot vectors again.
State of the art results for language modelling
Language modelling is an actively researched field in the AI community, and recently the model GPT-2 have achieved a breakthrough in language modelling accuracy, producing almost human like text with a special component added, the attention layer. Attention basically maps the "memory states" of the encoder and feed it as input to the decoder. The data teh model is trained on is also very large, with over 20GB of web scraped data from sites like Reddit.
Limits of language modelling
One limit of language modelling is the size of generated text. As GPU don't have unlimited memory, language model usually limits the input token size to a specific number, padding or trimming to this number. The number is usually 500-1000, which includes a paragraph or two, but not an entire book. You can only generate short paragraphs and essay with language modelling. For long text it is much harder.
Resources to help you get started
GPT-2 open AI blog: https://openai.com/blog/better-language-models/
GPT-2 online interactive site for text generation: https://talktotransformer.com/
How to train and fine tune GPT-2 in python: https://minimaxir.com/2019/09/howto-gpt2/
Hope I can help you

So what to do when I'm trying to generate large texts that for sure will exceeds 1000 words? should I do it piece by piece?

In short, you simply cannot. The training data of the model is just not a piece of continuous long text, so at the end the AI is going to predict the end token and the sentence/paragraph is going to end.

Recurrent Neural Networks (RNNs) have been applied to generate text. In this blog post you will find a couple of interesting text examples (the author also has made his code available on github), e.g. their Shakespeare-like texts generated by an RNN:

PANDARUS:
  Alas, I think he shall be come approached and the day
  When little srain would be attain'd into being never fed,
  And who is but a chain and subjects of his death,
  I should not sleep.
Second Senator:
  They are away this miseries, produced upon my soul,
  Breaking and strongly should be buried, when I perish
  The earth and thoughts of many states.
DUKE VINCENTIO:
  Well, your wit is in the care of side and that.
Second Lord:
  They would be ruled after this chamber, and
  my fair nues begun out of the fact, to be conveyed,
  Whose noble souls I'll have the heart of the wars.
Clown:
  Come, sir, I will make did behold your worship.
VIOLA:
  I'll drink it.

As you can see the RNN is able to somewhat mimic the "flow" of the texts it has been trained on but some sentences (like at the very end) do not make much contextual sense.
Moreover, RNNs have been trained to generate other content, e.g. drawing numbers (see here) or creating music (see here).

is it a good approach to train the RNN on a large amounts of data (sentences, paragraphs...) ?

Well, that depends on a number of things: what you define as "large amount", your computing power, the amount of data needed to learn the respective task etc.  In the linked blog post the author used up to 100MB to train his Wikipedia article writing RNN. Using a rough ball park figure 100MB of pure text equals about 100 books with 500 pages each.

okay, I got you! so the best approach here is using RNNs and go there! Thank you for your time!


text:
Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers
The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result!
I tried changing the learning rate and iterations but so far no luck!
Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]
Expected result: 467
Result found : 2,795.5
Cost: 504579.43
This is what I've done so far:
import numpy as np

# Init sequence
data =\
    [
        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],
        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]
    ]

X = np.matrix(data)[:, 0]
y = np.matrix(data)[:, 1]

def J(X, y, theta):
    theta = np.matrix(theta).T
    m = len(y)
    predictions = X * theta
    sqError = np.power((predictions-y), [2])
    return 1/(2*m) * sum(sqError)

dataX = np.matrix(data)[:, 0:1]
X = np.ones((len(dataX), 2))
X[:, 1:] = dataX

# gradient descent function
def gradient(X, y, alpha, theta, iters):
    J_history = np.zeros(iters)
    m = len(y)
    theta = np.matrix(theta).T
    for i in range(iters):
        h0 = X * theta
        delta = (1 / m) * (X.T * h0 - X.T * y)
        theta = theta - alpha * delta
        J_history[i] = J(X, y, theta.T)
     return J_history, theta
print('\n'+40*'=')

# Theta initialization
theta = np.matrix([np.random.random(), np.random.random()])

# Learning rate
alpha = 0.02

# Iterations
iters = 1000000

print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial
theta: {}\nInitial J: {:.2f}\n'
  .format(alpha, iters, theta, J(X, y, theta).item()))
print('Training model... ')

# Train model and find optimal Theta value
J_history, theta_min = gradient(X, y, alpha, theta, iters)
print('Done, Model is trained')
print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'
  .format(theta_min[1].item(), theta_min[0].item()))
print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))

# Calculate the predicted profit
def predict(pop):
    return [1, pop] * theta_min

# Now
p = len(data)
print('\n'+40*'=')
print('Initial sequence was:\n', *np.array(data)[:, 1])
print('\nNext numbers should be: {:,.1f}'
  .format(predict(p).item()))

UPDATE Another method I tried but still giving wrong results
import numpy as np
from sklearn import datasets, linear_model

# Define the problem
problem = [1, 3, 7, 8, 21, 49, 76, 224]

# create x and y for the problem

x = []
y = []

for (xi, yi) in enumerate(problem):
    x.append([xi])
    y.append(yi)

x = np.array(x)
y = np.array(y)
# Create linear regression object
regr = linear_model.LinearRegression()
regr.fit(x, y)

# create the testing set
x_test = [[i] for i in range(len(x), 3 + len(x))]

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regr.predict(x) - y) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regr.score(x, y))

# Do predictions
y_predicted = regr.predict(x_test)

print("Next few numbers in the series are")
for pred in y_predicted:
    print(pred)

Cross-posted here: https://datascience.stackexchange.com/q/51033/924

are you hunting my posts? @Anony-Mousse, I posted the same thread on multiple Stack platforms

@nbro please be clear to suggest to only ask in one place, and not to duplicate questions. Otherwise future users will have to look for answers in many places. For the rationale of not allowing crossposting see https://meta.stackexchange.com/questions/64068/is-cross-posting-a-question-on-multiple-stack-exchange-sites-permitted-if-the-qu/64069#64069 Thank you.

I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:
1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like $y = 2x$ or maybe affine function like $y = 2x + 5$. So you would have a sequence like:
$2, 4, 6, 8 ...$ or $7, 9, 11, 13, ...$
If you manage to get that working try a nonlinear function like $x^2$ for example.
2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions
(Universal approximation theorem) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.

well I found a pattern and it goes like this: we take the first element in the array and then we take it's index and next index and then the AI makes the decision which number to choose in that interval, that's how we got the number for example we got 3(2nd number in array) by applying the same logic like this [2^1(index of 2nd element), 3(number 3 exists somewhere between this interval), 2^2(next index)] and so on and so forth for each element (his index and the next...) but since I have no previous knowledge in AI and machine learning I have no idea how to apply this

Like others said you can't approximate this with a linear regression model.
A PRM that approximates a solution could give you the following:
$y = 0.948 + x + 0.00085*x^6$ ~
$y = 237/250 + x + (17/20000)*x^6$
For $x = 9$, $y \simeq 462$
or
$y = 0.9258 + x + 0.00086*x^6$
For $x = 9$, $y \simeq 466.965$
UPDATE
An approximation of course, may be in the range of:
$y = 2^{(x + 1)} - 2^x$ -the model you propose-
Goodness of fit: 0.968475 and Mean Square Error = 685.111
Based on this range a better approximation would be:
$y = 2^x + (-1/2)*x^2$
with $R^2$
Goodness of fit = 0.995
Mean Square Error: 89.0278

I actually found a pattern it is not random the pattern goes like this: you take the element's current index and it's next index and you apply this interval : [2^index, 2^next index] for example for the number 7 in the given example above we do this [2^2, 2^3] the number 7 is somewhere between that interval, the AI must predict based on this pattern. if you have any solution proposed please be kind and guide me through!


text:
How can an AI play Flow Free?
The game "Flow Free" in which you connect coloured dots with lines is very popular. A human can learn techniques to play it.
I was wondering how an AI might approach it. There are certain rules of thumb that a human learns, e.g. connecting dots on the edges one should keep to the edge.
Most of the time it appears the best approach is a depth-first search, e.g. one tries very long paths to see if they work. Combined with rules of thumbs and inferences such as "don't leave gaps". Also "Don't cut off one dot from another dot of the same colour".
But there are ways to "not leave gaps" such as keep within one square of another line. That humans seem to be able to grasp but seems harder for an AI to learn.
In fact I wonder if the rule of thumb "keep close to other lines" might even require some kind of internal language.
I mean to even understand the rules of the game one would think one would need language. (Could an ape solve one of these puzzles? I doubt it.)
So basically I'm trying to solve how an AI could come up with these technqiues for solving puzzles like Flow Free. (Techniques that might not work in all cases).
Perhaps, humans have an innate understanding of concepts such as "keep close to the wall" and "don't double back on yourself" and can combine them in certain ways. Also we are able to spot simple regions quickly bounded by objects.
I think a built in understanding of "regions" would be key. And the key concept that dots can't be joined unless they are in the same region. And we have got to a dead-end if:

There is an empty region
There is a region with a dot without it's pair

Still I don't think this is enough.

I think that these "rules of the thumb" you speak of can come from training. They don't need to be hard coded or anything. All in all I wouldn't imagine agents having much difficulty in learning flow free. We have seen AI agents solving arguably harder puzzles (e.g. rubik's cube) and the environment seems fairly simple to model and simulate.

@Djib2011 Rubik's cube is a simple permutation problem. But I would say this flow game you need an visual understanding of things.

A simple permutation game with $43 \cdot 10^{18}$ states and with no indication if you're actually heading in the right direction. It should be an indication that only in the past couple of years have there been AIs capable of solving the Rubik's cube reliably without any human knowledge.

@Djib2011 Yes, that's true. I should take a look at how it does that! I wonder how... I don't even know how a human worked out how to solve it!

A few of us have spent quite a bit of time thinking about this. I summarised our work in a Medium article here: https://towardsdatascience.com/deep-learning-vs-puzzle-games-e996feb76162
Would love to hear what you think.
Spoiler: so far, good old SAT seems to beat fancy AI algorithms!

Thanks for showing me your article! Very interesting. Have you heard of Kaggle? This would make a nice Kaggle competition.

@zooby glad to hear you liked it! I have indeed heard of kaggle, but hadn't thought about making this a competition, great idea. Do you know if a reward on the lower side (a few hundred USD) would be enough of an incentive for the community?

Don't see why not. Some competitions don't have any prizes.

Welcome to SE:AI!  This answer has been flagged for being link-only.  Could I impose on you to give a short summary of the Medium article here?


text:
How are LSTM's trained for text generation?
I've seen some articles about text generation using LSTMs (or GRUs) for text generation.
Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained it with text which includes the string:
"The dog chased the cat"
and also includes the string:
"The lion chased the ca"
Both should be acceptable. But although they are very similar they differ entirely after the 4th character. So they will result in two very different vectors. Further the longer you roll it out the more they will differ.
How is it then possible for an LSTM to learn that "The [something] chased the cat" is an acceptable phrase?
Equally if you train it to try and learn to pair up parentheses. I can see how you could manually design it to do this, but how exactly could it be trained to do this just by entering strings like "(das asdasdas) axd"?
What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.
Any ideas?
(Also, I've only ever seen one article that showed a LSTM that can pair parentheses. So has this study ever been replicated?) I get that it is possible for an LSTM to do this but I don't get how it can learn to do this!
What I'm getting at is you usually train things with the input phrase and then compare it to the expected phrase to get the error. But in text generation there might be millions of possible acceptable expected phrases! So how can you compute an error?

You compute the error in the same way as normal - treat the actual output as "ground truth" even if other options are possible. The LSTM will output probabilities of next character, it will not learn to associate a single "true" output except for rare circumstances (such as completing a word that can only be done in one way according to the training data). In practice that means that understanding acceptable loss values in NLP training is different to understanding them in other supervised learning environments, typically being higher than for datasets where ground truth is more predictable from the data.
In this way, it is a lot like a Markov or n-gram model, except that it learns its own internal state representations (they are not prescribed by the model, but are limited by structure and data available). The gated RNN memory models like LSTM and GRU include a mechanism for handling long term memory so can relatively easily learn things like quote, paren pairs etc.

What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.

A Markov model can in theory represent any sequence knowledge that an LSTM can. The difference is in terms of efficiency. RNNs have more compact internal representations than explicit state representation in say HMMs. This makes them harder to work with in some ways (it is hard to see what a RNN state vector represents), but means that they can store more complex knowledge about state transitions in less memory. It also means that RNNs can generalise (although not guaranteed correctly) more readily, they will not fall back to random behaviour given a brand new state without any training history.

In practise does this mean you have to set the learning rate very low so it doesn't try to get too close to one particular answer for example?

@zooby I don't think it needs to be lower than normal, but deep NNs often have low values e.g. 0.001 or 0.0001 and use gradient adjustment (RMSProp seems to be a popular choice for LSTM) with mini-batches (which will smooth out some of the differences)


text:
How to determine the probability of an "existence" question
I am having a go at creating a program that does math like a human. By inventing statements, assigning probabilities to statements (to come back and think more deeply about later). But I'm stuck at the first hurdle.
If it is given the proposition
   ∃x∈ℕ: x==123

So, like a human it might test this proposition for a hundred or so numbers and then assign this proposition as "unlikely to be true". In other words it has concluded that all natural numbers are not equal to 123. Clearly ludicrous!
On the other hand this statement it decides is probably false which is good:
 ∃x∈ℕ: x+3 ≠ 3+x

Any ideas how to get round this hurdle? How does a human "know" for example that all natural numbers are different from the number 456. What makes these two cases different?
I don't want to give it too many axioms. I want it to find out things for itself.

A human has an abstract concept of numbers in mind. So 456 is a unique entity which is by definition unlike any other number because that are other unique entities. If you give ∃x ∈ ℕ: x==123 to your system it could check the property of natural numbers by counting from 0 to 123 to conclude that the statement is true. A human does it in another way. A human would "see" that it is a natural number because it has no decimal point. Because of the concept of natural numbers the statement is immediately clear. To get a faster result here the machine could check just the decimal point.
In your second case you have to apply the commutative property of addition and you are done because the syntax is equal then.
The second problem is more syntactic while the first is semantic. Your machine may "know" the commutative property of addition but not the concept of natural numbers. Therefore, it has to count.

You need some sort of interpretation abstraction before your mathematical reasoning. While the text might read "123", you need to parse this into a literal of type Natural Number or Integer. Similarly, "x" could be a member variable. Then your deduction becomes, is literal 123 a Natural Number? Yes.
As for the second statement, you should hopefully be able to reason to definitely false. Your internal representation of a sum should not be order dependent since addition is commutative. Then, the check for equality must handle this unordered property.


text:
How to create a task-graph based neural network?
I'm trying to design a neural network with a task hierarchy. This is my idea so far:
 [Desires]
    |
[Layer 1] [T0]
    |    /
[Layer 2] [T1]
    |    /
[Layer 3] [T2]
    |    /
[Layer 4] [T3]
    |    /
  [Action]

The way this would work is that each layer represents a task as a binary number. Layer 1 is the main task, layer 2 the sub-task etc. Each task consists of 2 sub-tasks determined by T={0,1}. In this way the neural network represents a binary task graph with T=0 being the left child and T=1 being the right child of a node.
You can think of it as T3 changing every second T2 changed every 2 seconds and so on. So {T0 T1 T2 T3} gives the binary time in seconds in a 16 second cycle.
So far this only makes the output a sequence of 16 actions in order. But if some of the layers could be "if" gates they might control the T-values and so act as switches and so have more complicated programs.
Do you have any suggestions to improve this? Or has this kind of binary task graph representation been done before in a neural network?
Also importantly how would you train such a neural network? (At the moment I just assume that the model is pre-trained and just trying to find a good architecture).


text:
Would this neural network have short term memory?
I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, the number of words it could store is $10^7$.
Here is my design:
$$out_{n+1} = f(out_n, in_n)\mathbf{N} + out_n.\mathbf{M}$$
$$action_n = \sigma(\mathbf{N} \cdot out_n)$$
Where $f$ represents some layered neural network. Some of the actions would be physical actions and some might be internal (such as thinking of the letter 'C').
Basically I want $out_n$ to be an array that keeps the last 6 action values and puts them back in. So $M$ will be the matrix:
$$\begin{bmatrix}
0&1&0&0&0&0\\
0&0&1&0&0&0\\
0&0&0&1&0&0\\
0&0&0&0&1&0\\
0&0&0&0&0&1\\
0&0&0&0&0&0
\end{bmatrix}$$
i.e. it would drop the 6th item from it's memory.
and $N$ would be the vector:
$$\begin{bmatrix}
1&0&0&0&0&0&0
\end{bmatrix}$$
I think this would be equivalent to an equation of the form:
$$out_{n+1}=F(in_n,out_n,out_{n-1},out_{n-2},...,out_{n-6})$$
So I think this would be an advantage over an RNN since this model remembers precisely it's last 6 actions. But would this be better than an RNN or worse? One could increase it's memory to more than 7 quite easily.
I think it's basically the same archececture as an RNN except elinimating a lot of the connections. Is this a new design or a common design?
One problem with this design is that you might also want a memory that is over longer time periods (e.g. for actions that take more than one tick.) But that might be solved by enhancing the archecture.

Congrats, you have invented 1d convolution. Convolution combined with RNN would have some advantage over just RNN. Think about the perception field.
In this layer, you do aggregate $6$ values to one. Imagine two of them - it will be $36$ already, etc. But, in the end, you still need RNN at the end to aggregate a variable length to constant length.

Well that's good! Glad I'm on the right track! (Not sure what you mean at the end about variable lengths).

@zooby This is not a 1D CNN, its a non differentiable RNN. (actions must be sampled under some categorical distribution based on whats described). The only similarity to a 1d cnn is the sliding window

Why is it non-differentiable ?

do you train with sequences of different lenght, right? also if you put output as input think about output may be wrong so you can consider to force-feeding ( expected data instead of output)

I could be wrong but generally actions are drawn from a distribution (that’s why you show one hot encodingns) and you can’t differentiate through a categorical distrib


text:
What is a good language for expressing replacement or template rules?
Say I have a game like tic-tac-toe or chess. Or some other visual logic based problem.
I could express the state of the game as a string. (or perhaps a 2D array)
I want to be able to express the possible moves of the game as rules which change the string. e.g. replacement rules.
I looked into regex as a possibility but this doesn't seem powerful enough. For example, one can't have named patterns which one can use again. (e.g. if I wanted to name a pattern called "numbers_except_for_8". To be used again.
And it also should be able to express things like "repeat if possible".
In other words I need some simple language to express rules of a game that has:

modularity
simpleness
can act on other rules (self referential)

There are languages like LISP but these on the other hand seem too complicated. (Perhaps there is no simple language hence why the English language is so complicated).
I did read once about a generalised board game solving software program which had a way to express the rules of a game. But I can't seem to find a reference to it anywhere.
As an example rules for tic tac toe might be:
Players-Turn:
"Find a blank square"->"Put an X in it"->Oponent's turn
Oponents-Turn:
"Find a blank square"->"Put an O in it"->Player's turn
So I think the ingredients for rules are: searching for patterns, determining if an object is of a particular type (which might be the same as the first ingredient), and replacing.

What about the prolog?  Express the rules and moves using a logical expression.

@OmG I think that may be the language that the Stanford Generalised Game Playing program uses. Might be a good idea. The syntax does get a bit complicated beyond simple rules though. But it does seem similar to what I need like this tic-tac-toe game: https://swish.swi-prolog.org/p/Tic-Tac-Toe.swinb

Actually just seen a variant called ProbLog which uses probabilities as well.


text:
Is there an algorithm for "contextual recognition" with probabilities?
Example 1
An object is composed of 3 sub-objects.

Object 1: 90% looks like an eye 10% looks like a wheel
Object 2: 50% looks like an eye 50% looks like a wheel
Object 3: 90% looks like a mouth 10% looks like a roof

OK. So now we want to determine what the whole-object is. Using this evidence maybe we find:

Combined Object: 90% looks like a face 10% looks like an upside-down car.

But now, given this, we go back an reclassify the sub-objects.

Given whole object is a car object 2 is 99% an eye.

I'm looking for an algorithm that sort of goes back and forth between the context and the subobjects to classify both an object and it's parts.
(This is related to the rabbit-duck illusion, where such an algorithm once decided something is a rabbit, it classifies it's parts as rabbit-parts).
In other words the algorithm needs to calculate conditional probabilities P(A|B) but the B depends on what all the A's are! So it's a feed back loop.
Example 2
There is a word "funny". The sub-letters are classified (wrongly) as F U M Y. It guesses that the word is FUNNY and then goes back and tries to reclassify the letters. Using this reclassification it is even more certain the word is FUNNY. And much more certain the letters in the middle are NN and not M.
(Perhaps later the word is used in a sentence "The fumes from this fire are really fumy". And then using this new evidence it has to go back and reclassify the word again and now it thinks the word is FUMY with 80% probability).
I have an idea that I would write all the condition probabilities in a matrix $M$ then gives starting probabilities $S_0$. Then iterate like this: $S_{n+1}=M S_n$ until hopefully it converges on something.


text:
What are some common heuristics that might be innate?
Here's a question I might ask an AI to solve:
"Colour the states of the USA using just 4 colours".

Now, a common heuristic a human might use is to start at one state and "work their way out". Or start at an edge state. Now this seems to work best rather than just colouring states in a random order like a computer might do. And it means a human is often better than a computer because a computer might just start colouring random states and get into trouble very quickly.
(Also I wonder if this is a learned heuristic or would a child develop this on his/her own?)
Now the question is, whether this heuristic is an innate optimisation strategy, or just laziness on the part of the human. i.e. colouring things close together takes less effort. Either way it leads to a good strategy.
But I wonder if there are any other examples of heuristics that humans inately use without realising it, that lead to good strategies?
One heuristic that computers often don't know is
"If you're trying to play a game don't keep turn around and go the other way for no reason."

Again, a human would not do this, but again this could be laziness on the part of a human. It takes more effort to turn around than keep going in one direction to explore it.

Evolution has provided countless heuristics to humans for addressing problems.  Following the path of least resistance. i.e.  laziness, is just one of them that sometimes helps and sometimes hurts.

@George That's a good point. But it means we may have to simulate laziness in a computer even though for a computer, even for a computer it takes no physical effort to "move a pen" from one side of the screen to another.

I was not trying to emphasize laziness at all but to indicate that laziness is but one of the uncountable "heuristics" humans operate under.

Perhaps Occam's razor counts.
Occam's razor is the meta-heuristic that "the simplest explanation is the most likely to be correct". I consider it a meta-heuristic because itself doesn't provide explanations, only means of comparing them among another. It is a heuristic because like all heuristics, it can lead to false conclusions, but is usually right.


text:
Is there a mathematical formula that describes the learning curve in neural networks?
In training a neural network, you often see the curve showing how fast the neural network is getting better. It usually grows very fast then slows down to almost horizontal.
Is there a mathematical formula that matches these curves?

Some similar curves are:
$$y=1-e^{-x}$$
$$y=\frac{x}{1+x}$$
$$y=\tanh(x)$$
$$y=1+x-\sqrt{1+x^2}$$
Is there a theoretical reason for this shape?

As you mention, "It usually grows very fast then slows down", so, of course, there isn't a single mathematical formula that would describe all learning curves. So, I suggest you clarify your question, which seems to be quite interesting.

@nbro OK. By usually I mean always. And of course it slows down because the neural network can't keep increasing intelligence forever. But these curves do almost always look the same and do usually keep growing for a long time but very slowly. I'm just wondering if one can theortically model this curve and produce a formula for it.

Someone changed it to "how fast the nerual network is converging". This is not what the curves show. The curves might show, for example, how the neural network improves in its game of chess.

I had changed that sentence because you had written something like "network is gaining intelligence", which is a little bit arguable. Maybe "convergence" isn't the most appropriate word and I didn't think much about it. Anyway, learning curves look a bit like logarithmic curves.

I don't think there is a universal mathematical formula that describes the learning curve, but what I know is that not all curves look like your diagram. In Q-learning there is almost no overfitting so the curve continue to rise but in other task there maybe overfitting so it may rise then fall slowly again. Maybe one could fit a function to for the curve?

I don't think there's a theoretical reason for the shape, it's just showing the diminishing returns as training goes.
As for a mathematical function that describes this, exponential decay may fit well here (in case of loss, or inverse exponential decay if accuracy).
Also, I would not assume that this abrupt flat line of your examples are a universal.