text:
Steps to train and re-train a good model
I'm still a bit new to deep learning. What I'm still struggling, is what is the best practice in re-training a good model over time?
I've trained a deep model for my binary classification problem (fire vs non-fire) in Keras. I have 4K fire images and 8K non-fire images (they are video frames). I train with 0.2/0.8 validation/training split. Now I test it on some videos, and I found some false positives. I add those to my negative (non-fire) set, load the best previous model, and retrain for 100 epochs. Among those 100 models, I take the one with lowest val_loss value. But when I test it on the same video, while those false positives are gone, new ones are introduced! This never ends, and Idk if I'm missing something or am doing something wrong.
How should I know which of the resulting models is the best? What is the best practice in training/re-training a good model? How should I evaluate my models?
Here is my simple model architecture if it helps:
def create_model():
  model = Sequential()
  model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', input_shape=(300, 300, 3)))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(BatchNormalization())
  model.add(Dropout(0.2))
  model.add(Flatten())
  model.add(Dense(256, activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(64, activation='relu'))
  model.add(Dense(2, activation = 'softmax'))

  return model

#....
if retrain_from_prior_model == False:
    model = create_model()
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
else:
    model = load_model("checkpoints/model.h5")

better to re-train your model with a smaller lr than the last one. Also, if you keep re-training your model this way several times, your model gets accustomed to that specific images which have been added recently. I think it is better to collect all images and re-train once or twice. If you happen to get images afterwards, you collect them as one again and re-train with the best model of the base dataset.

authors: Δημήτρης Παναγιώτου


text:
Specifying resolution for objects with known dimensions using CNN
I would like to ask you for advice. I deal with beekeeping but I am also a bit a programmer and an electronics specialist. And this is where my 3 interests come together, actually 4 because recently deep learning has joined this group.
I would like to analyze (I already run some tests) bee behavior using deep learning mechanisms based on images (photos or videos) with bees. These images, of course, can be very different, this applies primarily to the area that can be shown. It can literally be 2x3cm like a macro picture or an area of 40x30cm and 3000 bees on it. Trying to create a network analyzing such different areas is a nightmare. And because I am interested in aspects of bees separately, it seems logical to divide the image into these parts, which will contain single bees in their entirety, reduce their resolution to a minimum ensuring recognition of the necessary details and only then analyze each part of the large image separately.
This approach seems even more right to me when I started to read the details, e.g. YOLOv3. Thanks to this, I would avoid a large amount of additional calculations.
And here comes the first stage to be implemented. I need to specify the image resolution, more precisely the number of pixels that on average falls on one bee length. If it is too small, I do nothing but say that the resolution is too small for analysis. If it is suitable (at least 50 pixels for the length of the bee), I algorithmically cut the input image into smaller ones so that they contain individual insects and I subject them to further analysis.
Fortunately, the sizes of bees and bee cells are precisely known to within 0.1mm. And there may be hundreds of such objects (a bee or a hexagonal bee cell) and I can decide how many of them will be at the stage of preparing training data. I have thousands of photos and videos from which I can easily create training data for the network in the form of pairs of objects image bee size expressed in pixels from this image.
Time for question:
At the output I want to get a number saying that one bee in the image is X pixels long, e.g. 43.02 or 212.11
Has anyone of you dealt with a similar case of determining resolution for known objects using a neural network (probably CNN with an output element with rel function) and can share your experience, e.g. a network structure that would be suitable for this purpose?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Using word embedding to extend words for searching POI names
I am developing my own mobile app related to digital map. One of the functions is searching POIs (points of interest) in the map according to relevance between user query and POI name.
Besides the POIs whose names contain exact words in the query, the app also needs to return those whose names are semantically related. For example, searching 'flower' should return POI names that contain 'flower' as well as those that contain 'florist'. Likewise, searching 'animal' should return 'animal' as well as 'veterinary'.
That said, I need to extend words in the query semantically. For example, 'flower' has to be extended to ['flower', 'florist']. I have tried to use word embeddings: using the words corresponding to most similar vectors as extensions. Due to the fact I don't have user review data right now and most of the POI names are very short, I used trained word2vec model published by Google. But the results turn out to be not what I expect: most similar words of 'flower' given by word2vec are words like 'roses'and 'orchid', and 'florist' is not even in the top 100 most similar list. Likewise, 'animal' gives 'dog', 'pets', 'cats' etc. Not very useful for my use case.
I think simply using word embedding similarity may not be enough. I may need to build some advanced model based on word embedding. Do you have any suggestions?

I think word embeddings are overkill in this particular case.
My suggestion would be to go for a simple dictionary based approach: compose sets of semantically related words, and then use those to expand your query terms. This might take a bit longer to set up, but has several advantages:

simplicity: you can't make many mistakes with this
transparency: you know exactly why a certain term matches, and another one doesn't
accuracy: you have tight control over the whole process; if some term is wrong, you remove it from the set. You cannot do that with embeddings
resources: a dictionary-based approach is far simpler and needs less storage

'Old tech' doesn't sound as sexy as the latest deep learning stuff, but unless you want to have this as a toy project to learn about how to do things with embeddings I would say the latter are the wrong tool for the job. At least you can be sure that it works, and if it doesn't you can easily fix it.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can I provide a CNN with hints?
Let's say I want to classify a dataset of handwritten digits (CNNs on their own can get 99.7% on the MNIST dataset but let's pretend they can only get 90% for the sake of this question).
Now, I already have some classical computer vision techniques which might be able to give me a clue. For instance, I can count the intersection points of the pen stroke

1,2,3,5,7 will usually have no intersection points
6,9 will usually have one intersection point each
4,8 will usually have two intersection points each (usually a 4-way crossover yields two intersection points which are close together)

So if I generate some meta-data telling me how many intersection points each sample has, how can I feed that into the CNN training so that it can take advantage of that knowledge?
My best guess is to slot it into the last fully connected layer just before classification.

"Distilling the Knowledge in a Neural Network" by G. Hinton et al.
https://arxiv.org/abs/1503.02531

You could add the meta-data along all the pixels to your feature vector and let the model learn the correlation between number of intersections and digits.

@Feodoran thanks. Is that different from what I said about adding it into the last fully connected layer? Is "feature vector" different than that? Just makings sure I've got the terms right.

It is different in that I suggest a single, combined feature vector feeding into the first layer, while you are suggesting two feature vectors feeding into two different layers of the model. You have to try which one works better.


text:
time-series prediction : loss going down, then stagnates with very high variance
I am trying to design a model based on LSTM cells to do time-series prediction. The ouput value is an integer in [0,13]. I have noticed that one-hot encoding it and using cross-entropy loss gives better results than MSE loss.
Here is my problem : no matter how deep I make the network or how many fully connected layers I add I always obtain pretty much the same behavior. Changing the optimizer also doesn't really help.

The loss function quickly decreases then stagnates with a very high variance and never goes down again.
The prediction seems to be offset around the value 9, I really do not understand why since I have one-hot encoded the input and the output.

Here is an example of a the results of a typical training phase, with the total loss :

Do you have any tips/ideas as to how I could improve this or could have gone wrong ? I am a bit of a beginner in ML si I might have missed something. I can also include the code (in PyTorch) if necessary.

What is the size of your data? And the number of trainable parameters in your network?

The dataset is quite small (about 200 time-series of 50 points each). The number of trainable parameter depends, as said, I have tried various number of layers and hidden sizes but it didn't change that much.

I found the issue, I should have done more unit testing. Upon computing the batch loss before backpropagation, one of the dimension of the "prediction" tensor was not corresonding to the "truth" tensor. The shape match but the content is not the one that was supposed to be.
This is due to how the NLL loss is implemented in pytorch which I was not aware of ...


text:
Can a NN be configured to indicate which points of the input influenced its prediction and how?
Suppose I want to classify a dataset like the MNIST handwritten dataset, but it has added distractions. For example, here we have a 6 but with extra strokes around it that don't add value.

I suppose a good model would predict a 6, but maybe with less than 100% certainty (or maybe with 100% certainty - I don't know that it matters for the purpose of this question).
Is there any way to get information about which pixels most strongly influenced the decision of the CNN, and which pixels were not so important? So to represent that visually, green means that those pixels were important:

Or conversely, is it possible to highlight pixels which did not contribute to the outcome (or which cast doubt on the outcome thereby reducing the certainty from 100%)

Yes there definitely is, and research into this has actually resulted in some really cool behaviour.
One of the simplest ways is to simply back propagate the gradient all the way back to the input. Areas of the input that affected the final decision will receive larger gradients. Interestingly, this also sort of works as a rudimentary form of semantic segmentation.
Other ways are to change segments of the input and to see how that affects the output (like add 0.1 to a pixel).
You can also determine what each filter in a convolutional layer is looking at using similar techniques.
It's a super interesting field of machine learning, and I highly recommend taking a look at this lecture that is completely free and one of the most interesting ones I've personally seen. it will explain all this much better than I have done.

There are many frameworks which allow you to do that.
One of them, which supports many different techniques for visualization, can be found here:
https://github.com/marcoancona/DeepExplain


text:
Crossing of training and validation loss
During training of my models I often encounter the following situation about training (green) and validation (gray) loss:

Initially, the validation loss is significantly lower than the training loss. How is this possible? Does this tell me anything important about my data or model?
One explanation might be, that training and validation data are not properly split, i.e. the validation might primarily contain data, that the model can easily represent. But then why do the curves cross after epoch 30? If this is because of overfitting, then I would expect the validation loss to increase, but so far both losses are (slowly) decreasing.
There is a related question at Data Science SE, but it doesn't give a clear answer.

authors: Божидар Гемеджийски


text:
Simple sequential model with LSTM which doesn't converge
I'm actually trying to create a sequential neural network in order to translate a "human" sentence in a "machine" sentence understandable by an algorithm. Like It didn't work, I've try to create a NN that understands whether the input is a unit or not.
Even this NN doesn't work and I don't understand Why. I tried with different optimize/loss/metrics/with Rnn/with LSTM.
So there is one array of unit and one array with lambda words.
I send to the NN :
input -> word in OneHotEncoding where each char is a vector
output -> a vector with a 1 at the relative position of the unit in the array, ex : [0,0,0,1,0]. If It's not a unit, the vector is composed of 0.
I'm actually using LSTM layers and sigmoid activation because I will need it for my "big" NN.
model = tf.keras.Sequential()
model.add(tf.keras.Input(shape=(13,vocab_size),batch_size=80))
model.add(tf.keras.layers.LSTM(32, return_sequences=True))
model.add(tf.keras.layers.LSTM(6, return_sequences=False))
model.add(tf.keras.layers.Dense(6, activation="sigmoid"))
model.reset_states()
model.summary()
model.compile(optimizer= 'adam', loss='categorical_crossentropy', metrics=['categorical_accuracy'])
model.fit(encoded_word, my_targets, batch_size=80, epochs=100, validation_data=(encoded_word, my_targets))

Model: "sequential_38"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
lstm_57 (LSTM)               (80, 13, 32)              6400
_________________________________________________________________
lstm_58 (LSTM)               (80, 6)                   936
_________________________________________________________________
dense_32 (Dense)             (80, 6)                   42
=================================================================
Total params: 7,378
Trainable params: 7,378
Non-trainable params: 0

Epoch 1/100
10000/10000 [==============================] - 6s 619us/sample - loss: 0.7441 - categorical_accuracy: 0.2898 - val_loss: 0.6181 - val_categorical_accuracy: 0.3388
Epoch 2/100
10000/10000 [==============================] - 2s 233us/sample - loss: 0.5768 - categorical_accuracy: 0.3388 - val_loss: 0.5382 - val_categorical_accuracy: 0.3388
Epoch 3/100
10000/10000 [==============================] - 2s 229us/sample - loss: 0.5039 - categorical_accuracy: 0.3979 - val_loss: 0.4640 - val_categorical_accuracy: 0.5084
Epoch 4/100
10000/10000 [==============================] - 2s 229us/sample - loss: 0.4207 - categorical_accuracy: 0.4759 - val_loss: 0.3709 - val_categorical_accuracy: 0.5041

My NN is converging towards 0.5 all the time.
Thank you in advance for your answers !


text:
Using tensor networks as machine learning models
Tensor networks (check this paper for a review) are a numerical method originally introduced in condensed matter physics to model complex quantum systems. Roughly speaking, such systems are described by a very high-dimensional tensor (where the indices take a number of values scaling exponentially with the number of system constituents) and tensor networks provide an efficient representation of the latter as an outer product and contraction of many low-dimensional tensors.
More recently, a specific kind of tensor network (called Matrix Product State in physics) found interesting applications in machine-learning through the so-called Tensor-Train decomposition (I do not know of a precise canonical reference in this context, so I will abstain from citing anything).
Now, over the last few years, several works from the physics community seemed to push for a generalized use of tensor networks in machine learning (see this paper, a second one and a third one and this article from Google AI for context). As a physicist, I am glad to learn that tools initially devised for physics may find interdisciplinary applications. However, at the same time, my critical mind tells me that from the machine learning research community's perspective, these results may not look that intriguing. After all, machine learning is now a very established field and it takes probably more than a suggestion for a new machine learning model and a basic benchmarking on a trivial dataset (as the MNIST one) -which is what the papers essentially do in my humble opinion- to attract any attention in the area. Besides, as I believe to know, there already exists quite a solid body of knowledge on tensor analysis techniques for machine learning (e.g.  tensor decompositions), which may cast doubt on the originality of the contribution.
I would therefore be very curious to have the opinion of machine learning experts on this line of research: is it really an interesting direction to look into, or is it just about surfing on the current machine learning hype with a not-so-serious proposal?

I do not feel comfortable proclaiming that I would be a machine learning expert.
But I want to point out that there is indeed interest in applying tensor networks in machine learning settings.
Let me highlight three particular settings in the following.
They can be used to find the governing equations behind dynamical systems analogous to the SINDy algorithm.
The reasoning is that the dynamics of your system may not be sparse but may indeed have a low-rank structure. (See this paper for a comparison of sparse and low-rank structures and this paper for a discussion about why low-rank matrices so often appear in big data.)
The resulting modification of SINDy is the appropriately named MANDy algorithm.
But if your dynamical law exhibits additional structure (despite being low-rank) then this structure is not used. (Requiring you to collect unnecessary samples.)
This work tackles this issue by allowing you to specify the structure that you expect your law to exhibit.
Another area of interest is the application of tensor networks as a means of compression.
Tensor networks can be used to approximate multivariate functions.
Either directly (see this article or this article for further details) or by representing their high-dimensional coefficient tensor (see e.g. here).
The last paper also highlights where such functions may occur: when solving stochastic or parametric PDEs.
"What does this have to do with machine learning?" you may rightfully ask.
These PDEs are hard to solve and it is often easier to minimize a residual than to apply a Galerkin method. The results can be shown to be equivalent with high probability. (See this paper for a proof of this statement and an application to some stochastic PDEs or this paper for an application to an optimal control problem.)
Moreover, there is the problem of tensor completion - a generalization of matrix completion to higher dimensions. (This and this are references for algorithms.)
These methods find application in data science where they can be used to decompose or denoise data. (Unfortunately, I don't have a reference with examples but I remember a presentation where these methods where used on EEG signals.)
Additionally, I recently found this interesting symposium covering even more potential applications of tensor networks in machine learning:
https://itsatcuny.org/calendar/quantum-inspired-machine-learning


text:
How is the receptive field of a CNN affected by transposed convolution?
When computing receptive field recursively through a CNN, does a transposed convolution affect the receptive field the same way that a convolution does if the kernel and stride is the same?


text:
When stacking LSTM's, should the hidden units increase?
I'm using Weights and Biases to do some hyperparameter sweeping for a supervised sequence-to-sequence problem I'm working on. One thing I noticed is that the sweeps with a gradually increasing number of hidden units tend to have a lower validation loss:

I'm wondering if this is generally true or just a function of my particular problem?


text:
Language Learning feedback with AI
Is there a program under development that uses AI technology, like Siri, to "hold hands" so to speak with a language learner and coach them on accent, colloqiual expressions, or to let them guide the language learning process using an archive of language knowledge?
Also, could this sort of program be used to learn things in a language one already knows, or in a new language, say for the purposes of travel or to learn about related hyperlinks in an online database?


text:
OpenAI spinning up convolutional networks with PPO
I am using pytorch version of PPO and I have image input that I need to process with convolutional neural networks, are there any examples on how to set up the network? I know that stable baselines  support this to some extend, but I had better performance with spinning up so I would prefer to keep using these.

I have written a tutorial on using OpenAI Spinning Up in a image-based PyBullet + Gym environment here
In order to be able to use spinup for an image-based environment I had to fork it here and add CNN to PPO's core.py


text:
How does adding noise to the action in DDPG help in learning?
I can't understand how playing with the action generated by the actor network in DDPG by adding the noise term helps in exploration.

Do you understand how does epsilon-greedy help exploration in Q-learning for example? Because the idea is basically the same.

Yes, in Q-learning you have discrete actions and we choose actions randomly during exploration and then we choose the action that gives the highest value. Let me explain my confusion by this example. Lets say my actor network generated a force of 1 N, but I need 10 N for the same state so I added big noise to make the action almost 10 N. Now the 10 N and the state go into the critic network and it will give a good value and the actor will know that 1 N is enough, but it will not know that I need 10 not 1.

That's not true because you're not updating actor through action that was modified by noise, you're updating it through action that it wanted to initially take. So in your case the update would be through action 1N and not 10N. Only critic is updated through noise modified action. The reason why this helps exploration is because the critic will eventually get more accurate in representing state-action values so it will "criticize" the actor better through updates

Got it ! I will try it out and revert back to you. Thank you

Hi, I hope you are doing great. I tried to update the critic through the noisy actions and the actor through the action its self. However, the actor is not learning, the only thing that changed is that the actions are shifted like all actions become shifted by 2N then 5N and so on. Please not that I am applying the noise for some episodes.

There are many things that can go wrong, I would advise you to study pseudocode  and read DDPG paper. After that you can study this github repo, it has beginner friendly implementations of some RL algorithms, including DDPG, in PyTorch and TensorFlow 1.

Appreciate your help and thank you for the continous support


text:
Drone Deployment Platform for Neural Networks
Good day everyone,
I would just like to ask if anyone part of a lab or company doing research on aerial robotics has any suggestions of a good platform for deploying computer vision algorithms for aerial robots?
Currently our lab has a set of DJI Matrice drones but are too heavy for our liking. We really wanted to use the Skydio R2 drone for out future research projects but found out later that the SDK does not allow access for implementing our own Deep Learning or Reinforcement Learning networks (only allows the user to code their own preset movements in python). We also took a look at the Parrot AR drones but found that they were discontinued and do not have the computing power that the Skydio has although as an alternative, it does have the capability to stream the video feed to an external computer for online processing instead of on-board.
I suggested that we stick with the DJI Matrice and just use an Nvidia Jetson for deployment but I am still curious to know if anyone knows of other available platforms. Does anyone know of a more compact platform available for research purposes?
Thank you :)


text:
How to choose the activation function in neuroevolution?
I am developing a NEAT flappy bird game, and it doesn't work, the system stays stupid for 300 generations. I chose tanh() for activation, just because it's included in JS.
I can't find a good discussion on the internet of activation functions in the context of neuroevolution, most of what I see is about derivative and other gradient descent issues which I suspect are irrelevant to forward only networks.
If you need a fixed point to answer, I have 8 inputs, one output and the problem is a classification ("jump", "don't jump"). But please explain your answer. I currently use tanh() for all the hidden and output nodes, and the output is considered "jump" if the output neuron value is >0.85
For some context, the code is here: https://github.com/nraynaud/nraygame and the game here: https://nraynaud.github.io/nraygame/

Why do you use only 1 output neuron? Why not use 2 output neurons? 1 for jump and 1 for not jump?

it feels more binary to have number and a threshold, and simpler to have less nodes, but if you have a good explanation as to why 2 is better than one, I just want to learn.

I don't have many experience in NEAT, but I have made some reinforcement learning NNs. When you are training your NN using the MDP logic,Your output is the value of each action. If you have 2 output neurons this would be the value of jumping and the value of not jumping. when comparing these 2 values, you can then take the action with the biggest number. When you only have one output neuron, you can't train these values.However, when you use supervised learning instead of reinforcement learning, you can use your method (because the system learns from your predefined data, not from experience).

I just checked, NEAT works with reinforcement learning methods. So using 2 output neurons could make it learn better.

thanks, that could be a teaching moment, I'm googling this vocabulary.

What is the difference between 2 output neurons whose value is compared in the code and having one output neuron and 2 hiddens neurons right before it, whose output would be combined with a negative sign in the NN? Shouldn't the system discover this structure?

Reinforcement learning works by training the network with the values of the actions. If you have 1 output neuron you will train that output neuron to be higher if you got far and lower if you died immediatly. In this case, it will be trained randomly. If you have 2 output neurons, you will do a run, select a action, and train 1 of those output neurons (which will be the action you selected). This way, the other output neuron will be unaffected. In your way, the network won't learn the 2 preceding hidden neurons the way you want to, because it is trained on the reward on the next output neuron.

@nraynaud Did you just fixed your problem? Because I tested your game, and the birds are really good at the after only 30 generations. If you found the solution to your problem, you can answer your own question to help others who have the same question.

@DaraKong Sadly no, I never could make it understand the link between pipe2 and pipeX. It's still always looking at the leftmost pipe even when it's already behind the bird. When the game becomes hard, the birds stay stupid.

@nraynaud I don't know why, but when every time I go to your game's url you gave us in your question, the agents are performing very well after only 25 generations. For example : generation 26 survived to time 50000, and the graph was showing that your birds were actually learning fast. I also tried to download your code, and test it locally. Everything was looking fine, and the agents were as good as in the online version. I can't figure out why you are experiencing problems, the results are supposed to be the same right ? What are your results precisely ? What is the best time you got ?

thank you very much for your interest. Over time the game gets harder and harder, the pipes gets closer, and there is a point where the system will have to look at the second pipe before the first one disappears on the left if it doesn't want to die. I am trying to make it evolve a behavior smarter than just "match the height of the leftmost pipe".

When it comes to genetic algorithms (neuroevolution), you can pretty much evolve any kind of parameters. You just need to have a fairly complex system where each parameter changes the way the inputs are affecting the outputs.
So, to answer your question, any activation function should work! It would be surprising that the activation function is your problem. But if you want to test some activation functions, here's a list of activation functions. But tanh should work perfectly fine.
As I already mentioned in another answer, the key to have a great neuroevolution algorithm is the fitness function. Your rewards should reflect the goal you're aiming for. You didn't mentioned what fitness function you used, but here's what I suggest: reward a player ONLY if he progresses in the game. For example, your fitness function could simply be the distance traveled by your birds. This could maybe solve your problem...


text:
Training an unsupervised convolutional neural network to learn a general representation of a Lua module
I am trying to train a CNN in keras to learn a general representation of a Lua module, e.g. requires at the beginning, local variables, local functions, interface (returns) and in between some runnable code (labeled "other"). So for each module (source code) I generate an AST which I then encode in a json file. The file contains the order of the node in the AST, the text it represents and the type of node it is (require, variable, function, interface, other). It can contain other metrics but so far I have settled on these three, where only the order and type of node will be converted into a vector to serve as input to the CNN. Now I don't have any labels for these modules (I want to treat one module as one input to the CNN), so I have concluded that I need to use unsupervised learning. In keras this should translate to using autoencoders, where I use the encoder part to learn weights for the representation and then connect a fully-connected layer and generate an output. Now before I specify the output, I want to specify the input more closely. In my mind It should be a 3D vector let's say (x,y,z). x represents the number of nodes of an AST that are taken into consideration, y represents the local neighborhood of said node (for now I have settled on 5 nodes) and z should represent the node itself, the order and type of node. So with that, I would want the output of the network to be in the (almost) same dimension. I want x outputs for every node that was taken as input and a number (ideally between 0-1) to specify how "correct" the node under consideration is in response to the learned representation. My question as a beginner to neural networks is, how feasible is this and are there any points which are simply impossible to do or are wrongly interpreted on my part?

What are you trying to achieve? I don't quite understand that. When the NN is trained, what will the input and desired output be?

The input will be a "module" - a file with source code. From this module an AST is created, and nodes of AST are selected as input. For each node a local neighborhood of nodes will be selected (of size 5 nodes). The information a node contains is its position in the AST (1, 2, ..., x) and the node type (require, function, ...). So the input vector for one module will be (x,y,z), with x the number of nodes, y the size of local neighborhood and z the node information. So a 3D vector. The output should be a 1D vector of size X, that say how well placed a node in input module is.


text:
Recurrent neural Network for survival analyses: Dealing with forecast data as feature which can exceed the number of days untill a event occurs
I am building a Recurrent Neural network (LSTM) for predicting the number of days until a Pollen season starts (when the cumulative of the year exceeds X). One of the features I am including in my model is the weather forecast.
However, I do not feel confident about the way I defined the model while including this weather forecast; currently, the weather forecast of 7 days is included as one of the predictors, However, when the label (number of days until the season starts) is smaller than the forecast I am training te model on forecast data which is completely irrelevant in determining the start of the season (e.g. if the season starts in 2 days and I am including the forecast of 7 days as predictor I am also training the model on the 5 days after the season already started while these are completely irrelevant).
My feeling is that it is not right when training RNN's for survival analyses. Does anyone know a way to deal with this? Or have an example where someone dealt with a similar issue?
Thanks a lot!


text:
CBIR and object detection
How does CBIR (content based image recognition) fit into the problem of object detection? Let's say we want to detect 4 types of dogs (Golden Retriever, Cocker Spaniel, Greyhound, and Labrador). We have an "average" model trained using YOLOv3.   So it might, for example, have a lot of false positives and false negatives.
How could we use CBIR to improve the detections from this "average' YOLOv3 model?


text:
Is it possible to derive meaning from text by providing multiple ways of saying the same thing to a neural network?
Let's say I feed a neural network with multiple string sentences that mean roughly the same thing but are formulated differently. Will the neural network be able to derive patterns of meaning in the same way that it is being done with images. Is this an approach currently used in Natural Language Processing?
With images of dogs the neural network will get the underlying patterns that define a dog. Could it be the same thing with sentences?

No, it will not derive patterns of meaning, as the network has no understanding of language. What will happen, is that it picks up surface features (usually letter sequences) which are common between sentences with the same (or a similar) meaning.
This approach is often used in chatbots for intent recognition. Sometimes it picks up subtle patterns that humans would not notice, but they are also often not reliable: you get wrong classifications without knowing why.
Having said that, it works fine if you want to distinguish between a limited number of different intents (you don't even need a lot of training examples — 4 to 5 examples are often sufficient if they are well-selected). In this fairly limited scenario yes, otherwise no.


text:
Can two planning PDDL actions be taken simultaneously?
We are discussing planning algorithms currently, and the question is to describe the steps to check if actions could be taken simultaneously. This is a really open-ended question so I'm not sure where to start.

First place to look is how the preconditions/effects of different actions interact.

We expect answers to provide more details. So, I would suggest that you review this answer to make it more complete.

I don't see any principal problem with that. The way I would approach it is to have a resource model and durations attached to actions.
For example, movement would put a lock on your legs. You can't have another movement at the same time, as your legs are already busy. But your attention might only be partially occupied, so you can make a phone call while you're moving. You won't be able to read a book, because your eyes might be partially busy monitoring the walking action. This can be encoded in pre- and post-conditions.
What will probably be easier is to parallelise the execution of the plan. Once the plan has been created, organise the actions in a Gantt-chart like structure. You can have mutual exclusion in there, so all 'movement' would be restricted to one single row, so no more than one movement can take place at the same time. But 'making a phone call' could be in a separate row, and thus execute in parallel. The details depend on the requirements of the actions.
I can't easily think of a way that it would impact the planning process itself; unless there are critical timings involved. So leaving the planner to do its thing might keep it simpler, and then the optimal execution could be a post-processing step.


text:
Monte Carlo updates on policy gradient with no terminal state
Consider some MDP with no terminal state. We can apply bootstrapping methods (like TD(0)) to learn in these cases no problem, but in policy gradient algorithms that have only a simple monte carlo update, it requires us to supply a complete trajectory (which is impossible with no terminal state).
Naturally, one might let the MDP run for 1000 periods, and then terminate as an approximation. If we feed these trajectories into a monte carlo update, I imagine that samples for time period t=1,2,...,100 would give very good estimates for the value function due to the discount factor. However, the time periods 997, 998, 999, 1000, we'd have an expected value for those trajectories far different than V(s) due to their proximity to the cutoff of 1000.
The question is this:

Should we even include these later-occurring data points when we update our function approximation?

OR

Is the assumption that these points become really sparse in our updates, so they won't have much effect in our training?

OR

Is it usually implied that the final data reward in the trajectory is bootstrapped in these cases (i.e., we have some TD(0)-like behavior in this case)?

OR

Are monte carlo updates for policy gradient algorithms even appropriate for non-terminating MDPs due to this issue?

It'd be better to only have one question; you can have follow-ups in separate posts.

@OliverMason It isn't really four different questions; it's 4 hypothetical explanations that the OP has come up with to explain his own confusion, and he's asking which of them are on point. So I vote to leave it open

Naturally, one might let the MDP run for 1000 periods, and then terminate as an approximation. If we feed these trajectories into a monte carlo update, I imagine that samples for time period t=1,2,...,100 would give very good estimates for the value function due to the discount factor. However, the time periods 997, 998, 999, 1000, we'd have an expected value for those trajectories far different than V(s) due to their proximity to the cutoff of 1000.

I think your intuition is... partially right here, but not entirely precise. Recall that a value function $V(S)$ is generally defined as something like (omitting some unimportant details like specifying the policy):
$$V(S_i) = \mathbb{E} \left[ \sum_{t=0}^{\infty} \gamma^t R_{i+t} \right].$$
The "samples for time period $t = 1, 2, \dots, 100$ that you mention are not estimates for this full value function. They're estimates for the corresponding individual terms $R_{i+t}$. Indeed, in general, your intuition is right that the closer they are to the "starting point" $i$, the more likely they'll be to be accurate estimators. This is because larger $t$ are typically associated with larger numbers of stochastic state-transitions and stochastic decision-making, and therefore often exhibit higher variance.

Should we even include these later-occurring data points when we update our function approximation?

Theoretically, you absolutely should. Suppose you have an environment where almost every reward is equal to $0$, and only after 1000 steps do you actually observe a non-zero reward. If you don't include this, you'll learn nothing! In practice, it can often be a good idea to give them less importance though. This already happens automatically by picking a discount factor $\gamma < 1$.

Is it usually implied that the final data reward in the trajectory is bootstrapped in these cases (i.e., we have some TD(0)-like behavior in this case)?

OR

Are monte carlo updates for policy gradient algorithms even appropriate for non-terminating MDPs due to this issue?

It would be possible to do some form of bootstrapping at the end yeah, cut off and then have a trained value function predicting what the remainder of the rewards would be. TD($\lambda$) with $\lambda$ close to $1$ would be much closer in behaviour to true MC updates than TD($0$) though. Either way, it would be technically incorrect to still call it Monte-Carlo then, it would no longer be pure Monte-Carlo. So yes, strict Monte-Carlo updates in the purest sense of the term are not really applicable to infinite episodes.


text:
How do we define intention if there is no free will?
There is an idea that intentionality may be a requirement of true intelligence, here defined as human intelligence.
But all I know for certain is that we have the appearance of free will.  Under the assumption that the universe is purely deterministic, what do we mean by intention?
(This seems an important question given that intention is not just a philosophical matter in relation to definitions of AI, but involves ethics in the sense of application of AI, "offloading responsibility to agents that cannot be meaningfully punished" as an example.  Also touches on goals, implied by intention, whether awareness is a requirement, and what constitutes awareness. I'm interested in all angles, but was inspired by the question "does true art require intention, and, if so, is that the sole domain of humans?")

Under the assumption the universe is deterministic, I would say you can't argue humans are intelligent either. Without free will, how are we any different than a complex algorithm that has many responses to different inputs?

On the contrary, the difference between a deterministic universe and a random one wouldn't be noticeable on the macro scale, so that brings up the interesting argument of why should something that doesn't have an affect on the macro scale all of a sudden mean some things (on the macro scale) are no longer possible (in this case true intelligence)? None the less, I believe AI is capable of exhibiting the same level of intent that humans do, whether that be "fake" (no free will) intent or not.

See Free Will and Incompatibilist (Nondeterministic) Theories of Free Will (two Stanford Encyclopedia of Philosophy articles).

Free will is religious concept, not scientific.

can you source the claim that " intentionality may be a requirement of true intelligence"? this seems to be an artifact of simon-newell era goal-directed intelligence definition, but surely aesthetic appreciation counts as a mode of intelligence? what about emotional intelligence?

@k.c.sayz'k.csayz' I'll see if I can dig up some formal references.  (The entire sentence is informal, and I break my own rule of never using the term "true intelligence", which is, imho, counter-productive because it's fuzzy and not a formal statement of what constitutes intelligence, for instance, as compared to Legg and Hutter's formal, mathematical definition.)

@k.c.sayz'k.csayz this was also a philosophical question where I'm posing an assertion, so a valid answer could absolutely include a refutation "intentionality is not a requirement of intelligence".  I'll ponder on it though, b/c that part is really just an opener to the actual question.

i hesitate to put this as an answer, but you can perhaps argue that intentionality of an external agent can only be "attributed", and not "decided", much like a cute face on a plushie doesn't really have any "cuteness" intrinsic to it outside of the person who sees the plushie, and thus it doesn't matter.

My thoughts.
The short answer is: you can't.
The long answer is that since we're searching for a new definition of a term when removing a necessary (in my opinion) precondition for it to exist, the question becomes "can you make up a new definition for what you intuitively and empirically understand as intention, while removing free will from the picture?". I'm going to give it a shot.
First of all, there's a lot to be said about whether or not the idea or intuition of intention even exists in our collective discourse based on the latent assumption that free will is a thing. As in, before any rigorous definition, even the intuitions encoded in discourse, philosophy, art, and other languages as "intention" could very well be as invalid as the assumption of free will itself.
That being said, I'm a fan of Deleuze's model for people and other entities as machines of input, output and internal state (not his wording, but I paraphrase and in consequence, interpret and alter for the purposes of my point). It's not perfect, but I run to it a lot to answer these question as I find it very refreshing, often lacking in bias and having good explanatory power compared to the usual romance-foo that dominates these conversations. If that's the case you could pretty much define intention not as a self-started force but as a product of a much blurrier mechanism, namely the non-deterministic characteristic of this rhizomatic soup. Whether or not an input or output will exist, what kind will it be, what internal state will it find or cause in the machine and the long term dependencies between these interactions seem to me like a convincing enough candidate for the cause of any intuition (or illusion if you like a more cynic vocabulary) of "intention". It's pretty much an emergent symbol we use, assuming the form of a force for the setup and function of new connections, that will in their complication or pure non-determinism spawn even more intention in the network.
tl;dr: Intent could be the most basic expression of the RNG of the universe.

Thanks for this excellent answer and welcome to SE:AI!   I'm strongly persuaded that, although we cannot yet model the human brain, it is some kind of computer, in that it takes input and typically produces output.  It seems to me that intention can merely be a expression of pursuit of some goal, thus intention does not require free will or even non-deterministic process. Bell's Theorem strongly indicates randomness in nature, so I also like your point about it's potential relationship to intent, precisely because a coin flip is the opposite, unless its part of a search function.

@DukeZhou I like the way you approach it, I also think that if probabilistic versions of typical macro-level physics can in some way hold truth then it seems that intention is a sort of inefficient or probabilistic search for a predetermined goal, delineated by our specific universe (as opposed to any counterfactual).

Of course we are again constrained by the language we use, since in a way the ability for a system to have a goal in the first place is what I took to be a product of free will, which is why I had that whole intro. If goals can occur as contributions to some universal one, then of course no free will as we understand it is required but my personal intuition of what a goal is kind of breaks down at that point and it seems like we are talking more about a ball rolling down a rocky hill and search for paths down instead of choosing to fly vertically simply because it isn't concerned with gravity.

In order to answer the question requires that the definition of intelligence (human or otherwise) be grounded into the physical model.  There are two forms of intelligence in the physical model, one subconscious (general) and the other conscious (specific).
The subconscious is responsible for managing the brain’s process and performing all the physical reactions.  It exist in a deterministic universe where the decisions are known and reactions automated.  In the brain, the shape of the deterministic universe is time.  For something to be deterministic, it must have guaranteed results within a specific time cycle.  This requirement precludes the use of reasoning and long-term memory whose completion time cannot be fixed.  The subconscious runs on general intelligence.  To understand the format of general intelligence, think in terms of a jigsaw puzzle.  General intelligence only needs the border of the puzzle.  With just the basic object structure, the subconscious can move and process data throughout the brain without regards to its content.  It is functionally agnostic and can complete its automation cycle without reasoning.
Now, consciousness is responsible for mapping the non-deterministic environment to the deterministic universe.  To do this, it must exist outside of that universe so its own non-deterministic nature doesn’t interfere with the subconscious process.  Consciousness runs on specific intelligence.  Its job is to decipher the contents of the puzzle.  In humans, specific intelligence takes on the form of visual symbols.  These symbols are used by the conscious state for reasoning and long-term memory access.  How decisions are reached has no predefined form or technique.  It is a non-deterministic process based on the accumulation of individual experience and previous selection.  This process allows the biological life-form to adapt decision-making to its own environment.  In the most basic sense, the subconscious allows us to live in a deterministic universe and consciousness allows us to adapt to a non-deterministic environment.
The two forms of intelligence are functionally incompatible. The non-deterministic environment has no common translation to a deterministic universe.  If it did, it wouldn’t be non-deterministic.  To overcome the translation problem, the lower occipital lobe is a functional “Black Box” that allows the two different formats to be written into the same memory construct.  By simple association, the two systems coexist in a non-invasive relationship where the hippocampus (short-term) memory serves as an interchange point.
With a grounded physical model, I can now attempt to answer some of the questions.

We have “free will”, but it can only exist in the non-deterministic environment of the conscious state.  There is no such thing as “free will” in a deterministic universe.  The results of that universe are already known and decided.
A deterministic universe cannot change.  Its own nature precludes
adaption from within.  To make alterations requires a separate state
that can rise above the process in order to change the process.
Biological consciousness serves this role and requires “free will”
to make decisions outside of the scope of what is already known.
There are no ethics in a deterministic universe.  Abstracting right and wrong is a reasoning function which is not allowed.  AI constructs like agents which do pattern matching to develop predictions/reactions are not directly attached to any conscious state.  They are deterministic functions whose capability is predefined. Simply put, agents cannot exceed the sum of their programming because they cannot adapt to that which is not known or anticipated.
Liability only exists where “free will” exists and does not exist in the deterministic universe.  Otherwise, you are attempting to blame your hand for a decision originated in your frontal lobe.
“Free will” becomes a prisoner of the deterministic universe.  As experience and decision knowledge grow, more and more decisions are shifted to the subconscious.  This creates a dependency that restricts the range of “free will” and the decisions it is called upon to resolve.
Creativity does not exist in the deterministic universe.  It is an essential skill used by the conscious state to employ related memory to formulate new decisions.  Creativity is used by animals to develop survival skills to adapt to environmental changes.  The production of art work in an esoteric sense is purely human, but as a basic function is not.
Intent does not exist in the deterministic universe.  Within the non-deterministic universe of consciousness, intent is buried in an ocean of previous experience that may or may not reflect current “free will” goals.  Especially, if these goals threaten the survival of the biological process.

Can you please cite any research work that supports your claims about the subconscious, conscious, general intelligence, specific intelligence, etc.? I've never heard of these theories, you'd better just cite the theories behind your claims.

Sorry, I can’t cite any research other than my own.  The information I supplied in this answer was provided from an Artificial General Intelligence prototype that I have been developing.  The design of the prototype is based on a neurological algorithm derived from the physical model.  Actually, I would have been stunned if you had heard anything about this approach.  My thanks for confirming that belief. I am about to release a paper describing the full algorithm possibly later this week.  If you are interested, I can circle back around and provide a link.

The following two papers explain the brain’s physical model and basic algorithm.  I have found by using this model, I can begin to answer some of the hard questions such as “free will”.  It makes things less mystical, more systematic.
https://medium.com/@davidedgar_5686/neural-synchronization-in-the-human-brain-1c912890c831
https://medium.com/@davidedgar_5686/general-intelligence-in-the-human-brain-7c908acd6651

The term "intentionality" has two quite different senses. One is a very technical concept in philosophy of AI and means (roughly) aboutness in the sense that beliefs, desires, fears, etc., are about things (snakes, tax, chocolate). That's the sense central to searle's Chinese room argument. The other sense is the common idea of intending to do (or not do) something. A really key issue for AI according to Searle is how can a computer have intentionality in the first sense.

I like both answers but am going to propose something simpler:

Intention can be understood as an expression of pursuit of a goal, and thus does not require free will.

Deterministic algorithms can have and pursue goals, even without being consciously "aware" of the goal.  Thus, a simple NIM solving algorithm can be said to have the intention of winning at NIM, even there the goal is not a part of the algorithm, but embedded undefined in the simple algorithm itself.
This becomes even more true with neural networds, where, unlike the NIMATRON, goals are typically defined.


text:
What's the best method to predict/generate signal from one sensor (source) to signal from another another (target)?
I was wondering what is the best method out there to find relationship between two 1D signals so that I can predict/generate one (source) from the other (target). For example, let's say that in response to an event, my sensor A's readings vary in a certain way for 5 seconds. For the same event sensor B's readings vary as well for 5 seconds. Sensor A and B are not measuring the same physical quantities but respond to the same event and seem to have a relationship.
What can I do to use the signal from sensor A to learn how the signal from sensor B would look like for that event? What is the state of the art in deep learning?


text:
How do 3 channels affect a network when detecting human skin (CNN)?
Yeah I know, best title ever. Anyway,
I want to make a neural network which is fed with frames coming from an usb camera.
Don't wanna be so specific, so I'm just gonna say that the network's goal is to classify human hand gestures, therefore I need to make sure it can effectively learn how the hand moves around.
My problem is that I've no idea about what happens when having 3 channels instead of 1, I only know that (for 3 channels) it does 3 separate convolution operations with the same kernel, resulting actually in 3 separate layers.
How do this 3 channels affect the network? Does it learn from the movement 3 parallel times, then it mixes toghether this 3 "separate movements"? Do I need to make it single channel to help him detect the hand?
PS: the text is problably confusing, but that's because I'm confused to, that's why I'm asking.

3 channels will give you more information about the color of the object and surroundings which might be important in certain cases. For example if you want to classify blue cars and red cars then color of the object is very important. Since your problem is to classify hand gestures then color might not be that relevant. You're not very interested in the color of the hand you only care about it's position so there is a good chance that grayscale images might be enough. You should try with grayscale first and if that works good, if not, try with 3 channels.


text:
How to model personalized threshold problem with machine learning
Assume that I have a candidate selection system to generate product/user pairs for recommendation. Currently, in order to hold a quality bar for the recommended product, we trained a model to optimize for the click of the link, denoting as pClick(product, user) model, the output of the model is a score of (0,1) representing how likely the user will click on the recommended product.
For our initial launch product, we set a manually selected threshold, say T for all users. For all users, only when the threshold pass T, we will send user the recommendation.
Now we realize this is not optimal: Some users care less about recommendation quality while some other users have a high bar of recommendation quality. And a personalized threshold, instead of the global T can help us improve the overall relevance.
The goal is to output the threshold for each user, assume we have training data for each user's activity and user/product attributes.
The question is: How should we model this problem with machine learning? Any reference or papers is highly appreciated.


text:
Using ML for Enemy Generation in Video Games
I am attempting to make a 2-D platformer game where the player traverses through an evil factory that is producing killer robots. The robots spawn at multiple specific locations in each level and impede the player's progress.
Enemies are procedurally generated using machine learning. Early levels have "garbage" robots that plop down and can't really do anything. After generations of training, the robots begin having more refined bodies and are able to move about and attack the player. Later levels produce enemies that are more challenging.
Enemies consist of a body and up to 4 limbs. The body is simply a circle of a certain radius, while the limbs are just a bar with a certain length. Limbs can pivot and/or contract/extend. Additionally, each limb can have one of three types of "motor" (wheel, spring, or hover). This makes for about 20-25 input parameters:

BodySize, Limb1Enabled, Limb1PivotPoint, Limb1Length, Limb1Angle, Limb1MotorType, Limb1MotorStrength, Limb2Enabled, Limb2PivotPoint, Limb2Length, Limb2Angle, Limb2MotorType, Limb2MotorStrength, Limb3Enabled, Limb3PivotPoint, Limb3Length, Limb3Angle, Limb3MotorType, Limb3MotorStrength, Limb4Enabled, Limb4PivotPoint, Limb4Length, Limb4Angle, Limb4MotorType, Limb4MotorStrength

My thoughts are that a genetic algorithm (or something similar) would be used to generate a body, while a neural network would control that body by using the same inputs to generate outputs that control the limbs and motors.
There would actually be 3 "control brains" that would have to be trained using the same inputs, but having different fitness goals: Moving Right/Left, Moving Up, and Attacking the Player. (Gravity exists in 2-D platformers, so moving down isn't necessary.)
A fourth, "master brain" would take the player's relative location, score, and maybe time elapsed, as inputs, and would output one of the goals for the robot to achieve (move left, move right, and attack).
The master brain's fitness would be determined by the "inverse" of the player's "progress", while each control brain's fitness would be determined by how well it was able to perform the task assigned by the master brain. Finally, the overall fitness for the body's genetic algorithm would be an average (or some other function like min, max, etc.) of the three control brain's fitness values.
Now that I have all this "down on paper", where do I start? I had planned on doing this in Unity, but early attempts have been a bit confusing for me. I've been able to procedurally generate a body with random limbs (no motors) that wiggle about randomly, but there's no neural network or any machine learning going on whatsoever. I am not exactly sure how to expose my parameters to be used as inputs, and am barely grasping how I should take those outputs to control what I want them to. Are there any libraries I should look at, or should I write this all from scratch?
Also, before I get too far ahead of myself, what are the flaws in my approach (as I'm sure there are plenty). I want my project to be something practical in scope, if training can't be done feasibly while a player traverses a level, this might just be a dead project idea.
Anyways, that all being said, thank you for your help.


text:
How to adapt MTCNN to large images with relatively small ROIs
This question could be generalised to how to adapt state-of-the-art object detection models to large images with small ROIs.
In my particular case I'm trying to use this implementation of MTCNN to get bounding boxes for the faces of images of statues.
One challenge is that the face could take up a large proportion of the image like this:

Or a very small proportion of the image like this:

Where I'll zoom in on the statue's face so you can see it:

Bonus
If anyone has additional comments on my overall approach to this particular problem, happy to hear them.

The paper approach use image pyramid with downscaling only. That create problem for big images with small faces. You may be able bypass it with brute force sliding window approach: 1. Train on upscaled low-res images 2. for big images apply detector in sliding subwindow on several scales, with subwindow upscaled to detector resolution.

Thanks. I decided to ditch MTCNN for a while in favor of an approach of getting the statue with Faster R-CNN, then we'll see where I go from there


text:
Expected duration in a state
I am going through Rabiner 1989 and he writes that the discrete probability density function of duration $d$ in state $i$ (that is, staying in a state for duration $d$, conditioned on starting in that state) is $$p_i(d) = {a_{ii}}^{d-1}(1-a_{ii})$$
($a_{ii}$ is the state transition probability from state $i$ to state $i$ - that is, staying in the same state).
He then continues to say that the expected durations in a state, conditioned on starting in that state, is $$\overline d_i = \sum_{i=1}^\infty d p_i(d) = \frac{1}{1-a_{ii}}$$
Where does the coefficient $d$ (in $\sum_{i=1}^\infty d p_i(d)$) come from?

Looks like a typo and sum should be over $d$, not $i$, in the second equation. But I am not familiar with that tutorial.

@NeilSlater Still, even if it is summing over $d$, what does the coefficient $d$ represent? Is it adding $p_i(d)$ each time it stays in state $i$?

You already state that: duration

@NeilSlater Sorry I was not clear. Why do we multiply $p_i(d)$ by $d$?

. . . .because it is taking a mean/expected value by multiplying each possible value by its probability of occurring


text:
How should I go about selecting an optimal num_units within a LSTM cell for different sequence sizes
I am currently working on a stock market prediction model which incorporates sentiments along with historical price for next day price prediction.
I wanted to test different window / sequence size e.g (3 days, 4 days .. 10 days) to identify which window size is most optimal in predicting the next day prices.
However the selection for num_units in model.add(LSTM(units=num_units)) for different window sizes are varying.
If a smaller window size is paired with a larger num_unit, there is over-fitting in the data where the model prediction for the price at day t+1 is almost equal to the price at day t.
Hence I am unable to make a fair comparison between different window sizes without varying num_units
I have referred to this How to select number of hidden layers and number of memory cells in an LSTM? however am unable to come to a conclusion.
Is there a predefined guideline for the num_units to use within a LSTM cell for timeseries prediction based on the sequence length?


text:
How to calculate the confidence of a classifier's output?
I'm training a classifier and I want to collect incorrect outputs for human to double check.
the output of the classifier is a vector of probabilities for corresponding classes. for example, [0.9,0.05,0.05]
This means the probability for the current object being class A is 0.9, whereas for it being the class B is only 0.05 and 0.05 for C too.
In this situation, I think the result has a high confidence. As A's probability dominants B's and C's.
In another case, [0.4,0.45,0.15], the confidence should be low, as A and B are close.
What's the best formula to use to calculate this confidence?

There's no best formula, this is heuristic based. It depends on what accuracy you're looking for, but if you want a place to start, consider anything > 0.85 in the correct class a confident prediction, anything between 0.3 and 0.85 low confidence, and anything beneath 0.3 wrong

If you have a well-calibrated method that does indeed output probabilities, your problem is already solved. In the first instance, the classifier says there's a 90% probability that the object belongs to class A, but in the second instance, it's only 45% sure that it belongs to class B. What more do you want?

After your human curators have marked a sample of the output, you can use https://scikit-learn.org/stable/modules/calibration.html

I believe that there is no "best formula" here, as there are many Calibration metrics out there, depending on what you want to calibrate. This paper introduces three metrics for different purposes:

Expected Calibration Error (ECE): provides a single scalar summary of calibrations.
Maximum Calibration Error (MCE): use when we wish to minimize the worst-case deviation between confidence and accuracy
Negative log likelihood (NLL): this is the same as Cross-entropy loss.

There is also a related paper about more metrics.
Just like Accuracy, F1, and ROC-AUC, Calibration metric should depend on the use case.

Obvious answer for a binary (2 classes)classification is .5. Beyond that the earlier comment is correct. One of the things I have seen done is to run your model on the test set and save the prediction probability results.  Then create a threshold variable call it thresh. Then increment thresh from 0 to 1 in a loop. On each iteration compare thresh with the highest predicted probability prediction call it P. If  P>thresh declare that as the selected prediction, then compare that with the true prediction. Keep track of the errors for each value of thresh. At the end select the value of thresh that has the least errors.
There are also some  more sophisticated methods for example "top 2 accuracy" where thresh is selected based on  having the true class within either the prediction with the highest probability or the second highest probability . You can construct a weighted error function and select the value of thresh that has the net lowest error over the test set.
For example an error function might be as follows. If neither P(highest) or P(second highest) = True class, error=1. If P(second highest) = true class, error=.5. If p(highest)=true class error=0. I have never tried this myself so I am not sure how well this works. When I get some time will try it on a model with 100 classes and see how well it does. I know in the Imagenet competition they evaluate not just the top accuracy but also the "Top 3" and "Top 5' accuracy. In that competition there are 1000 classes.
I never thought of this before but I assume you could train your model specifically to optimize say the Top 2 accuracy by constructing a loss function used during training that forces the network to minimize this loss.

I assume you want a model that uses the Softmax as the output layer.
Basically, the Softmax will produce a set of probabilities that all sum up to 1. So if you have three classes in your data the Softmax will produce these confidence values by default, even though this is not exactly its main functionality.
The Softmax is commonly used on multiclass data.

Maybe its possible to use Entropy to evaluate confidence?
import torch

def entropy(cls_prob:torch.tensor)->torch.tensor:
    """
    Calculate entropy
    The larger the number, the lower the confidence level.

    Args:
        cls_prob (torch.tensor): The output score of the AI model,
        the expected input shape is (batch size, number of categories)

    Returns:
        torch.tensor: entropy of each sample
    """
    score = -torch.sum(torch.log(cls_prob)*cls_prob, dim=1)
    return score

prob_score = torch.tensor([[0.9,0.05,0.05], [0.4,0.45,0.15]])
confidence = entropy(prob_score)

and the confidence be like tensor([0.3944, 1.0104])


text:
Intuition behind single-shot object detection
Is there a good way to understand how single-shot object detection works? The most basic way to do detection is use a sliding-window detector and look at the output of the NN to detect if a class is there or not.
I'm wondering if there is a way to understand how many of the single-shot detectors work? Internally is there some form of sliding window going on? Or is it basically the same detector learned at each point?

Single shot detectors are very black box, so you're not going to know how it works internally, all you can look at is the structure. I suggest looking at YOLO, and how it operates. You can find out more here: https://towardsdatascience.com/yolo-you-only-look-once-real-time-object-detection-explained-492dc9230006

Most have internal regression layer which produce object box coordinates.

Most start with pretrained bounding boxes which are then regressed/removed/classified etc. as the layers go deeper. Most SSD's have different parts such as feature proposal and bounding box regressor, whereas yolo is a single network, SSD has multiple parts which you might have to look into.

As an obvious question, if the item of interest is moved some place else on the picture is the detection performance the same? It seems like a successful SSD would be very hard to train since position information is encoded in image which would result in poor generalization.


text:
Convolutional Feature Encoding Methods in DCNN
In Computer Vision, feature encoding methods are used on pre-trained DCNN to increase the feature robustness to certain conditions such as viewpoint/appearance variations ref.
I was just wondering if there are already available well established methods among AI community with probably python implementations.
I found the following ones in literature but without any tutorial or code example:

Multi-layer pooling ref
Cross convolutional layer Pooling ref
Holistic Pooling ref


text:
How to use convolution neural network in Deep-Q?
I currently have a grid of pixels 20x20. Each pixel can be red green blue or black. So I have one hot-encoded the pixels giving a 20x20x4 array for each screen.
For my Deep-Q Network, I have attached two successive screenshots of the screen together giving a 20x20x4x2 array.
I am trying to build a Convolutional Neural Network to estimate the Q values but I am not sure if my current architecture is a good idea. It currently is as shown below:
    def create_model(self):
        model = Sequential()
        model.add(Conv3D(256, (4, 4,2), input_shape=(20,20,4,2)))
        model.add(Activation('relu'))
        model.add(Dropout(0.2))

        model.add(Conv3D(256, (2,2,1), input_shape=self.input_shape))
        model.add(Activation('relu'))

        model.add(Flatten())
        model.add(Dense(64))
        model.add(Dense(self.num_actions, activation='linear'))
        model.compile(loss='mse', optimizer=Adam(self.learning_rate), metrics=['accuracy'])
        return model

Is a 3d convolution a good idea?
Is 256 filters a good idea?
Are the filters (4,4,2) and (2,2,1) suitable?
I realise answers may be highly subjective but I'm just looking for someone to point out any immediate flaws in the architecture.

Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable?

It's not so much that answers are subjective, but you are performing an experiment, and this should be driven by results. If you can find something published about a similar environment that might help you narrow down your choices.
That said, intuitively you don't gain much going from 2d to 3d convolutions when one of your dimensions is only sized 2. Purely from gut feeling I would suggest simply concatenating your two frames by having 8 channels instead of 4, and use 2d filters. This is simple to enough try and compare though, so perhaps you can do both.
You will likely want to explore deeper networks that have fewer initial filters and build up over a few more than two convolutional layers.
Definitely try your network without the dropout layer. I have not had much luck with using dropout in DQN, and I have seen others having similar problems. I am not sure what the exact issue is.
The accuracy metric won't do you much good on a regression task, so you can drop that and just use MSE loss. Bear in mind that training loss is a less useful metric overall in RL, since the prediction target (action value) is continuously changing as the policy changes. Low loss values mean that the value predictions are self-consistent, they don't necessarily mean that the learning has converged to an optimal policy.

Thanks, I was following a tutorial for the programming myself and was also wondering why the accuracy metric was there


text:
Use cases for AI inside the software company
This question is a bit philosophic and is about making new use cases for software companies. Let me describe what exist for now, why it is not enough, and what is needed.
I know that there are a lot of existing researches in applying ML for software (please don't simply point to this one!), but none of them consider the application of ML for software company, not the software alone.
Existent approaches that apply AI for software engineering tasks consider it as follows:
human1 -> software (big code) <- human2

That means that human1 makes some part of software (that is a part of big code), and human2 reuses some knowledge from it. It may be a bugfix pattern (as e.g. DeepCode does), API usage pattern, repair of code, summarization of code, code search, or whatever else. I think the main reason for this is the original hypothesis of naturalness:

The naturalness hypothesis. Software is a form of human communication; software corpora have similar statistical properties to natural language corpora, and these properties can be exploited to build better software engineering tools.

(from Allamanis et al, page 3)
But imagine one software company. It has:

Some number of engineers,
Some number of managers,
The software product,
Information related to the software product (documentation, bug/task tracking system, code review),
Some number of formal management processes (waterfall, scrum or whatever else),
Some number of informal processes

But none of these models consider the software as a product itself. I mean that we should consider the model as follows:
company -> software product -> customers
              |
              v
           big code

or even
engineer1 -> |
    |
engineer2 -> |
    |
...          | ----> software product ----> customers
    |                   |
engineerN -> |          |
    |                   |
manager  --> |          |
                        v
                     big code

So my questions are:

Are there any cases of investigation of such models?
Are there any similar cases in related fields, say in general companies (not specifically software ones)?
Are there any analogies (not specifically from software-related domains) where some knowledge can be transferred from a bigger object (big code in our case) to a smaller one (software product)?

Any ideas are welcome.


text:
How to update edge features in a graph using a loss function?
Given a directed, edge attributed graph G, where the edge attribute is a probability value, and a particular node N (with binary features f1 and f2) in G, the algorithm that I want to implement is as follows:

List all the outgoing edges from N, let this list be called edgelist_N.
For all the edges in edgelist_N, randomly assign to the edge attribute a probability value such that the sum of all the probabilities assigned to the edges in the edgelist_N equals to 1.
Take the top x edges (x can be a hyperparameter).
List the nodes in which the edges from step 3 are incoming.
Construct a subgraph with node N, the nodes from step 4 and the edges from step 3.
Embed the subgraph (preferably using a GNN) and obtain it's embedding and use it with a classifier to predict say f1/f2.
Propagate the loss so as to update the edge probabilities, that was assigned randomly in step 2.

I do not understand how to do step 7, i.e. update the edge attribute with the loss, so that edges which are more relevant in constructing the subgraph can be assigned a higher probability value.
Any suggestion would be highly appreciated.
Thank you very much.

Hi and welcome to AI SE? Is this a homework problem? Is this related to geometric deep learning? What's the name of the course?


text:
How to incentivise snake to go straight to apple?
I have made a Deep Q Network for the game snake but unfortunately, the snake exhibits some unwanted behavior. It generally does quite well but sometimes it gets stuck in an infinite loop that it can't escape and at the start of the game it takes a very long route to the apple rather than taking a more direct route.
The discount factor per time step is 0.99. The Snake gets a reward of +9 for getting an Apple and -1 for dying. Does anybody have any recommendations on how I should tune the hyperparameters/reward function to minimize this unwanted behavior?
I was thinking that reducing the discount factor may be a good idea?

Your network finds the infinite loop and notices that this has the best reward (0). This is probably because it hasn't found a path to eating the apple (through exploration).
Reducing the discount factor will only make the long term rewards less valuable. So it will learn eating the apple even slower.
I don't know what you are using as inputs for your network, but maybe changing your reward system could help. For example, you could give your network a reward if it advances in the direction of the apple. This way, your network will be encouraged to find the apple more than it is now.

I'm using a convolutional neural network so I don't think that approach would be easily implemened I think.

My thinking was that reducing the discount factor would incentivise the snake to get the reward sooner rather than later because if it waits a while to get it then it will recieve less reward

How is this not easily implemented? After each decision you measure distance to the apple, if it is smaller than last distance you get reward +1.

You are right. I feel that may cause some unwanted behavior though. It might keep going back and forth between the apple and not actually getting it as it can keep getting reward this way. I will try it out and see if it works

This is of course the problem with reward shaping - spare rewards. Try this out, using different rewards for moving away form the apple, towards the apple, eating the apple, and dying. you can even give a negative reward for not eating the apple within a certain time period

Another approach is that you should give negative rewards for wandering too long. also during training, help your agent to experince wandering by not allowing to randomize exploration for small period of time. this way it helps with unnecessary detours.

authors: ปวีณา เสนีวงศ์ณอยุธยา


text:
Dealing with very similar object classes in object detection
I'm working on an object detection problem using Faster R-CNN. I need to identify two object classes, and they are very similar to one another. Furthermore they are similar to a third type of object which should be considered as background. Also, all three of these objects have a lot of variation within them.
In my particular example the two objects of interest are 1) a statue of a particular named person who appears in many statues, 2) a statue of anyone else
Examples:

Also, I want to treat living flesh people, or non-humanoid statues as background.
Now here are some interesting results:

The RPB losses follow the expected trajectory for such a problem, but on the other hand, I really had to have faith and hang in there for the detector losses. They take a while to start decreasing, and I presume it's because there is a relatively sharp trough leading to the the minimum of the loss function with respect to the weights (because the labelled classes are so similar to one another). Miraculously (at least I think so), it does start to kind of work, but not as well as I'd like it to.
My question is as in the title but here are some of my thought processes:

Does the class similarity spoil the bounding box regression? And then does that spoil the class inference in turn?
Would it be better to just detect humanoid statues in general, then train a classifier from scratch on the output of that? (I don't know about this one. Maybe the relevant info is already encoded in the detector of the Faster R-CNN and the added bonus is that the Faster R-CNN gets context from outside of the bounding box... unless the bounding box regression is being spoiled by the class ambiguity)


text:
How to deal with features which are here just for training?
I'm new to the Data Science field and last week I started to learn about Neural Networks and Deep Learning. To practice, I decided to do a small project: design a Neural Network to predict the winner of an NBA game given the two teams playing. Also, for each match I have 2 stats (let's say number of points and number of free throws) for each of the teams.
In the end, the dataset looks like:
|  ID |  Home |  Away | H_Pts | H_Fts | A_Pts | A_Fts | H_win |
|:---:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|  1  | Team1 | Team2 |   45  |   10  |   47  |   8   |   1   |
|  2  | Team3 | Team4 |   56  |   6   |   70  |   13  |   0   |
| ... |  ...  |  ...  |  ...  |  ...  |  ...  |  ...  |  ...  |

I implemented the model with TensorFlow/Keras (with the help of this tutorial: Classify structured data with feature columns | TensorFlow Core).
The code is pretty concise:
batch_size = 16
train_ds, test_ds, val_ds = get_datasets()  # The function mainly uses tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
team_names = get_team_names()

feature_columns = []
for column_name in ['Home', 'Away']:
    team = feature_column.categorical_column_with_vocabulary_list(column_name, team_names)
    feature_columns.append(feature_column.indicator_column(team))

for column_name in ['H_Pts', 'H_Fls', 'A_Pts', 'A_Fls']:
    feature_columns.append(feature_column.numeric_column(column_name))

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)

model = tf.keras.Sequential([
    feature_layer,
    layers.Dense(128, activation='relu'),
    layers.Dense(128, activation='relu'),
    layers.Dense(1)
])

model.compile(optimizer='adam', loss=tf.keras.losses.BinaryCrossentropy(from_logits=True), metrics=['accuracy'])

model.fit(train_ds, validation_data=val_ds, epochs=10)

loss, accuracy = model.evaluate(test_ds)
print(f"Model evaluation: Loss = {loss} | Accuracy = {accuracy}")

Trained with just 100 games, I get a great accuracy: 99%. Of course: as it is, the test dataset given to model.evaluate(test_ds) contains everything except the target label H_win. Because H_win can easily be deduced from H_Pts and A_Pts, I get a high accuracy. But this model can't work because by definition you don't know the number of points of each team before the game...
How should I deal with features like these ones which I do not want to predict (so they're not labels) but that should still be considered during the training? Does this kind of feature have a name?

Afaik, those auxiliary features are still predictions of your model that you later use to derive the classification label. Why do you say they are not predictions of your model in this case? Am I misunderstanding your question?


text:
Is it okay to have wide variations within one of the classes for binary classification tasks?
Say I am using a convolutional network to classify pictures of my face versus anyone else's face in the world.
So let's take 10000 pictures of me, and 10000 pictures of other people.
And let's do three experiments where we train a binary classifier:
1) The 10000 "other" pictures are of 1 other person.
2) The 10000 "other" pictures are of ~10 other people (approximately balanced, so about 1000 pictures per person).
3) The 10000 "other" pictures are of ~10000 other people.
I only have one question but here are some different perspectives on it:

Are any of these cases categorically harder to solve than the others?
Are they the same difficulty, or close?
Are there known considerations to make when tuning the model for each
of the cases? (like maybe case (3) has a sharper minimum in the loss
function than (1) so we need to use a different optimisation approach)

It depends on the context whether it's "harder". Are each of these options representative of what the network will see in testing data? Which ever matches the testing data most closely will have the best result.


text:
How do you perform a gradient based adversarial attack on an SVM based model?
I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing
Adversarial Examples.

I am struggling to actually calculate the gradient of the SVM cost function with respect to the input (I am assuming it needs to be w.r.t input).
Is there a way to avoid the maths (I am working in python if that helps?)

A way to avoid computing the SVM loss by hand is to use a differentiable programming framework, such as JAX. These frameworks will automatically calculate gradients using automatic differentiation.
If you can write down the SVM loss using numpy operations then you can use the framework's tools to get a function which evaluates the gradient with respect to any argument.
In JAX this would look like:
import jax
import jax.numpy as jnp
def hinge_loss(x, y, theta):
    # x is an nxd matrix, y is an nx1 matrix
    y_hat = model(x, theta) # returns nx1 matrix, model parameters theta
    return jnp.maximum(0, 1 - y_hat * y)

hinge_loss_grad = jax.grad(hinge_loss)
# hinge_loss_grad takes an x, y, theta and returns gradient of hinge loss wrt x


text:
Model for supervised sequence classification task
The Problem
I am currently working on a sequence classification problem I try to solve with machine learning.
The target variable is the current state of a system.
This target variable is following a repeating pattern (eg. [00110200033304...]).
So Transitions are only allowed from or to the "0" state if you imagine the system as a state machine.
The only deviation is the time the system stays in one state (eg. iteration_1 = [...0220...], iteration_2 =[...02220...]).
My Question
What would be the best choice of (machine learning) model for this task if one wants to optimize for accuracy?
Restrictions

No restrictions regarding time / space complexity of the model
No restrictions regarding the type of model
The model is only allowed to make wrong classifications in the state transitions phases (e.g. true: [011102...], pred: [001102...]) but must not validate the sequence logic (e.g. true: [011102...], pred: [010102...])

Additional Info / Existing Work

With a lstm neural network (many to 1) I achieved an overall accuracy of 97% in an unseen test set.
Unfortunately the network predicted sequences which violate the sequence logic
(e.g. true: [011102...] predicted: [010102...]) even tough the window length was wide enough to cover at least 3 state transitions.
with simple classification models (only one times step per classification, tested models: feed forward neural network, xgboost / adaboost) an accuracy of ca. 70% are reachable
The input signal is acoustic emission in the frequency domain; Ca. 100 frequency bins / 100 features

Ideas

Maybe the lstm would work better in "many to many" designed with a drastic reduced input dimensionality by increased window size?
Maybe a combination of the probability output of the lstm with
a timed Automaton (a state machine with time dependent probability density functions
about the state changes) or a Markov chain model could significantly improve the result?
(But this seems really inelegant)
Is it eventually possible to impose the restriction of valid sequences onto the lstm model?


text:
What algorithm to use for finding artists/bands in text and differentiating between artists that share the same name
Here's the data I have:

Text from articles from various music blogs & music news sites (title, summary, full content, and sometimes tags).
I used a couple different NLP/NER tools (nltk, spacy, and stanford NER) to determine the proper nouns in the text, and gave each proper noun a score based on how many times it appeared, and how many NLP tools recognized it as a proper noun. None of these tools are very accurate by themselves for my data
For each proper noun I queried musicbrainz to find artists with that name. (musicbrainz has a lot of data that may be helpful: aliases, discography, associations with other artists)
Any links in the article to Spotify, YouTube etc. and the song name & artist for that link

I have three goals:

Determine which proper nouns are artists
For artists that share the same name, determine which one the text is referring to (based on musicbrainz data)
Determine if the artist is important to the article, or if they were just briefly mentioned

I have manually tagged some of the data with the correct output for the above 3 goals.
How would you go about this? Which algorithms do you think would be best for these goals?
Is there any semi-supervised learning I can do to reduce the amount of tagging I need to do?


text:
I created a snake game and fitted the NEAT algorithm and there's issues
Below are my Inputs Outputs and fitness function. The snake is learning at a slow rate, and seems to be stagnant, additionally when the snake collides with the food, it gets deleted from the genome, which doesn't make any sense because that's not specified in the collision. Any input would be greatly appreciated
for x, s in enumerate(snakes):
        # inserting new snake head and deleting the tale for movement

        # inputs
        s.x = s.snake_head[0]
        s.y = s.snake_head[1]
        snakeheadBottomDis = win_h - s.y
        snakeheadRightDis = win_w - s.x
        snake_length = len(s.snake_position)
        snakefoodDistEuclidean = math.sqrt((s.x - food.x) ** 2 + (s.y - food.y) ** 2)
        snakefoodDisManhattan = abs(s.x - food.x) + abs(s.y - food.y)
        xdis = s.Xdis()
        ydis = s.Ydis()
        s.dis_list1.append(snakefoodDistEuclidean)
        s.dis_list2.append(snakefoodDisManhattan)
        s.dis_list3.append(s.Xdis())
        s.dis_list4.append(s.Ydis())
        s.hunger_list.append(s.hunger)
        #print('Euclidean: ', dis_list1[-1])
        #print('Manhattan: ', dis_list2[-1])
        #print('X distance from Wall: ', dis_list3[-1])
        #print('Y distance from Wall: ', dis_list4[-1])

        output = nets[snakes.index(s)].activate((s.hunger, s.x, s.y, food.x, food.y, snakeheadBottomDis,
                                                 snakeheadRightDis, snake_length, xdis,ydis,
                                                 snakefoodDisManhattan, snakefoodDistEuclidean,s.dis_list1[-1],s.dis_list1[-2],
                                                 s.dis_list2[-1],s.dis_list2[-2],s.dis_list3[-1],s.dis_list3[-2],
                                                 s.dis_list4[-1],s.dis_list4[-2],s.hunger_list[-1],s.hunger_list[-2]))

        #snake moving animation
        s.snake_position.insert(0, list(s.snake_head))
        s.snake_position.pop()
        s.hunger -= 1

        # Checking distance Euclidean and Manhattan current and last
        if s.dis_list1[-1] > s.dis_list1[-2]:
            ge[x].fitness -= 1

        if s.dis_list1[-1] < s.dis_list1[-2]:
            ge[x].fitness += 0.5

        if s.dis_list1[-1] > s.dis_list2[-2]:
            ge[x].fitness -= 1

        if s.dis_list1[-1] < s.dis_list2[-2]:
            ge[x].fitness += 0.5

        #checking hunger number and if its decreasing
        if s.hunger_list[-1] < s.hunger_list[-2]:
            ge[x].fitness -= 0.1

        # move right
        if output[0] >= 0 and output[1] < 0 and output[2] < 0 and output[
            3] < 0:
            #and s.x < win_w - s.width and s.y > 0 + s.height:
            # ge[x].fitness += 0.5
            s.move_right()

        # move left
        if output[1] >= 0 and output[0] < 0 and output[2] < 0 and output[
            3] < 0:
            #and s.x < 500 - s.width and s.y > 0 + s.height:
            #ge[x].fitness += 0.5
            s.move_left()

        # move down
        if output[2] >= 0 and output[1] < 0 and output[0] < 0 and output[
            3] < 0:
            #and s.x < 500 - s.width and s.y > 0 + s.height:
            # ge[x].fitness += 0.5
            s.move_down()

        # move up
        if output[3] >= 0 and output[1] < 0 and output[2] < 0 and output[
            3] < 0:
            #and s.x < 500 - s.width and s.y > 0 + s.height:
            # ge[x].fitness += 0.5
            s.move_up()

        #adding more fitness if axis aligns
        if s.snake_head[0] == food.x:
            ge[x].fitness += 0.1
        if s.snake_head[1] == food.x:
            ge[x].fitness += 0.1

        # checking the activation function tanh
        # print ('output 0: ', output[0])
        # print('output 1: ', output[1])
        # print ('output 2: ', output[1])
        # print ('output 3: ', output[1])

        # snake poping on other side of screen if screen limit reached
        if s.snake_head[0] >= win_w - s.width:
            s.snake_head[0] = 12
        if s.snake_head[0] <= 11 + s.width:
            s.snake_head[0] = win_w - s.width - 1
        if s.snake_head[1] >= win_h - s.height:
            s.snake_head[1] = s.height + 15
        if s.snake_head[1] <= 11 + s.height:
            s.snake_head[1] = win_h - s.height - 1

        head = s.snake_position[0]
        #s.x < 0 + s.width or s.x > win_w - s.width or s.y < 0 + s.height or \
                #s.y > win_h - s.height or

        #if run into self you die
        if head in s.snake_position[1:]:
            ge[x].fitness -= 10
            snakes.pop(x)
            nets.pop(x)
            ge.pop(x)

        #if hunger reaches 0 you die
        if s.hunger == 0:
            ge[x].fitness -= 5
            snakes.pop(x)
            nets.pop(x)
            ge.pop(x)

        #if snake collides with food award fitness
        if s.getRec().colliderect(food.getRec()):
            ge[x].fitness += 100
            s.hunger = 100
            score += 1
            s.snake_position.insert(0, list(s.snake_head))
            food.y = random.randint(0 + 24, 500 - 24)
            food.x = random.randint(0 + 24, 500 - 24)

    # print(s.hunger)

Firstly, the key of implementing a good genetic algorithm like the NEAT one is fitness. Now fitness is everything, it tells basically what your snakes will learn. If you have a bad fitness function, your AIs will target the wrong goal. You shouldn't give fitness when a snake is aligning with food, because that's not what you want. What you really want your snake to do is eating food.
So, I suggest you to try giving fitness only when your snake is eating food, and maybe remove some as time passes (because of hunger). Your fitness could even be just the time a snake survived, since to survive your snakes have to eat, and avoid touching themselves! Definitely try this: just giving fitness as time passes!
Secondly, you have to give good inputs to your AIs, so that they have enough information about their environment to optimize their strategy. I suggest you to try inputs that are relative to your snake's position, and direction. X and Y position of the snake and the food are bad inputs, because they're not relative to the snake, they're relative to the origin of your game. So, every second the position of the snake is changing, and this could lead to some distraction.
The snake doesn't need to know his position, he just need to know the distance between him and the food, and the angle between the direction of the snake (him) and the food.
Lastly, you may review your outputs. But this is a minor problem, you should definitely focus on the fitness of your snakes. But, if you want to go deeper, try having two outputs instead of four: speed, and turning velocity. That way, it's easy for your snakes to go straight forward. When a snake wants to go forward with left, right, up, down outputs, it's way harder.