text:
In how few updates can a multi layer neural net be trained?
A single iteration of gradient descent can be parallelised across many worker nodes. We simple split the training set across the worker nodes, pass the parameters to each worker, each worker computes gradients for their subset of the training set, and then passes it back to the master to be averaged. With some effort, we can even use model parallelism.
However, stochastic gradient descent is an inherently serial proces. Each update must be performed sequentially. Each iteration, we must perform a broadcast and gather of all parameters. This is bad for performance. Ultimately, number of updates is the limiting factor of deep model training speed.
Why must we perform many updates?
With how few updates can we achieve good accuracy?
What factors affect the minimum number of updates requires to reach some accuracy?

In Don't Decay the Learning Rate, Increase the Batch Size, Smith et al. train ResNet-50 on ImageNet to 76.1% with only 2500 updates. Has anyone done it in less?
In The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Sankararaman et al. present the concept of gradient confusion which slows convergence, and show that depth increases gradient confusion while overparameterization on width and skip connections reduces gradient confusion.
In Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flash, Mikami et al. train ResNet-50 on ImageNet to 75.29% with some number of updates, but I can't do the math to compute out how many updates they must have used.
To summarise:

Larger mini-batches help, but give diminishing returns and cause other issues.
Deeper models tend to require more updates.
Wider layers need fewer updates.
Skip connections, label smoothing, and batch norm might help.
The best I have found so far on ImageNet/ResNet-50 to >75% is 2500 updates.


text:
Can ConvLSTMs outuput images?
I am working on an image to image regression task which requires me to develop a deep learning model that takes in a sequence of 5 images and return another image. The sequence of 5 images and the output images are conceptually and temporally related. In fact, the 5 images in the sequence each correspond to timestep in a simulation and the output, the one I am trying to predict, corresponds eventually to the 6th timestep of that sequence.
For now, I have been training a simple regression-type CNN model which takes in the sequence of 5 images stored in a list and outputs an image corresponding to the next timestep in the simulation. This does work with a small and rather simple dataset (13000 images) but works a bit worse on a more diverse and larger dataset (102000 images).
For this reason, I have been researching a bit now in order to find a better way to carry out this task and I found the idea of ConvLSTMs. However, I have seen these applied to the prediction of feature and the output of a sentence describing that image. What I wanted to know is whether ConvLSTMs can also output images, but more importantly if they can be applied to my case. If not, what other types of deep learning network can be suitable for this task?
Thanks in advance!

If you only need 5 frames to predict the next frame, then I'd recommend a U-Net architecture, wich is basically a CNN encoder/decoder network in which the decoder uses the intermediate features produced in the encoder as well as its own features to produce an output image. Also, in additional to using a conventional L2 loss for the output image, you can always add an additional GAN loss to make the image look more realistic.
If using a longer history of frames can help, then I recommend taking a look at "Recurrent Environment Simulators" and combining it with the ideas above.
Hope it helps!

Thanks a lot for your answer @Mehran Shakerinava, I can't bump up your answer yet becuase my reputation is under 15, but still thanks. Can I adapt U-net architectures for tasks other than image segmentation?

@lr99 You're welcome! The U-Net architecture can be used to produce any image. In image segmentation, this output image is suggesting how the image can be segmented, but in general it could be anything.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How does one create a non-classifying CNN in order to gain information from images?
How do I program a neural network such that, when an image is inputted, the output is a numerical value that is not the probability of the image being a certain class? In other words, a CNN that doesn't classify. For example, when an input of an image of a chair is given, the model should not give the chance that the image is a chair but rather give the predicted age of the chair, the predicted price of the chair, etc. I'm currently not sure how to program a neural net like this.

This can be thought of as a loss function design problem. If you optimize your network weights for something like multi-class classification, then expect your network to learn weights for this task (You will use cross entropy loss for this task). If you optimize your network to output a single value at the last layer and treat it as a regression problem for age prediction, then your network can learn weights for this particular task (You may use something like Mean Square Error loss here).
Let me give you a weak guidline on how to do this. Suppose say your input is images of chairs and you want to predict their age using a pretrained resnet, this is how you may do it in pytorch.
Definition
X: Input pictures
Y: List of ground truth values which is age here. i.e. : [1.4, 2.5, 2.2, ....]
Modify your neural network to give one output at the last layer
model = torchvision.models.resnet18(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = torch.nn.Linear(num_ftrs, 1)
Design your loss function appropriately. You can treat age prediction as a standard regression problem. Of course, there are better loss designs here.
criterion =  torch.nn.MSELoss()
So use this loss during training
loss = criterion(output, target) where output is your neural network prediction and target is your ground truth values.
This is how you can modify an existing architecture for your task. Hope it helps.

You generally use SoftMax Layer as the output layer for a neural network that is used as a Classifier.
Now, if you want your neural network to predict the age of a chair, predicting price of a chair like in linear regression (output is continuous), you have to remove the SoftMax Layer and add one or multiple layers such that the output of the final layer gives only one value at the output. (which is the prediction for you age or price). And, instead of logit loss you can use MSE for back propagation.
So, like keshik mentioned in another answer, it's all about the final layer used and the loss function used. Based on what you use, your weights get trained.
This is what is done in transfer learning also. Based on the task you want to achieve you change the last layers and retrain your network.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What are some examples of LSTM architectures?
I've been doing some class assignments recently on building various neural networks. For convolutional networks, there are several well-known architectures such as LeNet, VGG etc. Such "classic" models are frequently referenced as starting points when building new CNNs.
Are there similar examples for RNN/LSTM networks? All I've found so far are articles and slides explaining recurrent neurons, LSTM layers, and the math behind them, but no well-known examples of entire multi-layered network architectures, unlike CNNs which seem to have in abundance.

In the paper, LSTM: A Search Space Odyssey (2017), by Klaus Greff et al., eight LSTM variants on three representative tasks (speech recognition, handwriting recognition, and polyphonic music modeling) are compared.
The compared variants are

Vanilla LSTM features three gates (input, forget, output), block input, a single cell, an output activation function, and peephole connections (connections from the cell to the gates). The output of the block is recurrently connected back to the block input and all of the gates. The vanilla LSTM is trained using gradient descent and back-propagation through time (BPTT). The original LSTM (which is not the vanilla LSTM) does not contain, for example, the forget gate or the peephole connections (but the cell possesses a constant error carousel, a constant weight of $1$).
LSTM trained based on the decoupled extended Kalman filtering (DEKF-LSTM), which enables the LSTM to be trained on some pathological cases at the cost of high computational complexity.
Vanilla LSTM trained with an evolution-based method (called evolino), instead of BPTT.
LSTM block architectures evolved with a multi-objective evolutionary algorithm, so that to maximize fitness on context-sensitive grammar.
LSTM architectures for large scale acoustic modeling, which introduces a linear projection layer that projects the output of the LSTM layer down before recurrent and forward connections in order to reduce the number of parameters for LSTM networks with many blocks.
An LSTM architecture with a trainable scaling parameter for the slope of the gate activation functions, which improves the performance of LSTM on an offline handwriting recognition dataset.
Dynamic Cortex Memory, an LSTM composed of recurrent connections between the gates of a single block, but not between different blocks, which improves the convergence speed of LSTM.
Gated Recurrent Unit (GRU), which simplifies the architecture of the LSTM by combining the input and forget gate into an update gate.

There are other related neural network architectures, such as the neural Turing machine (NTM) or differentiable neural computer (DNC). In general, there are several architectures that use LSTM blocks, even though they are not just recurrent neural networks. Other examples are the neural programmer-interpreter (NPI) or the meta-controller.


text:
Finding optimal Value function and Policy for an MDP
I am solving an RL MDP problem which is model based. I have an MDP which has four possible states S1-S4 and four different actions A1-A4, with S4 being terminal state and S1 is the beginning state. There is equal probability of applying any of the available actions on S1. The goal of my problem is to get from S1 to S4 with the maximum possible reward. I have two questions in this regard -

Will this be a valid MDP model if I have this rule for my model - If i perform action a1 on S1 and next state is S2 then the set of available valid actions on S2 will be only a2,a3 and a4. If i apply any of the actions a2,a3 or a4 on S2 and it takes me to S3 then now i am left with a set of two valid actions for S3(except the one that was taken on s2). Can i do this? Because in my problem an action once taken does not require to be taken again later.
I am confused about finding the optimal value and policy for my MDP. Both the optimal value and policy functions are not known to me. The objective is to find the optimal policy for my MDP which would get me from S1 to S4 with max reward. Any action taken on a particular state can lead to any other states (i.e there is a uniform equal state transition probability of 25% in my case for all states except S4 since it's terminal state). How can i approach this problem? After a lot of google search I vaguely understood that I must start with choosing a random policy (equal probability of taking any valid action) -> find value functions for each state -> iteratively compute V until it converges and then from these value functions I need to compute the optimal policy. Those solutions mostly use the Bellman Equations. Can someone please elaborate on how i can do this? Or if there is any other method to do it?

Thank you in advance

You have jumped too far into the solution and are already getting some fundamental things wrong. For instance, if an action choice is restricted after being taken once, that information needs to be part of the state. So you probably don't have four states. I suggest that instead of diving straight in to your (maybe incorrect, and most definitley confusing) analysis of your problem as a MDP, you start with a description of the problem you are trying to solve, present your MDP analysis and get feedback on it. Once you have that, then it may be possible in a new question to look for solutions.

okay i will work on the MDP and put up a new question with the problem description and all. I could not understand this statement - "if an action choice is restricted after being taken once, that information needs to be part of the state". Could you please elaborate? I am sorry i am new to this field and trying to implement these algorithms in my domain ..

Did you mean embed that information in the state transition probability matrix??

No, I mean that you don't just have states S1-S4, if one of the rules is that actions cannot be repeated. Even if the state carried no information at all except which actions were restricted, you would have at least 15 states. If the state S1-S4 is already something meaningful in your curent setup, then you have at least 60 states. If action sequence is important, then you will have more

Hi Neil, as per your suggestion i have included a more elaborate MDP analysis and opened a new question. here's the link - https://ai.stackexchange.com/questions/14296/can-someone-please-help-me-validate-my-mdp . Would be grateful if you could look at it and help me out ! Thank you !


text:
Small Machine Translation Model
I would like to perform Machine Translation in tensorflow.js, in the browser. The issue is that state-of-the-art models have many gigabytes (fairseq ensemble), and translation is slow.
Do you know some good small models for Machine Translation? Probably something below the 50 million parameter threshold.
I found some older papers with models below 100 MB for training on small datasets (IWSLT'15 English-Vietnamese attention-based models), but these are probably superseded.

There's a 300MB model in OpenNMT, which would perhaps be compressed with ERNIE. The paper states that 5x compression keeps 98% accuracy.


text:
Applying ML algorithms to data-sets with similar meta-features?
Is there any grounds for assuming an algorithms applied to a data-set that created a decently accurate model will perform as well on a different data-set with meta-features chosen and evaluated by meta-learning? What meta-features are even worth considering when evaluating similarity between data-sets with the goal of finding an optimal combination of algorithm application to this new data-set to create an accurate model?


text:
How does a single neuron in hidden layer affect training accuracy
I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer  can model any mathematic function with accuracy dependant on number of hidden layer neurons. Try as I might I cannot find any explanation as to why that occurs - could someone maybe explain the question why that is?

Possible duplicate of Where can I find the proof of the universal approximation theorem? You're asking two distinct questions: 1) how does the number of neurons in a hidden layer affect the accuracy of the model and 2) why an NN, with a single hidden layer with an arbitrary number of neurons, can approximate any function. Please, ask just one question per post. The answer to your 2nd question is already implicitly given in the linked post, so I suggest you ask the first question.

The claim that Neural Network with a single hidden layer can model any functions is proven in Cybenko's Approximation by superpositions of a sigmoidal function.
https://link.springer.com/article/10.1007/BF02551274
check also: https://en.wikipedia.org/wiki/Universal_approximation_theorem
The thing is that the neural network using sigmoidal functions, which are non-linear functions can.


text:
Sliding Window Detection
Suppose that we have a labeled training set of $n$ closely cropped images of cars $(x_1, y_1) , \dots, (x_n, y_n)$. We then train a CNN on this. Let's say we have $m$ test images. Then for each of the $m$ images, do we use the trained CNN on a cropped out portion of the box to detect whether there is a car or not? If the object is large, wouldn't having a large sliding window have better performance than a smaller sliding window?

Hi and welcome to this community! I think it would be useful if you can explain more in detail the relationship between the sliding window and the test dataset: why do you need a sliding window in the first place? Are you referring to a specific model that uses a sliding window? If yes, edit your question to add these details. What is the context of your question, model and problem?

@nbro: https://www.pyimagesearch.com/2015/03/23/sliding-windows-for-object-detection-with-python-and-opencv/   The idea is to use the "non-convolutional" version of YOLO.


text:
I am trying to create a network with dynamic connections for every different training example
I have developed a new algorithm which has dynamic connections. it is based off of the following paper:
https://www.researchgate.net/publication/334226867_Dynamic_Encoder_Network_to_Process_Noise_into_Images
i have figured out how to effect the dynamism. The pseudo code is in the diagram.
Additionally i would really like to know how to do the rest of the code in terms of a general overview. i would need to have the same deconvolutional layer for all training examples, as well as different current_weight_matrices for the different training examples. so how do i go about adding the dynamic layers to the deconv neural network . Will i have to have many copies of the same network?


text:
Reinforcement learning with hints or reference model
In Reinforcement Learning, when I train a model, it comes up with its own set of solutions. For example, if I am training a robot to walk, it will come up with its own walking gait, such as this Deep Mind robot that has learned to walk in a bizarre gait. It can surely walk/run although the movements does not quite look like a human.
I was wondering how can I train a model by providing it some kind of reference motion data? For example, if I collect motion data from a walking human and then provide it to the training, can the training be made learn the walking movements that looks similar to the reference motion data?
Searching online I did find some links that shows this is possible. For example, here is a research where the researchers did exactly what I am trying to do, they fed motion data captured from humans to a simulation and made it learn the movements.
So, my question is: How can I give some kind of hints or reference data to a reinforcement learning model instead of just leaving it all by itself? how exactly is this done? What is it even called? What are some terms and keywords that I can search for to learn more?
Many thanks in advance

You can look into the techniques used in GANs (genereative adverserial networks). These networks work by having 2 learning agents. 1 to create images and 1 to learn the difference in a human made image and a computer generated image. This works because the 2 agents drive each other to be better and ultimately make the generator create images which can't be distinguished from real life images.
In your case you can make a agent trying to tell if the data is human or computer generated. The agent learning to move will then get negative rewards when the other agent can identify it as computer movement. This way the mover will learn to move like your reference data.
UPDATE:
I just found this video and paper which does exactly the same as you asked. Instead of using a GAN like structure they use a task specific reward and a imitation reward, which is based on reference motion data they have.
https://www.youtube.com/watch?v=vppFvq2quQ0

To my knowledge it hasn't been done yet. Is it implementable? sure, if you look at the successes of GANs it should be able to work. to only real hard problem is getting enough reference data and figuring the right balance between moving fast and moving like a human I would guess. This is from someone who hasn't implemented a GAN before though, so I don't know how valuable my opinion on this is.


text:
Is Value Iteration better than Policy Iteration for first few iterations?
In Policy Iteration (PI), the action generated by the policy, whether it's optimal or not w.r.t the current value function $v(s)$. Whereas, in Value Iteration, the action is greedily generated w.r.t current $v(s)$, which is an approximation of the objective function (as I understand). As a consequence, in the first few iterations, will Value Iteration perform better than Policy Iteration?

@DuttaA That's what I'm trying to figure out. I'm new to reinforcement learing. I'm used to notations in optimal control and classic optimization, where objective function is used. Since reinforcement learning is solving an optimization problem, I assume the value function will eventually converge to the objective function (or the optimal value function)? Then the greedy action w.r.t the optimal value function corresponds to the optimal control action that minimizes the objective function in optimal control.

@DuttaA I totally agree. I'm still taking the first steps to figure out what everything means in RL and their corresponding counterparts in my problems. After that, I'll confidently formulate my problems with terminology the RL community agreed on.

@qiangli In general, an objective function is just the function you want to optimize (either minimise or maximise). It seems that you're using the expression objective function as a synonym for the function you want to learn or find.


text:
Probabilistic classification - normalize results
I have a probabilistic classifier that produces a distribution over my 3 classes - C1, C2, C3.
I want to compare some new points I'm classifying to each other, to see which one is the best fit for a specific class.
for example:.
for a new point X1 the classifier will output something like [0.2, 0.2, 0.6]
for another new point X2 it will produce [0.2, 0.4, 0.4]
so for both X1, X2 - the chosen class would be C3.
Now I want to know - which of X1, X2 is a better fit to C3
I cannot simply choose the one with the highest probability for C3, because it's probability for C3 depends on it's probabilities for the other classes. X1 got 0.6 and X2 0.4, but it's possible that X2 is closer to C3 in the hyper-plane than X1, it is just less unique for C3 than X1, and therefor X1 got a higher probability.
here's a visual, in 2 dimensions:

X2, who got a lower probability, is clearly a better fit to the Red class then X1, which is truly unique to the Red class, but is further from the class cluster
My questions are:

how do I normalize the results of a probabilistic classifier so I can compare predictions to each other?
given an output of a probabilistic classier - how can I get the actual distance from the probabilities. It must be possible because there's an exact mapping between a set of probabilities to a point in the classified hyper-plan.

Thanks a lot!
Amir


text:
Improving Recall of a Certain Class
Let's say that we have a test data set with $20,000$ observations for which we want to make a binary prediction for. When we apply our best trained model to this data set (e.g. logistic regression with threshold = 0.5, data_size = 4000 rows, 5 fold cv), only about $1 \%$ of the predictions are positive. That is, $p(\text{positive}) \geq 0.5$ is true only for about $1 \%$ of the predictions. We expect many more positives since the recall of the positive class of the best trained model is about $40 \%$. If we manually lower the threshold to $0.45$, then about $10 \%$ of the predictions are positive. Assume that the $20,000$ observations come from the same distribution as the training/validation data and are independent samples.
Questions.

Why would a a model with decent recall for the positive class predict very few positives in the out of sample data?
If (1) is true, then is it appropriate to lower the threshold for positive (e.g. $0.5$ to $0.45$) to increase the number of predicted positives in the test set?


text:
Handwritten digits recognition during the process of writing
I know how to train a NN for recognizing handwritten digits (e.g. using the MNIST database).
I'm wondering how to accomplish the same "online", which is during the process of writing e.g. I'm writing down a digit, and during the process and while keeping "the pen down", the NN recognizes the digit.
To make it simpler, I could assume one stroke-order only for each digit (e.g. each digit is written with a certain order of strokes, like for digit 1 is just a vertical line going up to down).
Which is the most suitable NN for the purpose and how to accomplish this?

Have you tried to use a neural network trained on MNIST to achieve that? You can pass the partial input the this NN and observe the output of it and how it evolves.

@nbro I haven't tried. However it would surely be sub-optimal, as a static partial input lacks "temporal" information (e.g. direction of stroke).

This is a good point!


text:
Clustering of very high dimensional data and large number of examples without losing info in dimensions
I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from $50$ genres. Each song has characteristics - some of which are common across all or most genres, some of which are common to only a few genres and some of which are genre-specific.
Common attributes could be something like song duration, artist, label, year, album, key, etc. Genre-specific attributes could be like lead guitarist, trombone player, conductor, movie name (in case of movie soundtracks), etc. Assume that there are, say, $2000$ attributes across all possible genres.
The aim is to identify attributes that characterize subgenres of these genres. So of course, let's say for rock I can just collect all the attributes for all rock songs, but even that set of attributes may be too broad to characterize the rock genre - maybe there are some that are specific to subgenres and so I won't have the desired level of granularity.
Note that for the purpose of this example, I'm not assuming that I already know the subgenres a priori. For example, I'm not going to categorize songs into subgenres like post rock, folk rock, etc. and then pick out attributes characterizing them. I want to discover subgenres on the basis of clustering, if that makes sense.
In a nutshell, about a million songs belonging to $50$ genres and all songs collectively have $2000$ attributes. So for each song I'll make a vector in $\mathbb{R}^{2000}$ - each dimension corresponds to an attribute. If that attribute is present for that song, the corresponding element of the vector is $1$, otherwise $0$ (e.g. a jazz-related attribute will be $0$ for a rock song). Now I want to do genre-wise clustering. For each genre, not only do I want to cluster songs of that genre into groups, but I also want to get an idea which attributes are the most important to characterize individual groups.
On the basis of this clustering, I can identify subgenres (e.g. of rock music) that I can characterize using a subset of the $2000$ attributes.
My first question is: is there a better way to initialize the problem than forming 2000-dimensional vectors of ones and zeros?
Secondly, given the vast number of dimensions and examples, what clustering methods could be tried? From what I've surveyed, there are graph-based clustering methods, hierarchical, density-based, spectral clustering and so on. Which of these would be best for the toy example? I've heard that one can project the points on to a lower-dimensional subspace, then do clustering. But I also want which attributes define different clusters. Since attributes are encoded in the dimensions, with dimensionality reduction techniques I'll lose information about the attributes. So now what?


text:
Get the position of an object, out of an image
I have some images with a fixed background and a single object on them which is placed, in each image, at a different position on that background. I want to find a way to extract, in an unsupervised way, the positions of that object. For example, us, as humans, would record the x and y location of the object. Of course the NN doesn't have a notion of x and y, but i would like, given an image, the NN to produce 2 numbers, that preserve as much as possible from the actual relative position of objects on the background. For example, if 3 objects are equally spaced on a straight line (in 3 of the images), I would like the 2 numbers produced by the NN for each of the 3 images to preserve this ordering, even if they won't form a straight line. They can form a weird curve, but as long as the order is correct that can be topologically transformed to the right, straight line. Can someone suggest me any paper/architecture that did something similar? Thank you!

By fixed, you mean the background is always the same image ?

@Astariul yes! and the object that changes position is also the same in each image (same size, shape, orientation etc.).

@Silviu-MarianUdrescu you mightn't need machine learning for this, it sound's like the object is very defined. If you can code something up that works 100% of the time why not do that?

@Recessive Ideally I want a NN that is able to learn the 2 numbers representation of an image, for any set of background and object moving. Coding it by hand works for only a fix background and a fix image (which is indeed what my post is about), but I want a NN approach so I can later generalize i.e. if I pass a new set of images (all with the same background and object, but different from the ones in the previous set of images) the NN would identify the "x" and "y" just as easily without any coding modifications. Coding something up manually would require a new code for each set of images.

@Silviu-MarianUdrescu There should be a few ways of doing this. You could create a regression CNN that outputs (x,y) coordinates (i wouldn't recommend this, it's very hard to get this to work from my experience). You could use deconvolutional layers to produce an output image of similar dimensions to the input image, using a softmax on the entire image to produce a probability of the image being at each location (you could also produce a downscaled version if approximate location is ok). Other then that I think there's so great resources online for object detection, just google search.

I am not sure how can I use a regression CNN at all in an unsupervised way in this case. I am not sure I understand what you mean by using deconvolutions and softmax. How can I use that to extract the x and y of the object?

@Silviu-MarianUdrescu Ahh sorry, I didn't see unsupervised. Ok well that changes things slightly. If you really want to use unsupervised learning, I'm sorry I can't help as I have no experience with that. However, this problem sounds like it's possible to complete using supervised learning. As I suggested earlier, you could perhaps hard code a program to extract the (x,y) point of the objects over a few hundred/thousand images, and using that you could then train a neural network. This should then improve robustness to future input changes.

@Silviu-MarianUdrescu As for deconvolutions perhaps ignore that, you might be able to get away with the following: Take a standard CNN, and just have the output be, say 49 nodes for a 7x7 image. This means that if node 4 has a value of 0.9, point (4,1) is 90% likely to contain the object. As for softmax, search it up online, it essentially converts the inputs to a probability distribution across all the output nodes.

As said in the comments, I wouldn't use Machine Learning for that.
You can achieve that result using something like OpenCV.
For example:

Get the "Naked" Background image: If you don't have it, you can easily calculate it by making an average of each image: background = np.mean(images, axis=0)
For each image, calculate the pixel difference between image and background. diffs = [img - background for img in images]
Diff's pixels can be negative, so take the absolute value of each pixel before converting it to grayscale.
If all goes well, you now have a dark noised image, with a bright silhouette of your object.
Set a threshold (i.e. threshold = diff.percentile(95)) and make a binary mask, so now each pixel indicates 1 for image silhouette and 0 for background.
Find the centroid of the object (like calculating the average coordinates for each pixel=1). And there you have it!

Of course, I just described one clear and easy way to do it. But you can find your own best solution.

✅ Don't need to train a neural network
✅ Don't need to label data
✅ Works for any set of image / background
✅ Precise coordinates
✅ Easy to make, debug and adapt.
✅ Runs fast


text:
What is the State-of-the-Art open source Voice Cloning tool right now?
I would like to clone a voice as precisely as possible. Lately, impressive models have been released that only need about 10 s of voice input (cf. https://github.com/CorentinJ/Real-Time-Voice-Cloning), but I would like to go beyond that and clone a voice even more precisely (with subsequent text-to-speech using that voice). It doesn't matter if I have to provide minutes or hours of voice inputs.


text:
What sort of Neural Network is best suited to predicting a future purchase?
I have previously implemented a Neural Network with Back-Propagation that was able to learn Tic-tac-toe and could go pretty well at Connect-4.
Now I'm trying to do a NN that can make a prediction. The idea is that I have a large set of customer purchase history, so people I can "target" with marketing, others I can't (maybe I just have a credit-card number but no email address to spam). I've a catalogue of products that changes on a monthly basis with daily updates to stock.
My original idea was to use the same NN that I've used before, with inputs like purchased y/n for each product and an output for each product (softmax to get a weighted prediction). But I get stuck at handling a changing catalog. I'm also not sure if I should lump everyone in together or sort of generate a NN for each person individually (but some people would have very little purchase history, so I'd need to use everyone else as the training set).
So I thought I'd need something with some ability to use the purchase data as a sequence, so purchased A, then B, then C etc. But reviewing something like LSTM, I kind of think it's still not right.
Basically, I know how to NN for a game-state sort of problem. But I don't know how to do it for this new problem.


text:
How can we conclude that an optimization algorithm is better than another one
When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 score .etc, and do the same for an old optimization algorithm, or do we need to compute the average performance,i.e.,the average value of accuracy or f1 scores for these runs, to show that it is better than the old optimization algorithm? Because when I read the papers on a new optimization algorithm, I don't know how they calculate the performance and draw the train-loss vs iters curves, because it has random effects, and for different runs we may get different performance and different curves.

Hi and welcome to AI SE. Could you please link any paper that you talk about in your question?

See here for a potential way to do it:
http://infinity77.net/global_optimization/#motivation-motivation
http://infinity77.net/global_optimization/#rules-the-rules
You basically test the two (or more) optimization algorithms against known objective functions, with several random (but repeatable) starting points and then analyze the outcome.

Thanks!So, do you mean we choose several random seed(initial value) and run once for each of them? And then do we choose the best performance among of these runs or do we choose just the average performance, and then use it to compare different optimization algorithms?

Normally you would use the average of the results. In my tests I have run multiple optimization algorithms with 100 random starting points (but the same for all algorithms) and then created a few graphics to compare performances: http://infinity77.net/global_optimization/multidimensional.html . Please don’t be caught into believing papers or website using CPU time or elapsed time as a performance measure, those are useless metrics.


text:
How is Average Recall (AR) calculated for an object detection model?
After looking around the internet (including this paper, I cannot seem to find a satisfactory explanation of the Average Recall (AR) metric. On the COCO website, it describes AR as: "the maximum recall given a fixed number of detections per image, averaged over categories and IoUs".
What does "maximum recall" mean here?
I was wondering if someone could give a reference or a high level overview of the AR calculation algorithm.
Thanks!


text:
What kind of neural network architecture is suitable for variable length block-like time series data?
I'm not sure what this type of data is called, so I will give an example of the type of data I am working with:

A city records its inflow and outflow of different types of vehicles every hour. More specifically, it records the engine size. The output would be the pollution level X hours after the recorded hourly interval.

It's worth noting that the data consists of individual vehicle engine size, so they cant be aggregated. This means the 2 input vectors (inflow and outflow) will be of variable length (different number of vehicles would be entering and lraving every hour) and I'm not sure how to handle this. I could aggregate and simply sum the number of vehicles, but I want to preserve any patterns in the data. E.g. perhaps there is a quick succession of several heavy motorbike engines, denoting a biker gang have just entered the city and are known to ride recklessly, contributing more to pollution than the sum of its parts.
Any insight is appreciated.

I have come across the same issue but in language. Where each input was a sentence, hence of different lengths.
The easier solution is to just find the longest sequence, extract its length, and 0 pad all other values to be able to get all of them to the same size, and then use any recurrent neural network architecture (Since you're dealing with a time series), where these padded values act as input. (When 0 padding, it is best if fewer of your layers have a bias value, because you want all multiplication with 0's to just give 0's as they represent a pad, having bias would mean after the weight multiplies with 0 it adds a small bias value, which is undesired because it is irrelevant information, One can always assume backprop will learn 0's for the biases even if present, but it never actually does, they mostly learn extremely small values, which can also hinder results)
Another thing to experiment with is creating a small network, to create an embedding for each input. The padded input can be input to a small neural network which generates a fixed size embedding (Activations of a particular layer ; Backprop will learn a decent representation when trained end-to-end ) that is the input to the recurrent neural architecture. - Adding a network to create an embedding almost always helps, as long as kept simple. Experimenting with the embedding algorithm (network), can help you find an idealistic one, which can significantly boost your results.

(this response should be a comment but I don't have yet the reputation to comment).
If I'm understanding your problem correctly you have a variable number of input which have an order and only one output ? It look like the kind of task where you could use recurrent neural network (the most common ones are the LSTM and GRU).
If you use a recurrent neural network you could (if you have the timestamps of your data) cut the hourly interval into smaller time step to help detect pattern.


text:
Puzzle solving AI?
I have a book containing lots of puzzles with instructions like:
"Find a path through all the white squares."

"Connect each black circle to a white circle with a straight line without crossing lines".

"Put the letters A to G in the grid so that no letters are repeated in any row or collumn"

I thought it might be fun to (1) Try and write a program to solve each individual puzzle (2) Write a program that can solve more general problems, and even more interesting (3) Try to write a program that parses the English instructions and then solves the problem.
I think that in general there would be common themes like, "draw a path", "connect the dots", "place the letters in the grid", and so forth.
The program would have general knowledge of thing like squares, cells, rows, collumns, colours, letters, numbers and so on.
I wondered if there is anything similar out there already?
If an AI could read instructions and solve the puzzles could we say that it is in someway intelligent?

The program would have general knowledge of thing like squares, cells, rows, collumns, colours, letters, numbers and so on.

What do you mean by "general knowledge" of this objects ? If you intend to give it similar knowledge that a human would have about them you are going to have a realy hard time.
It's a broader task but the Cyc project is attempting to assemble a comprehensive knowledge base that spans the basic concepts that every little child know and have worked on it for the last 35 years (https://en.wikipedia.org/wiki/Cyc). You could check what they have on their database about the thing that interest you but even so it will be a really hard task.
Basically what Cyc is trying to do is to create a huge dataset which contain every "common knowledge" about the world. In your case the world would be limited to just the dot, line, grid, letter, etc but it would still be a huge work to create this database.
This is the way AI community dealed with "general knowledges" for a long time. An alternative to this approach would be to use machine learning but I have no idea how it could be used for this objective (apart from using NLP to help you create the database which is what Cyc is now doing)

I wondered if there is anything similar out there already?

As far as I know there is nothing of the sort as of now. There exist AI which can learn to play multiple video game but they don't read the rule (like : "Find a path through all the white squares."), they just try things until they find the "good" way to play a game. This AI generally use reinforcement learning.

If an AI could read instructions and solve the puzzles could we say that it is in someway intelligent?

The concept of intelligence is not something on which everyone agree and even so most of the people think that there is different level of intelligence.
From my point of view this AI would have a certain level of intelligence if it had the possibility to learn a new kind of game (even if somewhat similar to the ones it already know) in just a few try (I can't give you a number but definitively not the thousands that actual method would require). Just like you presented it this ability would probably require your AI to have some level of understanding of the different common concepts between games (lines, dots, path ...).

Yes, I think the tricky thing would be to not over engineer it so it could adapt to new puzzles. But I think every time a new puzzle comes along I would probably say "Well I just need to update my program to account for this new puzzle" which is not really the idea! It should just be able to work out what to do from the instructions.


text:
RF: How to deal with environments changing state due to external factors
I have a use case where the state of the environment could change due to random events in between time steps that the agent takes actions. For example, at t1, the agent takes action a1 and is given the reward and the new state s1. Before the agent takes the next action at t2, some random events occurred in the environment that altered the state. Now when the agent takes action at t2, it's now acting on "stale information" since the state of the environment had changed. Also, the new state s2 will represent changes not only due to the agent's action, but also due to the prior random events that occurred. In the worst case, the action could possibly have become invalid for the new state that was introduced due to these random events occurred within the environment.
How do we deal with this? Does this mean that this use case is not a good one to solve with RF? If we just ignore these changing states due to the random events in the environment, how would that affect the various learning algorithms? I presume that this is not a uncommon or unique problem in real-life use cases...
Thanks!
Francis


text:
how to use Softmax action selection algorithm in atari-like game
I'm currently writing a program using keras (python 3) to play a game similar to Atari games, only in this one there are objects moving in the screen in different angles and directions (in most of Atari games I've encountered the objects you need to shoot are static). The agent's aim is to shoot them.
after executing every action I get feedback from the environment: I get the locations of all the objects on the screen, the locations of the collisions that happened, my position (angle of the turret) and the total score (from which I can calculate the reward)
I defined that each state will consists from the parameters mentioned above.
I want to use softmax algorithm in order to choose the next action, but I'm not sure how to do it. I'd be very grateful if anyone could help me or refer me to a source that can explain the syntax? currently I'm using decay epsilon-greedy algorithm.
Thank you very much for your time and attention.

What reinforcement learning algorithm are you using? Softmax isn't an algorithm its a function that scales vector components so that they all add up to 1. If you're using epsilon-greedy I assume that you're using some of the value function algorithms (e.g. DQN algorithm) so there's no need to use softmax, simply apply epsilon-greedy action choice procedure on the raw output

You would not change from $\epsilon$-greedy directly to softmax without also changing from .e.g. DQN to a policy gradient method like A3C. There is Gibbs sampling though, which is softmax plus a "temperature" hyperparameter that replaces $\epsilon$ and could be applied in a DQN agent. Is that what you are looking for?


text:
What is the best variant of darknet to use?
pjreddie's official darknet version (link from official website here) has been forked several times. In particular I've come accross AlexeyAB's fork through this tutorial. I assume the tutorial's author used AlexeyAB's fork because he wanted to use it on a Windows machine, which pjreddie's darknet cannot do AFAIK.
I am not really concerned about that (I am a linux user), but I am very interested about the half precision option (CUDNN_HALF) that AlexeyAB's darknet has, whereas pjreddie's darknet does not. Of course I've checked that this option was handled by the graphic card (RTX2080) we use at my office.
Nevertheless, I wonder: how stable/robust is that fork? Of course I want high-performing software, but I also want a certain level of stability! On the other hand, the latest commit on pjreddie's darknet is back from September 2018 (ie 1 year old), whereas AxeleyAB's darknet is active…
More broadly, there seems to be a lot of darknet forks: which ones to prefer?
What does the neural network community think?

I used Trieu's Darkflow and it was trained well up to around 30 to 50 epoch with maximum 30 GB dataset. At the same time, it was quite often crashed after these epochs.
Half precision...I don't remember Darkflow have that.
One way to estimate each fork reputation is by number of Star and Issues, for example Alexey's Darknet 5,000 and 2,500 and Darkflow 5,000, 539 respectively.
Reference: OS Ubuntu 16.04

Thanks for sharing your user experience with darkflow! :)


text:
Intelligent reflecting surface
I wanted to know about Intelligent reflecting surface (IRS) technology.
what is the application of IRS in wireless communication?
what are the competitive advantages over existing technologies?


text:
What order should I learn about Neural Networks?
I'm interested in learning about Neural Networks and implementing them. I'm particularly interested in GANs and LSTM networks.
I understand perceptrons and basic Neural Network configuration (sigmoid activation, weights, hidden layers etc). But what topics do I need to learn in order, to get to the point where I can implement GAN or LSTM.
I intend to make an implementation of each in C++ to prove to myself that I understand. I haven't got a particularly good math background, but I understand most math-things when they are explained.
For example, I understand backpropagation, but I don't really understand it. I understand how reinforced learning is used with backpropagation, but not fully how you can have things like training without datasets (like tD-backgammon). I don't quite understand CNNs, especially why you might make a particular architecture.
If for each "topic" there was a book or website or something for each it would be great.

For CNNs, use the cs231n stanford lectures on youtube - they're free. They teach back propagation very well in those lectures as well. As a recommendation, maybe try making your models in python, it will be a lot faster then c++, and the benefit of speed that c++ gives doesn't make sense in the context, as it will be very hard to implement a model as efficiently as Tensorflow is in c++. For an LSTM, I found information from this tutorial nice and concise https://medium.com/@aidangomez/let-s-do-this-f9b699de31d9, although the structure is drawn a bit differently.

Many people will try to draw the full back prop as a series of equations, like this: https://miro.medium.com/max/1920/1*PBu6tUmPMxBqxEpZHujfPA.jpeg (you need to copy paste the full link) but please keep in mind this is over-complicating a rather simple algorithm once you come to understand it. If you get the chain rule, and the basic multi-variable calculus rule that f'(a,b) = f'(a) + f'(b) at points where gradients meet, you can do it entirely visually with no need for huge long strings of equations, which is how it's mostly done in the example I gave earlier.

After understanding all of this for any model, you'll probably have the hardest time just finding the correct input and output dimensions of an architecture. A lot of articles either: don't understand the dimensions, or assume you already know them. CNNs are especially tricky, and I ended up programming one missing an entire dimension for filter depth (it still works fine, just learns a bit slower). It may take a while to learn any or all of this, but from experience it is very satisfying to finally see a model work and well worth the time.

Hey thanks @Recessive. Is there a progression that you can think of that makes sense? I'm working through http://neuralnetworksanddeeplearning.com/chap1.html#learning_with_gradient_descent but was hoping to use more than one source

Try to get to a position where you have learnt how to code from scratch a standard perceptron neural network. That tutorial looks good, although the back prop equations look a bit confusing. You only really need to know one for a sigmoid perceptron, and thats: https://wikimedia.org/api/rest_v1/media/math/render/svg/991c8f020800ec1da130849e20a3a415613e9bdb where that represents the gradient at the end of each node going backwards (aka the delta) (from https://en.wikipedia.org/wiki/Backpropagation, keep in mind, sigmoid = logistic function).

If you are able to follow this: https://en.wikipedia.org/wiki/Backpropagation#Derivation_for_a_single-layered_network you will completely understand a perceptron. Also, just as an overview of what a perceptron is doing (how it's doing it isn't explained well enough that you can do it on paper though), watch 3blue1brown's youtube series: https://www.youtube.com/watch?v=aircAruvnKk . So there's a few other sources that you should be able to work with.

I believe I've got a handle on a perceptron and how they're combined into a network. Working through back-prop now, but I think I still need to understand the prediction vs categorisation applications for back-prop (even for something simple like sigmoid). Is there may a single book I can read?

Yep, there's a book here you can read: https://www.reddit.com/r/MachineLearning/comments/5z8110/d_a_super_harsh_guide_to_machine_learning/

I think, once you are covered with the common stuff, you can probably go on and study all kinds of neural network variants.

The common stuff:
a) An undergraduate level Linear Algebra course -- covering matrix calculus. You might find this useful.
b) An undergraduate level study in statistical inference. Concepts from this topic will come up most of the time and you might have hard time getting around even though you understand the rest of the math. I would recommend this.
c) A starter book on neural networks. Ex- Neural networks by Raul Rojas.

After all these are covered you will certainly be ready for learning the variants of neural networks with ease. For LSTM I would recommend Alex Graves.

Would personally recommend deeplearning.ai's course to begin with. There may be more comprehensive or better MOOC's for covering basic MLP's, CNN, RNN's, tuning and training of neural networks but this is probably the most common one and the one that I can personally vouch for.
After this I'd recommend you get a physical or pdf copy of Deep Learning by Goodfellow et al. and use it as reference material for any new idea you'd want to learn. Personally would not recommend reading the whole book and its better as a reference material as it is quite comprehensive.
This should essentially be able to give you enough knowledge to be able to cover almost any paper/material on deep learning. The Course mentioned (most courses) would cover LSTM's as they are quite an old idea (~1997 I think) and GAN's are well covered in the book mentioned (The author invented them) since they are more of a recent advancement (2014).
Hope this was helpful!


text:
Looking or the simplest framework to train keypoint detector
I currently use an object detector to detect an object and specific parts of it (a crop and its stem). Such detector is not the best choice for detecting parts that could be represented by a point (typically a stem) so I'm planning on moving to a keypoint detector.
After reading the literature it appears that there are many solutions. I'm particularly interested in using a hourglass network to predict a set of heatmaps abstracting the keypoints positions.
The problem is that many of the existing frameworks are dedicated to human pose estimation (for instance OpenPose) but I don't need all this complexity.
By best choice for now is this framework that is a Tensorflow implementation of Hourglass Networks but it is still too specific to human pose estimation.
Do you have any suggestion of frameworks that best suit my application? i.e. simple keypoint detection.


text:
Trying to separate spiral data with neural network, learning tensorflow
I am learning how to use tensorflow without keras, just to make sure I understand tensorflow directly.
I created a spiral-looking datasets with 100 points of each class (200 total), and I created a neural network to classify it. For the life of me I can't figure out how to get a good accuracy. Do you mind looking at my code to see what I did wrong?
From what I've gleaned from various forums, it seems like if I do 4 hidden layers and 14 neurons per layer, I should be able to perfectly separate this dataset. I tried with with learning rate of 0.01, and 20k epochs.
I've tried different combinations of activation functions (tanh, sigmoid, relu, even alternating between them), but the best that I've gotten is around 60 percent, whereas people with less layers and less neurons have gotten close to 90 percent.
What I did NOT do is to create additional features (for example, r and theta), and this was intentional. I'm just curious to see if I can do this by looking at x and y alone.
The code is pasted below, and it includes the code to create the data (and it plots the data).
Thank you in advance!
import numpy as np;
import matplotlib.pyplot as plt;
import tensorflow as tf;
import random;

# Part A
# Gather data and plot
def chooseTrainingBatch(X, Y, n, k):
    indices = range(0,n)
    chosenIndices = random.choices(indices,k=k)
    batchX = X[chosenIndices, :]
    batchY = Y[chosenIndices, :]
    return (batchX, batchY)

def doLinearClassification(n, learning_rate=1, epochs=20, num_hidden_layer_1=100, num_of_layers=4, batch_size = 20):
    d = 1;
    plt.figure();
    X = np.zeros([2*n, 2]);
    Y = np.zeros([2*n, 2]);
    for t in np.arange(1,n+1,d):
        r1 = 50 + 0.2*t;
        r2 = 30 + 0.4*t;
        phi1 = -0.06*t + 3;
        phi2 = -0.08*t + 2;

        x1 = r1 * np.cos(phi1);
        y1 = r1 * np.sin(phi1);
        x2 = r2 * np.cos(phi2);
        y2 = r2 * np.sin(phi2);

        plt.scatter(x1, y1, c='b');
        plt.scatter(x2, y2, c='r');

        X[t-1,0] = x1;
        X[t-1,1] = y1;
        Y[t-1,0] = 1;

        X[n+t-1,0] = x2;
        X[n+t-1,1] = y2;
        Y[n+t-1,1] = 1;

    # declare the training data placeholders
    x = tf.placeholder(tf.float32, [None, 2])
    y = tf.placeholder(tf.float32, [None, 2])

    # Weights connecting the input to the hidden layer 1
    W1 = tf.Variable(tf.random_normal([2, num_hidden_layer_1]))
    b1 = tf.Variable(tf.random_normal([num_hidden_layer_1]))

    # activation of hidden layer 1
    hidden_1_out = tf.nn.relu(tf.add(tf.matmul(x, W1), b1))

    last_hidden_out = hidden_1_out
    for i in range(1,num_of_layers):
        # weights connecting the hidden layer i-1 to the hidden layer i
        next_W = tf.Variable(tf.random_normal([num_hidden_layer_1, num_hidden_layer_1]))
        next_b = tf.Variable(tf.random_normal([num_hidden_layer_1]))

        # activation of hidden layer
        if (i%2 == 0):
            next_hidden_out = tf.nn.tanh(tf.add(tf.matmul(last_hidden_out, next_W), next_b))
        else:
            next_hidden_out = tf.nn.tanh(tf.add(tf.matmul(last_hidden_out, next_W), next_b))

        # update for next loop
        last_hidden_out = next_hidden_out

    # and the weights connecting the last hidden layer to the output layer
    W_end = tf.Variable(tf.random_normal([num_hidden_layer_1, 2]))
    b_end = tf.Variable(tf.random_normal([2]))

    # activation of output layer
    y_ = tf.nn.sigmoid(tf.add(tf.matmul(last_hidden_out, W_end), b_end))

    # loss function
    y_clipped = tf.clip_by_value(y_, 1e-10, 0.9999999)
    cross_entropy = -tf.reduce_mean(tf.reduce_sum(y * tf.log(y_clipped) + (1 - y) * tf.log(1 - y_clipped), axis=1))

    # add an optimiser
    optimiser = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)

    # finally setup the initialisation operator
    init_op = tf.global_variables_initializer()

    # define an accuracy assessment operation
    correct_prediction = tf.equal(tf.argmax(y_, 1), tf.argmax(y, 1))
    blah = tf.add(tf.matmul(last_hidden_out, W_end), b_end)
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

    # For the purpose of separating dataset, testing data is the entire set
    testingX = X;
    testingY = Y;

    # start the session
    with tf.Session() as sess:
       # initialise the variables
       sess.run(init_op)
       for epoch in range(epochs):
            this_cost = 0
            trainingX, trainingY = chooseTrainingBatch(X, Y, n, batch_size)
            _, c = sess.run([optimiser, cross_entropy], feed_dict={x: trainingX, y: trainingY})
            this_cost += c
            print("Epoch:", (epoch + 1), "cost =", "{:.3f}".format(this_cost))

       #debugging statements
       #this_y = sess.run(y, feed_dict={x: testingX, y: testingY})
       #print("y")
       #print(this_y)
       #this_blah = sess.run(blah, feed_dict={x: testingX, y: testingY})
       #print("blah")
       #print(this_blah)
       #this_y_ = sess.run(y_, feed_dict={x: testingX, y: testingY})
       #print("y_")
       #print(this_y_)
       this_accuracy = sess.run(accuracy, feed_dict={x: testingX, y: testingY})
       print("Accuracy: ", this_accuracy)
       return this_accuracy

# this is the real thing
#doLinearClassification(100, learning_rate = 0.01, epochs=20000, num_hidden_layer_1=14, num_of_layers=4, batch_size=20);

# this is just to debug the code
doLinearClassification(20, learning_rate = 0.01, epochs=200, num_hidden_layer_1=14, num_of_layers=4, batch_size=2);
```

Most probably you are over-fitting the data. Consider decreasing the number of hidden layers and also the number of nodes in the hidden layers.


text:
Indoor positioning with variable number of distance measurements in tensorflow
Currently I have a setup where I'm determining the position of a transmitter using the RSSI of 4 receivers. Its a simple feed-forward network with some hidden layers, where the input is the RSSI values, and the output is a 2d coordinate.
Now, if I decide to add/remove receivers, I have to train the network again, since the input size changes. This is not ideal, since the receivers can move around, dissapear, etc. I have looked at some alternatives, but being pretty new to machine learning, it's difficult to pick which direction to go.
I have looked at a potential solution (stolen from another question), but I'm lost at how to implement it using tensorflow:

Any help is appreciated.

The solution mentioned seems feasible but you'd probably encounter a lot of problems, such as -

Since you're outputting coordinates, each one of the input networks must be trained differently. Considering $N$ should ideally be a variable, how many input networks do you train?
You'd use an intermediate encoding of sorts as co-ordinate representation which would be averaged and then passed through the output network. - Interpolation properties of the encoding might not be too great, meaning a linear change in the representation might not lead to a linear change in the co-ordinate, causing an averaging function to give skewed results.

Just a suggestion but I think it would be better if instead of learning multiple estimates of co-ordinate representations and then averaging them it might be better if you tried to learn some sort of distance function.
Essentially, each receiver would have a fixed position (Assumption) and the input network (or two input networks preferably) would output the distance  ($r$) of the transmitter from that receiver and the angle ($\theta$) it makes with a common axis. The problem with this approach is you wouldn't be leveraging multiple receivers as they would not be learning together, and the advantage being that whatever $N$ is, the RSSI readings pass through the same network always.
Once an $N$ number of $(r, \theta)$ values are obtained - a robust algorithm
$f(r_1, r_2,..., \theta_1, \theta_2, ...) = (x_t, y_t)$ could be found to output the transmitter co-ordinates.
I'm not too confident if it would work too well but just a suggested direction I could think of! Hope this helped!


text:
How mAP is unfair evaluation metric for Object Detection?
The following figure is from the last page in YOLOv3 paper highlighting how mAP is unfair metric for evaluating Object Detectors:

The figure shows two hypothetical Object Detector results which the author say they give the same perfect mAP, while visually the first detector is clearly more accurate than the other.
According to my understanding, the two detectors do not give the same mAP. This is how I calculate it for each detector:
Detector 1, 'Dog' class AP table:
 ______________________________________
| Object  | True? | Precision | Recall |
|_________|_______|___________|________|
| Dog_99% | Yes   |     1     |    1   |
|_________|_______|___________|________|
Hence, AP_dog = 1

Detector 1, 'Person' class AP table:
 ________________________________________
| Object    | True? | Precision | Recall |
|___________|_______|___________|________|
|Person_99% | Yes   |     1     |    1   |
|___________|_______|___________|________|
Hence, AP_person = 1
And by continuing doing so for the other 7 classes in the dataset, mAP=1.

Detector 2, 'Dog' class AP table:
 ______________________________________
| Object  | True? | Precision | Recall |
|_________|_______|___________|________|
| Dog_48% | Yes   |     1     |    1   |
|_________|_______|___________|________|
Hence, AP_dog = 1

Detector 2, 'Bird' class AP table:
 _______________________________________
| Object   | True? | Precision | Recall |
|__________|_______|___________|________|
| Bird_90% | Yes   |     1     |    1   |
| Bird_89% | No    |     0.5   |    1   |
|__________|_______|___________|________|
Hence, AP_bird = 0.75
And by continuing doing so for the other 7 classes in the dataset, mAP is less than 1 because AP for at least one class is less than one (AP_bird).

Hence, according to my understanding mAP for the first detector is 1, and for the second detector is less than 1. What is the mistake I'm doing in the calculation? Or is there some assumption in the paper that I'm not considering?


text:
How would an AI learn the concept of the words "repeat twice"?
In a hypothetical conversation:
Person A - "Repeat the word 'cat' twice".
Person B - "cat cat".

I'm thinking about how a human or AI can learn the concept of "repeat twice". In reinforcment learning it would require that after the first sentence the AI would go through every random sentence until it got it right and hence got a reward.
Another way might be the AI or human overhearing the conversation. Then on hearing a repetition or a word it may trigger some neurons in the brain related to detecting repetition. Thus by pavlovian learning associate the word "repeat" or "twice" with these neurons. When given the stimulus of the word "repeat" these neurons may get triggered making the brain do some repetition algorithm. (This is my favorite theory).
I suppose a third way might be as follows:
Person A - "Hello! Hello!"
Person B - "Stop repeating yourself".

It might learn to associate repeating with the word "repeating" in this way.
I think either way the brain must have some neurons dedicated to detecting repetitions and possibly inacting them. (I don't think any standard RNN has this capability).
What do you think is the most likely way?


text:
How can I write out the Real-TIme Recurrent Learning Gradient equations for a network?
This question is about Real-Time Recurrent Learning Gradient on a Recurrent neural network .
How can I write out the RTRL equations for a network ?
Before present an example give let's introduce some notation :
Notation

So the network for which we want to write the RTRL equations is the following :
Network

A similar question can be found here at page 561 for another network .


text:
Training and inference for highly-context-sensitive information
What is the best way to train / do inference when the context matters highly as to what the inferred result should be?
For example in the image below all people are standing upright, but because of the perspective of the camera, their location highly affects their skeletal pose.  If the 2D inferred skeleton of the person on the right were located where the middle person is in pixel space, it should not be considered upright even though it should be considered upright where it is now.
I assume the location would be fed in during both training and inference somehow, but I don't know the names of the techniques that should be used and are there any best practices when doing this type of scenario?


text:
Can't figure out what's going wrong with my dataset construction for multivariate regression
TL;DR: I can't figure out why my neural network wont give me a sensible output. I assume it's something to do with how I'm presenting the input data to it but I have no idea how to fix it.
Background:
I am using matched pairs of speech samples to generate a model which morphs one persons voice into another. There are some standard pre-processing steps which have been done and can be reversed in order to generate a new speech file.
With these I am attempting to generate a very simple neural network that translates the input vector into the output one and then reconstructs a waveform.
I understand what I'm trying to do mathematically but that's not helping me make keras/tensorflow actually do it.
Inputs:
As inputs to my model I have vectors containing Fourier Transform values from the input speech sample matched with their counterpart target vectors.
These vectors contain the FT values from each 25ms fragment of utterance are in the form $[r_1, i_1, ..., r_n, i_n]$ where $r$ is the real part of the number and $i$ is the imaginary one.
I am constructing these pairs into a dataset reshaping each input vector as I do so:
def create_dataset(filepaths):
    """
    :param filepaths: array containing the locations of the relevant files
    :return: a tensorflow dataset constructed from the source data
    """
    examples = []
    labels = []

    for item in filepaths:
        try:
            source = np.load(Path(item[0]))
            target = np.load(Path(item[1]))

            # load mapping
            with open(Path(item[2]), 'r') as f:
                l = [int(s) for s in list(f.read()) if s.isdigit()]
                it = iter(l)
                mapping = zip(it, it)

            for entry in mapping:
                x, y = entry
                ex, lab = source[x], target[y]
                ex_ph, lab_ph = np.empty(1102), np.empty(1102)

                # split the values into their real and imaginary parts and append to the appropriate array
                for i in range(0, 1102, 2):
                    idx = int(i / 2)

                    ex_ph[i] = ex[idx].real
                    ex_ph[i+1] = ex[idx].imag
                    lab_ph[i] = lab[idx].real
                    lab_ph[i+1] = lab[idx].imag

                examples.append(ex_ph.reshape(1,1102))

                # I'm not reshaping the labels based on a theory that doing so was messing with my loss function
                labels.append(lab_ph)

        except FileNotFoundError as e:
            print(e)

    return tf.data.Dataset.from_tensor_slices((examples, labels))

This is then being passed to the neural network:
def train(training_set, validation_set, test_set, filename):
    model = tf.keras.Sequential([tf.keras.layers.Input(shape=(1102,)),
                                 tf.keras.layers.Dense(551, activation='relu'),
                                 tf.keras.layers.Dense(1102)])

    model.compile(loss="mean_squared_error", optimizer="sgd")

    model.fit(training_set, epochs=1, validation_data=validation_set)

    model.evaluate(test_set)
    model.save(f'../data/models/{filename}.h5')
    print(model.summary())

and I get out... crackling. Every time, no matter how much data I throw at it. I assume I'm doing something obviously and horribly wrong with the way I'm setting this up.


text:
What is the input for the prior model of VQ-VAE?
I'm trying to implement the VQ-VAE model. In there, a continuous variable $x$ is encoded in an array $z$ of discrete latent variables $z_i$ that are mapped each to an embedding vector $e_i$. These vectors can be used to generate an $\hat{x}$ that approximates $x$.
In order to obtain a reasonable generative model $p_\theta(x)=\int p_\theta(x|z)p(z)$, one needs to learn the prior distribution of the code $z$. However, it is not clear in this paper, or its second version, what should be the input of the network that learns the prior. Is it $z=[z_i]$ or $e=[e_i]$? The paper seems to indicate that it is $z$, but if that's the case, I don't understand how I should encode $z$ properly. For example, a sample of $z$ might be an $n\times n$ matrix with discrete values between $0$ and $511$. It is not reasonable to me to use a one-hot encoding, nor to simply use the discrete numbers as if they were continuous, given that there is no defined order for them. On the other hand, using $e$ doesn't have this problem since it represents a matrix with continuous entries, but then the required network would be much bigger.
So, what should be the input for the prior model? $z$ or $e$? If it is $z$, how should I represent it? If it is $e$, how should I implement the network?

Note: This answer just uses images and pixels as an analogy for simplicity. It's not formal.
Some notes about VQ-VAE:

In the paper, they used PixelCNN to learn the prior.
Remember, PixelCNN is a good choice for images of discrete values (e.g. 256 value, 3 channels).
The discrete latent variables are just the indices of the
embedding vectors. For example, you can put your embedding vectors
in an array.

For a single input, the number of the output channels of the encoder before quantization equals the dimensionality of the embedding vectors. For an analogy, the output of the encoder is like an image but with a number of channels that equals the dimensions in the embedding space. With that, each pixel is a vector in the space of the embedding vectors. The quantization is done on each pixel by mapping each pixel to the nearest embedding vector $e$.
Now, each pixel can be represented by a single integer number which equals the index of its nearest embedding (what you call the discrete latent variables). Thus, you have two representations for the quantized output, one is a large tensor with many channels (embedding vectors), and another is a simple "discrete image" with one channel (discrete latent variables).
The one-channel "discrete images" are used to train PixelCNN. This is great because they are small-size images. while the large tensor with the embedding vectors is used for the decoder as it holds the information needed for reconstruction.
So, the discrete latent variables are used later to learn the prior and the embedding vectors are used as inputs for the decoder.


text:
What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?
I try to solve some easy functions with a neuronal network (aforge-lib):
This is how I generate the dataset:
const int GesamtAnzahl = 200;
float[,] tempData = new float[GesamtAnzahl, 2];
float minX = float.MaxValue;
float maxX = float.MinValue;

Random rnd = new Random();
var granzen = new List<int>()
{
    rnd.Next(1, GesamtAnzahl-1),
    rnd.Next(1, GesamtAnzahl-1),
    rnd.Next(1, GesamtAnzahl-1),
    rnd.Next(1, GesamtAnzahl-1),
};
granzen.Sort();

for (int i = 0; i < GesamtAnzahl; i++)
{

    var x = i;
    var y = -1;
    if ((i > granzen[0] && i < granzen[1]) ||
        (i > granzen[2] && i < granzen[3]))
    {
        y = 1;
    }
    tempData[i, 0] = x;
    tempData[i, 1] = y;
}

So this is quite easy: The output is 1 if the input is between the 2 lower random generated "borders" or between the 2 higher numbers. Otherwise the output is 1.
The input values are standardices to fit between -1 and 1. So 0 is -1 and 200 is 1.
As a network I used a BackPropagationLearning with a BipolarSigmoidFunction and several configurations like:
Learning Rate: 0,1
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons

Learning Rate: 0,1
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons
Hidden Layer 3: 2 neurons

Learning Rate: 0,2
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons
Hidden Layer 3: 2 neurons

and so on. None of them worked. As described here: https://towardsdatascience.com/beginners-ask-how-many-hidden-layers-neurons-to-use-in-artificial-neural-networks-51466afa0d3e it should be enough to have 2 hidden layers. The first one with 4 neurons and the second one with 2.
The configurations which worked best were:
Learning Rate: 0,01
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 4 neurons
Hidden Layer 3: 4 neurons

Learning Rate: 0,02
Momentum: 0
Sigmoids alpha value: 2
Hidden Layer 1: 4 neurons
Hidden Layer 2: 2 neurons

This solves the problem about 50 % of the times.
As this is a quite simple problem I wonder if I am doing something wrong. I think there has to be a configuration which has better results.
What is the best configuration for this problem and why?
Additionally I tried:

Having more data does not help. I created 5000 a dataset of 5000 points ( GesamtAnzahl = 5000). Then the networks have a even worse sucess rate.
I tried to add an extra constant input (always 1) to the dataset but this also lowered the sucess rate

I ran a lot of randomly created networks to solve this problem, but none of the structures where able to reliably "solve" this problem.
Of course, some of them where able to solve it one time, some of them even twice but there where only one which solved it 3 times:

LearningRate: 0,510141694690167
Momentum: 0,962972165068133;
Layer/Neuron-Count: 2 (14, 9)
SigmundAlphaValue: 2;

There is no uniformity in the requirement for a working neural network. Different datasets require different network architectures for optimum performance.

But this question is not about any dataset. The dataset is described.


text:
Reinforcement learning possible with big action space?
I’m experimenting with reinforcement learning for a 2D pixel plotting task, and am running into an issue that (I think) has to do with the big action space. It goes like this:
The Agent gets two vector inputs each step.
Each describes an (n x n) 2d matrix composed of zeros and ones.
One is the (n x n) target matrix, containing a certain shape of zeros
The other is an (n x n) state matrix, containing another shape
Every step, I want my agent to pick an (x, y) coordinate:
x (picks one of n)
y (picks one of n)
This will turn a zero into one, or one into zero.
every step, if correct, I give a small reward, and it’ll get punished when incorrect.
I’m training the agent (a network with 3 layers with 256 hidden units) with PPO, and curiosity in the loss, and for a 12 x 12 matrix it works quite well, not 100% but okay. (see image). Note that the agent doesn't get enough steps here to fully delete the initial shape when the target shape is empty, that's why it doesn't fully make it. Takes about 800K steps to converge though.

But the agent starts struggling in local minima when I increase beyond 32 x 32.
This one is at 32 x 32:

Is this even scalable to bigger matrices even? I was hoping to go 3D eventually, by reaching 100x100x100 .
I do realize that i have a huge input and action space when working with such a grid.
Is something like that even possible with an RL paradigm? I’ve tried increasing the network size, and decreasing learning rate, but I’m not satisfied. Any ideas or alternative approaches to plot pixels like this?
Any input is very much appreciated!
Thanks!

this paper might be relevant for you

seems very relevant indeed. Will have a look!


text:
How does a batch normalization layer work?
I understood that we normalize to input features in order to bring them on the same scale so that weights won't be learned in arbitrary fashion and training would be faster.
Then I studied about batch-normalization and observed that we can do the normalization for outputs of the hidden layers in following way:
Step 1: normalize the output of the hidden layer in order to have zero mean and unit variance a.k.a. standard normal (i.e. subtract by mean and divide by std dev of that minibatch).
Step 2: rescale this normalized vector to a new vector with new distribution having $\beta$ mean and $\gamma$ standard deviation, where both $\beta$ and $\gamma$ are trainable.
I did not understand the purpose of the second step. Why can't we just do the first step, make the vector standard normal, and then move forward? Why do we need to rescale the input of each hidden neuron to an arbitrary distribution which is learned (through beta and gamma parameters)?

Look, this is currently a quite contentious issue. J3soons answer already links to the original paper and another one. However, most of the evidence for batch normalization is still empirical, and there is very little theoretical explanation for why it works.
There are a number of competing theories out there. Also, there are a number of papers providing explanations for how batch normalization works in certain settings. However, there is no general theorem/explanation that can explain why the technique works.
Here's one paper analyzing how it works in simplified settings. This one provides another analysis of the batch norm (you can find their appendices on Arxiv). Here's another one which is the most recent peer reviewed one I could find. To quote this last paper:

Perhaps at the core of the confusion is that BatchNorm has many
effects. It has been correlated to reducing covariate shift (Ioffe and
Szegedy, 2015), enabling higher learning rates (Bjorck et al., 2018),
improving initialization (Zhang et al., 2019), and improving
conditioning (Desjardins et al., 2015), to name a few. These entangled
effects make it difficult to properly study the technique.

Definition and Explaination
For how Batch Normalization works exactly, I'll suggest you to read the following papers:

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
How Does Batch Normalization Help Optimization?

The recent interpretation on How BN works is that it can reduce the high-order effect as mentioned in Ian Goodfellow's lecture. So it's not really about reducing the internal covariate shift.
Intuition
For how it works intuitively, you can think that we want to normalize the intermediate outputs (zero mean and unit variance) if the normalization won't remove too much useful information.
However, normalization may not be suitable for all intermediate outputs. So $\beta$ and $\gamma$ is introduced to provide additional flexibility, if normalization removes too much useful information then $\beta$ and $\gamma$ will learn to become the original mean and variance, making the BN layer an identity transformation, as if it doesn't exist.
In practice, $\beta$ and $\gamma$ won't become the original mean and variance, since all intermediate outputs can be normalized in some certain way without losing too much useful information. So you can think of it to be a customized normalization for each BN layer.
tl;dr
BN layer normalize the intermediate outputs in default, however, if the neural network find out that these intermediate outputs should not be normalized, then the neural network undos or provide more flexibility to the normalization.

What is the "high-order effect"?

@nbro You can see Ian Goodfellow's lecture or this post. It's mainly about the high-order interactions.

It's funny because the post doesn't actually explain how high order terms interact with normalized vs not. Only saying that it does.

Andrej Karpathy did a video explaining initialization of neural networks.
To summarize, activations like tanh and relu cause too many vanishing gradients if the distribution of inputs has too larger standard deviation (or the mean negative for relu).
Putting batch norm (without β and γ) before your tanh will prevent it becoming saturated (i.e. prevent vanishing gradients) as less input values will be on the flat tails (i.e. $<-3$ or $>3$).
However, adding in β and γ will allow the neural network to control the saturation optimally of the tanh.
Similarity, zero mean input to relu will mean half of the inputs are saturated (and have vanishing gradient). A positive β parameter will stop this from happening.
So you can think of the step 2 as optimizing the saturation of the activation function.
Of course you can put the batch norm after the activation function, and perhaps the second step wouldn't be as needed here.

I thought it is weird to put BN between the model weights and its activation.  IMO the order should be BN -> weights -> activation. so beta and gamma are absorbed to the layer's weights and don't show up as separate parameters to be optimized. I'm sure this has caused lots of discussion here and elsewhere.

@NikoNyrh yes I probably should have said beta and gamma are completely useless before a linear layer. Although I'm not sure about conv layers


text:
Flattened vector observation or convolutional neural network input?
This is more of a general question of how to model/preprocess 'visual' state-observations to an Agent in Reinforcement Learning that I'll illustrate with an example.
Say you have a reinforcement learning problem where the agent has to draw pixels in an n * n 2D state-matrix of 0's and 1's. Say n = 100. The agent can move one step (up, down, left, right) and on its location can additionally switch 0's into 1's or the other way around.
Each step, it needs to take action so that the state-matrix resembles an n * n target-matrix (that has a certain shape). It is rewarded accordingly each step.
The agent will know its location from an x and y position that are given in addition to the state- and target-matrix each step.
Now I'm curious to the question what the best way is to represent the state to the agent. Using a visual 'prior', or not. Here's two ways:

Based on that you want to give only the essential information to the agent: The agent is presented with a matrix (with target subtracted from state), that will be flattened into one array of n^2. Additionally it'll know its current location as an additional (x, y) vector observation.
Based on that (1) would be more difficult to solve for a human, because you'll have to learn from a flattened array how different points are connected (think about how hard a flattened game of chess would be), you can also use a convolutional neural network to encode the current scene. In this case the agent will be e.g. a red dot. Given that it's such a visual task, it seems to me that using this would give the agent a better model of how the environment works, since the spatial relations are kept intact. Also it feels that keeping the 2D shape intact with a CNN would mean that it'd form better representations that generalize to other shapes, but I can't really say why.

On the other hand one could say that it's arrogant to assume that our 'human' spatial way of interpreting visual information is the best way for this case. Maybe there's a mathematical solution?
Any ideas?


text:
N-tuple based tic tac toe diverges in temporal difference learning
I have n-tuple based tic tac toe. I already have perfect minimax player and perfectly trained table-based player. My n-tuple network consists of 8 different rows of 3 of the board as triplets having possible empty, X or O, and one bit defining who's move is now, so totally 2 * 3^3 = 54 states in tuple. I train and update weights with the idea of the pseudo code from "Learning to Play Othello with N-Tuple Systems" by Simon Lucas:
public void inGameUpdate(double[] prev, double[] next) {
  double op = tanh(net.forward(prev));
  double tg = tanh(net.forward(next));
  double delta = alpha * (tg - op) * (1 - op * op);
  net.updateWeights(prev, delta);
}

public void terminalUpdate(double[] prev, double tg) {
  double op = tanh(net.forward(prev));
  double delta = alpha * (tg - op) * (1 - op * op);
  net.updateWeights(prev, delta);
}

And the score is the sum of weights of those rows of 3. The temporal difference training generally works for n-tuple based tic tac toe and after several thousands games it mostly plays perfectly. But after a while it diverges from the perfection and oscillates between perfect and near perfect. I realized it was in situations like this:
OXO
X-O
-XX

I suspect because row that prevent opponent from winning has big value. And having two of such rows seems to be better than losing later.
I know I can have perfect player basing on this particular n-tuple network. I could just stop training after I reach perfection, but in bigger games I can't do that. I fiddled with different alpha in ranges 0.1-0.0001 and e-greedy epsilon 1%-50%, or adaptive. Increasing epsilon to about 50% somewhat mitigates this effect, but this value is mostly very big to use in other games.
Here are couple questions:

Does this effect have a name in the machine learning world? It values preventing opponent from winning. But if opponent has more opportunities to win, its value will be bigger, so that it will exceed the (negative of) losing value.
Aside from probably using different n-tuple networks and tweaking hyper-parameters, what can I do to mitigate or eliminate this effect?
In bigger games, this learning and n-tuple system give fairly good result, but I see big oscillations after certain points. I.e. in breakthrough game against 1-ply minimax, after it reaches about 60% winrate (testing 10000 games after training every 10000 games against itself), its winrate goes slitghly up, but in testing its winrate oscillates between 45-65%. Can this effect be caused by the problem I mentioned in 1.?


text:
Find the nearest object in a image which is captured from camera?
Objective :  To find the nearest object (closer distance object)  in the single camera image. But Image Contains multiple objects shown below:

I searched in the net and found this formula to calculate the distance of object from camera
F = (P x  D) / W Further Detail click on the Link
Is there any other better approach to find the nearest object in a image?
Thanks in Advance!!!

What components are you able to work with, and how much time/resources can you spend on this problem? If you are hoping to use an existing pre-trained YOLO model, and don't have any training data, just want to guess at nearest object, then something like your link is going to be best you can do. If you have time and ability to train a new model, you may be able to do better, but it could take a lot of effort to do so.

Also, do you have a specific use case that limits what types of object and their possible locations? If your camera is above ground level, with a horizontal horizon plus reasonable to assume all detected objects are on the ground (not the case in the image shown, but also not unusual assumption), then you could just rank objects by lowest point of bounding box. Other simplifications may apply in your case.


text:
Improving graph decoder network
I have been using a network to generate graphs. The architecture that I have been using is the following:

In this figure, $D_1$ is the signal generator and $D_2$ is the graph topology generator, which is a square, symmetric matrix which indicates which node is connected to which. In this network, $l$ shows linear layers, $a$ shows activation functions. Here we are using leaky relu activation function.
The problem that I am experiencing is that after training the network, my output is only a chain of nodes, meaning that only subdiagonal and superdiagonal elements have non-zero values and it is very rare to have other forms of graph. I was wondering if anyone has a suggestion for improving the output. Note that my training data is diverse and has every kind of graphs.


text:
Can an image recognition model used for human pose estimation?
I am currently writing my thesis about human pose estimation and wanted to use Google's inception network, modify it for my needs and use transfer learning to detect human key joints. I wanted to ask if that could be done in that way?
Assuming I am having n-keypoints, generating the n-feature maps, use transfer learning and cut off the final classification layers and replace it by a FCN which guesses the key joints. I am asking myself if this might be possible.
However, these feature maps should output heatmaps with the highest probability as well. Is this assumption valid?


text:
How to deal with nonstationary rewards in asymmetric self-play reinforcement learning?
Suppose we're training two agents to play an asymmetric game from scratch using self play (like Zerg vs. Protoss in Starcraft). During training one of the agents can become stronger (discover a good broad strategy for example) and start winning most of the time, which causes big portion of the state values (or Q(s,a) values) become very high for this agent and low for another, just because the first is generally stronger and receives most of the rewards. Some training time later the other one finds a weakness in the first's play (in many states too) and starts dominating and the reward stream shift the other way.
The problem is, we have to retrain function approximator (deep neural net) to wildly different value/Q states, this slows and destabilizes learning. For each of the agents this is similar to highly nonstationary environment (the opponent), that can be harsh or easy at times.
What do people usually do in such a case? I think what is needed is some kind of slowly changing baseline (similar to advantage in A2C), but applied to the reward values themselves.


text:
Categorizing text into dynamic amount of categories
I'm looking for a supervized system/approach, that could learn how to categorize incoming texts/documents, where new categories can be added over time and the training set will be small. The trained model should not be static and should be able to evolve with adding new categories or evaluating new documents.
For each document it should first give it's suggestion that can be then corrected.

Transfer Learning allows you to add new categories to be predicted to the output layer without needing to re-train the entire model every time a new category needs to be classified. Rather, the weights of all initial layers up to the last few layers of the network can be frozen and only the last one or two layers can be made trainable for fine-tuning the classification rather than training from scratch. This approach would be relatively fast than other methods since the dataset in your case is small.
In order for model to flag the need of a new output class by inferring on a test document, you could include a class "Unknown" in addition to existing N classes (hence, the output layer now contains N+1 classes). If the model predicts "Unknown" with highest probability, you could add a new class to the output layer after examining the test data.


text:
Is it possible to combine multiple SVMs that were trained on sublayers of a CNN into one combined SVM?
I have created a CNN for use on the MNIST dataset for now (so I have 10 classes). I have trained SVMs on the sublayers of this trained CNN and wish to combine them into a combined SVM as to give a combined score.
So far, I trained two individual SVMs at two of the sublayers of my neural network.
What is the best method I can go about combining the two SVMs and what are the different options available to me? Is it simply a case of taking the maximum/average of each SVM prediction for a class and using that as the score for the combined SVM class prediction?
Thanks

You can google "deep svm", there are several approaches in papers


text:
How to understand my CNN's training results?
I created a multi-label classification CNN to classify chest X-ray images into zero or more possible lung diseases. I've been doing some configuration tests on it and analyzing its results and I'm having a hard time understanding some things about it.
First of all, these are the graphs that I got for different configurations:
Results of CNN with different configurations
Note 1: I've only changed the dataset size and the number of color channels in each configuration
Note 2: In case you're wondering why I tested the network with both 1 and 3 color channels, it's because the images are technically grayscale, but I am using the AlexNet architecture, which was made to take as input 224 x 224 images with 3 channels, so I wanted to see if the network somehow performed better with 3 channels instead of just the one
These are the things about it I don't understand:

Why does the sensitivity and specificity of the network vary so much between different epochs?
Is it normal for the validation loss of the network barely ever change as the number of epochs increase?
Looking at the results I got, it looks like 2 epochs is where there tends to be the best results. Does that make sense? I've heard of people training their networks with dozens of epochs sometimes.
Why is it that, many times, when the sensitivity of the network increases between epochs, the specificity tends to decrease, and vice-versa?

Sorry if some of these questions are dumb, I'm still a newbie at this. Also, my total dataset is drastically larger than what I present in these results (~110,000 images). I just haven't done tests with more images due to the time the network takes to train.
Network Architecture:

Base Architecture: AlexNet
Loss Function: Sigmoid Cross-Entropy Loss
Optimizer: Adam Optimization Algorithm with learning rate of 0.001

EDIT: I forgot to mention that the number of diseases to predict is 15, and that the network sees 0's much more than 1's due to the imbalance of classes. I've considered changing the loss function to a weighted version of sigmoid cross-entropy because of that, but I'm not sure if that would help the network much.


text:
Ambulance dataset needed
Could I get a dataset that can classify ambulances?
I have searched everywhere, but, couldn't seem to get hold of a set of annotated images for ambulances.

Answering my own question here.
Looked at the Open Image Dataset by Google @ https://storage.googleapis.com/openimages/web/index.html
They provide image-level labels, object bounding boxes, object segmentation masks, and visual relationships.


text:
Hyperparameter optimisation over entire range or shorter range of training episodes in Deep Reinforcement Learning
I am optimising hyperparameters for my deep reinforcement learning project (using PPO2, DQN and A2C) and was wondering:
Should I find the optimum hyperparameters to get maximum reward from training over my entire range of training (e.g. 50 million steps) or can I optimise over less time (e.g. 1 million steps)?
What is the conventional approach and why?

There is no conventional approach. It's a happy event if RL method converge at all, hyperparameter optimization is a luxury few can afford. If computational resources permit better to do it over a longer period, RL methods have notoriously high variance.