text:
How to obtain SHAP values
I want to obtain SHAP values with kernel SHAP without using python but I don't really understand the algorithm. If I have a kNN classifier, do I have to run the classifier for all the coalitions possible? For $n$ variables, $2^n$ predictions?
Also, how do I obtain the SHAP values after that? Linear regression?

Hi and welcome to AI SE! What is a SHAP value? Are you referring to values produced by https://github.com/slundberg/shap? You should contextualize a little bit your problem.

thank you! yes, I am referring to that. I need to do it in MATLAB but I don't understandit quite well how to obtain those values


text:
Why care about the value of the action which I'm not gonna take in policy iteration?
In this article, there is an explanation (with an example) of how policy iteration works.
It seems that, if we replace all the probabilities of moves in the example by new probabilities where the best action is taken 100% of the time and the other moves are taken 0% of the time, then the final policy will end up to be (south, south, north) as in the example provided. However, if we are certain of our moves, we can go north and then south in the example to get most of the reward.
In other words, it seems to be incorrect to calculate the value of a state by summing up rewards for all possible actions out of the state, because, like in the case I described above, or a case where one action gives you a huge penalty, you are 100% gonna avoid it, therefore the value state is unfairly weighted.
Why care about the value of the action which I'm not gonna take?

If you knew that you should take an action $100 %$ of the time, then, of course, you should always take it, but this is rarely the case. The transition probabilities should reflect how likely you should take the actions, but it's possible that whoever wrote the MDP didn't do it right. To give you a more concrete answer, I would read that article, but I can't unless I sign up/in with Google or Facebook, which I won't do.

@nbro Weird, I can open it without registration. Maybe there's some pop up you just have to close? Anyways, even if it's not 100%. Check this image: https://i.imgur.com/5hzZ5k7.png. If I go up and lets say upper path has 90% probability to go +1 when I choose to and 90% go -3 if I choose to and 10% the latter and lower path splits on 50% (no matter what I choose), then lower path will seem better even though is clearly not because for the upper path I will sum both moves 90/10 +1 and 90/10 -3 even though one would never choose to willingly play move where 90% chance is to end up in -3.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How to determine when the image is steady enough in a video sequence to take photos?
How do I calculate the points in a video sequence where the images are steady enough for a photo.  For example, I want to take maybe 20 photos for a facial recognition dataset.  Instead of asking the subject to hold still 20 times, I take a moving video of him and filter the images for 20 good photos.  The problem is that the subject face has to move into different poses and facial expressions and many times the movements blur the quality of the image.
Thanks in advance.

If you have a video you could filter based on the optical flow between images. There are also "blurness" metrics, such as the variation of the Laplacian, that you can use to determine the "amount of blur" an image has.

Thanks. What I ended up doing was to go in straight with the face detector and filtering it for highest confidence levels.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to prevent image recognition of my dataset with neural networks and make it hard to train them?
Suppose I have a private set of images containing some objects.
How do i

Make it very hard for the neural networks such as ImageNet to recognize these objects, while allowing humans to do it at the same time?

Suppose I label these private images - a picture of a cat with a label "cat" - how do I make it hard for the attacker to train his neural network on my labels? Is it possible to somehow fool a neural network so that they couldn't easily train it to recognize it?

Like random transforms etc, so that they couldn't use a neural network to recognize these objects, or even train it on my dataset if they had labels.

Just curious, but what's the purpose of this set of images? Are you trying to make your own set of CAPTCHA images (or whatever the image versions of the anti-robot tests are called)?

If the model is trained and held constant, then there are so-called adversarial attacks to modify images such that the model classifies them incorrectly (see Attacking Machine Learning
with Adversarial Examples).
However, if you want to make images that are untrainable, you are probably out of luck. Deep neural networks can learn to recognize even random images with random labels (see Understanding deep learning requires rethinking generalization), though if there's no rhyme or reason to the randomness, they won't generalize in meaningful ways.


text:
Is such a captcha AI-resistant?
Let's say we have a captcha system that consists of a greyscale picture (of a part of a street or something akin to re-captcha), divided into 9 blocks, with 2 missing pieces.
You need to choose the appropriate missing pieces from over 15 possibilities to complete the picture.
The puzzle pieces have their edges processed with glitch treatment as well as they have additional morphs such as heavy jpeg compression, random affine transform, and blurred edges.
Every challenge picture is unique - pulled from a dataset of over 3 million images.
Is it possible for the neural network to reliably (above 50%) predict the missing pieces? Sometimes these are taken out of context and require human logic to estimate the correct piece.
The chance of selecting two answers in correct order is 1/15*1/14.

You're thinking of making the task visually more challenging, which ultimately isn't going to help much. I think captcha uses other variables like mouse tracking to discern human movement from an AI-based decision.

Related: https://stackoverflow.com/a/27299487/8770170

Well to give you a short answer, I would say that YES, it would be MORE resistant than a more standard captcha approach...
This being said, I would still go as far as to predict something like a 75-80% successful prediction rates, for a custom model which is designed specifically for defeating a mechanism such as what you describe. The reason why I am fairly confident in such an appraisal, is primarily because of the following:

New techniques which researchers have begun to explore, which are intended to be "Structure Preserving Convolutions" which utilize a higher dimensional filter to store the extra correlation data.

I think that the obfuscation efforts that you mention will definitely help to some degree, although they can be easily defeated by training the model on a dataset which you pull out some portion of the samples during pre-processing and inject the same sort of noise and glitch treatments, etc.

An idea that would be worth exploring would be to process your dataset with an adversarial model which you could then use to generate Adversarial Noise that could then be fed into a pre-process step for your images and replace (or extend) the obfuscation efforts!

TL;DR: If you cant beat 'em, then join 'em! Just train a model to defeat your captcha implementation, and then use the model to generate adversarial examples and then apply obfuscations to your data set accordingly!
For more information on what I am talking about in my suggestion for further obfuscation efforts explore some of the papers you can find on Google Scholar - Ensemble Adversarial Training Examples

The problem with use of adversarial approach here is that you have no measure of whether an example is still easily solved by humans. Without a suitable loss function, you would need to sample test large numbers of outputs with humans, looking for high percent human solvers (and it will need to be very high to be a useful CAPTCHA), and low percent machine solvers.

yeah but what about universal adversarial perturbations?

I find that to be a very weak argument against my answer. If you would investigate the literature on adversarial training, you would find that with most adversarial inputs the image is not even visually altered from a human point of view, although completely defeats the ML models attempting to process them. We are talking about slight changes in pixel color values, tiny perturbations, etc.

for being so arrogant, his attack ideas are mediocre at the very best, we have been aware of this kind of attack and introduced safety measures against it. However, we would still like to introduce these perturbations, do you think an Uap will do the job?

very specific claims and numbers are cited without any justification. it is up to you to justify your claims, of which no part of your answer provides asides from a few name drops and no explanation as to why these techniques might suffice towards this goal.

also your idea of training an adversarial network, though laudable in theory, is ignorant towards the simple fact that adversarial networks training doesn't very often converge towards a stable solution.

well reCaptcha uses adversarial examples.

no arguments on your part bro. Also, you are a neuroscientist not an AI professional

@EricJohn reCaptcha uses adversarial examples because..... its object categorization. what is your adversarial example training against?

This is not resistant at all. A simple comparison on the similarity of edge pixels between borders should be very sufficient to break this method completely.
We can do a very simple calculation. Assume the picture is 8bit black and white with each border being 50x50 pixels. Also assume the distribution is continuously uniform between 0-255 (should probably be normally distributed, but whatever). You have a total of 200 pixels that border each other in between borders. Assume that the naturally generated image  is continuous in brightness with respect to dimension in at least 10% of the image with +/- 10 units of brightness is acceptable. Thus we have 20 pixels to work with.
In the case where the image is incorrect, we assume the pixel brightness to be iid in [0-255], thus giving us roughly a 8% (21/256) chance of each pixel around the border to be of acceptable similarity. Which gives us about a 10^-22 chance of this algorithm being fooled. You might disagree with my assumed parameters, but to be frank I am probably being too generous in estimating a lower bound.
There are two lessons here: 1. Just because you and others can't think of a way to break your secure system doesn't mean it's actually secure. 2. Modern ML techniques are not strictly stronger than handcrafted algorithms, though I would also imagine that a simple NN would be able to solve this problem easily.

The edges are processef with glitch treatment. That alone is enough to make tjis attack unfeasible. we ar every well aware of this method.

The decoys are chosen on the basis of similarity of the lightness and the similarity of the pixels around the edges.

too much noise and no human subject will be able to recognize the object. even if we increase the threshold of noise to something ridiculous like 20% we are still talking about a false positive rate of 10^-14. 2. the identical calculation will demonstrate that the chance of finding a natural scene with such parameters will be something similar. did you even do any math before you make a claim such as "That alone is enough to make tjis attack unfeasible"? there is a good reason why this method is not used even historically (because it doesn't work)

if you don't believe me, go ahead and run your systems with this method of security

https://ieeexplore.ieee.org/abstract/document/5692499/

your method works only in theory. You havent even read what I said and you keep repearing yourself. Your attack was patched 3 months ago.

And you don't even know what kind of noise i am talking about.

@EricJohn no. it is you who didn't specify the type of noise to be used. what else should i assume asides from gaussian noise on the pixel strength? just look at figure 5: it should be obvious that adding some fuzzing to your boundary search algorithm to look for more adjacent pixels than just the direct border would be very sufficient to solve this problem. just because this paper was featured in a conference doesn't mean it is correct, this should be obvious.

there is a very good reason why text distortion was used in early captcha: because text segmentation is/was /very/ hard and the images are easy to generate.


text:
Network design to learn multiple sequences of multiple categories
For learning a single sequence, LSTM only should suffice.
However, my situation is different here. I have a list of sequences to learn:

The sale volumes of 12 months, these are the sequences

And each sequence above belongs to a category.
I'm trying it out by consider [category,sequence] as a sequential sample, the loss can be reduced to 1% but it gives wrong values in inferring real data.
The second try is considering [category,sequence] as a sample of 2 inputs:

X1 = sequence
X2 = category

Feed the sequence thru' LSTM layers to get H, and then concat with X2, and feed again the pair [H,X2] thru' some dense layers, the results aren't better.
Any popular solutions (network shape, network design) for learning this kind of data: sequential data in different categories?

You can use MathJax/latex on this site, in case you didn't know it.

i closed the question, training was done correctly, bug in inference code


text:
CIFAR-10 can't get above 10% Accuracy with MobileNet, VGG16 and ResNet on Keras
I'm trying to train the most popular Models (mobileNet, VGG16, ResNet...) with the CIFAR10-dataset but the accuracy can't get above 9,9%. I want to do that with the completely model (include_top=True) and without the weights from imagenet.
I have tried increasing/decreasing dropout and learning rate and I changed the optimizers but I become always the same accuracy.
with weights='imagenet' and include_top=False I achieve an accuracy of over 90% but I want to train the model without those parameters.
Is there any solution to solve this? It is possible, that the layers of those Models are not set to be trainable?
train_generator = ImageDataGenerator(
                                    rotation_range=2,
                                    horizontal_flip=True,
                                    zoom_range=.1 )
val_generator = ImageDataGenerator(
                                    rotation_range=2,
                                    horizontal_flip=True,
                                    zoom_range=.1)

train_generator.fit(x_train)
val_generator.fit(x_val)

base_model_1 = MobileNet(include_top=True,weights=None,input_shape=(32,32,3),classes=y_train.shape[1])

batch_size= 100
epochs=50

learn_rate=.001

sgd=SGD(lr=learn_rate,momentum=.9,nesterov=False)
adam=Adam(lr=learn_rate, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)

model_1.compile(optimizer=adam,loss='sparse_categorical_crossentropy',metrics=['accuracy'])

model_1.fit_generator(train_generator.flow(x_train,y_train,batch_size=batch_size),
                      epochs=epochs,
                      steps_per_epoch=x_train.shape[0]//batch_size,
                      validation_data=val_generator.flow(x_val,y_val,batch_size=batch_size),validation_steps=250,
                      verbose=1)

Results of MobileNet:
    Epoch 1/50
350/350 [==============================] - 17s 50ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1021
Epoch 2/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1030
Epoch 3/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016
Epoch 4/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1014
Epoch 5/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1040
Epoch 6/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1009
Epoch 7/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035
Epoch 8/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1013
Epoch 9/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1029
Epoch 10/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1023
Epoch 11/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1017
Epoch 12/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1020
Epoch 13/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1020
Epoch 14/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1033
Epoch 15/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1011
Epoch 16/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016
Epoch 17/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024
Epoch 18/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024
Epoch 19/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1041
Epoch 20/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1010
Epoch 21/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022
Epoch 22/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1014
Epoch 23/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035
Epoch 24/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1032
Epoch 25/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012
Epoch 26/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1018
Epoch 27/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022
Epoch 28/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1031
Epoch 29/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022
Epoch 30/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1015
Epoch 31/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028
Epoch 32/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1015
Epoch 33/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1030
Epoch 34/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1003
Epoch 35/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1044
Epoch 36/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012
Epoch 37/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1022
Epoch 38/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1021
Epoch 39/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028
Epoch 40/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1012
Epoch 41/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1035
Epoch 42/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1009
Epoch 43/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1034
Epoch 44/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1024
Epoch 45/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016
Epoch 46/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1028
Epoch 47/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1016
Epoch 48/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1033
Epoch 49/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1018
Epoch 50/50
350/350 [==============================] - 17s 49ms/step - loss: nan - accuracy: 0.0990 - val_loss: nan - val_accuracy: 0.1023

<tensorflow.python.keras.callbacks.History at 0x7fa30b188e48>

try rescaling your inputs (between 0 and 1)

@SajanGohil thanks for your answer but I don't know what do you exactly mean, how can I do that?

Your loss is NaN : your model is not learning anything. So your accuracy is the same as random choice. You first need to fix your loss.

add rescale parameter in your generators or divide your inputs by 255 beforehand. I am assuming they are in uint8 format (0-255 values)

Hi @SajanGohil could you take a look here? https://ai.stackexchange.com/questions/27687/visualisation-for-mobilenet-feature-extraction


text:
Is there an upper limit to the maximum cumulative reward in a deep reinforcement learning problem?
Is there an upper limit to the maximum cumulative reward in a deep reinforcement learning problem?
For example, you want to train a DQN agent in an environment, and you want to know what the highest possible value you can get from the cumulative reward is, so you can compare this with your agents performance.

In any reinforcement learning problem, not just Deep RL, then there is an upper bound for the cumulative reward, provided that the problem is episodic and not continuing.
If the problem is episodic and the rewards are designed such that the problem has a natural ending, i.e. the episode will end regardless of how well the agent does in the environment, then then you could work it out by calculating the max possible reward in each step of the episode; however this is potentially non-trivial depending on your environment.
For an example in a trivial setting, however, imagine the problem of cartpole -- I could define the MDP to have a reward of +1 for every time step that the agent is able to balance the pole upright, and 0 when the pole falls. If I also defined that the problem terminates after 200 time steps then the upper bound on cumulative rewards for this problem would be 200.
In general, if the problem is continuing then in theory the problem goes on infinitely and so there is no upper bound, as the episode never ends -- this is partly why we use the discount factor, to ensure that $\sum_{k=0} \gamma^k R_{t+k}$ converges.

"In any deep reinforcement learning problem, not just Deep RL", you mean in any reinforcement learning problem without deep right ? Also, just because problem is episodic doesn't guarantee that sum is finite. You could set up your rewards such that agent never wants to end episode even though there is a terminal state which would make environment episodic. You would need to add discount factor to guarantee boundness.

My answer to:Is there an upper limit to the maximum cumulative reward in a deep reinforcement learning problem?
Yes but depending on the environment, if dealing with the theoretical environment, where there are infinite number of time steps.
Calculating the upper bound
In reinforcement learning (deep RL inclusive), we want to maximize the discounted cumulative reward i.e. Find the upper bound of:  $\sum_{k=0}^\infty \gamma^kR_{t+k+1}, where$ $\gamma$ $\epsilon$ $[0, 1)$
Before we find the upper bound of the series above, we need to find out if the upper bound exists i.e. whether it converges according to the environment specifications such as the reward function.
I will provide one example environment where the series converges. It is an environment that has simple rules and goes on for infinite time steps. It's reward function definition is as follows:
-> A reward of +2 for every favorable action.

-> A reward of 0 for every unfavorable action.

So, our path through the MDP that gives us the upper bound is where we only get 2's.
Let's say $\gamma$ is a constant, example $\gamma = 0.5$, note that $\gamma$ $\epsilon$ $[0, 1)$
Now, we have a geometric series which converges:
$\sum_{k=0}^\infty \gamma^kR_{t+k+1}$ = $\sum_{k=1}^\infty (1)(2\gamma^{k-1})$ = $\sum_{k=1}^\infty 2\gamma^{k-1}$ = $\frac{2}{1 - 0.5}$ = $4$
Thus the upper bound is 4.
For environments that go on for a finite number of time steps the the upper bound does exist but for certain environments, likewise for the infinite time step environments, it may be a bit difficult to calculate but not necessarily impossible, the environments I speak of are ones with complicated reward functions and environments i.e. the environments are stochastic or the reward function's possible values are dependent on the state, they always are but we can loosely say that a reward function is independent of state when all possible reward values for an environment can be given in any state, obviously with regards to the actions taken though.

Lets assume $\sup_{s,a} r(s,a)<b$. Then for continuing problems the upper bound can be obtained by
\begin{align}
\sum_{t=0}^{\infty} \gamma^{t}r(s_t,a_t)
&\le \sum_{t=0}^{\infty} \gamma^{t} \sup_{s,a}r(s,a) \nonumber \\
&=\sum_{t=0}^{\infty} \gamma^{t} b = \frac{b}{1-\gamma}.
\end{align}
We can use the same bound for episodic tasks with discounted return.
For episodic tasks without discounting ($\gamma=1$) the above sum goes to infinity. However, if we know the episode length $T$, we can use $Tb$ as an upper bound.

The last part, where you say Tb can be used as an upper bound, may not work since e.g. The reward is 2 for every favorable action during the episode and 100 is given for the action that takes your agent to the goal state is an edge case. Technically Tb will be an upper bound but won't be sufficiently accurate for use. I think the solution for continuing problems is good.

I agree. There might exist tighter upper bounds, but they require further information about the problem. $Tb$ can be used for any problem.


text:
Why can't we train neural networks in a peer-to-peer manner?
I have recently been exposed to the concept of decentralized applications,
I know that neural networks require a lot of parallel computing infra for training.
What are the technical difficulties one may face for training neural networks in a p2p manner?

I think the transfer of data will take too long compared to similar operations in existing solutions.

what would be the purpose of doing so? e.g. for data privacy, for speed, etc

Data management and bandwidth are key issues for interconnecting multiple GPUs.  These are such big issues that it is hard to think about other challenges like neural network architecture, metrics, etc.  The key to success for interconnecting multiple GPUs on a single computer is NVIDIA's NVLink:
NVLink is a wire-based communications protocol for near-range semiconductor communications developed by Nvidia that can be used for data and control code transfers in processor systems between CPUs and GPUs and solely between GPUs. NVLink specifies a point-to-point connection with data rates of 20 and 25 Gbit/s (v1.0/v2.0) per differential pair.
Compare 25 Gbit/s to a typical peer to peer connection over the web of 100Mbps.  NVLINK provides a 250x advantage assuming everything else is equal which it is not.  This means that, considering bandwidth only, a neural network which takes one day to train on a computer with two GPUs connected with NVLINK could take 250 days over the internet using two computers with the same GPU!

Does it mean the neural network training is an iterative process (every tweak I made to the network influences next training step) and thus is not possible to split the dataset into parts that could be trained independently and then merged?


text:
How can the expectation-maximization improve the classification?
I am learning the expectation-maximization algorithm from the article  Semi-Supervised Text Classification Using
EM. The algorithm is very interesting. However, the algorithm looks like doing a circular inference here.

I don't know am I understanding the description right or wrong, what I perceived is:
Step 1: train NB classifier on labeled data.
Repeat
Step 2 (E-step): use trained NB to add label to unlabeled data.
Step 3 (M-step): train NB classifier by using labeled data and unlabeled data(with tags from step 2) to get a new classifier
Until convergent.
Here is the question:
In step 2, the label is tagged by the classifier trained on the labeled data, which is the only source containing the knowledge on a correct prediction. And the step-3 (M-step) is actually updating the classifier on the labels generated from step 2. The whole process is relying on the labeled data, so how can the EM classifier improve the classification? Can someone explain it to me?

Are you essentially asking why EM actually converges to a good solution (i.e. good $\hat{\theta}$)?

Yea, sorry for my bad description.

If you search for the word "convergence" in the same paper from which you took the screenshot, you will find some information about the convergence of the EM algorithm.


text:
Best ROC threshold for classifier?
Suppose I have a neural network $N$ that produces the output probabilities $[0.3, 0.8]$. Normally, I would specify a threshold of 0.5 for the argmax of the prediction, let's say, second arg > 0.5 means that the image is attractive, and if both probabilities are lesser than 0.5 we don't have that good of a prediction.
My question is, can we plot this threshold on a ROC curve so we can figure out the best value?

When you say "can we plot this threshold" do you mean "can we plot different thresholds"?

Well the ROC as far as I understand it is plotting points based on the specificity and sensitivity that’s coming at a specific threshold. So, yes, i’m asking if we can see what threshold is the best

authors: นิรันดร์ ใจพุธ


text:
Neural Network is not learning a very simple task
I am a complete beginner in the area. I implemented my first neural network following the online book "Neural Networks and Deep Learning" by Micheal Nielsen. It works fine with classifying handwritten digits. Achieving ~9500/10000 accuracy on the test data.
I am trying to train the network to determine whether $x > 5$ where $x$ is in the interval $[0,10)$, which should be a much simpler task than classifying handwritten digits. However, no learning happens and the accuracy ever the test data stays exactly the same with every epoch. I tried different structures and different learning rates but always the same thing happened. Here is the code I wrote that uses libraries in Nielsen's book:
import networkCopy
import numpy as np
# Creating training data
x = []
y = []
for n in range(1000):
    to_add = 100*np.random.rand()
    x.append(np.array([to_add]).reshape(1,1))
    y.append(np.array([float(to_add > 50)]).reshape(1,1))
training_data = zip(x, y)
# Creating test data
tx = []
ty = []
for n in range(1000):
    to_add = 100*np.random.rand()
    tx.append(np.array([to_add]).reshape(1,1))
    ty.append(np.array([float(to_add > 50)]).reshape(1,1))
test_data = zip(tx, ty)

# Creating and training the network
net = networkCopy.Network([1, 5, 1])  # [1, 5, 1] contains the number of neurons for each layer
net.SGD(training_data, 300, 100, 5.0, test_data=test_data)
# 300 is the number of epochs, 100 is the mini batch size
#5.0 is the learning rate

The way I generated the data may not be optimal, it is an ad hoc solution to make the data in the proper form for the network. This is my first question so I apologize for any mistakes that might be in the format of the question.

I don’t understand the point of your task. If you have got it to predict the digit already then why not just add some code that takes the predicted digit and returns prediction > 5?

Yes the task is pointless, I am doing it just to build a better understanding of how neural networks behave. This is the reason I chose the task extremely simple.


text:
Finding the 'ultimate resolution' of an ANN
I want to use a neural network to predict the refractive index of a solution. My thinking is, instead of immediately training on many samples, I will first find the 'ultimate resolution' of the network given the experimental apparatus I am using. What I mean is I will make two different solutions which have refractive indices near the middle of the range of which I am interested. Then I will train the network to classify these two solutions based on reflectance measured from the solution. If it works, say with at least 95% accuracy, then I will make two different solutions in which the difference in refractive index is smaller than before. I will repeat this until the ANN classifies, say below 95%.
Will this method of finding the 'resolution' by classification extrapolate well to regression with many more training examples?

What is a "refractive index of a solution"? Are you talking about the refractive index of a material?

I am measuring the refractive index of a solution based on reflectance data. I'm confused why your confused, do you need me to clarify what a solution is?

Well, not everyone may be familiar with your terminology, but, nevertheless, could help, so it may be a good idea to explain certain terms.

Solution is a solute dissolved in a solvent. Reflectance meaning I will put the solution in the apparatus and shine light on it and measure the reflected light.


text:
Is there a way to reduce the RMSE error when training a neural network to recognise MNIST digits using ANFIS?
I wanted to build a digit recognition neural network using MATLAB ANFIS kit.
I started by using the MNIST database and I figured out it's almost impossible to classify 784 dimension data using ANFIS. So, I reduced my data dimension from 784 to 13, using an autoencoder in Python. With the new data, I had about 80 percent accuracy in classification using a sequential model. I implemented my data in MATLAB too.
Since MATLAB treats the problem as a regression problem, I had about 1.5 RMSE after 10 epoch of learning, in both grid partitioning and subtractive clustering, and also the error curve almost seems constant in the process.
Is there any way that I can have less error?

ANFIS is too shallow to learn much better.  What do you see as state of the art (SOA) results with ANFIS?  It is not a modern solution.  Why are you considering it?  This paper, "A Deep neuro-fuzzy network for image classification" (https://arxiv.org/pdf/2001.01686v1.pdf), shows that a deeper variant of ANFIS can achieve an accuracy of 99.58%.

@BrianO'Donnell I really don't want to work on it that much but I have to,it's my final course project ,is there any tool for deep ANFIS that I don't have to implement all the stuff by myself?


text:
Using transformer but masking in reverse direction/smart sampling for desired final word?
I'm trying to generate rhymes, so it would be very helpful to have a language model where I could input a final word, and have it output a sequence of words that ends with that word.
I could train my own model and reverse the direction of the mask, but I was wondering if there was any way I could use a pretrained model but apply a different mask to achieve this goal.
If this isn't possible, what's the best way to sample a forward-predicting model to achieve the highest probability sentence ending with a particular word?
Thanks!

What the particular final word depend on? Does that depend on previous sequences? Or previous words?

@pedrum It depends on the previous sentence.  It's exactly like the usual transformer case except predicting from right to left instead of left to right


text:
Is it a good idea to train a neural network to classify images without base-hypothesis?
I'm a relative beginner in deep learning (understand by that, I'm doing my first Kaggle competition right now, and I have loads to learn still) and I was just wondering something.
Let's say you have pathology/biopsy tissue images from patients dying from a disease A and patients dying from other causes (whatever causes actually but not related to disease A).
To date, I think we can say that nobody actually really knows what causes at the level of a biopsy the disease A.
My idea, as my group could have actually a lot of these biopsies for both groups, would be to use them to fuel a neural network.
Why would I do that? Biopsies images are rather complex images, and maybe some fine details are hard to guess for a human being, or maybe the sum of some details is actually important to tell whether disease A kills the patient or not. But again, I don't think anybody could come and say: on those tissue biopsy, the sign(s) for disease A are x, y, z.
My question then becomes a bit more theoretical: given the fact that you have enough data to actually give chances to the algorithm to find differences, is it a good idea to train a neural network without having actually any idea of what could differentiate the two groups?
Do you know examples of such a strategy? How hard is it afterwards - in the case of a rather good accuracy - to understand what makes it so recognisable?

It's not clear to me what you actually want to do: are you saying that you have a labelled dataset of $N$ instances where the labels are either $l_1$ (it has the disease) or $l_2$ (it does not have the disease), and your goal is to train a neural network to predict the label of an unseen image? If that's your setting, what is your question? Are you asking if you (the human) need to know what patterns in the images are associated with "diseases" in order to train the neural network? The answer is clearly no: you don't need that, but maybe you need that to understand if the net is doing well.

Thanks for answering. My question was indeed: does one need to have an hypothesis to build a neural network or can one build hypothesis based on what the neural network found (if successful)


text:
Finding whether an input column is missing
I am working on a problem similar to this one:(supervised, artificial data)
x=np.random.uniform(-100,100,10000)
y=np.random.uniform(-100,100,10000)
z=np.random.uniform(-100,100,10000)
a=np.random.uniform(-100,100,10000)
b=np.random.uniform(-100,100,10000)

i= x**2/1000 + 2*x*y/1000 + 4*np.sqrt(abs(z)) + z*a + 4*a + abs(b)**1.5 -1/((b+1)**2) * a + 10*y

Since I am not creating the data myself I want to make sure, that my customer provided all the relevant input features. Is there a way to find out, whether the input is complete and not lacking a feature, say "a"? Obviously if the input is the same and the output differs it would be evidence of missing data but it isn't guaranteed that any two input samples are the same. Another way I thought of would be to use an autoencoder to find the dimension of the dataset(including the output) and hope it is exactly the input dimension but in my case it is also possible that there are redundant features. Is there any other way to check whether a function is computable from the given inputs?

I might not have understood it well, but for handling missing data you don't need AI, a simple dimensionality check would do the trick. Also if you miss "a" then when evaluating "i" an error will be raised

Dimensionality checks aren't "simple" if your data is high dimensional and possibly not even continous. Also I'm fully aware that I would get an error but as I'm not creating the Data myself I can't check that. The problem is rather that my customer might have forgotten to include some input columns and there might also be some redundant or completely unnecessary ones. The question is essentially whether there is a way to check, whether I can actually learn the function given my data


text:
What are some solutions for dealing with time series data that are recorded at uneven intervals?
Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic intersection. It only records when something occurs, like a car passes, a pedestrian crosses, etc... But otherwise doesn't record information.
I want to produce a LTSM NN (or some other memory based NN for time series data), but since my features don't occur at even time intervals, I am not sure how having a memory would help. For example let's consider two sequences:
Scenario 1:

At 1PM, I recorded a car passing.
At 105, some people cross.
At 150 some more people cross.
At 2PM, another car passes.

Scenario 2:

At 1 PM a car passes
At 2 PM a car passes

In the first scenario, the last car passed 3 observations ago. In the second scenario, the last car passed 1 observation ago. Yet in both scenarios, the last car passed 1 hour ago. I am afraid that any model would treat the last car passing in scenario 1 as 4 time periods ago and the last car passing in scenario 2 as 1 time period ago, even though in reality, the time difference is the same. My hypothesis is that the time difference is a very important feature, probably more so than the intermediate activity between the two car passing. In other words, knowing that the last car passed 1 hour ago is equal or likely more important than knowing that there were some people crossing in the last hour. With that said, knowing that people crossed is important too so I can't just remove that feature.
Another example of my issue can be scene below:
Scenario 1

1PM Car passes
2PM Car passes

Scenario 2

1PM Car passes
10PM Car passes

Once again, in my data set, this would be treated as adjacent observations, but in reality, the time gap is vastly different and thus, the two scenarios should be viewed as very dissimilar.
What are some ways to solve these issues?

I've thought of just expanding the data set by creating a row for every possible time stamp, but I don't think this is the right choice as it would make my dataset humongous and most rows would have 0s across the board. I have observations that occur in microseconds so it would just become a very sparse dataset.
It would be nice to include time difference as a feature, but I am not sure if there's a way to include a dynamic feature in your data set. For example, in the first scenario, at 105, the 1PM observation needs a feature that says this occurred 5 minute ago. But at 150, that feature needs to be changed to this occurred 50 minutes ago, and then at 2PM, that feature needs to now say that it occurred 1 hour ago.
Would the solution to just give the NN the raw data and not worry about it? When building a NN on word prediction, I guess if you give the model enough data, it'll learn the language even if the relevant word happened 10 paragraphs ago...  However, I am not sure if there are enough examples of the exact same sequences (even with the amount of data) for it to obtain the predictability I want.

Any ideas on ways to solve this problem while keeping in mind that the goal is to build a NN?  Another way to think about it is, the time when a data point occurred relative to when the prediction will be made, in my situation, is a crucial piece of information for prediction.

I would suggest reviewing the literautre first. If you search for "irregularly sampled time series" on google scholar, you're gonna find already some preexisting solutions such as: https://arxiv.org/abs/2003.09291 and https://arxiv.org/abs/1907.03907

Thanks,  I will take a look.

Have a look through this list


text:
Which is the best RL algo for continuous states but discrete action spaces problem
I am trying to train an AI with an environment where the states are continuous but the actions are discrete, that means I can not apply DDPG or TD3.
Can someone please help to let know what should be the best algorithm for discrete action spaces and is there any version of DDPG or TD3 which can be applied to discrete action spaces on partially observable MDPs.

there are lots of options you can choose from! You might want to start with vanilla DQN and then see if you can use any variants to improve upon this. The same with actor-critic. I usually like to start with the simplest model for my problem and then adapt it until it works.

This resource gives an interactive guide on selection of RL algorithms. https://intellabs.github.io/coach/selecting_an_algorithm.html It might not give a direct answer for this particular case, but can be useful

Hi there i saw recently this question, i would like to know if you reached your goal and found a way to use TD3 also in pomdp environments and discrete action space.

Hi TD3 is not for discrete action space..I found the best solution as r2d2...you can check any good github RL dipository for this.


text:
How to modify the Actor-Critic policy gradient algorithm to perform Safe exploration in Reinforcement Learning
I am trying to implement safe exploration technique in [Ref.1]. I am using Soft Actor-Critic algorithm to teach an agent to introduce a bias between 0 and 1 to a specific state of interest in my environment.
I would like to ask for your help in order to modify the critic update equation -which is originally  based on the return or the RL cost function- from this:

which is based on the return functions:
  =

to be based on the following cost function to make the RL objective risk-sensitive and avoid large-valued actions (bias values) at the start of the agent's learning,

How can I include the second part of the cost function - in which the variance of the reward is evaluated- in the update equation?
[Ref.1] Heger, M., Consideration of risk in reinforcement learning. 11th International Machine Learning Conference (1994)

what is $c_t$ here?

It’s the return at instant t. When compared to the $R_t$ equation, $c_t $ ~ $r_{t+k+1}


text:
How to calculate the attention loss in the paper "Tell Me Where to Look: Guided Attention Inference Network"?
I have been reading the research paper Tell Me Where to Look: Guided Attention Inference Network.
In this paper, they calculate the attention loss, but I didn't understand how to calculate it. Do we have to calculate it like outcome[c]? If it is so, then why do arrows connect with each other from the middle FC and the last FC?
Here is the image:

From this figure, there are apparently 3 FC, and the second (or middle) FC does not seem to be connected to the last FC. So, can you clarify what you mean by "why do arrows connect with each other from the middle FC and the last FC?"?

FC (where the red box is turned on) layers connected by green attention mining loss.


text:
Time Series Forecasting - Recurrent Neural Networks (tensorflow)
I am attempting to forecast a time series using tensorflow with the following code:
X = mytimeseries
scaler = MinMaxScaler()
scaled = scaler.fit_transform(X)

length = len(X)-1
generator = TimeseriesGenerator(scaled,scaled,
                            length=length,batch_size=1)

model = Sequential()
model.add(LSTM(units=100,activation='relu',input_shape=(length,n_features)))
model.add(Dense(units=100))
model.add(Dense(units=1))

model.fit(generator,epochs=20)

Then I just run a loop to forecast, but it's giving me nothing more than a straight line after a few points, as observed below.
Obviously there is a trend for the data to go down, and I would expect to see that.
Is this because my architecture is not sophisticated enough / not the right one to pick up on the general decline of the known data? Have I inappropriately chosen any parameters?
I have tried increasing the number of neurons in the dense layer, units in the LSTM cell, etc. At the moment, the thing that looks like to most effect the resultant curve is to change the length parameter in my code above. But all this does is make the predictions more sinusoidal.

Thanks for your help!

Can you share some context? Time series modeling do not happen in a vacuum --- what is your goal, what does the series represent, ... ?

probably stocks XD


text:
How to train a hierarchical DQN to play the Montezuma's Revenge game?
Would anybody share the experience on how to train a hierarchical DQN to play the Montezuma's Revenge game? How should I design the reward function? How should I balance the anneal rate of the two-level?
I've been trying to train an agent to solve this game. The agent is with 6 lives. So, every time when the agent fetches the key and loses his life for the instability or power of the sub-goal network, the agent restart at the original location and simply go through the door, thus gets a huge reward. With an $\epsilon$-greedy rate 0.1, the agent is possible to choose the subgoal key network and fetch the key, so the agent always chooses the door as the subgoal.
Would anyone show me how to train this agent in the setting of one life?

When you say HDQN, are you referring to any particular hierarchical DQN algorithm? If yes, please, provide a link to the paper.

Here is the linklink. In my implementation, every sub goal is considered to be a neural network. The architecture is the smame as that of the paper.


text:
How to normalise image input to backpropogation algorithm?
I am implementing a simple backpropagation neural network for classifying images. One set of images are cars another set of images are buildings (houses). So far I have used Sobel Edge detector after converting the images from black and white. I need a way to remove the offset (in other words normalise the input) of where the car or where the house is in the image.
Will taking the discrete Fourier cosine transform remove the offset? (so the input to the neural network will be the coefficients of the discrete cosine Fourier transform). To be clear, when I mean offset I mean a pair of values (across the number of pixels, and vertically the number of pixels) determining where the car or the building is in the 2D image from the origin.

For image classification, in general, I'd recommend going for Convolutional Neural Networks. Is there a particular reason/constraint why you are not using them, but trying to hand-craft features, instead?

@DanielB. The reason I am using  simple backpropagation neural network  is because I am doing this image classification to learn to implement a  simple backpropagation neural network ; I have not programmed a neural networks before. I cannot see any reason why a  simple backpropagation neural network  would not work.


text:
How can I build a recommendation system that takes into account some constraints or the context?
I am building a recommendation system that recommends relevant articles to the user. I am doing this using simple similarity-based techniques (with the Jaccard similarity) using as features the page title, the tags, and the article content.
Now my problem is I have different "adult articles" and some are articles that expire (for example, an article about a movie in Jan 2019 would not be relevant in Dec 2019).
I want to keep these adult articles separate, as a person who is reading about history does not want to be led to an adult article and not recommend articles that have expired or would not be relevant in the present moment.
Should I just improve the quality of my features or tags? Or is there any other way to achieve this?

wouldn't it better to have two systems, one for adult and one else if you don't want them to connect? Considering the fact that you will add items to your system you could also remove th ones that are more ephemeral or find a way to use date as a distance metric between two items. Though I'm not familiar with recommendation systems.

Is there some reason you could not hardcode this??

@hisairnessag3 it would not be scalable right?

@Alrl I thought of doing that in the pre-processing step but the problem is just identifying the tags with sex etc in it does not help me get an exact match for an adult article. I also get different articles which talk about sex but are not adult articles.

@m2rik i fail to see why it wouldnt be


text:
What are some good models to use for spelling corrections?
I used OCR to extract text from an image, but there are some spelling mistakes in it :
The text is as follows :
'gaRBOMATED WATER\n\nSFMEETENED CARBONATED 6\nBSREDERTS: CARBONATED WATER,\nSUGAR. ACIOITY REGULATOR (338),\n\nCFFENE. CONTAINS PERMITTED NATURAL\nCOLOUR (1506) AMD ADDED FLAVOURS QUcTURAL,\nSATIRE: OENTICAL AND ARTIFICIAL PLIVOUREE\n\nCOLA\nl 1187.3 PIRANGUT, TAL. MULSHI,\nGBST. PUME 612111, MAHARASHTRA.\nHELPLINE: 1800- 180-2653\ntet indishetptine@cocs-cola.com\nAUTHORITY OF THE COCA-COLA\n‘COCA-COLA PLAZA, ATLANTA, GA 36313, USA\nme DATE OF MANUFACTURE. BATCH NO. &\nLP CNL. OF ae TAXES}:\nSE BOTTOM OF CAN.\n\nTST Fone Sor MOTHS FROM\nWe, RE WHEN STORED ft.\n\nY PLACE.\nChe coca conn\nnee\n\n| BRA License uo:\n‘ eS wo:\n\n \n\x0c'

I would like to know if there are some NLP models/libraries that I can use to correct spelling mistakes(like correcting gaRBOMATED to CARBONATED

Of course there are methods and libraries to identify likely correct spelling, however, their accuracy depends on the possible vocabulary and the amount of errors. If you're dealing with soft drink ingredient lists only, the vocabulary might be sufficiently small but the OCR quality seems to be pretty awful. Don't know whether such a model would correct "BSREDERTS" to "INGREDIENTS", for example. And sorry, I can't suggest a specific library as I don't have experience with that area yet.

Thanks a lot. And yes the OCR quality is not very good. I have to work on that code.

Since most blatant errors seem to be at the beginning of lines, it's also possible that the image is partially blurry (if taken from a convex shape, highly possible) and therefore pretty unreadable. In that case, better OCR won't help but better imaging might do.


text:
What is the time complexity for training a single-hidden layer auto-encoder?
What is the time complexity for training a single-hidden layer auto-encoder, for 1 epoch?
You can assume that there are $n$ training examples, $m$ features, and $k$ neurons in the hidden layer, and that we use gradient descent and back-propagation to train the auto-encoder.

I removed the "stacked" part because I don't think that's necessary. What did you mean by "stack auto-encoder", btw?

When decoder weights matrices (except bias term) are just the transpose of encoder weight matrices.

Are you sure that's the definition of stacked AE? Where did you take that definition from?

@nbro It might be this: https://medium.com/@venkatakrishna.jonnalagadda/sparse-stacked-and-variational-autoencoder-efe5bfe73b64#:~:text=A%20stacked%20autoencoder%20is%20a,of%20the%20successive%20hidden%20layer.
A stacked autoencoder is a neural network consist several layers of sparse autoencoders where output of each hidden layer is connected to the input of the successive hidden layer.
By the way, it's not a single layer stacked AE!

@OmG The OP is saying that the decoder has the same weights as the encoder, but only transposed weights. This definition doesn't seem correct, according to your definition or other definitions I've seen so far. That's why I was asking for a reference/source of that definition.

@nbro Right, I see the point.


text:
Deep learning based physics engine
Ridgid body simulation is a well known field with well established methods. It's still fairly computationally expensive to simulate things.
I am interested in approaches to training deep learning networks to predict rigid body dynamics and interactions to reduce the computational load associated with simulations.
Has this been done before and what approaches have been used?

I think such an engine will need to be LSTM based since states naturally depend on prior states

2 minute papers: https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg Does a huge amount of videos specifically on this exact topic. You'll find many many papers on his channel that discuss what you're asking.

@Recessive You may want to provide a formal answer, if you have the time, based on the information in that video/channel ;)


text:
Strategy to input and get large images in VGG neural networks
I'm using a transfert-style based deep learning approach that use VGG (neural network). The latter works well with images of small size (512x512pixels), however it provides distorted results when input images are large (size > 1500px). The author of the approach suggested to divide the input large image to portions and perform style-transfert to portion1 and then to portion2 and finally concatenate the two portions to have a final large result image, because VGG was made for small images... The problem with this method is that the resulting image will have some inconsistent regions at the level of areas where the portions were "glued". How can I correct these areas ? Is there an alternative approach to this dividing method ?
thanks

You could add more layer to each end in order to downscale and upscale the images. But you would have to retrain the network. You could try downscaling your images and then use another network or a simple method to upscale them if this is what you desire.

I would like to use the same network if possible


text:
Extract features with CNN and pass as sequence to RNN
I read an article about captioning videos and I want to use solution number 4 (extract features with a CNN, pass the sequence to a separate RNN) in my own project.
But for me, it seems really strange that in this method we use the Inception model without any retraining or something like that. Every project has different requirements and even if you use pretrained model instead of your own, you should do some training.
And I wonder how to do this? For example, I created a project where I use the network with CNN layers and then LSTM and Dense layers. And in every epoch, there is feed-forward and backpropagation through the whole network, all layers. But what if you have CNN network to extract features and LSTM network that takes sequences as inputs. How to train CNN network if there is no defined output? This network should only extract features but the network doesn't know what features. So the question is: How to train CNN to extract relevant features and then passing these features to LSTM?

The approach that you don't train the whole net, but just the latter part of it (all starting with lstm in our case), can actually work. The idea is that the inception was already pretrained a very large dataset (imagenet for instance). And it's capable of extracting some useful information from it. Actually there are different domains of images in the imagenet and the inception net needed to capture a vast variety of input information to classify images well. The idea is that the pretrained inception is already capable to extract almost everything what could possibly be useful (unless your images aren't something completely different from imagenet, but that a rare case). Then you adapt the lstm layers and the fully connected layers to correctly process that information. Maybe you aren't going to get the perfect score with this approach and maybe it's better to train the whole large net including the inception part on the new data to lower the distributional shift and that's what people usually do in fact, but it takes more time to train and if you don't have enough data you won't be able to achieve results that are significantly better than those with a frozen CNN part.

Thank you for a good reply! But this creates another questions. You say that in order to have good results I need to train the whole network, not only the lstm part. But in this case I should create the model which takes sequence of images as input (for example 5 frames) and train this big model. It requires more data and time or gpu power to do this, but it's not a big problem. But there is another problem.

It may be very slow while testing. Because you take 5 frames, you do CNN part, then LSTM part and you have the output. And then you capture new frame. And you must do the whole CNN part once again for 2,3,4,5 and 6 frame. So you do CNN calculations second time for 2,3,4 and 5 frame which is a terrible waste of time. But your model expects 5 images as input so you must do it.

But if you have separate CNN to extract features, you can extract features for last 5 frames and then pass these features to RNN. And then you do CNN part for 6th frame and you pass the features from 2,3,4,5,6 frames to RNN which is better.
The task I want to do is autonomous driving using sequences of images. I don't know if using pre-trained Inception or VGG network I will have good results, but I could try. But I am just curious how to do it from scratch. I may have to do this in the feature or maybe even in this project if pre-trained networks fail.

Should I train CNN with one image as input and steering command as output and define LSTM network with a sequence of features as input and steering command as output. So in other words, should the output of the CNN and LSTM part be defined the same way? And then you train CNN on a lot of images, and at the end you train LSTM using the middle conv/pooling layer with features as inputs?

No, it doesn't work like that. You can compute the CNN part frame-wise only once and then feed the outputs to the lstm just once. It's like you take a frame, compute output of the CNN, feed it into the lstm part and update its hidden and cell states. After that you take the second frame and do it all over again. That's the conventional way to work with sequential data.

And you can't make some parts of your pipeline slower adding some variables into training. A training step consists of one forward and one backward pass. An inference step consists of one forward step. The fact that you unfreeze some layers affects only a backward step, the forward step stays the same.

Ok, but how CNN can calculate something if it's not trained? How can I train the CNN part? Because my goal is to have two separate networks (CNN and LSTM), not one huge network with CNN part at the beginning and LSTM at the end. So in order to compute useful features that will be passed into LSTM I must train CNN first. How I should do this? How I should define output of the CNN? The same way as output of the LSTM? As a steering command?

you could also just use a Task-agnostic CNN as an encoder to get extract features like in (1) and then use the output of the last global pooling layer and then feed that as an input to the LSTM layer or any other downstream task. Add another small Neural Network (projection head) after the CNN. And then use contrastive loss on output of this projection head to improve upon the model.
(1) Big Self-Supervised Models are Strong Semi-Supervised Learners (Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton ) Link: https://arxiv.org/abs/2006.10029


text:
Are there any python implementations of GGP games or how to use game logic written in GDL in python?
I'm writing a General Game Playing (GGP) AI in python. I'd like to test it on some GGP games. So are there any python implementations of GGP games?
I found in http://games.ggp.org/base games written in Game Description Language (GDL). How to use them in python if it is possible to do so?

Hello. Welcome to AI SE! What is "GDL" and "GGP"? You should use the long version of these terms the first time you use them, so that everyone knows what the abbreviation exactly refers to. Moreover, it's not fully clear how your question is related to AI and is on-topic according to our on-topic page. Please, have a look at https://ai.stackexchange.com/help/on-topic for more details.

Sorry, I have edited the question now. I think its on-topic as it comes under developmental aspects of AI.

Your question may be more appropriate for https://gamedev.stackexchange.com/help/on-topic, but I am not sure it's on-topic there too.

I'm not sure if the specific question asking for python-based solutions is on-topic, but GGP/GDL have certainly been big AI topics and would be on-topic (it's academic game AI research stuff, probably wouldn't be suitable for gamedev @nbro). To respond to the question; most GDL people I know use Java/C++, sometimes Prolog. Python is probably way too slow to be useful for GDL. Except maybe if you first do the expensive parts in C++, and then wrap Python around it (this seems to be done in https://github.com/richemslie/galvanise_zero/tree/dev/src/ggpzero I think)

Personally I'd recommend using Ludii (https://ludii.games/) instead of GDL these days, but I'm a part of the Ludii group myself so that recommendation may not be 100% objective :)

@DennisSoemers Thanks, I'll check out ludii.


text:
Are there any examples of state-of-the-art NLP applications that are still n-gram based and use Naive Bayes?
As far as I can tell, most NLP tasks today use word embeddings and recurrent networks or transformers.
Are there any examples of state-of-the-art NLP applications that are still n-gram based and use Naive Bayes?

There exists a field in text forensics thats about identifying an author based on their writing style. Pan - https://pan.webis.de, is the most active conference about this topic. And as far as I know, each year the winner submissions tends to be handcrafted features that uses word and character n-grams amongst other handcrafted stylometric features, they are usually trained with SVMs or other simple methods. I believe the reason for this is that when the feature vectors are embeddings of the whole input, it takes time to learn style, it tends to learn topic instead, which is very misleading.

Also, have a look at this paper https://arxiv.org/pdf/2009.03116.pdf, here we evaluate different text representations on the task of sentence similarity, as you can see in Table 2, a svm with char n-grams are very competitive to state of the art if you consider the computation time and the slight drop in performance ;)

@Isbister thanks! I'll take a look


text:
Which 6-bit string would represent an optimal solution for trap-3 in the Linkage Learning Genetic Algorithm?
I am struggling to learn certain Evolutionary algorithm concepts and also relations between each of them. I am going through the Linkage Learning Genetic Algorithm (LLGA) right now and came across this question:
Which 6-bit string of LLGA would represent an optimal solution for trap-3?
Can anyone give me an answer or explain it?

What exactly is the question? Can you quote it? Because I cannot really understand this "what 6-bit string of LLGA that would represent an optimal solution for Trap-3". Also, please, put your question in the title.

I have edited your post to clarify what I think is your question. Please, make sure that this is the case. It would also be useful if you can tell us which resource you are using to learn about LLGA and what are your definitions of "trap-3".

Thank you for the edit @nbro. I am searching and reading research papers about these topics. I don't have a specific resource that I am following.

Which research papers are you reading on these topics? Context can be very useful to help the potential answerers to help you more and more specifically. So, it may be worth citing them (or provide a link to them), and maybe quote relevant parts of those papers that caused you to ask this question. So, I suggest that you edit your post to do this.


text:
Should I use additional empty category in some categorical problems?
I try to create autonomous car using keyboard data so this is a multi class classification problem. I have keys W,A,S and D. So I have four categories. My model should decide what key should be pressed based on the screenshot (or some other data, I have some ideas). I have some API that I can use to capture keyboard data and screen (while gathering data) and also to simulate keyboard events (in autonomous mode when the car is driven by neural network).
Should I create another category called for example "NOKEY"? I will use sigmoid function on each output neuron (instead of using softmax on the all neurons) to have probabilities from 0 to one for each category. But I could have very low probabilities for each neuron. And it can mean either that no key should be pressed or that the network doesn't know what to do. So maybe I should just create additional "artificial" category?
What is the standard way to deal with such situations?

It is not fully clear what your problem is. What do you mean by "create autonomous car"?  And how is this related to "keyboard data"? Are you a simulated environment?

I edited my question. My model should decide what key should be pressed based on the screenshot. I have some API that I can use capture keyboard data (while gathering data) and also to simulate keyboard events (in autonomous mode when the car is driven by neural network).

In short: yes, you must allow "do nothing" decision as a first level result.
Your system must decide the action to be taken, including "do nothing" action. This is different to low network outputs, that can be translated as "don't know what to do".
In other words, the network can result in:

"I don't know what to do now" when all results in the output have low probabilities. (Obviously, this is a bad network result, to be fixed as much as possible).
"I know I must do nothing", when "do nothing" action has high probability, greater than the others.
"I know I must do W", when "W" action has high probability, greater than the others.
...

Kind regards.

I know this is not a straight answer to your question, but I couldn't comment on your post so decided to post it (so maybe I will delete it after you received a better answer).
I think this playlist by sentdex can be handy as he goes through a lot of details to teach a neural network model that can drive cars in GTA-V by simply looking at each frame of the game. You can find the code of each step in this link.

Yes, I know this series. I was inspired by this tutorial, but I have some ideas how to improve this. How to make even better cars than in this tutorial. I will see if I will be able to do this.


text:
Variance of the Gaussian policy is not decreasing while training the agent using Soft Actor-Critic method
I've written my own version of SAC(v2) for a problem with continuous action space. While training, the losses for the value network and both q functions steadily decrease down to 0.02-0.03. The loss for my actor/agent is negative and decreases to about -0.25 (I've read that it doesn't matter whether it is negative or not, but I'm not 100% sure). Despite that, the output variance from the Gaussian policy is way too high (make all the outcomes nearly uniformly likely) and is not decreasing during training.
Does anyone know what can be a cause of that?
My implementation is mostly based on https://github.com/keiohta/tf2rl/blob/master/tf2rl/algos/sac.py, but I resigned from computing td_errors.
Here is the code (in case you need it).
import tensorflow as tf
from tensorflow.keras.layers import *
from src.anfis.anfis_layers import *
from src.model.sac_layer import *
from src.anfis.anfis_model import AnfisGD

hidden_activation = 'elu'
output_activation = 'linear'

class NetworkModel:
    def __init__(self, training):

        self.parameters_count = 2
        self.results_count = 1
        self.parameters_sets_count = [3, 4]
        self.parameters_sets_total_count = sum(self.parameters_sets_count)

        self.models = {}
        self._initialise_layers()  # initialises self.models[]

        self.training = training

        self.train()

    def _initialise_layers(self):
        # ------------
        # LAYERS & DEBUG
        # ------------

        f_states = Input(shape=(self.parameters_count,))
        f_actions = Input(shape=(self.results_count,))

        # = tf.keras.layers.Dense(10)# AnfisGD(self.parameters_sets_count)
        #f_anfis = model_anfis(densanf)#model_anfis(f_states)
        f_policy_1 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_states)
        f_policy_2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_policy_1)
        f_policy_musig = tf.keras.layers.Dense(2, activation=output_activation)(f_policy_2)
        f_policy = GaussianLayer()(f_policy_musig)

        #self.models["anfis"] = tf.keras.Model(inputs=f_states, outputs=f_anfis)
        #self.models["forward"] = tf.keras.Model(inputs=f_states, outputs=model_anfis.anfis_forward(f_states))

        self.models["actor"] = tf.keras.Model(inputs=f_states, outputs=f_policy)

        self.models["critic-q-1"] = generate_q_network([f_states, f_actions])
        self.models["critic-q-2"] = generate_q_network([f_states, f_actions])

        self.models["critic-v"] = generate_value_network(f_states)
        self.models["critic-v-t"] = generate_value_network(f_states)

        # self.models["anfis"].compile(
        #     loss=tf.losses.mean_absolute_error,
        #     optimizer=tf.keras.optimizers.SGD(
        #         clipnorm=0.5,
        #         learning_rate=1e-3),
        #     metrics=[tf.keras.metrics.RootMeanSquaredError()]
        # )
        # self.models["forward"].compile(
        #     loss=tf.losses.mean_absolute_error,
        #     optimizer=tf.keras.optimizers.SGD(
        #         clipnorm=0.5,
        #         learning_rate=1e-3),
        #     metrics=[tf.keras.metrics.RootMeanSquaredError()]
        # )
        self.models["actor"].compile(
            loss=tf.losses.mean_squared_error,
            optimizer=tf.keras.optimizers.Adam(
                learning_rate=1e-3),
            metrics=[tf.keras.metrics.RootMeanSquaredError()]
        )

    def act(self, din):
        data_input = tf.convert_to_tensor([din], dtype='float64')
        data_output = self.models["actor"](data_input)[0]
        return data_output.numpy()[0]

    def train(self):
        self.training.train(self, hybrid=False)

def mean(y_true, y_pred): #ignore y_pred
    return tf.reduce_mean(y_true)

def generate_value_network(inputs):
    # SAC Critic Value (Estimating rewards of being in state s)
    f_critic_v1 = tf.keras.layers.Dense(5, activation=hidden_activation)(inputs)
    f_critic_v2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_v1)
    f_critic_v = tf.keras.layers.Dense(1, activation=output_activation)(f_critic_v2)
    m_value = tf.keras.Model(inputs=inputs, outputs=f_critic_v)
    m_value.compile(
        loss=tf.losses.mean_squared_error,
        optimizer=tf.keras.optimizers.Adam(
            learning_rate=1e-3),
        metrics=[tf.keras.metrics.RootMeanSquaredError()]
    )
    return m_value

def generate_q_network(inputs):
    # SAC Critic Q (Estimating rewards of taking action a while in state s)
    f_critic_q_concatenate = tf.keras.layers.Concatenate()(inputs)
    f_critic_q1 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_q_concatenate)
    f_critic_q2 = tf.keras.layers.Dense(5, activation=hidden_activation)(f_critic_q1)
    f_critic_q = tf.keras.layers.Dense(1, activation=output_activation)(f_critic_q2)

    m_q = tf.keras.Model(inputs=inputs, outputs=f_critic_q)
    m_q.compile(
        loss=tf.losses.mean_squared_error,
        optimizer=tf.keras.optimizers.Adam(
            learning_rate=1e-3),
        metrics=[tf.keras.metrics.RootMeanSquaredError()]
    )
    return m_q;

from src.model.training import Training
import numpy as np
import tensorflow as tf
from src.constructs.experience_holder import ExperienceHolder

class SACTraining(Training):

    def __init__(self, environment):
        super().__init__()
        self.environment = environment
        self.models = None
        self.parameters_sets_count = None
        self.parameters_sets_total_count = 0
        self.parameters_count = 0

        self.gamma = 0.99
        self.alpha = 1.0
        self.beta = 0.003
        self.tau = 0.01

        self.experience = ExperienceHolder(capacity=10000, cells=5)  # state, action, reward, state', done

    def train(self, simulation_model, **kwargs):
        self.models = simulation_model.models
        self.parameters_count = simulation_model.parameters_count
        self.parameters_sets_count = simulation_model.parameters_sets_count
        self.parameters_sets_total_count = simulation_model.parameters_sets_total_count

        self.train_sac(
            self.models,
            epochs=300, max_steps=200, experience_batch=128, simulation=self.environment)

    def train_sac(self, models, epochs, max_steps, experience_batch, simulation):

        # deterministic random
        np.random.seed(0)

        history = []
        epoch_steps = 128
        simulation.reset()
        update_net(models['critic-v'], models['critic-v-t'], 1.0)

        for i in range(epochs):
            print("epoch: ", i)
            episode_reward = 0
            reset = False
            j = 0
            while not(j > epoch_steps and reset):
                j += 1
                reset = False
                # ---------------------------
                # Observe state s and select action according to current policy
                # ---------------------------

                # Get simulation state
                state_raw = simulation.get_normalised()
                # state_unwound = [[i for t in state for i in t]]

                state = [state_raw[0]]  # TODO
                state_tf = tf.convert_to_tensor(state)

                # Get actions distribution from current model
                # and their approx value from critic
                actions_tf, _, _ = models['actor'](state_tf)
                actions = list(actions_tf.numpy()[0])

                # ---------------------------
                # Execute action in the environment
                # ---------------------------
                reward, done = simulation.step_nominalised(actions)
                episode_reward += reward

                # ---------------------------
                # Observe next state
                # ---------------------------

                state_l_raw = simulation.get_normalised()
                state_l = [state_l_raw[0]]  # TODO

                # ---------------------------
                # Store information in replay buffer
                # ---------------------------

                self.experience.save((state, actions, reward, state_l, 1 if not done else 0))

                if done or simulation.step_counter > max_steps:
                    simulation.reset()
                    reset = True

            # ---------------------------
            # Updating network
            # ---------------------------
            if self.experience.size() > 500:  # update_counter_limit:
                exp = self.experience.replay(min(experience_batch, int(self.experience.size() * 0.8)))
                states_tf = tf.convert_to_tensor(exp[0], dtype='float64')
                actions_tf = tf.convert_to_tensor(exp[1], dtype='float64')
                rewards_tf = tf.convert_to_tensor(exp[2], dtype='float64')
                states_l_tf = tf.convert_to_tensor(exp[3], dtype='float64')
                not_dones_tf = tf.convert_to_tensor(exp[4], dtype='float64')

                with tf.GradientTape(watch_accessed_variables=True, persistent=True) as tape:

                    q_1_current = models['critic-q-1']([states_tf, actions_tf])
                    q_2_current = models['critic-q-2']([states_tf, actions_tf])
                    v_l_current = models['critic-v-t'](states_l_tf)

                    q_target = tf.stop_gradient(rewards_tf + not_dones_tf * self.gamma * v_l_current)
                    q_1_loss = tf.reduce_mean((q_target - q_1_current) ** 2)
                    q_2_loss = tf.reduce_mean((q_target - q_2_current) ** 2)

                    v_current = models['critic-v'](states_tf)
                    actions, policy_loss, sigma = models['actor'](states_tf)
                    q_1_policy = models['critic-q-1']([states_tf, actions_tf])
                    q_2_policy = models['critic-q-2']([states_tf, actions_tf])
                    q_min_policy = tf.minimum(q_1_policy, q_2_policy)

                    v_target = tf.stop_gradient(q_min_policy - self.alpha * policy_loss)
                    v_loss = tf.reduce_mean((v_target - v_current)**2)

                    a_loss = tf.reduce_mean(self.alpha * policy_loss - q_min_policy)

                backward(tape, models['critic-q-1'], q_1_loss)
                backward(tape, models['critic-q-2'], q_2_loss)
                backward(tape, models['critic-v'], v_loss)
                update_net(models['critic-v'], models['critic-v-t'], self.tau)

                backward(tape, models['actor'], a_loss)

                del tape

                print('Loss:\n\tvalue: {}\n\tq1   : {}\n\tq2   : {}\n\tactor (ascent): {}'.format(
                     tf.reduce_mean(v_loss),
                     tf.reduce_mean(q_1_loss),
                     tf.reduce_mean(q_2_loss),
                     tf.reduce_mean(a_loss) #Gradient ascent

                ))
                print('Episode Reward: {}'.format(episode_reward))
                print('Batch sigma: {}'.format(tf.reduce_mean(sigma)))

def update_net(model, target, tau):
    len_vars = len(model.trainable_variables)
    for i in range(len_vars):
        target.trainable_variables[i] = tau * model.trainable_variables[i] + (1.0 - tau) * target.trainable_variables[i]

def backward(tape, model, loss):
    grads = tape.gradient(loss, model.trainable_variables)
    model.optimizer.apply_gradients(
        zip(grads, model.trainable_variables))

from tensorflow.keras import Model
import tensorflow as tf
import tensorflow_probability as tfp

class GaussianLayer(Model):
    def __init__(self, **kwargs):
        super(GaussianLayer, self).__init__(**kwargs)

    def call(self, inputs, **kwargs):
        mu, log_sig = tf.split(inputs, num_or_size_splits=2, axis=1)

        log_sig_clip = tf.clip_by_value(log_sig, -20, 2)
        sig = tf.exp(log_sig_clip)

        distribution = tfp.distributions.Normal(mu, sig)
        output = distribution.sample()
        actions = tf.tanh(output)

        return actions, \
            distribution.log_prob(output) - \
            tf.reduce_sum(tf.math.log(1 - actions ** 2 + 1e-12), axis=1, keepdims=True), \
            tf.stop_gradient(tf.keras.backend.abs(actions - tf.tanh(mu)))

Can you post your actual code? Not sure how looking through someone else's code will really help. Also what do you mean by "I resigned from computing td_errors"? That sounds like a red flag to me, how else are you updating your critic or value function?

I've edited the question (added the code). As far as the 'td_errors' are concerned, I was refering to the code I've given a link to. There there is a function 'compute_td_error', however it is not called during training so I thought it might be related to metrics.


text:
Loss function for better class separability in multi class classification
So I am trying to enforce better separability in my deep learning model and was wondering what I can use besides cross entropy loss to do that? Could maybe using logarithm with different basis in cross entropy (i.e. using lower basis of logarithm than $e$ to gain steeper losses on small values, or bigger basis of logarithm to enforce plateaued losses). What would you suggest on doing?

What do you mean by "better class separability"? Can you explain maybe with an example? What exactly do you want to achieve?

@nbro I want to force the network to give higher softmax outputs for the class it thinks are the correct ones. i.e. to force it to give instead of for three class example 0.3,0.5,0.2 to give something like 0.2 0.8 0.1, but to avoid doing it in programming, but actually force different loss function that would force this kind of behavior.


text:
In LSTMs, how does the additive property enables better balancing of gradient values during backpropagation?
There are two sources that I'm using to to try and understand why LSTMs reduce the likelihood of the vanishing gradient problem associated with RNNs.
Both of these sources mention the reason LSTMs are able to reduce the likelihood of the vanishing gradient problem is because

The gradient contains the forget gate's vector of activions
The addition of four gradient values help balance gradient values

I understand (1), but I don't understand what (2) means.

Sources are SLIDE 119 on http://cs231n.stanford.edu/slides/2020/lecture_10.pdf

Sentence beginning "Another important property to notice is that the cell state" on https://medium.com/datadriveninvestor/how-do-lstm-networks-solve-the-problem-of-vanishing-gradients-a6784971a577

Any insight would greatly be appreciated!

can you please mention the slide number, where the point 2 is mentioned?

It's mentioned in the original post.

In order to vanish the gradient, the value should be zero for every path in the LSTM. If any path have the nonzero gradient the gradient will not be vanished. I think thats what they are trying to say.


text:
What is the best neural network model to classify an x(t) signal according two classes?
I am a beginner in AI methods. I have a collection of x(t) data, where x are some signal amplitudes and t is a time. My testing data are divided into two classes, say those from good and bad experimental samples. I need to classify the signals from unknown samples as good or bad according to their similarity to these two classes. What kind of a neural network is the best in this case? Could you recommend me some example in the literature where such a problem is considered?

RNN is the usual NN in case of variable length sequences as temporal ones.

@pasabaporaqui Thank you.


text:
Deep Learning based image restoration using multiple frames
Suppose we have a sequence of still images each of which has been contaminated by some particles(ex, dust/sand/smoke) making the images very poor in certain areas.
What architecture would be best to teach image regeneration using multiple frames? The simplest technique is to simply find a way to detect what parts of the image are contaminated and uncontaminated and pull uncontaminated sections from each frame.

I don't know a lot about it, but I think the technique is called Video Inpainting and you can find a lot of papers by searching the topic. And I was easily able to find review papers on this topic

We have denoising autoencoders to remove noise so we can use the difference between original and denoised picture to define areas with good or bad quality. After that, either segmentation or clustering algorithms can be used to determine the exact areas with bad quality.


text:
Unable to meet desired mean squared error
I wish to get MSE < 0.5 on test data (https://easyupload.io/zr7xf3) which is 20% of given data chosen randomly. But I am reaching 0.73 using both plain Ridge Regression as well as a neural network with about 6 layers with some elementary regularization, dropout and choice of other parameters. Overfitting also occurs.
Suggest. I believe a Bayesian optimization or a genetic algorithm for parameters is required.
I did no feature selection (as top 4 features showed no improvement) and non-linear methods exploration.
My solutions -
Ridge - Alpha = 0.002 (Grid searched)
Neural Network efforts =
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,
                              patience=10, min_lr=0.001)

es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=10, restore_best_weights=True)

model_b = Sequential()

model_b.add(Dense(2048, kernel_initializer='he_uniform',input_dim = X.shape[1], activation='relu', kernel_regularizer=regularizers.l2(l2=1e-6)))
model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))

# The Hidden Layers :
model_b.add(Dense(1024, kernel_initializer='lecun_normal',activation='selu',kernel_regularizer=regularizers.l2(l2=1e-6)))

model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))

model_b.add(Dense(1024, kernel_initializer='lecun_normal',activation='selu',kernel_regularizer=regularizers.l2(l2=1e-6)))

model_b.add(BatchNormalization(beta_regularizer = regularizers.l2(0.00001)))

model_b.add(Dropout(0.5))

model_b.add(Dense(512, kernel_initializer='normal',activation='relu'))

model_b.add(Dense(512, kernel_initializer='normal',activation='relu'))

model_b.add(Dense(256, kernel_initializer='normal',activation='relu'))

# The Output Layer :

model_b.add(Dense(1, kernel_initializer='normal',activation='linear'))
optimizer = SGD(lr = 0.0001)

model_b.compile(loss='mean_squared_error', optimizer= optimizer)

model_b.fit(X_train, y_train, batch_size=70,
              epochs=256,
              validation_data=(X_test, y_test),callbacks = [es])

predb = model_b.predict(X_test)

If anyone has free time, may answer.
Best

Regardless of where you post this though, could you explain why you have set a target MSE of 0.5? For instance, is it something you know can be achieved with the given data (e.g. other people are reporting error of 0.5 MSE or better using exact same test set)? Have you done some analysis and have reason to believe 0.5 MSE is achievable? Please use [edit] to add details to the question.

@Slater; I was asked at an interview


text:
Determining observation and state spaces for viterbi algorithm in a simple word recognition system using HMM
The system I'm trying to implement is a microcontroller with a connected microphone which have to recognise single words. the feature extraction is done using MFCC (and is working).

the system have to recognise [predefined, up to 20] single words each one up to 1 seconds length
input audio is sampled with a frequency of 10KHz and 8 bits resolution
the window is 256 sample wide (25.6 ms), hann windowed with a 15ms step (overlaying windows)
the total MFCC features representing each window, is about 18 features

I've done the above things, and tested the outputs for accuracy and computation speed so there is not much concern about the computations. now I have to implement a HMM for word recognition. I've read about the HMM and I think these parameters need to be addressed:

the hidden states are the "actual" pieces of the word with 25.6ms length represented in 18 MFCC features. and they count up to maximum of 64 sets in a single word (because the maximum length for input word is 1sec and each window is (25.6 - 10)millisecs)
I should use Viterbi algorithm to find out the most probable word spoken untill the current state. so, if the user is saying "STOP", the Viterbi can suggest it (with proper learning of course) when the user has spoken "STO.." . so it's some kind of prediction too.
I have to determine the other HMM parameters like the emission and transition. the wikipedia page for Viterbi which has written the algorithm, shows the input/output as:

from the above:

what is observation space? the user may talk anything so it seems indefinite to me
the state space is obviously the set containing all the possible MFCC feature sets used in the learned word set. how I learn or hardcode that ?

thanks for reading this long question patiently.

Hi. Please, ask 1 question per post. If you have $N$ questions, create $N$ posts, one for each question, otherwise, I doubt that anyone will be able to answer all your questions.

@nbro OK I'll simplify the question soon. thanx for advice.

Can you also please put your main question in the title?


text:
What are the implications of storing the alternative situation (that could have been experienced) in the replay buffer?
Consider an environment where there are 2 outcomes (e.g. dead and alive) and a discrete set of actions. For example, a game where the agent has 2 guns $A$ and $B$ to shoot a monster (the monster dies only if the correct gun is used).
Let's say we store the experience $e_1 = (s,a_1, r_1, s'_1)$ in the replay buffer $D$, where

$s$: we have the monster to kill
$a_1$: choose and use gun $A$
$r_1$: $-1$ (the wrong gun)
$s'_1$: monster is alive

But we also save the alternative situation $e_2 = (s, a_2, r_2, s'_2)$ in the replay buffer $D$, where

$s$: the same state (we have the monster)
$a_2$: choose and use gun $B$
$r_2$: $(-1) * r = 1$
$s'_2$: the monster is dead

I can't find a topic about this technique, or don't know what to look for.

It looks like for this scenario "if" statement could be the best option. Something like this: "if beast.kind == 1: gans.use.A() else: gans.use.B()".

When you ask "What are the implications", what do you mean exactly? Do you mean the "implications in terms of convergence to the right solution during learning/training"? Please, edit your post to clarify this point. Anyway, the closest thing to what you're proposing that I am aware of is Hindsight Experience Replay, but they don't store the alternative tuple, but change the trajectories that are experienced so that they become "falsely successful".

What do you mean by using "IF" statement, the state has +100 features (image and sound), i don't know what should be used "hence the use of the RL".

nbro; it's more of a problem of Sample efficiency, and i'm thinking of reversing the states to extend the sample size (as data augmentation). logically it should help, but the loss diverge after a while.

@KimbooRasta Please, tag me with @nbro (like I'm tagging you), otherwise I will not see your comments. I saw your comments only by chance.

@KimbooRasta Well, if you do what you suggest, you're basically assuming that those transitions were observed by the behaviour policy. Are you using DQN (i.e. an off-policy algorithm) or something else?


text:
Generating data from a High-Res. RGB image for a CNN
Say I want to build a detection model that detects the existence of X or NO X.
The only piece of information I have, though, is a high res. RGB image, say 100k (width) x ~1000 pixels (height).
Let's also assume I cannot browse the internet to grab more data. I am stuck with this High resolution image. Can I somehow "slice" this image into multiple images and use said images as input data for my CNN?
How would I do so?

I don't think you could make out a lot from just one image....however you could try adding  some noise  to the one image you have with different intensities and try training your CNN with those....

It's not clear to me what you are really asking. Are you asking how to convert a high-resolution image to a low-resolution image?

Well no. Since we generally input images of around 200x200x1 (give or take) into CNNs, how would we generate extra data, from super high resolution images (i.e. in the MP range)? (that's if we can)


text:
How to classify anomalies between two sound datasets?
I have two sound datasets and each one has 80% normal and 20% anomalous data points. The first one is a rock song and the second one is a mellow indie song. I use half of the normal data as a baseline in each dataset. I identify anomalies using isolation forest in each dataset and found 25 anomalies in the first rock song dataset and 12 in the mellow threshold. Now my question is how can I classify an anomaly as a rock song specific one? Do you think building a simple linear regression classifier should work?

When you ask "Now my question is how can I classify an anomaly as a rock song specific one?", are you asking how to find anomalies that are specific to the rock song rather than "general anomalies"?

yes. sorry for the confusion.


text:
CNN to detect presence/absense of label on images with mixed labels
Here's my problem: I work with medical image classification, and currently I have 3 classes:

class A: images with lesion 1 only; and images with lesion 1 and N other lesions
class B: images with 2 other lesions (no lesion 1)
class C: images with no lesion

The goal is to classify into "lesion 1", "other lesion", "no lesion". I'd like to know some approach/method/paper/clue for this classification. I think the presence of other lesions on both class A and B is confusing the model (the validation accuracy and f1-score are very low).
Thanks in advance.

You say "I think the presence of other lesions on both class A and B is confusing the model". Can you please explain a little bit about how you are training your model? What model are you using? What is your loss function? How many training and validation data do you have?

Hi. I'm using resnet50, and used categorical cross entropy and kullback leibler divergence losses. Both with similar results. I have 2000 images for class A (1300 wth mixed lesions), 1500 for class B, and 800 for class C. I'm applying 10-fold cross validation.


text:
What is the space complexity of iterative deepening search?
When using iterative deepening, is the space complexity, $O(d)$, where $b$ is the branching factor and $d$ the length of the optimal path (assuming that there is indeed one)?

As stated in my other answers here and here, the space complexity of these search algorithms is calculated by looking at the largest possible number of nodes that you may need to save in the frontier during the search.
Iterative deepening search (IDS) is a search algorithm that iteratively uses depth-first search (DFS), which has a space complexity of $O(bm)$, where $m$ is the maximum depth, with progressively bigger depths, and it turns out that its space complexity is $O(bd)$.
(Note that there is also the informed version of IDS, which uses the informed search algorithm A*, rather than DFS, known as iterative deepening A*).