text:
Adding data to training results in loss random peaks
I have succesfully trained ssd_mobilenet_v2_keras for object detection, with a dataset of about 3700 images. Now I have more images to add. I tried adding only a few images (150-300) to see what happened, but what I obtain is that the trainig looks good in the first steps, but then there are some really high peaks in the loss function.  At first, I thought the problem was the quality of the pictures, so I removed them and tried to add more or less 300 bigger pictures: nothing changed. Then I tried to add only good pictures (no shadows or lights that may confuse the net, no interference with the object, only images where the objects I want to find are big and centered), but nothing.  All the things I have tried leads to the same results:     As you can see, The training looks good at the beginning, but then there are those extremely high peaks that seems to happen at random steps (sometimes after 20.000 staps, sometimes after 2.000).  I tried to train both with and without some data augmentations (random contrast, brightness and saturtion adjust, random rgb-to-grayscale, random horizontal flip, ...) but the results are more or less the same (with data augmentations it's a little better, but still far from good). Any suggestions on why this happens and how to fix?

EDIT: unfortunately I didn't take a screenshot at the end of the succesful training, I only have this one taken after 6.000 steps (total number of steps is 50.000), but then the chart followed this trend and ended with this values: - classification loss: 4.16e-3  - localization loss: 1.11e-3  - total loss: 0.077

Could you share the loss graph before adding those images, so, how did the loss graph look before adding the images? I have a pretty good hypothesis of what is happening, but need more info

@JVGD unfortunately I don't have the screenshot of the final result...I added one taken after 6.000 staps (total steps number is 50.000), I hope it is enough for you...let me know if you need more

Ok, so the loss function is ok. I have seen the same in my experiments with custom loss functions that are not well defined or when data is not correctly annotated or not consistent.  It might be happening that the new images you added to the dataset are not correctly annotated (maybe a parsing error or a format error)

@JVGD mmm this sounds strange to me...I have personally annotated them, with the same method used for the 3400 starting images (I use the software labelImg, that generates .xml files). Also, I had added 300 pictures a couple of weeks ago (that's why in the question I said 3700) and I had no problems. From a quick check, the format looks the same...I don't think it's an annotation problem


text:
Is there a survey that describes the most effective approaches for an answer retrieval problem?
I have a dataset that contains pairs of a question and an answer. My problem is to train a model that can search for the right answer from the pool of my answers given the newly input question, so this is a kind of answer retrieval problem.
Can anyone provide me a survey and effective approaches for this problem?

What research have you done already?

As suggested in the previous comment, before asking a question, you should do a little bit of research and tell us what you have or not found. See https://ai.stackexchange.com/help/how-to-ask for more details of how to ask a good question.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Do deep learning researchers generally visualize intermediate steps?
Many researchers in deep learning research come up with new CNN architectures.
The architectures are (just) combinations of a few existing layers.
Along with their mathematical intuition, in general, do they visualize intermediate steps by execution and then (do trial and error) brute force for achieving the state-of-art architectures?
Visualizing intermediate steps refers to printing outputs in the proper format for analyzing them. Intermediate steps may refer to feature maps in CNN, hidden states in RNN, outputs of hidden layers in MLP, etc.

This question seems to be related/similar to this, this and this.

Moreover, note that the term "feature map" only typically applies to convolutional neural networks. If you're referring only to CNNs, it's better that you edit your post to clarify that. Also, what do you mean by "visualize intermediate steps", do you mean the outputs of a hidden layer? I also roughly understand what you mean by brute-force, but it may be better to given example to clarify what you mean in that case too.

Okay @nbro. I will improve the question.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to measure the significance of an input feature for the output of a linear layer in a neural network
Suppose I have a simple linear layer $y = xA^T + b$ that is part of a neural network trained on some dataset. The weight matrix $A$ for this layer has the shape [num_outputs, num_inputs].
For each layer input, I would like to find a value between 0 and 1, based on the weight matrix, representing the significance of that input to the layer output.
Intuitively, if the values in the i-th column of the weight matrix are close to 0, then the significance of i-th inputh should also be close to 0. Conversely, if the values are close to maximum or minimum of the entire weight matrix, the significance should approach 1.
This statistic should also adequately recognize cases where the vast majority of values in a column are close to 0, but at least one is not. Then the significance of such an input should not be close to 0, because it is important for a single neuron that detects, for example, an edge case.
Can anyone point me in the right direction?

Conversely, if the values are close to maximum or minimum of the entire weight matrix, the significance should approach 1. How minimum values can be assigned weightage of 1?

@hanugm Weights can also be negative numbers. An input with a weight of -0.2 has much less impact on the output than an input with a weight of -2.4.


text:
Is there any geometrical interpretation on overcoming gradient related problems by adjusting/changing loss function?
There are instances in literature where we need to change loss function in order to escape from gradient problems.
Let $L_f$ be a loss function for a model I need to train on. Some times $L_f$ leads to the problems due to gradient. So I reformulate it to $L_g$ and can apply the optimization successfully. Most of the times the new loss function is obtained by making a small adjustments on $L_f$.

For example: Consider the following excerpt from the paper titled Evolutionary Generative Adversarial Networks

In the original GAN, training the generator was equal to minimizing
the JSD between the data distribution and the generated distribution,
which easily resulted in the vanishing gradient problem. To solve this
issue, a nonsaturating heuristic objective (i.e., “$− \log D$ trick”)
replaced the minimax objective function to penalize the generator

How can one understand those facts geometrically? Are there any simple examples on either 2d or 3d that shows two types of curves: one gives no gradient issues and the other gives gradient issues yet both obtains the same objective?

Could you link one or more examples that you say you have read in the literature? This is not something I have heard of, or I do not understand properly what you are asking. The original text would help

@NeilSlater https://ai.stackexchange.com/questions/29985/what-does-it-mean-by-strong-or-sufficient-gradient-for-training-in-this-context

That doesn't appear to mention vanishing gradient? Could you link or clarify about vanishing gradient as per this question? Or is it possible that you are not asking about vanishing gradient, but about gradient issues discussed in the paper?

@NeilSlater I think the latter one. I thought vanishing gradient is mostly the problem related to gradients...

Vanishing gradient is specifically the problem associated with gradient signals decaying through multiple layers or multiple time steps. I think your linked paper is more to do with an original signal that can get weaker when it is certain distance away from the objective

It is more common to talk about derogates and vanishing gradient problem when discussing SGD. when discussing loss issues it is more common to discuss regularization methods.


text:
Training a sequential model that can only evaluate after several hundred cycles
I'm attempting to build a neural network to play the card game, Lost Cities.
A brief overview of the game:

The game involves two players taking turns to play cards on expeditions.
Expeditions incur a debt when you play the first card. Subsequent cards will buy out of that debt and return a profit.
Each player must either play or discard a card (with restrictions), and then draw a new card from the deck or the discard pile.
The game ends when the last card is drawn.

The expanded rules can be found here.
I'm attempting to train a sequential model to play this game, with the state of the board/hand/discard pile as a set of inputs and a three heuristic values for each card (value of playing, value of discarding, value of picking up from this discard) as the output values. However, it's unclear to me how I should approach this from a training perspective.
My biggest hurdle is that the network's success can only be evaluated by wether or not it can "beat" itself or a competing network in a game. A player's raw score during the game is not viable due to the nature of play. To me, this means that the network will have to be used with several hundred different sets of input data (for each turn of state of the game board during a match) before any meaningful results are generated.
So far, the only solution I've had minor success with is a generational algorithm that creates "fuzzy" children to compete against each other for the most "wins". This was done in Python's standard library, and was obscenely slow, even with a reduced sequential network.
My question:
Is there an established method to deliver delayed feedback to a network (after several uses of the network)?
I'm very new at this, so any and all feedback is more than welcomed.

Hello, welcome to Artificial Intelligence Stack Exchange. I didn't read your question in detail. But, it seems to me like a programming question in Tensorflow. If yes, please note that general programming questions are off-topic for or site. Please read here for more details and you can ask on CV.SE.

Hello @hanugm, thanks for your response. I've edited my question to specifically ask about the high-level architecture for my network, rather than the specifics regarding the TensorFlow platform. Sorry for the inconvenience.


text:
Tensorflow Probability Implementation of Automatic Differentiation Variational Inference with Mixtures
In this paper, the authors suggest using the following loss instead of the traditional ELBO in order to train what basically is a Variational Autoencoder with a Gaussian Mixture Model instead of a single, normal distribution:
$$
\mathcal{L}_{SIWAE}^T(\phi)=\mathbb{E}_{\{z_{kt}\sim q_{k,\phi}(z|x)\}_{k=1,t=1}^{K,T}}\left[\log\frac{1}{T}\sum_{t=1}^T\sum_{k=1}^K\alpha_{k,\phi}(x)\frac{p(x|z_{k,t})r(z_{kt})}{q_\phi(z_{kt}|x)}\right]
$$
They also provide the following code which is supposed to be a tensorflow probability implementation:
def siwae(prior, likelihood, posterior, x, T):
  q = posterior(x)
  z = q.components_dist.sample(T)
  z = tf.transpose (z, perm=[2, 0, 1, 3])
  loss_n = tf.math.reduce_logsumexp(
  (−tf.math.log(T) + tf.math.log_softmax(mixture_dist.logits)[:, None, :]
  + prior.log_prior(z) + likelihood(z).log_prob(x) − q.log_prob(z)), axis=[0, 1])
  return tf.math.reduce_mean(loss_n, axis=0)

However, it seems like this doesn't work at all so as someone with nearly no tensorflow knowledge I came up with the following:
def siwae(prior, likelihood, posterior, x, T):
  q = posterior(x) # distribution over variables of shape (batch_size, 2)
  z = q.components_distribution.sample(T)
  z = tf.transpose(z, perm=[2, 0, 1, 3]) # shape (K, T, batch_size, encoded_size)
  l1 = -tf.math.log(float(T)) # shape: (), log (1/T)
  l2 = tf.math.log_softmax(tf.transpose(q.mixture_distribution.logits))[:, None , :] # shape (K, 1, batch_size), alpha
  l3 = prior.log_prob(z) # shape (K, T, batch_size), r(z)
  l4 = likelihood(tf.reshape(z, (K*T*x.shape[0], encoded_size)))
  l4 = l4.log_prob(tf.repeat(x, repeats=K*T, axis=0)) # shape (K*T*batch_size, )
  l4 = tf.reshape(l4, (K, T, x.shape[0])) # shape (K, T, batch_size), p(x|z)
  l5 = -q.log_prob(z) # shape (K, T, batch_size), q(z|x)
  loss_n = tf.math.reduce_logsumexp(l1 + l2 + l3 + l4 + l5, axis=[0, 1])
  return tf.math.reduce_mean(loss_n, axis=0)

There are no errors when I try to use this as
siwae(prior, decoder, encoder, x_test[:100, ...], T)

but after a few training steps I get only nans. I really don't have any idea of this is an due to a wrong implementation or wrong usage of the loss - especially as I don't have much experience with tensorflow. So any help would be greatly appreciated.
For a full, minimal example I created this colab.

Hi, Jonas. Unfortunately, this AI stack is not focused on too technical and computational problems. You should try asking the same question on other Stacks, like: https://datascience.stackexchange.com

Hi Andre, thanks for the suggestion, I was a bit unsure on where to post and ended up here but in this case I'll try datascience.

Or would you suggest to even choose stackoverflow instead @AndreGoulart? My problem is that I'm not even sure myself whether the problem is just the tensorflow code (-> stackoverflow) or more on the side of getting the theory wrong.


text:
What's the best way to feed stories to a neural network?
I'm trying to train a model that would generate stories. I have a dataset of 2000 stories prepared. They are tokenized and one-hot encoded. I can't load them all at once as a one big dataset, because of memory limits.
What would be the best way to fit my network so that i can reset the states after each story?
I tried doing it in a nested for loop (for epoch/for story: model.fit) but it's working really slow cause it takes 3 seconds to fit a single story but almost 10 to load the next file and setup model.fit again.

What is the largest size, that your model can accept? I would suggest to form a batch of stories as a tensor, and the feed it to the network. Like 100, 500 stories, whatever fits into your memory

Would probably have to try different sizes and guess what would fit since the stories have different lenghts. The issue is that I want to reset my network after each story so that it models the structure. But if I made a batch of for example 10 stories, then as far as i know i can only reset per batch so I wouldn't be able to reset after each of the stories. Im wondering if there could be a way to create for example a data generator that would feed stories (right now  a story is 10-20 batches ) with resetting after each.

I'm guessing that the best way would be to feed a story in one batch but I don't think my memory can handle that since my vocab size is 10k

Nevermind, I realised I'm able to create a data generator that will load a whole story as one batch and it fits in my memory. Don't know why I haven't thought about it before. Thanks


text:
Is it possible to identify which feature maps were generated from a particular image after convolutional operation
Let's say I have a video that contains 3 grayscale sequential frames having a combined shape of (3, 24, 24). After inputting these frames together into a CNN, multiple feature maps will be generated from each of the images. Would it be possible for me to separate the temporal aspect by identifying which frame generated which feature maps?

Is $3$ your batch size?

Yes you could say that. Then the shape of my input would (3,1,24,24) => 3 images, 1 channel,  Height, Width. @hanugm


text:
Which model is more efficient and why?
Suppose, I have two NN models:

CNN model
Sequential NN model

They are solving the same problem. The data points have the same number of features.
In the case of #1, we used 0.6 million data points, 35k epochs, and the model achieved 80% accuracy in the training.
In the case of #2, we used 1.4 million data points, 1k epochs, and the model achieved 90% accuracy in the training.
Which model is better/more efficient and why?

You quote "% accuracy in the training". Could you clarify how this was measured - is it reported accuracy on the training dataset, the best result from cross-validation, or a final report against a hold-out test dataset?

@norbertk, I got the answer to my query. thanks.


text:
Expression Transfer Deep Learning Problem
I have old video and I want to keep the person's face in the video but I want to transfer my facial expressions to that video. Is there any better alternative to first order motion model for that task ? I tried deepfacelab but it has kind of steep learning curve

What do you mean by "first order motion model"? Maybe you should also show us what you're expecting and what exactly you have tried.

I was using this: https://github.com/AliaksandrSiarohin/first-order-model


text:
What should the value of $ρ$ in the $w(n+1) = w(n) + \rho*\text{error}(i)x(i)$ formula of Least Mean Squares be?
I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically.
If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ where $w(n + 1)$ is the new weight of the classifier function, $w(n)$ is its current weight and $x(i)$ is the $i$th element of a training dataset, what should $\rho$ be?
From what I have found online, $ρ$ is supposed to be $0 < \rho < \frac{2}{trace(X^TX)}$, where $X$ is a matrix with all the training data the algorithm has processed at that point. One idea that I had, was to take $\rho = \frac{1}{trace(X^TX)} < \frac{2}{trace(X^TX)}$, but I do not know if that is correct. Also, one characteristic that this value has is that it changes with each iteration of the algorithm, as more samples are added to matrix $X$.
So, what is a good value for $\rho$? Should it change during the execution of the algorithm or should it stay the same?

Hello. To clarify which algorithm you're referring to, are you talking about any of the ones mentioned here https://en.wikipedia.org/wiki/Least_mean_squares_filter?

@nbro Hi. Yes, the formula that I am using can be found in the section "LMS algorithm summary" of that article, although with different symbolism.


text:
Cost functions for reducing Tensors to 1-dimensional arrays?
I'm interested in the IT side, here, specifically how I most efficiently store a tensor in a one dimensional data structure. My assumption is that certain approaches will be more expensive than others, but I'd like to be able to validate that assumption, and show it to be wrong. Is there any work on this subject?

You need to clarify how this is related to AI, otherwise, this would be off-topic.

@nbro I hope it falls under computational aspects of AI as Tensors are mandatory for neural networks.


text:
What deep reinforcement learning algorithm should I use for my problem?
So here is a description of my problem:
Essentially, I have a large amount of files filled with code for a number of different tasks. However, lets say these codes are inefficient, and should be edited to be made more efficient. There exists a program to edit this code, and essentially what is does is analyze the code to look for what possible edits can be made, and classifies these edits into 5 classes (Class A, Class B, Class C, Class D, Class E). There is also a program to test the efficiency of this code and yields numeric value quantifying the efficiency. However, choosing some edits benefit the efficiency of a some code better than other edits. For example, a piece of code that has had 5 different edits applied to it (2 of Class A, and 3 of Class C) can be more efficient than the same code that has had 10 edits applied to it (5 of Class A, 5 of Class D). Ideally, the less edits and the higher the efficiency the better.
I want to use deep reinforcement learning to  address this problem.
Here are the goals of my model:

Takes the code in a form which it can understand (I have looked into this and stumbled upon code2vec, so I am looking into that)
Also can take in the edits that can be applied to the code (which is the action space) and their respective classes.
Makes decisions for edits which yield the highest efficiency for the code

I want to train this model on the files filled with the inefficient code. The ultimate goal is to end up with an AI that takes code as an input and makes the best decision of which edits it should apply to the code.
What deep reinforcement learning techniques/algorithms/architecures should I use to approach this problem and how can I go about implementing these in Python? As you can probably tell from this question, I am not very experienced in deep reinforcement learning, but I am willing to learn. Any help would be appreciated!

not specific to the code fixing case, but you might want to google "adaptive operator selection" especially the RL based methods among them. perhaps also google the "learn to improve" paradigms from RL for combinatorial optimization.

thanks! ill look into that


text:
Uniform representation of images for machine learning
I'm new to the field of ML so please bear with me while I try to explain what I'm looking for. In most machine learning pipelines that deal with images there is a requirement to "normalize" the data in some way so that images of different dimensions can be used as inputs for the function that is being optimized. As in, if the function takes its input as an $n\times n$ grid of pixels (assuming we're dealing with 1-channel images) then any image that is not of the right shape must be re-shaped so that it can be used as input. We can assume $n = 2$ without losing any generality because any larger image can be reduced to the $2\times 2$ case for what I'm about to describe.
So if we assume we have a $2\times 2$ image then there is an obvious way to map such an image to a function defined on $[0,1]\times [0,1]$ ($f:[0,1]\times[0,1]\rightarrow\mathbb{R}$) by using convex combinations of the points in the image. If the points of the image are labeled as $x_{00},x_{01},x_{10},x_{11}$ where $x_{00}$ is the top left corner and $x_{11}$ is the bottom right corner then given a point $(a,b)\in[0,1]\times[0,1]$ the value of $f$ at $(a, b)$ can be defined as $$f(a, b) = (1-b)((1-a)x_{10}+ax_{11})+b((1-a)x_{00}+ax_{01})$$
Assuming I got all the signs right it's obvious that this idea can be extended to any grid of pixels by mapping the horizontal and vertical dimensions to $[0,1]$ and then interpolating between the grid points as in the $2\times 2$ case. So this mapping from grids of pixels to functions provides a uniform representation for all images as functions defined on $[0,1]\times[0,1]$.
Now my question is the following: Is there any work that tries to use this kind of representation of images and if there isn't does anyone know what exactly are the obstructions to doing so? It's possible I'm missing something that makes this approach non-viable but I wasn't able to find anything that explained one way or the other why the usual tensor representation is preferable to a functional one as above that reduces all images to functions on $[0,1]\times[0,1]$.

You seem to be trying to describe bilinear interpolation, I think?

Yes, this is bilinear interpolation followed by rescaling to make everything fit into the unit square. It's obvious to me that this representation subsumes the usual tensor one because every tensor representation can be reduced to a functional one by adding the dimensions as another set of inputs to the learning algorithm, e.g. $(w, h, f)$ where $w$ is the width and $h$ is the height of the image. The question is why aren't there more learning algorithms structured around this representation?

To explain a bit more. Images have a topology that the usual tensor representation does not account for because it treats each pixel independently from its neighbors but it's obvious that there is a continuity in natural images because pixels that are close together often have similar values but the tensor representation does not account for this natural topology. In the representation I'm proposing this natural continuity is made obvious because the function that represents the image is continuous so it has more structure that the tensor representation doesn't.

Why do you think the tensor representation doesn't account for this natural topology? Pixels that are close together in the image are also close together in the tensor, and convolutional layers take advantage of that (as well as the fact that we want to look for the same thing in any part of the image)

Can you describe the topology of the tensor representation? The topology of both the unit square and $\mathbb{R}$ are the standard ones taught in most undergrad math classes with $\epsilon$ neighborhoods. What is the equivalent for the tensor/matrix representation?


text:
How to assess the goodness of a text generation algorithm
Take a RNN network fed with Shakespeare and generating Shakespeare-like text.
Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how can one assess and refine the goodness of the result ?
Only human eyes can judge of the readable character of a text, its creativity, its grammatical correctness etc.
QUESTION : Which systematic approach can be used to refine a generative model (text) ?

How would you measure the quality of the output text?

That's a philosophical question, at first, it should be readable and correct, which is already a challenge when tuning generative models

So you could, as a very basic metric, use readability scores (though they have loads of faults).


text:
What are the different types of geometry in literature that may be used for deep learning?
Recently, I asked a question on the concept of a manifold and received an answer that points to a relatively new subfield of deep learning named geometric deep learning.
In the preface of the paper titled Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges, there is a mention of three types of geometry that do exist in the literature.

For instance, Euclidean geometry is concerned with lengths and angles, because these properties are preserved by the group of Euclidean transformations (rotations and translations), while affine geometry studies parallelism, which is preserved by the group of affine transformations. The relation between these geometries is immediately apparent when considering the respective groups, because the Euclidean group is a subgroup of the affine group, which in turn is a subgroup of the group of projective transformations.

The three types of geometry they mentioned are Euclidean, affine and projective. I want to know the complete list of types of geometry that do exist in the literature, if relevant to geometric deep learning.
What are the types of geometry in the literature that may be used for deep learning?

Hi there, if you are interested in GDL, you can check out those lectures https://www.youtube.com/playlist?list=PLn2-dEmQeTfQ8YVuHBOvAhUlnIPYxkeu3, they are given by the authors of this paper and closely realted to this paper.

Yeah, thanks @SwaksharDeb


text:
What is meant by domain in the notations of geometric deep learning?
While reading the Notation of the paper titled  Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges, I came across the following notations.
$$
\Omega = \text{ Domain} \\
 u = \text{Point on domain}
$$
I can only understand domain here as a set and point as an element in the set $\Omega$.
I have a simple doubt here.
Why do we call the set $\Omega$ as domain? Is it due to the notation related to mathematical function?
$$\text{function_name} : \text{domain} \rightarrow \text{co_domain}$$
Or is it due to the usage of the word domain in the sense of application domain: A specified sphere of activity or knowledge. such as computer vision, natural language processing etc?
or a combination of both?

Consider this example for an image, here domain is the grid(whole 2D grid), the function (image) is over this domain f(x,y), here a point  of this domain is (x,y).

So, it is same as the domain of a function right? @SwaksharDeb

We call it domain for mathematical scence like your first example

yes, it's the sample domain as we use in mathematics


text:
What are practical methods to acquire a large number of tasks for Meta-learning?
It appears that it may be necessary to acquire a very large number of tasks for meta-learning , because MAML for example says that each task is analogous to a single training example in regular learning.
This is slightly confusing to me because it appears that outside of automated techniques like N-way classification where you randomly sub-select classes (& training examples) to include in a given task.
Adding tasks seems to be quite laborious right? I mean it sounds like you would need to get a different micro dataset for each task? And if meta-learning needs so many tasks then how do you satisfy that need?

Yes, it does seem that way. You are trying to train something to learn tasks. To learn dogs you need logs of dogs. To learn tasks you need lots of tasks and if the task is detecting dogs then you need lots of dogs for just that one task.

I'm not sure if I'm right. But it seems maybe you don't need a lot of dogs. Because usually meta-learning is few-shot. You maybe just need a few images of lot of animal species.
Unless you are doing an automated method for generating tasks.


text:
"Hot Word Detection", bur for different applications
I have been looking in detecting a specific rhythm/pattern within the temporal domain for a time-series signal. For this purpose, how "Wake Up" words work for devices like Alexa have gained my interest!
I have 2 questions with regards to this matter:

Is this just, say, training a model (e.g. CNN) on a specific word (I know audio processing is not this simple, I meant it in it's simplest form) and running it continually until a hot word is detected? if so is the name just a formalization and not a different method of implementing a ML model?

If this type of model can be used for a time series signal, namely audio, is there any restriction on applying it on a different set of temporal signals? for example, detecting the onset of an earthquake?

Also, I would really appreciate it if someone could reference a good paper to have a read through on this matter, I have looked online but have not been able to find interesting stuff.

For question 2, yes there would be no reason it couldn't be applied to another temporal signal, it just depends on how easy that signal is to detect.

Hello. If you have 2 questions, it's better that you split this post into 2 separate posts, one for each question, but make sure to provide enough context/details in each post for us to understand your problem and question.


text:
Why isn't my perceptron having the expected costs?
I want to implement a single perceptron for linear regression using the following formulas:

The input data for the first case is one column (x(392, 1); y(392, 1)) and for the second case is (x(392, 7); y(392, 1)). The NaN values have been removed and x values have been standardized x-x.mean()/x.std()
This is my Python implementation:
class LinearRegression(object):

    def __init__(self, x, y, n_iter):
        self.x = x
        self.y = y
        self.n_iter = n_iter
        self.cost_iteration = []
        # Initializing model parameters (w, b) to zeros
        self.weights = np.zeros((1, self.x.shape[1]))   # w: weights
        self.biases  = np.zeros((1, 1))                 # b: bias

    def feedforward(self):
        # return the feedforward value for x
        #self.weights, self.biases = self.update_params()
        z = self.x @ self.weights.T  + self.biases
        return z

    def loss(self):
      # return the loss value for given x and y
      z = self.feedforward()
      loss = self.y-z
      cost = np.sum(loss**2)/self.y.shape[0]
      return loss, cost

    def backpropagation(self):
      # return the derivatives with respect to weight matrix and biases
      loss, cost  = self.loss()
      db = -2*np.sum(loss)/self.y.shape[0]                    # dJ/db
      dw = -2*np.dot(self.x.T, loss)/self.y.shape[0]          # dJ/dw
      return dw, db

    def update_params(self):
      # update weights and biases based on the output
      dw, db = self.backpropagation()
      self.weights -= dw.T
      self.biases  -= db
      return self.weights, self.biases

    def fit(self):
      # fit method for the training data
      for iter in range(self.n_iter):
        self.update_params()
        print(self.biases)
        l, c = self.loss()
        self.cost_iteration.append (c)
      return self.cost_iteration

The final cost should be approximately 23.9 and 11.6 for the two models, respectively. But I can't figure out why it's not the case when I use my code.

Hello. Could you please put your specific question in the title?

I am not sure how to formulate the question correctly. But I think the problem is my implementation of the loss and update methods. I think they are wrong based on the resulting cost.

Ok, I tried to give a more descriptive title to your post. Please, make sure that it's consistent with what you were asking. How do you know that the cost should approximately be 23.9 and 11.6?

It is said at the end of the document (from which I took the screenshot above).


text:
Usefulness of the state_values calculation in Dynamic Programming
State values are always presented as a central concept in RL, notoriously in the bible, the Sutton&Barton’s book.
I have done some exercises trying to improve my understanding, but it is clear that I am missing some important point(s), at least in the tabular case.
I do not understand, in the case where we have a complete MDP model of the environment, why we should bother doing value iteration or/and policy iteration if we can directly find the optimal policy by calculating each state-action value.  I mean that having the state-action values, we can by the argmax function obtain the optimal policy for any starting point.
In my learning/example, link, with an observation space of one million states and three different actions, using a desktop notebook it took 21 iterations in 19 minutes to find every state action value, and consequently,  the optimal policy.
Perhaps it could be done quicker, selecting only the interesting states. For instance, if there is a unique starting state and the simulation has a finite-small number of steps per episode, probably most of the states are uninteresting because never visited. In that case, we can select the target states using some test simulations, but, is it worth?

Could you please explain more what you mean by "if we can directly find the optimal policy." What would you intend to do differently to state/value calculations? I'd like to understand if you have some other mechanism in mind, or are more concerned about basic dynamic programming calculating all the state values (including ones yuo may not care about). Your last paragraph mentions that point, but the title and first four paragraphs are not, and kind of imply you are looking for an alternative to any kind of value calculation

ok. I'll edit my question

Thanks for the edit. I am confused though, where do you think the state-action values that you want to use for your policy are coming from? They are not part of the environment model. The dynamic programming is calculating them (or the state values normally, but it can easily calculate action values if that's what you want). But you seem to want to not do that calculation - where will you get the values from instead?

Ies. As I said, I calculate them, and the state_action_value is solved. What for solving the "state_value"? In some sense, I think I do not understand the interest of the prediction problem. It seems to me that solving the control problem is enough.

Your question is about not calculating them though, and still using them? How are you calculating the state action values without using a value-based method? How, exactly are you calculating $q^*(s,a)$ without using dynamic programming in your case where you want to go straight to the optimal policy?

I also don't understand what you mean by "if we can directly find the optimal policy by calculating each state-action value". Are you asking why do we estimate V(s) if we can simply estimate Q(s, a) (from which we can directly derive the policy)?

@nbro. Ies. This is exactly what I meant.

@nbro Also, why calculate Q(s,a) following a specific policy instead of calculate it following an exhaustive sweep search ? In the tabular situation, it seems reasonable.

@HermesMorales Ok, I suggested that you edit your post to clarify your question and leave the question in the form I've written it. After you've edited it, more people should see it.


text:
How to initialize the coefficient vector of Deep Tensor Neural Network
In Quantum-Chemical Insights from Deep Tensor Neural Networks, I would like to ask a question about how to initialize the coefficient vector of the network, because I could not understand it even after reading the paper. In the paper, it says

All presented models use atomic descriptors with 30 coefficients. We initialize each coefficient randomly

If it is initialized randomly like this, why do we initialize it randomly since we cannot use the information of nuclear charge as mentioned in the paper?

You should provide a direct link to the paper: https://www.nature.com/articles/ncomms13890.pdf

Are there 30 different nuclear charges per atom?


text:
In this paper, if region $R_{2}$ moves in a sliding window manner, won't the saliency map have a smaller size than the original image?
In the paper Salient Region Detection and Segmentation, I have a question pertaining to section 3 on the convolution-like operation being performed. I had already asked a few questions about the paper previously, for which I received an answer here. In the answer, the author (JVGD) mentions the following:

So for each pixel in the image you overlap on top $R_{1}$ and then $R_{2}$ on top of $R_{1}$, then you compute the distance $D$ for those 2 regions to get the saliency value of that pixel, then slide the $R_{1}$ and $R_{2}$ regions in a sliding window manner (which is basically telling you to implement it with convolution operation).

Regarding the above, I had the following question:
If the region $R_{2}$ moves in a sliding window manner, won't the saliency map (mentioned in section 3.1) have a smaller size than the original image (like in convolution the output image is smaller)? If this is so, wouldn't it be impossible to add the saliency maps at different scales since they each have different sizes?
The following edit re-explains the question in more detail:

In the animation above, you can see a filter running across an image. For each instance of the filter, some calculations are happening between the pixels of the filter and the image. The result of each calculation becomes one pixel in the output image (denoted by "convolved feature"). Here, the output image is smaller than the input image because there are only 9 instances of the filter. From what I understood of the salient region operation, a similar process is being followed i.e. a filter runs across an image, some calculations happen, and the result of each calculation becomes one pixel in the output image (saliency map). Hence, won't the saliency map have a smaller size than the original image? Furthermore, when the filter size is 3 x 3, the output image size is 3 x 3. However, if the filter size was 5 x 3, the output image size would only be 1 x 3. Clearly, the output image size is different for different filter sizes. This makes the output images (saliency maps) impossible to add. There is clearly something I am missing / misunderstanding here, and clarity on the same would be much appreciated.
P.S. There is no indication of padding or any operation of that sort in the research paper, so I don’t want to assume anything because the calculations would then be wrong.

Have you considered they just pad the input?

The paper you have provided, discusses *detecting salient regions by using a contrast determination filter

which operates at various scales to generate saliency maps containing “saliency values” per pixel.*

The Contrast detection filter used in this paper is better explained with the figure 2. Here R1 is the inner most region and R2 scale is varied. Note this is the filter and not the image itself

The below fig 3 shown in the paper section 3.1 indicates how the image appears when the filter of varying scales is applied(filtered images). When R2 has the highest scale(i.e., maximum width) the background is also shown that is the non-salient parts are also taken into consideration. As the R2 takes up lesser width, the non-salient parts becomes almost invisible which you can see in 3 to 8 images in figure3. It helps to focus on what's really important in the image, which is the man riding the horse.

If you have noted in the figure 3, though R2 width varies, the filtered image size is the same as the original image.
To answer your question, the saliency map is the same size as the original image size as only the filter was scaled not the image itself
This is also quoted in the paper section 3.1, page 4

A change in scale is affected by scaling the region R2
instead of scaling the image.
Scaling the filter instead of the image allows the
generation of saliency maps of the same size and resolution as the input image.

Update
Let me provide you with a much more simpler explanation ,
suppose you as a human have couple of reading glasses(of varying sizes) and you are trying to read a book.

The smallest reading glass will help you focus on smaller part of the book, and you would often have to look in different parts of the book
A medium reading glass will focus on a more wider part of the book
If you have a large reading glass, you can cover the whole book

Now in above example, reading glass acts as the filter, the book acts as your input image, the output image is what you can see through the reading glass.
No matter the size of your reading glass(filter) the dimensions of the book hasn’t changed only the focus on what is observed i.e., content or the information of book you see through your reading glass is more focused and everything else is a blur.
So the output image will be the same size as the input image with the out of focused will be like a blur

@ArchanaDavid I don't think this really answers my question. I have edited my original question to provide more detail and clarity for what the question actually is. Hopefully, you can provide more insight on how exactly "scaling the filter instead of the image allows the generation of saliency maps of the same size and resolution as the input image".

@learning123 have updated my answer to help you better understand the concept

@ArchanaDavid I am trying to code the algorithm in the research paper for a project, hence I would highly appreciate it if you could explain the process exactly (maybe with a little diagram or something or something like how I explained my logic with the above animation) or tell me what is the flaw in my reasoning because although I understand your analogy, it doesn't explain how the output image sizes won't be different for different filters (you can refer to my question for my logic). Sorry if I am stretching this, but I am still confused about what is exactly happening with the filter.

@ArchanaDavid I think I am having some confusion with the filtering process itself in thinking that it is like a convolution operation and that is why I am having issues.  If you can explain that process in a little bit if detail, I think it will answer my question. Once again, sorry for constantly pestering you.

"operates at various scales...pixel" appears to have been copy-pasted from the paper.  Please see our rules on copying from external sources: https://ai.stackexchange.com/help/referencing.  In particular, whenever you copy from another source, we require you to clearly indicate that you are quoting from them and credit the source.


text:
How to decide a train-test split?
In almost every ML model, a train-test (or train-test-val split) is critical to assess the model's performance. However, I have always wondered what the rationale is to decide a particular train-test split. I've seen that some people like an 80-20 split, others opt for 90-10, but why? Is it simply a matter of preference? Also, why not 70-30 or 60-40, what is the best way to decide?

I don't think there is any rationale behind choosing 80/20 over 75/25 or others. But those are the numbers for rather small datasets. If your dataset is large enough (like hundreds of thousands of samples), you can even work with 98/1/1 percents for train/val/test as discussed by Andrew Ng in this video. Neural networks thrive with big data and it is always a good idea to make use most of it.


text:
What makes Sequential Bayesian Filtering and Smoothing tractable?
I'm currently diving into the Bayesian world and I find it pretty fascinating.
I've so far understood that applying the Bayes' Rule, i.e.
$$\text{posterior} = \frac{\text{likelihood}\times \text{prior}}{\text{evidence}}$$
are most of the time intractable because of the high dimensional parameter space in the denominator.
One way to solve this is by using a prior conjugate to the likelihood, as then the analytical form of the posterior is known and calculations are simplified.
So far so good. Now I've read about bayesian sequential filtering and smoothing techniques such as the Kalman Filter or Rauch-Tung-Striebel Smoother (find references here). As far as I understood,  assuming a time step $k$,  instead of calculating the complete posterior distribution $p(X_k|Y_k)$ with $X=[x_1, ...,x_k]$ and $Y=[y_1,...y_k]$, a Markov Chain is assumed and only the marginal $p(x_k|Y_k)$ is estimated in a recursive manner. That is, the posterior calculate at time step $k$ serves as prior for the next time step. I guess, Bayes' Rule is somehow involved in these calculations
Furthermore, both techniques assume the posterior always to be Gaussian and therefore closed-form solutions are obtained. Now I was wondering what restriction does make the whole process tractable, i.e. eliminates the need to compute the evidence?
I guess it's the Gaussian assumption, i.e. the prior, the predicted, and the posterior distribution are all assumed to be Gaussian, and therefore updated distributions are obtained without computing the evidence - is this correct and does this refer to conjugate distributions?
Or is it the fact that we assume a Markov Chain and do not consider all states at each time step?

It's the Markov assumption which let's you forget the prior states.

so the Gaussian assumptions of prior and posterior distribution have nothing todo with it?


text:
In this paper, how does scaling the filter instead of the image generate saliency maps of the same size and resolution as the input image?
In this paper, in section 3.1, the authors state

Scaling the filter instead of the image allows the generation of saliency maps of the same size and resolution as the input image.

How is this possible?
From what I have understood, the process of filtering the image is similar to that of a convolution operation, like this:

However, if this is true, shouldn't we get different sized outputs (i.e. saliency maps) for different filter sizes?
I think I am misunderstanding how the filtering process really works in that it actually differs from a CNN. I would highly appreciate any insight on the above.
Note: This is a follow-up to this question.

I didn't read the paper, but the verb "to scale" may not be used here to mean "to reduce the size of the filter" but "to normalize the values of the filter", but I could well be wrong. So, my question to you is: what do they mean by "scaling the filter"? In any case, the size of the output of a convolutional layer (often called feature map) depends not only on the filter's dimensions but also on the stride and if the image is padded.

@nbro In the paper, it is stated that "A change in scale is affected by scaling the region $R_{2}$ instead of scaling the image." This leads me to believe that "scaling" does indicate a change in the size of the filter. As for strides and padding, there is no mention of such and so I am not inclined to make assumptions on their existence.


text:
Masked Autoencoder Structure
In the following structure when we use MADE due to the constraints for making a masked autoencoder, it seems some inputs do not have any connection to the next layer, and also there is the output that does not have a connection to the previous layer!
Can someone clarify?

Hello. Welcome to Artificial Intelligence Stack Exchange. Rather than asking "Can someone clarify?", can you please reformulate this in order to ask a more specific question? Is your question "In MADE, is it true that some inputs are not connected to the next layer and some outputs not connected to the previous layer?" Moreover, can you please clarify where you took this screenshot from and provide a paper or article that introduces this "MADE" (for people not familiar with it).

Hello, Thank you for your consideration, this is the official paper link: https://arxiv.org/abs/1502.03509 @nbro


text:
Is the graph considered as overfit?
I have a training dataset of 2000 images and 500 images for validation. I have executed 50 epochs, however I realized that my graph seems to be different as my accuracy is smaller than my loss. I am not sure on whether my graph is considered as overfitting. If it is, are there other ways to resolve it? I am currently running on Jupiter notebook. Attached below is my code
Edit:
By the way, I have made some changes to my code, such as reducing the dropout to 0.2. I have uploaded the new graph output as shown below. Is it considered as normal?
Code
import numpy as np
import keras
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
import os
import cv2

Training_dir = './datasets/train'

train_datagen = ImageDataGenerator(rotation_range =15,
                         width_shift_range = 0.2,
                         height_shift_range = 0.2,
                         rescale=1./255,
                         shear_range=0.2,
                         zoom_range=0.2,
                         horizontal_flip = True,
                         fill_mode = 'nearest',
                         data_format='channels_last',
                         brightness_range=[0.5, 1.5])

train_gen = train_datagen.flow_from_directory(
    Training_dir,
    batch_size = 64,
    class_mode='binary',
    target_size=(150, 150)
)

Validation_dir = './datasets/val'

val_datagen = ImageDataGenerator(rotation_range =15,
                         width_shift_range = 0.2,
                         height_shift_range = 0.2,
                         rescale=1./255,
                         shear_range=0.2,
                         zoom_range=0.2,
                         horizontal_flip = True,
                         fill_mode = 'nearest',
                         data_format='channels_last',
                         brightness_range=[0.5, 1.5])

validation_gen = val_datagen.flow_from_directory(
    Validation_dir,
    batch_size = 64,
    class_mode = 'binary',
    target_size = (150, 150)
)

imgs = os.listdir(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\train\cat")
imgs1 = os.listdir(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\train\dog")
imgs2 = os.listdir(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\val\dog")
imgs3 = os.listdir(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\val\cat")

for img in imgs:
    img=cv2.imread(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\train\cat" + "\\"+img)
    x = img_to_array(img)
    x = x.reshape((1,) + x.shape)

    i = 0
    for batch in train_datagen.flow (x, batch_size=1, save_to_dir =r'C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\Preview\train\cat', save_prefix ='cat', save_format='jpg'):
        i+=1
        if i>5:
            break

for img in imgs1:
    img=cv2.imread(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\train\dog" + "\\"+img)
    x = img_to_array(img)
    x = x.reshape((1,) + x.shape)

    i = 0
    for batch in train_datagen.flow (x, batch_size=1, save_to_dir =r'C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\Preview\train\dog', save_prefix ='dog', save_format='jpg'):
        i+=1
        if i>5:
            break

for img in imgs2:
    img=cv2.imread(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\val\dog" + "\\"+img)
    x = img_to_array(img)
    x = x.reshape((1,) + x.shape)

    i = 0
    for batch in val_datagen.flow (x, batch_size=1, save_to_dir =r'C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\Preview\Validation\dog', save_prefix ='dog', save_format='jpg'):
        i+=1
        if i>5:
            break

for img in imgs3:
    img=cv2.imread(r"C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\datasets\val\cat" + "\\"+img)
    x = img_to_array(img)
    x = x.reshape((1,) + x.shape)

    i = 0
    for batch in val_datagen.flow (x, batch_size=1, save_to_dir =r'C:\Users\User\Documents\IM4483 Mini Project (Tang Soon Loong Jefferson_U1921181B)\Preview\Validation\cat', save_prefix ='cat', save_format='jpg'):
        i+=1
        if i>5:
            break

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, Dense, MaxPool2D, Flatten, Dropout

'''
SRC : https://www.pyimagesearch.com/2018/12/31/keras-conv2d-and-convolutional-layers/
----------------------------------
STRIDES = The stridesparameter is a 2-tuple of integers, specifying the “step” of the
convolution along the x and y axis of the input volume.
PADDING = Typically, we set the values of the extra pixels to zero 'valid' or 'same'
KERNEL_INITIALIZER = The kernel_initializercontrols the initialization method used to initialize
all values in the Conv2D class prior to actually training the network.

FLATTEN = Return a copy of the array collapsed into one dimension.
DROPOUT = dropout refers to ignoring units (i.e. neurons) during the training
phase of certain set of neurons which is chosen at random. When created, the dropout rate can be specified to the
layer as the probability of setting each input to the layer to zero. TO PREVENT OVER-FITTING.!

DENSE = A Dense layer feeds all outputs from the previous layer
to all its neurons, each neuron providing one output to the next layer.
----------------------------------
'''

model = Sequential()
model.add(Conv2D(16, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal',input_shape=(150, 150, 3)))
model.add(Conv2D(16, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(MaxPool2D((2,2)))
model.add(Dropout(0.2))

model.add(Conv2D(32, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Conv2D(32, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Conv2D(32, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(MaxPool2D((2,2)))
model.add(Dropout(0.2))

model.add(Conv2D(64, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Dropout(0.2))
model.add(Conv2D(64, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Dropout(0.2))
model.add(Conv2D(64, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(MaxPool2D((2,2)))
model.add(Dropout(0.2))

model.add(Conv2D(128, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Dropout(0.2))
model.add(Conv2D(128, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(Dropout(0.2))
model.add(Conv2D(128, (3,3), strides=(1,1), padding='same', activation='relu',kernel_initializer='he_normal'))
model.add(MaxPool2D((2,2)))
model.add(Dropout(0.2))

model.add(Flatten())
model.add(Dense(units=256, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(units=256, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(units=256, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(units=1, activation='sigmoid'))

import tensorflow as tf
check_point_path = './best.h6'
model_checkpoint = tf.keras.callbacks.ModelCheckpoint(
    filepath = check_point_path,
    monitor = 'val_accuracy',
    save_weights_only=False,
    save_best_only=True,
    verbose=1
)

model.compile(optimizer = tf.keras.optimizers.Adam(0.0005,decay=1e-5),
             loss = 'binary_crossentropy',
             metrics = ['acc'])

tb_callback = tf.keras.callbacks.TensorBoard(log_dir="logs/", histogram_freq=1)

print('Num Params : ',model.count_params())

'''
VERBOSE = By setting verbose 0, 1 or 2 you just say how do you want to 'see' the training progress for each epoch.
'''
model_history = model.fit(
    train_gen,
    epochs=50,
    batch_size=128,
    verbose=1,
    callbacks = [tb_callback],
    validation_data=validation_gen
)

%matplotlib inline

import matplotlib.image  as mpimg
import matplotlib.pyplot as plt

#-----------------------------------------------------------
# Retrieve a list of list results on training and test data
# sets for each training epoch
#-----------------------------------------------------------
Acc=model_history.history['acc']
Val_acc=model_history.history['val_acc']
Loss=model_history.history['loss']
Val_loss=model_history.history['val_loss']

epochs=range(len(Acc)) # Get number of epochs

#------------------------------------------------
# Plot training and validation accuracy per epoch
#------------------------------------------------
plt.plot(epochs, Acc, 'r')
plt.plot(epochs, Val_acc, 'b')
plt.title('Training and validation accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend(["Training Accuracy","Validation Accuracy"])
plt.show()

#------------------------------------------------
# Plot training and validation loss per epoch
#------------------------------------------------
plt.plot(epochs, Loss, 'r')
plt.plot(epochs, Val_loss, 'b')
plt.title('Training and validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(["Training Loss","Validation Loss"])
plt.show()

Epochs output
Num Params :  3273585
Epoch 1/50
32/32 [==============================] - 97s 3s/step - loss: 0.7618 - acc: 0.4925 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 2/50
32/32 [==============================] - 70s 2s/step - loss: 0.6916 - acc: 0.5330 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 3/50
32/32 [==============================] - 71s 2s/step - loss: 0.6946 - acc: 0.5250 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 4/50
32/32 [==============================] - 72s 2s/step - loss: 0.6911 - acc: 0.5205 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 5/50
32/32 [==============================] - 73s 2s/step - loss: 0.6949 - acc: 0.5095 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 6/50
32/32 [==============================] - 74s 2s/step - loss: 0.6933 - acc: 0.5170 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 7/50
32/32 [==============================] - 74s 2s/step - loss: 0.6926 - acc: 0.5190 - val_loss: 0.6932 - val_acc: 0.5000
Epoch 8/50
32/32 [==============================] - 72s 2s/step - loss: 0.6936 - acc: 0.5055 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 9/50
32/32 [==============================] - 74s 2s/step - loss: 0.6940 - acc: 0.5120 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 10/50
32/32 [==============================] - 72s 2s/step - loss: 0.6920 - acc: 0.5265 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 11/50
32/32 [==============================] - 73s 2s/step - loss: 0.6929 - acc: 0.4975 - val_loss: 0.6934 - val_acc: 0.5000
Epoch 12/50
32/32 [==============================] - 92s 3s/step - loss: 0.6935 - acc: 0.5035 - val_loss: 0.6933 - val_acc: 0.5000
Epoch 13/50
32/32 [==============================] - 89s 3s/step - loss: 0.6924 - acc: 0.5115 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 14/50
32/32 [==============================] - 72s 2s/step - loss: 0.6929 - acc: 0.5095 - val_loss: 0.6935 - val_acc: 0.5000
Epoch 15/50
32/32 [==============================] - 71s 2s/step - loss: 0.6928 - acc: 0.5175 - val_loss: 0.6940 - val_acc: 0.5000
Epoch 16/50
32/32 [==============================] - 71s 2s/step - loss: 0.6916 - acc: 0.5170 - val_loss: 0.6941 - val_acc: 0.5000
Epoch 17/50
32/32 [==============================] - 71s 2s/step - loss: 0.6922 - acc: 0.5205 - val_loss: 0.6945 - val_acc: 0.5000
Epoch 18/50
32/32 [==============================] - 71s 2s/step - loss: 0.6894 - acc: 0.5280 - val_loss: 0.6949 - val_acc: 0.5000
Epoch 19/50
32/32 [==============================] - 71s 2s/step - loss: 0.6887 - acc: 0.5205 - val_loss: 0.6956 - val_acc: 0.5000
Epoch 20/50
32/32 [==============================] - 72s 2s/step - loss: 0.6870 - acc: 0.5400 - val_loss: 0.6963 - val_acc: 0.5000
Epoch 21/50
32/32 [==============================] - 72s 2s/step - loss: 0.6864 - acc: 0.5380 - val_loss: 0.6974 - val_acc: 0.5000
Epoch 22/50
32/32 [==============================] - 74s 2s/step - loss: 0.6836 - acc: 0.5505 - val_loss: 0.6965 - val_acc: 0.5000
Epoch 23/50
32/32 [==============================] - 74s 2s/step - loss: 0.6722 - acc: 0.5640 - val_loss: 0.6903 - val_acc: 0.5280
Epoch 24/50
32/32 [==============================] - 72s 2s/step - loss: 0.6712 - acc: 0.5755 - val_loss: 0.6980 - val_acc: 0.5100
Epoch 25/50
32/32 [==============================] - 72s 2s/step - loss: 0.6709 - acc: 0.5735 - val_loss: 0.6946 - val_acc: 0.5180
Epoch 26/50
32/32 [==============================] - 74s 2s/step - loss: 0.6687 - acc: 0.5890 - val_loss: 0.7016 - val_acc: 0.5120
Epoch 27/50
32/32 [==============================] - 73s 2s/step - loss: 0.6626 - acc: 0.5825 - val_loss: 0.7043 - val_acc: 0.5060
Epoch 28/50
32/32 [==============================] - 72s 2s/step - loss: 0.6675 - acc: 0.5830 - val_loss: 0.7053 - val_acc: 0.5060
Epoch 29/50
32/32 [==============================] - 72s 2s/step - loss: 0.6580 - acc: 0.5895 - val_loss: 0.7074 - val_acc: 0.5100
Epoch 30/50
32/32 [==============================] - 72s 2s/step - loss: 0.6632 - acc: 0.5875 - val_loss: 0.7036 - val_acc: 0.5200
Epoch 31/50
32/32 [==============================] - 73s 2s/step - loss: 0.6705 - acc: 0.5685 - val_loss: 0.6877 - val_acc: 0.5400
Epoch 32/50
32/32 [==============================] - 73s 2s/step - loss: 0.6595 - acc: 0.5880 - val_loss: 0.6821 - val_acc: 0.5520
Epoch 33/50
32/32 [==============================] - 72s 2s/step - loss: 0.6751 - acc: 0.5600 - val_loss: 0.7052 - val_acc: 0.5020
Epoch 34/50
32/32 [==============================] - 72s 2s/step - loss: 0.6536 - acc: 0.6015 - val_loss: 0.6853 - val_acc: 0.5580
Epoch 35/50
32/32 [==============================] - 72s 2s/step - loss: 0.6549 - acc: 0.6010 - val_loss: 0.6675 - val_acc: 0.5860
Epoch 36/50
32/32 [==============================] - 73s 2s/step - loss: 0.6530 - acc: 0.6050 - val_loss: 0.7023 - val_acc: 0.5160
Epoch 37/50
32/32 [==============================] - 75s 2s/step - loss: 0.6559 - acc: 0.5910 - val_loss: 0.6965 - val_acc: 0.5380
Epoch 38/50
32/32 [==============================] - 73s 2s/step - loss: 0.6423 - acc: 0.6225 - val_loss: 0.6719 - val_acc: 0.5920
Epoch 39/50
32/32 [==============================] - 72s 2s/step - loss: 0.6426 - acc: 0.6135 - val_loss: 0.7164 - val_acc: 0.5260
Epoch 40/50
32/32 [==============================] - 73s 2s/step - loss: 0.6430 - acc: 0.6080 - val_loss: 0.6936 - val_acc: 0.5640
Epoch 41/50
32/32 [==============================] - 72s 2s/step - loss: 0.6440 - acc: 0.5980 - val_loss: 0.6894 - val_acc: 0.5720
Epoch 42/50
32/32 [==============================] - 72s 2s/step - loss: 0.6431 - acc: 0.6135 - val_loss: 0.7083 - val_acc: 0.5560
Epoch 43/50
32/32 [==============================] - 73s 2s/step - loss: 0.6344 - acc: 0.6205 - val_loss: 0.6800 - val_acc: 0.5860
Epoch 44/50
32/32 [==============================] - 72s 2s/step - loss: 0.6687 - acc: 0.5735 - val_loss: 0.6699 - val_acc: 0.5940
Epoch 45/50
32/32 [==============================] - 72s 2s/step - loss: 0.6344 - acc: 0.6190 - val_loss: 0.7070 - val_acc: 0.5620
Epoch 46/50
32/32 [==============================] - 72s 2s/step - loss: 0.6340 - acc: 0.6160 - val_loss: 0.7012 - val_acc: 0.5740
Epoch 47/50
32/32 [==============================] - 74s 2s/step - loss: 0.6424 - acc: 0.5985 - val_loss: 0.7255 - val_acc: 0.5340
Epoch 48/50
32/32 [==============================] - 73s 2s/step - loss: 0.6449 - acc: 0.5985 - val_loss: 0.7120 - val_acc: 0.5420
Epoch 49/50
32/32 [==============================] - 72s 2s/step - loss: 0.6437 - acc: 0.6120 - val_loss: 0.6708 - val_acc: 0.6140
Epoch 50/50
32/32 [==============================] - 73s 2s/step - loss: 0.6497 - acc: 0.6030 - val_loss: 0.7339 - val_acc: 0.5280

Graph output

It's hard to say with that kind of variance. Main issue seems to be high learning rate. You'd better reduce the learning rate (or just use Adam) as the model converges.

Hi, my apologies I forgot to include my full code. I have just made changes to it. At that time, I have included the adam optimizer, but the graph still remains the same

"my accuracy is smaller than my loss" this is like comparing apples to oranges, and not an issue. Overfitting occurs when your training loss continuous to decrease while your validation loss start increasing; nothing of the sort happens here (there is no trend in the graphs), so this is not overfitting.

@desertnaut I see, thanks alot

What is the difference between underfitting and overfitting?

@Joseph See these slides: https://www.deeplearningbook.org/slides/05_ml.pdf


text:
Which AI algorithm to use for identifying API for a specific use from a list of APIs?
We have a legacy code solution in C#. We have to change the code so that it fetches internal data via APIs and not via DB calls.
E.g. if the current code GETS Payment object from DB, we have to replace the logic so that the code calls the GET PAYMENT API instead.
Since there are 100s of code files and multiple DB hits in a single file, doing this manually is not feasible.
I was thinking of building an AI-based tool that would take my code file as input and point me out where I would need to replace the existing code and suggest what API to call at that place.
I have never worked on AI and it would be great if anyone suggests which algorithm to refer for my tool and also how should I proceed with solving the above problem.

Welcome to AI stack exchange. Although an AI solution to your problem might be interesting, with current AI tools this would be far harder than the manual process you are trying to avoid. I would expect some automation using regular expressions to be a workable compromise, and it may save you time overall

Thanks, @NeilSlater for your suggestion. Nearly everyone suggested not choosing AI as it would increase complexity! However, upon doing some google research, I stumbled upon something known as Rule-based ML (RBML). Do you think it can be an option?

I am not sure what RBML is (you could give a link and I will check - I'd guess something similar to decision trees). The issue will be with the ML part = "Machine Learning". ML works by processing a number of examples, using them to build a function that relates some input (e.g. text of your code) into a value (e.g. true/false on whether to replace with your new call). To do so it typically needs large numbers of examples. Chances are your entire code base does not contain enough examples, so your first task would be to complete the change manually.

https://en.wikipedia.org/wiki/Rule-based_machine_learning Here's the link. I got your point @NeilSlater. Using ML would need huge data and thus the approach would be too complex. Thanks! If you feel I should research the topics in the above link, kindly let me know. Else I'll proceed with traditional automation :)

OK so RBML is a broad category of ML. It would include decision trees as one example - these work by iteratively applying boolean rules that best split a group of examples into similar cases. I think my other comments still stand. Unless someone has already written a useful AI tool to help you find the code you need to re-factor, then creating one and training it on your data yourself will be a large time sink. There are reasons you might do it, e.g. to learn about AI for coding, but saving time on your task is not one of those reasons.

Got it @NeilSlater. Thank You so much! :)


text:
Why am I getting a very small number as CNN prediction?
I created a CNN using Tensorflow to identify pneumonia and sometimes it returns a very small number as a prediction. why is this happening?
I have attached the link for the dataset
Here I how I process and load the data.
from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator( rescale = 1.0/255. )
val_datagen  = ImageDataGenerator( rescale = 1.0/255. )
test_datagen = ImageDataGenerator( rescale = 1.0/255. )

train_generator = train_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/train/',
                                                    batch_size=20,
                                                    class_mode='binary',
                                                    target_size=(350, 350))

validation_generator =  val_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/val/',
                                                         batch_size=20,
                                                         class_mode  = 'binary',
                                                         target_size = (350, 350))
test_generator = test_datagen.flow_from_directory('/kaggle/input/chest-xray-pneumonia/chest_xray/chest_xray/test/',
                                                         batch_size=20,
                                                         class_mode  = 'binary',
                                                         target_size = (350, 350))

And here the Model, compile and fit functions
import tensorflow as tf

model = tf.keras.models.Sequential([
    # Note the input shape is the desired size of the image 150x150 with 3 bytes color
    tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(350, 350, 3)),
    tf.keras.layers.MaxPooling2D(2,2),
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    # Flatten the results to feed into a DNN
    tf.keras.layers.Flatten(),
    # 512 neuron hidden layer
    tf.keras.layers.Dense(1024, activation='relu'),
    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')
    tf.keras.layers.Dense(1, activation='sigmoid')
])

compile the model
from tensorflow.keras.optimizers import RMSprop

model.compile(optimizer=RMSprop(learning_rate=0.001),
              loss='binary_crossentropy',
              metrics = ['accuracy'])

model fit
history = model.fit(train_generator,
                              validation_data=validation_generator,
                              steps_per_epoch=200,
                              epochs=2000,
                              validation_steps=200,
                              callbacks=[callbacks],
                              verbose=2)

The evaluation metrics as followings,
loss: 0.2351 - accuracy: 0.9847
The prediction shows a big negative number for the negative pneumonia, and for positive it shows more than .50.
I have two equations?

why I getting a very small number as 2.xxxx * 10e-20

why I can't get the following values as null?
val_acc  = history.history[ 'val_accuracy' ]
val_loss = history.history['val_loss' ]

I am a still student in machine learning and I really appreciate your effort to solve my question.

Hello. Welcome to Artificial Intelligence Stack Exchange. It seems to me that this is or could be a programming issue. Let me know if I am wrong or right. Generally, questions about programming problems are off-topic here and should probably be asked on Stack Overflow. Here, we focus on the theoretical, philosophical, social and historical aspects of Artificial Intelligence. Please, take the time to read our on-topic page. So, if you think this is not a programming problem, you should edit your post to clarify that.

Thank you so much let me know and I will remove this question from here and I am really sorry!!!!


text:
Is there a way to adapt Particle Swarm Optimization to an incremental/online learning setting?
As stated in the title, is there a way to adapt PSO to an online scenario where new data samples arrive continuously?
In more detail: suppose that I have a classifier with several parameters for which the optimal values are to be chosen automatically, instead of being predefined. I want to use PSO to select the parameters. I know this is doable in a static scenario, where the data set is fixed. However, if new data samples arrive over time (and in large amounts), is there a way to make PSO work on such dynamic data streams?
Also, I am open to other ways to implement self-adaptive parameters. PSO is a possible choice but if it's not possible I'd love to hear your suggestions about other approaches.

Hello. By "parameters", do you mean "hyper-parameters" (e.g. learning rate in stochastic gradient descent)? PSO is sometimes used for hyper-parameter optimization and you tagged the question with that tag, so that's why I am asking this question. Which parameters are you optimizing specifically and would you need to optimize in an online setting (and why)?

a latest survey may hint you the existing approaches to using evolutionary algorithm (e.g., pso) to solve dynamic optimization problem : https://doi.org/10.1109/TEVC.2021.3060014

Thank you both for your comments! Yes Sanyou I'll take a look at it thanks for the reference! And nbro specifically I want to perform clustering on data streams and the algorithm I want to use has several hyper-parameters that are normally predefined based on some pre-analysis of a data stream. I want to set the hyper-parameters automatically instead. So that's why I'm trying to see if PSO is applicable in this case.


text:
What to predict in a limited transaction dataset?
I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification.
The columns are as follow:

Transaction ID
Quantity purchased
Product name
Coupon code
Transaction Date
City (where transaction was made)
Delivery fee (if any)
Total amount spent

I am having a rough time figuring out what to predict using regression or classification given only these columns.
i.e: Given a full row how much is the total spent... etc
In other words I need help deciding what will be the label of my dataset and what would be the reason behind choosing that label.

Do you want to know what is the output variable in the dataset?

@hanugm Essentially I am looking to predict 1 column from the others so what would be the best label given this dataset as it seems you can't really predict any of them as its only transaction with no mention of customer data.

Oh, you want to decide the classifying label without data.

@hanugm I have the data but the columns seem very unrelated and the task I was given says I need to predict one of the columns given the others which looks very unlikely to me.


text:
In general, when are the normal, uniform and zero initializers used?
I came across a Conv2D layer in a fully convolutional network, which used a kernel_initializer='zero' for regression. Why is a kernel_initializer of 'zero' used here?
In general, when are 'normal', 'uniform' and 'zero' initializers used?

Hello. Welcome to Artificial Intelligence Stack Exchange. Given that answer to your second question may also contain the answer to the first one, I put that more general question in the title. I would recommend that you provide the link to the TF implementation that uses the zero initializers for the kernel in the case of regression.

Hello, @nbro . I can't provide the link to the implementation, but the network I was looking at was a region proposal network. One of the outputs was classification (background vs non-background) and the other was regression (bounding box coordinates). The regression part of it had the zero kernel initialization.


text:
What's the best way to train data with unbalanced targets?
Suppose I have data I want to use for supervised learning, but there is a pretty bad target/class/labels imbalance. Should I:

Limit the size of the training set to make sure there is a flat target/class balance distribution (the training set is designed such that there is an equal number of training samples for each class based on splitting the lowest-occuring class as high as possible). For example, if my lowest-occuring class appears only 50 times in my data, and I want an 80-20 train-test split, then I decide I take 40 of the samples for training, and for an even target balance, take 40 samples for all other samples in training, even if the highiest-occuring class appears 100,000 times, for instance.

Ignore target balance and just focus on the ratio for the train and testing split. So, if it's 80-20, take 40 of the samples out of 50 for my lowest-occuring class, and 80% of 100,000 for my highest occuring class, and so on.

Something else?

Let's suppose I can't just get more data. I know there's some stuff to be said regarding undersampling and oversampling, but what can I do to tell if either one is working better if model accuracy might be disingenuous?

What I would advise is to try and get a representative training set for your test and application set. If your eventual application is also heavily imbalanced, then it makes sense for your train/test sets to also be unbalanced. You have to realise that if your application also is unbalanced, it is much more important to correctly classify/predict the biggest category than the smaller ones, so it makes sense to represent this in the train/test set. Do pay attention to also make your test set and training set 'similarly unbalanced'

If your final application data is balanced, I would probably try to get my training/test sets to be balanced as well


text:
If the model always underfits, do I really need a larger model?
I train my neural network on random points generated for a data set that theoretically consists of approximately $1.8 * 10^{39}$ elements. I sample (generate) tens of thousands of random points on each epoch with uniform distribution. For every model I try, it appears that it cannot get past $10-12\%$ accuracy on the data, even if I increase the size of the model to over ten million parameters.
Each feature of the data set is of two Rubik's cube positions, and the corresponding label is the first move of a possible solution to solve from the first provided position to the second provided position within twenty moves.
So, it's a classification model for $18$ distinct classes, one for each of the possible moves on a Rubik's cube.  Seeing that it has $12\%$ accuracy (being greater than $1/18 \approx 5.6\%$) is nice because it does mean that it is learning something, just not enough.
I also notice that loss tends to go down to a hard minimum over many epochs, but accuracy stops increasing after only around ten epochs.  On epoch 36, it reached a loss of $2.713$, and it repeatedly comes back down to $2.713$, but never any lower even after 2000 epochs.
I concatenated a convolutional layer with a fully connected layer to use it as the first layer of the model.  Convolutional layers might not work for this as well as I'd hope, so I throw in the fully connected layer as a safeguard.  Some Keras code below:
inp_cubes = Input((2,6,3,3,1))

x = TimeDistributed(TimeDistributed(Conv2D(1024,(2,2),(1,1),activation='relu')))(inp_cubes)

output_face_conv = TimeDistributed(TimeDistributed(Flatten()))(x)
flatten_inp_cubes = TimeDistributed(TimeDistributed(Flatten()))(inp_cubes)
x = TimeDistributed(TimeDistributed(Concatenate()))((output_face_conv, flatten_inp_cubes))

x = TimeDistributed(TimeDistributed(Dense(1024,'relu')))(x)
x = TimeDistributed(TimeDistributed(Dropout(0.3)))(x)
x = TimeDistributed(TimeDistributed(Dense(1024,'relu')))(x)
x = TimeDistributed(TimeDistributed(Dropout(0.3)))(x)
x = TimeDistributed(TimeDistributed(Dense(1024,'relu')))(x)
x = TimeDistributed(TimeDistributed(Dropout(0.3)))(x)
x = TimeDistributed(TimeDistributed(Dense(1024,'relu')))(x)
x = TimeDistributed(TimeDistributed(Dropout(0.3)))(x) # face logits
x = TimeDistributed(Flatten())(x)
x = TimeDistributed(Dense(1024,'relu'))(x)
x = TimeDistributed(Dropout(0.3))(x)
x = TimeDistributed(Dense(1024,'relu'))(x)
x = TimeDistributed(Dropout(0.3))(x)
x = TimeDistributed(Dense(1024,'relu'))(x)
x = TimeDistributed(Dropout(0.3))(x) # cube logits
x = Flatten()(x)
x = Dense(1024,'relu')(x)
x = Dropout(0.3)(x)
x = Dense(1024,'relu')(x)
x = Dropout(0.3)(x)
x = Dense(1024,'relu')(x)

outp_move = Dense(18,'softmax')(x) # solution logits

I tried using only one of the two types of input layers separately, and nothing quite worked.
Loss is measured as categorical cross-entropy.  I make use of time-distributed layers so that each of the two Rubik's cube positions from the input is processed equivalently, except when determining how they relate.  I'm making sure to scale my data, and all that stuff.  It really seems like this should just work, but it doesn't.
Is there any way to increase the model's performance without using hundreds of millions of parameters, or is that actually necessary?
I would have thought that there would be some relatively simple correlation between positions and solutions, although it's hard for us to see as humans, so maybe this comes down to the Cayley diagram of the Rubik's cube group being innately random, as though they're prime numbers or something.
EDIT: I guess I really did just need a bigger neural network. This new on has 75 million parameters. The second image shows how that model is able to learn the data set quite easily. It takes a long time to process, though.

I have a feeling that underfitting is not the problem here. Are you sure that there is a pattern for the model to learn? The target/data relation that you are giving (first move of a possible solution to solve from the first provided position to the second provided position within twenty moves) seems pretty ambiguous and I am not sure that it is even unique. How about training the model first with a simple one move relation and expanding further?

I did that. The problem is solved now. It just needed 75 million parameters. Even with 25 million on single-move solutions, it couldn’t learn. But now it’s at 30% accuracy overall, which is actually pretty impressive when there are 1.8 duodecillion possible inputs. So yeah, it was underfitting. Probably, the model architect is not perfect, but I can’t quite think of anything better for now. At least it’s working now! It just has to be very deep. Also, the “the first move of a possible solution” does not imply uniqueness. There are always many “fast” solutions, and this model will generalize.


text:
Is it possible to find a good neural network structure without training it?
Neural networks consist of so many parameters. Researchers could create as many possible neural networks as they wish. So I want to ask a general question. Could we devise an evolutionary algorithm which learns an efficient structure without optimization?
Are there some important works in this area?
If we look at sparse neural networks, it seems that there are so many topologies that perform as well as a dense network.
So a single task has so many solutions which differ slightly. So getting rid of optimization for many problems shouldn't be hard at all.
Edit: I add some more information. I want to know whether we could find sparse topologies by mutating them like adding layers and changing the connections without optimizing the loss function directly?

I was thinking about this subject after reading this paper: http://ceur-ws.org/Vol-2993/paper-10.pdf. The paper suggests that it might be possible to create generative neural networks which don't need backpropagation or any optimization. If it could be proven that we only need to devise an evolutionary algorithm for many tasks, we no longer need resource-heavy optimization.

Are you aware of hyper-parameter optimization techniques? Moreover, what do you mean by "without optimization"? Evolutionary algorithms are also optimization algorithms, but you're proposing to use EAs to solve this problem, but then you say "without optimiztion", so it's not clear what you have in mind. Could you please edit your post to clarify what you're really looking for?

Please, put your specific question in the title. Your question is not the one in the title "Is it possible to find a good neural network structure without training it?". Your question seems to be "could we find sparse topologies by mutating them like adding layers and changing the connections without optimizing the loss function directly?". Having said that, it seems that your question is answered here and here. Let me know if that's the case. If not, edit your post to clarify why not.

Don't forget to upvote those posts if you think they are useful! Meanwhile, I closed this post. Feel free to ask another question, if you were looking for something else, but please be clearer and specific.


text:
Augmented an Image with other data when training CNN
In the typical RL/MDP framework, I have offline data of $(s,a,r,s')$ of expert Atari gameplay.
I'm looking to train a CNN to predict $r$ based on $(s, a)$.
The states are represented by a $4 \times 84 \times 84$ image of the Atari screen, where 4 represents 4 sequential frames, and $84 \times 84$ is the size of the image. The action is an integer from 0 to 3.
I'm not sure how best to merge these two inputs $(s, a)$ together. How should I incorporate the action into the CNN?

So, are you trying to create some kind of inverse RL algorithm?

The conventional way is to have a network with output size of 1x4 (for the 4 actions), to predict the output corresponding to each actions. Since action space is small, this would be a good bet. If you really want to merge S and A, you might want to encode A as onehot and concatenate with 1D feature vectors from the CNN.

My question can be taken out of the RL context for simplicity. Imagine if I had just images (states in the RL example), and additional information, say integers (actions in the RL example), giving a slight hint as to how "good" these images are (rewards in the RL example). How would you add the "additional information" to the CNN? Currently, I just add it in the last linear layer as an additional feature. Are there any obvious pitfalls I am making?


text:
What I Should Do to Reduce Solution Size for Simulated Annealing Algorithm?
I am trying to find the best solution for radar placement problem with using multi objective simulated annealing algorithm.
So there is an area (in real map) and I want to put minimum count of radar as possible with the maximum area coverage. I am using multi-objective simulated annealing algorithm to do that and in small maps it works well, but when the map size increases the solution size also increases and updating it for better solutions. It becames hard to find a good solution at the end and the calculation times increases a lot.
The problem is here. What should I do to reduce the solutin size? For example my real world map is 250x500 and the solution size is 250x500 = 125000. It is hard to seperate radars randomly at on solution with this high size, it doesn't work well and consumes a lot of time. How can I reduce that solution size? For example, there are 125k different possible place to put a radar, but if it become 1k or 10k different place, it becomes easier to get a good result.

so its just placement binary matrix (or vector)? supposedly sparse matrix, one simple trick crossed my mind is segment the solution into 16 bits and represent that 16 bits as integer [0.2^16-1]. In other words, find other solution representation, maybe binary vector compression also can come in handy.

@Sanyou my solution is a binary vector like this: [0,0,0,1,0,0,0,0,0,1,......,0], I could not understand well how can I represent this with 16 bit representation.

every N bit u turn it into [0,2^N-1] integer. example: 2x4 dimensions, and N=4. x = [0,0,1,0,   1,0,0,1]. x' = [2,9]. 0,0,1,0 => 2. 1,0,0,1 => 9

@Sanyou iy my solutions the bits represents just one place and 1=true, 0=false. if it is true there is placed a radar. So "1" represents "there is placed a radar". After all I look at where are 1's and calculate the coverages of areas. Your suggestion is logicfull but doesn't solve my problem.

maybe you missed that besides encoding (binary vector into sequence of integer), you also can decode (sequence of integer into binary vector). Now if you can define simulated annealing operation for the sequence of integer (or any other solution representation you choose), then your optimization process will be the same as the previous one but with smaller solution's size.


text:
Impact of imbalanced dataset on CNN model performance
I trained a 1D CNN model to model bacterial plate count based on time series data of water temperature. Bacterial place count is numerical, based on which I created two category variables, namely "Label 1" and "Label 2". There are two possible values for "Label 1" - "risky" and "non-risky" based on a threshold of 100 for bacterial place count . Eg. 1 if Bacterial place count >100; 0 otherwise. "Label 2" is (1 - value of "Label 2"). This gives a balanced "Label 1" data of 43.1% for 1 and 56.9% for 0. The model observed an 80% accuracy for prediction.
Then I wanted to see if changing the threshold would change the performance of the model. So I increased the threshold to 500 and trained the model, The ratio of 1 and 0 for "Label 1" is now 12.5% for 1 and 87.5% for 0. And at 1000 threshold, the ratio is 8.1% for 1 and 91.9% for 0.
Then I calculated the confusion matrix results for the predictions at each threshold, as below:
Confusion matrix using 100 threshold:

Confusion matrix using 500 threshold:

Then I further increased the threshold to 1000:

Calculated Results:

**Precision**
# 100 threshold
151/(151+246) = 38%

# 500 threshold
85/(85+30) = 74%

# 1000 threshold
47/(47+12) = 79%

**Recall**
# 100 threshold
151/(151+23) = 86%

# 500 threshold
85/(85+159) = 35%

# 1000 threshold
47/(47+127) = 27%

**F1-score**
# 100 threshold
2*(0.38*0.86)/(0.38+0.86) # 52.7%

# 500 threshold
2*(0.35*0.74)/(0.35+0.74) # 47.5%

# 1000 threshold
2*(0.79*0.27)/(0.79+0.27) # 40.2%

Something I observed:

during training, there were consecutive training batches getting the same validation accuracy, for instance:

83/83 [==============================] - 1s 11ms/step - loss: 0.4065 - acc: 0.8347 - val_loss: 0.4527 - val_acc: 0.8152
Epoch 81/120
83/83 [==============================] - 1s 10ms/step - loss: 0.3943 - acc: 0.8552 - val_loss: 0.4350 - val_acc: 0.8152
Epoch 82/120
83/83 [==============================] - 1s 10ms/step - loss: 0.3800 - acc: 0.8408 - val_loss: 0.4601 - val_acc: 0.8152
Epoch 83/120
83/83 [==============================] - 1s 11ms/step - loss: 0.4026 - acc: 0.8396 - val_loss: 0.4351 - val_acc: 0.8152

I wonder if the model stops to learn as the data is highly imbalanced and so that the model would achieve higher accuracy simply by always predict the value with the higher weight. Is this a sign of the model freezing/not learning?
The graph looks like this:

And my main questions are, does the numerical threshold used to define the classes able to have a significant impact on the accuracy level of the model? How do we interpret the confusion matrix and choose the model which gives the best performance? What are the metrics we should be considering?
Any contribution is much appreciated.

Not sure I understand your first question (plus, you should really not ask multiple questions per post); isn't it obvious from your own experiments that the choice of the threshold can indeed have a (very) significant effect on the performance metrics?

Hi @desertnaut thanks for the reply. Indeed the confusion matrix results are different. What I don't understand is if one threshold is necessarily better than the others. If yes, what are the criteria to select the best and the different consequences. My understanding is that precision and recall represent a trade-off between quality and quantity, since "Precision can be seen as a measure of quality, and recall as a measure of quantity". Also found this link but I'm not sure how exactly all this applies to my case.


text:
Are Neural Networks only really useful for Image Classification?
Let's say I wanted to predict whether someone was Male or Female based on what they answered to certain survey questions.
I can see something like a Random Forest or KNN being useful here, but is there any reasons you'd apply a Neural Network? Or would it just be overkill?

Random forest would probably do fine enough. KNN maybe not as well if survey questions have discrete data - continuous data may also be fine, but i don't imagine it will be as robust as random forests. With a lot of data, neural networks would do very well though. Only concern with using neural nets would be overparameterizing and not having enough data.
I would argue that it's not really an overkill if you have enough data and don't over parameterize. Generally, you don't know which model you build is more generalizable unless you try and see how it does on held out test sets. Neural nets can definitely applied in more fields than just computer vision.

Thank you. That makes sense. I guess I'm just seeing most applications of Neural Nets revolving around image classification (but I'm maybe just looking in the wrong place). Do you mind elaborating on why KNN wouldn't be as robust?

A distance in discrete data is not well defined usually (ofc you can define it, but it may not make sense for 0/1 variables). I don't think KNN would be as good for continuous data but that would depend on your data. If doing image class, imgs of dogs to other dogs may look similar (there is still large variation still to make this questionable based on distance measure - maybe color plays too much of a role) vs dogs to cats. If survey questions, to me there would be more trouble interpreting distances of questions meaningfully so KNN may not do as well

a combination of discrete and continuous variables would further complicate because of distances to resolve the different data types and meanings of distances. Maybe you interpret a discrete variable as continuous , but it may not lend itself to an ordinal scale


text:
Confusion about reward in reinforcement learning
I noticed that there is no single "reward" convention in RL community and as
a new RL learner this confuses me a lot.
For example, it is not clear to me if
Case 1:  reward is a state property (shown next to a state, see example 1)
Case 2: reward is a state transition property (shown on the transition arcs, see example 2).
It is not clear also how you defined "returns" in both cases.
Ex1:

Ex2:

@ nbro,   Thank you. It answers partially: to see their differences. What left is, how do you define returns in each case? I mean how  do you define returns separately for  cases of  using R(s), R(s,a) and R(s, a. a')?

Returns are defined exactly the same in all cases - the discounted sum of rewards from any given timestep onwards. The form of the reward function, or its distribution doesn't change that at all. I.e. $G_t = \sum_{k=0}^{\infty} \gamma^k R_{t+1+k}$

If you try to calculate the expectation of that return given a policy, the transition model, and one of the reward functions, then you might care a little about the form of the reward function, but only because you could move some forms of the function out of the sums that contain them for efficiency.

@Neil Slater, what confuses me is this: for example, if you have a given sequence in ex1 (such as Class1-Class2-Class3-Pass-Sleep) and want to calculate the return associated to this sequence, you need the rewards associated to arcs (R(s, a ,s')) in my opinion, not the rewards associated to states (R(s)). But in David Silver's notes (this is where ex1 is taken), he calculates  returns for a sequence  from rewards associated to states (from R(s)). This confuses me.

You can just associate the rewards to the arcs if you want, just copy all the rewards associated with states in ex1 to actions leading away from them (IIRC David Silver uses the reward values when leaving the associated states, i.e. they are considered to be rewards accrued from completing a time step in that state). It is functionally the same, because using $R(s)$ is a shortcut for stating $R(s,a,s')$ is the same for all $a,s'$ as per the duplicate question


text:
What did Bronstein et al 2021 mean by "conductance"?
On page 24 of Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges they make the following statement:

An essential aspect of Fourier transforms is that they reveal global properties of the signal and the domain, such as smoothness and conductance.

What do they mean by "conductance" here?

I haven't time to read the paper, but Conductance in graph theory controls how fast a random walk on G converges to its stationary distribution., is this paragraph in any related to such? I don't have the context but it's my best guess. In terms of signals, the only "conductance" I could find would relate to electrical conductance, but I don't see how that would be relevant here unless the paper went on quite a tangent

@Recessive Thank you! That is a plausible interpretation.