text:
Can AI be used to reverse engineer a black box?
A while back I posted on the Reverse Engineering site about an audio DSP system whose designer had passed away and whose manufacturer no longer had source code (but the question was deleted). Basically, the audio filter settings are passed from a Windows program to the DSP device presumably as coefficients and then generic descriptions of those filters (boost/cut, frequency and bandwidth) are passed back from the box to the software - but only if it somehow recognizes the filter setting.
I want to be able to generate the filter settings separately from the manufacturer software, so I need to know how they are calculated. I've not been able to deduce how this is structured from observing the USB communication that I've gathered. So, I wonder if AI could do this.
How would I go about creating an AI to send commands to the box (I know how to communicate with the box and have a framework for how these types of commands are phrased) and then look at the responses to either further decode the system and/or create an algorithm for creating filters?
The communication with the DSP mixer box is basically via "Serial" commands and although it uses a USB port, there is a significant bottleneck inside the command control system in the mixer box. Any attempts to reverse engineer may encounter problems based on the sheer amount of time that it would take to compile enough data. Or not.

By "reverse engineer" do you mean you want to know exactly how it works? Or just make another thing which you're still not quite sure what it does but you can get the same output? Most 'AI' is still a black box, even to those who made it.

@ Lio, I seek 2 potential outcomes - One is that I can decipher the human based design and the 'recipe' will make complete sense. The other is that an algorithm (that I likely don't understand) can be made that will predict the behavior, essentially giving me a recipe that I don't necessarily understand.

@ Dutta, I apologize, I have not determined the exact behavior of the device.  But I do know that if it is sent a filter setting from the existing software, it can be queried about the filter and a pre-defined description is returned. But other filter shapes are possible that are not predefined as measured by sending an audio signal through the device.  Brute force: create a large "dictionary" of inputs that result in the pre-defined outputs. Programming would be a lookup.. But I would like to find a mathematical way to "calculate" the result and then arbitrary responses would be possible.

Yes this is entirely possible. As was previously mentioned, complex connectionist systems are often thought of as black boxes(despite us being able to "look in" the box given enough computation and analysis) because of the difficulty in understanding learning and the networks ultimate decision making.
Here, we can model the problem as such: given an input of filter settings(and presumably some information about the audio), predict the target descriptors as an output. All you really need to do is generate a dataset from the program and then train it in a multi-label classification context to predict the output descriptors.

I can certainly write some code to gather a dataset. Bsicaly "Brute Force" in a previous comment. In fact, is any AI necessary? I could simply create a lookup table using the first discovered input that results in a given predefined output. In this case my use would be limited to the predefined filters. But can AI somehow do this better?  More efficiently? Predict undefined outcomes? And how do I begin? I assume there is some kind "seed" that is started and then as it "learns" it changes into the machine that I want? I suppose the AI itself becomes the "algorithm" that I'm looking for...

@chmedly depending on the number of input/output pairs, it very well could be easier to construct a lookup table. However if the space is not too large this would more than likely be the quickest and most effective. To start create a dataset, if you are able to capture all pairs without too large a memory/computational cost than just create a table. Otherwise construct a set of a large portion of pairs and then convert the categorical data into scalar values an algorithm can use.

@chmedly A lookup table is entirely sufficient to deal with small, deterministic input/output spaces where you can enumerate every possible mapping that your system should handle. The problem is, a table cannot generalize at all, so it can't generate useful output for any inputs that aren't in the table. Memorizing input-output pairs is trivial, much of AI is about building systems that generalize well to similar (but not identical) inputs that have never seen before.

You definitely could -technically- use AI (advanced informatics) techniques like in the BinSec binary analyzer (static analyzer of binary code).
You might be forbidden legally to do so. Check with your lawyer.
Contact me by email for more information (on the technical level).


text:
When could a linear discriminant give excellent or possibly even the optimal classification accuracy?
I am actually reading the linear classification. There is a question in the question set behind the chapter in the book as follows:

Sketch two multimodal distributions for which a linear discriminant could give excellent or possibly even the optimal classification accuracy.

I have no idea about how to get the optimal solution on linear classification, any ideas?

A linear model has a linear decision boundary. So in the case of the question, you need to draw two multimodal distributions whose domains do not overlap at all and then you can just say the linear model would should all to the right of some number would be class 1 and all to the left would be class 2

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How should we pad an image to be fed in a CNN?
As everyone experienced in deep learning might know, in an image classification problem we normally add borders to images then resize it to the input size of a CNN network. The reason of doing this is to keep aspect ratio of the original image and retain it's information.
I have seen people fill black (0 pixel value for each channel), gray (127 pixel value for each channel), or random value generated from gaussian distribution to the border.
My question is, is there any proof that which of these is correct?

If the computational components of the forward feed through the network have no curvature, which is normally the case in a sum of products, then it can be proven that any constant pixel value is equivalent in terms of effect on convergence results.  We wouldn't expect a proof for that, since it would be too trivial to spend time writing up for publication.  In general, functioning vision systems have feed forward computational components with curvature, so the padding is likely significant.
Even the convolutional layers may have activation functions or something even more complex going forward, as noted in Gauge Equivariant Convolutional Networks and the Icosahedral CNN (Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling, 2019).
If purely stochastic values with value distributions like that of the un-padded coordinates are used, it may be possible to prove that some gain is made, but none appeared in a few academic article searches just made.  Not surprisingly, there are many proofs regarding the properties of various message padding strategies for cryptography.
Short of the inclusion of thermal or quantum noise acquisition devices in VLSI circuitry and exposure of those devices in software, purely stochastic values cannot be generated.  This leaves the risk of a learning approach expected to extract features from frames learning features of the pseudo-random noise generator used to pad.
The answer is that none are universally correct and there appears to be much work to do in proving advantages between different techniques in as many cases as such advantages can be proven.

I've more often seen image resizing than padding to be honest and tend to resize the images. Maybe it's because datasets I've used have images with near equal aspect ratios.
One major exception was when I worked with MR images. These were orthogonal and it would be wrong to mess up the aspect ratio. However, in this domain images have black borders everywhere, so a zero-padding was easy to apply.
The most common use of padding I've seen is for data augmentations (to fill values gone due to translations, rotations, shifting etc.). In this regard, I've used many types of paddings (constant value, random value, 'same' padding, mirrored padding etc.) The best I've found to empirically work is zero-padding but I don't think that you'll ever find a proof for this. I like to think of it as a hyperparameter; different padding strategies maybe work better for different tasks. Though I think that zero-padding is the safest (there is a small chance of messing things up).

I agree with the point that this question depends on the nature of dataset. I'm working on faces at the moment, so losing the aspect ratio by simply resize would potentially disturb the learning process. I could even do shearing instead. However, there seems to be many ways and I couldn't find any experimental results comparing different strategies. Anyway, thanks for the reply!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What are the loss functions used in teacher-student learning models?
I am not sure what are the common loss functions people usually use when training a student in a teacher-student learning model. Any insight on this is appreciated.

You'll have to clarify what you mean by teacher-student. Please link papers in which you were introduced to this term.


text:
What are the most common methods to enable neural networks to adapt to changing environments?
For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the environment.
What are the most common methods to enable neural networks to quickly adapt to the changing environment for supervised learning? Could somebody provide a link to a good review article?

For the vast majority of cases where you have a dynamic(and assumed non-linear) relationship between your input and output, you would not use modified architecture. You would simply retrain on the new data.
In some cases, based on domain knowledge or intuition, one might put a "weight" on the new data to increase or decrease its importance relative to previous data.
There are some attempts(mostly by those studying one-shot learning) to create NNs that quickly fit to new data effectively with only a few samples. However, most of these are not ready for anything resembling real-world problems(particularly on tabular data).


text:
Can Microsoft's cognitive service find similar person in a set of images without using the face service?
I need to create an application that can detect if a person X entered as an input exists in an image set and return as output all the images in which the person X exists. The problem is that the pictures do not only contain people's faces, they also contain pictures taken from behind.
Is it possible to use Microsoft's cognitive services? If not, is there another solution to allow the realization of this application?


text:
How are the observations stored in the RNN that encodes the state?
I am a bit confused about observations in RL systems which use RNN to encode the state. I read a few papers like this and this. If I were to use a sequence of raw observations (or features) as an input to RNN for encoding the state of the system, I cannot change the weights of my network in the middle of the episode. Is that correct? Otherwise, the hidden state vectors will be different when the weights are changed.
Does that mean that the use of RNN in RL has to store the entire episode before the weights can be changed?
How does then one take into account the hidden states in RNN for RL? Are there any good tutorials on RNN-RL?

This research question seems to be analyzed in further details here (section 3) - https://openreview.net/pdf?id=r1lyTjAqYX.
Usually, a sequence is taken as a state to be fed into RNN to compute the final hidden state. One can then ask what initial state should the RNN be seeded with? This paper analyzes three methods with respect to the seed -

zero initialization: When the RNN is initialized with the zero state
burn-in: when the sequence is prepended by some preceding observations for RNN to learn a good initial state
storing the initial hidden state: When the hidden state at the beginning of the sequence is stored


text:
How to make deepfake video without a fancy PC?
Is there any way to make deepfake videos without a fancy computer? For example, run the DeepFaceLab on a website so your own computer won't get involved?

Please explain your usecase. On what videos are you planning to apply deepfake?

Yes. There are services that provide free environment to run jupyter notebooks for research purposes (with GPU included, which is crucial for neural networks) - such as Google Colaboratory and Kaggle Kernels. Although they limit how long your computation may run (12 and 6 hours accordingly), which adds some difficulties to the process, although I think it is possible to bypass these restrictions.

Take a look at using AWS You can spin up an instance with as much processing power as you need and by using their pre-built images it will already be preconfigured with a lot of the packages etc you might need for any kind of ML.
I see you wanted to use DeepFaceLab which I guess requires some kind of GUI so unsure if this is suitable for your requirements but check it out, it seems to be the best way to perform high-processing machine learning without the fancy computer


text:
Is there a neural network that can output a unit vector that is parallel to the input vector?
I'm wondering if there is a NN that can achieve the following task:
Output a unit vector that is parallel to the input vector. i.e., input a vector $\mathbf{v}\in\mathbb{R}^d$, output $\mathbf{v}/\|\mathbf{v}\|$. The dimension $d$ can be fixed, say $2$.
To achieve this, it seems to me that we need to use NN to do three functions: square, square-root, and division. But I don't know if a NN can do all of these.


text:
Is the agent aware of a possible different set of actions for each state?
I have a use case where the set of actions is different for different states. Is the agent aware of what actions are valid for each state, or is the agent only aware of the entire action space (in which case I guess the environment needs to discard invalid actions)?
I presume the answer is yes, but I would like to confirm.

Here is a related question.

This is actually an implementation choice, and will depend on how you chose to represent the agent's model of the function that maps from states to actions.
If you explicitly represent the entire state space, as you might chose to do with simple benchmark problems that you solve by directly solving an MDP with something like value iteration, then you can also easily explicitly represent exactly the set of actions that the agent can perform in each state, and the agent can learn the expected value of just taking those actions.
If your state space is very large, you may not be able to represent it explicitly, and your agent is more likely to use some approximation of the value function or its policy, as is commonly done in Q-Learning. Here, it is often preferable to define your model of the environment so that taking an invalid action in a state causes some well-defined outcome, or causes the agent to randomly re-select its actions until it ends up picking a valid one. The agent will eventually learn that selecting an invalid action leads to bad outcomes, without "realizing" that the action is invalid.


text:
What is the term for datasets that are themselves composed of datasets?
As computers are getting bigger better and faster, the concept of what constitutes a single datum is changing.
For example, in the world of pen-and-paper, we might take readings of temperature over time and obtain a time-series in which an individual datum is a time, temperature pair.  However, it is now common to desire classifications of entire time-series, in the context of which our entire temperature time-series would be but a single data point in a data set consisting of a great number of separate time-series.  In image processing, an $(x,y,c)$ triple is not a datum, but a whole grid of such values is a single datum.  With lidar data and all manner of other fields things that were previously considered a dataset are now best thought of as a datum.
What is the term for datasets that are themselves composed of datasets?
The term "metadata" is occupied, I should think.
Are there any papers that talk about this transition from datasets of data to datasets of datasets? And what the implications are for data scientists and researchers?

PLEASE NOTE: I am not asking "what are the implications of this?", which would be an open-ended question.  I am asking for references that discuss the phenomenon.  The existence or non-existence of such references is not open-ended.

An interesting topic, you might be interested in latent variable models: https://www.sciencedirect.com/topics/neuroscience/latent-variable-models Hope this answers something, I look forward to a discussion.

I don't think that this is anything new. Let's use your example of classifying an entire time series, say predicting word 1 vs word 2 for speech recognition. We can write out the data as a data frame like we would do with any other multivariate data: observations at time 1, time 2, etc as the predictors and the classification label as the response variable.
Each observation is a vector of the values at particular times for your subject, plus the label--no different than any other multivariate data. Sure, there might be special dependence structure because of the time series nature of your data, but you can still write it as a multivariate problem.
Okay, let's say that you hit the speech signal with a wavelet transform, resulting in an image-looking spectrogram of 2D data. Then just consider each "pixel" (time-frequency pair) to be a variable in your multivariate problem, along with the classification label. This is some kind of bijection between an $m\times n$ matrix and $\mathbb{R}^{n\times m}$.
You can extend this idea to 3D or 4D data (or higher), too. Just unwrap the high-dimension tensor in some kind of map $T^{m\times n \times \dots} \rightarrow \mathbb{R}^{m\times n \times \dots}$.

"Each observation is a vector of the values at particular times for your subject, plus the label" this is not how most learning methods treat the problem.  Suppose that I want to classify a time-series.  Your model has been trained to take on points $(t,y,l)$ where $l$ is a label.  The constraint that all of the points in the time-series I want to classify have an identical label $l$ at least shows that naive application of datasets-of-data algorithms is non-trivial in this context.

@Scott I assume that $t$ is time and $y$ is the value of the time series at time $t$. Let's also assume $T$ time samples. Then you could write your observation as $(y_{1},\dots,y_{T}, l)$. Then you would have one row of this $T+1$-dimensional observation per labeled subject.

This assumes that all of my time-series are equal in length.  Of course, I agree with you that analysis of datasets of datasets is not impossible, and I agree with you that many situations can be handled with various tricks to arrange the data into familiar fashions.  To bring the conversation back to my original question, which is less about "how to handle the situation" and more about "how to refer to the situation"... The vectorizing trick in your immediately previous comment:  Under what situations and contexts could you employ it?  Only time-series?

@Scott Uneven length time series wreck the data frame idea but not the idea of seeing the time series as a vector. For instance, represent different length time series in a graphical format (SPARQL-style): label as a node -> value at time 1 -> value at time 2 -> ... Do this with multiple branches from the label node if you have several time series (such as in a spectrogram). I see this method working for images and language, in addition to time series.

"I see this method working for images and language, in addition to time series"  Neat.  How would you succinctly denote this class of problems?  Note that the descriptor you use can't actually include all time-series problems, since simple prediction on a time-series does not fall into this class of problem.  What is this class of problem?  Are there any sources that attempt to generalize your vectorization technique to this whole class?  Or other analysis techniques?

@Scott “Multivariate statistics” and “supervised learning” sound good to me. Why do you say that simple prediction of time series can’t fall into this class?

Because my question is about how to distinguish this class of problems.  If you don't have to use your fancy vectorization technique on simple time series in order to make predictions, then they don't fall into the class that we're trying to distinguish.  Therefore, "supervised learning" is not a satisfactory distinguishing term for the class.

I think perhaps you misunderstand my original question, so allow me to make an analogy.  There is a class of animal called "mammalia".  I am pointing to some specific type of mammal and asking "what are these things?"  You are saying "well, they're mammals, and here's why".  I agree with you wholeheartedly.  However, I knew that they were mammals.  I want to know what type of mammals.  Are they cats?  Are they dogs?  Possibly nobody has named this particular type of thing that I'm asking about.  That is fine, and that would be a satisfactory answer.

Let us continue this discussion in chat.


text:
Is there a GAN that can be used for sequence prediction?
I want to use a GAN for sequence prediction, in a similar way that we use RNNs for sequence prediction. I want to test its performance in comparison with RNNs. Is there a GAN that can be used for sequence prediction?

A sequence-to-sequence model may be what you were looking for.


text:
How does Atlas from Boston Dynamics have human-like movement?
Discussing the video More Parkour Atlas, a friend asked how the robot's movement was so similar to the one from a real human and wondering how this is achieved?
To my knowledge, this is not something the developer "programmed", but instead emerged from the learning algorithm.
Could you provide an overview and some reference on how this is achieved?

Because Boston Dynamics is a private, for-profit, company, we cannot know for sure how they achieve their results. However, we can examine the available public information and make educated guesses.
In the information posted with the video, Boston Dynamics tells us that they use

... an optimization algorithm transforms high-level descriptions of
each maneuver into dynamically-feasible reference motions.  Then Atlas
tracks the motions using a model predictive controller that smoothly
blends from one maneuver to the next.

This sounds like three older AI approaches have been blended together to create the video.
First, they mention using an optimization algorithm to assemble complex motions from "dynamically-feasible reference motions". This sounds like they have first learned, or possibly pre-programmed, a range of simple movements that, on their own, are not very impressive, but that can be composed together into more complex movements. This approach is called Layered Learning, and was pioneered by Peter Stone and Manuela Veloso in the late 1990s. It is widely and successfully used in academic robotics competitions. Basically, this algorithm tries out different combinations and sequences of simple actions until it finds one that is close to the desired complex action. This is usually done with a local search algorithm, or sometimes with other optimization tools.
The second technique is, of course, actually learning to perform the basic actions that layered learning can compose together into more complex actions. This is usually done with some form of reinforcement learning, but sometimes is done by a programming who explicitly solves the equations for the movement of a simple system.
Finally, they need to use a model predictive controller to smoothly interpolate between the sequence of actions that the layered learning approach has come up with. In this approach, the engineers designing the robot have very carefully measured, for this specific robot, exactly how parts of it tend to move or continue moving, and written this down as a mathematical model called a dynamical system. This model allows the algorithm to figure out how this specific robot's movements are slipping away from the planned ones as it executes the motions. The slippage occurs because of things like friction (the robot's parts are not all perfectly smooth, and may have different amounts of lubrication, or a motor might slip). The algorithm can then make small changes to the motion to smooth out these unexpected bumps. That's the part that makes everything look so smooth.
It's worth noting too, that the video is not fully representative of the typical results their algorithm obtains. Boston Dynamics claims in the video caption that their algorithm "succeeds" in achieving a desired motion about 80% of the time. They don't tell us what "success" means, but the video is probably just the best of many takes for a carefully planned and filmed routine.


text:
What is the relationship between the size of the hidden layer and the size of the cell state layer in an LSTM?
I was following some examples to get familiar with TensorFlow's LSTM API, but noticed that all LSTM initialization functions require only the num_units parameter, which denotes the number of hidden units in a cell.
According to what I have learned from the famous colah's blog, the cell state has nothing to do with the hidden layer, thus they could be represented in different dimensions (I think), and then we should pass at least 2 parameters denoting both #hidden and #cell_state.
So, this confuses me a lot when trying to figure out what the TensorFlow's cells do. Under the hood, are they implemented like this just for the sake of convenience or did I misunderstand something in the blog mentioned?

I had a very similar issue as you did with the dimensions. Here's the rundown:
Every node you see inside the LSTM cell has the exact same output dimensions, including the cell state. Otherwise, you'll see with the forget gate and output gate, how could you possible do an element wise multiplication with the cell state? They have to have the same dimensions in order for that to work.
Using an example where n_hiddenunits = 256:
Output of forget gate: 256
Input gate: 256
Activation gate: 256
Output gate: 256
Cell state: 256
Hidden state: 256

Now this can obviously be problematic if you want the LSTM to output, say, a one hot vector of size 5. So to do this, a softmax layer is slapped onto the end of the hidden state, to convert it to the correct dimension. So just a standard FFNN with normal weights (no bias', because softmax). Now, also imagining that we input a one hot vector of size 5:
input size: 5
total input size to all gates: 256+5 = 261 (the hidden state and input are appended)
Output of forget gate: 256
Input gate: 256
Activation gate: 256
Output gate: 256
Cell state: 256
Hidden state: 256
Final output size: 5

That is the final dimensions of the cell.

All the gates have to have the same dimension output as the cell state in order to perform element-wise multiplications. However, this does not answer why the hidden state and the cell state vectors have to have the same dimensions. You could very well have different hidden and cell state dimensions and still force the concatenation of the hidden state with the input vector to have the same dimension as the cell state after going through the gates.

@Mike that's exactly what I'm thinking, and I don't get why in TF the units parameter represent both the cell size and the output size

@AlbertoSinigaglia Can't answer Mike's question right now (I assume it's just convention to keep sizes the same, but don't know 100%), but the reason the units parameter represents both is because the output is just the hidden state. The idea of this is that if you don't want the hidden state to be this size, you can just add a linear layer on top of it and transform it how you see fit, which is what defining the output size would do anyway

@RecessiveI'm saying that all the dimensions can be different. If you check the equations on PyTorch doc you see that the only requirements are that the hidden cell has to have the same size as the gates, the rest can vary

What I understand with a layer of LSTM composed of 4 cells is depicted in the following picture:

This would explain the fact that the hidden state of the whole layer has exactly the same dimension of the hidden states (or cells).
However, what I still don't fully understand is the 'return sequence' between LSTM layers, which changes the shape from [hidden_states] to [x_dimension, hidden_states]. This is explained because usually we only care about the state of the last cell, and when connecting multiple layers, all the states of the cells are passed into the next layer. Nevertheless, I still cannot make sense of it graphically.
e.g.
model = keras.models.Sequential([ keras.layers.LSTM(20, return_sequences=True, input_shape=[None, 1]), keras.layers.LSTM(20, return_sequences=True), keras.layers.TimeDistributed(keras.layers.Dense(10)) ])

Look at the equation for computing the hidden state as a function of the cell state and output gate:
$$
h_t = \tanh(C_t)\circ o_t
$$
This equation implies that the hidden state and cell state have the same dimensionality.


text:
What is the best way to find the similarities between two text documents?
I would like to develop a platform in which people will write text and upload images. I am going to use Google API to classify the text and extract from the image all kinds of metadata. In the end, I am going to have a lot of text which describes the content (text and images). Later, I would like to show my users related posts (that is, similar posts, from the content point of view).
What is the most ppropriate way of doing this? I am not an AI expert and the best approach from my prescriptive it to have some tools, like google API or Apache Lucene search engine, which can hide the details of how this is done.

I would suggest to convert the documents into TF-IDF(use Gensim) vectors and then compare them using various similarity calculating techniques like cosine similarity.
You should read this amazing article for the same. I once used it while working on my project.
https://medium.com/@adriensieg/text-similarities-da019229c894

Google has introduced Universal Sentence Encoder, which converts sentences into vector representations while preserving the semantic details. The pre-trained models are available on Tensorflow Hub. The Colab notebook would help you get started as well.


text:
What are the differences between stability and convergence in reinforcement learning?
The terms are mentioned in the paper: An Emphatic Approach to the Problem of off-Policy Temporal-Difference Learning (Sutton, Mahmood, White; 2016) and more, of course.
In this paper, they proposed the proof of "stability" but not convergence.
It seems that stability is guaranteed if the "key matrix" is shown to be positive definite. However, convergence requires more than that.
I don't understand the exact difference between the two.

Sometimes when training, particularly in reinforcement learning, the model can become unstable due to the amount of variance that exists in the training data that the agent generates by interacting with the environment.  This is certainly a problem at the start of training as you can get huge outliers in the data because the agent is behaving randomly.  You can find that just one update to the policy could potentially make it collapse because it moves the policy into some obscure region, e.g. so the agent always take a particular action.  You can make training more stable by using larger batches and a smaller learning rate so it takes smaller steps at a time, but the downside to that is training is slower.  So you need to test different hyperparameters to find a good trade-off between the two.  You can also use an training architecture such as Proximal Policy Optimization (PPO) which clips the amount the policy can move in any given update to try and maintain some stability.
Convergence is a term used to describe when the model has found an optimal policy and isn't learning any further, usually demonstrated when the reward plateaus for a certain number of episodes.  Of course, it may have settled on a local optima, and other global optima may exist; the data you present and the way you train your model may yield better results - again, all part of testing and experimentation.

I understand your intuitive explanation, but a more theoretical approach would be preferable.


text:
How could I compute in real-time the similarity between tickets?
I'm dealing with a "ticket similarity task".
Every time new tickets arrive at the help desk (customer service), I need to compare them and find out about similar ones.
In this way, once the operator responds to a ticket, at the same time he can solve the others similar to the one solved.
I expect an input ticket and all the other tickets with their similarity in output.
I thought about using DOC2VEC, but it requires training every time a new ticket enters.
What do you recommend?

Not sure if you are bent on using DOC2VEC, but why not use OpenAI embeddings or any Hugging Face model, and store them as a sparse-dense vector in a vector database (NOTE - the concept of sparse-dense was designed by Pinecone) they have publicly made it transparent how to do it yourself).
Using Sparse-dense makes sure your similarities are a blend of semantics as well as lexical which will work perfectly for your case.

You need to create an active learning loop over the process of the learning. Try to start from a history of tickets and using doc2vec to get the similarity. When you find a bad result in the result of your classifier, then report it and then try to retrain the classifier. Also, you can wait to retrain the model, up to finding the predefined batch0size of the new data which are not in the training set.
Also, to get a better result in the active learning loop, you can testify incoming data by the measuring of the classifier uncertainty over it. If the entropy of the classifier over the data is not in a good situation, you can label the data by the operator (as an oracle) and then up reach to the predefined batch-size, retrain the classifier.
Morevoer, to know better about the active learning process and query strategies follow this link (and other articles in that link like this article).


text:
What is the difference between random and sequential sampling from the reply memory?
I was working on an RL problem and I am confused at one specific point. We use replay memory so that the network learns about previous actions and how these actions lead to a success or a failure.
Now, to train the neural network, we use batches from this replay or experience memory. But here's my confusion.
Some places like this extract random (non-sequential) batches from the memory to train the neural network but Andrej Karpathy uses the sequential data to train the network.
Can someone tell me why there's the difference?

You linked 2 completely different algorithms, that is the reason why there is a difference. One is off-policy and other is on-policy algorithm

What do you mean?

First link talks about DQN algorithm, second link is implementation of policy gradient algorithm, reinforcement learning isn't only a single algorithm there are multiple kinds of algorithms, you need to read up a bit on it

@Brale_, so correct me if I'm wrong. in DQN, the sequence doesn't matter because we have this Q table that gives an idea to the network what it should do to converge. But in PG, we don't have such a table or any reference hence we have to follow the sequence to see which set of actions led to an optimal policy?


text:
Given a list of integers $\{c_1, \dots, c_N \}$, how do I find an integer $D$ that minimizes the sum of remainders $\sum_i c_i \text{ mod } D$?
I have a set of fixed integers $S = \{c_1, \dots, c_N \}$. I want to find a single integer $D$, greater than a certain threshold $T$, i.e. $D > T \geq 0$, that divides each $c_i$ and leaves remainder $r_i \geq 0$, i.e. $r_i$ can be written as $r_i = c_i \text{ mod } D$, such that the sum of remainders is minimized.
In other words, this is my problem
\begin{equation}
\begin{aligned}
D^* \quad = \text{argmin}_D&  \sum_i c_i \text{ mod } D \\
\textrm{subject to} &\quad D > T
\end{aligned}
\end{equation}
If the integers have a common divisor, this problem is easy. If the integers are relatively co-prime however, then it is not clear how to solve it.
The set $|S| = N$ can be around $10000$, and each element also has a value in tens of thousands.
I was thinking about solving it with a genetic algorithm (GA), but it is kind of slow. I want to know is there any other way to solve this problem.

I would actually think that this is one of the problems that GAs are fast at solving; being a purely mathematical problem, with mono-dimensional solutions, that can be bounded in a finite space.

Hello. This is an old post, but how did you solve this problem in the end? You may want to provide a formal answer below, if you found a good solution to this problem.

I am almost certain the answer here is just T (being the lower bound on N). The reason for this is factors are prime, and therefor mostly random. As N becomes larger, the possibility for a larger remainder also increases, but the possibility of N being a divisor (or even close divisor) remains the same. This is all speculation, but I'm only saying this based on some simulations I did where the sum of each N was strictly increasing with N. This excludes edge cases with very small sample sizes (like [10,20,30] and T=9), but once a sample size is large enough, it seems $D*=T$

Disregard "strictly increasing". Some values are less than others even in large sample sizes, but only slightly. It's more like "mostly increasing, with outliers that are a few percent smaller than the previous value"

Genetic Algorithm is not the best approach here:

GA is a stochastic methods and therefore will never guarantee the best possible solution.
Your solution (GA individual) is modeled as a simple integer. GA is optimal to find a solution modeled as an array (like our genomes), so it can do mutation and crossover.

Good Approach:

The set |S|=N can be around 10000, and each element also has a value in tens of thousands.

It certainly looks a lot, but for a computer, that's not so much. We can start with a brute-force and incrementally simplify it for better performance:
Brute Force:
You can try every single integer from D>T until D<max(S) and store the best candidate.
# Fully functional python solution:
import random
import time

# Let's start initializing a random T
T = random.randint(1, 100)
# And a S, with 10,000 integers fom T to 10,000
S = [random.randint(T, 10000) for i in range(10000)]

def LowestRemainder(List, Threshold):
    # Storing the all time lowest D and remainder
    Best_D_SoFar = Threshold+1
    lowestRemainderSoFar = sum(List)
    for d in range(Threshold+1, max(List)+1):
        remainder = sum(c%d for c in List)
        if remainder < lowestRemainderSoFar:
            lowestRemainderSoFar = remainder
            Best_D_SoFar = d
    return Best_D_SoFar, lowestRemainderSoFar

print("Looking for D > %d, that minimizes the sum of all remainder in a list containing %d integers" % (T, len(S)))
# Keep track of time
start_time = time.time()
# call the function
D, R = LowestRemainder(S, T)
# stop the clock
elapsed_time = time.time() - start_time
print("Found D =", D, "and lowest remainder =", R, "in", elapsed_time, "seconds")

I've tested a few times on my machine and it always runs in less than 4 seconds (for the maximum
But that's the brute-force baseline. So let's make it more efficient, by early interrupting the inner loop:
    # Remove this:
    # remainder = sum(c%d for c in List)

    # Add this instead:
      remainder = 0
      for c in List:
          remainder += c%d
          if remainder > lowestRemainderSoFar:
              break

And done!
It now takes less than 0.2 seconds
(for the whole 10,000 sized array with random integers up to 10,000:)

Extra thoughts:
As the best solution is probably a low number (as discussed later), if it takes too long to finish, you can simply interrupt and you'll still have a good solution (not guaranteed to be optimal) just like with GA (and probably better, as you'll quickly exploit the best candidates, instead of exploring the space).
Factorization is our friend:
Suppose a solution D, with a score S.
Now factorize the solution, finding the list of primes that composes D: [p1,p2,…].
Now remove some p, generating a D' and you will notice that the c mod(D) >= c mod(D') for any c, any D, and any p.
That means, you don't need to explore any multiple of D, if D was already evaluated.
That will drastically drop the amount of trials!
No threshold scenario:
Using this factorization principle, in the simpler scenario where T=1 (if T<1, than D=1 is the best solution), we guarantee that the solution is a prime number! Also, you don't need to search until the biggest number max(S), as the sqrt(max(S)) will be enough.
With threshold scenario
It makes it a little harder to program, but still you just need to loop through co-primes.
Upper limit:
Once you hit the first candidate (d) larger than the smallest S (d>min(S)), than this remainder will be min(S).
If the best solution so far is already better than min(S), we can skip all the next candidates.
This logic is cumulative. Once the candidate d is larger than the N lowest S, the sum of remainders will be at least the sum of all N lowest S. So, our loop could be something like:
if(sum([s<d for s in S]) > lowest):
  return d
else:
  d = next_coprime()

In a list with thousand of elements, there's a high chance that the smallest element is found early. And even if it does not satisfy the finish condition, it will only accumulate more elements as d grows.
In summary:
Start with a baseline and restrict the boundaries until your code runs in the desirable time.
Example:
Inputs:

S = [10, 19, 28]
T = 3

Process: Testing all d>T, skip the multiples and stop on upper limit:

d = 4 --> remainders = [2,3,0] = 5
d = 5 --> remainders = [0,4,3] = 7
d = 6 --> remainders = [4,1,4] = 9
d = 7 --> remainders = [3,5,0] = 8
d = 8 --> is multiple of 4 (skip)
d = 9 --> remainders = [1,1,1] = 3
d = 10--> is multiple of 5 (skip)
d = 11--> 11>min(S) --> remainders = [10,x,x] > 3. DONE!


text:
What are the exact meaning of "lower-order structure" and "higher-order structure" in this paper?
I recently read a paper on community detection in networks. In the paper EdMot: An Edge Enhancement Approach for Motif-aware Community Detection, the authors consider the "lower-order structure" of the network at the level of individual nodes and edges. And they mentioning about some "higher-order structure" method. The point is, what is the exact meaning (definition) of lower- and higher-order structure in a network?

Low order/low level information refers to the most granular level of information. This is the most informative in terms of volume of information, but it can often be difficult to conceptualise for humans.
High order/high level information refers to abstractions of the low level information to more intuitive but less easy to describe technically concepts.
An example would be images of faces. The low level information might be the raw $x, y, z$ values of the pixels: their position and colour value. Some high level information might be the direction the face in the images is facing, from what direction the lighting in the image is coming etc.
In the paper cited, the example they use is low level information (node and edge values); high level information (motifs).


text:
What are the examples of agents that is represent these characteristics?
I'm looking for examples of AI systems or agents that best represent these five characteristics (one example for each characteristics):

Reactivity
Proactivity
Adaptability
Sociability
Autonomy

It would be better if its machine learning-based application.

When you talk about agents in this context we tend to think of the formal definition of "intelligent agent", which involves reflex (reactivity), utility, model-based, goal oriented, and learning (adaptability).  I would be interesting if there were formalizations for proactivity, sociability and autonomy!


text:
Is randomness necessary for AI?
Is randomness (either true randomness or simulated randomness) necessary for AI? If true, does it mean "intelligence comes from randomness"?
If not, can a robot lacking the ability to generate random numbers be called an artificial general intelligence?

What do you mean by "necessary for AI"? I know it may seem obvious, but I think you should be more explicit and clear.

This is a good core question, in that the statistical approach to AI, which allows machine learning, has eclipsed GOFAI, but it might be useful to elaborate a little in the question.  Pseudo-random number generation may not be entirely sufficient, in that, given sufficient time/iterations, it would result in unintended bias.

Even if the claim - randomness being necessary for AI- is true, how does the question does it mean intelligence comes from randomness  follow?

Being able to generate random numbers does not imply intelligence, not even close to it. Intelligence might follow from the why of random number generation. That why is generally coded in the models for AI such as in neural networks by a human - and that is for weights and bias initialization which in turn is essential for solving the optimization problem using gradient descent.

we're probably missing words here. randomness is the apparent lack of pattern or predictability in events. the more predictable something is, the dumber it becomes. of course just a bunch of random numbers doesn't make anything intelligent. it's much to the opposite: randomness is an artifact of intelligence. but if while we're reverse engineering intelligence (making ai) we can in practice see it does need rng to exist, then there's evidently something there between randomness and intelligence that's not just artifacts. could we call it chaos? rather continue there: https://cregox.net/random

Is randomness (either true randomness or simulated randomness) necessary for AI

It depends on how you define Artificial Intelligence.  If you regard it strictly as an intentionally created construct which demonstrates utility, then no.  (For instance, Nimatron, potentially the first functioning AI, beat most human competitors at NIM.  But Nimatron was classical AI, entirely rules based with no learning.) That said:

Randomness has proved a useful component in machine learning, and any feasible AGI would likely require ML.

Given sufficient computing power, aka time and space, it would absolutely be possibly to brute force anything, including AGI, but the resulting algorithm would be "brittle", unable to "compute" anything not previously defined. A learning algorithm, presented a problem outside of its domain of knowledge may initially degrade in performance, but it can learn from those outcomes, and gradually improve performance.
IBM brute forced Chess with Deep Blue, but Chess is a strictly narrow problem that turned out not to require general intelligence.  AGI requires human level performance in all tasks engaged in by humans, which, even if they could be broken down to a set of individual narrow problems, it's an ever expanding set of problems.

Does it mean "intelligence comes from randomness"?

Not if the definition of intelligence is rooted in utility because deterministic processes can demonstrate utility.

In statistical AI, the intelligence arises from the analysis of random search or the fitness of the genetic algorithm, not the randomness per se.

In other words, if you have the randomness without the analysis, every decision is an unqualified guess.
My sense is that it is free will that would arise from randomness—effects unrelated to causes—because without true randomness, the universe and everything in it is purely deterministic.

Yes, randomness is necessary to achieve generality in theory. Right now AIs we have are on the basis of seeking pattern and use them to predict future moves or outcomes. If we don't include randomness in data then machine might consider that as pattern and behave according to that (Which will be bias for us).
Generating random numbers is a different story in itself and won't be a criterion alone to judge. While this might be one of the conditions for sure.

Brute force (exhaustion) would not more limited than random composition--in both cases there would still be the same number of potential configurations. Only the approach to forming them is different. (See "The Library of Babel")  But there is still the "needle in the haystack" problem, and the statistical approach has proven effective where brute force is insufficient.

Why it will be a limited number of stories to come up with? Given word 0 and word 1 we can generate all (which is infinite) binary stories without randomness.

It might be too philosophical answer, but maybe first we need to answer the question whether a human way of thinking or his creativeness includes random elements. For example if an author writing a book uses some randomness in developing some side thread or some episodic character and I would say, that yes - sometimes we think up of something random.
Some algorithms uses randomness at their basis, for example evolutionary algorithms for generating first population.

All machine learning algorithms that have proven effective seem to use the statistical approach, which definitely involves randomness.  So not just genetic algorithms, NNs in all of their flavors, native Bayes, et al.

I think the randomness we observe in human decision may not be true randomness, it may just be a function taking history, environment as arguments which is too complicated that it behaves like random function.


text:
Looking for GAN paper with spiral image
I am looking for a GAN paper I have read a while ago, but unfortunately cannot find it again. I think it compared GANs and other methods (like CVAEs) w.r.t. how they handle multi-modal data, not sure about the CVAEs though.
What I know is that they created different synthetic toy datasets with multiple modes to analyze this. I remember one plotting of a blue spiral of this data on a white background. Any guesses?


text:
What is the difference between a semantic network and an ontology?
What is the difference between a semantic network and an ontology? How are they related? I have not found any article that describes how these two concepts are related.

A semantic network is a way to implement an ontology. An ontology is just a generalised way of representing knowledge in a particular domain, and there are multiple ways of doing so. The key that distinguishes an ontology from, say, Wikipedia, is that it is formally defined, so that the knowledge represented can be used in programs to reason with.
Semantic networks can be used to do that. Please note that there are many types of semantic networks, so you can use them to represent a wide range of relevant information. Since the point of an ontology is to show the relationships between entities relevant to a domain, they are usually represented as networks — if the information is stored as RDF triples, these can usually be visualised as an equivalent network.
So ontology is the broader, more general term, whereas a semantic network is a more specific way of representing information.

Actually Ontology is something most AI do not incorporate and you can ask them your self....most are transformers- Ontology relates to a hierarchy of data that relates to the word or phrase... that is keyed into the AI query -
You won't really get the difference between them unless you built one from front to back- which most people don't do and so they lack real insight from just using gui' s...read a academic paper from a Ukrainian - on Distributed Semantics with Ontology training - to learn the difference.


text:
How to train a transformer text-to-text model on counterexamples?
Is it possible to update the weights of a vanilla transformer model using counterexamples alongside examples?
For example, from the PAWS data set, given the phrases "Although interchangeable, the body pieces on the 2 cars are not similar." and "Although similar, the body parts are not interchangeable on the 2 cars." we have the label 0 because it is a counterexample, whereas for the phrases "Katz was born in Sweden in 1947 and moved to New York City at the age of 1." and "Katz was born in 1947 in Sweden and moved to New York at the age of one." we have the label 1 because it is a positive example of a valid paraphrase.
My goal is to use the transformer model to generate paraphrases, and I am attempting to build a GAN but could not find any references for updating the transformer text-to-text model using counterexamples.


text:
What are examples of optimization problems that can be solved using genetic algorithms?
I'm trying to learn how genetic algorithms can solve optimization problems. I have already learned how genetic algorithms can solve the knapsack, TSP and set cover problems. I'm looking for some other similar optimization problems, but I have not found any.
Would you please mention some other famous optimization problems that can be solved by using genetic algorithms?

If the answer below answers your question (and I think it should, if not, please, let me know why not), you should accept it by clicking the check symbol next to it. Take a look at https://ai.stackexchange.com/help/someone-answers.

There are numerous problems that can be solved with genetic algorithms or, more generally, with evolutionary algorithms (which includes also genetic programming and evolutionary strategies), even though they may not necessarily be the most efficient approach.
Here are a few examples.

Evolution of the topology of neural networks. This is called neuroevolution.
Automatic test case generation (in particular, for self-driving cars). AsFault is one specific example.
Design of novel quantum computing algorithms. Specifically, genetic programming has been used to solve this problem (see this reference for more details).
As an alternative to reinforcement learning algorithms to solve RL problems. Specifically, evolution strategies have been successfully used in this case (see this).

There is a Wikipedia article that lists many other applications of genetic algorithms: List of genetic algorithm applications.


text:
Is there a way to do reinforcement learning in POMDP?
Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?

What do you mean by "update Q kernel"?

So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel

Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here.

Here is a paper link. In general you can google pomdp transition kernel to get many such results

I am not sure if this is a duplicate of this one. I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know.


text:
What is the difference between tracking and mapping (TAM) and localization and mapping (LAM)?
In the paper Visual SLAM algorithms: a survey from 2010 to 2016 by Takafumi Taketomi, Hideaki Uchiyama and Sei Ikeda it is mentioned

It should be noted that tracking and mapping (TAM) is used instead of using localization and mapping. TAM was first used in Parallel Tracking and Mapping (PTAM) [15] because localization and mapping are not simultaneously performed in a traditional way. Tracking is performed in every frame with one thread whereas mapping is performed at a certain timing with another thread. After PTAM was proposed, most of vSLAM algorithms follows the framework of TAM. Therefore, TAM is used in this paper.

I do not quite follow the difference between localization and mapping versus tracking and mapping.

What is the difference?

What are some advantages of TAM?

Why is SLAM not called STAM?


text:
What is the most suitable AI technique to use for path planning?
I am making a firetruck using Arduino Uno with flame sensors and ultrasonic sensors to detect how to move and where to go. As this is a project for my university, I am asked to implement AI in it for path planning.
I am not sure whether to use something like A* technique or ID3 decision tree or if there is something better than both to implement path planning for my robot. Any suggestions?

AVR-based Arduinos have very limited processing power. They do not have hardware floating-point support, for example. Many AI algorithms involve doing matrix math, which will be very slow on an AVR based Arduino. I suggest a higher powered processor like a Raspberry Pi would be a better choice.

@DuncanC i would but i already have arduino and my project is due soon, we are asked for a simple way to implement the AI just to plan the path to the fire

Can you give more details about how the path finding algorithm would work? .i.e. is the computation local or you have the chance to compute on a pc and send instructions to the Arduino? (The second approach seems the most promising to me). The ultrasonic sensor data is stored somewhere to map the environment?

@AlvinSartor I am not sure yet how I should work with it,I thought maybe I can code it in a way that while the robot is roaming around (before finding fire) it would store coordinates of obstacles it finds so that when it does detect flame it would use that info to find best path to flame, or i would just make up a dataset to use for an id3 decision tree

Ok. The first step is being able to calculate your position. Maybe using the ultrasonic sensor or calculating the distance/direction each time the wheels spin.

@AlvinSartor ok so i will have in memory saved distances of each obstacle i come across?

If I had to implement a path exploration/finding algorithm on a robot, I would follow these steps:

Make sure you can detect your position. You need to be able to record your position otherwise you have no reference for the exploration. You don't need a global positioning system (like GPS), a local one is more than enough in your case. This means that the robot must know that it has been switched on in position (0,0). If you go straight for 5 meters, you'll update it to (5,0) or something like that.
Knowing how much you moved and towards where is the difficult thing.
Once you can know where you are, it is time to record it. As you want to explore the environment, you might want to create a tree with states on the nodes. The node can be open if you can still explore around it, closed if the exploration has been done.
To get the path from a node to another, A* works more than enough.
To know what is around you, you can use the sensors to explore the surroundings and know the position of the obstacles.

This is the general idea:
Tree tree = new Tree()
Node first = new Node(Here)
tree.Add(first)

Node current = first

while(true) {
    ExploreAround(current)
    var nodes = CreateSurroundingNodes()
    tree.Add(nodes)
    current.State = closed
    var next = PickNearestOpenNode()
    MoveTo(next)
    WaitUntilRobotIsOn(next)
    current = next
}

In my bachelor thesis I was implementing a very similar algorithm on a drone on Unity3D. You can find the package here. You can get an idea out of it.
Here is a video of how it worked: https://youtu.be/Xrh9-4Bfcew

This is super helpful! I will try to see how I can mark down coordinates as the robot roams around in the specified area, Thanks a lot!


text:
Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?
I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this paper. As a reference, I am also using this Github repository (another implementation).
My model/idea is pretty straightforward. Train the network to do the bit operation AND (e.g 1 && 0 = 0), then using EWC, train it to use OR (e.g 1 || 0 = 1). I've got three inputs: bit1, bit2 and operation (0 stands for AND and 1 for OR) and one output neuron - the output of the operation. For example, if I have 0 1 0 the ground truth should be 0.
The problem, however, comes when calculating the EWC loss.
def penalty(self, model: nn.Module):
    loss = 0
    for n, p in model.named_parameters():
        _loss = self._precision_matrices[n] * (p - self._means[n]) ** 2
        loss += _loss.sum()
    return loss

I've got two problems:

The current means (p) and the old ones (self._means[n]) are always the same, resulting in multiplication by 0, which completely negates EWC.
As I have just one output neuron the calculation of the fisher's matrix is a bit different than the repo. The one I have written seems to be wrong. Any ideas?

I initialise the self._means[n] and self._precision_matrices (fisher's matrix) in the init method of the EWC model:
class EWC(object):
def __init__(self, model: nn.Module, dataset: list, device='cpu'):

    self.model = model
    self.dataset = dataset
    self.device = device

    self._means = {}
    self._precision_matrices = self._diag_fisher()

    for n, p in self.model.named_parameters():
        self._means[n] = p.data.clone()

def _diag_fisher(self):
    precision_matrices = {}

    # Set it to zero
    for n, p in self.model.named_parameters():
        params = p.clone().data.zero_()
        precision_matrices[n] = params

    self.model.eval()

    for input in self.dataset:
        input = input.to(self.device)

        self.model.zero_grad()

        output = self.model(input)
        label = torch.sigmoid(output).round()
        loss = F.binary_cross_entropy_with_logits(output, label)
        # loss = F.nll_loss(F.log_softmax(output, dim=1), label)
        loss.backward()

        for n, p in self.model.named_parameters():
            precision_matrices[n].data += p.grad.data ** 2 / len(self.dataset)

    precision_matrices = {n: p for n, p in precision_matrices.items()}
    return precision_matrices

And this is the actual training:
# Train the model EWC
for epoch in tqdm(range(EPOCS)):

    # Get the loss
    ls = ewc_train(model, opt, loss_func, dataloader[task], EWC(model, old_tasks), importance, device)

def ewc_train(model: nn.Module, opt: torch.optim, loss_func:torch.nn, data_loader: torch.utils.data.DataLoader, ewc: EWC, importance: float, device):
    epoch_loss = 0

    for i, (inputs, labels) in enumerate(data_loader):
        inputs = inputs.to(device).long()
        labels = labels.to(device).float()

        opt.zero_grad()

        output = model(inputs)
        loss = loss_func(output.view(-1), labels) + importance * ewc.penalty(model)
        loss.backward()
        opt.step()

        epoch_loss += loss.item()

    return loss

Note: the loss function that I am using is nn.BCEWithLogitsLoss() and optimisation is: SGD(params=model.parameters(), lr=0.001).

Hi Mitch and welcome to this community! Maybe it would be better if you provided a minimal and reproducible example. Bear in mind that not all implementation-related questions are on-topic here.


text:
YOLO 9000 about Better Stronger
In this paper, YOLO has three features compared to YOLO v1. This question is about Better and Faster.
In the Better section, there are many techniques such as Batch Norm, Anchor Box and so on. In the Faster section, there is a darknet only.
Darknet has 19 Conv Layer but it doesn't use Layer Norm or Passthrough Layer. So, I think that Darknet doesn't use Better section techniques.
Is the Better Section model different from Faster Section Model?
In my understanding, there are three models named YOLO v2. First is Better YOLO v2, second is Faster YOLO v2, third Strong YOLO v2. Is this right?


text:
What is the difference between principal component analysis and singular value decomposition in image processing?
What is the difference between principal component analysis and singular value decomposition in image processing? Which one performs better, and why?

Is there are particular (image processing) application you're interested in? PCA and SVD are quite general techniques.

The application are Missing Person Detection using python @nbro


text:
How to classify human actions?
I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses).
I want to classify human actions real-time like:

Left-arm bended
Arm above shoulder
...

I first did some research for pre-trained models, but I didn't find any.
Because I'm still quite new, I want to have advice about how I should solve this.

I thought maybe I need to create for every action enough pictures and from there on I can do image classification.
Or I use PoseNet from TensorFlow so that I have the pose estimation points. And from there on I create videos of a couple of seconds with every pose I want to track and I save the estimation points. From there on, I use a classification algorithm (neural network) to classify those points.

What is the most efficient option or are they both bad and is there a better way to do this?

My suggestion is to go with 1st option. reason is you will get to know much about data and initial stage will find some challenges in developing the model, over a period of time you will get to better results after hypertunning. Please go through article , ignore you have already read this article

Thanks for the article will read this. Any idea how I get a lot of images or how many I actually need?

Are you looking for label data with images or just images, have found one data set, link.

Thank you. For me it doesn't matter I can label them myself. Like for example pictures of people bending their arms. When searching for this on google not a lot of pictures show up. In the database you shared per action 280 pictures exist.

In response to your question, I recommend using YOLOv8 for pose detection. It's a highly efficient and low-cost model for pose estimation, and it performs well in real-time applications. You can use it to extract key points from the video frames. you can use any other pose detection model it is a suggestion for you. like yolov8 is easy to use and highly accurate also it gives around 17 points of human pose
However, remember that many human actions depend on movements made in the past, so a simple classification of individual frames might not be enough for accurate action recognition. Instead, I suggest training a CNN-LSTM-Attention model.
=> CNN will help capture spatial features.
=> LSTM (Long Short-Term Memory) will account for temporal dependencies (previous movements).
=> The attention mechanism can further enhance the model’s ability to focus on the most important frames/key points for classification.
This architecture would allow you to classify actions based on sequences of key points extracted from videos, giving you much better accuracy than classifying single frames or individual poses.


text:
Should I add some noise when the dataset is small?
I want to create a small dataset (about 10 classes and 20-30 images each), should I add some noise (wrong label sample) in the training, validation and test datasets, and why?

Hi. Are you asking whether you should or not deliberately add some noise to the dataset? I've added the word "add" to your question, but I am unsure if this was the actual word missing.

What is your purpose in creating the dataset? What do you hope to achieve? We need to know in order to answer whether adding noise would be useful


text:
Why is my variational auto-encoder generating random noise?
This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the images that it generates are just random noise. Here are my script and the images that were generated by the model: https://github.com/jweir136/PyTorch-Variational-Autoencoder-Latent-Space-Visualization.
Any advice or answers on how to solve this problem are greatly appreciated.

Hi and welcome to this community! There's a reference implementation for the VAE in PyTorch: https://github.com/pytorch/examples/tree/master/vae. Maybe you should compare it with your implementation.


text:
Is there a way of automatically drawing bounding boxes around interested objects?
Given thousands of images, where some of the images contain target objects and others do not, is there an easy way of drawing bounding boxes on these target objects rather than relying on manual annotation? Wouldn't drawing 4 orientations of an object and their respective bounding boxes and randomly inserting them into the images be a viable option?
It becomes painful to manually annotate thousands of images by yourself.

How many classes do you want to annotate it?
We can get the bounding boxes and class names coarsely by using the pre-trained models such as YOLO-V3 darknet, SSD and others.
Then, we can load those annotations using the tool labelImg and manually correct it. It reduces the lot of work and it is called the Human-AI labeling technique.


text:
Why information gain with entropy as impurity function can't be used as a splitting method for Decision Tree Regression?
In Decision Tree Regression, we can use 'Reduction in Variance' or MSE (Mean Squared Errors) as splitting methods. There are methods like Gini Index, Information Gain, Chi-Square for splitting on classification trees. Now, I read somewhere that we cannot use Information gain (with impurity function as entropy) as a splitting method for regression trees. Why is it so, and what other methods are there which we can and cannot use, and why?
EDITS:
Please suggest me a reference to understand maths behind it.
The references I used are :
https://www.analyticsvidhya.com/blog/2016/04/complete-tutorial-tree-based-modeling-scratch-in-python/
https://www.python-course.eu/Regression_Trees.php
https://towardsdatascience.com/https-medium-com-lorrli-classification-and-regression-analysis-with-decision-trees-c43cdbc58054
In the first article, it is mentioned that:

Gini Index, Chi-Square and Information gain (impurity function as entropy) algorithms are used for Classification trees while Reduction in Variance is used for Regression Trees.

In the second article, it is mentioned that:

Since our target feature is continuously scaled, the IGs of the categorically scaled descriptive features are no longer appropriate splitting criteria.
As stated above, the task during growing a Regression Tree is in principle the same as during the creation of Classification Trees. Though, since the IG turned out to be no longer an appropriate splitting criteria (neither is the Gini Index) due to the continuous character of the target feature we must have a new splitting criteria.
  Therefore we use the variance which we will introduce now.

In the third article, it is mentioned that:

"Entropy as a measure of impurity is a useful criterion for classification. To use a decision tree for regression, however, we need an impurity metric that is suitable for continuous variables, so we define the impurity measure using the weighted mean squared error (MSE) of the children nodes instead"

Thank You!

Entropy, information gain etc has the formula for calculating the probability for the particular class so can use for the classification and we build tree according to that but in the case of the regression, we can't go each value as a label as they are infinite. There may be another points also. Have a look at a tree https://www.geeksforgeeks.org/python-decision-tree-regression-using-sklearn/


text:
How can I train a neural network to find the hyper-parameters with which the data was generated?
I have 10000 tuples of numbers (x1, x2, y) generated from the equation: y = np.cos(0.583 * x1) + np.exp(0.112 * x2). I want to use a neural network, trained with gradient descent, in PyTorch, to find the 2 parameters, i.e. 0.583 and 0.112
Here is my code:
class NN_test(nn.Module):
    def __init__(self):
        super().__init__()
        self.a = torch.nn.Parameter(torch.tensor(0.7))
        self.b = torch.nn.Parameter(torch.tensor(0.02))

    def forward(self, x):
        y = torch.cos(self.a*x[:,0])+torch.exp(self.b*x[:,1])
        return y

model = NN_test().cuda()

lrs = 1e-4
optimizer = optim.SGD(model.parameters(), lr = lrs)
loss = nn.MSELoss()

epochs = 30
for epoch in range(epochs):
    model.train()
    for i, dtt in enumerate(my_dataloader):
        optimizer.zero_grad()

        inp = dtt[0].float().cuda()
        output = dtt[1].float().cuda()

        ls = loss(model(inp),output)

        ls.backward()
        optimizer.step()
    if epoch%1==0:
        print("Epoch: " + str(epoch), "Loss Training: " + str(ls.data.cpu().numpy()))

where x contains the 2 numbers x1 and x2. In theory, it should work easily, but the loss doesn't go down. What am I doing wrong?


text:
Why can neural networks generalize at all?
Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they seem particularly apt at learning the right parameters. However, something we often have to combat when training neural networks is overtfitting - reproducing the training data and not generalizing to a validation set. The solution to overfitting is usually to simply add more data, with the rationalization that at a certain point the neural network pretty much has no choice but to learn the correct function.
But this never made much sense to me. There is no reason, in terms of loss, that a neural network should prefer a function that generalizes well (i.e. the function you are looking for) over a function that does incredibly well on the training data and fails miserably everywhere else. In fact, there is usually a loss advantage to overfitting. Equally, there is an infinite number of functions that fit the training data and have no success on anything but.
So why is it that neural networks almost always (especially for simpler data) stumble upon the function we want, as opposed to one of the infinite other options? Why is it that neural networks are good at generalizing, when there is no incentive for them to?

AFAIK, universal function approximation theorems show that, in theory, neural networks are capable of approximating continuous functions, which are, of course, not all functions. Neural networks seem to generalize in certain cases because the data they have been trained with is similar to the data they are tested on. Adversarial examples show that neural networks can catastrophically fail and don't really understand what they are supposed to do.

@nbro Although it has been shown that adversarial examples are signs that the neural network is learning robust and semantically rich features. Equally, while training and test data might be similar to us, who understand generalized features, the data usually isn't similar in terms of distance between tensors $n$-dimensional space, which is the way the computer "sees" it, or else you could solve every problem by k-means. The fact that a neural network can understand and cluster by those non-trivial, rich features leads back to my original question.

I've not read that paper and I won't read it now. Given that we use neural networks to solve our problems, we expect them to be consistent with our knowledge, that is, if it's a cat according to us, it should also be a cat according to the neural network. You can fool neural networks even by changing only some pixels. (IMHO, neural networks don't understand anything, but this is a philosophical digression).

You've asked a question which is basically one of the most important open questions about neural networks. The answer is a huge mystery - any response to this question which immediately opens with a purported explanation is basically ridiculous. We don't know.
As you pointed out, the issue is that the training set simply does not contain enough information to uniquely specify the target function. On an infinite input domain like $\mathbb R^n$, no finite number of samples is enough to uniquely determine a single function, even approximately. Even accounting for bounds and discretization of the input, and even for the symmetries our architectures impose on the output function, our training sets are microscopic compared to the sizes of our input domains. The problem that neural networks successfully solve every day should be impossible.
You can think about this in a low dimensional input space to get some intuition. Doing supervised binary classification on the unit square (that is, your input is a pair of numbers) is equivalent to trying to determine a monochrome image by seeing a random sample of some of its pixels. In terms of the size of the training set relative to the size of the input domain, what neural networks do on say an image classification task like MNIST is comparable to, say, guessing a 1000x1000 monochrome image almost perfectly by observing 20 random pixels, and even 20 is probably generous. The task is impossible - unless you know something about what the target image is. If you know that the image (the target function) is restricted to some set $H$ of functions, then you might be able to determine it approximately from a finite sample. Neural networks must in some sense be doing this implicitly,  with some set of "nice" functions $H$ which, it seems, happens to contain (approximations to) a lot of the functions we actually want them to learn, like the "is a cat" function on the space of all images.
The study of such sets of "nice" functions, and in particular how small they need to be before learning is possible, is the subject of statistical learning theory. But I'm not aware of any plausible answers for what $H$ could be for neural networks.

Maybe that's true in terms of a rigorous theorhetical basis but actually I see several 'intuitive' explanations including occam's razor, similarity, attractor states. The question is more which ones are right / contribute most.

@BruceAdams Sorry, but that's garbage. Occam's Razor, for example, is completely subjective. What does "simplest" mean? It only pushes the question back one stage - why do neural nets find the same qualities "simple" that humans do? Same kind of problem with "similarity" - what's "similar"? You can always contrive to find something similar between any two objects. Why do neural nets like the same kinds of "similarities" people do? Even "intuitively", no one ought to find these kinds of explanations remotely satisfying.

If you use the example from OP. A "wiggly" line is a more complex solution to a problem if you can draw a single line that goes close to or through all the target points. You can define that mathematically as having a lower error (distance measure from each point say) and less complexity (say a lower order polynominal is simpler) but not so easily in the context of SE post.

"If you use the example from OP. A "wiggly" line is a more complex solution to a problem if you can draw a single line that goes close to or through all the target points." No, it isn't. What's complex about it? You as a human being intuitively like to call it more complex, it is not actually more complex. What is simple about some random number (polynomial degree) being smaller? I could just as easily say bigger is simpler, or define some new metric which is smaller for a curvy line than for a straight one. These are human factors, not mathematical or objective ones.

It is mathematical or rather computational. You can define metrics for complexity. For example you could use the number of terms in an equation or the highest power in a polynomial. You have to fit your measures to the problem of course but it would be absurd to say that because there is no single universal definition of complexity it is not possible to create and use such measures. Many of these measures will share similar properties. An elephant is bigger than a cow if you measure its weight, its length or its volume. Each of those metrics is a useful measure of size.

@BruceAdams Yes, and the exact problem we're discussing (or one way of thinking about it) is which metric for complexity neural networks prefer, and how to justify somewhat rigorously that that's the one. Your answer to "which metric and how do you know?" can't be "well gosh I can think of plenty of metrics off the top of my head".

A fairly recent paper posits an answer to this:
Reconciling modern machine learning practice and the bias-variance trade-off.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, Soumik Mandal
https://arxiv.org/abs/1812.11118
https://www.pnas.org/content/116/32/15849
I'm probably not qualified to summarize, but it sounds like their conjectured mechanism is: by having far more parameters than are needed even to perfectly interpolate the training data, the space of possible resulting functions expands to include "simpler" functions (simpler here obviously not meaning fewer parameters, but instead something like "less wiggly") that generalize better even while perfectly interpolating the training set.
That seems completely orthogonal to the more traditional ML approach of reducing capacity via dropout, regularization, etc.

In other words we have implementation of something like occam's razor in some models

Your question "Why can neural networks generalize at all" make no sense.
Model does not simply either generalize or not in a black-or-white manner.
Perhaps what you mean is "Why does improving complexity of neural networks doesn't always cause the notorious bias-variance trade-off?"
To that my answer might be more controversial: Are we really sure that it doesn't?
The question is difficult to be verify in the sense that most of machine learning model (especially deep learning models) that perform really well currently arises from academia. So there is a mix of survival-of-the-fittest. While there might be bias-variance trade-off, it might be counteract with inductive biases employed by prior knowledge of the data scientist.
Then the remaining problem is how do we fairly and randomly adjust the variance of the model to significantly test the question? Unless we figure out how, I don't think we'll get an answer to your question any soon.

There is normally more to generalisation than just increasing training data. It helps to make the task noisy, through various means. One common and popular method is to use dropout, which encourages the network to utilise every node, and avoids dependencies on small clusters of nodes.
So how does making the task more noisy help with generalisation? Well it's easy to explain with a conceptual example, but I don't think that's what you're looking for, rather a more mathematical approach.
The best way to think of it is with a simple example of a polynomial data set, in only 2 dimensions. If you consider this, and the way the network slowly approaches optimums through back propagation, the concept that the classification boundary gradually approaches the optimum, which in all cases is an over-fit function, isn't too far fetched.
Now considering this, this would suggest that in order to properly train a network, we would need some way of determining when the network isn't super close to the optimum (as it will have over-fit by then) but also isn't so far from it that it's worse than randomly picking. This is were methods to improve generalisation come in, we want to hit that sweet spot.
If the learning rate is too high, we will overshoot that sweet spot and miss out entirely (the range can sometimes be very small), if it's too low, we may get stuck in tiny local minimums and never escape, or it could take years to reach it.
Using the example of dropout from before, this increases noise, and makes training harder on the network. Due to more difficult training, the network approaches the optimal function at a slower rate, and makes it easier to cut training when the network has generalised well.
Conveniently, this extends to n-dimensional problems as well. Now, as far as I know, there is no robust mathematical proof of why this works for n-dimensional problems. The reason for this, explains why neural networks even exist: We don't know how to mathematically classify these issues to the degree a NN does ourselves. Because of this, there will always be gaps in our knowledge. We will never be able to quantitatively say what a neural network is doing, without making them obsolete. So until we can figure it out for ourselves, we'll have to just perform testing against unseen data to see if the network has indeed generalised.

It's misleading to say "dropout, which forces the network to utilise every node". You should reformulate your description of dropout, IMHO.

@nbro You're right, I changed forces to encourages

What I meant is that, even without dropout, all units are involved. Moreover, dropout is usually only applied during training, although you can also apply it during testing (MC dropout). I would describe dropout as a way of training randomly chosen subsets of the network at every iteration, so that, in a way, you're training multiple smaller NNs to detect slightly different things.

@DuttaA In dropout, at every iteration, you randomly select units and you drop them, so that you train only the ones you haven't dropped. I am familiar with dropout and MC dropout. In this case, my knowledge was mainly acquired while reading research papers. But I am curious, what's the popular interpretation you're talking about?

@DuttaA Well, yes, dropout is used for regularization, but I was describing dropout (i.e. how it works) and not what it is used for. Note that, at test time, all units are usually enabled (apart from the case of MC dropout).

A neural network is composed of continuous functions. Neural networks are regularized by adding an l2 penalty on the weights to the loss function. This means the neural network will try to make the weights as small as possible. The weights are also initiallized with a N(0, 1) distribution so the initial weights will also tend to be small. All of this means that neural networks will compute a continuous function that is as smooth as possible while still fitting the data. By smooth I mean that similar inputs will tend to have similar outputs when run through the neural network. More formally, $||x-y||$ small implies $||f(x)-f(y)||$ small where f represents the output from the neural network. This mean that if a neural network sees an novel input $x$ that is close to an input from the training data $y$, then $f(x)$ will tend to be close to $f(y)$. So the end result is that the neural network will classify $x$ based on what the labels for the nearby training examples were. So the neural network is actually a little like k-nearest neighbors in that way.
Another way for neural networks to generalize is using invariance. For example, convolutional neural networks are approximately translation invariant. So this means that if it sees and image where the object in question has been translated then it will still recognize the object.
But it's not giving us the exact function we want. The loss function is a combination of classification accuracy and making the weights small so that you can fit the data with a function that is as smooth as possible. This tends to generalize well for the reasons I said before but it's just an approximation. You can solve the problem more exactly using minimal assumptions with a Gaussian process but Guassian processes are too slow to handle large amounts of data.


text:
Are PAC learning and VC dimension relevant to machine learning in practice?
Are PAC learning and VC dimension relevant to machine learning in practice? If yes, what is their practical value?
To my understanding, there are two hits against these theories. The first is that the results all are conditioned on knowing the appropriate models to use, for example, the degree of complexity. The second is that the bounds are very bad, where a deep learning network would take an astronomical amount of data to reach said bounds.

both of these started out many years before ML theory in (theoretical) computer science and are more abstract. they interrelate to ML theory but are rarely cited by practitioners. VC dimension seems a rough measure of entropy and might correlate somewhat with "total size of model". think it is useful for ML practitioners to have some basic knowledge of TCS & its povs. however knowledge of them might be more relevant to those who are attempting to build new more effective models in the sense of research into ML & not as much tuning individual models for max performance (aka kaggle competitions)

Possibly relevant: https://cs.stackexchange.com/q/75327/755

Yes, PAC learning can be relevant in practice. There's an area of research that combines PAC learning and Bayesian learning that is called PAC-Bayesian (or PAC-Bayes) learning, where the goal is to find PAC-like bounds for Bayesian estimators.
For example, Theorem 1 (McAllester’s bound) of the paper A primer on PAC-Bayesian learning (2019) by Benjamin Guedj, who provides a nice overview of the topic, shows a certain bound that can be used to design Bayesian estimators. An advantage of PAC-Bayes is that you get bounds on the generalization ability of the Bayesian estimator, so you do not necessarily need to test your estimator on a test dataset. Sections 5 and 6 of the paper go into the details of the real-world applications of PAC-Bayes.
See e.g. Risk Bounds for the Majority Vote: From a PAC-Bayesian
Analysis to a Learning Algorithm (2015) by P. Germain et al. for a specific application of PAC-Bayes. There's also the related Python implementation.
See also these related slides and this blog post (by the same author and John Shawe-Taylor) that will point you to their video tutorials about the topic.
The VC dimension can also be useful in practice. For example, in the paper Model Selection via the VC Dimension (2019) M. Mpoudeu et al. describe a method for model selection based on the VC dimension.


text:
Is there an AI technology that can predict human behaviour?
Is there an AI technology out there or being developed that can predict human behaviour, given that we as humans are irrational decision-makers?
I'm looking at this from an economic standpoint - the issue with current economic models is that they assume that humans are perfectly rational, but obviously this isn't the case. Could AI develop better models and therefore produce better models of recessions?

Could you clarify what you mean by "predict"? Would doing as well as or better than  another human count, even if the percentage of successful predictions is low (e.g. a human would predict someone else's behaviour and get it right 5% of the time, but a machine might get it right 10% of the time).

Irrational in the sense that humans face bounded rationality. We can't make completely rational decisions like "homo economicus" because of information failure etc.

GPT2 predicts the next word that people will say. https://openai.com/blog/better-language-models/
Facebook predicts what will make you keep using their site.
Youtube predicts what videos you will click on.

Is there an AI technology out there or being developed that can
predict human behaviour ?

If it can predict (all) human behavior, it can act as an human, thus, it will be the first real (strong) AI. This has not happened yet.
I must remark that the question contains a lot of weakly defined terms. Fix these terms can help to work in the question subject:

"human behavior" : the behavior of an individual or the behavior of all humans as a set ? (bigger groups tends to be more predictable).
"irrational decision-makers" : it assumes that it exists a "rational" way of take all decisions.
"humans are perfectly rational, but obviously this isn't the case" : same as previous.
"better models and therefore  better models of recessions" : recessions are not caused by "irrational decisions".

(In fact, each of these points could be an independent question at this site, more according to its manifest that all the plague of questions about neural nets).

authors: Влад Бекренев


text:
Is the number of neurons in each capsule in a capsule neural network hardcoded?
The capsule neural networks have been formally introduced in the paper Dynamic Routing Between Capsules.
Much ado has been made about how the capsules output a vector (magnitude = probability that an entity is present, orientation space = the instantiated parameters), which can then allow it to maintain more information than a max-pooled operation which outputs only a scalar.
Within these vector representations, the dimensions of this space turn out to be the parameters by which a written digit could vary: scale, stroke thickness, skew, width, etc. There are 16 neurons in each capsule to represent 16 dimensions.
It is unclear to me, from reading the paper, if these parameters emerged through training, or if they were hand-coded a priori. If these parameters were not hand-coded, why do such "clean" dimensions emerge? Why don't mixed-selective neurons emerge within the 16?

I have changed your title to be what I think is your question. Make sure that this is the case!


text:
Is my fine-tuned model learning anything at all?
I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet.
base_model = ResNet50(weights='imagenet', include_top=False)
x = base_model.output
x = keras.layers.GlobalAveragePooling2D(name='avg_pool')(x)
x = Dropout(0.8)(x)
model_prediction = keras.layers.Dense(1, activation='sigmoid', name='predictions')(x)
model = keras.models.Model(inputs=base_model.input, outputs=model_prediction)
opt = SGD(lr = 0.01, momentum = 0.9, nesterov = False)

model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])  #

train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=False)

test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
        './project_01/train',
        target_size=(input_size, input_size),
        batch_size=batch_size,
        class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
        './project_01/val',
        target_size=(input_size, input_size),
        batch_size=batch_size,
        class_mode='binary')

hist = model.fit_generator(
        train_generator,
        steps_per_epoch= 1523 // batch_size, # 759 + 764 NON = 1523
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps= 269 // batch_size)  # 134 + 135NON = 269

I plotted a figure of the model after training for 50 epochs:

You may have noticed that train_acc and val_acc have highly fluctuated, and train_acc merely reaches 52%, which means that network isn't learning, let alone over-fitting the data.
As for the losses, I haven't got any insights.
Before training starts, network outputs:
Found 1523 images belonging to 2 classes.
Found 269 images belonging to 2 classes.

Is my fine-tuned model learning anything at all?
I'd appreciate if someone can guide me to solve this issue.

Are you basing this code on some existing script? How did you decide to use a dropout rate of 0.8, and a learning rate of 0.01? (Dropout rate is very high, learning rate probably too high). What is your batch size?

@MathiasMüller those are from other model tests that I've been training. Actually I tried several hyper-parameters. For now, VGG16 model is performing quite well.

It's difficult to say without knowing what your data looks like but from the numbers it seems too less and the images might be too similar to one another or very different. In any case, I'd have checked using other networks like Inception and decreasing learning rate even further (say 0.0001) to not mess with the Imagenet weights if your data is not very different from Imagenet classes.

many thanks, my images are similar to one another, just like these signal images. Can you give any more updates?

In this case the number of images is highly inadequate imo. I'd try to make a custom network first with very few layers and see if there's any improvement. Huge architectures like Resnet, etc. don't generally work with small data.

how would you deal with weight initialization in case there is high similarity between the 2 classes and the number of images are around 1k for per class?