text:
Is there an AI technique (or general programming technique) suitable for seeing if two articles deal with the same event?
I'm looking for a way to work out if two or more articles deal with the same event or issue and I'm not sure where to start.
For example, back in August 2022 there were a few articles on the latest developments in AlphaFold, such as:
Nature: https://www.nature.com/articles/d41586-022-02088-x
The Economist: https://www.economist.com/alphafold-pod
Chemistry World: https://www.chemistryworld.com/opinion/why-alphafold-wont-revolutionise-drug-discovery/4016051.article
If I'm downloading hundreds or thousands of articles, what's the best way to pick out those three articles and work out that they're dealing with the same piece of new news? I imagine there must be a way given Google News groups similar news items together, but I'm not sure how difficult it would be.
Any guidance or directions would be much appreciated! Thanks.

There is one rather simple way to classify articles based on relevance if you use publication date and article title as the input to a supervised model. To learn more fine-grained distinctions, check out the event-dataset:
https://www.sciencedirect.com/science/article/pii/S2352340919304019#fig1.


text:
How to encode categorical data for a convolutional model?
Is there a way to encode categorical nominal (no ordered) data to be used in CNN models?
Let's say I need to create a 1D CNN model for categorization of time series but the values are not measurements, but categories like for example colors:

red, red, blue, green, blue, red, green
blue, red, green, green, red, red, green
green, blue, green, red, red, blue, red
and so on (can be 2D array of colors input as well)

It seems to me it would be hard to one-hot encode it. On the other hand If I change colors into integers like: 1-red, 2-green, 3-blue, then wouldn't convolutional layers assume these are ordinal or discrete data and proceed as such?

This sounds like processing sentences where the words (categories) are in some order. It’s up for debate if a convolutional architecture would be preferred to a recurrent architecture (maybe a combination), but is absolutely can be done. Brandon Rohrer discusses this around 23:00, and that approach seems perfectly reasonable for your task, if you’re determined to use a convolutional architecture.

The key lies in how the categorical data is transformed into a numeric representation before being fed into the CNN. I believe what you are asking  can easily be done through one-hot encoding.
Here is how it can be done without embeddings:
One-Hot Encoding: Represent each category as a one-hot vector. This approach avoids imposing any ordinal relationships between categories.
For example, with categories red, green, and blue, you encode them as:
red: [1, 0, 0]
green: [0, 1, 0]
blue: [0, 0, 1]
Input Transformation: If your data is a 1D time series of categorical values, transform it into a 2D input where each category is represented as its one-hot vector. The input for the CNN would then be a 2D tensor of shape (sequence_length, num_categories).
Convolution Operations: A 1D CNN kernel can then slide over this 2D input. The kernel weights interact with the one-hot vectors directly. Since each vector uniquely represents a category, the model learns patterns over these vectors just as it would for numerical time series.
Note:

The convolutional operation doesn't assume ordinality in one-hot encoding, as each category is represented independently in a distinct channel.
CNNs can learn spatial relationships between categories by examining combinations of one-hot vectors in local windows.

It is a common practice to encode categorical data using embedding layers.
In your case, it seems like you have a stream of discrete (categorical) input and you want to use a 1d convolution (you could alternatively use an LSTM, transformer or any other time-aware architecture)
The standard way to work with this data is to transform each of the discrete variables into a learned vector, which will be learned during the training of the model (i.e. an embedding layer - you can find these layers in pytorch, tensorflow and basically any other deeplearning framework).
Once the variables are transformed, you have a 2D  signal which you can use your 1D convolution on.
Converting the labels to integers is not going to work well with neural networks. As you said, this kind of transformation assumes order and would make it significantly more difficult for your network to learn the right function.

This is just my opinion, but I don't think that what you are proposing can be done on categorical data, nor do I think you can meaningfully encode categorical data so that it can be a substrate for 1D CNNs.  In a 1D CNN, a 1D kernel (a vector of numbers) would slide over your 1D dataset and carry out numerical operations on that dataset.  This cannot be done with categories (one-hot encoded or not).
This article has a very good and simple explanation of how 1D CNNs work:  Understanding 1D and 3D Convolution Neural Network | Keras.
The Kaggle exercise in the Computer Vision course also provides valuable information (see Lesson 4, Exercise)

Isn't 1D convolution commonly used in NLP tasks, where inputs are for example words (so they are categorical)?

@NikoNyrh, you might be right.  This article and this article seem to thinks so.  I think this is a good question and deserves an answer from someone more knowledgeable than I am.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Purchase prediction - What minimum feature will "upgrade" A rule-based algorithm into an "AI" algorithm?
Let's say I have an algorithm that tries to predict if a given item will be bought or not, within an X timeframe, based on it's price and other attributes.
To Do this right now, I'm collecting historic data about similar items, like prices, and I assume that if a new given item is cheaper then let's say, the avg. price of similar items, I can predict it will be bought within X timeframe, and in that case I will output "1" instead of "0".
All of this is rule based - the rule is to look at the average price, if below, output "1".
So, This isn't considered as machine learning, right? What kind of features will "upgrade" this algorithm to be considered as "AI"?

You are correct.  This is not machine learning.  Your current approach is a rule-based approach to binary classification.  However, you misunderstand what makes something machine learning.  It is not that you need an "upgrade in features".  Instead, you need to use an algorithm that takes examples (many pairs of features and truth values) and "learns" the relationship between the features and truth values.  This is what we call training.  Then, when you give the model new features, the model will be able to predict the corresponding truth values.  This is what we call inference.  There is no "hard-coding" / no "rules" that you need to program with machine learning.

Thanks, What kind of machine learning algorithms are possible to handle my use case? and will they all be "black box", or can some of them show me an actual formula that I might be able to manually modify myself?
Also, I think that some ML algorithms might be looked at as "upgrade". For example, Amazon product recommendation algorithm, could have been rule based, like if users that are similar to me in A,B,C, bought Z, recommend Z to me too. But adding ML algorithm might "upgrade" the user experience, so what kind of upgrade could that be?

None of the machine learning algorithms are truly "black boxes".  They all yield "formulas", even deep neural networks, and all can be manually modified.  However, in practice the formulas of deep neural networks can be complex and difficult to interpret by humans, which makes them effectively "black boxes".

I agree that machine learning algorithms may yield improved performance over rule based approaches and can be considered "upgrades."  (This is context dependent and won't be true in all cases.)  This usage, however, is different from your usage of "upgrade" in your post, where you talk about "upgrade of features".

Algorithms that may be useful include decision trees, random forests, support vector machines, logistic regression, and neural networks.  Decisions trees are often the simplest to understand (least like a "black box") and surprisingly can yield competitive results as more sophisticated models.  Which model is right for you depends on a lot of things like the type of features you have (categorical vs. numerical; high vs. low dimensional; etc.).  Ultimately, the best performant model must be empirically determined.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Initial State of RNN
Can I initialize the initial state of my RNN to be non-zero? I have some initial condition of the sequence and I want to use this initial condition as the initial state.

Can I initialize the initial state of my RNN to be non-zero?  Yes, the initial state / weights of a neural network can be initialized to non-zero values.  In fact, the trainable weights can be initialized any way you want.
Exactly how you do this depends on the deep learning package that you are using (if you are using a package).  For example, if you are using Keras/TensorFlow, there is built-in functionality to initialize layer weights and set model weights.  Here is a nice article demonstrating how this can be done:  "How to set custom weights in keras using NumPy array".  The article uses random normal initialization, but you can use your desired initial state / weights.

I think OP was maybe asking about the initial state, i.e. first hidden state not the initialization of the network. - @wrek?
In that case, here is a related thread that might yield an answer: https://ai.stackexchange.com/a/35465/52211

yes, the $h_0$, the initial hidden state.


text:
Epsilon-greedy action selection question
Is the following sentence correct?
The estimated values $Q(a)$ do not converge to the true values $q_*(a)$ because $\epsilon$-greedy action selection behaves randomly from time to time.
My Answer: The sentence is wrong since eventually the estimated value $Q(a)$ will converge to $q_*(a)$ and the randomness of epsilon greedy action selection won't be able to influence it if we run it for a long time.
Is my answer correct?

@Faizy Thanks but could you please explain why? I don't get why randomness is a problem for the convergence of estimated Q to the true value q.

i think you correct.... Running the epsilon greedy algorithm for a long time will minimize the influence of randomness on the estimated value $Q(a)$. The epsilon greedy action randomly selects actions with probability epsilon and otherwise acts greedily with respect to the current estimated value of $Q(a)$. Over time, the algorithm will tend to converge on the true values of $q_∗(a)$, since the random actions will be less influential.

I believe, that you are correct in saying the sentence is incorrect. Since, as you mention, the epsilon greedy action selection will just allow for the Q values to converge to the optimal q* value since the actions of the policy just provide more observations for Q which will allow for the q* to be converged to. However, this is assuming that the updating of the Q values is in such a way that each new observation has less of an effect on the Q(a). Though it is not the case that the epsilon greedy will converge to the optimal policy/action due to the randomness mentioned by the answer.


text:
How to deactivate kernels from a trained CNN model
I trained a 1 layer CNN model with 128 3x3 kernels. I evaluated the model with a prescribed test data set and now I want to evaluate the performance of this model where we only consider select kernels (meaning other kernels are set to 0). My question is, how exactly can I go into the model and set these unwanted kernels to 0? Any help would be greatly appreciated.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

You could manually create a copy of the model, reset some of the weights to zero and do the experiment that way.
But an easier way is to pass two inputs to the model: the image, and a kernel mask. Assuming the convolution layer uses ReLU, it is natural to multiply its output with the mask element-wise before passing it to subsequent layers. The mask would have values of all one when training, but during the experiment you could set the selected of the 128 values to zero. This way you don't need to modify the weights ad-hoc.
Actually this is very similar to what dropout does, except you are doing the dropout at evaluation stage rather than during training.

Thank you for your response. I indeed have a copy of my model, but I do not know how to reset such weights to 0. I tried looking at the tensorflow documentation, but I haven't been able to find anything relevant. If you happen to know how or can direct me to the appropriate resource, I would appreciate it very much.


text:
How can I use my neural network model, trained on MNIST database, on "real word" digits such as my handwritten digits?
I have developed a feed-forward ANN from scratch trained (and evaluated) on MNIST database, which contains 60,000 + 10,000 handwritten digits samples.
Can I test my model on other digits, for example I write the digit 7 on a paper with my pen and check whether my model would recognize it or not? How can I do it?

Is this more of a technical question, regarding how do you write digits on a paper, take photos, crop them and feed them to the network or something else?

Can I test my model on other digits, for example I write the digit 7 on a paper with my pen and check whether my model would recognize it or not? How can I do it?
Yes, definitely.  One of the main goals of machine learning is to be able to train a model and use it on your own data to make predictions.  BUT you must ensure that the handwritten digits are produced and processed in the same way as the training data.

Example 1:  If the MNIST digits dataset uses black pen, you must also use black pen (preferably of the same type).
Example 2:  If the MNIST digits dataset has digits on a white background, you must also produce digits on a white background.
Example 3:  If the MNIST digits dataset has digits that are centered in a 28x28-pixel image, then you must do the same.

This will help keep your dataset "within distribution" as the training dataset.  Departures from the pipeline that generated the training data will result in degradation in performance.  By the way, there is usually some degradation in performance anyway when models are deployed in the real world.
After you have captured and processed your "custom" handwritten digits, you must package them in a way that can be processed by your model.  For example, models in Keras/Tensorflow will take images as Tensorflow tensors and Numpy arrays.  You should preferably package your data the way MNIST dataset is packaged in your download.  Keras has the .predict() method that allows you to make predictions / inferences on new data (see article).


text:
SQL Machine Learning using matrix multiplication
What is the easiest classification algorithm in SQL when my data looks like this?
A = 101000101110
B = 010101110010
C = 100101101000
B = 100101101101
C = 100010000001
A = 100010010101

These are binary vectors that indicate the presence of a certain attribute.
I was thinking about training some kind of regression using python that gives me a vector.
Then I would only need to multiple the vector with the binary vectors and I would bet some number indicating me what kind of label I have using a threshold method.
Are there any python algorithms that allow me to do that?

Unless SQL has changed in the last 10 years and I missed it, SQL doesn't have machine learning algorithms... It's a language for specifying a query out of a database.  I suspect you might need to reword your question

The question, the data, and the goal are all unclear to me.  @Ohumeronen, please review this article on "How do I ask a good question?" and revise your post accordingly:  https://ai.stackexchange.com/help/how-to-ask.  You are more likely to get the help you need if you help us help you.

You have a few overlapping topics here:

Fitting a model to your data in Python
Storing model parameters to the database
Executing SQL queries to apply the model

I would start with a linear model, such as logistic regression or linear discriminant analysis.
You should think carefully how you structure the data in the database. This is more of a programming topic, but I would have a table with few columns and several rows to represent each item. This way you don't need to modify the schema if your data gets a new input feature.
Then as a final step you'll need to "just" write the SQL query, joining attribute data and model parameters and output. But this is off-topic on an AI forum.
Getting back to linear regression, you must re-arrange the binary representations into separate columns:
array([[1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0],
       [0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0],
       [1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0],
       [1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1],
       [1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1],
       [1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1]])


text:
Fine tuning a Deep Learning model post training
I have trained a CNN in a binary classification problem, however the original problem has 6 different classes, of which, I am only interested in classifying one, so if it is that certain class or not.in this case, let's say class 2.
After looking closely into the model's performance on test dataset, I have found that the model confuses class 2 with class 1 often. Is it common practice, to make a balanced dataset from the data that I have only from class 1 and class 2, and further train the model on that dataset? Are there any pieces of research/papers on this? If no, what other possible solutions would there be, of course other than making a new model?

You can use the technique of Transfer Learning to fine-tune your model. You can take the weights from the pre-trained model and then use them as initializations for your own model.
Yes!, It is common practice to make a balanced dataset when training a machine learning model, as this can help prevent the model from overfitting to one class or the other. A model trained on a balanced dataset would be more accurate at predicting the classes.
In your case, if you have more data from class 1 than class 2, you may want to downsample the class 1 data so that both classes are represented equally. You can then train your model on this balanced dataset and see if it improves performance on the test set.
There is a great deal of research has been conducted on the topic of data balancing and its impact on machine learning models. few papers on the topic include

On the effectiveness of preprocessing methods when dealing with different levels of class imbalance

Learning from Imbalanced Data

A review on data preprocessing methods for class imbalance problem


text:
How to make my neural networks designs more robust
Whenever, I design a neural network to solve a novel problem (requires a novel loss function i.e. not image classification) it always ends up being hypersensitive to batch size and learning rate.
Andrej Karpathy compared training neural networks to balancing a pencil on your figure. It seams things like image classification are only workable because people have stumbled on to magic tricks like batch norm, skip connections, relu. And as you venture away from image classification, machine learning returns to a hypersensitive balancing act.
Andrej Karpathy recomends understanding backprop/autograd. I imagine reducing a batch of losses by mean rather than sum, will help make the gradients more invariant to the batch size resulting in the neural net being more robust to choices of the learning rate. Currently, my loss function involves calculating a covariance matrix and probably dividing by $n^2$ would help with the hypersensitivity.
I'm wanting to know if there are any references of how you can use your understanding on backprop to design neural networks that are more robust (less hypersensitive) to arbitrary choices of hyperparameters.
Is the job of machine learning engineers to balance hypersensitive hyperparameters (i.e. find lucky choices) or is the job to design networks that are robust to arbitrary choices?
It seems like most people just use exactly what architectures the gods at google research have handed to them (i.e. transformers haven't really changed) rather than taking a more humanistic approach of making these problems more tractable.


text:
What can go wrong using batch norm?
Andrej Karpathy said he's shot himself in foot over and over gain with batch norm.

I mentioned no one likes this layer. It causes a huge amount of bugs and intuitively it's because it is coupling examples in the forward pass of a neural net and I've shot myself in the foot with this layer over and over again in my life and I don't want you to suffer the same so basically try to avoid it as much as possible. Some of the other alternatives to these layers are for example group normalization or layer normalization.

How exactly can you shoot yourself in the foot with batch norm?

I think the problems with batch normalization arise mainly due to the large misunderstanding of what it does, combined with a simple but confusing name.
Despite the simple computation (everybody knows what normalization is), it is not so clear why batch normalization provides the benefits it does. In the original paper the main explanation given was that batch normalization reduced covariat shift for the parameters in very deep layers, but there are many papers out there who proved already that batch normalization also smooth classic loss functions, which means it can be considered as a regularization layer. And indeed some papers pointed out that combining batch norm with other regularization techniques like dropout doesn't really provide any benefit. You can find all the papers link in the first answer to this question.
So this already answer how you can shoot yourself in the foot with batch normalization, said that it's hard to say a priori when you should use it or not, unfortunately like many other aspects of deep learning this is still a matter of trials and errors.


text:
How are OCR training datasets constructed?
For the sake of concreteness: let's suppose that the word "OCR" refers to any OCR system build on an R-CNN architecture. Similarly, in aims of simplicity, let's declare that we are interested in reading digits between 0 and 100.
Question: How should I construct a dataset (given the aforementioned goal and architecture) ?
My understanding is that I need to collect images of all the digits from 1 to 100 and label them with its corresponding digit. Is this premise correct?
My struggle is that I can't fully understand how this seemingly tedious procedure is generalized to OCR that read more general types of characters (language-characters for example or if I generalize the problem to detect the numbers from 0 to $10^{10}$).
Thanks in advance!

OCR - optical character recognition identifies individual characters. So in the case of numbers, there are just 10 classes to learn. And this doesn't depend on how large numbers we are looking at. There is a separate step to segment observed numbers (or words) into single digits (or characters).
But on low-quality images this character segmentation is non-trivial, so better (and more complex) systems work with a larger context.


text:
AI in signal processing: how to reduce data volume
I am studying 5G technology, where AI/ML is integrated to imporve performance. I am not expert in AI/ML and I am sorry if my question is stupid.
Honestly i am quite not sure If I understand the main concept of AI
My question: AI/ML can predict received data, right? I understand it as channel estimation methods in signal processing/ receiver side processing. If we can predict the received data, can we decrease data volume using AI?

5G technology incorporates Αrtificial Intelligence in a number of ways.

It can help to manage Network traffic and optimize its performance.
It can also be used to improve the accuracy of predictions about future network conditions. In addition, AI can be used to develop new 5G applications and services.
It can be used for Predictive Maintenance.
Site Selection
Network Design

Yes, AI can predict received data by analyzing Past Data and trends in digital signal processing.
AI can definitely be used to decrease data volume. By using prediction methods, AI can learn to anticipate what data is coming in and only send the necessary data. it can be used to automatically extract features from signals, which is a difficult and time-consuming task for humans.
here are a few ways where AI can be used to reduce data volume in Signal Processing:

Pattern Recognition Technique in digital signal processing to automatically identify patterns in digital signals. This can then be used to reduce the overall data volume.

Denoising can be used to reduce the amount of data required to represent a signal.By removing noise from a signal, AI can improve the Signal-to-Noise Ratio (SNR), which can lead to improved communication quality in 5G systems.

Dimensionality reduction & Feature Selection Techniques that can help identify a smaller set of features that are most relevant to the task and remove redundant data, thereby helping to reduce the number of features that need to be considered, which can again reduce the amount of data that needs to be processed.

Finally, prediction can be used to generate new data that can be used to represent a signal, which can reduce the overall volume of data that needs to be processed.

For example, if we know that a particular data stream will be mostly static for a period of time, we can send fewer data during that time period, since we know that the prediction will be accurate. This can help to reduce bandwidth usage and improve efficiency.

Here are some other potential uses of AI in signal processing include:

Automated signal enhancement –  Removing background noise from an audio signal.

Automated feature extraction –  Extracting relevant features from an image for further analysis.

Automated pattern recognition – Identifying patterns in data that indicate a particular phenomenon.

Optimizing signal processing algorithms –  Finding the most efficient way to process a given signal.

Developing new signal processing methods –  Using AI to design novel algorithms for processing signals.

Selecting optimal Sensor and actuator placements-  Selecting the best locations for sensors and actuators to get the most information from or control over a system.

Designing digital filters- Designing filters that remove specific types of noise from a signal.

Incorporating domain knowledge into signal processing – Using expert knowledge to design more effective algorithms.

Interpreting results – Providing meaning to the results of signal processing for decision-makers.

Generating hypotheses – Formulating hypotheses about how a particular signal was generated based on its characteristics.

Explaining results – Providing justification for the results of signal processing to stakeholders.

Hello Faizy! Thank you very much for taking your time and giving this explanation!

can we improve link budget by using AI/ML?


text:
Encoding data in which the order of features does not matter
My input to the model is a set of features that I encode in the form of five vectors of the same size consisting only of 0 and 1. I now want to combine them into one vector in such a way that their order does not matter.
My first idea was to run each vector through the same activated linear layer and sum the results.
Is there maybe a better way?

Could you give us a concrete example? And what do you mean by "the order of features does not matter", is it that if you permute their order the network's output should be unchanged?

@NikoNyrh Simply put, I have to encode a set of sets of features. By set I mean an unordered collection of unique elements. I can encode each inner set as a vector of zeros and ones. Now I need a function that will turn the outer set into a single vector. Simply adding or multiplying these vectors seems to me not to be the right way.

Is there maybe a better way [to combine them into one vector in such a way that their order does not matter]?
There are many ways to perform feature fusion (see ref).  The way you described is one way of doing it.  However, for most simple cases where feature fusion is for the type of data you are describing (i.e., tabular data), it is more common to concatenate the features at the input level.  The order of concatenation does not matter to the neural network.

Could you elaborate on how for a neural network, the order of concatenation does not matter? From the same vectors, depending on the order of concatenation we will get completely different inputs to the network. How could it not matter?

Each of the input features feed into each one of the neurons in the first hidden layer.  The inputs are, therefore, all treated the same.  Thus, the order of concatenation doesn't matter.  This diagram shows how that is true.

I disagree that concatenation would be invariant to the order of the inputs. In the given example the values of h_1 will change if you permute the inputs, assuming that is weights are distinct for each input.

The important thing is to be consistent about the order in which the inputs are presented to the model, not the absolute ordering itself.  Think about it this way:  If you are feeding a neural network two features - petal length and sepal length for example - to predict the iris species, wouldn't it be a bizarre property of the model if it mattered which feature was in column 1 and which was in column 2 of your datatable?  Regardless, the order invariance is a directly related to the commutative property of addition.  w1input1 + w2input2 + b1 = w2input2 + w1input1 + b1.

You are right, @NikoNyrh. You cannot change the order randomly.  Once you have decided that feature 1 is feeding into input neuron 1, you must be consistent.  My point though is that the initial ordering of which feature you decide to be feature 1 which which one you decide to be feature 2 and so on is not considered by the model.

My first idea was to run each vector through the same activated linear layer and sum the results.

That was my first thought as well, but actually it doesn't need to be a single layer but instead it can be a more complex neural network. Basically it will learn an embedding $R^n \rightarrow R^m$, and you can use the sum (or average) of such embeddings to represent the set of your features. I wouldn't use any activation in the final layer, or maybe at most tanh to preserve "symmetry" between negative and positive values. Or who knows, maybe ReLU would work in you problem as well.
I think average of the embeddings would be better than a plain sum, in case the number of "active" features may change and absent features have an embedding of all zeros. Or you normalize the sum so that the L1 or L2 norm is 1.
Come to think of it, the $R^n \rightarrow R^m$ can be implemented as a stack of 1D convolution with "kernel size" and "stride" of one (Keras terms). Or it can be a "stand-alone" model of which outputs you sum together.


text:
how to apply crossover and mutation rates in genetic algorithm?
I'm working with genetic programming and let's say I have the following operator:

pop_size = 100
Crossover ratio = 0.4
Mutation Ratio = 0.2
Selection Ratio = 0.1

What is exactly the next generation will look like?

Select 10% from the last pop with any selection scheme.
Fill 40% of the next generation with the crossover between parents.
Mutate the new offspring based on that mutation ratio before adding it to the next generation.
That will be 50% of the next generation filled up, and we have to fill the remaining 50%, how to do this? by mutating the individuals from the last generation according to the mutation ratio? add some random genes to the next generation?

The remaining will be models selected from the original population based on the best performers (or some other user defined criteria).  You don't do anything to those models - no crossover and no mutations - just keep them as they are.

so first of all we gonna select 10% with any selected method, after that, we gonna fill the remaining 50% of the next generation with the old individuals' best performers; if we sum this we gonna find 60% of the last individuals will be added to the next generation?


text:
Propagating gradients through an "Item Selector" network
Consider the following problem:
There are $N$ items and $S$ slots. Each item is a vector of length $D$.
The goal is to train a neural network to select one item per slot in order to minimize the loss function $L$, given any arbitrary list of items. This loss function could also be thought of as a reward function; it's not comparing labels, it just outputs the 'value' of some slots/items configuration, and it's differentiable. (In my particular application, this function is another MLP which can be trained separately; it's an actor-critic setup. But this problem applies to any sort of differentiable loss/reward function).
Assumptions:

The same item can be used multiple times in different slots.
We have an infinite number of training examples available; they don't need to be labelled so they can be generated automatically.
$S \times N$ is large, such that we can't practically enumerate the value of $L$ for all possible configurations of items in slots for a particular example. (If we could, then this would turn into a simple supervised learning case).

Here's an example where $N = 8$, $S = 4$, $D = 1$. The neural network is an MLP which outputs an $S \times N$ matrix of logits. These are then converted to a binary matrix which is multiplied with the original list of items to get the selected item in every slot, on which the loss can be calculated.

The problem is what should happen between the neural net output and the binary 'selection' matrix (The ??? bubble in the diagram).

It's natural to do an argmax/one-hot operation on the logits (i.e. push the max logit to 1 and the rest to zero). But this operation isn't differentiable, so the MLP can't be trained.
You could do a softmax instead, which has good gradients, but then the problem is broken because the selected items are weighted combinations of items in the original list, which isn't allowed.
You can also do softmax with "temperature", e.g. softmax(logits * 10e5), to get a more argmax-like behavior in a differentiable function. But in practice, the gradients are low quality and the network doesn't train properly.
You can try argmax for the forward pass, but softmax for the backwards pass - this kind of works, but again in practice the neural network does not converge to a good solution.

Is there any kind of neural network architecture or training regimen that can solve or sidestep this problem? Or is this application fundamentally unsuited for back-propagation?

As soon as you discretize the selection, i.e. make a hard selection (argmax) instead of a soft selection (softmax), you have a biased gradient. This is because the things you didn't select are not included in the gradient calculation.
However, it is possible and one way to create such a biased gradient is to apply both techniques. You can use the argmax method in the forward pass and the softmax (with temperature) in the backward pass .
Here is what that could look like, independent of framework:
def differentiable_select(logits, temp=0.6):
    # turn logits into a categorical distribution (soft-selection)
    logits_dist = softmax(logits / temp)
    # get the maximum probability for the forward pass
    hard_selection_idx = logits_dist.argmax(-1)
    hard_selection_mask = one_hot(hard_selection_idx, depth=logits.shape[-1])
    # for the forward pass, return the hard_selection_mask,
    # for the backward pass, return the logits_dist (soft-selection)
    return logits_dist + stop_gradient(hard_selection_mask - logits_dist)

# generate the selection mask
select_mask = differentiable_select(logits)
# select from x (x-shape: [n_items, n_dims])
x_select = x * selection_mask

The central part is the stop_gradient function. In PyTorch this corresponds to Tensor.detatch() and in tensorflow you'd call tf.stop_gradient.
That way, in the forward pass, you have the hard selection:
logits_dist + hard_selection_mask - logits_dist == hard_selection_mask

and in the backward pass, you have the soft selection by using logits_dist. The temperature ($\tau$) is a hyperparameter that trades training stability (if $\tau$ is close to $1$) vs. similarity to the forward pass (if $\tau$ is close to $0$). In my experiments, a temperature of $\tau = 0.6$ worked pretty well.
This solution is derived from the Straight-Through Gumbel-Softmax (ST-Gumbel-Softmax) function. In the original implementation, the ST-Gumbel-Softmax function includes non-deterministic sampling from the categorical distribution.
For further reading, you can look at the following sources:

[Paper by Bengio et al. on ST-Gumbel-Softmax]
[PyTorch Code Example]
[Blogpost by Fabian Fuchs]
[Maddison et al.]

Thank you for your in-depth response @Chillston! I had looked into Gumbel Softmax before posting the question - as you can see I attempted both a temperature-based approach as well as a "hard" softmax. However I found in practice that it did not converge to a solution.  Here's a gist of a Keras implementation of the problem, where the goal is to select the option with the value '5'. It's also confusing to me that all the literature focuses on stochastic networks and Gumbel noise, but this problem is totally deterministic.

Ah, right - I must have missed that you wrote that. Anyways, in your case, I could imagine that providing numbers from 0 to 10 as inputs may cause the inability to converge, you could try providing numbers 0.0, 0.1, .., 1.0 and train on selecting 0.5.
Beyond that the method should work like you implemented it.

Regarding the sampling from the Gumbel distribution: My current understanding is that this method has an exploration/exploitation problem (as in Reinforcement Learning), so sampling instead of selecting the argmax should improve exploration of the solution space. In your case this shouldn't be an issue though, as you only select once and not cascading multiple consecutive selections. But I'm interested in that as well, so it would be cool if you'd let me know if scaling the dataset values to [0, 1] works.

Scaling the inputs doesn't have an effect (I had tried this before, but left them unscaled for simplicity in the example). In both cases, If you check the network outputs, the logits trend towards being equal (presumably because the backwards pass softmax loss kind of cancels out numbers above and below 0.5)

Okay, another idea: Your model does not use permutation symmetry, because you flatten the input directly after passing it to the model. Thus the model has to learn all possible combinations of inputs, when instead you could apply the model to the options separately. You can try leaving out the flatten (L.31) and setting the output neurons of the Dense layer in L.35 to 1. Maybe this helps?

I see what you're going for, and I do something similar in my "full" application with 1d convolution layers to apply the same kernel to every item individually. But At the end of the day it is order-dependent because the output needs to match up to the input in order for the selection to work

I don't fully understand these points. When you use a 1D-Convolution, do you set the kernel-size to 1? Otherwise you'd still have the sequential bias in your model. And from what I know about your project so far, I'd disagree with you on the order-dependency. The model should not be concerned with order if the only important thing is that input and output are ordered similarly - here is an example: If you use an MLP and apply it separately to each item in the sequence, you can still ensure that each output $y_i$ belongs to the corresponding input $x_i$, right? Or maybe I'm missing something

Hey @Chillston - just to follow up on this. It turns out the secret is actually all in the Gumbel noise. If you apply Gumbel noise to the example I posted, it works flawlessly. And it translated to my real application as well! A really interesting result that I don't fully understand...

Hey thanks so much for sharing! That's really interesting! You are only selecting once, right? So it's probably not the case that the noise boosts exploration of the solution space.. Are you planning on investigating this further?

what do you mean by only selecting once? It only selects once (per item in the batch, per training step) if that's what you mean?

Exactly, in a current project of mine, we have cascading selections where the pool of options depends on a prior selection and in that scenario it makes sense to me that the randomness is a benefit. However, it is not intuitive to me at all why it helps in your case :D If you ever find out, I would love to hear what it is!


text:
How does Tesla's autopilot generate 3d reconstruction of road?
Tesla uses stereo vision and in many applications it works OK but ground reconstruction is uniquely difficult since there aren't features to match.
I'm wondering how 3d reconstruction works here? Does the autopilot assume the ground plane is flat or is something else going on to infer road curvature?

Tesla's autopilot likely uses a combination of sensor data and mapping data to generate a 3D reconstruction of the road. The exact details of how this is done are likely proprietary information. However, it is likely that the autopilot uses the data to generate a 3D model of the road that it can then use to navigate.


text:
why validation accuracy be greater than training accuracy for deep learning models?
I hope you are well.
I had a problem and didn't understand the answers given on questions similar to my question.
If possible, please answer this problem in a simpler way.
Val_acc : %99.4 _
Train_acc : 97.24
Thank you for attention.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

Why validation accuracy be greater than training accuracy for deep learning models?
You are probably thinking that the training process is supposed to reduce the training loss (and increase training accuracy).  The influence on the validation loss (and validation accuracy) should be secondary, no?  And you are 100% right to think that because the loss function only considers training labels (y_train) and how far model predictions (y_pred) deviate from those training labels.  So why is the validation accuracy higher than the training accuracy?  So long as you are not overfitting (which you clearly are not), the training process should have a benefit for both the training accuracy and validation accuracy because the datasets come from the same distribution.  The reason why you may have small differences like you are seeing here has to do with random sampling of the two datasets from that identical distribution.  (You may have seen i.i.d. elsewhere.  It means independent identically distributed.  Each example in the two datasets are presumed to be i.i.d.)
Bottom line:  The two samples are different, so they have some differences in accuracy.  It just so happens that the accuracy is higher for the validation set in this case (probably within some margin of error).
On a separate note, to get a better estimate of your true accuracy along with error bounds, you may consider performing a cross-validation of some sort.  You can review the scikit learn article to see how this is done.

Let consider this example:
You have a Neural Network for binary classification. The NN is classifying A and B. Let say the NN is not very sophisticated, and it does just always pick A for classification. So on balanced Test-Set the accuracy would always be 50%.
The Validation accuracy depends on your Validation-Set. Imagine you have the NN from above and your validation-set consists only of A-labeled examples. The validation accuracy would be 100%, while your Training metric would be 50% with a balanced data set.
When validation metrics perform better than the training ones, this could be the case if the validation set is "simpler" than the training set.
I also strongly disagree with the other answer, that "you clearly do not overfit". You can't say for sure based on the data we see. I don't say you do, neither. It depends on whether your data is balanced and diverse.
One of the best ways to see if you do overfit is to make a k-fold split.

Validation accuracy being higher than training accuracy can typically happen for three major reasons:

You have some kind of a regularization that is switched off during validation. Usually this happens with dropout regularization.

Validation score is computed once after the whole epoch (after the net have seen whole dataset), while training score is averaged over the  epoch.

Random fluctuation.


text:
Alternatives to brute forcing neural network plateau
Below is the loss of the same training run at different scales illustrating the plateau phenomenon.

Source p3.
It seems to me that each dip adds constraints to the neural network optimization, and that neural networks must be highly over parameterized to enable the optimizer to find paths through to the next dip.
I am trying to train timeseries features for a covariance matrix (i.e. einsum('bic,bjc->bij', model(s0), model(s1))). The first dip corresponds to the neural net maximizing the cosine similarities, while being linear in the magnitude i.e. $\gamma\lVert s \rVert =\lVert model(s)\rVert$. The next dip corresponds to the the neural network being nonlinear in magnitude (i.e. silence, and everything else, being mapped onto the unit sphere).
Here are a few responses to plateauing

Regularize the architecture (an architecture that is non-linear in magnitude)
Regularize the loss function (punish being linear in magnitude)
Iterate on the training set to find architectures/hyperparameters that plateau less.
Brute force it. Just leave the neural network training.

I'm sceptical about 1 and 2 as it would only get you past the first plateau. I don't hear much about 3, although I think it might be a good idea. Does anyone do this in practice?

I think the primary solution to plateauing is improving the dataset.
The iteration process should be

Train the neural network
Identify what properties are causing the neural network to plateau.
Gather more data, improve the targets, argument the existing dataset, to target these undesired properties.
Find the next plateau, and repeat until the desired results are achieved.

We are changing the loss landscape using the dataset.


text:
Are there free and open-source audio versions of Generative AI programs like Stable Diffusion?
Generative AI is being used to create amazing art; first through paid services like Midjourney and now also with free, open source alternatives like Stable Diffusion. Now you can even generate art in a particular style, first with Google's Dreambooth and later with open-source implementations of the same.
Is there a Generative AI program for audio that allows you to create a model/style of a particular voice exemplar?
I am looking to train a voice model in a particular style using the aural equivalent of Dreambooth, and then apply that voice model/style to written text. Ideally I could review several options, and then pick certain ones to expand upon and later upscale.
I have researched this extensively but all of the voice generation services I have found are 1) paid 2) closed source 3) don't allow you to train the software on specific audio samples or 4) don't sound very natural at all.
Are there modern generative AI services for text-to-audio?

You can check sample-generator, an implementation of stable diffusion for audio data from Harmonai. I'm playing with it recently and I can say it works pretty well already out the shelf. With some hacks I was also able to reduce the model size and run a training on cpu.
And if you have the proper gears they also provide pretrained weights of some of their models.


text:
Are there metrics for image complexity for informing neural network design?
BACKGROUND: I am trying to think of rational approaches to designing deep learning models for image classification.  One thought is to quantify the complexity of image datasets and use that to inform model design.  By the way, I know that rational model design is much more complex than just quantifying image complexity, but right now, I'm in the brainstorming phase.
In the below examples, I qualitatively describe the complexity of the images as a function of the number of channels, complexity of foreground object, complexity of the background, and the number of classes.  Certainly, there can be many other factors, such as image dimensions or bytes/pixel.  I think all images below are 28x28-pixels and 8-bits/pixel.
MNIST digits           --> greyscale, simple objects belonging to 10 classes and a white background
MNIST digits corrupted --> greyscale, same as above but with added noise
MNIST fashion          --> greyscale, more complex objects belonging to 10 classes and a white background
CIFAR-10               --> RGB, even more complex objects belonging to 10 classes and complex backgrounds
CIFAR-100              --> RGB, same as above but with 10X more classes

Based on the above one has a subjective sense that image datasets can be ordered as shown below based on complexity.  It is reasonable to hypothesize then that the computer vison models should be progressively more complex as well (more neurons, more layers, more parameters, etc.).
MNIST digits > MNIST digits corrupted > MNIST fashion > CIFAR-10 > CIFAR-100

SPECIFIC QUESTION: Are there any existing quantitative measures of image complexity that capture these aspects of image datasets?
PRIOR RESEARCH: Various searches lead me to computational complexity and model complexity, which are not what I'm looking for.

In the realm of computer vision and machine learning, "task" complexity is more important than image complexity. This is one of the major reasons that research has not formally described any specific measures for image complexity. Fundamentally image complexity is actually defined by task complexity. For example, classifying objects which look very similar to the background is more difficult than objects that distinctly stand out from the background. Additionally model complexity is something dependent on task complexity as well rather than image complexity. You can generally overcome challenges with image complexity by training strategies (say with large variety of samples) even with simple models.
For your specific question, I think the natural approach is to maybe define a basic task and then derive complexity measures that affect the quality of the task. For example, let's say classifying fish images (task), there the image quality metrics can be items like brightness, blurriness of image, amount of bluish tint in the image, etc.


text:
MCTS for trick-taking game?
I'm trying to implement a MCTS-based AI for a trick-taking card game.
The game : (Belote)

The play consists of 8 tricks.
A trick consists in each of the 4 players play successively 1 (legal) card from their hand (starting with 8 cards).
A player can only play certain cards from his hand (depends on the current state of the trick). Thus, the other players can infer information of some other players hand based on the current and past tricks.

My implementation so far:
The node:
The current state of the round (previous tricks, current trick, the play order...) + the position of the "smart" player.
Expansion function:
I can think of 2 possibilities :

a card is played by a player
a whole new trick is played

Other tools that I implemented:

A method that gives the legal cards of a hand (based on the current trick).
A method that gives the smart player all the possible cards of another player (based on the current and previous tricks).
A method that generates every possible next (legal) trick (based on the 2 previous methods).

Problem :
If I want to apply MCTS, I need to ensure that the succession of the simulated remaining tricks (or played cards) until the end of the round (ie the path in the tree) will not be a dead-end before reaching the final (32th) card.
Indeed, a player could run out of legal cards during the simulation...
How can I deal with that constraint in an MCTS context ?
Is MCTS even a viable algorithm for what I want to achieve ?
I could of course precompute the whole possible tree ahead of time, but his size would be huge, and the interest of the MCTS is kind of lost.

How do the game rules deal with situations where there are no legal cards left for a player?

There is always at least one legal card in every hand. The rules only specify a "priority order" of cards in a hand with respect to the current trick. "If you don't have these cards in your hand, then you should play these, if you still don't have these cards, then you have to play these... etc.". The legal cards in a hand depend on the current trick and the whole hand itself.

This is in real life. Now, in my MCTS implementation it's different because I make the opponents play cards that they possibly have, but the smart player doesn't know their actual hands of course.

Ah, I see. Could you randomly sample one possible set of cards at the start of the rollout and then keep taking cards from that set until the end of the rollout? More generally with hidden information everything becomes a lot more complicated game-theoretically, with things like bluffing and trading off good moves vs hiding information. Unfortunately I don't think there are any simple extensions to MCTS to deal with this (yet).

If I understand correctly, you suggest to distribute the 3 other players the 24 remaining cards (32 - 8 known cards from the smart player's hand) and rollout from this node. And repeat N times with different initial distributions.
I've tried that but unfortunately it did not work very well. I suppose that it because there is $\frac{(3*8)!}{{8!}^3}$ ~ 9 billions possible initial distributions. And even a 1 card change can have a great impact.
If not with MCTS, do you see any possible way to achieve that ? Reinforcement learning ? Thanks.

Yeah I'm not too surprised it doesn't work that well, naive MCTS seems a bit hopeless it this case. For hidden information games two recent approaches (indeed using reinforcement learning) are Player Of Games and ReBeL, so consider looking into those or some of the related work they cite.


text:
Group image classification for whether containing unrelated images
I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.
Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.
For example,
if the images are {apple, apple, apple} then the label is false.
if the images are {apple, basketball, basketball} then the label is true.
I would like to build a model to classify whether those group of images contain such unrelated images.
Seems the idea appeared in my mind is to have a model such as following

image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.

Loss function would just be the binary cross entropy with the actual label.
I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!

This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.
Here is a survey paper Multiple Instance Learning: A Survey of Problem Characteristics and Applications

Thanks for pointing to the keyword! that's exactly what I'm looking for :)


text:
How are gradients of individual layers computed?
I have been reading some papers recently (example: https://arxiv.org/pdf/2012.00363.pdf) which seem to be training individual layers of, say, a transformer, holding the rest of the model frozen/constant. In the case of the paper I read, this was done in an attempt to minimize parameter changes so as to reduce knowledge "lost" by a model when it is updated for new information.
My question is, how are individual layers of a transformer trained? Like, if we run the transformer and get a gradient, how can we use that gradient to train, say, the first layer, without affecting the rest of the layers at all?

The gradient that we use to train neural networks is the gradient of the loss function with respect to the parameters of each layer.
The parameters usually form a very large vector, concatenating the parameters of each layer, and you compute gradients with respect to that large vector.
Then to train a single layer, you just take the gradient of the loss with respect to the parameters of that layer only. Mathematically it is very simple.

The way to "train individual layers" is to fix the weights of all other layers during training.  This notebook describes an implementation of how to fix specific layers of model weights (see section Fine-tuning a pretrained model, Freezing all layers until the fourth from the last).  This example is for a convolutional neural network, but the same approach can be applied to transformers.  Those weights become non-trainable, meaning that during training they cannot be updated.  Only the weights of the layer(s) of interest will remain trainable (and updatable).
This is useful when the model was previously trained on other data (pre-trained model), and while you do not want to undo all of that training (i.e., you don't want to change the weights that were previously learned), you do want to refine the model with new data that you have.  Allowing weights of just some of the layers to be updated during training is one way of doing that.


text:
Interpretation of the roc curve on test set
Hi i got the following roc curve:

What does this mean? has this to do with overfitting? Is my data wrong preprocessed?
i do not understand and would appreciate an answer.

You cannot see overfitting in this curve, because to evaluate overfitting, you need to have training and validation losses, and a ROC curve does not present that.
The only informationn you can obtain from this curve is that for your dataset, a random forest is better classifier than KNN or logistic regression, nothing more.

The Receiver Operating Characteristic (ROC) Curve that you are showing helps in evaluating and comparing the performance at binary classification of machine learning models (see article).  For different thresholds (not shown) of the model's output probability of the positive class, the ROC curve shows the Sensitivity (True Positive Rate) vs. 1-Specificity (False Positive Rate) for the binary classification task.  Thus, you can see how changing the threshold changes the Sensitivity and Specificity.  Generally, setting the threshold low results in high Sensitivity and low Specificity.  As the threshold is increased, the Sensitivity decreases and Specificity increases.
The area under the ROC curve (AUC) is a measure of how well the model performs.  If the model were presented with all possible pairs of positive and negative examples from your dataset, the AUC is the proportion of pairs that the model would correctly identify which is which.  The maximum value that AUC can have is 1, and this is the AUC a "perfect" classifier would have.  The diagonal line indicates the performance of a naïve model (a dummy classifier) that predicts randomly, and as such, the AUC for the diagonal is 0.5 (i.e., coin flip).  Therefore, in your example, the Logistic Regression model has the best performance and the Random Forrest model has the worst performance.
You only have the ROC curves for the test set.  If you had the same for the training set, then you would be able to compare the AUC (i.e., a measure of model performance) on both the training and test sets and be able to evaluate for overfitting (or underfitting).  For example, the model performing well on the training data (high AUC) but poorly on the test data (low AUC) would be an indication of overfitting.


text:
Can i train xgboost on multiple time series csv files at the same time?
I built an xgboost model to predict stock it now trains on 1 stock at a time its a csv file I use pandas to load it.
Is there a way to train the model on multiple stocks at the same time? What would be the best approach?
I don't need code just guidance as all my research has failed

The short answer is no.
Time series models model a single time series. If you want to model N time series you need N time series models. XGBoost is no different when used for time series modeling. The only exception to this would be deep learning using transfer learning but that is a different animal.


text:
Ways to improve DQN model learning snake?
I'm training a Deep Q-learning model on a snake game and I would like some ideas on how to improve the model and maybe also efficiency of training it.
The game is currently set to a 12x12 grid, a blue snake with a green head and a red apple. The network is fed with 3x12x12 input parameters (RGB, width, height) and gets a positive reward when an apple is eaten and a negative reward when it collides with something.
It does learn, but plateaus around 12-13 apples per round (on average) after 3 million steps:

What I have tried:
Giving a partly reward on the steps before a "real" reward. For example:
Step    Action        Reward
N       Go straight      100
N-1     Go straight       50
N-2     Go straight       33
N-3     Go left           25

That was just an idea but it does not seem to work as I hoped.
What else can I try? What I don't want to do is tinker with the game, I just want the visual input and nothing else.

Hi @DanneP and welcome to AI Stack Exchange! If possible, could you supply a few more details, such as the specific variant of the DQN algorithm you are using? That would help us determine what improvements could be made. Thanks!

Here are your options:

Try different starting points in the path.
Try creating more paths to train your snake.
Try using genetic algorithms to "evolve" your algorithm more systematically.
I don't think that snakes eat apples, so try changing the reward to a mouse if you are serious about getting good performance out of your snake.
Some snakes are smarter than others, so try using a cobras, pythons, etc.  Do not use titanaboas because they are extinct, probably for a reason.

I would suggest looking at various optimisations used by Deep Mind when they originally developed DQN. You can check their "rainbow" paper for ideas that improved performance.
In addition, I think you are missing the idea of frame stacking, which should help the agent better understand velocity which is currently not in your state representation, although the position of the head will often be a clue. To frame stack, change the representation to include the last few  turns (3 turns total might be enough for you) and concatenate the channels. You could also take the opportunity to flatten the representation of a single frame*, since each grid point only ever contains a single entity (e.g. you could use 0 for empty, 1 for apple, -1 for snake body and -0.5 for snake head). That should reduce the number of parameters required for the estimator, which in turn may improve model performance.
Reward values like +100 might be a problem in that they could cause large gradients in a neural network whilst learning them. It is worth checking to see what happens when you normalise your reward values to just +1 for eating an apple. Termination of an episode when the snake crashes into itself, and largish discount factor, $\gamma = 0.99$ should be enough to discourage the snake from hitting itself or walls.

* In DQN they turned the game inputs into greyscale, but if you do that with your current colour choices, all the entities would have the same value. However using greyscale could work for you as well, if you had all the entities display with different colour intensities.


text:
How to develop an Entity linking system able to detect custom entities?
I am trying to build an entity linking system that links entities found in a text to entities on Wikipedia. If no Wikipedia page seems to match this entity, the systems will assume to have found a new entity and adds it to an internal knowledge base.
I am currently working to build a system that works like this:

Extract mentions of entities in the text: in the sentence "Napoleon commissioned a grand reconstruction of Paris", Napoleon is extracted as it refers to some entity.
Search for the alias "Napoleon" in Wikipedia and compute the embeddings of the pages that appear.
Compute the embedding of the sentence where the mention is found and compare it against the embeddings computed from Wikipedia. In this case, Napoleon III should be the best fit, although Napoleon I has a higher prior probability, given his popularity.
If no page from Wikipedia surpasses a given similarity threshold, we assume that we have found an entity with no corresponding page on Wikipedia; thus, we store it in our entity kb.
Save on our kb the previously computed embeddings of Wikipedia pages.

Can this approach work effectively? How can I improve it?
Any help you can provide would be greatly appreciated!

You may want to use Wikidata: https://www.wikidata.org/wiki/Wikidata:Main_Page


text:
How would one go about checking numerical stability of the deep learning algorithms?
Let's say I have a trained deep learning model. It would be good if it would be numerically stable, so if I change input by small amount, the output will also change by small amount.
How should I approach checking numerical stability of deep learning algorithm? Should I just test many different inputs and check whether the outputs for similar inputs are similar or are there any other ways to check that?

You need to define "numerical stability". It seems that you're not using the precise mathematical definition and that you're not aware of certain issues in ML.

One way to do this would be to add a small amount of noise to all of your inputs and determine the change in output.  Steadily increase the noise and measure impact on output.  Local gradients around the model's final weights are a measure of stability.


text:
How do Neural Networks deal with ambiguity of words?
The word "shot" has several meanings. For example, when one says they shot an animal, it could mean that they shot it with a gun, or even shot it out of a cannon.
How can NN trained models be used to identify the appropriate meaning of the word shot?

It depends on the actual neural network model being used but in general NNs won't be looking into individual words. They'd be fed with context (other words) surrounding the subject word. You can find some more info on most basic approach (Word2Vec) here https://en.wikipedia.org/wiki/Word2vec
Also NN models are not being trained to identify meaning. It's very complicated to explain to a NN such a complex concept as "identifying meaning". It's in general complicated to actually "explain" or "teach" anything to NN. It will regularly identify "shortcuts" that would allow NN to sand-bag on its training. It would often find strategies to trick your loss function.
It's all about how you craft your training process and your loss function. Typically a word "meaning" would appear (if I may be so bold) as a bypass product from NN doing it's job so good. I might be wrong but it can just be a mere human feeling that NN truly understand concepts. In reality it's more a numbers game, a reflection of something that might feel real yet there's nobody on the other side of the mirror.


text:
How to generate longer text with GPT-2?
I am currently using Huggingface transformers to generate text using GPT-2. The problem is, that it only generates 1024 tokens, what seems to be a hard limit in the code of the script, which enforces a maximum length of the model generation length model.config.max_position_embeddings.
As far as I understood it, one should be able to generate infinite text, with the drawback that only the last 1024 tokens are considered as context. I tried feeding a paragraph from the output into the text run, but the input prompt seems to be limited to much fewer tokens than 1024 (I am not sure how many).
So it may only be the script that's limited, but I am not sure how one would use the model to continue generation with the state the models has after generating the last token.


text:
Why does data augmentation using synthetic data generated by one model improve the performance of another model?
I understand from articles like this one that synthetic data generated by one model based on real data can improve the performance of a second model. Can anyone help me understand the intuition behind why this works? Specifically, why not directly use the real data to train the second model? Since the data generated by the first model is at most as good as the real data used for that model, won't the performance be actually better if the real data are directly used to train the second model?

The reasoning behind synthetic data is the same behind classic data augmentation,so the goal is to increase the amount of training instances to improve generalization.
The difference with classic data augmentation techniques is that classic data augmentation rely on linear transformations like flipping or rotation that do not provide as much variation as we would like. Synthetic data instead are usually generated with generative models, which sample new data from a posterior distribution learned from real data. This difference is huge. Imagine augmenting data with a flipped pictures of the same dog flipped around vs augmenting the data with a pictures of different dogs not present in your current real data.
Now the problem of course is that training generative model to produce good samples is as hard. Training a GAN on MNIST or other toy datasets is rather easy, doing so on real datasets with much less instances and all sort of imperfections (picture taken with different cameras to give an example) is many times a rabbit hole in itself. So you always want to do a good list of pros and cons before deciding to invest time in training a model just for synthetic data generation.
On the other hand, it's not impossible to find synthetic datasets ready to be used out of the shelf. In that case exploring the data to check they fit your use case is usually a no brainer and time worth spending for sure. An example is SynthCity, a dataset for point cloud segmentation, I found it very handy for a project I was working on where we didn't have lot of point clouds to start with.
Update
Why not feeding the real data used to train the generative model directly?
First answer is that you always feed the real data as well, i.e. you never train only on synthetic data, the point again is to augment you initial dataset, not replacing it.
So you move from a situation like this:
real data + naive augmentation -> model
to this:
real data + synthetic data + naive augmentation -> model
Second answer is that not always synthetic data are generated trough machine learning, there are situations in which you can use analytic methods to produce them (for example fluid simulations trough differential equations). So in this case you're not even starting from real data at all.

Thanks. Yes, I can see how giving a model new data that does exist in the original sample can improve performance, but digging deeper, my question is really why we need a separate model to do the generation of new data. Why can't we input the real data (from which the first model learns the distribution) directly into the second model? Can the second model not both learn the distribution and generalize? Or is it hard to architect such a model that does both?

I've updated the answer, hope it's more clear!


text:
Object recognition based on its movement
Models like YOLO, Detectron2, etc. classify objects based on their appearance.
This is great when we have single images, and we want to classify objects within those images. However, video contains a lot more data, and the movement of an object within a sequence of frames should also be a great way to classify objects.
For example, a bird and an aircraft may look visually similar, but their movement over a period of e.g., 2 or 5 seconds will give a much better indication as to what the object is (e.g., the bird may make a sharp turn and appear erratic, while an aircraft follows a predictable trajectory).
Are there any open source models that consider the motion of objects within a video, in order to do classification?


text:
Privacy implications of storing and transmitting GPT-3 embeddings?
We are exploring implementing a feature where a user might enter "which product has optional all wheel drive" into a search input, which would be transformed to GPT-3 embeddings, and compared against a set of embeddings for product descriptions to return similar products.
If a user enters PII and/or sensitive data, for example "my name is Jim, what will help with my COPD", then the embeddings that are generated will include this sensitive information.
Is it possible to reverse the embeddings GPT-3 creates back to the exact phrase entered?  If not, are there metrics like anonymity that can be used to reason about exactly how sensitive the embedding are?


text:
Use multiple embeddings with the Transformer architecture
In their article about their system "MuseNet" OpenAI state the following:

Embeddings
We added several different kinds of embeddings to give the model more
structural context. In addition to the standard positional embeddings,
we added a learned embedding that tracks the passage of time in a
given sample. This way, all of the notes that sound at the same time
are given the same timing embedding. We then add an embedding for each
note in a chord (this mimics relative attention, since it will be
easier for the model to learn that note 4 needs to look back at note
3, or else at note 4 of the previous chord). Finally, we add two
structural embeddings which tell the model where a given musical
sample is within the larger musical piece. One embedding divides the
larger piece into 128 parts, while the second encoding is a countdown
from 127 to 0 as the model approaches the (end) token.

I was wondering how to implement and make use of multiple different embeddings for the Transformer architecture. At the moment, the only things that come to my mind would be to sum the embeddings in a similar fashion as is done with the positional encoding, or to somehow concatenate them, where (if the embeddings are of length l each) the final embeddings would be of length 2l for each of the elements.


text:
How do I know how to design my ANN so that it fits my specific problem?
We developed a neural network-based protein reconstruction tool to reconstruct the main chain from only CA atoms.

we generated data from some selected PDBs from the RCSB website to train an NN model.
we then use those data to train the NN model.
we select some test PDBs, strip all atoms except CA atoms, and save them in files.
we pass those CA-only PDBs through the NN model, obtain a reconstructed main chain, and save them in files.
we compare original and reconstructed PDB files and calculate CRMSD values.

The data set is large: a sample of 1398438 rows and 102 columns (data points). The model is Keras via a 4-layered MLP. There is no feedback loop or convolution applied.
We obtained a CRMSD value of 0.3559, which is not satisfactory.

How can I improve or redesign the NN model?
How do I know what NN type would best serve our purpose? How do I know the number of layers we need in the model? How do I know if we need a feedback loop or not?

Evidentially, one of the most important concepts in NN design are inductive biases and symmetry. A good way to approach the concept of symmetry is to ask: Given a datapoint $x \in \mathcal{D}$ from my Dataset $\mathcal{D}$, how can I transform $x$ without changing it's meaning. Indictive biases, on the other hand, are essentially assumptions that you have about your data and that you embed into your model architecture.
Here are some examples of what I mean:
Images: Images have a translational symmetry, meaning the identity of an object doesn't change when you move it to another position in the image - a cat in the left corner of an image remains a cat even if you move it to the right corner.
If you now want to classify images and where to take an MLP instead of a ConvNet, your model had to learn to identify a cat in every possible position of the image, because the inputs to your model change with the slightest translation. In contrast, ConvNets have a translational inductive bias, i.e. are translation equivariant. This means they have a build in robustness against translations that requires no learning. This is one of the reasons, they are so good with images. This property of ConvNets arises from applying the same weights (kernel) over the space of the image. Another inductive bias of ConvNets is locality: The full image is not so important - whats important is local patterns and groups of pixels.
Text:
In AI research text was understood as sequential data. This is why people came up with RNNs like GRUs and LSTMs, thus implementing models with a sequential inductive bias. However, since transformers took over it has become clearer that the position of a word in a sentence is important but not primarily encoding the meaning. Instead a graph-like relation between words is more discriminative, which IMO, is why attention-based models are now the SOTA. How I understand this is that words have a specific permutation symmetry that depends on grammar and retains meaning if you permute words in a sentence in a specific way.
Point Clouds:
A 3D scan (point cloud) remains the same under rotation and translation in 3D, i.e. point clouds are SE(3)-symmetric. In the paper [2], this is biased into a transformer model by a clever usage of self-attention.
Graphs: The identity of a graph doesn't change under permutation as long as the relations between nodes remain the same. Hence, graph neural networks are invariant to permutation.
The list goes on like this and I put some references at the end for further reading - there is a lot of material regarding these topics and IMO it is super important. I'd recommend reading [1] which is very clearly written, [3] is very technical and I found it pretty hard to understand in detail, but it provides a nice overview of symmetries and provides an intuition of what constitutes groups and how this is related to equivariance and invariance in neural networks.

In your specific use case, you could ask: What changes can I apply to the input without changing the expected prediction and is there some sort of hierarchy or any other bias that could be useful. I'm no protein expert, but I assume a reflection symmetry (reversing the input sequence). I'd also assume that sequence length is important but not very descriptive. There might be local motives that are more important (Like image size is not important but local groups of pixels are). This would direct you to a localized model such as 1D Convolutions. Providing the model with reflection equivariance is straight forward in the case of 1D convolutions: You can, for every learned kernel, add the reversed kernel to the convolution operations.
This got pretty long by now, but I hope it is somewhat clear what I mean by all of this.

[1] Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
[2] SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
[3] A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
[4] Video: Nice talk of Max Welling on the topic of symmetry

I'll try to answer, because even this is beyond the scope of my area I think it may help.

How do I know what NN type would best serve our purpose? How do I know the number of layers we need in the model? How do I know if we need a feedback loop or not?

This generally depends on the level of experience. There is not deterministic way how to decide and find an answer for these.

How can I improve or redesign the NN model?

By playing with it I suppose.
I have not much experience in this area and zero in protein reconstruction tasks. But I think that the best way would be as in any other area of any work.

I would to simplify the task and model based on gained knowledge.
Use this network to solve couple more similar problems, prune it and compare pruned networks.
I like the idea of Generative adversarial network.
Design neural net to help to design your solution.
Or evolution algorithm should to find some answers also.

I think it is mainly about to get more knowledge about the solution.

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.


text:
Crossover and Mutation function for value encoding
I have been trying to attempt writing a Genetic Algorithm using value encoding (fixed-length vectors of real numbers) instead of binary encoding. So far the code I have written works, but needs quite a lot more population than the binary encoded code to converge to 0.
This is the code I am using:
#one-point crossover

def crossover(p1, p2, r_cross):
c1, c2 = p1.copy(), p2.copy()
# check for recombination
if rand() < r_cross:
    # select crossover point that is not on the end of the string
    pt = randint(0, len(p2))
    # perform crossover
    c1 = p1[:pt] + p2[pt:]
    c2 = p2[:pt] + p1[pt:]
return [c1, c2]

# mutation operator

def mutation(nval, r_mut):
    for i in range(len(nval)):
    # check for a mutation
    if rand() < r_mut:
        # flip the bit
        nval[i] = 1 - nval[i]

I have tried working it out following the instructions in the below link to rewrite my code, but am not sure if my crossover or mutation function works as it is supposed to:
How do mutation and crossover work with real-valued chromosomes?

With real-valued encodings, you need to adjust your crossover and mutation operators. Basically binary encodings let something like one point crossover pick a crossover point that’s "inside" a parameter. If you have two parameters p1 and p2 that are say (0.4, 2.6) in one parent and (1.2, 1.7) in the other parent, your children can only be drawn from {(0.4, 2.6), 0.4, 1.7), (1.2, 2.6), (1.2, 1.7)}. That’s just not enough ability to explore new solutions. Binary encoding those same two parameters with say 10 bits each gives you 20 possible crossover points and thus 20 possible offspring instead of just four, because it’s able to split the encoding inside a parameter. That gives you much greater power to create offspring that are different than their parents while still being similar enough that selection is meaningful.
There are specific types of operators designed to work with real-valued encodings. Look up "simulated binary encoding" or SBX for an example. Similar story for mutation. You need an operator that is able to do a better job of generating variation. A common choice is to mutate a parameter by adding a random number to it drawn from a Normal distribution with 0 mean and some standard deviation that’s customized to be appropriate for each specific problem or parameter.


text:
Replay Buffer taking long time to construct (Reinforcement Learning DQN with tf-agetns)
I'm new to Reinforcement Learning and I have some question. I am actually training some DQN using the tf-agents from tensorflow. And I recently learned that it's not possible to train a DQN using continuous actions. So the continuous actions that I had had, well I just converted that to discrete actions.
So now, it's about to set up my replay buffer (note I haven't started the actual training part yet, but just setting up the buffer. But I notice that it's taking up a long time than usual. Initially, I had a 0-1 action space and the replay buffer construction was quick. But then I realized I was missing something in my action space that I had to convert it from a discrete to a continuous action space from [0,1]. But when I realized that training DQN with continuous action space is not possible (and that the replay buffer is taking very long to construct), I reverted back to discrete space with min action value of 0 and max action value of M, for some integer M (but now it's action space is discrete and yet it is still taking time in the construction of the replay buffer).
So I am not sure what's going on here? Why is my replay buffer taking a long time to work? Does that have to do with my action space? My integer M is a parameter, but I'm testing it on a data with an M value of 4,235. So that means the action space has 4,236 possible actions. Could it be the reason that it's taking long is I have so many actions?
replay_buffer = tf_uniform_replay_buffer.TFUniformReplayBuffer(
                                data_spec=agent.collect_data_spec,
                                batch_size=tf_env.batch_size,
                                max_length=100000)


text:
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive value is a noise). if I ignore the positive value, then the predicted results of the data testing will be bad. but if I change the positive value to 0 then the prediction results will be good. what should I do if I want to keep positive value without changing it but have good prediction result.
Force Plate Shape

2050,1

Smart Insole Shape

2050,89

below are my code:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import math
from tensorflow.keras.layers import Dense,RepeatVector, LSTM, Dropout
from tensorflow.keras.layers import Flatten, Conv1D, MaxPooling1D
from tensorflow.keras.layers import Bidirectional, Dropout
from tensorflow.keras.models import Sequential
from tensorflow.keras.utils import plot_model
from tensorflow.keras.optimizers import Adam
from sklearn.model_selection import train_test_split
from keras.callbacks import ModelCheckpoint, EarlyStopping
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.preprocessing import MinMaxScaler
%matplotlib inline

## Load Data
Insole = pd.read_csv('1113_Rwalk40s1_list.txt', header=None, low_memory=False)
SIData =  np.asarray(Insole)

df = pd.read_csv('1113_Rwalk40s1.csv', low_memory=False)
columns = ['Fx']
selected_df = df[columns]
FCDatas = selected_df[:2050]
## End Load Data

## Concatenate Data
SmartInsole = np.array(SIData[:2050])
FCData = np.array(FCDatas)
# FCData = np.where(FCData>0, 0, FCData) #making positive value to 0
Dataset = np.concatenate((SmartInsole, FCData), axis=1)
## End Concatenate Data

## Normalization Data
scaler_in = MinMaxScaler(feature_range=(0, 1))
scaler_out = MinMaxScaler(feature_range=(0, 1))
data_scaled_in = scaler_in.fit_transform(Dataset[:,0:89])
data_scaled_out = scaler_out.fit_transform(Dataset[:,89:90])
## End Normalization Data

steps= 50
inp = []
out = []
for i in range(len(data_scaled_out) - (steps)):
    inp.append(data_scaled_in[i:i+steps])
    out.append(data_scaled_out[i+steps])

inp= np.asanyarray(inp)
out= np.asanyarray(out)

x_train, x_test, y_train, y_test = train_test_split(inp, out, test_size=0.25,random_state=2)

## Model Building
model = Sequential()
model.add(LSTM(64, activation='relu',  return_sequences= False, input_shape= (50,89)))
model.add(Dense(32,activation='relu'))
model.add(Dense(16,activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss = 'mse', optimizer=Adam(learning_rate=0.002), metrics=['mse'])
model.summary()
## End Model Building

## Model fit
history = model.fit(x_train,y_train, epochs=50, verbose=2, batch_size=64, validation_data=(x_test, y_test))
## End Model fit

## Model Loss Plot
import matplotlib.pyplot as plt

plt.figure(figsize=(10,6))
plt.plot(history.history['loss'], label='Train Loss')
plt.plot(history.history['val_loss'], label='Test Loss')
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epochs')
plt.legend(loc='upper right')
plt.show()
## End Model Loss Plot

## Prediction and Model Evaluation
model.evaluate(inp, out)
predictions=model.predict(inp)

print('MSE: ',mean_squared_error(out, predictions))
print('RMSE: ',math.sqrt(mean_squared_error(out, predictions)))
print('Coefficient of determination (r2 Score): ', r2_score(out, predictions))

#invert normalize
predictions = scaler_out.inverse_transform(predictions)
out = scaler_out.inverse_transform(out)

x=[]
colors=['red','green','brown','teal','gray','black','maroon','orange','purple']
colors2=['green','red','orange','black','maroon','teal','blue','gray','brown']
x = np.arange(0,2000)*40/2000
for i in range(0,1):
    plt.figure(figsize=(15,6))
    plt.plot(x,out[0:2000,i],color=colors[i])
    plt.plot(x,predictions[0:2000,i],markerfacecolor='none',color=colors2[i])
    plt.title('LSTM Regression (Training Data)')
    plt.ylabel('Force/Fx (N)')
    plt.xlabel('Time(s)')
    plt.legend(['Real value', 'Predicted Value'], loc='lower left')
    plt.savefig('Regression Result.png'[i])
    plt.show()

## End Prediction and Model Evaluation

## Model Validation
Test_Insole = pd.read_csv('1113_Rwalk40s2_list.txt', header=None, low_memory=False)
TestSIData =  np.asarray(Test_Insole)

Test_df = pd.read_csv('1113_Rwalk40s2.csv', low_memory=False)
Test_columns = ['Fx']
Test_selected_df = Test_df[Test_columns]
Test_FCDatas = Test_selected_df[:2050]

test_SmartInsole = np.array(TestSIData[:2050])
test_FCData = np.array(Test_FCDatas)
# test_FCData = np.where(test_FCData>0, 0, test_FCData) #making positive value to 0
test_Dataset = np.concatenate((test_SmartInsole, test_FCData), axis=1)

test_scaler_in = MinMaxScaler(feature_range=(0, 1))
test_scaler_out = MinMaxScaler(feature_range=(0, 1))
test_data_scaled_in = test_scaler_in.fit_transform(test_Dataset[:,0:89])
test_data_scaled_out = test_scaler_out.fit_transform(test_Dataset[:,89:90])

test_steps= 50
test_inp = []
test_out = []
for i in range(len(test_data_scaled_out) - (test_steps)):
    test_inp.append(test_data_scaled_in[i:i+test_steps])
    test_out.append(test_data_scaled_out[i+test_steps])

test_inp= np.asanyarray(test_inp)
test_out= np.asanyarray(test_out)

model.evaluate(test_inp, test_out)
test_predictions=model.predict(test_inp)

test_predictions = test_scaler_out.inverse_transform(test_predictions)
test_out = test_scaler_out.inverse_transform(test_out)

x=[]
colors=['red','green','brown','teal','gray','black','maroon','orange','purple']
colors2=['green','red','orange','black','maroon','teal','blue','gray','brown']
x = np.arange(0,2000)*40/2000
for i in range(0,1):
    plt.figure(figsize=(15,6))
    plt.plot(x,test_out[0:2000,i],color=colors[i])
    plt.plot(x,test_predictions[0:2000,i],markerfacecolor='none',color=colors2[i])
    plt.title('LSTM Regression (Testing Data)')
    plt.ylabel('Force/Fx (N)')
    plt.xlabel('Time(s)')
    plt.legend(['Real value', 'Predicted Value'], loc='lower left')
    plt.savefig('Regression Result.png'[i])
    plt.show()

## End Model validation

the Result without changing the positive value

the Result if I changing the positive value to 0


text:
Is transfer learning effective when the new task has more classes than the original?
All examples of transfer learning I have seen for classification use initial weights of a network trained on a larger number of classes (say 1000 in the case of networks trained on ImageNet data) to address a new task that has a smaller number of classes. Can transfer learning be effectively used when the new task has more classes than the original? For example, can I effectively build on ResNet50 or parts of it for a new task that has 1500 classes? Thanks.

Yes, transfer learning can usually also be utilized when the number of classes differs from the original. Your model will however be more 'transferable' if it has been trained on a wide variety of data or data that is somewhat similar to your new data.
What one would usually do is freeze the weights of the lower layers of the network and only retrain the upper part of the network (e.g. the fully connected part). The most important question is, thus, not how many classes each network was trained on but instead: How high is the representational distance between the old and the new dataset?
Recently self-supervised pretraining has increased in popularity, this shows that general features are useful somewhat independent of how they were acquired.

The emphasis on representational distance concept is apropos.  For example, it has been shown that ImageNet trained models do not perform well on medical images.

Also, how many layers in the pre-trained model you decode to freeze could depend on how well you believe the convolutional layers are able to extract the feature of your new images.  If your images are a subset of the pre-training images, for example, then you may not need to train any of the convolutional layers.   If you are adding novel images, you may also have to train some of the deeper convolutional layers.  This can be a "hyperparameter" in your training.


text:
Keep weights of output layer in transfer learning?
I'm seeing conflicting info on what to do with the fully-connected output layer of a pre-trained network when it's used in transfer learning. A previous answer seems to imply that the network is kept intact and a new output layer is added on top of the existing output layer. This approach means that the new network will build on all of the pre-trained weights of the existing network, including the weights in the output layer. However, when I look at this tutorial on PyTorch, the fully-connected output layer is replaced with a new output layer to match the number of classes in the new task. This second approach means that the pre-trained weights in the existing output layer (512*1000 parameters for ResNet18) are lost. The first approach retains everything that was learned. The second approach also looks reasonable since the weights of the final output layer are like very task-specific and so can be discarded when learning a new task. Which approach is recommended in general?

Transfer learning is a complete field of research, and there are multiple possibilities for what might work best in each situation.
There are various ways in which you can employ a pretrained model for transfer learning. You can indeed keep the complete model intact, but it is more common (as is done in pytorch) to delete the last (several) layers. In addition, one can vary with which model weights are frozen during training. You can, for example, delete the last layer and unfreeze the second to last, meaning you train 2 layers, but one is pretrained.
If you do not simply want to extend the model, it is usually advised to look into unfreezing (some of) the last pretrained layer(s), as it helps adjust the model to the new training data. Do not put the learning rate too high as it might result in the unfrozen pretrained layer forgetting what it had learned (usually termed catastrophic forgetting).
I personally have had more success with deleting the last 1 or 2 layers and unfreezing another 1 or 2 layers. You could look at some academic papers to see if there are any large-scale tests for what generally works better.


text:
computational complexity for batch normalization technique
Could you please let me know weather it is possible to have a computational complexity formula for batch normalization technique or not? If someone can help me in this regard I will be appreciated.

Batch Normalization is achieved through a normalization step that fixes the means and variances of each layer's inputs. For a Batch of size N, computing the mean and variance will have a complexity of O(N) per layer. Actual normalization can be considered an operation of O(1) with parallelism or vectorization (even without that, re-centring and re-scaling will be O(N) operations.)
Reference: Wikipedia

thank you. since I am not good enough at math could you please let me  know what does O(1) and O(N) mean? is it possible to share the answer with an example?

Those refer to Big O notation. From Wikipedia: "In computer science, big O notation is used to classify algorithms according to how their run time or space requirements grow as the input size grows." Please note that, out of convention, constant factors are disregarded in Big O notation. So, for instance, computer scientists will consider O(k*N) or O(N/2) both as O(N).