text:
How to measure classification accuracy in object detection task?
Object detection uses mAP as the metrics. But if we are only interested in classification once the object in a bounding box is extracted, what metrics should we use? Thanks!

For classification problems, using crossentropy and bce is always my go-to option. It usually performs best for these types of problems. However, using MSE or its variants is also common in some cases. Hinge loss is another one that you might want to look at if the previous options didn't work.

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

You could use top-n accuracy with a couple different n values like is normally done in simple classification tasks.
For a given class prediction, you score it as 1 if the ground truth label is in the top $n$ prediction probabilities, and 0 if it isn't in the top $n$. Then you average all these values over all your relevant predictions. (see https://stats.stackexchange.com/questions/95391/what-is-the-definition-of-top-n-accuracy for more on this)
For object detection specifically you could only count boxes towards your accuracy score that have sufficient IOU with a ground truth box so that it is only evaluating class accuracy on accurate bounding boxes.
You can also make a confusion matrix (https://en.wikipedia.org/wiki/Confusion_matrix) to see which classes are being commonly confused.
The problem with using class loss as your metric, as suggested in one of the other answers, is that it's one of the parameters you'd want to play with while aiming for a better accuracy so you wouldn't have a consistent way to score your experiments.


text:
Surrogate model to produce time series from parameter set
Say I have a model $M$ that takes in a parameter vector $\beta$, and produces a (numerical) time series. This could be a complicated model (e.g. a bespoke enzyme reaction model), or something simple like a VAR(1) (e.g. $x_{t}=\beta_0+\beta_1 x_{t-1}+\varepsilon_t$). I can run this model multiple times with the same parameter vector, and get a different time series each run.
(Even for the same parameter vector, the output may be different each run, as is the case when you simulate a VAR(1) with known variance $\sigma^2_\varepsilon$)
How can I design/implement a neural network that acts as a surrogate to my model - i.e. when given a parameter set $\beta$, it produces plausible time series sequences?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How to Train a Decoder for Pre-trained BERT Transformer-Encoder?
Context:
I am currently working on an encoder-decoder sequence to sequence model that uses a sequence of word embeddings as input and output, and then reduces the dimensionality of the word embeddings.
The word embeddings are created using pre-trained models. I want to be able to decode the word embeddings returned by the decoder of the Sequence to Sequence model back to natural language.
Question: How can I train a Decoder that works with the sequence of word embeddings and the original sentence for this task?
See below for the code that generates the word embeddings:
from typing import List

import numpy as np
import torch
from transformers.tokenization_utils_base import BatchEncoding
from transformers import BertTokenizerFast, BertModel

TOKENIZER = BertTokenizerFast.from_pretrained('bert-base-uncased')
MODEL = BertModel.from_pretrained('bert-base-uncased')

def get_word_indices(sentence: str, separator=" ") -> List:
    sent = sentence.split(sep=separator)
    return list(range(len(sent)))

def encode_sentence(sentence: str) -> BatchEncoding:
    encoded_sentence = TOKENIZER(sentence)
    return encoded_sentence

def get_hidden_states(encoded: BatchEncoding, layers: list = [-1, -2, -3, -4]) -> torch.Tensor:
    with torch.no_grad():
        output = MODEL(**encoded)
    hidden_states = output.hidden_states
    output = torch.stack([hidden_states[i] for i in layers]).sum(0).squeeze()
    return output

def get_token_ids(word_index: int, encoded: BatchEncoding):
    token_ids = np.where(np.array(encoded.word_ids()) == word_index)
    return token_ids

def embed_words(sentence: str) -> torch.Tensor:
    word_indices = get_word_indices(sentence)
    encoded_sentence = encode_sentence(sentence)
    hidden_states = get_hidden_states(encoded_sentence)
    word_embeddings = []
    for word_index in word_indices:
        # Get the ids of the word in the sentence
        # Important, because BERT sometimes splits words into subwords
        token_ids = get_token_ids(word_index, encoded_sentence)
        # Get all the hidden states for each word (or subwords belonging to one word)
        # Average the hidden states in case of subwords to retrieve word embedding
        word_embedding = hidden_states[token_ids].mean(dim=0)
        word_embeddings.append(word_embedding)
    return torch.stack(word_embeddings)
```

You can follow this example code https://github.com/microsoft/CodeBERT/blob/master/CodeBERT/code2nl/README.md
This example generally loads pre-trained Bert (encoder) and plugs a custom decoder. Regarding its encoder, the example utilizes transformer for loading pre-train.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Coherence is classifying time series data
I have been told that models that are continually running to detect a temporal pattern, say wake up words in Alexa or Echo, the model establishes this coherence or another word I heard, statistical trust, and does this via classyfying multiple times and looking at the past classifications in some manner.
I wanted to ask if this methods has got a specific name I can look up? any papers?

The following paper from Amazon Alexa Research refers to this topic as keyword spotting (KWS), and more specifically, to wake up words as wake word (WW) spotting.

Jose, C., Mishchenko, Y., Senechal, T., Shah, A., Escott, A., Vitaladevuni, S. (2020). Accurate detection of wakeword start and end using a CNN

Some other papers you may find interesting:

Luo, M., Wang, D., Wang, X., Qiao, S., & Zhou, Y. (2022). Error-Diffusion Based Speech Feature Quantization for Small-Footprint Keyword Spotting. IEEE Signal Processing Letters, 29, 1357–1361. https://doi.org/10.1109/LSP.2022.3179208
Arik, S. O., Kliegl, M., Child, R., Hestness, J., Gibiansky, A., Fougner, C., Prenger, R., Coates, A. (2017). Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting. arXiv.
Chen, G., Parada, C., & Heigold, G. (2014). Small-footprint keyword spotting using deep neural networks. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4087–4091. https://doi.org/10.1109/ICASSP.2014.6854370
Qin, H., Ma, X., Ding, Y., Li, X., Zhang, Y., Tian, Y., Ma, Z., Luo, J., & Liu, X. (2022). BiFSMN: Binary Neural Network for Keyword Spotting. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22)
Tsai, T. H., & Lin, X. H. (2021). Reduced Model Size Deep Convolutional Neural Networks for Small-Footprint Keyword Spotting. 2021 28th IEEE International Conference on Electronics, Circuits, and Systems (ICECS), 1–4. https://doi.org/10.1109/ICECS53924.2021.9665618


text:
The Small Set Expansion Hypothesis, this problem was solved or is open problem yet?
I found this problem by article called "Ten Lectures and Forty-Two Open Problems in the Mathematics of Data Science" published at 2016. I`m looking for an open problem at Data Science or/and Machine Learning field that use many Analysis Math and I'm new in this fields. Anybody can answer about? I looking for any paper the about possibility solved this problem, but I can't find any paper published definitively solved about this problem...
Thanks!

How about a reference or a link to the paper you mention? Problems can have partial solutions, and sometimes depend on exact specification

In this paper, https://people.math.ethz.ch/~abandeira/TenLecturesFortyTwoProblems.pdf, at the chapter 3, has a conjecture 3.10. This conjecture is called Small-Set Expansion Hypothesis. A saw this problem have many analysis and has strong relationship with machine learning. I interested in this problem, but this paper was published in 2016... I trying looking for any other current paper that said about solution of this problem, but I can't find and I don't know if I'm looking for correct form....

one idea is to e-mail the author, most authors are very happy to answer emails regarding their work.


text:
If an event has a statistical probability of only 50%, is it possible to use a neural network to predict it with more than 50% accuracy?
For example using a neural network to predict a coin toss. Can a trained neural network to predict it with more than 50% accuracy?

The short answer is "No." :)

Could you give more details? Do you have any information about the process of the event before it starts that could feasibly be used to make a prediction, as opposed to simply knowing that the event has been observed in the past (or is expected due to its nature) to occur with $p=0.5$? For instance, are sequential events not independent (a theoretical coin toss is independent of previous coin tosses)?

@NeilSlater Knowing more information also does not prove that the additional information is related to the probability of the event occurring, as I know the quality of the dice, the material from which it is made, etc. I put this information into the neural network and it doesn't increase the accuracy either.

If you're evaluating your neural network on the same set of data used to train the model, you could definitely get higher than 50% "accuracy". But, the accuracy would go down to 50% once evaluated on unseen data if the event is truly random (i.e. truly unrelated to any features fed into the neural network).

@clementzach True randomness may be a false proposition, since we can't find features that accurately predict it, it's true randomness, and once we find features that accurately predict it, it's no longer true randomness.

@JoeHuang Yes, I agree. The extent to which an event is "truly random" to an observer doesn't have anything to do with the type of event itself, but it does have to do with the other information an observer has.

When does the NN predicts the outcome? 1 ms before the coin stops moving? When the coin is in the air? In my hand? in my pocket? These are different levels of uncertainty.

Using the result of a coin flip as a feature to predict the result the person calling it will say, leads to an event with about 50% probability that you can predict with about 100% probability. It's a trivial example, but it should demonstrate the point.

This is a question of marginal vs. conditional distribution
The marginal distribution of the coin may be a Bernoulli random variable with 50% probability for either outcome.
However, the conditional distribution of the outcome given information about other factors (e.g. the angle, throw height, ... see other answers) may look entirely different. Provided these features determine the outcome in some way, a neural network can absolutely predict the outcome with more than 50% accuracy.
A neural network could not exceed 50% accuracy, if

The information determining the throw outcome is not available
The function is of a nature that can not be learnt by the neural network
The coin toss is truly random

A coin toss is often used as a casual example of a "truly random" event, so in this sense the answer to your question is "No". In reality however, it is very hard to find any truly random events (at least outside quantum mechanics), which is why random number generation is a big challange and neural networks can predict a lot of things.

What nature of function  can prevent neural networks from learning?

@JoeHuang For example, a neural network with only linear layers can only predict linear functions. (Nobody uses this type of neural network). Or the neural network could be too small for the complexity of the prediction.

@JoeHuang, another example is the sorts of functions you'd find in modern cryptography. Although they can support some non-linearities, a neural network still relies on at least local smoothness for gradient descent and generalization. Hash or encryption functions, by design, do not have that. (In principle a sufficiently large neural network could encode such a function, just by memorising the mapping, but it wouldn't do any better than a lookup table.)

This is the best answer, hands down. The question cannot be answered with "Yes" or "No" without knowing the nature of randomness.
I find it misleading to even begin or end any answer with such a definitive summarization, even if it is clarified in the details (which not everybody reads carefully after seeing a simple, bold answer).

What's the logical nature of your bulleted list? Do all these conditions need to be satisfied, or any of them?

Good point - any of the criteria listed will suffice to prevent the network from learning.

No.
If there are no patterns, relations or correlations in your data, AI can do nothing to improve what essentially is just guessing.
My last 5 tosses were Heads, Tails, Tails, Heads, Tails. Can you predict the next toss outcome? How would you explain your guess? If you give AI this same data, it cannot do better than just guessing.
The question changes if you have data that is related to the outcome of the coin toss, such as the direction and force the coin was tossed before it lands. In this case, it isn't "an event with a statistical probability of 50%" anymore. If you measured everything perfectly, you could have 99.9% accuracy on what the outcome of the coin toss would be.
AI can only produce accurate results if a super smart human could theoretically also produce accurate results.

I do agree with the first part (no patterns = guessing), but I don't think the second part is strictly speaking correct. You don't need more types of data (like force or direction), you just need any patterns at all. While I can't say much from your dataset of five throws (neither can AI), if you gave me results of many more of your throw, it might show that you have a slight tendency for heads. Maybe that's the side you intuitively put the coin on your hand and you don't throw with many rotations. Maybe the coin is not perfectly balanced.

I'm not saying that the probability of a right prediction will be 90% in such case, but it still might be some number above like 50,01% (which is > 50%)

YES
If you obtain information about the force and angle of the throwers thumb striking the coin at release, that would give insight into how many times the coin would be expected to rotate. Combine this with what faces up when the coin releases, and you should be able to do better than 50/50.
I don’t have a firm source (perhaps there is something on Skeptics), but it seems that people have trained themselves to flip coins to reliably land on one of the sides, so there are some features that dictate how the coin rotates.
Really, this is kind of the point of regression. You think some process has a 50/50 chance of the two outcomes, but once you know a bit more (features), you can sharpen that estimate. Formalizing this mathematically involves the conditional vs marginal distribution discussed in the answer by Scriddie.

this is cherry picking an example. To be more precise, I think OP's example relates more to e.g. a Random Variable $X \sim \mbox{Bernoulli}(0.5)$, in which case a NN cannot predict $x \sim X$ with accuracy (significantly) different to 50%.

@DavidIreland Maybe, but the whole point of doing a regression model like a neural network is to take a process that seems unpredictable (e.g., flipping a coin) and use related information (e.g., force of the flip) to reliably make better predictions.

This does not predict statistical probability. AI may highlight patterns that imply deviation from pure luck, e.g. a roulette wheel being imbalanced or a person being more likely to spin full circles with the coin, but this is not of statistical significance, rather priorly unknown factors that cause deviations from expected probability.

@user3819867 That’s what regression is: using outside factors to predict something better than you could without that outside factor. If you’re unclear about this, you might consider asking for clarification on a statistics website like Cross Validated Stack Exchange.

It seems like your answer to "can a neural network predict a 50/50 outcome with more than 50/50 accuracy?" is "yes, as long as the outcome isn't 50/50"...?

@Daevin The whole point of using features in a regression is to expose the fact that what appears to be 50/50 is not 50/50 once those features are considered, so there is a sense in which your comment is correct, yes. // This is really a statistics issue, and if it is confusing, I recommend asking about it on the Cross Validated Stack Exchange I linked earlier. It might be worth linking that question in the comments here.

@Dave As you say Dave, this is an issue of statistics. You are assuming the existence of the extra information that you can condition on knowing. Assume this extra information is denoted by $Z$, then you're actually proposing to learn approximate the distribution of $X | Z$, which is a totally different random variable to simply $X$. So, again, referring to my original comment, it depends on what exactly OP is asking. (If this is confusing, perhaps you can ask on the CVSE that you linked to earlier).

You need to ask yourself, what is the limiting factor in the accuracy for whatever you are trying to predict.

If the limiting factor is in the quality of the algorithm being
used to calculate the prediction, then perhaps you could find a
better algorithm that would improve the accuracy.
If the limiting factor is in the very nature of the problem itself,
such as a coin flip, then there is no method of calculation that
could improve the accuracy.

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Although the question is a little vague, I'll treat it as a statement about the mapping of inputs and outputs in the underlying random process - no matter what conditions/inputs/features we observe, there is not a consistent mapping from input to output. A statistical probability of 50% suggests in two cases with identical inputs, we may find different outputs. A traditional deterministic neural network cannot do this, as it is really just a mathematical function, which by definition maps every possible input to exactly one output - it is not possible to use the same inputs and get different outputs. Because of this, a deterministic neural network can't achieve more than 50% accuracy in the long run in this case. No matter what set of features is input, there are in reality two possible outcomes, but the neural network can only return one outcome for any particular input. On average, the neural network will return the correct output only half the time - it can't achieve more than 50% accuracy.

If a neural network was only able to be as reliable as random guessing, they wouldn't be much use!
Let's suppose there's an election on, and the result is finely balanced between the yellow party and the purple party. At a top level, 50% of people will vote for each colour. If you know nothing else about the people, "Who will the next person in the polling station vote for?" is intrinsically an even guess.
It would still be possible to use a neural network (or a decision tree, or a human!) to predict at much better than 50% if you have additional input. For example, if you look at them and can read their apparent wealth, race, gender presentation, or whether they come accompanied or alone, it may then be possible to identify membership of a sub-population which is more likely to vote yellow.
The fundamental limit on performance isn't the baseline 50:50 probability. Instead it is the component which is caused by some inputs (which may but doesn't have to be true randomness) that are simply not available to the network. Suppose for example that there's a 5% chance that someone has been bribed to flip their vote and the network can't know that. In this case it won't get to more than 95% reliable predictions, but can still do much better than 50% with demographic data.


text:
Pseudo Label Generation for Generative Cooperative Learning
I am trying to implement this paper for unsupervised video anomaly detection.
The gist of the paper seems to be:

Create a dataset for an unsupervised setting, by mixing up the train and anomalous videos (section 4)
Divide each video into $p(=16)$ segments each, so for video $i$ we have the segments $s_{ij}$, $j= 1,2...p$.
Using a Feature Extractor (ResNext) ,compute a '$d$' dimensional feature vector $f_{ij}$ for each $s_{ij}$. (Section 3.1)
Use this to (pre)train the Generator (a simple Autoencoder). Once G is (pre)trained, it is used to generate pseudo labels. The pseudo labels from G are then used to (pre)train the Discriminator (D). (section 3.3)
Pretrained G and D are then put into a collaborative loop, where pseudo labels from D are used to improve G and likewise.

I am having trouble with generating the pseudo labels using G. The paper clearly mentions in section 3.3: "Once G is pretrained, it is used to generate pseudo labels".
So does the pseudo label generation come into play once training is done? Like after the model is ready, we pass the dataset again and keep note of the reconstruction losses, and generate labels accordingly?
Or are they generated during training?
(Section 3.2.2 mentions the pseudo label generation and section 4 mentions the threshold selection).


text:
What does AUSE metric mean in uncertainty estimation
I am reading the paper "Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision", I do not understand the definition of AUSE metric in this sentence "but only in terms of the AUSE metric which is a relative measure of the uncertainty estimation quality." There is no explanation for this term in the paper and googling it didn't bring me anything useful either. Can anyone tell me what does AUSE stand for? Thank you so much!

The paper you read refers to the "Uncertainty Estimates and Multi-Hypotheses
Networks for Optical Flow" paper for the AUSE metric. AUSE stands for Area Under the Sparsification Error. There are 2 core concepts here c.f. Sparsification Plot and Sparsification Error (defined in Section 5.2)
Concept 1: Sparsification Plot
Here, we plot our "metric-of-interest" on the y-axis and "percentage-of-removed-samples" on the x-axis. The idea is to remove samples (hence the term sparsification) in descending order of uncertainty and to evaluate your "metric-of-interest" at each removal. Then, your models sparsification plot is compared to an "Oracles" sparsification plot. The "Oracles" plot removes samples in descending order of their error on the basis of your models predictions. The closer your models curve is to the "Oracles" curve, the better your model is in context of uncertainty-error correspondence (or uncertainty calibration).
Concept 2: Sparsification Error
But, how do you compare your models performance with other models since they all have different errors and hence different sparsification plots (also for their respective Oracles). The authors propose to simply calculate the difference between a model and its Oracle's sparsification plot (hence the term error) and calculate the area-under-the-curve (AUC) for that. Unlike most cases where high AUC is better, here a lower AUC is considered better.
One can stop reading here if they just need an answer to the OP's question. Rest of the content is just some of my thoughts on this.

My thoughts: Global Implication
For future readers of this post, the authors of the paper referenced by the OP refer to the AUSE metric as "relative" and instead propose their own "absolute" metric called "AUCE" - Area under the Calibration Curve. They argue that AUSE can be gamed if the models uncertainty is consistently underestimated. My understanding of this is that if your model assigns similar(or same?) uncertainty to each sample, then the removal process to compute the sparsification plot is just like an Oracle. And then the Sparisifcation Error is epsilon(~0). In spite of this theoretical edge case, I have seen other papers using the same concept as a sparsification plot though termed as a "Risk-Coverage" curve instead [e.g. Automatic segmentation with detection of local segmentation failures in cardiac MRI ]
My thoughts: Uncertainty calibration vs Uncertainty-Error Correspondence
As a side note, most folks use the term uncertainty-calibration, though in my opinion it should be termed as "uncertainty-error correspondence".
Sstrictly speaking, it is counter to the definition of calibration for a predictive model. Historically, calibration is spoken of in context of probabilities of a particular event (The well-calibrated Bayesian, Journal of the American Statistical Association,1982). Since uncertainty (e.g. entropy, variance etc) values are not directly indicative of probabilities, even though they are derived from them, I believe it is semantically incorrect to say uncertainty calibration.


text:
Hyperparameter tuning methods for neural networks
I have a fully connected feedforward classifier neural network that uses the leaky ReLU activation function. I would like to apply a state-of-the-art hyperparameter tuning method to my methodology. Currently, the fitness function associated with each parameter setting is a weighted combination of (i) accuracy of both training and test data sets ($acc_{tr}$ and $acc_{ts}$, respectively), (ii) numbers of activated hidden layers ($nL$) and their neurons ($nN_l$), and (iii) number of epochs ($nE$) - length of training. The fitness function is computed as follows:
\begin{equation}\label{eqq}
 FF = 0.7 \times (acc_{tr} + acc_{ts}) + 0.1 \times \frac{nL}{10} + 0.1 \times \frac{\sum_{l \in \mathcal{L}} nN_l}{1000} + 0.1 \times \frac{nE}{2000}
\end{equation}
Note that I have determined the weights in function $FF$ based on my experimental observations. Potential values for the parameters of this neural network are given in the following table:

$\alpha$ here is the leakage parameter of the ReLU activation function. Considering that I am new to this field, my questions are as follows:

Is the weighted fitness function an appropriate one? Should I include both training and test accuracies at the same time? Why?
Do the potential values make sense for a fully connected feedforward classifier neural network with the leaky ReLU activation function? Are there any other parameters that I have to tune? Please consider that I have developed this neural network using the Keras library.
The above table also shows the elite configuration. I have found this parameter setting using the iRace package. Do you recommend any other method for hyperparameter tuning in neural networks?
I have seen many papers use genetic algorithms to tune their parameters. What are the advantages of using a genetic algorithm in comparison to the iRace package (if there is any)?

Please kindly answer my above questions with supporting scientific references. Also, please let me know if there are any other considerations for the hyperparameter tuning of neural networks. Thank you!

There is a Python package called Tune is fairly easy to use, and contains most modern hyperparameter tuning methods such as Bayesian optimization and Hyperband. It should interface with both Keras and PyTorch.
To answer some of your questions:

If you are doing intensive hyperparameter tuning, you need to have three separate datasets: the training set, validation set, and test set. When you train the network, you train on the training set, and evaluate the accuracy metric on the validation set. The validation accuracy is then used to tune the hyperparameters. After you find the best hyperparameters, you then evaluate again on the test set to get the true test accuracy. The reasoning for this is because the act of hyperparameter tuning is itself an optimization procedure, and will suffer from overfitting as well. This is best explained in one of Andrew Ng's video. A more robust way is to do k-fold cross-validation, which I think is the default method of Tune.

Yes I think your hyperparameter makes sense. Is $\alpha$ the learning rate? You can also try to include batchsize from some set, say $\{2, 4, 8, 16, ...\}$, given your GPU VRAM.

I am not familiar with iRace enough to say it's advantages/disadtanges compared to genetic algorithms. But I know two of the major drawbacks of genetic algorithm is that it doesn't perform well when the number of hyperparameters (or genetic elements) are large, and it has a tedency to converge to local minima.


text:
How can I train GPT-3 (or similar) to search for answers my text file
My company has large sets of standards that are quite dry to read through and find answers to questions.  How can I have an AI perform the search?
The text files are too long to submit with every query, so it seems like I would need to train the model with the data.  What's the simplest solution out there?

there is a variant of BERT which is specifically trained to answer questions given text data. more specifically, it was trained to return $start$ and $end$ indexes that should in theory mark the answer in the text that corresponds to the question. Might have to fine-tune it a bit by giving it your specific data, I would start of with few-shot (before transfer learning for example)


text:
How to use strong labels in image classification?
I have a dataset where I have the labels cancer & non-cancer, and I also have localized pixel-level annotation masks of important regions/features in the images.
In a binary classification task, how do I optimize my deep learning model to focus on the annotated regions of the image and ignore or pay less attention to the regions outside the annotations?
Additionally, I have a set of images with only image level labels. Is there a way to use both localized labels (strong labels) and image level labels (weak labels) in the same model?

This is a great question.  Sorry I only have 1 reputation point, so I can't upvote.  The optimal solution likely has to be empirically determined.  I don't think it can be determined a priori.  It would also depend on how you define as optimal (high sensitivity, high specificity, high AUC, or high accuracy).
One approach would be to develop a segmentation model and any image with "tumor pixels" would be classified at the image level as "tumor".  I don't know for sure (and no one really knows for sure), but I think this method would yield higher sensitivity (but possibly lower specificity) than a straight image classifier.  Which one is likely to be better depends on a lot of things, including how much training data you have, the quality of annotations, and the quality of images.
To answer your second question, yes there is.  There are probably several ways to integrate the different levels of information.  One simple way is to develop a model with weak labels and a model with strong labels and then do a probability fusion, similar to how it is done for multimodal data (see article).


text:
How to use RL on a robotic moving arm?
I'm working on a simulation of a motor that is attached to a wing (Later, this will also have a real-life counterpart once I'll assemble all the components in our lab), and I can control the forces/torques that the motor applies. I want to use RL and find an optimal action in terms of

"what force should the motor apply to maximize lift".

To make things clearer, check out the following figure

So for example, I can find $\phi$ at every single time step $t$ (using some 1st-year physics equations and python integrators) and this will be my state $s$.
Are you familiar with approaches that deal with this problem?

Hey, check this answers. Also, take a look at these: one, two. Did they answer your question?

The core part of RL is the reward function. It belongs to the environment and it is the only way the agent can explore the world given a state.
By constructing an appropriate reward function we can make an agent do exactly what we want.
Here is an example of a double-jointed arm that can move to target locations in the Unity environment.
A reward of +0.1 is provided for each step that the agent's hand is in the goal location. So, the goal of the agent is to maintain its position at the target location for as many time steps as possible.
Initially, the agent performs random actions (random forces on the joints) and eventually learns to follow the target. A high-level idea is described in this video. And here is also related DeepMind's article and paper.


text:
Shaping reward so that it maximizes multiple components together
I am fairly new to RL and I compete in AWS DeepRacer student league. The main task there is to create a reward function. All the hyperparameters and action space are fixed. So far, I know how to shape a single component like speed or steering. I always do like steer_reward^2 or speed^2. But when I try to combine the components together, I am not seeing good results. Is it best to mutliply or add components and are there any resource where I find various maximization functions?

Hey, check these answers: one, two. Did they answer your question?

@ArayKarjauv Nope. I am trying find out functions that have a good global minimum or maximum where the variables of the functions are the speed or steering or other params.


text:
During batch normalization is the mini-batch gone through twice, one to calculate the mean and variance and then to normalize them?
I am asking this question because while designing my own model, I had repeated gradient explosion issues, so I wanted to try batch normalization. I really want to understand the details and math surrounding how it is implemented. Specifically wouldn't batch normalization effectively change the output of each layer as the activations are now normalized? Also, any details on alpha and beta parameters would be greatly appreciated! Thank you!

I think it'd be helpful to refer to the batchnorm formula given in the PyTorch implementation.
In particular, given an input $x$, you would get the mean and variance ($\mathbf{E}[x]$ and $\text{Var}[x]$), and normalize it as such:
$$ \hat{x} = \frac{x - \mathbf{E}[x]}{\sqrt{\text{Var}[x] + \epsilon}} $$
where $\epsilon$ is a small number to improve stability. There is also an additional affine transform applied in most implementations of batchnorm in standard libraries, so the output is really $\gamma \hat{x} + \beta$, where $\gamma$ and $\beta$ are the affine parameters (scale and shift respectively).
To answer your question (if I understood correctly), yes batchnorm would change the output of the previous layer, due to the normalization and affine transformation.
There are some details to consider:

In most implementations, the $\mathbf{E}[x]$ and $\text{Var}[x]$ are running mean estimates, meaning that there would be additional momentum hyperparameter to control how fast the estimates are updated as new inputs come in.
For batchnorm, you would normalize and apply the affine transformations to each output channel separately, meaning that each output channel would hold its own set of running mean estimates and affine parameters.

Thanks for the reply, just have a quick question. When you say moving mean do you mean in the mini-batch or for the final inference stage? Specifically are these moving means and variances used only during the inference stage? I am asking this because it seems like a lot of work to go through the training sample all over again with normalized outputs for each node, rather than just having a moving mean and average from the start of the mini-batch.

@liyuzerihun The moving means are only updated during training, and updated per mini-batch when fed forward through the network. For inference, the standard practice is to "freeze" them (meaning that they do not get updated).


text:
variational inference but with a weighted loglikelihood
I  would like to know if it's  correct if I substitute in the ELBO formula

a weighted sum of the loglikelihood
$$\sum E_{q_{\theta}(w)}[w_i \ln{p(y_i|f^{w}(x_i))}]$$
in place of the traditional sum.
My problem is that my dataset comes with  the errors on the target variable, which means that i have access to the weights and i would like to give more value to the measurements with a lower error rather than considering all of them equals.
The addition of the weights would mean that the formula for the posterior probabilitychanges in
$$  p(w,D)= \frac{ p(D,w)p(w)}{p(D)} = \frac{ (\prod p(y_i|f^{w}(x_i))^{w_i})p(w)}{P(D)} $$
does doing something like this correct in the bayesian framework?


text:
Deep Learning model to predict ELO chess rating from games?
I was wondering if anyone had already tried building a regression model that could predict the ELO rating of a chess player based on his last N games? Inputs could be anything, from chess notation of the plays to Stockfish's analysis of the moves. (Excellent/Innacuracy/blunder/...).
Basically, anything but the actual way to calculate ELO, which requires (if I understood correctly), to get the wins/loss of other players and their respective ELO. And I think it's impossible for a player to get to ELO 2500 (for example), in only 10 or 20 games.
I might have misunderstood some intricacy of the ELO system which makes my question unfeasible, forgive me if so.
Thank you.

Elo is relative accuracy and changes in Elo are zero-sum after a rating has stabilized. By the way, Elo is not an acronym but the name of the person who invented it. If you are interested in modeling this, I would be interested in collaborating! Lichess publishes all their game data freely. Regarding the input of Stockfish analysis, we would need to run it for every game (it's only their on Lichess if a given user has chosen to analyze it with a computer after the game ended).

It was a kaggle competition - many of the participants wrote up their results. https://www.kaggle.com/competitions/finding-elo


text:
How to combine pretrained language models with additional feature set?
Are there any techniques to combine a feature set (other than the text itself) with pretrained language models.
Let's say I have a random NLP task that tries to predict a binary class label based on e.g. Twitter data. One could easily utilize a pretrained language model such as BERT/GPT-3 etc. to fine-tune it on the text of the tweets. However the tweets come with a lot of useful metadata such as likes/retweets etc. or if I want to add additional syntactic features such as POS-Tags, dependency relation or any other generated feature. Is it possible to use additional features I extracted for the finetuning step of the pretrained language model? Or is the only way of doing so to use an ensemble classifier and basically write a classifier for each of the extracted features and combine all of their predictions with the finetuned LMs predictions?

Although barely related, this brings to my mind a concept I first learned in conditional GANs - there, if you wanted an output of some specific class, you would take your random noise input $z$ (that is given to your generative model $G$) and concatenate it with some representation of the class instance. On the MNIST digits dataset, for example, you can provide $[z,(1,0,...,0)]$ where $(1,0,...,0)$ represents a one-hot encoding of the digit $0$. So perhaps you could use this concatenation concept for your task:

extract features $\{a_i\}_{i=1}^N$ from the text itself - this can be done using a transformer, as suggested, or some RNN architecture like LSTM or GRU.
extract features $\{b_i\}_{i=1}^N$ from the metadata - there are some approaches that deal with tabular data, but I would start with a basic MLP
concatenate the two feature vectors $c_i=[a_i,b_i]$
yield a final prediction using yet another model that takes $\{c_i\}_{i=1}^N$ as input

Edit: I've looked this up online, and found some supporting evidence - check out this blog post, for example.

If you're eventually building a classifier with added features, concatenating the LM output embeddings with those additional features should work, I believe.
It seems similar to incorporating non-sequential data, like location, in RNN models, where the final set of features for classification are last hidden layer output + non-sequential features.


text:
Which positional encoding BERT use?
It is a little bit confusing that someone is explaining that BERT is using sinusoidal functions for BERT position encoding and someone is saying BERT just uses absolute position.
I checked that Vaswani 2017 et al., used a sinusoidal function for position embedding, but not sure BERT used the same technique since the BERT paper does not mention it much.
My question is

Did BERT use absolute position embedding that is learnable, not with sinusoidal function?
If so, is the absolute position the same dimension as the input embedding dimension for an addition?

BERT uses trained position embeddings. The original paper does not say it explicitly, the term position embeddings (as opposed to encoding) suggests it is trained. When you look at BERT layers in HuggingFace Transformers, you will the dimension of the trained positions embeddings (768×512), which is also the reason why BERT cannot accept input longer than 512 tokens.
from transformers import AutoModel
model = AutoModel.from_pretrained("bert-base-cased")
print(model.embeddings)

BertEmbeddings(
  (word_embeddings): Embedding(28996, 768, padding_idx=0)
  (position_embeddings): Embedding(512, 768)
  (token_type_embeddings): Embedding(2, 768)
  (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
  (dropout): Dropout(p=0.1, inplace=False)
)


text:
Why Phasic Policy Gradient (PPG) can update value function in auxiliary phase?
My questions is that how could we train the value network (separated from shared network) by using data from previous policies, which varies a lot since we collect data from different policies with many training phases done to start a auxiliary phase, especially returns are calculated by those policies. Wouldn't it hurt the stability of fitting value function?

PPG can update the value function in the auxiliary phase because it uses a technique called "reparameterization" which allows for the gradient of the value function to be computed more efficiently. Additionally, PPG can take advantage of the structure of the problem to provide a better estimate of the value function.
I think you can train the value network on data from any policy, but
it will have more variance. If you have access to more data, it could help to smooth out the value function.

One approach would be to update the value network after each training phase of the policy network. This would ensure that the value network is always fitted to the current policy. However, this might not be practical if the training phases are very long.

Another approach would be to fit the value network using a rolling window of data from the previous training phases. This would strike a balance between stability and accuracy.

"Wouldn't it hurt the stability of fitting value function?"$\rightarrow$
$\color{red}{\textbf{No}}$, it should not hurt the stability of fitting the value function. In fact, using data from different policies can actually help to improve the stability of the value function estimate. This is because of "reparameterization" which allows for the gradient of the value function to be computed more efficiently as discussed above.

Aren't we estimating the value function w.r.t $\pi$? If we fit the value function with returns coming from different policies, we are moving towards different $V^{\pi}$?


text:
Can action space be part of the state space?
I'm working on a project where I have access to position coordinates and velocity components of multiple agents in an environment. Assuming that one agent is controllable while others are not controllable, the velocity of the controllable agent represents the action in the defined MDP. can I consider the velocity of the controllable agent as part of the state space as well? i.e. for time step t the state includes the "current" velocity of the agent, and the action represents the velocity at the "next time step". If this is not possible, then what are the consequences of doing so?
I was looking for references that may give directions to answer the question, but couldn't find something useful, unfortunately.

There's no reason I can think about that would prevent you from doing this. A question though, what environment are you trying to model ? If you use velocity as an action, your agent will move in an unusual manner, being able to go very fast or to stop instantly, is that what you want ? Otherwise using acceleration as an action might be more desirable.

@nathanraynal Thanks Nathan for your comment! I'm trying to model self driving cars environment. I don't have access to "instantaneous" acceleration information. But, can I control the behavior of the agent via reward reshaping so as to avoid unusual behavior?

You could define your set of available actions as a function of your state, if your state contains the velocity of the car. A simple choice would be for state $S_t = (..., ..., ..., v_t)$.

A simple choice would be for state $S_t = (..., ..., ..., v_t)$ to choose something like the set of action to be  $A_t = f(S_t) = [v_t - \alpha; v_t + \alpha]$, with $\alpha$ a small value. Maybe you can also shape your reward as to discourage velocity too far from the current one too.


text:
What is the difference between a diffusion model and GANs?
Recently, I hear a lot of people claiming that diffusion models beat GANs, also providing less training time. I've searched about these two type of models, and I am confused, because somehow they both use gaussian loss to blur and then train to reconstruct the image. They also seem to use autoencoders in the process. Can someone please explain and elaborate in more detail about these two types of models?

Google :).
See, example, this, https://www.youtube.com/watch?v=fbLgFrlTnGU&t=3s and https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-introduction/ and ...


text:
Is data augmentation beneficial even if the dataset is large/diverse enough?
I'm training a deep learning model to map binary images to grayscale values of the same shape. For the dataset, I can genearate one as large and diverse as I want it to be.
My question is: let's say the original dataset I created contains 100k images. I can either generate another 900k unique images (so that my training set is 1M in total), or to use data augemntation on the ones I already have and somehow (flipping, rotating, etc.) generate another 900k (I know that in 2D, there's probably only 8 different types of unique images that can be generated by flipping/rotating or a combination, but that's beside the point here).
Which one would you go for and why? Thank you so much!

You are going to generate the images by flipping, rotating, etc. which will happen anyways in augmentation. Augmentation can happen on the fly so you don't waste memory storing those new images, thus, you can train your network fast. You can use RandAugment or AutoAugment for augmentation.

Let me clarify: I can (a) generate "unique" new images, or (b) generate "augmented" new images. Which one is preferred?

Before taking this decision, taking the embeddings of your current dataset through Resnet50 or EffNetB0, then, doing PCA for plotting in 2D, would give us an idea about the distribution of data. If you find it wide enough, then, augment, if not, generate more and perhaps improve the method of generation if you don't find enough diversity.

Thanks! That's great advice, I'll give it a try. Just one question: since I'm not doing image classification, do you think the embeddings of ResNet50 (or similar classification NNs) would be accurate enough? Thanks once again!

Yes, they are useful since they tell you about the general properties of images. Out of ResNet and EffNetB0, go for EffNet though (it has proven to better across many-many domains). It beats the usual method of flattening the images and then performing PCA.


text:
How do you train a model with low success rate?
I'm training a model where the samples success rate is low.
I mean how do I tackle such situation - maybe only show the samples which match but then the ones that doesn't may match too.
But on the other hand if I train it with all the samples it seems like it never fully matches when it should.
Any ideas?

I think you need to specify your question: What do you mean by 'samples success rate', 'samples that match' and what is 'it' referring to in 'seems like it never fully matches when it should'?.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

Are you referring to something related to class imbalance?

You have a situation where class imbalance is quite high. There are two ways to tackle this:

Use Focal loss which focuses on underrepresented classes.
Train a Siamese network and allied techniques to it.


text:
Reinforcement Learning with PPO - entropy loss dropping, but so is performance. Why?
I'm using PPO with an action-mask and I'm encountering a weird phenomenon.
At first during training, the entropy loss is decreasing (I interpret this as less exploration, more exploitation, more "certainty" about policy) and my mean reward per episode increases. This is all exactly what I would expect.
Then, at a certain point, the entropy loss continues to decrease HOWEVER now the performance starts consistently decreasing as well. I've set up my code to decrease the learning rate when this happens (I've read that adaptively annealing the learning rate can help PPO), but the problem persists.
I do not understand why this would happen on a conceptual level, nor on a practical one. Any ideas, insights and advice would be greatly appreciated!
I run my model for ~75K training steps before checking its entropy and performance.
Here are all the parameters of my model:

Learning rate: 0.005, set to decrease by 1/2 every time performance drops during a check

Gamma: 0.975

Batch Size: 2048

Rollout Buffer Size: 4 parallel environments x 16,834 n_steps = ~65,500

n_epochs: 2

Network size: Both networks (actor and critic) are 352 x 352

Is this an environment where the agent can stumble upon the "right solution" by trying out things until something works? Conceptually, it could be that when the policy is still stochastic enough (high entropy), the actions that "solve" the environment are chosen after enough tries. But during your training entropy decreases quickly enough that the policy is close to deterministic (low entropy) before the agent has had a chance to reliably learn the right actions, instead converging on the "wrong" ones and choosing them consistently.

@mikkola Yes, the agent can definitely get rewards through random actions. I understand your overall point, but I guess I don't understand why the agent would converge on the wrong ones, rather than the right ones, given that both are available since the reward is initially high?

Does the agent respond to the environment change as a policy change? It happened to me that due to insufficient training of the model using the SGD optimizer, the entropy of behavior decreased, that is, the probability of some actions increased significantly in relation to others, but this probability did not depend on the environment in any way. The problem was solved by choosing the Adam optimizer

PPO is an algorithm in a class of actor-critic methods. In this class of methods, the training is broken in two counteracting parts: the "actor" parts learns the policy and the "critic" part learns the value (advantage) function for that policy. Such approach suffers from a fundamental instability inherent to all adversarial optimization methods - if one part of the adversarial loop gets way too ahead of another part, then the whole training loop stops.
That looks like what is happening in your description - your "actor" bit got too certain about the policy so the "critic" bit is unable to progress to better learn the value (advantage) function of the environment.

authors: Андрей Петров


text:
Is it overfitting?
hi i'm new in this field. I am trying to do a video classification project by using 3DCNN and I plotted the loss curves & accuracy curves. I have some questions.

i'm using kfold Cross validation. Should i save the parameters after every fold and choose the best one?
how to reduce the loss?
is it overfitting?

one of the fold

another fold

There's a few questions here so I'll address them in turn:

What to do with the results of a K-Fold Cross Validation? Generally, a Cross Validation (or any other form of data partitioning) is used as part of a hyper-parameter seach. Your final model would then be the trained using the best hyper-parameters found during the search, and all of the available training data. This final model should then be tested on a held-out set of testing data, which was not using in any stage of training,  to ensure no leakage of information.

How to reduce the Loss? This is a tough question and there is no one direct answer that works in every situation. However improving the performance of a model can be crudely summaraised as: better data, better model, and more regularisation.
Aquiring better data in many contexts is often "more data", but there are analyses that suggest data cleaning and rebalancing can cause signficant improvements, but it's hard to measure these improvements. These must be carried out with care to avoid leaking information between the training/testing sets. It is also almost always the hardest thing to do!
Better models can be achieved by changing your architecture or ensembling multiple models. You've mentioned you're using 3D CNNs on video input, so possible better model options while keeping the same overall achitecture are changing the number of layers and experimenting with different shapes of convolutional kernel.
Finally, more regularisation usually helps avoid overfitting and improves your model's ability to generalise to unseen data. This can be achieved by applying layers such as Dropout, Batch Normalisation, or applying a regularisation constraint to your model's weights.

Is it overfitting? Probably, but that's not a death knell for your modelling approach per se. Your graphs show that for these K-fold splits, the performance of the model on the validation set stabilises out as worse than the performance on the training set. If we can assume that the distribution of targets in these validation sets is the same as in the training set, then the model has measurably learned patterns that are unique to the training sets. Thus, overfitting.
However, it's worth checking that this assumption of equal distributions is true - and if not I'd recommend using Stratified K-Fold Cross Validation to ensure that it is true for future experiments to make a fair comparison.

If you re-run with difference architectures, regularisation and stratification (if needed) then the generalised performance of the model may improve. There's a lot of fun experimentation to try from here, and I wish you a healthy dose luck and patience! :)


text:
Left-to-Right vs Encoder-decoder Models
Xu et al. (2022) distinguishes between popular pre-training methods for language modeling: (see Section 2.1 PRETRAINING METHODS)

Left-to-Right:

Auto-regressive, Left-to-right models, predict the probability of a
token given the previous tokens.

Encoder-Decoder:

An encoder-decoder model first uses an encoder to encode an input
sequence, and then uses a left-to-right LM to decode an output
sequence conditioned on the input sequence.

My question is, what are the differences between those two methods?
Do they suggest that the first method is a decoder-only? If so, what is the input to this decoder?
Based on what I know about auto-regressive models and the above definition, I understand that in Left-to-Right, we predict the $i$-th token given the $1,...,i-1$ tokens (which could be our past predictions).


text:
Are neural networks a strict special case of a transformer?
Since transformers contain a neural network, are they a strict generalisation of standard feedforward neural networks? In what ways can transformers be interpreted as a generalisation and abstraction of these?

https://deepai.org/machine-learning-glossary-and-terms/transformer-neural-network

Neural network is a generic term used in literature as a sort of umbrella for all types of architectures, architecture being a set of distinct forward operations and hyper parameters (such as number of layers/nodes, kernels size).
Feedforward neural networks, multi layer perceptrons, convolutional neural networks, recurrent neural networks, autoencoders, transformers (and many more) are all types of neural networks (deep neural networks to be precise, the 'deep' is usually assumed). Also edge cases like generative adversarial networks (which is more of a training approach than a strict architecture) are usually referred to as neural network, which allegedly might be confusing.
So "since transformers contain a neural network" is not really a correct way of putting it. Also in case you meant "since transformers contain a feed forward neural network" it would still be incorrect cause transformers use operations such as convolutions which are not used in feed forward neural networks, so they are still very distinct type of architectures.

Transformers can be used for a variety of tasks that standard feedforward neural networks $\color{gray}{\textbf{cannot}}$, such as Natural Language Processing and Time Series Forecasting & they are $\color{gray}{\textbf{more efficient}}$ than standard feedforward neural networks, as they can share parameters across multiple tasks.
$\color{maroon}{\textbf{No}}$, Transformers are not a Strict generalization of standard feedforward neural networks. Since there is no Strict definition of what a "standard feedforward neural network" is, but they Can be seen as a Generalization of a Feed-Forward Neural Network in Several ways.

Transformers can be interpreted as a generalization of recurrent neural networks (RNNs). In contrast to RNNs, which propagate hidden states through time, Transformers propagate hidden states through a self-attention mechanism. This allows them to model long-range dependencies without the need for RNNs, which are difficult to train.

Transformers can also be interpreted as a generalization of convolutional neural networks (CNNs). In contrast to CNNs, which extract features from local patches of an input, Transformers extract features from the entire input sequence. This allows them to model global dependencies without the need for CNNs, which are limited by their local receptive fields.


text:
What is the difference between Restricted Boltzmann Machine and Artificial Neural Network?
In the deep learning course I took at the university, the professor touched upon the subject of the Restricted Boltzmann Machine. What I understand from this subject is that this system works completely like Artificial Neural Networks. During the lecture, I asked the Professor the difference between these two systems, but I still did not fully understand. In general, there is an input layer and a hidden layer in both, and the weights are updated with forward-backward propagation. Can someone explain the exact difference between them?

A Boltzmann Machine is a probabilistic graphical model which follows Boltzmann distribution:
$$p(v,h) = \frac{e^{-E(v,h)}}{\sum_{v,h} e^{-E(v,h)}}$$
where $E(v,h)$ is known as the energy function.
An RBM is a Boltzmann machine with a restriction that there are no connections between any two visible nodes or any two hidden nodes, which gives it the structure similar to a 2-layer Artificial Neural Network.
The difference is that RBM, being an unsupervised model, is trained by minimizing the energy function. While an artificial neural network can have many hidden layers along with an output layer, and is trained by optimizing the loss between the values of output layer and the values of target variable.
A special case of RBM which has binary visible nodes and binary hidden nodes, also known as Bernoulli RBM has an API available in scikit-learn. They have also documented the learning algorithm here. In this example, they show how Bernoulli RBM can be used to perform effective non-linear feature extraction which can be fed to the LogisticRegression classifier for digit classification.

You can find in this paper that RBM is a specific type of artificial neural networks. Hence, the term Artificial Neural Network is more general than RBF.

It is an important property that single as well as stacked RBMs can be reinterpreted as deterministic feed-forward neural networks. Than they are used as functions from the domain of the observations to the expectations of the latent variables in the top layer. Such a function maps the observations to learnt features, which can, for example, serve as input to a supervised learning system. Further, the neural network corresponding to a trained RBM or DBN can be augmented by an output layer, where units in the new added output layer represent labels corresponding to observations. Then the model corresponds to a standard neural network for classification or regression that can be further strained by standard supervised learning algorithms. It has been argued that this initialization (or unsupervised pretraining) of the feed-forward neural network weights based on a generative model helps to overcome problems observed when training multi-layer neural networks.

Of course, we have other types of neural networks like Recurrent Neural Networks expressing them as an RBF is not straightforward.


text:
What loss function will be correlated with classification metrics?
Recently I developed a custom training algorithm for deep learning models, based on evolutionary algorithms. Details are not important, except that it also uses decreasing regular cross entropy loss as its fitness function.
What I observed is that it very well decreases the loss function but the classification metrics such as accuracy, precision or recall also decrease along the training. This got me confused, as I was sure that decreasing loss such as cross entropy should always increase these metrics. After researching I found out this is possible due to fact that cross entropy can decrease in case where confidence on few samples is greatly increasing, but many other samples are meanwhile getting incorrect scores, but the profit from these few correct ones are dominant over many incorrect ones: https://www.quora.com/What-is-the-matter-when-loss-decreases-and-accuracy-decreases-too-on-training-neural-network?top_ans=238980470
So my question is: is there a loss function that when decreasing will always be increasing classification metrics?

Does not answer the question (which I think is actually ill-posed, for reasons I hint at at the end of this answer of mine at SO), but may be helpful: https://stackoverflow.com/questions/47817424/loss-accuracy-are-these-reasonable-learning-curves/47819022#47819022

Different metrics measure different quantities, so there is no reason to expect different metrics to move together unless one is a function of the other (such as MSE and RMSE).
Further, metrics like accuracy use thresholds that can prove problematic, versus evaluating the probability outputs directly. This is discussed extensively on the statistics Stack, Cross Validated. I will leave a few links.
https://stats.stackexchange.com/a/312787/247274
https://stats.stackexchange.com/a/359936/247274
https://stats.stackexchange.com/a/312124/247274
However, even strictly proper scoring rules do not have to move together. Since, for instance, Brier score and log loss (both of which are examples of the strictly proper scoring rules discussed in the first link) are measures of different quantities, this is desirable. Depending on what we value, we might prefer one over the other.


text:
Distributions over outputs for randomly initialized neural networks
Does anyone have any pointers to resources about the properties of randomly initialized neural networks (with no training)? I'm guessing this might depend on the network architecture and initialization scheme, but I'm most interested in properties that seem to be mostly true across architectures and initializations.
In particular, I'm wondering what the distributions over outputs look like. Intuitively, one would expect most output distributions to be random, but I'm sure there are also "adversarial examples" that could be crafted which leads the NN to output highly-confident predictions.

In your questions, you specify untrained networks. So, the distribution will remain random. But, one work which sounds like this one is - Feedback alignment (https://arxiv.org/abs/1609.01596).

Thanks for the pointer! However, this doesn't quite answer my question (what kind of randomness): say you have all weights initialized as random normals(0,1), and RELUs, and your data distribution is uniform over a box. What will be the distribution of outputs f(x) where x ~ data-distribution and f(•) is the neural network? Wondering if this kind of analysis (formal or empirical) has been done anywhere


text:
Why do iterative deepening search start from the root each iteration in the context of the minmax-algorithm?
Consider the graph below for an understanding on how IDS work.
Now my question is:
why do IDS start at the root every iteration, why not start at the previously searched depth in the context of minmax?
What is the intuition behind it?

Normally in minimax (or any form of depth-first search really), we do not store nodes in memory for the parts we have already searched. The tree is only implicit, it's not stored anywhere explicitly. We typically implement these algorithms in a recursive memory. As soon as we've finished searching a certain of the tree, none of the data for that subtree is retained in memory.
If you wanted to be able to continue the search from where you left off, you'd have to change this and actually store everything you've searched explicitly in memory. This can very quickly cause us to actually run out of memory and crash.
Intuitively, at first it certainly makes sense what you suggest in terms of computation time, i.e. it would avoid re-doing work we've already done (if it were practical in terms of memory usage). However, if you analyse exactly how much you would save, it turns out not to be much at all. Due to the exponential growth of the size of the search tree as depth increases, it is usually the case that the computation effort for the next level ($d + 1$) is much bigger than the computation effort already done for all previous depth levels ($1, 2, 3, \dots, d$) put together. So, while in theory we're wasting some time re-doing work we've already done, in practice it actually rarely hurts.
In the specific context of minimax with alpha-beta pruning, we get an additional benefit when re-doing the work. We get to make use of the estimated scores from our previous iteration to re-order the branches at the root, and with alpha-beta prunings this can actually make our search more efficient!

I could be mistaken since I don't know the source of the image you have provided, but that image appears to be showing how the tree is built, not how it is searched.  Even so, when a balanced tree of the sort you have illustrated is searched, it will start with the root node, though in a search the maximum number of nodes traversed will be minimized and all operations (like min or max) will be performed in $O(h)$ where $h$ is the height of the tree.


text:
Can neural network be used to predict deltas between numbers?
I have a list of increasing numbers with no duplicates for example : 3,6,11 and so on. Difference or deltas between these numbers such as in above case : 3, 3, 5 are frequent and with greatest difference being 9. Can a neural network be used to predict next 3 deltas for next 3 numbers. Which type of neural network will be useful and what kind of limitations will there be ?

Note : List comprises of millions of numbers.
These numbers and deltas are not random.

Yes, predicting deltas is often easier than predicting values directly. The most obvious way to perform this task would be to use a simple MLP with MSE loss. If the deltas are discrete, and there are only 9 possible values, a multi-label cross entropy loss would probably perform better than using MSE.
Depending on what the relationships between the numbers are, you might want to feed the last W numbers to the neural network, so that it can learn to use context. To do that, just make your input dimension W, and have your output dimension be 1. If you need a very long context window for good prediction, you should consider using an RNN.
One thing that would help to improve training is that the scale of the inputs shouldn't be too different (e.g. some inputs being ~10^5 and some being ~10^0). If you can rescale your inputs to always be close to 0, that's helpful. For example, if the rule which relates numbers to one another is +3, +5, +3, +5, ... you could always normalize the first input to be 0, and un-normalize after you get your neural network output, without any loss of information.


text:
What kind of neural network and GPU should I use to classify images into > 10 000 classes?
I am trying to developp an image classifier that would have more than 10 000 classes but I don't know what kind of neural network I should use ?
Some Other questions arise from this one :

How big should be my training dataset ? (10,100,1000 or even more image/classe)
What kinf of GPU should be used to train/run such a network ?

Thanks in advance !

You could look for papers that trained models on the Open Image Dataset, which contains around 6k classes, so pretty close to your final use case.
Regarding the dataset size, most datasets include at least 1k examples per class, this is a rule of thumb more than a mathematical restriction. Of course the more the better, like is always the case in machine learning.
The GPU question is an ill posed one. All framework by now support more or less any GPU, nvidia being the choice of preference when it comes to driver compatibility, especially in cloud platforms like aws or google gloud. The specific type is really up to you. If you have a limited amount of time to allocate for training then definitely use as many GPUs as you can afford. If time is not a constrain any GPU will do the work just fine.

For good performance, you want a GPU with enough memory to fit the entire model in its memory, and I guess ideally double or triple that - not sure what amount of overhead is needed for training. If you're training a model from scratch you get to choose how big it is.

You're actually right. I'm used to google cloud and aws where on average every GPU has a lest 12 or 16 GB of memory, more than enough for any model (with exceptions like GPT and similar, which won't fit in any card anyway). But indeed a 3GB card or other small gaming card would probably be limiting, good call!


text:
How to use Categorical Cross Entropy for Multi-Label Classification?
Say my target with classes A, B, C, D, E is [0, 1, 1, 0, 0].
And my output layer is of B x N where N is the number of classes.
How do I use Categorical Cross Entropy for this?

You want to treat the problem as binary classification, and of course use binary cross-entropy.
The only requirement is to ensure to use a sigmoid activation in your model's last layer.


text:
How do I use the N correctly in NEATs speciation delta function?
When implementing NEAT I'm having some issues with the speciation distance/delta function, specifically the term N (number of genes in biggest genome).
Won't term $N$ in $δ=c1*E/N+c2*D/N+c3*W$ just out-scale $E$ and $D$, and reduce their impact on the function as the genomes grow?
The paper(3.3) recommends limiting N = 1 when genomes are small, but what happens when they grow? The impact of E and D tapers off.
For example two genomes with no similarities (silly example, but just for demonstration):
#1: 67 connections
#2: 40 connections
No similar innovation numbers, so $W$ will be 0.
$E$ = (difference in genes count) 67-40 = 27
$D$ = (total count of all genes found in one but not in other, minus excess genes) 67+40-27 = 80
$N$ = (Size of biggest genome) = 67
$c1$ = $c2$ = 1.0
$δ=1.0*27/67+1.0*80/67+0 = 1.5970$
This is pretty low compared to the default limit of $3$ that is used in the NEAT paper(4.1) examples, and by some other examples online.
Am I computing this incorrectly?


text:
Extract a document page from a photo
I am trying to extract a document as an image from another image. Let's say that we take a photo of a document on a surface. My ultimate goal is to be able to digitize this document but as an image, not as OCR. This is already done in phones like my Samsung, but I want to do this using my Linux machine and as a command line tool if possible.
As an example see the attached image (created by the default application of my mobile phone). The system recognized the orthogonal area and proposed its bounding box with yellow color. After that of course it is easy to extract this area and make it straight as a regular orthogonal image. What I am interested is how to "reconstruct" this yellow box from scratch.
Any ideas how to do this? I tried to search various methods and tools, but when I search something there's always information on how to convert this through OCR, which is something I am not interested. I even found some imagemagick techniques that promise to do this, but unfortunately failed :(

I think what you are searching for is something like this, it is sufficient to extract the lines in the image and choose the lines that are most connected as your object

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.


text:
How do Transformer decoders handle arbitrary length input?
I am working through a Tensorflow Neural Machine Translation tutorial (https://www.tensorflow.org/text/tutorials/transformer) and am confused about how the decoder handles inputs when making inferences after it has been trained.
In the section where we create a class for translating a sentence (https://www.tensorflow.org/text/tutorials/transformer#run_inference) it appears that we feed the decoder an array populated with the START token only, then append the last element from the predicted sequence it makes to the growing translated sentence. This does not make sense to me as we trained the transformer on a fixed length sequence that was padded to length MAX_TOKENS=128, so I can't figure out why it would be able to accept input of an arbitrary length tensor.
Here is the code for inference in question that confuses me:
    # As the output language is English, initialize the output with the
    # English `[START]` token.
    start_end = self.tokenizers.en.tokenize([''])[0]
    start = start_end[0][tf.newaxis]
    end = start_end[1][tf.newaxis]

    # `tf.TensorArray` is required here (instead of a Python list), so that the
    # dynamic-loop can be traced by `tf.function`.
    output_array = tf.TensorArray(dtype=tf.int64, size=0, dynamic_size=True)
    output_array = output_array.write(0, start)

    for i in tf.range(max_length):
      output = tf.transpose(output_array.stack())
      predictions, _ = self.transformer([encoder_input, output], training=False)

      # Select the last token from the `seq_len` dimension.
      predictions = predictions[:, -1:, :]  # Shape `(batch_size, 1, vocab_size)`.

      predicted_id = tf.argmax(predictions, axis=-1)

      # Concatenate the `predicted_id` to the output which is given to the
      # decoder as its input.
      output_array = output_array.write(i+1, predicted_id[0])

      if predicted_id == end:
        break

self.transformer takes in the "encoder_input" which is fed to the encoder and "output" which is fed into the decoder, where we then take the last prediction in the sequence returned from the decoder, append it to "output", then repeat the process. But I don't understand how the decoder can process a tensor of length 1, then 2, then 3, and so on if it is always expecting a tensor of length 128.
It seems like the input to the decoder should be padded to 128 tokens and then the prediction at the i-th index should be appended to the output (rather than always the last element in the predicted sequence) before repeating.
Is the tutorial mistaken in the implementation on of how this transformer preforms inference? Or am I missing something that Tensorflow does behind the scenes that allow this to work?
Note, I realize the title to this question is similar to How can Transformers handle arbitrary length input? but the answer given appears to verify that the input to the decoder should be fixed length and padded, which contradicts how inference is done in the tutorial I am referencing. So I hope this question isn't considered a duplicate but I am asking a similar question in the context of the mentioned tutorial.
Thank you for any insight someone might be able to provide!

At training time, the decoder uses a so-called triangular mask that prevents the self-attention to "look" at positions that are yet to be decoded. Because of that, nothing prevents the decoder to use shorter sequences than those it was trained with. The padding would be masked out anyway.
A more interesting question is whether the decoder can work with longer sequences than it was trained for. The answer is: it depends. In the original Attention is all you need paper, analytically defined position encoding was used that should in theory generalize beyond the lengths it was trained for. This is also used in the tutorial the question refers to. However, in practice, learned position embeddings are used (e.g., BERT uses a fixed length of 512 tokens) that seem to deliver better results in practice. In such a case, the decoder simply cannot generalize beyond the maximum length, simply because it does not have embeddings for those positions.


text:
Why is the greedy heuristic admissible and consistent for food at corners problem, but not for food anywhere problem?
UCBerkley has a great Intro to AI course (CS188) where you can practice coding up search algorithms. One of the exercises (question 6), asks to generate a heuristic that will have Pacman find all 4 corners of the grid.
My implementation used a greedy approach that found the closest corner by Manhattan distance, moved Pacman to that corner, and then repeated the process. This is a relaxation of the original problem by removing all the walls. Here is my code below:
def cornersHeuristic(state: Any, problem: CornersProblem):

    corners = list(problem.corners) # These are the corner coordinates

    position, corners_status = state

    bools = [x==0 for x in corners_status]
    unexplored_corners = []

    for index, bool_val in enumerate((bools)):
        if bool_val == True:
            unexplored_corners.append(corners[index])

    current_sum = 0
    while unexplored_corners != []:

        dists_temp = []

        for corner in unexplored_corners:

            d = util.manhattanDistance(position, corner)
            dists_temp.append(d)

        min_dist = min(dists_temp)
        current_sum = current_sum + min_dist
        position = unexplored_corners.pop(dists_temp.index(min_dist))

    return current_sum

This implementation ended up working correctly and the autograder recognized it as consistent.
However, upon closer inspection, I think the autograder made a mistake. Here is an example of where my greedy heuristic would find a path longer than the theoretical shortest path available for Pacman:
The blue line is the path my heuristic would take and the green line is the correct shortest path.

What am I missing here? Why is this heuristic consistent? And why is this greedy heuristic consistent for the corners problem, but not for the "food anywhere" problem? Any visualization showing a simple scenario where it would fail for the "food anywhere" problem would be greatly appreciated since Im a visual learner.

Someone asked a very similar question on Stackoverflow just the other day: https://stackoverflow.com/questions/73680177

I disagree. My question is asking why this greedy heuristic is consistent and I provided a figure showing how its not a lower bound on the closest manhattan distance


text:
Is there a best practice for creating multiple convolutional layers from small image inputs?
With all the work being done on larger and larger images, I'd like to ask if a best practice(s) has arisen for allowing multiple convolutional layers on small image inputs?
For instance, in my case I have a 7x7x4 input that I convolve (coming from a small visual piece of a RL environment). Obviously if I want to allow a few layers with increasing depth/channels to allow the network to learn abstractions, many kernel/padding combinations will swiftly reduce the first two dimensions to a point where arguably convolutional layers don't make sense.
For my own case, I've managed to get 3 seemingly reasonable layers by playing with kernel size, stride, and padding, and while it "works" for my use-case, it feels outright arbitrary:
# Values for the first conv layer
self.ksize_1 = 3
self.stride_1 = 1
self.padding_1 = 1
# retains the input_dimz x input_dimz shape

# Values for the second conv layer
self.ksize_2 = 4
self.stride_2 = 1
self.padding_2 = 1
# roughly makes dimension even (looks like input_dimz - 1)

# Values for the third conv layer
self.ksize_3 = 3
self.stride_3 = 1
self.padding_3 = 0
# outputs dimensions roughly 2 less in width by height from above
# should be (input_dimz - 3) x (input_dimz - 3)

# Now our actual conv layers
self.conv1 = nn.Conv2d(self.input_chan, self.conv_dim, self.ksize_1,
                       self.stride_1, self.padding_1, padding_mode=self.padding_mode)
self.conv2 = nn.Conv2d(self.conv_dim, self.conv_dim * 2, self.ksize_2,
                       self.stride_2, self.padding_2, padding_mode=self.padding_mode)
self.conv3 = nn.Conv2d(self.conv_dim * 2, self.conv_dim * 3, self.ksize_3,
                       self.stride_3, self.padding_3, padding_mode=self.padding_mode)

(where conv_dim is essentially a hyperparam to tweak the channel depth)
Are small dimensional cases like this still pretty arbitrary and the above reasonable, or has a methodological approach arisen that can be followed?

Hi @Josh, are you saying that you have 4 "channel dimensions"?  Is it RGB + alpha?  Also, are your images really only 7x7-pixels?

@SnehalPatel yep 4 channel dimensions and indeed 7x7. In my case it's visual-like info coming from a piece of a reinforcement learning environment

A bit late to the party but the only thing I could come up with is VGG-oriented architectures. What I mean by that is use blocks of convolutional layers that leave the input the same by for instance using stride one and same padding. Finish the previous block with a convolutional layer that has stride 2 or valid padding and then create a new block that utilizes a zero padding layer.
For tensorflow you would do something like this:
   inputs = keras.Input(shape=input_shape)
   x = layers.Conv2D(filters=num_filters, kernel_size=kern_size, padding="same", strides=1,activation='relu')(inputs)
   x = layers.Conv2D(filters=num_filters, kernel_size=kern_size, padding="same", strides=1,activation='relu')(x)
   ...
   x = layers.Conv2D(filters=num_filters, kernel_size=kern_size, padding="same", strides=2,activation='relu')(x)
   # new block
   x = layers.ZeroPadding2D(padding=((padding_num)(x)
   ...

This way you can have several layers that leave the input size untouched until you want to go further. Ofcourse instead of changing the stride, you could add a MaxPooling layer etc.
My answer is based on subjective empirical evidence with small images and with trying to expand networks without reducing the size of the initial input to too small dimensions.

Upvote as this suggestion is helpful. I don't think it should be "the answer" as it seems either the answer is "no, there is no best practice" or at least a consensus approach hasn't arisen. That said, I used pretty much this approach ultimately!


text:
How does Seq2Seq with attention actually use the attention (i.e. the context vector)?
For neural machine translation, there's this model "Seq2Seq with attention", also known as the "Bahdanau architecture" (a good image can be found on this page), where instead of Seq2Seq's encoder LSTM passing a single hidden vector $\vec h[T]$ to the decoder LSTM, the encoder makes all of its hidden vectors $\vec h[1] \dots \vec h[T]$ available and the decoder computes weights $\alpha_i[t]$ with each iteration -- by comparing the decoder's previous hidden state $\vec s[t-1]$ to each encoder hidden state $\vec h[i]$ -- to decide which of those hidden vectors are the most valuable. These are then added together to get a single "context vector" $\vec c[t] = \alpha_1[t]\,\vec h[1] + \alpha_2[t]\,\vec h[2]+\dots +\alpha_T[t]\,\vec h[T]$, which supposedly functions as Seq2Seq's single hidden vector.
But the latter can't be the case. Seq2Seq originally passed that vector to the decoder as initialisation for its hidden state. Evidently, you can only initialise it once. So then, how is $\vec c[t]$ used by the decoder? None of the sources I have read (see e.g. the original paper linked above, or this article, or this paper, or this otherwise excellent reader) disclose what happens. At most, they hide this mechanism behind "a function $f$" which is never explained. I must be overlooking something super obvious here, apparently.

Hello, can you check this post I just created and lmk which part of it you would like to further break down to get a clear answer? https://datascience.stackexchange.com/a/115015/102852

@hH1sG0n3 Like all of the other sources I linked to, you calculate the attention weights and the context vector, but you never show how to use the context vector, only how to get it.

Evidently you can only initialize it ($\vec{c_t}$) once

As I see it, $\vec{c_t}$ depends on $\vec{h}[1] \ldots \vec{h}[n]$ AND $\vec{s_{t-1}}$ (because $\alpha_i[t]$ depend on $\vec{s_{t-1}}$, and so is different for every calculation of a new $\vec{s_t}$).
And so $\vec{c_t}$ is different for every $\vec{s_t}$.
It can be used by the decoder by e.g., concatenating it with $\vec{s_{t-1}}$, or with $\vec{y_{t-1}}$, or by adding instead of concatenating, or ...

Your clarification of it is incorrect. it in that sentence stands for $\vec s$, the decoder hidden state. See the preceding sentence: (...) the decoder as initialisation for its hidden state. Evidently, you can only initialise it once. I also indicated that $\alpha_i[t]$ is time-dependent, both with the $[t]$ parameter and by stating (...) the decoder computes weights $\alpha_i[t]$ with each iteration -- by comparing the decoder's previous hidden state $\vec s[t−1]$ (...).
Do you have a source for your last paragraph? It's the usage of $\vec c[t]$ I'm looking for (hence the title).

Right, I misinterpreted what you meant by 'it'.
It seems to me you can only initialize $\vec{s_0}$ once, sure, but for the calculation of each subsequent $\vec{s_t}$, a different context vector $\vec{c_t}$ is used, combining the (fixed) encoder-hidden-states $\vec{h}[1] \ldots \vec{h}[n] $ with different $\alpha$ weights.

For the last paragraph, this is just to illustrate how it could be done. However, it seems that the appendix in the Bahdanau papers shows the details of how $\vec{c_t}$  is used:  https://arxiv.org/pdf/1409.0473.pdf#page=12, apparently roughly by adding (after projection) it to (projected versions of) $\vec{s_{i-1}}$ and $\vec{y_{i-1}}$ .

If our similarity function is defined as
$$e^{t,t'}= f(y^t,h^{t'})$$
for a $(t, t')$ pair, this would give you an attention weight/score for each hidden state $h^{t'}$. In this way you end up with $[0, ...,  t-1]$ weighted hidden states, which you combine using a sum. This weighted sum is the "context vector"
$$ c_t = \sum_{i=0}^{T}a_{t,i}h_i$$
To answer your question, for each position you end up with a context vector $c_t$ which fully replaces the hidden state $h^{t}$ at the exact same position of your computation graph.
$$ h_t \rightarrow c_t,\ as\ \ LSTM_{simple}\ \rightarrow\ LSTM_{attentive} $$
Hope it helps!
References:
- Neural Machine Translation by Jointly Learning to Align and Translate, 2014 \

This seems unlikely to me. What is this $h_t$ at the end of your post? It isn't an encoder state, so you must mean a decoder state $s_t$. Yet, Bahdanau on page 3 writes that $s_i = f(s_{i−1}, y_{i−1}, c_i)$, which suggests to me that $s_{t-1} \neq c_{t-1}$, otherwise Bahdanau would have written $s_i = f(c_{i−1}, y_{i−1}, c_i)$.

Let me take a step back. In simple LSTM, can you define the difference between $h_t$ and $y_t$, so that i understand your pov better.

It doesn't matter how I define these things. That equation is a literal quote from the Bahdanau paper linked in my question. It also doesn't matter whether it's with or without attention, because the Seq2Seq decoder LSTM feeds its own output $y_t$ back to its input regardless. $h_t$ are the encoder's hidden states. $s_t$ are the decoder's hidden states.


text:
What does the adversarial loss in a GAN represent?
I'm working on Pix2Pix an image-to-image translation GAN, and I noticed that there is an adversarial loss implemented using BCE, and a L1 loss implemented using MAE. I know L1 loss represents the difference between the predicted image and actual image, but I am not sure what does the GAN adversarial loss represent?
This is the official definition
The adversarial loss influences whether the generator model can output images that are plausible in the target domain
but the meaning is tough to understand. Is it representing the difference between the predicted probability distribution and actual probability distribution?

The adversarial loss in a GAN represents the amount of information that the generator is able to trick the discriminator into believing is true.
"The adversarial loss influences whether the generator model can output images that are plausible in the target domain"$\rightarrow$ means that the generator model should output images that look like they could plausibly belong to the target domain. This is important because if the generated images look too fake, it will be difficult for the discriminator to learn to correctly classify them. It's the data that determines whether and how it improves.
In addition, the adversarial loss is based on the generator model's experience of its successes and failures in the data.

"if the generated images look too fake, it will be difficult for the discriminator to learn to correctly classify them" -> in that case wouldn't it be easy to classify them as fake?

The adversarial loss in a GAN represents the difference between the predicted probability distribution (produced by the discriminator) and the actual probability distribution (i.e., the distribution of real samples in the target domain). It is used to train the discriminator to correctly classify generated images as fake. If the generated images look too fake, it will be easy for the discriminator to classify them as fake, resulting in a high adversarial loss. This loss is used to improve the discriminator's ability to distinguish real from fake images.


text:
Difference between sequence length and hidden size in LSTM
It does not come clear to me how the seq_length is not the exact same as the hidden_size in LSTMs.
For example, in the next image, the seq_length is T since there are $x_{1}, x_{2}, ..., x_{T}$ time steps and there are is one LSTM cell (green LSTM blocks) per input $x_{i}$. So in this image seq_length == hidden_size

In addition, every LSTM cell requires 3 inputs:

$x_{t}$ the input value at time t.
$h_{t-1}$ the hidden state value from the previous time step.
$c_{t-1}$ the cell state value from the previous time step.

This can be seen in the following image:

How can the hidden_size be lower or higher than the seq_length? In any of both cases the LSTM cells would not have the $x_{i}$ input which is required.

You are correct: you get one hidden state per time-step. The hidden size, however, refers to the number of features that are used to compute each hidden state, and are therefore independent of your sequence length.


text:
How code analysis part of ChatGPT works and trained
ChatGPT can explain given code snippet we also ask question like "What does this variable do" , "Why this is used" and all. I gave C++ function snippet from an popular Open Source project it was able explain complete context how this function can be used and it is used in project even though snippet I gave only contained that particular function. This has baffled me. I understand that they might have fed all Open Source project codebases to it.
But what I wonder how might they have trained it. Like given the code how to make Neural networks do analysis on it and make it answer the query? I'm really curious to learn it.
I know OpenAI did not really publish proper research paper on this. But still anyone can tell me or guide me how might they have done it.