text:
How does Chat GPT encode a question?
Chat GPT is based on a decoder-only Transformer so it does not have an encoder. Given that, how is a user's question passed as input to Chat GPT's decoder? In a regular encoder-decoder architecture, the final embeddings of the encoder are passed to the decoder along with the  token. Then, the decoder auto-regressively outputs tokens. How would this work in a decoder-only architecture?
Let's say I have the following question: "How many countries are there in the world?" and its token form is [3, 5, 8, 2, 10, 4, 1, 6, 7]. How will the decoder take in the input?
In an encoder-decoder architecture, I would just pass the  token to the decoder which would, based on the encoder embedding, auto-regressively output the next token in the sequence to form an answer.
In the decoder-only architecture, how will that work? If I pass the first token of the question (token "3") to the decoder, it will output what it thinks is the most likely token after "3" but it will have no context... So how is the context taken into account given we are not encoding it?
I am aware of this post which is similar but I would more specifically want to know how we go from a decoder-only model that inputs the most likely word in general, to a model that outputs a whole answer based on a specific question. Put differently, how does a problem which is inherently a sequence to sequence problem (question answering) get solved using an autoregressive model instead of a seq2seq model?
One potential answer I have been looking into is training fine-tuning but I am still unsure how a model that out puts a single token x[n+1] = f(x[n],...,x[1]) can be fine-tuned based on a training set of two sequences (one question and one answer per record).

Do you have a reference for "Chat GPT is based on a decoder-only Transformer"?

only from forums: https://community.openai.com/t/is-gpt-group-of-models-decoder-only-model/286586/2, https://datascience.stackexchange.com/questions/118260/chatgpts-architecture-decoder-only-or-encoder-decoder#:~:text=Here%20is%20one%20of%20its,has%20an%20encoder%2Ddecoder%20architecture.

In the case of ChatGPT's decoder-only architecture, the input question is typically passed as a prompt or initial context to guide the generation of responses. Rather than explicitly encoding the question and passing it to the decoder, the model uses the conditioning prompt to infer and generate relevant responses.
When you provide the question prompt, such as "How many countries are there in the world?" or its corresponding token representation [3, 5, 8, 2, 10, 4, 1, 6, 7], ChatGPT uses this information as the starting point. The model then processes the prompt and generates a response token by token, considering the context provided.

thanks. so let's say f() is the decoder-only model and the answer predicted is "there are 195" ([20, 2, 13]) . Then, we would have: 1/ f([3, 5, 8, 2, 10, 4, 1, 6, 7]) = 20; 2/ f([5, 8, 2, 10, 4, 1, 6, 7, 20]) = 2;  f([3, 5, 8, 2, 10, 4, 1, 6, 7, 20, 2]) = 13? I guess that would work but it's hard for me to understand how the model could infer enough context from the prompt to output the correct response given it is trained to output the most likely next token, in general, not in the context of a specific question.

@John In neural networks, information about the context of a question is usually conveyed through internal states.  These states are updated and adapted at each generation step.
In the context of sequential response generation, the model selects the next token at each step based on the statistical relationships learned during the training process.  It estimates the probability of possible tokens based on previous tokens and the current context of the question.

thanks, so those representations are learned during model fine-tuning? Also, is the example I gave above correct? Would a training set for such fine tuning look like the following? [[[3, 5, 8, 2, 10, 4, 1, 6, 7], [20]], [[5, 8, 2, 10, 4, 1, 6, 7, 20], [2]], [[8, 2, 10, 4, 1, 6, 7, 20, 2], [13]]]

@John    Yes. The example is correct.


text:
Approaching construction of model that interprets financial reports
I want to train a model to be able to interpret financial reports (from a company). Basically, I want to be able to extract relevant information without needing to read through hundreds of pages of text. I know I will need to use a database that can store vectors for the embeddings of the texts. However, I have some questions:

Assuming the texts are hundreds of pages long and, therefore, they go over a lot of stuff, what vectorization technique can be used?
I assume I will have to obtain a homogeneous training set. How do I do this without reading all the texts if I still don't have the final model (classification??)?
Let's say I have the final model. How do I "query" the things I want to know? In other words, would I have to input some text and then the model has to be able to interpret that as a query?

Note: I apologize in advance if im not being too descriptive, I currently don't have so much information about the details of the data and other things, I only have a general sketch about the idea of the project.

I think you should split this post into multiple ones because you're asking several questions.

Given the lengthy nature of financial reports, you can consider using techniques like document embedding or topic modeling. Document embeddings, such as Doc2Vec or Universal Sentence Encoder, can represent the semantic meaning of a document in a fixed-length vector representation. Topic modeling algorithms like Latent Dirichlet Allocation (LDA) or Non-negative Matrix Factorization (NMF) can help identify key topics or themes within the documents.
Achieving a homogeneous training set without reading all the texts can be approached through a combination of techniques. One way is to use supervised or semi-supervised learning, where you label a subset of the data and use it to train the initial model. You can then iteratively refine the model's performance by actively selecting representative texts for annotation based on the model's uncertainty or other criteria.
Once you have the final model, you can query it by inputting specific text or queries to obtain the desired information. You can predefine patterns or utilize natural language processing techniques to enable more flexible inquiries. The model will interpret the query based on its learned representations and patterns, providing relevant information in response.
Given the limited information available about your project's details, it's important to emphasize that the effectiveness of these techniques may rely on other factors, such as the quality and structure of the data, available computational resources, and the complexity of the information you seek to extract. Further research and experimentation will be necessary to fine-tune the process to suit your specific project requirements.

What model would be suitable for this type of problem? I mean, when I say training, should I be training a neural network for example?

@Hlkwtz This user probably user ChatGPT or something like that to generate this answer. I'd not trust this answer or user.

@nbro True, I should've looked at his profile first, thanks for notifying.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Are sentence-transformers the best available pretrained models for computing embeddings for a sentence?
I am playing around with the sentence transformers pretrained models to compute embeddings for a sentence and then compare it using the cosine similarity on the two vectors. I was using the sentence-transformers/all-MiniLM-L6-v2 model and used the inference API on the model page.
For the two sentences:

delivery fee not showing
delivery fee showing twice

the similarity score was nearly 0.72 which seems very high for the two sentences that are in fact kind of opposite of each other. I would rather believe that these are orthogonal and would like them to have a score close to 0.
Can you guys help me with any other model that might give better results? Also, read about Universal Sentence Encoder by Google. Any reviews about that?

The language model is not only about the appearance or not of the delivery fee. Therefore the sentences should not be expected to be polar opposites. Neither are they orthogonal concepts (with nothing in common) in the space of all possible sentences. Instead, it is quite reasonable that a significant part of the embeddings capture the shared concept of "factual statement about visibility of a delivery fee", which does not vary between the two sentences and thus you should expect a certain amount of similarity from that.
There should be a vector linking the two embeddings that is approximately about an observed number of entities. You can extract it by taking one embedding away from the other (always subtracting in the same sense). This vector may only be stable when comparing very similar pairs of sentences, it may not be as global as simpler, stronger concepts, e.g. gender of a single word subject (when using a gendered word like "king" or "queen"), because there are many ways to express the same idea, but the details of what specific words are used will be important.
You can check for this, and see if the embeddings are capturing useful information for your project by comparing this difference vector with difference vectors between visibility or not of any other entities. For example, you could switch "delivery fee" with "postcode" or any other component on your form, and you should get other pairs of vectors, and those pairs should have a similar difference between them.
Even if this is not the case, you may not need to find some other embedding system with different properties (and you most likely won't find any generic embedding system which has zero similarity between the two sentences). Instead you should test the utility of the embeddings in other ways. If you want to distinguish between the two sentences presented, you may need to create a model other than measuring vector similarity between embeddings.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Advanced / Complex Neural Network (System) Design
In addition to basic forward networks, many books cover other basic network designs like CNN's and RNN's. However, they don't really go any further than that, explaining things like common approaches to solve complex tasks with more complex systems, e.g. dividing the work/data between networks, chaining different types of networks for different tasks and combining different partial solutions to a final solution.
For example, I read that it's common to use multiple RNN's in serial and/or parallel for language processing like translating text, dividing the work to something like understanding the intent first, then translating it, avoiding a (probably wrong) literal translation of the plain words without any meaning. Another example I remember reading about is how game AI's, e.g. a chess AI, use different types of networks to analyze moves, weight its options and decide on strategies. Or a robot using a CNN to analyse camera footage and then using other networks to process the input and decide on what action to take.
I found this post, which mentions to books. I know the first one (NNDesign), though I haven't read it yet, but it doesn't seem to cover what I'm looking for. The second one seems to have a chapter on modular networks, which seems to go into the right direction. It's now much, though.
Does anyone have some good books/resources on this kind of topics?

"Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
"Neural Network Methods for Natural Language Processing" by Yoav Goldberg
"Deep Reinforcement Learning" by Pieter Abbeel and John Schulman

Thank you for your reply. I will take a look at these books.

@ThorstenSchmitz This user has used ChatGPT or a similar model to generate this and other answers. I'd not trust him or his answers. In fact, the first book mentioned does not really cover in depth complex neural networks. The others I don't know.


text:
Difficulty with 'group_by' in Polars
I am new to polars, trying to convert my pandas code to polars.
The below code is in pandas, (book --> pd.DataFrame)
book = load_data(0 ,'book' ,DataBlock.TRAIN)
book['wap1'] = calc_wap1(book)
book['wap2'] = calc_wap2(book)
x = book.groupby(['time_id'])['wap1'].apply(log_return)
book.shape, x.shape

Output : ((917553, 12), (917553,))
I tried to write the following code in polars (book -> polars.DataFrame):
book = load_book(0, block=DataBlock.TRAIN)
book = book.with_columns(pl.lit(calc_wap1(book)).alias("wap1"))
book = book.with_columns(pl.lit(calc_wap2(book)).alias("wap2"))

def log_return(wap1_series):
    """Calculates the log returns of a given wap1 Series."""
    log_wap1 = wap1_series.log()  # Take the log of wap1
    log_returns = log_wap1.diff()  # Calculate the difference (log returns)
    return log_returns

log_returns_df = book.groupby("time_id").agg(
    log_return_wap1=pl.col("wap1").apply(log_return)
)
log_returns_df.shape

Output :  (3830, 2)
log_returns_df.head(2)

3567    [null, -0.000222, … 0.0]
27153   [null, 0.000013, … 0.0]

I don't want this array, I need log_returns_df such that it can be added as new feature in book.
All the functions mentioned above are defined in code_base...
Please guide on how to approach this problem... Thank in advance


text:
Paper to cite about normalizing the inputs to a neural network
We can read that the normalization of the data that is input to a neural network is important and is considered as a best practice, for example SO #1, SO #2 and many other places. It looks to me that there is no consensus on why it works, but there are a wide variety of possible reasons, including that the initialization of the weights is assuming normalized data, and that the gradient descent might become unstable if the input variables are scaled very differently (see here).
I myself have also experienced in an application that applying Z-score normalization was essential in order to achieve good results. On the other hand, my understanding is that it is not theoretically proven that normalization will always allow to produce better results than without it, and it is just one of the techniques in the engineer's toolbox to try.
Now the question is, which paper to cite about that normalization of inputs is a best practice?
I tried to look for a good reference for it but I have not found a really good one. For example, I cannot find this in particular in the Goodfellow book. There is a lot about batch normalization (i.e., that is often applied in between layers, and the normalization parameters are learned), and some computer vision related normalizations, but not this. The Glorot paper does not mention normalization in this sense either (what they normalize is the initial weights). On the other hand, this technique is there in quite some blog posts and tutorials, e.g., this one.

If this is on-topic, I don't think we have yet agreed on this. It looks like you're doing research and ask us to help you do it. We should probably discuss this on meta. Please, open a post there to ask about if this question is on-topic: https://ai.meta.stackexchange.com/questions/ask

I’m voting to close this question because I think it's off-topic.

I have found a related part in

C. C. Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer International Publishing AG, 2018. isbn: 978-3-319-94462-3. doi: 10.1007/978-3-319-94463-0.

about "Additive preprocessing and mean-centering" and "Feature normalization" on page 127. It says:

It can be useful to mean-center the data in order to remove certain types of bias effects. Many algorithms in traditional machine learning (such as principal component analysis) also work with the assumption of mean-centered data. In such cases, a vector of column-wise means is subtracted from each data point. Mean-centering is often paired with standardization, which is discussed in the section of feature normalization.

The following paper is about time series and provides a good explanation.
Passalis, N., Kanniainen, J., Gabbouj, M. et al. Forecasting Financial Time Series Using Robust Deep Adaptive Input Normalization. J Sign Process Syst 93, 1235–1251 (2021). https://doi.org/10.1007/s11265-020-01624-0
PDF

One question that naturally arises from the aforementioned
observations is why normalization seems to affect the performance of
DL algorithms to this extent. To better understand this we need to
consider two different phenomena that often arise. First, the employed
normalization scheme significantly affects the generalization of the
model on unknown data. For example, when the raw price is used as a
feature, then a DL model operates on an fully unknown region of the
input space when the price exceeds the range of price values used
during the training. Methods such as sample-based standardization and
instance normalization [14] can mitigate the effects of this to some
extent, leading to improved generalization results. It is also worth
noting that recent literature demonstrated that applying the
appropriate input pre-processing Schemes, such as including the
normalized first order differences, emphasizing on specific
characteristics of the data, e.g., zero-crossing rate, can lead to
significant improvements in forecasting precision and trading
performance [18, 19], further highlighting the importance of
appropriately pre-processing the input time series. The second reason
is intrinsic to the nature of the gradient descent-based optimization
that is employed for training DL models. Even though the input
features carry the same amount of information regardless of the
scaling applied to the individual features, the gradient descent’s
updates that are used for training the model are affected by the scale
of the input features. Therefore, features with large magnitude have a
larger effect on the direction of the descent, often leading the model
toward different local minima than a model for which all the input
feature were normalized in the same scale. The normalization therefore
acts as a way to ensure that each feature will have “equal” chances
for affecting the optimization.


text:
Why is there a "reference free" option in DPO (Direct Preference Optimization)'s loss function?
There is a reference_free parameter in trl's loss function implementation of DPO, while the original DPO paper does not mention the concept of "reference free".
In trl's implementation:
    pi_logratios = policy_chosen_logps - policy_rejected_logps
    if reference_free:
        ref_logratios = 0
    else:
        ref_logratios = reference_chosen_logps - reference_rejected_logps

    logits = pi_logratios - ref_logratios

When reference_free is True, the loss is calculated as cross entropy between policy_chosen_logps and policy_rejected_logps, while reference_chosen_logps and reference_rejected_logps are not in use.
What are the pros and cons of the "reference free" DPO?

Hi @Yang Bo and welcome to AI Stack Exchange! To receive an answer on this question, it might help to add a few more details. Thank you for posting, and we hope to see more of your questions on this site in the near future!

@DeepQZero Thank you for your warm welcome message. I just added the related code and my interpretation of the implementation.


text:
How can a LLM optimise on it's own processing speed
Is it possible to give a large-language-model access to the time it takes to give it's answers and ask it to optimize on that?
An example prompt would be:

From now on, please measure the time it takes for you to give me an
answer and optimize that time so that it stays under 500 milliseconds.
It is alright if you give me incorrect information because of this
constraint.

We can assume that the human brain optimizes for processing speed to lower it's energy usage. How can this be applied to a LLM?

if you add "considering that on average every word you process takes X ms" it will probably already work, because you are pretty much just telling it to produce responses of length < 500/X

Yes, that would be limiting the output. But is the time needed for an LLMs to answer really linearly depending on the number of inputs tokens and output tokens.
Or are there some loops. Kind of how humans are able to reason, arrive at a conclusion, see that the have an edge case and spend more time thinking about the edgecase?
My request would be for the LLMs to output "there's an edge case I would need to analyze deeper if I had more time."

It is not currently possible to directly give a large language model access to measure the time it takes to generate its answers and optimize on that.
The model's response time is largely determined by the underlying hardware and software infrastructure, and optimizing it would require significant changes at the system level. While the human brain does optimize for processing speed to lower energy usage, applying this directly to a large language model would require a different approach due to the fundamental differences in their architecture and operation.
However, ongoing research in the field of artificial intelligence continues to explore ways to improve the efficiency and speed of language models.

Off topic, but this answer seem to be generated by an LLM. Could you confirm or deny that? And could you elaborate on the details of "require a different approach due to the fundamental differences in their architecture and operation.". Which approach would it require?


text:
Can a policy with gaussian distribution allow two distinct optimal actions to have distinctively high probabilities?
As an example to show the benefits of stochastic policy, I often have seen the below grid world example. Five blocks in a row. the first, third, and fifth are white(distinguishable states), and the second and fourth are gray(for agent, these two states are equivalent, non-distinguishable). positive reward if goes down in the middle state, negative reward if goes down in the first and the fifth states. They often say, in the gray region, it's best to put 0.5 probability to each left and right actions and it is possible only by stochastic policies.
Let Left = 0, Down = 1, and Right = 2 be action values. Let those three actions, left, down, right be available for all states, that down action in gray state will just let it stay with negative reward. My question is, for the gray region, if we use gaussian distribution for our policy, can we set up 0.5 probability for each left and right action? wouldn't it naturally make the probability to choose down action quite high as we only modify mean and variance value?
I just thought it's interesting that most RL paper seems to use gaussian distribution for stochastic policy but that distribution cannot even solve this simple setup, which is often used in teaching the benefits of stochastic policy. Or, am I wrong?

Assuming by distinct you mean that, for example, the euclidean distance between the two actions is sufficiently large, then no it cannot be true. This is because the Normal distribution is uni-modal. There is an interesting paper that uses this fact as motivation to replace Normally distributed action selection in on-policy optimisation of MuJoCo tasks with discretised variants since a discrete (e.g. softmax) distribution can be multi-modal.
An alternate to discrete actions could be to use a mixture of Gaussians. This allows the modelling of multiple-modes, and in particular if you have prior information about your action space that informs you how many modes there are likely to be, it goes a long way to knowing how many Gaussians in the mixture you'd need. An example of this being used, whilst not directly to parameterise the action distribution, is in Distributional RL, where the returns are modelled as a probability distribution. The authors of this paper look to use MoG as an alternative to the commonly used C51 parameterisation (modelling the returns with a discrete distribution of 51 evenly spaced atoms) since this typically requires knowing the lower/upper bounds of your returns, whereas MoG is defined over the whole real line so doesn't require this prior information.
For what it's worth, a Normally distributed policy would not be applicable to the example you have given, since this looks to be an environment with a discrete action space.

great answer, maybe can be also pointed that if we have some prior knowledge on the number of modes, we can use a mixture of gaussians, and that using enough of them (good luck with the "enough"...) can converge to the true distribution https://stats.stackexchange.com/questions/365155/a-gaussian-mixture-model-is-a-universal-approximator-of-densities


text:
How to find an argument of a NN function(which returns a distribution) to minimize a KL divergence?
Consider a neural network function $f:\mathbb{R}\to distribution$. For simplicity, maybe consider that it returns a gaussian distribution.
I want to find $\arg\min_{s\in\mathbb{R}}D_{KL}(f(s),q)$ for some fixed distribution $q$.
Is there an efficient closed-form method to find such $s$? or do I need to run gradient descent with respect to $s$ and it might get stuck into local-optima?

For someone who might want more context: I'm reading a paper about cross-domain transfer in RL.
Cross-Domain Transfer via Semantic Skill Imitation, by Karl Pertsch and et.al...
https://arxiv.org/pdf/2212.07407
Here, to map a state in the source domain to a state in the target domain, the author suggests finding a target domain state that minimizes below loss. Here $p_S(k|s^S)$ is a distribution of semantics for the source state $s^S$, and I think can be considered as fixed for each source state. $p_T(k|s^T)$ is a distribution of semantics for the target domain state $s^T$(trained NN function). so minimizing this loss means we consider a target domain state that has the most similar semantics to the source domain state to be the representation of the source domain state in target domain.
$\min_{s^T \in D_T} \left( D_{KL}(p_T(k|s^T), p_S(k|s^S)) + D_{KL}(p_S(k|s^S), p_T(k|s^T)) \right)$
However, as I know, state space is continuous, so I am confused how one can efficiently find such a target domain state that minimizes above KL term.

you made no assumption on $q$ thus, it's even very likely that you cannot even compute the KL (thus you can't even do GD)

@Alberto umm what if we consider q also to be a gaussian distribution?

Since you made no assumption on what $q$ is, the problem is untractable
If you assume $q$ to be Gaussian, it's easy to prove that the KL distance is minimized when you have $\mu_{f(s)}, \sigma_{f(s)}$ equal to $\mu_q, \sigma_q$

but is there a closed-form or clever way to find s that $\mu_{f(s)},\sigma_{f(s)}$ equal to $\mu_q,\sigma_q$?

@user3315463 for the question you posted, it looks like you assume to have $q$, this implies that you also have $\mu,\sigma of $q$... if not, please add further details on what's $q$


text:
Why some papers focus on constructing large dataset from real robots, instead of simulations?
Recently, I have seen papers about large datasets for robotics such as DROID(https://droid-dataset.github.io/) or Open X-Embodiment(https://robotics-transformer-x.github.io/).
As I see, the datasets are specific to some types of robots(although X-Embodiment allows one robot to learn from another robot's data) and environments. If one wants to add another robot into the dataset, they have to do all data sampling again, which is quite expensive. Some environments might be difficult to reproduce, especially as they collected data from all the labs in the world.
I am wondering: why don't they instead set up data collection procedure on simulation? it will make the data collection way cheaper. When they want to add a new robot and collect data with the same tasks and environments like other robots, they can do it easily. It is also easy to add a new task and collect data from all robots/environments. Then, why they collect data in real world while giving up on such reproducibility/extensibility? Is Sim2Real that bad, even if it can collect way more samples easily?


text:
Understanding the function of attention layers in a convolutional neural network (U-Net in a diffusion model)
I am trying to understand the neural network architecture used by Ho et al. in "Denoising Diffusion Probabilistic Models" (paper, source code). They include self-attention layers in the model, applying them to the feature maps output by previous convolutional layers (ResNet blocks). I understand self-attention in the context of sequential data, but here there is no sequence of vectors, just a single image to be processed by self-attention. I do not understand what the self-attention layer is doing to the feature maps.
Question: please could you explain the function of the self-attention layers in this CNN?
My guess is that the self-attention layer is treating pixels like sequence elements i.e. it uses keys/queries to find the pixels in the input feature map that are most relevant to the pixel at some spatial location in the input feature map, then forms a relevance weighted sum of these pixels (or rather their corresponding values) as the output at that location. But this perspective doesn't seem to be consistent with the mechanics of the operation. In sequential data, we have Q, K, V matrices which store embeddings pertaining to the different sequence elements as different rows in the matrices, and it seems that we're replacing these with Q, K, V matrices that are derived from a single image. So when we e.g. calculate $QK^\top$ we are not computing the dot-product similarity of the queries and keys of different elements=pixels, because a row of $Q$ and a row of $K$ does not correspond to one element=pixel in the image (contrast to sequential data where they would correspond to specific sequence elements).
Note: from reading the paper/source code, I think that the self-attention operation is working according to the diagram below, but please correct me if this isn't the case.

(Figure reproduced from this paper and a similar set-up can be found in the self-attention GAN paper.)

@DeepQZero Thank you very much for welcoming me and for the feedback on my question! I wanted to make sure that I didn't make the problem any more confusing than it needed to be since AI/ML is already fairly confusing intrinsically haha :)

The implementation of self-attention in the source code for the "Self-Attention Generative Adversarial Networks" (SAGAN) paper is somewhat easier to follow than that in the "Denoising Diffusion Probabilistic Models" paper. Assuming that both papers are using the same approach to self-attention on feature maps (which I think they are), the question can be answered by inspecting the definition of self-attention used in SAGAN:
class Self_Attn(nn.Module):
""" Self attention Layer"""
def __init__(self,in_dim,activation):
    super(Self_Attn,self).__init__()
    self.chanel_in = in_dim
    self.activation = activation

    self.query_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim//8 , kernel_size= 1)
    self.key_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim//8 , kernel_size= 1)
    self.value_conv = nn.Conv2d(in_channels = in_dim , out_channels = in_dim , kernel_size= 1)
    self.gamma = nn.Parameter(torch.zeros(1))

    self.softmax  = nn.Softmax(dim=-1) #
def forward(self,x):
    """
        inputs :
            x : input feature maps( B X C X W X H)
        returns :
            out : self attention value + input feature
            attention: B X N X N (N is Width*Height)
    """
    m_batchsize,C,width ,height = x.size()
    proj_query  = self.query_conv(x).view(m_batchsize,-1,width*height).permute(0,2,1) # B X CX(N)
    proj_key =  self.key_conv(x).view(m_batchsize,-1,width*height) # B X C x (*W*H)
    energy =  torch.bmm(proj_query,proj_key) # transpose check
    attention = self.softmax(energy) # BX (N) X (N)
    proj_value = self.value_conv(x).view(m_batchsize,-1,width*height) # B X C X N

    out = torch.bmm(proj_value,attention.permute(0,2,1) )
    out = out.view(m_batchsize,C,width,height)

    out = self.gamma*out + x
    return out,attention

The calls of .view(m_batchsize,-1, width*height) in the forward pass are flattening the spatial image dimensions from (W,H) to (N,) where N=H*W, so the 2D array of pixels in the feature map is being converted into a 1D sequence of pixels, then  self-attention is being computed over these sequence elements in the usual manner. So the guess in the question is correct; self-attention is being computed in a pixel-wise manner, and the complaints following the guess are resolved when flattening the feature maps before computing self-attention.

This makes sense by simple analogy to the transformer: Each pixel is like a token, and the feature maps are like the embeddings of the token.  (The similarities between feature-maps for a pixel and token embeddings is strong.)  Attention maps tokens to tokens, so it feels correct for attention to map pixels to pixels.. That this is O(w^2 h^2) does make this approach seem fairly expensive.


text:
Non differentiable loss function train with actor critic style
I'm working on a project where a non differentiable loss is there. I'm thinking about how should I deal with them.
My model is a very big lstm model (about 1M parameter), and after 500 steps (not sure if it's enough) of differential evolution, the loss didn't decrease at all.
I'm thinking that actor critic can see as follows:
The policy network updates by value function, which is calculated from a reward estimator (Critic).
I'm wondering if I can take the thought of "estimator", and use it on loss estimation, then I can replace the non differentiable loss as that estimator, then also train the estimator each step. Maybe someone has already done that, if so, I want to know the name of the method.

what's your loss? what differential evolution method did you use? CE/CMA-ES?

The part make the loss function not differentiable is "argmax" and "split the list by a special element". And I think DE is an algorithm just like cma-es.

i mean you can definitely try reinforce, not quite sure how to help without a formal definition of the loss


text:
Comparing the performances of GPTs with deep learning in the field of binary files and their related reports
Regarding the case study of a dataset including binary files (containing assembly code) and reports related to each file (the content of the static analysis of the file as well as the analysis of the dynamic functionality of the file, with a specific format), can it be generally claimed that the learning on this dataset with the use of GPTs will be more effective than training by other deep learning methods?
What exactly should be considered for this comparison?
An example of assembly code of a binary file may be this simple code:
global    _start

          section   .text
_start:   mov       rax, 1                  ; system call for write
          mov       rdi, 1                  ; file handle 1 is stdout
          mov       rsi, message            ; address of string to output
          mov       rdx, 13                 ; number of bytes
          syscall                           ; invoke operating system to do the write
          mov       rax, 60                 ; system call for exit
          xor       rdi, rdi                ; exit code 0
          syscall                           ; invoke operating system to exit

          section   .data
message:  db        "Hello, World", 10      ; note the newline at the end

And an example of features that a report can present are:
hash of binary file, sections of file (.text , etc.), strings, network connections, activities carried out in the registry, etc.

Welcome to AI stack exchange. It is not clear what kind of information you are hoping to extract from binary files, and what kind of training data you have access to. Please give a more concrete example of the function(s) you want to compare, including inputs and outputs.

@Neil_Slater Thank you very much, consider the problem in this way that our inputs consist of a set of pairs containing a binary file (the content of the assembly code of a program that is compiled from a high-level language) and a report file corresponding to this binary file. This file describes both the static characteristics of the file and its dynamic characteristics.

@Neil_Slater Static properties can include things like used libraries, system calls, etc. Dynamic properties also include what can be extracted from the execution of the file on a system.
Generally, GPT will use different parameters for learning. But consider the final goal that we intend to identify the binary files that are in a specific category (after which categories are considered). That is, after learning, when a binary file is given to the model, it will tell in which category it will be placed.

Thanks. Please give more concrete examples to go with this description (e.g. a snippet of the binary file, and of the report, and labels of some of the classes), and use [edit] to insert them into the question.

@NeilSlater It has been edited.

can it be generally claimed that the learning on this dataset with the use of GPTs will be more effective than training by other deep learning methods?

No. The only way to find out which would be better for your purposes here is to try both. A very critical factor here will be amount of training data. An LLM may perform better when you have little or no training data, whilst a deep learning model may be far more data hungry but capable of performing better above a certain threshold of data.
If your use case was closer to natural language comprehension, resulting in language manipulation outputs, then a tuned LLM may be better even with large amounts of training data for deep learning, since the LLMs capabilities may be a good superset of your task. However, the more specialist and unique your data analysis needs to be, the worse a general-purpose LLM will be at working with it, and there are strong limitations on how much you can tune such a model to perform other functions.

What exactly should be considered for this comparison?

Before starting tests, select a metric that measures the utility of results in a way that you care about for the task. Common abstract metrics for classification are accuracy, F1 score, area under ROC. If you can customise a metric to measure true rewards/costs to your project for correct and incorrect classifications, that is best. You may also want to consider cost/benefits of training effort, if there is a limited project budget to create your model, and cost per inference to run the models in production.
Ensure you have enough correctly labelled test data to train with, and plot learning curves for performance versus amount of training data used. These plots can be used to decide whether collecting more training data would be worthwhile, because hitting a plateau in performance versus training data means that more training data would not be likely to improve a model. Whilst conversely, if doubling the amount of training data improves your metrics noticeably then it may be worth the effort to double it again.


text:
Why CNN filters (kernels) are randomly initialized?
I learned that when CNN filters are defined, they are initialized with random weights and bias(Im not sure about bias).
Then as learning step goes on, the weight values change and each filter makes its own feature map.
What I don't understand is that, if filter is initialized with random values, is there any chance that

different filters make the same feature map
feature map varies every time it repeats.

It seems little unefficient to initialize the weights of every filter randomly. More precisely, I think (in most of the networks) the number of filters is too small to get meaningful features.
Is the second case is why CNN has randomness?

first of all its is best to initialize randomly according to some distribution, for instance if you are using Relu activation, initialize according He-normal, or He-uniform, see the corresponding papers for why. second, if you initialize all weights similarly well then they all be treated in the same way by the gradient descent, and converge to similar values. finally in stochastic optimization, you need some kind to diversity to be able to explore the search space and find the solution. You do not no where the optimum solution is you need to find it, diversity ensures that you have a chance.

Can you please put your specific question in the title? Thanks.

It is usual to initialize parameters with "a good guess", when you have prior information, in order to help the model converging.
However in deep learning most of the time you have no clue what the weights should be, so you initialize them randomly following certain schedules like Xavier initialization or Kaiming, that prevent gradients from vanishing or exploding (due to a bad initialization) by setting the mean activations to be zero and the variance of the activations to be constant accross all layers.
Regarding your questions :

Yes it is totally possible that multiples filters learn the same thing. In fact, a good part of the filters may be completly useless, which is the idea behing network pruning that tries to delete neurons in a network while keeping the same performances.

Yes, the network you'll obtain after training will depend on the initialization. You can have different results depending on the initialization (just permuting the filters, for instance, give a new network with same capabilities), since anyway you will never get the global optimum for your task but only sub-optimal weights. However if trained properly, the final results should not very too much.


text:
Neural networks with a continuous function as the input
I am rethinking image recognition algorithms, and want to use an idea I saw in a 3Blue1Brown video:
The basic idea is to map pixels on an image to a specific frequency—every point is distinct—and if the pixel is brighter, make the amplitude of that sine curve larger.
And we could simply map the pixels to frequencies going row by row—how it’s done in traditional implementations—but why not strive for something better? We can use a Hilbert Curve to map the pixels to frequencies such that the frequency associated with a specific location change minimally if we scale up our resolution. Full Video.
I was thinking of how this could extend to neural networks—and remembered how functions are just continuously infinite dimensional vectors. The input function here is the sum of frequencies with their respective amplitude.
So our first layer could be an infinite set of neurons, and the second layer could be a finite set of neurons. To connect these layers, we could have a series of functions (one for each neuron in the second layer).
To calculate the value of each neuron in the second layer, we could take the inner product with the input function (our image as a sum of sine curves) and a connecting function associated with this neuron.
Now, here’s where I’m having some trouble: how do we go about training such a network? Are there any gradient descent algorithms for such a neural network structure?
I was looking into the variational derivative, which we could potentially use in our gradient descent. But this seems to yield an analogue to a directional derivative—I’m not sure how to generalize this to a “variational gradient.”


text:
How is state space normalization done in off-policy algorithms like dqn?
There are 4 features in my state representation and they are in different ranges. So I'm thinking state normalization would reduce the bias on certain features. And also, in the problem I consider, there is a distribution shift in the inputs between train and test times. I think state normalization would help both issues.I found some suggestions in this question here
Can someone provide an example where this is done? It's a bit confusing to me.

Can you explain what you know about "state space normalization"? Also, why are you interested in this? Thanks for providing more context.

there are 4 features in my state representation and they are in different ranges. So I'm thinking state normalization would reduce the bias on certain features. And also, in the problem I consider, there is a distribution shift in the inputs between train and test times. I think state normalization would help both issues.

Thanks for clarifying this. Can you please include this info in your post.

Normalization, also known as scaling or standardization of input features, is essential not only in Reinforcement Learning but in machine learning in general. Normalization is widely accepted as a good practice, particularly when dealing with features that vary in scale and range. By doing so, you help ensure that the training process is both stable and efficient.
Here are a couple of main reasons:
Convergence Speed: Normalized inputs can accelerate the convergence of the algorithm by ensuring that the gradient descent steps are balanced across all dimensions. Without normalization, features with larger scales dominate the gradient updates, which can lead to slower convergence or divergence.

(source)
Equal Contribution: By scaling features to a similar range, you encourage the model to consider all features equally during learning. Otherwise, features with larger magnitudes may have a disproportionate influence on the distance metrics, loss calculation, and learned weights.
Better Learning Rate Generalization: When features are normalized, the learning rate parameter becomes more general and less dependent on the scale of the features. It's easier to find a suitable learning rate that works well across all features.

(source)
The normalization is done not only for the input data but also for intermediate layers. Features with large values can cause numerical instability, leading to issues such as exploding gradients in neural networks. Normalization keeps the values within a controlled range, contributing to the numerical stability of the training process. You can read more on it in Bach Normalization and Group Normalization papers. These papers provide theoretical information on normalization in general. We can consider the output of each hidden layer as an input to the next one. So some insights are also useful to understand normalization of the input data.
Depending on your data you may choose different approaches for normalizing your input data. For example, for images, it could be a simple mapping of pixel intensities [0, 255] to [0, 1] or [-1, 1] range.
Another common practice is to calculate the mean and standard deviation of your training set for normalization.
One more technique is min-max scaling, where each feature is scaled to a range [0, 1] or [-1, 1] by subtracting the minimum value and dividing by the range.
It's important to note that you should not recompute the mean and standard deviation for normalization during the testing phase or while the model is in production. Use the statistics computed from the training set to avoid any leakage and to ensure consistency between the training and testing normalization processes.

This is a lot of useful information about normalisation, but fails to address the question. The OP needs to understand how to approach normalisation when using an online + replay structure in a DQN. They cannot apply standard normalisation techniques because they don't have a complete training dataset at the start of training. Batch normalisation is not really in scope here either, they are asking about input features not NN architecture

@NeilSlater Since we are speaking about DQN (which is a neural network), the normalization directly addresses the question. Without further information about the source of the data, there is nothing to add. Regarding BN and GN, these papers provide theoretical information on normalization in general. We can consider the output of each hidden layer as an input. So this also applies to the normalization of the input data

Thanks for the detailed explanation @ArayKarjauv. But I don't understand how to apply this in the context of online + replay structure in a DQN. How do I calculate the mean and standard deviation to do this when I don't have a full dataset and if I am to use a running average how do I do this?

@AkiA can you provide us with more details on your data? otherwise, if you know the range of your data, you can always standardize it by mapping it to a smaller range


text:
Custom Loss Function Traps Network in Local Optima
I am working with a feedforward neural network to fit the following simple function:
N(1) = -1
N(2) = -1
N(3) = 1
N(4) = -1

But I don't want to use the Mean-Squared Error; I'm using a custom loss function that "guides" the network to the correct output in a different way. My custom loss function works as follows:
if input == 3:
  loss = -1 * output
else:
  loss = 1 * output

If the neural network has a tanh activation function, the minimizing function for this loss is clearly N. So the network should tend to fit N.
But my network is only outputting -1 (even when the input is 3). I wrote a small sample code with pytorch to show this:
Loss function:
import torch
import torch.nn as nn

class CustomLoss(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x, i):

        multiplier = -1 if i == 3 else 1

        return multiplier * x

Initializing Network, Optimizer, and Loss:
net = nn.Sequential(
    nn.Linear(1, 20),
    nn.Tanh(),
    nn.Linear(20, 20),
    nn.Tanh(),
    nn.Linear(20, 20),
    nn.Tanh(),
    nn.Linear(20, 1),
    nn.Tanh(),
)
optimizer = torch.optim.Adam(net.parameters(), lr=1e-3)
loss_function = CustomLoss()

Optimizing through Backprop:
for _i in range(100000):
    i = _i % 4 + 1
    input_value = torch.FloatTensor([i])
    output_value = net(input_value)

    optimizer.zero_grad()
    loss_value = loss_function(output_value, i)
    loss_value.backward()
    optimizer.step()

    print(i, output_value.item(), loss_value.item())

And after running it for a minute or so, the program outputs:
Input |         Output      |        Loss
1     | -0.9999958276748657 | -0.9999958276748657
2     | -0.9999961853027344 | -0.9999961853027344
3     | -0.9999962449073792 |  0.9999962449073792 <--- Loss can be minimized by making Output 1
4     | -0.9999962449073792 | -0.9999962449073792

As you can see, N(3) is not properly fitting. So I'm led to believe the model is trapped in a local optima. But I don't know exactly why, since the MSE version of this code easily works (I tested it with the exact same setup with an altered loss function). Does anyone have any ideas?
I also discovered something else: if I make N(1) = 1 instead of N(3) = 1 (i.e. swap the outputs of 1 and 3), the network fits perfectly. I have absolutely no idea why this happens. It seems the loss function can approximate:
N(1) = 1
N(2) = -1
N(3) = -1
N(4) = -1

But it can't approximate:
N(1) = -1
N(2) = -1
N(3) = 1
N(4) = -1

Somehow.

This is not meant as a complete answer to your question, but here are some things in your approach that arguably mess with model convergence.
Firstly, optimization is all about gradients, your loss function technically guides the model towards outputs of $-\infty$ and $+\infty$ (prior to $\tanh$), depending on the case. But what the gradient of the $\tanh(x)$-function does:
$\lim_{x \rightarrow \infty} \tanh(x) = 1$
$\lim_{x \rightarrow \infty} \tanh'(x) = 0 \quad$ ($\tanh'(x) = 1 - \tanh^2(x)$).
$\lim_{x \rightarrow -\infty} \tanh(x) = -1$
$\lim_{x \rightarrow -\infty} \tanh'(x) = 0$
As soon as the model overfits to the solution where everything is $-1$, there is no gradient to recover from it and this scenario is likely to happen, because $-1$ is the more frequent case in the training set.
<-- This would be my explanation why your model converges to -1 for all inputs. This is also why you statement "Loss can be minimized by making Output 1" is misleading. A gradient of 0 means no training regardless of the loss value.
Another thing is that you are using Adam optimizer - usually a great choice but it seems not suitable in this case. Its an optimizer with momentum, which cannot deal well with rapid slope changes during training, it will basically overshoot and therefore kind of ignore the slope change - this is amplified by training on individual samples and not on batches. (A slope change happens due to your if statement - it changes from $+1$ to $-1$ or vice versa.)
I'd argue its a mixture of the first two points why your special case of setting N(1) = 1 actually works: It's your very first training sample, thus it initializes the momentum of Adam which is a much better precondition not to converge to the $-1$ solution prematurely.
If you want to keep trying with this custom function, I'd try the following 1) reduce model size 2a) Use batched training instead of individual samples to make training less noisy 2b) balance the batches such that $+1$ and $-1$ occur in equal amounts. 3) Experiment with smaller learning rates 4) You can also try other optimizers, but if everything else is configured a bit better Adam should also work. (I'd try in that order and after every change see what happens).

Doesn't the gradient of $(tanh(x) - x_0)^2$ also vanish at $\pm \infty$ though? The MSE approach works, even though the model is technically overfitting to the case where every input outputs -1 in the beginning, right? Sorry for replying so late.


text:
Sutton & Barto: Exercise 7.11 mistake?
Exercise from the book:
7.11 Show that if the approximate action values are unchanging, then the tree-backup return (7.16) can be written as a sum of expectation-based TD errors:
$$
\begin{align*}
&G_{t:t+n} = Q(S_t, A_t) + \sum_{k=t}^{min(t+n-1,T-1)} \delta_k \prod_{i=t+1}^{k} \gamma \pi(A_i | S_i),\\
&G_{T-1:h} = R_T\\
&G_{h:h} = R_h + \sum_a (\pi (a|S_h) Q(A_h, S_h)) \text{, where }h < T-1\\
&\text{ where: } \delta_t = R_{t+1} + \gamma  \bar{V}_t(S_{t+1} - Q(S_t, A_t)
\end{align*}
$$
Question: Ain't $G_{t:t+n} = Q(S_t, A_t) + \sum_{k=t}^{min(t+n-1,T-1)} \delta_k \prod_{i=t+1}^{k} \gamma \pi(A_i | S_i)$ missing return for $G_{h:h}$ in case $ h < T-1$?
Here is my solution, I wonder if there is mistake or $G_{h:h}$ really should be included in sum?
my solution:
$$
\begin{align*}
&G_{t:t+n} = R_{t+1} + \gamma \sum_{a \neq A_{t+1}} \pi(a|S_{t+1}) Q(S_{t+1},a)  + \gamma \pi (A_{t+1} | S_{t+1})G_{t+1:t+n}\\
&=  R_{t+1} + \gamma \sum_{a \neq A_{t+1}} \pi(a|S_{t+1}) Q(S_{t+1},a)  + \gamma \pi (A_{t+1} | S_{t+1})G_{t+1:t+n}  + \gamma \pi(A_{t+1}|S_{t+1}) Q(S_{t+1},A_{t+1}) - \gamma \pi(A_{t+1}|S_{t+1}) Q(S_{t+1},A_{t+1})\\
&= R_{t+1} + \gamma  \bar{V}_t(S_{t+1}) + \gamma \pi (A_{t+1} | S_{t+1})G_{t+1:t+n} - \gamma \pi(A_{t+1}|S_{t+1}) Q(S_{t+1},A_{t+1}) = R_{t+1} + \gamma  \bar{V}_t(S_{t+1}) + \gamma \pi (A_{t+1} | S_{t+1}) (G_{t+1:t+n} -  Q(S_{t+1},A_{t+1}))\\
&= R_{t+1} + \gamma  \bar{V}_t(S_{t+1}) + \gamma \pi (A_{t+1} | S_{t+1}) (G_{t+1:t+n} -  Q(S_{t+1},A_{t+1})) + Q(S_t, A_t) - Q(S_t, A_t)\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) (G_{t+1:t+n} -  Q(S_{t+1},A_{t+1}))\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) (R_{t+2} + \gamma \sum_{a \neq A_{t+2}} \pi(a|S_{t+2}) Q(S_{t+2},a)  + \gamma \pi (A_{t+2} | S_{t+2})G_{t+2:t+n} -  Q(S_{t+1},A_{t+1}))\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) (R_{t+2} + \gamma \sum_{a \neq A_{t+2}} \pi(a|S_{t+2}) Q(S_{t+2},a)  + \gamma \pi (A_{t+2} | S_{t+2})G_{t+2:t+n} -  Q(S_{t+1},A_{t+1}) + \gamma \pi(A_{t+2}|S_{t+2}) Q(S_{t+2},A_{t+2}) - \gamma \pi(A_{t+2}|S_{t+2}) Q(S_{t+2},A_{t+2}))\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) (\delta_{t+1} + \gamma \pi (A_{t+2} | S_{t+2})G_{t+2:t+n} - \gamma \pi(A_{t+2}|S_{t+2}) Q(S_{t+2},A_{t+2}))\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) \delta_{t+1} +  \gamma^2 \pi (A_{t+1} | S_{t+1}) \pi (A_{t+2} | S_{t+2})(G_{t+2:t+n} - Q(S_{t+2},A_{t+2}))\\
&= Q(S_t, A_t) + \delta_t  + \gamma \pi (A_{t+1} | S_{t+1}) \delta_{t+1} +  \gamma^2 \pi (A_{t+1} | S_{t+1}) \pi (A_{t+2} | S_{t+2}) \delta_{t+2} + ... + \gamma^{t+n-t} \pi (A_{t+1} | S_{t+1}) \pi (A_{t+2} | S_{t+2})...\pi (A_{t+n} | S_{t+n}))(R_{t+n} + \sum_a (\pi (a|S_{t+n}) Q(S_{t+n}, a)) - Q(S_{t+n},A_{t+n})) \\
&= Q(S_t, A_t) + (R_{t+n} + \sum_a (\pi (a|S_{t+n}) Q(S_{t+n}, a)) - Q(S_{t+n},A_{t+n})) \prod_{k=t+1}^{min(t+n,T-1)} \gamma \pi (A_k, S_k) + \sum_{k=t}^{min(t+n-1,T-1)} \delta_k \prod_{i=t+1}^{k} \gamma \pi(A_i | S_i) Q(S_t, A_t) \\
\end{align*}
$$

+1 even only for having taken the time to write this whole thing down in latex


text:
Fine-tuning to generate accurate embeddings for specialized language?
I'm working on creating custom embeddings for a dataset of legal texts in a specific one language(for example, Uzbek. It can be any).
How should I select a model for fine-tuning to generate accurate embeddings, particularly for specialized language like legal terminology? Should I consider the languages/scripts on which the base model was trained? What other factors should I consider when selecting the model?
If possible, could you also outline the steps for doing this?

Please, focus on one question and specific problem per post. You can always create multiple posts.

We've released a framework to make LLM embeddings easier to finetune (i.e., customize), evaluate and use: github.com/nlp-uoregon/ullme (paper: arxiv.org/abs/2408.03402).

To create custom embeddings for a dataset of legal texts in Uzbek, you can fine-tune a pre-trained encoder-only language model that is capable of capturing nuances in legal language. XLM-RoBERTa is a multilingual variant of RoBERTa, designed to work across many languages, including low-resource languages like Uzbek. It has strong performance across a variety of tasks and can be fine-tuned for specific domains like legal texts.

It is based on Facebook’s RoBERTa model released in 2019. It is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data... trained on 100 different languages. Unlike some XLM multilingual models, it does not require lang tensors to understand which language is used, and should be able to determine the correct language from the input ids.

After loading the pretrained model in your fine tuning process, prepare your legal texts in a format compatible with the datasets library of Hugging Face, such as DatasetDict for train, validation, and test splits. You can fine-tune the model on a specific task like legal text classification, named entity recognition (NER), or directly train it to produce embeddings.


text:
Filter scientific papers based on abstract information
I have a data frame that contains different numbers of columns, the important ones are:
Title Abstract

The length (number of words) varies between each row, with a total of 850 rows, but all are scientific papers related to science (immunology context).
I created this file using a Python script that checks for different combinations of keywords in a specific context to search in the literature the papers in a straightforward way.
To move forward I would like to reduce the number of rows, in this case, the number of papers that are useful for my research. With this in mind, I started using transformers to analyze each abstract in my data frame and check if it is useful for my research based on a specific question and context, so my focus is the transformers related to the question-answering process.
I'm completely new in this world of neural network architecture (transformers), so I do not know if I'm doing the process correctly or if the approach that I chose is correct.
Based on the research that I did my current approach is:
1. Use transformers from Hugging Face.
2. Select a dataset that works with Biological data context.
3. Select a model that is for the question-answering process in the context of biological data.
4. Upload the dataset in my workspace and train the model (?)
5. Define your question and define the context.
6. Upload your dataset (data frame) that you want to analyze and generate output with long and short answers (?)

So far, I think that I'm in point 3, fixing some issues related to the question-answering model (not related to this post). However, I doubt if all of this is necessary to perform my goal, reduce the number of papers that are useful for my research (that I need to include to perform discussion process in a paper writing process) based on a question and context.
Is this process correct for my goal?
Do you know if is there a better way?
If the process is correct, what model do you recommend for biological context?

Hi, Welcome to AI Stack Exchange. What's ultimate objective? finding relevant papers  among 850 papers based on query/context? or make QA model? Just checking what you expect form model - generated answer or relevant paper.

Thank you for your reply, my main goal is to be able to check each abstract in my data frame an select the ones that answer my question-context information, because I selected those 850 papers based on keyword criteria, for example, keywords 1, 2, 3..., so now I want to check which one of them really work for me and be able to reduce the number of papers to read

@KulinPatel Could I have your feedback on my code? Here is the link if you have time to check it:
https://stackoverflow.com/questions/78737322/issues-with-irrelevant-similarity-search-results-using-rag-and-llms-on-pubmed-da

I can't see the question.

If your goal is to quickly find the most relevant papers from abstract dataset, you don't need to train a new QA model. You can use vector database and embedding model to efficiently search your abstracts.
Suggested approach:

Create embeddings: transform all abstract (or chunks of abstracts) into numerical representations called embeddings. These embeddings capture the semantic meaning of the text. Pre-trained models will do that for you.

Vector Database: store these embeddings in a vector database designed for similarity search (for example ChromaDB, Pinecone). Add each vector embedding with metadata, including the paper title and potentially other relevant information.

Retrieval: when you have a query/question, convert it into an embedding using the same embedding model. Use the vector database's retriever to find the most similar abstract embeddings. The retriever will return the top-k most relevant abstracts, ranked by their similarity to your query. They will also have paper title as metadata.

So now you know top relevant papers. Read them!

Why this approach:

Searching a vector database is much faster than running a QA model over every abstract.
Once you create and store the embeddings, you can easily experiment with different queries without having to reprocess all abstracts each time.
Also, you don't need to train a new model, saving time/resources.

This approach is used for  Retrieval Augmented Generation(RAG) with LLMs. These retrieved abstracts, along with question itself, are passed to LLM to generate informative answers.

So, as I see and if I understand correctly, I can apply this directly to my csv file that contain all my paper abstracts
I will try your idea, thank you for your help

I just created a script to perform this task, now that I see the complete picture, it looks like a promising way to accomplish this task, thank you again for your help!!!!!!!


text:
How to convert a positionally encoded predicted embedding from a decoder to its matching token?
Is it valid to just subtract the positional encoding from a predicted output if the decoder was also positionally encoded? Or does masking take care of this problem, and the decoder should only learn the embedding (not positionally encoded)

No need to remove positional encodings from the predicted output.  In most  Transformer architectures, positional encodings are added to token embeddings to provide the model with information about the order of the sequence, as the Transformer encoder/decoder block does not have sequence awareness.  Once position encodings are added to  token embeddings, they become part of the model's learned representation, which the decoder uses to generate the output.
Both the encoder and decoder learn to interpret and use this positional information as part of the sequence processing. This integrated learning process allows the model to capture both the meaning of the tokens and their positional relationships within the sequence.
Subtracting positional encodings after the model has made predictions is not recommended. While masking helps the model attend to specific parts of the sequence, it doesn't "undo" the effects of positional encoding.
There's no need to "take care of this problem" for positional encoding - they are, in fact, essential to the model’s ability to generate outputs.
To convert predicted embedding from a decoder to its matching token, you typically pass the predicted embedding through a linear layer followed by a softmax. The linear layer maps the embedding back to the vocabulary space, and the softmax gives you the probability distribution over all possible tokens. The token with the highest probability is then selected as the predicted token.


text:
Why are rows of Attention Weights in a Hopfield Transformer the same?
I'm working on building a Hopfield Transformer using the github code from the paper (https://github.com/ml-jku/hopfield-layers/tree/master/hflayers) to forecast a timeseries dataset with 48 variables, 20 past timesteps, and 4 future timesteps. I'm then working to extract the attention weight matrix.
I'm noticing that the attention weight matrix has rows that are all the same.
The way I'm building this Transformer is with a source dataset of shape [176, 48, 20] and a target dataset of shape [176, 48, 4] where 176 is the batch size, 48 is the "sequence" length, and 20 (or 4) is the embedding dimension. In the encoder, I first embed and perform sequential encodings on the 20 timesteps to obtain a [176, 48, 1280] tensor, which gets sent to the Hopfield Encoder from the github (or the hopfield attention layer, either way it works the same), and then to the Decoder, with a linear layer that reshapes the source and target data shapes to match.
**My model does appear to significantly outperform a vanilla transformer. However when extracting the attention weight matrix, I'm noticing that the rows appear the same: **

Is this supposed to happen, or am I missing something about Hopfield Transformer functionality in the context of my data shape? I'm not sure what to do to resolve this - I'm happy to provide code if it helps!

Could you please formulate the title as a specific question? Thanks.


text:
Are K and V values reused in each decoder layer's cross-Attention in the original "Attention is all you need" paper?
I'm working with Transformers and have a question about the encoder-decoder structure. In each decoder layer's cross-attention, are the K and V pairs from the corresponding encoder layer reused for all cross-attention operations within that decoder layer? Or are they recalculated for each operation?
Also, if K and V are reused, doesn't this limit the decoder's effectiveness? It seems like only the Query values are changing and propagating through the model. Since most of the meaning is stored in the V layer, wouldn't using the same K and V hinder the decoder's abilities?


text:
Do people realize that AI can't produce new results?
I read a good article about how AI raises the performance of low performing employees generally while not helping high performers as much. So employers might hire people with less experience, pay high performers less, fire high performers, etc.
But don't people realize that AI uses past information? How can it help with areas like programming when you need to improve, use new techniques, tools, languages and so on? It can't, because it is an encyclopedia, not an expert.
Will people learn this before some very bad choices are made? How can it be explained to them that recycling doesn't produce new stuff?

Yeah. I think the problem is much more complex. I think you're oversimplifying many of the real considerations of how AI is used and their different capabilities. You're being too broad in your statement. What type of AI are we talking about - LLMs, Transformers, Neural Networks, RL agents, Genetic Algorithms? I think each of these algorithms should be considered individually and not lumped together as "AI" because it can bring ambiguity. Personally, it is not that clear what AI will be able to do in the future and the argument that it is just recycling old stuff is not very well founded.

@CesarRuiz Ok, so focus on the area of producing computer program code. My understanding is that it cobbles together things from existing examples. That can never write anything original in the sense of generalizing a method to a new language or situation. Heck, most humans have trouble with this. A lot has been written about why many programmers seem to lack understanding of basic programming concepts. So we're going to trust "the wisdom of (extrapolated) crowds"? Would you fly on that plane? Why pay people to do that?

I was heartened when I heard someone say that if you sign off on something, you the human  are responsible for understanding it fully and providing it as your solution that you must stand by. Using tech is not a get-out clause. I hope that this becomes the bottom line for use of AI-generated material.

I think it's still ambiguous because the methods I think you're talking about - the use of instruction trained LLM models for coding - does not represent all the LLMs able to code. I will reference the work by Deepmind with FunSearch that does not use instruction based models in this case and demonstrates to be useful in generating "new ideas". Although there's could also be ambiguity on that term too. Just curious what do you think about their work?

Well I made a mistake - technically FunSearch uses things like StarCoder or Codey that are instruction LLM models for coding. But they could use a fully pretrained model and still I think my argument could remain. I said this because at least for me the pretrained model could be seen as a more creative autocompleter of text even though much of its generations doesn't make much sense.

As a programmer I often peruse many many pages of potential solutions and quickly bypass things that are far too complex or off-track. Quite frequently I abstract ideas from entirely different scenarios to create a simple, concise, understandable, maintainable solution.  Even if AI could suggest things that are relevant, you still need knowledge and experience to select and adjust the code to fit the situation. Coding is not just sticking Lego together, it is more like surgery. I found the idea laughable in the article of having junior Lawyers use AI suggestions. And in court they'll flounder.

This question needs work. It presupposes an answer to a question (can AI generate novel solutions to coding problems), then is half rant, half impossible question about non-specific "people". I suggest alter the question to be about coding capabilities of AI, around the original assumption. Also worth clarifying whether asking about LLMs, or current crop of code assistants. These are far from being all of AI around code generation, although they get a lot of attention currently, and presumably what triggered the question

@NeilSlater thanks, I'll see what I can do.

This is an interesting political/social question, but unfortunately it will only lead to opinions. My opinion is that people sometimes learn things too late or don't learn until they don't see the immediate effects, which may be too late to recover.

First of all, the statement 'it (AI) is an encyclopedia, not an expert', thus implies it cannot 'improve, use new techniques, tools, languages and so on' is wrong. AI can discover new knowledge.
Take the example of algorithm, which is the heart of programming. Do we agree that if someone invents a novel, effective algorithm, that makes him an expert?
Then take a look at this work, where AI taught us a new and better way to multiple two matrices. We human have been tackling this task for thousands of years and the latest improvement was made in 1969 (half a century back). In writing better algorithm, AI can beat us to it.
In fact it is not at all surprising; years back, the infamous version of AlphaGo, the AlphaGo Zero, which uses no (zero, nil) human knowledge, beat the whole mankind in the 3,000 years old game. In fact, it was discovered that if we incorporate human knowledge to the AI, its performance actually worsens. Human 'expertise' can be junk.
If there is anything we human are better at: be humble and modest, realize and admit our limits, and always learn from those better than us.

Ok, improving matrix multiplication by going from 80 multiples down to 76. With massive amounts of hardware and work. Reminds me of replacing a headphone wire with 2 complex transceivers (Bluetooth) and 2 computers, all to go the same 3 feet. Will we ever tire of ourselves?

I couldn't read the linked article without creating an account, but OP mentions specifically "low performing employees". I suppose were are talking about programs like ChatGPT in this context, which is meant for the masses to generate answers "out of thin air". AlphaTensor and AlphaZero are in a totally different league, although they are good examples on how reinforcement learning can solve surprisingly difficult tasks. But those solutions are built by experts for experts.

@NikoNyrh the AlphaGo idea of having the AI figure things out for itself seems reasonable. We could just chuck programming and have AlphaCode make up a whole new thing from scratch. Worth a try...

@EdificialInelegance you missed the point completely - AI has made a discovery, a new algorithm which can do a calculation 10-20% more effective than we ever know. The "massive amounts of hardware and work" was to discover the algorithm, much like human spending a lifetime researching. No effort is needed to enjoy the benefit. I suggest you to take some time to re-read the article and think about the implication.

@NikoNyrh If I read OP correctly, his point was "(because AI) uses past information... How can it help... to improve, use new techniques... and so on?" And my point is that 1: not all AI needs past (or more precisely, human's) information; 2: AI has shown that it can invent new techniques, tool, languages etc. which we never know nor imagined (recall the novel moves of Go, and the never-known matrix algorithm). Even we narrow down the scope to LLM, just like OP commented, what stops him to combine LLM with (say) AlphaCode, that can invent new algorithm per our request on-the-fly?

So let it rip, discover away. Just don't include old rubbish in the setup conditions. Garbage In, Garbage Out

@EdificialInelegance Yes, I long for the day where we ignorance people learn to step aside and let AI do the job. No more garbage from human's bias and prejudice. AlphaGo Zero already showed us how harmful human's garbage knowledge can be.

"'Tis a consummation devoutly to be wished."

See, an encyclopedia using past info is talking. Can't produce new result though.

From what you write it seems that you believe that human new ideas are not based on past experience. This is an extremly bold statement very difficult to prove.
Almost everybody would define a new idea idea of the connection of two previously unconnected dots. Ai can do that as much as we can.

No, human new ideas are based only on experience. I think that AI is quite lacking in this area. Have you seen any AIs lately? What are they learning about a world of humans with needs and wants? Probably nothing. So why should we have them managing things for us?


text:
In policy gradient algorithms, if the model always predicts any action with a probability of 1, will the gradient always be 0?
The loss of policy gradient:
$$
\nabla_{\theta}J(\theta)=\mathbb{E}_{\tau\sim\pi\theta}\left[\sum_{t=0}^{T}(R_t-b_t)\nabla_{\theta}\log\pi_\theta(a_t|s_t)\right]
$$
where $\pi_\theta(a_t|s_t)$ is the probability of the model $\theta$ predicting the action $a_t$ in the state $s_t$.
Howerver, if the model always predicts a certain incorrect action with a probability of 1, then it necessarily follows that $\log\pi_\theta(a_t|s_t)=0$. So the loss is alwasy 0.
I discovered this issue because I found that my model always overfits by outputting any action with a probability of 1, and there's a high chance it's an incorrect action. At this point, the loss is 0, and convergence stops. What could be the problem?

"So the loss is always 0" doesn't mean that there is no learning... indeed, it's called "policy gradient", not "policy loss", thus you have to consider the gradient of the logarithm evaluated at 1 (since $\pi(a|s) = 1)$)
And indeed, $\nabla \ln \pi(a|s)$ when $\pi(a|s) = 1$ is $1$, thus learning can definitely still happen (though it might be slow)


text:
What are efficient methods for classifying tremor intensity, given challenges with 3D CNN and memory issues with a pretrained VideoMAE model?
I have a training dataset of 80 videos (without augmentation). They are 15 seconds video recorded at 30fps and 240*480 dimensions. What options do I have to train a classification model? The problem statement is I have videos of people with tremor. I want to classify the intensity of tremor on the scale of 1 - 3. Please suggest some ways to work with this
I have tried using 3D CNN but it requires excessive compute power. I also tried to train the model by fine-tuning pretrained VideoMAE model from huggingface but it only accepts 16 frames instead of 450 frames (15s * 30fps). I tried to change the arch but then it gives me memory error.


text:
Relationship between regularization and (effective) discounting in deep Q learning
I have a deep-Q-network-type reinforcement learner in a minigrid-type environment. After learning I can put the agent in a series of contrived situations and measure its Q values, and then infer its effective discount rate from these Q values (e.g. infer the discount factor based on how the value for moving forward changes with proximity to the goal).
When I measure the effective discount factor this way, it matches the explicit discount factor () setting I used.
But if I add a very strong L2 regularization (weight decay) to the network, the inferred discount factor decreases, even though I didn't change the agent's  setting.
Could someone help me think through why this happens?
Thanks!

L2 regularization pushes weights to be more normally distributed (see here), so your output will get smoothed out. However, reward functions are pretty discrete (e.g. did the mouse get the cheese or not?). Maybe you could try other priors for the weights? The Cauchy distribution is sharper, and has loss
$$\frac{\gamma^2}{2}\ln\left(1 + \left(\frac{w}{\gamma}\right)^2\right)$$


text:
How does reward work while training a Reinforcement Learning agent?
I am using PPO to train my environment which I created using stable baselines 3. I am confused if I should make the reward = 0 in the step function or not.
Initially, I used to have self.reward = 0 in the init() and reset() function and I used to calculate reward in the step function like this.
def __init__():
    self.reward = 0
def reset():
    self.reward = 0
def step():
    self.reward = self.reward + slope + calculation

But I have seen some implementation where people do this
    def __init__():

    def reset():

    def step():
        reward = 0
        reward = slope + calculation

What is the right way of doing it and how does it work?

The environment needs to return immediate reward only, from the step function. It's the agent's task to sum received rewards, the environment should not normally do that.
In some environments it may make sense to add to something that accumulated on actions and influences reward. E.g. perhaps you want to simulate investments where value accumulates over time. You would want to store that as an instance variable for the environment, and reset on episode start. However, rather than call that reward, name it after whatever it represents within your environment model. Convert it to immediate reward received from the step function as needed.
In short, the second approach you show, where reward is reset and calculated on every call to step is the standard approach that you should follow in most cases.


text:
Tensorflow Fine-tune BERT error
I have a piece of Tensorflow code:
class Classifier(keras.layers.Layer):
    def __init__(self):
        super().__init__()
        self.classifier = keras_nlp.models.BertClassifier.from_preset(
            "bert_base_en_uncased",
            num_classes=1,
        )

    def call(self, inputs):
        out = self.classifier(inputs)
        out = tf.squeeze(out, -1)
        return out

def get_model_2():
    inp = {
            'token_ids': keras.Input((512,), dtype=tf.int32, name=None),
            'padding_mask': keras.Input((512,), dtype=tf.bool, name=None),
            'segment_ids': keras.Input((512, ), dtype=tf.int32, name=None)
    }
    layer = Classifier()
    out = layer(inp)
    model = keras.Model(inp, out)
    return model

model = get_model_2()
model.compile(
    loss=keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[
        keras.metrics.TruePositives(),
        keras.metrics.FalsePositives(),
        keras.metrics.TrueNegatives(),
        keras.metrics.FalseNegatives(),
        keras.metrics.BinaryAccuracy()
    ],
    optimizer=keras.optimizers.Adam(1e-4),
)

The problem is: when I call model.summary(),  then the result would be
Model: "model_9"
__________________________________________________________________________________________________
 Layer (type)                Output Shape                 Param #   Connected to
==================================================================================================
 input_47 (InputLayer)       [(None, 512)]                0         []

 input_48 (InputLayer)       [(None, 512)]                0         []

 input_46 (InputLayer)       [(None, 512)]                0         []

 classifier_5 (Classifier)   (None,)                      0         ['input_47[0][0]',
                                                                     'input_48[0][0]',
                                                                     'input_46[0][0]']

==================================================================================================
Total params: 0 (0.00 Byte)
Trainable params: 0 (0.00 Byte)
Non-trainable params: 0 (0.00 Byte)

The trainable parameter is 0 bytes! I believe that the issue arise when I want to train a model inside a layer is not possible, or even when I try training a model inside a model, which is also not working. I wonder how people fine tune this model? Thank you very much!
Edit 1: Still don't know the solution to this yet, but I guess that the bert implementation of keras is faulty, since the version that I switched to tensorflow_hub.KerasLayer works just fine
Edit 2: Thanks to @Keval Pandya, the code is the solution. The problem lies in the import (not inside the context given by the original question):
import keras
# or
from tensorflow import keras

works fine, but
import tf_keras as keras # Error

shall produce the error.
My version of package is:
keras                              3.4.1
keras-nlp                          0.15.1
tf_keras                           2.17.0
tensorflow                         2.17.0
tensorflow-hub                     0.16.1
tensorflow-text                    2.17.0
keras-nlp                          0.15.1
tf_keras                           2.17.0

The issue you are facing is how Keras layers and models interact when you embed one model inside another. When you use BertClassifier.from_preset(), it returns a pre-built Keras model. When you try to use that model inside a custom layer like Classifier, it may not correctly propagate the trainable parameters unless you explicitly set it up.
=> Your model has Total params: 0, meaning the pre-trained BERT model is not recognized as trainable.
=> This likely happens because the model inside the Classifier layer doesn't register the parameters correctly.
Keras treats your custom Classifier class as a plain layer, and if you don't explicitly mark the inner layers (like BERT) as trainable or don't pass them properly, Keras may not know there are trainable parameters inside.
to solve this issue you have to set the trainable as true you need to modify your code like this one
import keras_nlp
import tensorflow as tf
from tensorflow import keras

class Classifier(keras.layers.Layer):
    def __init__(self):
        super().__init__()
        # Ensure that the BERT model is marked as trainable
        self.classifier = keras_nlp.models.BertClassifier.from_preset(
            "bert_base_en_uncased",
            num_classes=1,
        )
        # Make sure the BERT model's layers are trainable
        self.classifier.trainable = True

    def call(self, inputs):
        out = self.classifier(inputs)
        out = tf.squeeze(out, -1)  # Adjust output shape
        return out

def get_model_2():
    # Define the input dictionary for token_ids, padding_mask, and segment_ids
    inp = {
        'token_ids': keras.Input((512,), dtype=tf.int32, name='token_ids'),
        'padding_mask': keras.Input((512,), dtype=tf.bool, name='padding_mask'),
        'segment_ids': keras.Input((512,), dtype=tf.int32, name='segment_ids')
    }

    # Instantiate the custom classifier layer
    layer = Classifier()

    # Pass inputs through the classifier
    out = layer(inp)

    # Define the final model
    model = keras.Model(inputs=inp, outputs=out)

    return model

# Create the model
model = get_model_2()

# Compile the model with an appropriate optimizer, loss, and metrics
model.compile(
    loss=keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[
        keras.metrics.TruePositives(),
        keras.metrics.FalsePositives(),
        keras.metrics.TrueNegatives(),
        keras.metrics.FalseNegatives(),
        keras.metrics.BinaryAccuracy()
    ],
    optimizer=keras.optimizers.Adam(1e-4),
)

# Check the model summary to see trainable parameters
model.summary()

Here the output of this summary is like this one

Here the tensorflow, Keras, and keras_nlp versions are as below
tensorflow==2.17
keras== 3.4.1
keras_nlp ==0.15.0


text:
Human trainers: collect data by 2-alternative force choice (2AFC)
To collect human intelligence data, we intend to use 2-alternative force choice (2AFC). A sample human intelligence task (HIT) is shown below. Can it be considered a 2AFC? Is such a HIT considered acceptable to collect data from trainers?
Two choices are provided, i.e. A and B. Trainer can pick A, B, or Not sure for multiple metrics. Optionally, the trainer can explain why they picked.
The Not sure choice is provided to avoid polluting the data with some random selections.

2AFC does not allow for a "not sure" option. The forced part strictly means that the user must select one of the two options. The data collection form design cannot be considered 2AFC, because you allow the user to effectively skip answering or say "I don't know".
This doesn't necessarily mean your form design is wrong, just that it is not 2AFC, and to avoid confusion you should not call it that in any documentation or report.

Is such a HIT considered acceptable to collect data from trainers?

That depends on how it will be used. You will need to consider what the "not sure" ground truth data collection means for your model or analysis. You have a few choices:

Add "not sure" to model outputs, making this a 3 class problem.
Model "not sure" as $p(A)=0.5$ in a two class problem. You will want to check that your model supports this, as some optimisers won't work with probabilities even if the underlying maths of the model would support them.
Do not add "not sure" records to the dataset.

Each of these choices will have a different impact on how the model behaves, how you report findings, and how you measure success.

The Not sure choice is provided to avoid polluting the data with some random selections.

This is a good rationale, and does not make the data collection "wrong". However, it prevents it from being 2AFC. If you have specifically been instructed to create a 2AFC data collection, then you should clarify with your manager/tutor/project lead whether that is a strict requirement.

Thanks, I'm studying you answer. It makes sense to me.


text:
Attention module (CBAM) in CNN tend to saturate values to 1
In the context of image classification, I am using a feature extractor based on a resnet-like architecture (ResNet12): four residual blocks, each of which is made of two consecutive conv3x3, batch norm and leaky relu activation function and a final conv3x3, batch norm, residual, leaky relu and maxpool. This architecture is mainly used in the context of few-shot learning, but for this case, consider a simpler image classification task.
I added an attention module (CBAM) to the main architecture. During training I plot a heatmap based on the attention score values retrieved and, over the first iterations, it seems like the feature extractor starts focusing on the correct regions of the image, which means that it should be able to identify the subject (some errors still occur, but consider that it is still the beginning of training).
The problem arises later when the attention "heatmap" starts covering the whole image, including the background and non-relevant parts. By debugging the values I noticed that the scores are all shrunk towards 1. Consider that the output of the CBAM layer is given by a sigmoid activation function, so the values are supposed to be in the [0-1] range (see the attached image).
In the first attempt, the attention module was embedded in every residual block, and then I tried to only use it in only one of them, but the results were conceptually the same. In addition, the attention module receives as input the values that come from the last batch normalization layer, at the end of the residual block.
Have you ever encountered such a problem? Do you have any clue on how could I solve it, or do you know any other attention mechanisms that could fit this feature extractor?


text:
Need Expert Advice for using sklearn pipeline to create a composite estimator with multiple models & features
I am trying to use sklearn pipeline to create a composite estimator. Please check attached image for model blueprint. Can anyone help me understand how this can be done in python.
Column Transformer with Mixed Types
Dataset has only 3 columns including target. [featur_1: categorical, feature_2: text, target]

Code (that didn't work!!)
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import FeatureUnion
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import OrdinalEncoder
from sklearn.linear_model import SGDClassifier
from lightgbm import LGBMClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
from sklearn.pipeline import Pipeline

# Split the data into train and test sets
X_train, X_test, y_train, y_test = train_test_split(df['description'], df['reject'], test_size=0.2, random_state=42)

# Separate transformers for text and categorical data
text_transformer = TfidfVectorizer()
categorical_transformer = OrdinalEncoder()

# Create a FeatureUnion for combining the outputs of different transformers
combined_features = FeatureUnion([
    ('text', text_transformer, 'description'),
    ('categorical', categorical_transformer, [['reject']])
])

# Create an instance of the SGDClassifier
sgd_classifier = SGDClassifier()

# Create a pipeline for the model
model_pipeline = Pipeline([
    ('features', combined_features),
    ('sgd', sgd_classifier),  # Stochastic Gradient Descent Classifier
    ('lgbm', LGBMClassifier())  # LightGBM Classifier
])

# Fit the pipeline on the training data
model_pipeline.fit(X_train, y_train)

# Predict on the test data
predictions = model_pipeline.predict(X_test)

# Evaluate the accuracy
accuracy = accuracy_score(y_test, predictions)
print(f'Accuracy: {accuracy}')

Error:
"name": "TypeError",
"message": "All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'SGDClassifier()' (type <class 'sklearn.linear_model._stochastic_gradient.SGDClassifier'>) doesn't",


text:
How to detect abnormal fetal head size with image classification?
I am a computer science student currently working on my final project, which involves finding a classification-based solution for predicting the head size of a fetus during its third month. Here is a reference for a regression solution utilizing image segmentation with the U-Net model, as illustrated in the Fetal-Head-Circumference-Prediction project.
Unlike that solution, I aim to use classification to detect if the head is of proper size.
My question is what approach is best to split a data set of gray images of fetal heads to classify if one is in abnormal size or not?
Here is a sample database for reference.
My initial thought was to organize the images into age groups, similar to the list at the end of the FetalCns_Final.ipynb file provided in the first link. However, I am uncertain if this is the most suitable approach when dealing with similar images to classify age.

Hi and welcome to our community! I don't see any specific question being asked in your post. Could you please edit your post to explicitly ask a specific question and also put it in the title? If you have more than one question (which seems to be the case), ask the others in separate posts. Note that "any advice or recommendations" is not a question or at least not specific enough. Thanks.

@nbro, I clarified the explanation regarding the question


text:
In VAE, why not maximize D[q(z)||p(z|x)] instead？
In VAE, the goal is to maximize $\log p(x)$, where $x$ is the data and $p(\cdot)$ is a parameterized distribution.
For any distribution $q(z)$, the following identity holds,
$$
\log p(x)=D[q(z)||p(z|x)]+E_{z\sim q(z)}[\log p(x|z)]-D[q(z)||p(z)].
$$
The usual approach is to learn $q_\phi(z)=p(z|x)$ so that $D[q(z)||p(z|x)]\rightarrow0$ and maximize the ELBO $E_{z\sim q(z)}[\log p(x|z)]-D[q(z)||p(z)]$.
My question is that if all we want is maximize $\log p(x)$, why not maximize $D[q(z)||p(z|x)]$ instead?