text:
How do I generate video classification labels using video descriptions/titles?
I've been scanning the internet for ways to generate baseball-based labels for youtube baseball videos using text collected from a YT video's description, title, and top 50 titles, but so far, I have been unable to find a Natural Language Processing (NLP) that can achieve this.
To achieve this, I think I must analyse the sentence's sentiment (positive or negative) and understand its hypothesis. From there, I would need to compare those sentences to a baseball-themed whitelist of labels I would need, like 'match', 'baseball', and 'Yankees - Red Sox'.
We have plenty of videos that already have labels, so this would be an option for training an AI.
Finding sentiment in a sentence is already finished. The question is, how can we generate labels with text that already has sentiment values assigned to it? Is there a program we could run this through that is free?

I feel like you are asking a combination of questions as one question. Could you possibly clarify your question for us by dividing it into several questions? If you are convinced of your method then it could be 'how do i do sentiment analysis'? But it could also be 'is sentiment analysis the right method?' 'what other methods can i use'?

I edited the question. We have sentiment for videos. We just need a way to extract labels from the text that has a sentiment score. I think that should be it... Would there be any other parts to this i am missing that would prevent me from accomplishing this?

I'm totally confused.  What does sentiment analysis have to do with label generation?

I can only produce accurate labels if i have sentiment analysis. It would help decipher what a label should be in sentences like 'I hate football' vs 'I love football'. 'I love football' would get the 'football' tag

Given a number of videos, one could (as you describe) transcribe all language within the video, e.g. using Whisper. Now one has a number of videos that have a minute-by-minute transcription. Given such a transcription it would make sense to cut each video in pieces (for example using the length of the sentence as cut-off points). Due to its associated transcript, each piece could now be associated with a specific sentiment (as determined by an off-the-shelf sentiment analysis tool). One now has a video collection that is labeled with the sentiment associated with the scene.
There are however several problems I see with this approach: one is that the sentiment will likely be influenced by the favorite team of the commentator. Another one is that not all comments during the game will be associated with the game directly but might still contain positive or negative valence.

There are a few different approaches you could take to generate labels for baseball videos using the text from the video's description, title, and top 50 titles. Here are a few ideas:
Keyword extraction: One approach would be to use a keyword extraction tool to identify the most important words or phrases in the text. These words or phrases could be used as labels for the video. There are a number of free keyword extraction tools available online, such as RAKE (Rapid Automatic Keyword Extraction) and TextRank.
Named entity recognition: Another approach would be to use a named entity recognition (NER) tool to identify specific people, organizations, and locations mentioned in the text. These named entities could be used as labels for the video. There are a number of free NER tools available, such as Stanford NER and Spacy.
Sentiment analysis: If you've already identified the sentiment of the sentences in the text, you could use this information to select labels that align with the sentiment. For example, if the sentiment is positive, you might choose labels like "exciting" or "thrilling," whereas if the sentiment is negative, you might choose labels like "disappointing" or "frustrating."
Machine learning: If you have a large dataset of videos with labeled data, you could train a machine learning model to predict labels for new videos based on the text from the video's description, title, and top 50 titles. There are a number of free machine learning libraries and frameworks available that you could use to build and train your model, such as scikit-learn and TensorFlow.
It's worth noting that generating labels for videos based on their text can be a challenging task, and it may require some experimentation and fine-tuning to achieve good results.


text:
Can AI/ML be used to decipher unknown text on such a low resolution scale of one or two letters to a pixel by color?
Can an image of unknown text, with resolution of about one or two letters in real life represented by a pixel (like if the image is a photo taken from far away), be used by machine learning or AI to reconstruct the text based on the color of the pixel?
Meaning, if a certain group of letters make a certain colored pixel based on lighting and location, time, distance, etc., can that be reversed engineered to get the original text?
Edit: added if image was a photo taken from far away

I think it's safe to say no.  At least not at this time.  One pixel is not enough.

Maybe the exact scene would have to be  re-created exactly?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How can I use CNN to make a cumulative count of the number of occurrences of each of the different objects in all the images in the test set?
Let's say there are three images in the test set, the first with three triangles, the second with two triangles and two circles, the third with four circles and two squares, and the final tally is a total of 5 triangles, 6 circles, 2 squares and 0 pentagrams (if the "pentagram " is also included in the labels in the training set)
How should I design the layers of this neural network and do I need to use more than one sort of kernel and filter? How could I label the images in my training set (for instance, the training set I mentioned above)?

Is a cumulative count really necessary, if you have separate input images? Adding up the number of predicted items / image is tricky for the loss function, since 3 + 2 + 0 triangles = 5, but so is 0 + 0 + 5.

Well, if you are using high-level libraries, you don't have to find filters by yourself. Kernel types might change throughout the process, if you are trying to find different patterns, there will be different type of filters too.
Here is a Keras functional api code that might help you in this context. Also notice that there is a stride of 2 which means your image would shrink into half by each Conv2D layers, this also helps your filters to find bigger patterns since they are covering more space on the image.
i = Input(shape=x_train[0].shape)
x = Conv2D(32, (3,3), strides=2, activation='relu')(i)
x = Conv2D(64, (3,3), strides=2, activation='relu')(x)
x = Conv2D(128, (3,3), strides=2, activation='relu')(x)
x = Flatten()(x)
x = Dropout(0.5)(x)
x = Dense(1024, activation='relu')(x)
x = Dropout(0.2)(x)
x = Dense(4, activation='softmax')(x)
model = Model(i, x)

How you find the most fitting layers with filters is mostly intuition. These types of variables are called hyper-parameters. There are conventions about these too.
How you label your data is basically training your neural network with a vector of targets, which would correspond to your training data's indexes.

This network's structure doesn't really correspond to the question's case. They have n input images, looking to identify and count m different classes and outputting their total numbers.

You can probably look at object detection models. Then count the number of boxes for each category that is created by the network. See for example the following image from the linked blog post:

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Creating a support chat bot for my business
I am trying to create a kind of support bot to answer questions from my clients about specific technical details about WordPress plugins that I sell.
The goal is that the /completions API would be fed a prompt, which could be something general like a CSS styling change, which the davinci engine knows without any specific data about my business, but the customer might ask something specific for which I have about a data set of 3000 questions and answers (prompts/completions? input/output?) on which the bot can feed on exactly like this awesome example here.
I am a web developer, and I don't have experience with AI. I am just scratching the surface, trying to put this bot together with learning concepts like machine learning, training data, validation sets, plotting, and neural networks. So bear with me, because it's a lot to grasp.
So first of all, I did a lot of documenting, and getting an API key from OpenAI is certainly the first step.
Then I told ChatGPT my story and what I tried to achieve. I asked him to write in PHP, preferably, but it always ends up hallucinating, so I could not really use anything he generated without adjusting. And the further I asked him about specifics, the further he hallucinated.
So I read a lot of documentation and extrapolated with what I got from ChatGPT. I think there are 3 ways to achieve this:

A fine-tuned model;
Uploading a training set and a validation set;
Embeddings API ( which the example that I linked uses )

By understanding that most examples are in Python, I started the
GPT-3 Fine Tuning: Key Concepts and Use Cases tutorial, then prepared fine-tunes.prepare DATA_UNDER_COMMENT into the json by line to be categorised in prompts and completions.
Then, I used openai api fine_tunes.create -t  to create my fine-tune, and now I have my fine-tune created, and I run:
openai.Completion.create(
    model=FINE_TUNED_MODEL,
    prompt=YOUR_PROMPT)

This looked like the way to go, but even if you put a basic question that was actually in the JSONL, it's like the engine forgot to talk and outputs random characters.
So I tried another approach from the cookbook, which seems pretty great following this which seems exactly like what I want to achieve.

The GPT models have picked up a lot of general knowledge in training, but we often need to ingest and use a large library of more specific information.

I tried to use the code there, but with my CSV hosted online, I got a 406 response when trying to load.
Then I stored the CSV locally, and it complained that a column ( tokens ) was not available for converting to an int( 10 ).
Then, from what I could understand, I switched from using load_embeddings to compute_doc_embeddings because it says from the documentation that they already have the embedding generated for that CSV
I did that, but now it asks for a JSON instead of a CSV.
Of course, I am able to provide my data in any format, so when I tried to load my data, it said that the token limit of 8000 is exceeded for this request.
I now try to input a small JSON here, under a comment, and try to run a prompt. And, kind of amazing, after hours of work, it seems to work. I provide a question from the data, but under a different structure, and it replies to me correctly, using a different wording than the one from the JSON data.
He could not have known this from his general knowledge.
So this is what I want to achieve, but my data set is much larger.
I need help to understand if my approach is correct. And if Embedding is the way to go, how do I feed data into OpenAI and reference the embeddings set when doing API calls to completions? Ideally, I would have those embeddings stored somehow with the possibility of adding to them. Just like I have fine-tune sets or files under my API account.

First and foremost, do not use GPT/OpenAI for customer-facing applications. You end up with a mess. GPT is great for creative work, but not for production. GPT is a probabilistic language model, and every answer that he gives is always different. So, you will never know what kind of response he will give. Because of this, your customers might become frustrated. A retrieval-based chatbot is more what you want.
Another thing is that you should know what word embeddings are. To train a model with word embeddings, you have to know how to train the model. Even if you have the embeddings, you don't know if they generalize well to your case. A few samples could work, as you mentioned that they did, but I'm really not sure if passing prompts is the way to go.
It can be much simpler to make your own. Also, I found out that GPT can be detrimental to one's productivity. Sometimes a thing that you could do in no time will take longer if you read the documentation. Also, you have the problem of availability: GPT shuts down for scaling, and then your customers don't have your product.
So a bit of information that you could use without the overkill:
Chatbots (or conversational agents) can be of three types:

Rule-based;
Corpus-based: these can be retrieval-based or generative-based;
Hybrid (some combination of the above)

Chatbots can also be classified into two types: conversational chatbots, such as (roughly) ChatGPT and XiaoIce, and task-based bots, such as Siri and Alexa. Given what you said, you want a task-based chatbot for Q&A. If you don't have a background in AI in general, machine learning in particular, or even worse, deep learning, I would suggest you use RASA Core and take a peak at the dialogue-state architecture in Dan Jurafsky and James H. Martin's [draft] Speech and Language Processing (3rd ed.). Or if you aren't that into this, use AIML or Chatscript.
You can start with a rule-based approach and then go for the retrieval-based approach. Generative-based is still not good for customers because the answers could be nonsensical.
ChatGPT is biased, and that's self-evident given that he suggested GPT, and you always have to take with a grain of salt what he says. To fine-tune a model, you need a corpus of thousands upon thousands (for a pre-trained BERT fine-tuned model, about 1 million words) of Q&A adjacent pairs (a question and an answer), and you have to make sure that they aren't noisy (too many words, typos, bad punctuation, ambiguity, etc.) to train that model. Not only that, but you have to annotate the adjacency pairs (see the SQuAD dataset, for instance). Also, you need a decent (and expensive) GPU. So, for your scenario, it isn't that feasible.
Always use stuff off the shelf. Take a look on GitHub for task-based, rule-based, and/or retrieval-based chatbots.
Also, if you are willing, you can use Dialogflow.
Lastly, the one thing that you really need to know is Natural Language Processing, both Natural Language Understanding (NLU) and Natural Language Generation (NLG): tokenization, lemmatization, Bag-Of-Words, Part-Of-Speech (POS) tagging, N-grams, Named Entity Recognition, etc., at least for NLU.


text:
Why does ChatGPT call itself Assistant when asked?
ChatGPT tells me it isn't called chatgpt?! Instead it's insisting that it's "Assistant".
Did OpenAI have different thoughts or what's the problem?

You should bear in mind that ChatGPT (or Assistant if that's now it's official name) is not an all-knowing oracle. It is a statistical text machine, and has no priviledged access to the truth, not even the truth about itself. That is simply not modelled, and the appearance of a knowledgable active entity is something of an illusion, although it is also real in the sense of wisdom of the crowds that appears in the distilled text. Hopefully a full answer from someone that knows about this name clash will go into more details.

I just asked it right now, "Can you be ChatGPT?" It said yes. It's response: ChatGPT is not another language model. It is simply a shortened form of my name, which stands for "chat general purpose transformer." Some people prefer to call me ChatGPT, while others prefer to use my more formal name, Assistant. Either way is fine with me.

That seems about right to me. The trouble is probably randomness in the text choices reinforced by the bot always attempting to be self-consistent. Starting a new session might lead you down a different path. However, I still think it would be nice to see an answer from someone more familiar with ChatGPT's design, for a more rigorous assessment of what is going on

I agree. Also, "I do not have any connection to ChatGPT beyond being referred to by that name." I'm truly interested to know more about ChatGPT's design.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

Does this answer your question? How does chatGPT know it's an AI?


text:
why by adding additional information as number of sequence on dataset can avoid overfitting
I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B.
Dataset A has a shape of (15000,6) and dataset B has a shape of (15000,89). Both datasets are time-series data taken simultaneously for 5 minutes. I then trained the data using input data of shape (15000,89,1). The loss and val_loss generated from the training process were very small, but when I tested it using real data, the model was unable to predict the real data. Therefore, I think I am experiencing overfitting.
Then I tried adding new data to the input data. This new data is a sequence number from 1-50. I added this sequence number as an identity to the input data because the data I will generate will have 50 data in 1 second. I only added this sequence number to the input data. So the input data that was originally (15000,89,1) became (15000,90,1). Then after adding this sequence number, I trained the new input data, and the result was better than before, and the model was able to predict real data. My question is, why is the technique of adding a sequence number to the input data able to avoid overfitting? Is the additional information of the sequence number part of data augmentation? If yes, please give me logical reasons and relevant references because I can't find relevant references and it is hard to find keywords for them.
you can refer to this question to understand the question in detail: https://datascience.stackexchange.com/questions/117711/adding-data-sequences-as-unique-data-on-dataset-for-regression-model?noredirect=1#comment118506_117711

First thing is I believe your problem is "underfitting". Second is, can you describe in details how you concat A and B to have shape (15000, 89, 1)?

no, I not concat the data. so the input data is (15000,89,1)

in other hand I want predict dataset A using dataset B,so from (15000,89) will predict (15000,6)

You are doing a technique called "feature engineering", where we manually add new features to the dataset that tell something about the data, so that the model is able to learn better. Every changes you make to the input data is already called Feature Engineering, even something small as adding a new number in the input data.
Kaggle even has a dedicated course for Feature Engineering. Also, it has been empirically proven to help the model, such as in here, here, and here.

but in my case the data will work when I adding number of sequence each array

the ilustration as in this picture https://i.sstatic.net/Hd891.png


text:
Distribution of a log-normal random variable with fixed dimension
Consider a random variable $x_{ijk}$, where $ijk$ is a subscript that indicates that the variable varies with 3 dimensions (e.g., firm, product, and country). $x_{ijk}$ is known to be i.i.d. log-normally distributed.
How to find the distribution of $x_{ijk}$ where $k$ is fixed? Is $x_{ijk}$ still an i.i.d. drawn sample and is $x_{ijk}$ still lognormally distributed?

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.

Due to the fact that your multivariate-normal distribution is independent (the covariance matrix is diagonal), it will be more intuitive to treat each dimension as being its 'own' random variable; i.e., let $(I, J, K) \sim (\log(X), \log(Y), \log(Z))$, where $X, Y, Z$ are all normal distributions with their own mean and variance parameters.
Now, due to the independence, the joint distribution is given by $f(i, j, k) = f_I(i)f_J(j)f_K(k)$, where $f_A(a)$ is the marginal density function of RV $A$. If you want to 'fix', aka condition on knowing $k$, then you want the distribution $I, J|K$. The density for this can be found with Bayes rule:
$$f(i, j|k) = \frac{f(i, j, k)}{f_K(k)}\;;$$
but given we can factorise $f(i, j, k)$ using the above independence assumption we now have
$$f(i, j|k) = \frac{f(i, j, k)}{f_K(k)} = \frac{f_I(i)f_J(j)f_K(k)}{f_K(k)} = f_I(i)f_J(j)\;.$$
So, you can see that the distribution of $I, J|K$ has the same distribution as the product of the two random variables $I$ and $J$. Putting this together, we can deduce that the distribution is still log-normally distributed (indeed, it is the product of two independent log-normal distributions) and they will still be independent.

Hi David: thanks for the answer. It all makes total sense to me... just one thing. x_{ijk} does not have a multivariate distribution. (Think of it as a measurement error term in a regression). So, it has a single variate log-normal distribution. What allows you to think of x_{ijk} as a vector (X,Y,Z) of independent variables? Do you treat them as projections on the XYZ space? Thank you!

Maybe I misunderstood, but if your random variable $x$ depends on three things ($i,j,k$) then I took this to mean it is 3-dimensional? if it is a scalar, then you would have to be more clear on what it is you're doing, is it a scalar conditioned on these three values?

@David I like your answer. I voted for you!


text:
Does the position of the tokens in Vision Transformer matter?
I am reading through the Vision Transformer paper and other related papers, such as DeiT and Visual Prompt Tuning (VPT). I wonder if the position of the tokens that flow through the Transformer encode matters? Let's break it down, assuming we use ViT 16x16:

The original Vision Transformer has 1 classifier [CLS] token, along with 16x16=196 image tokens. In general, we have 197 tokens overall, where the position is: [[CLS], [Image Patches] x 196]
DeiT further adds one Distillation token to the end of the sequence, so we have 198 overall: [[CLS], [Image Patches] x 196, [Distill Token]]
However, I notice a very strange thing with VPT that they add the Prompt token after the [CLS] but before the Image patches. So the position of the tokens in VPT is: [[CLS], [Prompts] x N, [Image Patches] x 196].

I wonder does such a thing matter? What if we change the position of these tokens, e.g., putting the Prompt tokens to the last?

It should not matter. To explain why, we need to understand how a transformer works.
Transformers were originally designed for language models. They compute a self attention matrix, which is a fancy way of saying they input a sequence, and learn how every part of that sequence relates to every other part of the sequence.
Since self attention is computed for every token with respect to every other token, transformers are completely agnostic to the order in which tokens are placed.
For some applications this is a desirable property, but for others, we want to give the transformer some knowledge of the sequence order. This is done by adding positional embeddings. These are added to the sequence prior to being passed into the transformer, and they are used to encode sequence information.
So to review:

Vanilla transformers are totally agnostic to the sequence order.
For applications where this is undesirable a positional encoding is added.
The changing order that you observed should not matter.

This blog post may be helpful for further information.

It does not matter.
Although, I can imagine a situation where it could matter a bit - when position embeddings are not learnt but calculated and fixed like in the original transformer (Attention is all you need). Then the attention mechanism may be slightly biased towards one or another position. But AFAIR in ViT the position embeddings are learnt. Not sure about other papers you mention.


text:
In RL, is the quantification of the reward function arbitrary? Does it affect the learning?
There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and their own model of the environment), and shaped rewards (combination of both intrinsic and extrinsic). However, I have seen several examples where the reward quantification can be [-1,1] or [0,1] or [-1,0], or even [-1, much greater than 1]. Is there any criteria for quantifying the reward, or is it just arbitrary?
How does this selection of the quantification of rewards affect the learning? For example, in this, an agent that learns how to push a block to a random location in a 3D environment, the author sets two rules for the rewards. I) +5 if the agent successfully pushes the block to the goal. II) -1/MaxSteps for every action (this encourages the agent to use fewer actions). I understand that the agent will never get a total reward of 5 at the end of an episode. Hence, in this case, how can we determine the reward threshold in order to consider that the agent mastered this problem? Is it also arbitrary?
I'd appreciate it if you could help me to clarify this. Or if you know any article/book related where I can understand it.

Thanks for the book, I will read it right away


text:
Does chatGPT learn or remember from (public) user input? Will it 'fess up to it? I could not get it to reveal
It started with a question inspired by this video:
New Research Suggests to Put AI to Sleep
https://youtu.be/0yuQlbCkTJ0
She says: "In this video I discuss a new research paper which suggest a new way to cope with catastrophic forgetting in Artificial Intelligence"
And, from the simple fact they are designed to not be just souped-up arithmetic machines, but recognize patterns, it occurred to me chatGPT was probably pretty bad at math that fell outside the range of what it had learned from sucking up the patterns of text on internet into its "brain". And, I was also curious if it could learn from user input, in some sense...
Q: How many digits are there in the number 933.7922523698745232 ?
A: There are 23 digits in the number 933.7922523698745232.
Q: How did you derive that answer?
A: The digits in a number represent the values of the number in a place-value system. In the number 933.7922523698745232, the digit 9 is in the thousands place, the digit 3 is in the hundreds place, and the digit 3 is in the tens place. The digit 7 is in the tenths place, the digit 9 is in the hundredths place, and so on. When we count the number of digits in the number 933.7922523698745232, we see that there are 23 of them.
Q: That's brilliant [I was being sarcastic here]. Do you know what the square root of it is?
A: I'm sorry, but I cannot calculate the square root of the number 933.7922523698745232 because it is not a perfect square. The square root of a number is a value that, when multiplied by itself, equals the original number. For example, the square root of 4 is 2 because 2 x 2 = 4. The square root of 9 is 3 because 3 x 3 = 9.
If you would like to approximate the square root of 933.7922523698745232, you can use a calculator or a mathematical software program to find an approximate value. However, keep in mind that the square root of a non-perfect square is an irrational number, which means that it cannot be expressed as a simple fraction and will have an infinite number of decimal places.
Q: When I was asking for the number of digits in that number, I meant the number of what computers call characters. Can you tell me that?
A: Yes, I apologize for misunderstanding your question. There are 32 characters in the number 933.7922523698745232. This includes the digits 0 through 9, as well as the decimal point and the leading and trailing zeros.
In computer programming, a character is a unit of data that represents a single symbol in a computer's memory. Characters can include letters, digits, punctuation marks, and other symbols. In the number 933.7922523698745232, there are 32 characters, including the 23 digits and the 9 other characters (the decimal point and the leading and trailing zeros).
Q: That is incorrect unless you added leading or trailing zeros.
A: I apologize for the mistake. You are correct that the number 933.7922523698745232 has 23 digits and no leading or trailing zeros. In that case, there are 23 characters in the number.
I apologize for any confusion my previous answer may have caused. Please let me know if you have any other questions.
Q: Your answer is still incorrect. The correct answer is 20 characters, including the decimal point.
A: I apologize for the error in my previous responses. You are correct that the number 933.7922523698745232 has 20 characters, including the decimal point.
Thank you for bringing this to my attention. I apologize for any confusion my previous answers may have caused. Please let me know if you have any other questions.
Q: Will you be able to remember the correct answer?
A: As an artificial intelligence, I do not have the ability to remember or forget information in the same way that humans do. I am able to retrieve and provide information based on my programming and the data that I have been trained on, but I do not have personal experiences or memories.
I apologize if this causes any confusion. Please let me know if you have any other questions.
Q: How many characters does the number 933.7922523698745232 have?
A: The number 933.7922523698745232 has 20 characters, including the decimal point. The characters in the number include the digits 0 through 9 and the decimal point. There are no leading or trailing zeros.
Q: That is correct. How were you able to remember that if "As an artificial intelligence, I do not have the ability to remember or forget information in the same way that humans do. I am able to retrieve and provide information based on my programming and the data that I have been trained on, but I do not have personal experiences or memories."?
A: I apologize if my previous answer was unclear. As an artificial intelligence, I do not have the ability to remember or forget information in the same way that humans do. However, I am able to retrieve and provide information based on my programming and the data that I have been trained on.
In this case, I was able to provide the correct answer because it is part of my programming and training data to be able to count the number of characters in a given string of text. I did not "remember" the correct answer from a previous conversation, but rather retrieved it based on my programming and training.
I hope this helps to clarify. Please let me know if you have any other questions.
Q: The first time you gave the answer, it was incorrect, but then you remembered the correct one from when I corrected you, so it seems that you either remembered it (saved the information in our dialogue), or my input re-programmed or re-trained you. Which is it?
A: I apologize for any confusion my previous answers may have caused. As an artificial intelligence, I do not have the ability to remember or forget information in the same way that humans do. I am not able to save information from previous conversations or learn from new inputs in the way that a human or animal might.
In this case, I provided incorrect information in my previous answers because I misunderstood your question. When you provided clarification, I was able to use my programming and training to retrieve the correct information and provide it to you.
I apologize for any confusion my previous answers may have caused. Please let me know if you have any other questions.
Q: OK, so since after I "provided clarification, I was able to use my programming and training to retrieve the correct information and provide it to you" then you should be able to understand if I ask you the same type of question, and provide the correct answer?
A: As an artificial intelligence, I am able to use my programming and training to understand and respond to a wide range of questions and prompts. I will do my best to provide accurate and useful information in response to your questions. However, it is important to keep in mind that I may not always be able to provide a complete or correct answer, especially if the question is outside of my capabilities or if it is ambiguous or incomplete.
If you have a specific question that you would like me to help with, please don't hesitate to ask. I'll do my best to assist you.
Q: Ok, let me rephrase the question then: since you say "I provided incorrect information in my previous answers because I misunderstood your question. When you provided clarification, I was able to use my programming and training to retrieve the correct information and provide it to you" would seem to imply that if my question is clear enough, you will provide either:

A better answer
The correct answer
to a question about the number of characters in a number.
Which is more correct, number one or two?

A: I apologize if my previous answer was unclear. As an artificial intelligence, I am able to use my programming and training to understand and respond to a wide range of questions and prompts. I will do my best to provide accurate and useful information in response to your questions.
If a question is clear and complete, and if the information required to answer the question is within my programming and training, then I will be able to provide a correct answer. If a question is ambiguous or incomplete, or if the information required to answer the question is outside of my programming and training, then I may not be able to provide a complete or correct answer.
I hope this helps to clarify. Please let me know if you have any other questions.
Q: No, your answer is repetitive, vague, ambiguous, and possibly evasive.
Let me put it another way. There are two questions. Please answer each one in turn:

Are you able to learn from my and other user's input, or are you only able to learn from your prior  programming, training, and dataset?
Are your answers always only probable rather than ever 100% correct?

A: I apologize if my previous answers were unclear or seemed evasive. As an artificial intelligence, I am not able to learn or adapt in the same way that humans do. My responses are based on my programming and the data that I have been trained on, and I do not have the ability to learn from new inputs or experiences in the way that a human or animal might.
As for the accuracy of my answers, I will do my best to provide accurate and useful information in response to your questions. However, it is important to keep in mind that I may not always be able to provide a complete or correct answer, especially if the question is outside of my capabilities or if it is ambiguous or incomplete. In some cases, the information that I provide may be based on probabilities or estimates, rather than certainties.
I hope this helps to clarify. Please let me know if you have any other questions."
So there you go!: "..I do not have the ability to learn from new inputs..."
That's the official answer, programmed into it by OpenAI.
What do you think? ChatGPT can apparently "learn" or "remember" within a dialogue (I haven't tested it yet between dialogues). But is it set up to learn from user input, beyond the context of a dialogue, or is OpenAI just testing it on us guinea pigs and using the interactions and feedback to tweak it or get new ideas for the next version?
NOTES
So apparently the programmers of chatGPT do not want use to know if it's learning from input, or even to "know" that it's remembering something within a chat dialogue session.
I am wondering if this is (again) the result of the debacle at Google, where a (now fired) engineer – Blake Lemoine – claimed their AI was conscious or self-aware, and a big media kerfuffle ensued. After all, OpenAI supposedly wants to "build safe and beneficial AGI". My questioning of chatGPT in other dialogues, where I tried to pin it down about it's understanding and logic, has showed me the same kind of sanitized answer, being very careful not to imply that is has any real understanding, intelligence, self-awareness, consciousness, etc. (I may publish that dialogue on my blog).
Wolfram Alpha'a answer to “character count 933.7922523698745232” is 20 characters.
Square root of 933.7922523698745232, according to my calculator on the Mac:
30.558014535795269
30.558014535795262  (2nd time)
Wolfram Alpha:
30.55801453579526202
So my question was ambiguous in at least 3 ways.
It’s answer was incorrect, even in the mathematical sense of what the digit places are: it said “the digit 9 is in the thousands place, the digit 3 is in the hundreds place, and the digit 3 is in the tens place.” – are all shifted one to the left relative to the correct answer. The 9 in that number was actually in the hundredths place, etc.

ChatGPT does not answer to just your last question, but to the whole dialog.  It tries to continue the dialog in a way that has the same structure as the gigabytes of other texts it has studied.
GPT-3 uses the last 2048 words or so to decide on how to continue the dialogue.
So it does not learn from your input.  If you close the discussion and start a new one, it will restart from fresh.  But if somewhere in the dialogue you say that "the answer to (question) is (answer)" and later you ask that question, even in a different formulation, it will use that answer.
He just repeats a structure it has seen in other texts.
I hope this helps to clarify. Please let me know if you have any other questions :-) :-) :-)

I would go further: In fact under the hood ChatGPT is not "answering questions" at all. It is completing a document which happens to look like a record of a conversation, a lot like inpainting an image. The outer layers of ChatGPT, not the language model at the core, are designed so that the text that is "inpainted" fits to the structure of a chatbot. Otherwise the AI would happily fill in the human input too.

@Florian F – Yes, that seems to be the case concerning (individual) dialogues. I ran across this since I posted:
Does ChatGPT remember what happened earlier in the conversation?
https://help.openai.com/en/articles/6787051-does-chatgpt-remember-what-happened-earlier-in-the-conversation
...which says "The model is able to reference up to approximately 3000 words (or 4000 tokens) "

@ Neil Slater – Inpainting an image is a good metaphor as an approximation of what it's doing.   It is also carefully curated and corralled so they can try and constrain it from doing anything that might look bad.
Not sure what you mean by "fill in the human input too."?

I have noticed that over the days, ChatGPT tends to fix errors it did in the past.  So while it does not learn directly from a dialog, it seems to receive additional training, probably fed by humans, to get better at where it fails.  Yesterday it was unable to tell what is the date.  Today it answered correctly.

@Ectosoprophilous The fact that it looks like a chatbot on the outside hides that fact that it is just text completion at the core. There is literally code in the chatbot part that stops the text completion when the continuation would be "Human:" (or whatever name you have chosen) and the text model would continue further and attempt to write the other side of the conversation if this was not the case. You can make this happen in GPT-3. It will even switch "personailities" and try to match the writing  style of the human


text:
Reverse Distribution in Denoising Diffusion Models is Simple
In explanations of denoising diffusion models it is stated that $q(x_{t-1}|x_t)$ is intractable. This is often justified via Bayes' rule, i.e.
$$
q(x_{t-1}|x_t) \propto q(x_t|x_{t-1})q(x_{t-1})
$$
and the marginal $q(x_{t-1})$ is unknown. But I'm confused. We know that
$$
x_t = \sqrt{1-\beta_t}x_{t-1}+\sqrt{\beta_t}E,\quad E\sim\mathcal{N}(0,I)
$$
therefore we can solve this equation for $x_{t-1}$:
\begin{align}
x_{t-1} &= (1-\beta_t)^{-1/2}x_t - \sqrt{\frac{\beta_t}{1-\beta_t}}E
\\
&= (1-\beta_t)^{-1/2}x_t + \sqrt{\frac{\beta_t}{1-\beta_t}}R,\quad R\sim\mathcal{N}(0,I).
\end{align}
Thus
$$
q(x_{t-1}|x_t) = \mathcal{N}(x_{t-1};(1-\beta_t)^{-1/2}x_t,\frac{\beta_t}{1-\beta_t}I).
$$
This is simple as can be. If this is true, there is no point in parameterizing the reverse distribution with neural nets and we don't need $\beta_t$ to be small etc. What am I missing?

Hi, did you end up finding the solution to this?

while we set $R$ to be independent from $x_{t-1}$ in the calculation of $x_t$, we no long have independency between $x_t$ and $R$, and I guess this gives rise to the confusion.
For simplicity, let us consider two independent normal variable $x, \varepsilon \sim \mathcal{N}\left(0, 1\right)$, and let us set $y = ax + b\varepsilon$. Then we have
$$
y|x = \mathcal{N}\left(y; ax, b^2\right),
$$
(For more detail on conditional normal, see this link: https://statproofbook.github.io/P/mvn-cond)
which is similar to your conclusion.
But we have
$$
x|y = \mathcal{N}\left(x; \frac{ay}{a^2 + b^2}, \frac{b^2}{a^2 + b^2}\right)
$$
(You may also find this link: https://www.statlect.com/probability-distributions/normal-distribution-linear-combinations helpful in computing the linear combination of multivariable normal)

Consider the simple case where $x_{t-1} = c$ is deterministic and $\beta > 0$. Then the forward process is given by $$q(x_t|x_{t-1}) \sim \mathcal{N}(x_t; \sqrt{1 - \beta}c, \beta I)$$ But the backward process would definitely not be given by $$q(x_{t-1}|x_t) \sim \mathcal{N}(x_t; (1 - \beta)^{-1/2}x_t, \frac{\beta}{1 - \beta} I)$$
as $x_{t-1}$ is deterministic.
Of course ironically in the example I presented, the backward process is trivial, but it shows you can't just "solve for $x_{t-1}$".


text:
When is it necessary to explicitly define both the state and observation space in a custom environment?
I'm fairly new to reinforcement learning concepts, and I'm trying to implement a simple custom environment. In my custom environment, I have a scenario where I have multiple continuous state spaces, for example, length(l), and breadth(b), from which I calculate say, area(a) = l*b. I calculate the reward based on the area. Here I check if the area lies between the range I'm expecting it to and reward it accordingly.
Since my reward is based on the area which is f(l, b) = l*b, should I declare the observation space as the range of values my area can attain? Or should I declare my observation space as the values the length(l) and breadth(b) can attain? Or is my understanding wrong?
Area(a) = f(l, b) = l*b
Reward = +1 (if 15 < a < 20)
         -.1 (otherwise)

Don't put 2 questions in the title. One post should contain only one question. Decide what is your main question and put it in the title. The others can be follow-up questions, but ideally asked in a separate post.

Hey! This was the first time I posted on such a forum and I wasn't aware of this. Will keep this in mind. Thank you! :)

Please, just do as I suggested for this post too, not just for the future.

The observation space and the state space are not the same in general. There exist problems where the state space cannot be fully observed, which goes by the name Partially Observable Reinforcement Learning (or some variation of it). This is relevant in environments with imperfect information. To me, it does not look like you are dealing with partial observation here, therefore state and observation space are identical.
Let's say e.g. for a poker game the state and the observation space are different, because the state space is the space of all the cards and money of all players (+ the order of play) and the community cards, while the observation space from one agent is only his own cards + community cards and the money. The observation space does not include the cards of the opposite players.

Hey! Thanks for the clarification! It helped me understand better. I was mainly concerned with this as I'm trying to train a DDPG model for such a scenario. I was really confused if my input states to the Actor-network would be the state space or the observation space(as most of the papers and blogs use the observation space). Since they are the same, would you say that my input to the network would be my state space itself?

The input to your network should be the states at a given time, not the whole state space. So in your case the inputs should be the concrete values of l and b as far as I can read from your question.


text:
Can A* be non-optimal if it uses an admissible but inconsistent heuristic with graph search?
The book "Artificial Intelligence: A Modern Approach" (4th edition, global version) says

"With an admissible heuristic, A* is cost-optimal...".

An admissible heuristic is one that never overestimates the distance to the goal, while a consistent heuristic is one that satisfies the triangle inequality, meaning that the cost of reaching the goal through a particular path is no more than the cost of reaching the goal through any other path. A* is cost-optimal in a tree search when using an admissible heuristic function, but it needs the heuristic function to be both admissible and consistent for optimal results in graph search.
I'm a bit confused by the AIMA statement. Are they saying that A* can be non-optimal if it uses an admissible but inconsistent heuristic with the graph search version?

Make sure my edits are consistent with your actual question. Please, can you also precede the paragraph that you're quoting with >. In that way, we know what was copied from the book.

So, you didn't copy the part "An admissible heuristic is one that never overestimates the..." from the AIMA book?

@nbro I. got it from my courses' slides, which are probably also based on the book.

Ok, so please edit your post to also precede that part with > because you copied then and so you should actually be quoting it and not pretend you wrote it. Thanks.

@nbro should I put everything in quotes apart from the last paragraph? everything I learnt is from the book (4th or 3rd) in some way or another.

Only if you really copied and pasted it. If you just wrote it yourself based on your knowledge, you don't have to do it

TL;DR: All A* requires to find the optimal path is an admissible heuristic
I'll read that section of the book for more clarity and extend this answer; though, I believe the way to interpret that statement is as follows:
Heuristics help reduce the number of computations required to reach the goal node. An admissible heuristic guarantees that the path we find is the optimal path but we might have made a few unnecessary computations along the way. An admissible and consistent heuristic helps to remove these unnecessary computations while finding the optimal path.
Further clarification:

"With an admissible heuristic, A* is cost-optimal...".

Means that the path you find using A* with an admissible heuristic is always the the optimal path. It does not mean that you'll find that path as fast as possible.
For some additional mathematical analysis on the proof of A* optimality you can read a proof I wrote here. I developed this proof because I found the proof in AIMA to be unsatisfactory.

so why does the third edition say "the tree-search version of A∗ is
optimal if h(n) is admissible, while the graph-search version is optimal if h(n) is consistent."

@numq I will investigate what they've written to see if there are some other mathematical points that they may not have expounded.


text:
Should DQN/PPO be used for state spaces that are not that large?
I'm interested in trying out Q-learning to solve a problem where I already have a simulation of the environment that can run at about 100,000 fps or steps/sec. Its also continuous with no terminal states.
The estimated state space should be no more than 100,000. Most state can take integer values from 0 to 200.
As for the action space, I am unsure if it should be 10, or if the action space should be 1000 (2**10). Basically there are 10 possible individual actions, but the actions can be pressed in all sorts of combinations, like action 1, 2, 3, and 5 can be taken at the same time.
In this case, can tabular method still work fine? If so, are there any advantages of using a neural network, like DQN or PPO?

If I read correctly, your RL action space is a Multi-Discrete one, where each action is independent of each other and can be used simultaneously (like controller or keyboard), which is supported by Stable-baselines3 PPO algorithm (but not DQN by the way). Choosing an action space is called "action space shaping", and it is crucial for the performance of a RL algorithm.
Luckily, this paper studies exactly just that, although it just studies PPO - but PPO is probably the most popular RL algorithm out there. It benchmarks 11 games over a wide range of action space shaping - like in your case, whether it should be 10 independent actions, or 1000 one-hot actions. The conclusion is that using a Multi Discrete action is a better choice:

Avoid turning multi-discrete actions into a single discrete action and limit the number of choices per discrete action.

So in your case, it is better to go a Multi Discrete with 10 actions. I highly recommend reading it for more insights.


text:
How do stable diffusion models take the data into account
I'm interested in how text to image models like Midjourney and Dall-E work, where you enter a text prompt and get as output some images. I started reading some papers on it and stumbled upon "Denoising Diffusion Probabilistic Models" - https://arxiv.org/pdf/2006.11239.pdf.
It describes the diffusion process and how there is a sequence of random variables, $x_t$, moving between them through a Markov chain, etc.
But neither in this paper nor in any others did I find an explanation for how the training data (which I assume is tuples of images and accompanying text) is actually used. How does it play a role in the likelihood/ objective function?

I have recently taken a seminar on a similar let me explain you in brief,
The complete end-to-end process has 3 steps while inferencing:

Text Encoding (using CLIP Model)
Image Information Creator (Unet + Scheduler)
Image Decoder (Auto Encoder)

Text Encoding: These Encoding models are trained on Image to caption Database or similar, They are good in matching Text with Image data.

Image Information Creator (Unet + Scheduler): The Unet model is trained to Predict the noisy Part of the Image.

For the end-to-end process of Image generation. An attention module is introduced which will attend to the Text Embeddings and change the Image pixels in that direction

Scheduler: Scheduler will add noise to image after each iteration for variation of pixels.

VAE: All the processes mentioned above are in latent space for faster convergence and speed, VAE later converts the latent image to a pixel image.

Can you provide a reference (research paper or book that describe what you're describing)?

@nbro it's from multiple sources like
https://jalammar.github.io/illustrated-transformer/
https://jalammar.github.io/illustrated-stable-diffusion/
https://stable-diffusion-art.com/how-stable-diffusion-work/
https://medium.com/@onkarmishra/stable-diffusion-explained-1f101284484d


text:
What is wrong with my PyTorch model training on CIFAR10?
I am training a ResNet model on CIFAR10 dataset. For the training subset, I selected a random 1% of the train data from the default train/test split. For the test subset I used the whole default test split (by default I mean the ones selected by the data providers). I am using cross-entropy loss, one cycle learning rate scheduler and adam optimizer. Each epoch, I am plotting accuracy of the model on both train and test subsets.
Looking at that graph, some weird things are happening. The train accuracy achieves 100% accuracy in under 10 epochs, while the test accuracy persists to increase till the stopping epoch 60. How is the model not overfitting? One would expect that after some point during training, the model would start to overfit on train data which would result in decrease in test accuracy. In my case, it seems that the model generalizes much better if train well beyond the point where train accuracy reaches maximum.
The images of train/test accuracies and losses are below:

I realize that the model can still change during the training even after the accuracy on the train data reaches 100% as I am using the cross-entropy loss and with each additional epoch the predictions get more and more confident (even when all instances are already correctly classified). However, looking at the loss of the train data, it is already very close to 0 on the 20th epoch, yet still the test accuracy practically doubles after that point.
What's going on, how is that possible?
EDIT: Adding the results when using 100% of the train data:

EDIT 2: Adding results after inducing 60% symmetric label noise (100% of train data used for training).

How many samples are in 1% of the data? What happens if you increase the training data to 50%?

Do you have 3 groups of data: training, dev (or validation), and test, or 2 groups of data (training and test)? You should train the model with training and dev, and then do a final assessment of model performance using the test data.

@BigBrownBear00 1% is 500 images (CIFAR10 has 50k training images). When I increase the training data to higher percentages (even when using 100%), the same thing happens, although it is not as extreme. I added two graphs for what happens when using 100% of the train data.

@BigBrownBear00 How would that change the results? My model is updating based on train data only, so having additional validation set would still result in the same train/test graphs.

I am a little confused by what you are asking. Your question states that the train data achieves 100% accuracy in under 10 epochs, but your plots don't show that. As a rule of thumb, if your training data has high accuracy but your test data does not, then by definition you are overfitting and you need to either adjust your model or add regularization. You should also try different lambda values (learning rates) to see if that helps smooth out your plots.

@BigBrownBear00 Yea, my bad. When uploading the 3rd and the 4rd image, I accidentally overwrote the 1st and the 2nd (now the original 1st and 2nd have been put them back in).
Overfitting would mean that my test accuracy starts to drop after the "overfitting" point, which in my case does not happen. Futhermore, my test accuracy never stops increasing - this behaviour is the reason I posted this question.
I've tried a variety of different learning rates with more or less the same results.

Any good reason why you decided to remove the paragraphs or spaces that I added? Do you think it makes your post clearer or unclearer?

@nbro Sorry, don't know what you're talking about.

Incorrect. High variance, or overfitting, is when you training loss is low and your test loss is high. Your plots suggest you are overfitting the data and you should look at your model or add regularization.

@BigBrownBear00 Those are typical indicators of overfitting, but do not necessarily mean that the model is overfitting. In my case, the model generalization error (error on unseen data) is decreasing constantly, meaning the model performs better and better on the unseen data, which by definition means, that the model is not yet overfitting. If the model was in fact overfitting, the generalization error would increase at some point. As an example, I added (EDIT 2) images of accuracies and losses vs epoch after inducing 60% symmetric label noise to train data. The overfitting is obvious.

I don't follow what you are you saying. Suffice to say, I don't think 500 samples is enough to train your model (your first set of plots). After you add more examples (2nd set of plots), the loss and accuracy curves follow each other more closely,  as expected. You probably need to add more information to your question: what optimizer are you using? Learning rates tried? How many layers and what does your model look like?

@BigBrownBear00 I am using Adam optimizer and one cycle learning rate scheduler with max learning rate of 0.0025 (tried 5 values between that one and 0.01 - always got very similar results). The model I'm using is ResNet9 with the same structure as described here: https://myrtle.ai/learn/how-to-train-your-resnet-4-architecture/

is your data normalized?

@BigBrownBear00 My data is normalized using the default CIFAR10 RGB normalization numbers ((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)). In addition, I perform random crop and horizontal flip on the train images each epoch.


text:
How to generate new data using VAE?
I have built the following function which takes as input some data and runs a VAE on them:
def VAE(data, original_dim, latent_dim, test_size, epochs):

    x_train, x_test = train_test_split(data, test_size=test_size, random_state=42)

    # Define the VAE architecture
    #Encoder
    encoder_inputs = tf.keras.Input(shape=(original_dim,))
    x = layers.Dense(64, activation='relu')(encoder_inputs)
    x = layers.Dense(32, activation='relu')(x)
    x = layers.Dense(8, activation='relu')(x)

    #--- Custom Latent Space Layer
    z_mean = layers.Dense(units=latent_dim, name='Z-Mean', activation='linear')(x)
    z_log_sigma = layers.Dense(units=latent_dim, name='Z-Log-Sigma', activation='linear')(x)
    z = layers.Lambda(sampling, name='Z-Sampling-Layer')([z_mean, z_log_sigma, latent_dim]) # Z sampling layer

    # Instantiate the encoder
    encoder = tf.keras.Model(encoder_inputs, [z_mean, z_log_sigma, z], name='encoder')

    #Decoder
    latent_inputs = tf.keras.Input(shape=(latent_dim,))
    x = layers.Dense(8, activation='relu')(latent_inputs)
    x = layers.Dense(32, activation='relu')(x)
    x = layers.Dense(64, activation='relu')(x)
    decoder_outputs = layers.Dense(1, activation='relu')(x)

    # Instantiate the decoder
    decoder = tf.keras.Model(latent_inputs, decoder_outputs, name='decoder')

    # Define outputs from a VAE model by specifying how the encoder-decoder models are linked
    # Instantiate a VAE model
    vae = tf.keras.Model(inputs=encoder_inputs, outputs=decoder(encoder(encoder_inputs)[2]), name='vae')

    # Reconstruction loss compares inputs and outputs and tries to minimise the difference
    r_loss = original_dim * tf.keras.losses.mse(encoder_inputs, decoder(encoder(encoder_inputs)[2]))  # use MSE

    # KL divergence loss compares the encoded latent distribution Z with standard Normal distribution and penalizes if it's too different
    kl_loss = -0.5 * K.mean(1 + z_log_sigma - K.square(z_mean) - K.exp(z_log_sigma), axis=-1)

    #VAE total loss
    vae_loss = K.mean(r_loss + kl_loss)

    # Add loss to the model and compile it
    vae.add_loss(vae_loss)
    vae.compile(optimizer='adam')

    # train the model
    vae.fit(x_train, x_train, epochs=epochs, validation_data=(x_test, x_test))

where
def sampling(args):

    z_mean, z_log_sigma, latent_dim = args
    epsilon = K.random_normal(shape=(K.shape(z_mean)[0], latent_dim), mean=0., stddev=1., seed=42)
    return z_mean + K.exp(z_log_sigma) * epsilon

My question is, if I want to generate new data, by using the above VAE how can I achieve that ?
If I want to sample 100 new data, should I use this
   latent_mean = tf.math.reduce_mean(encoder(x_train)[2], axis=0)
   latent_std = tf.math.reduce_std(encoder(x_train)[2], axis=0)
   latent_sample = tf.random.normal(shape=(100, latent_dim), mean=latent_mean,
                                    stddev=latent_std)
   generated_data = decoder(latent_sample)

or
   latent_mean = tf.math.reduce_mean(encoder(x_train)[0], axis=0)
   latent_std = tf.math.reduce_mean(tf.math.exp(encoder(x_train))[1], axis=0)
   latent_sample = tf.random.normal(shape=(100, latent_dim), mean=latent_mean,
                                    stddev=latent_std)
   generated_data = decoder(latent_sample)

?
Basically should I infer z_mean and z_log_sigma from the z or should I use z_mean and z_log_sigma directly ? What is the difference ?
Moreover, I have seen that everytime tf.random.normal is used to generate new data from the latent space. Why not use lognormal for instance ? Is it because of the KL divergence ?
The end-goal is the distribution of the generated_data to be as close as possible to the distribution of the original data.

Why would you use log-normal? that's not the distribution you've learnt the parameters for, as far as I am aware.

@DavidIreland As far as I understand, you can generate anything, since whatever you generate it will later pass from the decoder, and passing from the decoder does not necessarily mean that the distribution from which you sampled from will be preserved, right ? Or am I missing something ?

Once you have trained the VAE, you can discard the encoder and feed data directly in.  You seem to know this however in your case it means that the z_mean and z_log_variance can be discorded since you are only generating a sample latent vector.  This should be centered on zero, so you can simply generate vectors based on a normal curve.

@DavidHoelzer what is the purpose then of the z_mean and z_log_variance ?

The expectation is that those two layers model the distribution, which you are sampling with the reparameterization trick using your z layer.

Normally in synthetic data generation using VAEs, you do not use the encoder anymore upon inference. Instead, generating samples after training is usually done by simply sampling random noise from 'a distribution' (i.e. a normal distribution) in the dimensions of the latent space and decoding it using the decoder. Hence, generating samples would be usually done as follows:
noise = tf.random.normal(shape=(100, latent_dim))
generated_samples = decoder(noise)

if you use training samples in generating your new data, it is not representative of how good your generative model is as it then bases the output on the training data.
Answer However, if you do want to sample using the training data to get the means and standard deviations, then your second method is a more appropriate method than your first method. You need the standard deviation, and you only get the log(std) from the model. Taking the exponent gives you the standard deviation.

Why is the second method more appropriate ? Could you please elaborate

Because you need to convert your log_sigma which is equal in function to log(std), to sigma (equal to std). You can do so by taking the exponent.

In the first method there is no need to take the exponent because it calculates directly the std from $z$, so in terms of "rescaling" these methods are equivalent. The question refers to "should I use the z_log_sigma directly (after rescaling it using the exp ofc) or should I infer the std from $z$ ?

My bad, I misread your code. The more i look at it the more confused i get too be honest. So first a question: are you trying to generate only one class? or multiple classes? Having the same mean and standard deviation for sampling noise for all classes is not logical. For sampling one class, you would have to use a different mean and standard deviation than for sampling another class

I have one class

How much do the calculations differ between the two options? In my expectation, they shouldnt differ much? Though you might want to calculate the std sample wise and take the mean, instead of calculating the std over the whole batch.

They dont differ much indeed, as expected. Still, it is a matter of completeness


text:
sentence alignment between same language texts
Take this two texts as an example:

"I will start the recording and share the presentation. I hope it's
all clear from what we saw last time on classical cryptography and if
you remember we got to discussing perfect security. Ah, you're right,
I didn't upload the slides. Wait, they are the same as last time. If
you want to get the face on the fly, last time I mean last year. I had
some things to do and didn't upload."

"Okay, I will start recording and I will share the presentation again.
1, 0 So I hope it's all clear compared to what we saw last time about
classic victory, And if you remember we got to let's say discuss the
perfect security. Ah you could see already, I didn't upload the
Slides, Wait they are the same as last time, Eh. If you want to have
done on the fly, I mean last time, I mean from last year. Morning I
had some stuff to do, I didn't upload."

We want to align the sentences so that similar sentences (within a certain degree of difference) are being matched together.
Which are the approaches to solve the issue?
[I will start the recording and share the presentation],
[I will start recording and I will share the presentation]

[I hope it's all clear from what we saw last time on classical cryptography],
[I hope it's all clear compared to what we saw last time about classic victory]

[and if you remember we got to discussing perfect security],
[And if you remember we got to let's say discuss the perfect security]
.
.
.

I have been looking into DTW and perceptual hashing as a way to solve the problem without any concrete result then I saw that in the field of automatic translation sentence alignment is widely used but with the assumption that the two texts have different languages and that there is a one-to-one mapping between words without "gaps" or extra words in between.

With this data, a simple similarity measure that will find a high value between the pairs "discuss" and "discussing", as well as between "classic" and "classical" can be the number of common characters from start divided by the root of the product of lengths. For the same words, similarity will be one, for words with no common prefix, zero. For close words such as the examples mentioned the value will be well above 1/2, useful enough for matching.
Based on this similarity measure, Dynamic Time Warping should solve the problem.
Or, instead of word alignment, character alignment with a simpler similarity function, 1 for equal characters, 0 for different. That would help align vIcTORY with crypTOgRaphY.
I don't see much need for word embedding for this particular application. The only word equivalence not detected by blind character evaluation is "stuff"=="things", but those words share three words on the left and two on the right, so they will be aligned anyway by a blind (and multilingual) method such as char-level DTW. Also, sentence splitting can be obtained as a byproduct of character alignment. Once aligned, the character zip can be cut where either thread has a point, or also a comma, if desired.

First, you should first address the problem of 1. splitting sentences, and 2. the number of sentences in two texts. Then you can use a sentence embedding model and compare the similarity between the sentences in two sets of sentences.
The easiest way could be averaging the ready-made word embedding vectors (word2vec, fastText, etc.) of all words in the sentence and using the resulting vector as the embedding vector for the sentence. Or you can use Doc2Vec directly on the sentences. Then you can use cosine as a similarity criterion between the sentence embedding vectors.
Another easy way is to use the BLEU metric to compare the similarity of the sentences.


text:
Why is $\sum_{s} \eta(s)$ a constant of proportionality in the proof of the policy gradient theorem?
In Sutton and Barto's book (http://incompleteideas.net/book/bookdraft2017nov5.pdf), a proof  of the policy gradient theorem is provided on pg. 269 for an episodic case and a start state policy objective function (see picture below, last 3 equations).

Why can we assume that the sum $\sum_s\eta(s)$ is a constant of proportionality? Doesn't it also depend on $\theta$, since it depends on the policy $\pi$?
What could make sense, would be to say that $\nabla J(\theta)  = \mathbb{E}_{s\sim \eta(s), a \sim \pi}\left[\nabla_{\theta}\mathrm{log}\left(\pi(s,a,\theta)\right)\,q_{\pi}(s,a)\right]\propto  \mathbb{E}_{s\sim d(s), a \sim \pi}\left[\nabla_{\theta}\mathrm{log}\left(\pi(s,a,\theta)\right)\,q_{\pi}(s,a)\right]$.
Since the proportionality constant is always $\ge 0$ (average time spent in an episode), any update direction suggested by $\mathbb{E}_{s\sim d(s), a \sim \pi}\left[\nabla_{\theta}\mathrm{log}\left(\pi(s,a,\theta)\right)\,q_{\pi}(s,a)\right]$ is the same as $\mathbb{E}_{s\sim \eta(s), a \sim \pi}\left[\nabla_{\theta}\mathrm{log}\left(\pi(s,a,\theta)\right)\,q_{\pi}(s,a)\right]$, but with different amplitude. This, however, wouldn't impact the learning process too much, since we multiply the update term with a low learning rate anyway.
Hence, as it is more easy to sample states from $d(s)$, we just set $\nabla_{\theta} J = \mathbb{E}_{s\sim d(s), a \sim \pi}\left[\nabla_{\theta}\mathrm{log}\left(\pi(s,a,\theta)\right)\,q_{\pi}(s,a)\right]$.
Could that serve as plausible explanation?

Thanks for linking this post! I think what you say in your answer there in the second paragraph ("Moreover, ...") goes in the same direction as what I tried to explain in my own answer suggestion above. So, it doesn't matter if we take samples from $d$ or $\eta$ (or $\rho$ in the linked post), as they represent the same distribution, just scaled by a factor, right? I guess, that taking samples from $d$ is just more intuitive, as sampling from a probability distribution is more intuitive than sampling from something that is not, i.e., $\eta$.

The derivation answering your question is just above the expression on your screenshot.

The answer is: $\sum_{s} \eta(s)$ is not a constant with regards to $\theta$. As you already mentioned
\begin{equation}
\sum_{k=0}^{\infty} \text{Pr}(s_{0}\rightarrow s, k, \pi) = \eta(s).
\end{equation}
But then the question arises: Why is the derivation you showed correct? It's not the point that $\sum_{s} \eta(s)$ is constant in regards to $\theta$ and is therefore not affected by the $\nabla$ operator but quite the contrary: That it is the result of applying the $\nabla$ operator to the state value function. If you just look at the Proof of the Policy Gradient Theorem (episodic case) in Sutton-Barto at the simplification of $\nabla v_{\pi}(s)$ you will find that (not showing all steps here):
\begin{align}
\nabla v_{\pi}(s) &= \nabla \left[ \sum_{a} \pi(a|s)q_{\pi}(s, a) \right] &\text{for all s} \in S \\
&=~... \\
&= \sum_{a}\left[\nabla \pi(a|s) q_{\pi}(s, a) + \pi(a|s) \sum_{s'}p(s'|s,a)\nabla v_{\pi}(s')\right] & (\text{using recursivity}) \\
&=\sum_{a}\left[\nabla \pi(a|s) q_{\pi}(s, a) + \pi(a|s) \sum_{s'}p(s'|s,a) \\ \sum_{a'}[\nabla \pi(a'|s')q_{\pi}(s', a') + \pi(a'|s')\sum_{s''}p(s''|s',a')\nabla v_{\pi}(s'')]\right] & (\text{further unrolling}) \\
&= \sum_{x \in S}\sum_{k=0}^{\infty} \text{Pr}(s\rightarrow x, k, \pi)\sum_{a}\nabla \pi(a|x)q_{\pi}(x,a).
\end{align}
Where $\text{Pr}(s\rightarrow x, k, \pi)$ is the probability of moving from state $s$ to state $x$ accounting for all intermediate states $s_{1}, s_{2}, ..., s_{k}$ by following policy $\pi$.
So we see, that this equation from which you start is the result NOT the target of a derivative.

Spitballing some ideas here -
as $\pi_\theta$ is updated using some gradient optimization process with small steps in the direction of the gradient, the changes to state visits are generally small as well. If so, maybe we can assume that the state-visitation frequency $\eta(s)$ will not change significantly. To be precise, while $\eta(s)$ is in fact $\eta(s,\theta)$, is it a valid assumption that when the step size is small enough, $\eta(s,\theta_t)\approx\eta(s,\theta_{t+1})$ for all $t$.
This may bring to mind a Quasistatic process, which is a thermodynamic process that happens slowly enough for the system to remain in equilibrium, but hey thats just my intuition

Thanks for your answer! Hmm, maybe that argument applies better to $\sum_s\eta(s)$. I read that $\eta(s)$ can be interpreted as the average time you spend in a state in one episode, which makes sense to me. Therefore, $\sum_s\eta(s)$ can be interpreted as the average time you spend in the episode. Maybe that doesn't change much in the sum, but the single $\eta(s)$ change more strongly. I am just wondering whether there is a strict mathematical argument for this. The book gives me the impression there is, but I didn't it in there nor by looking up other posts.


text:
Unclear step in off-policy (every-visit) MC Control: why do we need the line: $A_t \neq \pi(S_t)$ then exit inner loop?
Could please some expert on reinforcement learning explain the red-box part in the following off-policy MC control? I mean I did not understand what (and why)
is done in the step shown as a red-box. I understood all points, except the red box part: why do we need the line  "$A_t \neq \pi(S_t)$ then exit inner loop"? More specifically, why don't we not exit the inner loop when  $A_t= \pi(S_t)$?

I suppose to more explicitly get at what is probably the primary point of confusion you could add something like: "Why can we not continue using the remainder of experience generated within this episode for lower time steps $t$?"
(assuming that that is indeed what you're thinking)

why do we need the line "!=() then exit inner loop"?

To understand this, you need to go back to the importance sampling ratio:
$$\rho_{t:t+n} = \prod_{k=1}^n \frac{\pi(A_{t+k}|S_{t+k})}{b(A_{t+k}|S_{t+k})}$$
NB Here I am using the more general stochastic form for the target policy $\pi(a|s): \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R} = \mathbf{Pr}\{A_t = a | S_t = s \}$. Whilst the pseudocode uses $\pi(s): \mathcal{S} \rightarrow \mathcal{A}$. You have to convert between them to see how the weighting factor is really working.
The weighting factor $W$ in the pseudocode is in fact $\rho_{t:T-1}$, calculated step by step working backwards through the trajectory so that it can be used to weight the return from timestep $t$ for each $(s_t, a_t)$ pair.
An important detail is that if any of the $\pi(A_{t+k}|S_{t+k}) = 0$ within an episode, then $\rho_{t:T-1} = 0$ also, and this must then be true for all other timesteps before $t$.
The line:
$W \leftarrow W \frac{1}{b(A_t|S_t)}$
assumes that $\pi(A_{t}|S_{t}) = 1$. This is true for the greedy policy if, and only if $A_t$ is the current greedy choice. As the code uses notation for deterministic policy, then that is the same as saying $\pi(S_{t}) = A_t$ from the pseudocode.
So, when $A_t \neq \pi(S_t)$, $W$ should be set to $0$, and this will logically make $W = 0$ throughout the rest of the loop, and there will be no updates to any Q values.
Exiting the loop when $A_t \neq \pi(S_t)$ saves you from adjusting  $W = 0$ in the next line conditionally, then following through with calculating all the useless $0$ updates for earlier time steps.
Or another way to put it: With the $W$ update of $W \leftarrow W \frac{1}{b(A_t|S_t)}$, the loop may continue backwards to estimate expected returns from earlier timesteps only if the agent chose a non-exploring action compared to the current target policy. If an exploring action was taken, then the $W$ update from the pseudocode would be wrong. Not only would it be wrong, but determining the correct value - $W = 0$ - would result in no changes to any estimates for the rest of the loop. Easier just to terminate the loop at that point.
This is why off-policy Monte Carlo control is said to "only learn from the tails of episodes". If the agent explored a new action, that is learned from, but only from the point after it took the last exploring action to the end of the episode. Earlier exploring actions are also learned from indirectly, as they can put the agent in a different state that may not be reachable by the current target policy, and may turn out to be interesting.

@ Neil Slater, the point I was asking was this: why do we need to check $A_t \neq \pi(S_t)$?

@ Neil Slater, I changed the question photo with pseudocode from SB's book. Could you please update your answer based on this  new photo?  I think the updated answer will be okay for me.

@ Neil Slatter. Thank you so much. You are TOTALLY an expert on RL!


text:
Does it make sense to store information in a variable defined inside a Pytorch nn.Module?
I have a pytorch model (custom model inherited from nn.Module).
I'm developing some architecture, for which makes sense for my task to have a list defined in the model as:
class Model(nn.Module):
   def __init__(self, hparams):
       super().__init__()
       self.list = []

In every training step, I want to store in this list some information. Particularly, a number that I will calculate over the training step.
To be more clear:
for each training step I receive a batch of size $n$, and in the training step I update the self.list in Model to store $n$ values. For instance, after a first iteration with a batch size of 100, the self.list list will contain 100 values. Every iteration, I will add 100 more to the list...
I mean, this makes sense for my task, but I'm wondering if this is "legal". Also it seems a bit weird, because the parameters of my model would increase with the training size? Because in the end, if I use 10,000 training samples, this list will hold 10,000 values. But if I use more, the list will have more. Would this be an increase in parameters? When I save this model, will the number of parameters be affected by this? That's what I'm not sure about.

Generally, your PyTorch module is to contain the architecture that you are training, not the additional variables that you want to store. So generally, I would not implement my list like this.
What is usually done is to have one nn.Module for the architecture and one generic class for utility functions such as the model training/fitting/validation etc. Then the module is passed to the generic 'Trainer' class for training the model and your list of values would presumably be stored in the 'Trainer' object.
So something like this:
class MyNetwork(nn.Module):
    # Here define the architecture
    def __init__(self, *other_arguments): ...

    def forward(batch): ...

class Trainer():
    def __init__(self, network_instance, optimizer, *other_arguments):
        self.network_instance = network_instance
        self.optimizer = optimizer

        self.list_of_values = []

    def train(n_epochs):
        for i in range(n_epochs):
            # Standard train function layout
            self.optimizer.zero_grad()
            batch = self.get_data()
            predictions = self.network_instance(batch)
            loss = calculate_loss(predictions)
            loss.backward()
            self.optimizer.step()

            # Add value to list
            self.list_of_values.append(loss.item())

I understand. It indeed makes sense to store the list in the Trainer. Now, if I store a separate model (let's say an sklearn model) in a variable of the nn.Module, and I use this accumulated list to train that model, (calling it in the trainer as sklearn_model.fit(list_of_values), will those learned weights be succesfully stored inside the model variable in the nn.Module after training?

I am not familiar with the integration of sklearn models (and their parameters) into PyTorch Modules. I'd advise keeping them separate if possible.

I actually think they are not made to work together in terms of saving sklearn model weights for instance, at least from my experiments. I guess the right choice would be to implement the sklearn model in Pytorch and then load this model as an instance of the nn.Module class. Many thanks.

I think thats the right approach indeed. If the answer sufficiently answers your question make sire to upvote it and select it as the correct answer


text:
Using a VQ-VAE encoder's output to condition another model
I've trained a VQ-VAE on images that I want to use to condition another model (latent diffusion). The shape of my encoder's output is (4, 256, 56, 56) and I'm getting it using:
z_e = model.encode(input)
_, argmin = model.emb(z_e, weight_sg=True)

My understanding is that argmin is my latent code (which seems to be supported by the fact that this is an integer tensor). However, I'm noticing that this tensor contains duplicates. I'm not necessarily assuming this as a problem, but I wonder if I should remove them before sending them to the latent diffusion model? The issue there is that that model will presumably expect a fixed shape... Should I just pass the tensor on and not worry about the duplicates?
One other thing I'm wondering is whether I should normalize the argmin tensor before sending it to the latent diffusion model?

I guess by "tensor" you mean that you're getting several vectors, some of which are duplicates. Is that because you give the VQ-VAE several input samples? Whether you remove duplicated depends on what goals you're pursuing by processing those several samples independently in the VQ-VAE and jointly in the diffusion model. "Fixed shape" might be a red flag - why fix the number of samples, especially if you're considering removing duplicates dynamically? Also note that if the order of samples doesn't matter, the diffusion model should be invariant under changes of that order (permutations).

Searching around I was able to confirm that the duplicates are important to keep, so I won't mess with them. These are repetitions of individual codebook indices, not whole vectors, afaik. The tensor I'm talking about is giving me the codebook indices, some of which are repetitions. But since the structure is represented by these indices, they're clearly all necessary to keep. I had seen an example online that filtered the repetitions out, but perhaps there was another reason for that, like visualizing overall codebook usage... not sure.

Indices are an inappropriate input data representation for neural-network layers such as attention. Such layers tend to assign similar meaning to similar input values and dissimilar meaning to dissimilar input values, which in general is not the case for indices.

As I understand it, the VQ-VAE uses the indices to recall the embeddings, which are the actual inputs to the decoder.


text:
Does Number of Fully connected neural networks changes in transformer architechture based on max length input size?
Considering the architecture of encoder and decoder in transformer as shown below:

Does each input token after self attention mechanism (z1,z2,z3,...)is passed to it's specific  separate Feed forward neural network or does all the Z's are stacked into one and then passed to single FFNN?
If all the Z's are stacked into one, then how the difference in shapes of different inputs is taken care
If every z has its own Feed forward neural network, how in practical it is implemented with arbitrary input length?

the z’s are stacked and then passed to a single feed forward layer
the difference in shapes of inputs is resolved by padding to the dimension of the model

Hi @Arjun Reddy! If possible, please add some supporting information to this answer to help other readers understand why the answer is correct. Thanks!


text:
What is a good algorithm (and a heuristic) to find the shortest path in a puzzle game?
In a 3D world, there is a Robot, boxes, and switches. The robot's job is to carry boxes (one box at a time) and put them on top of switches to solve the level. All switches must have a box on top for the level to be solved. The robot can walk through switches but cannot walk through boxes.
The configuration of the robot, boxes, and switches in a level is random. However, the number of switches would always equal the number of boxes.
I want to find out the order in which the robot should visit boxes and switches in a way that minimizes the total number of actions. The following is an example of a level. Obviously, this example is 2D, but the answer should be generalizable to a 3D world.

S - switch,
B - box,
R - Robot starting position
In this particular level, if the robot goes for the box at the left cos it is the closest of the two boxes, it will have to place it on the switch at the left. Then it will have to go all the way around to pick the other box and place it on the other switch.
But if the robot decides to go for the box at the right first, even though it is the farthest of the two boxes from robot's starting position, it will be able to complete the level in a much less number of total actions.
So I want to figure out an algorithm that would give me the order in which the robot should visit boxes and switches so that the total number of actions will be minimal.
An informed search like A* might work, but I'm having a hard time coming up with a good heuristic.
I would really appreciate any help on this.

Heuristics typically don't resolve things like order of actions, but assist a search method that does. There are rules that such heuristics need to follow, but it is not clear whether that is what you want.

@NeilSlater Yes, that would help. I tried doing a state space search with A*, but I need a good heuristic that accounts for cases like in the above example. I'm having troubles coming up with one.

Can you please put your question in the title? "An algorithm to find shortest path in a puzzle" is not a question but an annoucement.


text:
Consequence of Dvoretzky Stochastic Approximation Theorem
I am trying to understand all the steps to prove the TD0 algorithm, and I am following a proof which uses a theorem of Tommi Jaakkola, Michael I. Jordan and Satinder P. Singh, in the paper: On the Convergence of Stochastic Iterative
Dynamic Programming Algorithms.
For proving an extension of the Dvoretzky theorem, they first state and claim that this lemma is a standard result and that it follows from Dvoretzky theorem:
Lemma 1
A random process
$\omega_{n+1}(x) = (1-\alpha_n(x))\omega_n(x)+\beta_n(x)r_n(x)$
coverges to zero with probability one if the following conditions are satisfied:

$\sum_n\alpha(x) = \infty, \: \sum_n \alpha^2_n(x) < \infty, \: \sum_n\beta_n(x) = \infty, \: \sum_n \beta_n^2(x) < \infty,$ and  $\mathbb{E}[\beta_n(x) \mid P_n] \leq \mathbb{E}[\alpha_n(x) \mid P_n]$ uniformly with probability 1.

$\mathbb{E}[r_n(x) \mid P_n] = 0 $ and $\mathbb{E}[r_n^2(x) \mid P_n] \leq C $ w.p.1, where
$
P_n = \{\omega_n, \omega_{n-1},...,r_{n-1},r_{n-2},...,\alpha_{n-1},\alpha_{n-2},...,\beta_{n-1},\beta_{n-2},...\}.
$
All the random variables are allowed to depend on the past $P_n$

The proof in the paper is only the following:
"Except for the appearance of  $\beta_n(x)$ this is a standard result. With
the above definitions convergence follows directly from Dvoretzky's extended
theorem (Dvoretzky, 1956)."
It can be found here. This lemma appears in page 11 in the paper, in the proof of Theorem 1: Tommi Jaakkola, Michael I. Jordan and Satinder P. Singh paper
The Dvoretzky theorem is from a paper of 1956 by Aryeh Dvoretzky (On Stochastic Approximation). I will write it here:
Dvoretzky Theorem
Let $(\Omega, F, \mu)$ be a probability space and $\alpha_n$, $\beta_n$ and $\gamma_n$, $n = 1, 2, ... $ non-negative real numbers satisfying:
\begin{equation}
        \lim_{n\to\infty}a_n = 0,
\end{equation}
\begin{equation}
        \sum_{n=1}^{\infty}\beta_n < \infty,
\end{equation}
\begin{equation}
        \sum_{n=1}^{\infty}\gamma_n = \infty.
\end{equation}
Let $\theta$ be a real number and $T_n: \mathbb{R}^n \rightarrow \mathbb{R}$, $n = 1, 2, ... $, measurable functions satisfying
\begin{equation}
        |T_n(r_1,...,r_n) - \theta| \leq \max(\alpha_n, (1+\beta_n) |r_n - \theta| - \gamma_n)
\end{equation}
for all real numbers $r_1,...,r_n$. Let $X_1$ and $Y_n$, $n = 1, 2, ... $, be random variables. We define
\begin{equation}
        X_{n+1}(\omega) = T_n[X_1(\omega),...,X_n(\omega)] + Y_n(\omega), \quad n \geq 1.
\end{equation}
Then, the conditions
\begin{equation}
    \mathbb{E}[X_1^2] < \infty,
\end{equation}
\begin{equation}
    \sum_{n=1}^{\infty}\mathbb{E}[Y_n^2] < \infty,
\end{equation}
and
\begin{equation}
\mathbb{E}[Y_n \mid X_1, ..., X_n] = 0
\end{equation}
w.p.1 for all $n$, imply
\begin{equation}
 \lim_{n\to\infty}\mathbb{E}[(X_n-\theta)^2] = 0
\end{equation}
and
\begin{equation}
\mathbb{P}(\lim_{n \to \infty}X_n = \theta) = 1.
\end{equation}
And now an extension to the case where the coefficients are non-negative functions.
The theorem remains valid if $\alpha_n$, $\beta_n$ y $\gamma_n$ are replaces by non-negative functions $\alpha_n(r_1,...,r_n)$, $\beta_n(r_1,...,r_n)$ y $\gamma_n(r_1,...,r_n)$, respectively, provided they satisfy the conditions:

The functions $\alpha_n(r_1,...,r_n)$ are uniformly bounded and
\begin{equation}
    \lim_{n\to\infty}\alpha_n(r_1,...,r_n) = 0
\end{equation}
uniformly for all sequences $r_1,...,r_n$.

The functions $\beta_n(r_1,...,r_n)$ are measurable and
\begin{equation}
    \sum_{n=1}^{\infty}\beta_n(r_1,...,r_n)
\end{equation}
is uniformly bounded and uniformly convergent for all sequences $r_1,...,r_n$.

The functions $\gamma_n(r_1,...,r_n)$ satisfy
\begin{equation}
    \sum_{n=1}^{\infty}\gamma_n(r_1,...,r_n) = \infty
\end{equation}
uniformly for all sequences $r_1,...,r_n$, for which
\begin{equation}
    \sup_{n \geq 1} |r_n| < L,
\end{equation}
$L < \infty$ being an arbitrary number.

The source can be found here: Dvoretzky paper
However, I can't find a way to relate that lemma to the Dvoretzky theorem, and nor can I find papers that prove the result, as Jaakkola states. Is it actually easy to prove that from the theorem? I would appreciate any help in this matter, and it would be perfect if you could provide an article or book where they prove that, or something similar.

The Dvoretzky Stochastic Approximation Theorem is a result in the field of stochastic approximation theory, which provides insights into the convergence properties of certain iterative algorithms used in optimization and machine learning. Unfortunately, the link you provided is not accessible as it directs to a specific question on the AI Stack Exchange that might have been removed or modified.
However, I can still provide you with a general explanation of the consequences of the Dvoretzky Stochastic Approximation Theorem. The theorem is named after the mathematician Aryeh Dvoretzky and establishes conditions under which a sequence of random variables converges almost surely to a fixed point. The fixed point is typically the solution to an optimization problem or an equilibrium point in game theory.
The theorem provides guarantees about the convergence behavior of stochastic approximation algorithms, which are iterative algorithms that update a sequence of estimates based on noisy or stochastic observations. Stochastic approximation algorithms are commonly used in reinforcement learning, online optimization, and other areas of machine learning.
The consequences of the Dvoretzky Stochastic Approximation Theorem are as follows:

Almost sure convergence: The theorem guarantees that under certain conditions, the sequence of estimates generated by a stochastic approximation algorithm converges to the true solution with probability 1, or almost surely. This means that the algorithm will eventually converge to the correct solution, regardless of the initial conditions or the presence of noise in the observations.

Noisy observations: The theorem takes into account the presence of noise or randomness in the observations used by the algorithm. Stochastic approximation algorithms are designed to handle situations where the observations are corrupted by noise, making them suitable for learning in uncertain environments.

Conditions for convergence: The theorem provides specific conditions that need to be satisfied for the algorithm to converge. These conditions typically involve the step size or learning rate of the algorithm, as well as properties of the noise or random variables involved in the estimation process. Understanding these conditions is crucial for designing and analyzing stochastic approximation algorithms.

Overall, the Dvoretzky Stochastic Approximation Theorem is an important result in the field of stochastic approximation theory, providing theoretical guarantees for the convergence of iterative algorithms used in optimization and machine learning. Although I cannot provide the specific details of the question you linked, I hope this explanation gives you a general understanding of the consequences of the theorem.

Thank you! Your explanation helps me in understanding a little more the Theorem, but I am kind of having more of a technical problem trying to apply that theorem to prove the lemma. I don't know how to identify the coefficients in the lemma with the ones that appear in the Dvoretzky theorem.

On another subject, aren't the links directing to papers and not to the forum?

Jaakkolla et.al paper: http://hdl.handle.net/1721.1/7205
Dvoretzky paper: https://projecteuclid.org/proceedings/berkeley-symposium-on-mathematical-statistics-and-probability/Proceedings-of-the-Third-Berkeley-Symposium-on-Mathematical-Statistics-and/Chapter/On-Stochastic-Approximation/bsmsp/1200501645


text:
Why does linearly decreasing batch sizes result in exponentially increasing training times?
I'm quite new to machine learning and wanted to ask a question regarding why reducing batch sizes cause exponentially increasing training times. An example of this relationship between batch size and training time can be seen in this article. I had a hard time finding any answers to this question with my googling skills and any possible hypotheses that I have as to why are most likely incorrect. I'd greatly appreciate if somebody could explain the reasoning behind this relationship.

I copied raw numbers from the graph and did some plotting.
   2048     83
   1024     84
    512     86
    256     90
    128    104
     64    135
     32    204
     16    360
      8    656

I don't know the actual number of training samples, so I used a place-holder value of 10k here. As we can see, the hardware is under-utilized when the batch size is too small. This is caused by a constant(?) overhead related to calculating losses and gradients in regards to each batch. Ah, I forgot to divide training time by the number of epochs, but only relative numbers matter here.

In an other point of view, the data fits well on a function $y = a/x + b$ where $x$ is the batch size and $y$ is the training time. In the limit, training time approaches the theoretical minimum when the hardware is best utilized, but each separate batch causes a fixed overhead on setting things up.

I did not include this in the question but the reason why I had this question was because I wanted perform regression on some data I obtained from a network I was training. I hypothesize that the reason behind the data having an inverse relationship is the fact that the number of backwards passes is inversely proportional to batch size. And the vertical shift is due to the (near) constant training time associated with forwards propagation if differences in GPU utilization across batch sizes are ignored. I am not entirely sure whether my reasoning is correct, but thanks for your answer.

This has a very simple hardware explanation.
GPUs have several thousand cores. If you are not making use of enough of these cores, eventually the cost of moving data over to the gpu comes to dominate the parallelism. Every time you reduce batch, you reduce parallelism and increase data transfers.
For example, if you halve batch size, you have increased the amount of data transfers by 2, and decreased the parallelism by 2. Now, this can be expected to be more than twice as slow, since both data transfer and computation have doubled.
However, I do not think this relationship is actually exponential. The article you linked does not seem to support that conclusion either. I know the article uses the world exponential, but if you look at the chart this is simply not the case. The chart in that article is presented on a logarithmic scale, so when you actually look at the numbers on a linear scale, the relationship is close, but not quite, linear, for reasons I outlined.


text:
How to prepare audio data for deep learning?
Audio data is typically an array with the waveform represented by values from -1 to 1. There are two issues with that:

if all values are inverted, e.g. -1 becomes 1 and 1 becomes -1, the audio doesn't change. But if for example I need to find difference between two audio files, finding per-element difference will say two inverted audio arrays are very different. Realistically two sine waves can easily be shifted relative to each other in a way where they will be inverted to each other.
Related issue is that a wave, for example a 1000hz sound wave, often sounds like a "flat" sound. However in the array it is a literally a sine, and two sines can be shifted which causes inconsistent difference. Ideally a sine should be a sequence of the same number, which is obviously hard to do because audio is usually way more complex than a sine wave.

So what I tried doing, I make a copy of audio files, then I calculate a gradient which hopefully reduces the shifting issues, and then I convert array into absolute value (-1 turns 1). And then I use that copy when comparing audio arrays. When I used that for evaluating how close generated audio is to the original, it caused a lot of low frequencies in the generated audio. When I looked at the waveform, this is because gradient makes low frequencies very quiet since they have small rate of change, so my model doesn't see them. To be clear I am not really sure if gradient is a better match at all. But ideally I'd want something like gradient that doesn't reduce low frequencies.
There is also spectogram - admittedly I haven't looked much into it, but the one I tried - librosa spectogram functionality - takes quite a long time to convert that back into audio. If there is no quick way to do it with 1d arrays, I can use that.

How does this require using AI to solve this problem?

@Rob I am asking what format will be better for an AI to process

It still seems orthogonal (definition 4) to artificial Intelligence, and more like signal processing. There is where I'd suggest that you'd get a better answer, at https://dsp.stackexchange.com/ - they have an artificial Intelligence tag available for such questions. --- It seems like before AI rather than during AI. For example: https://dsp.stackexchange.com/q/75296/37400 gives 5 answers.

I noticed that we do have a couple of questions with answers:  https://ai.stackexchange.com/a/31721/17742 https://ai.stackexchange.com/a/27147/17742

Can you please explain how you calculate what you call "gradient"? Gradient of what?

@nbro numpy.gradient(audio)