text:
How to train Diffusion model with additional loss?
I would like to train a diffusion model with an additional loss on the created image. Without getting into too much details my intention is to do something like regularization, for example you may think that I want to make sure the created image is smooth, or something of the sort.
My thinking was to add an additional loss during training, if the vanilla training process is:
$$L = ||\epsilon-\epsilon_\theta(x_t, t)||$$
where $\epsilon_\theta$ is the model learning to predict the noise $\epsilon$ added to the original image. My suggested loss is:
$$L = ||\epsilon-\epsilon_\theta(x_t, t)|| + \lambda * L'(img_\theta)$$
where $L'$ is my additional loss (which may for example induce smoothness or whatever), $img_\theta$ is the image that we get after denoising $x_t$ using the predicted noise.
For standart models it is trivial that this makes sense. Due to the iterative nature of diffusion models I'm not sure if specifically for them it makes sense. I wasn't able to find any work that does something like this, would appreciate any help, does my additional loss makes sense to add?

Be careful not to mix different parameterizations: Here the first term is performing $\epsilon$-prediction (i.e you predict the noise part of the image), while your second term $img_\theta$, seems to be depending on a prediction of the denoised image. As @Peblo pointed out, training and sampling algorithms are typically different, and in your proposed loss you'd likely not be able to exploit the $\epsilon$-prediction reparameterization

I have nearly the same question, do you have tried it? I think the key issue is the physical meaning during the intermediate steps, do the loss work when the generated thing is still a kind of noise?

I don't see why it should not work.
Though, the tough part is how to implement this.  The algorithm requires to sample a singular time point along the timestep sequence, draw $\epsilon$, calculate $x_t$ for a batch, and evaluate the loss.
In your case you should also create the final image for the whole batch at each individual evaluation of the loss, which is extremely time consuming.  But, other than that, I think it should work.

It's time consuming because I'm gonna have to do the whole denoising process? So, for example, it may include 50 denoising steps instead of the single step that you usually do during training, that is what you mean?
Perhaps it would make sense to denoise using a single step only? then it would spare the full denoising process. Anyhow, I intend to work with low resolution (MNIST) images, so I'm not that worried about efficiency

@GiladDeutsch yes this is what I meant with "time consuming".  You may try to partially mitigate the problem by skipping a few denoising steps (look up the reply by Ciodar to this question), or by adding the penalization term at random only every N batches.
Keep me posted, I am curious to know about the solution you achieve.

are you sure that the fact that the diffusion model is recursive (i.e iteratively denoises the image) does not create an issue with this idea of learning using a loss on the created image? seems like this would create the same problems that RNNs have

I am not sure which RNN issue you refer to, however -- regarding the recursiveness: keep in mind that the reproduction, i.e., the whole inverse process at prediction time, is indeed recursive, but the learning phase is not.  During the learning phase the NN only needs to know the timestep $t$ under current evaluation, and upgrade the weights, it does not need to recur.  Look at Algorithm 1 in Ho et al. 2020, step 5.


text:
Could generative AI bootstrap?
In programming, if a new language could be improved by the language itself, it’s call self-hosting or bootstrapping.
To develop generative AI, there’s some steps, data preparing, model training, fine tuning. Is it possible to use AI it self to help with these steps and make big improvement in efficiency?

Could you clarify what you mean by "AI itself"? That's very broad, it's a bit like asking about using "computers themselves" to improve computers, which is trivially true (chip designs are made on computers).

Let me see if this helps-
I believe what you request is an understanding of whether or not Language models generate content to train themselves or other SOTA models.
In short yes, Language models can automate tasks like data pre-processing, and data-preparing, and given the right algorithm (consider OpenAI's new function calling feature) even make decisions on writing the code to train themselves in an optimal manner by mere means of arguing with themselves.
You might be aware of GANs which kind of do that (improve each other without much human interception).
I mean consider all the prompt styles being researched and used, Chain-of-Thought, Tree-based-Chain-of-Thought, etc. If we can have LLM reason itself why can we not have it engineer textual data for us? It is coming, if you search you will find research in this direction. Here is an example.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Machine Learning Math Requirements
What area of math do i need for AI and ML? and some recommendations for YouTube math's playlist
I've just started to learn to code in python and i want to go to the path of AI/ML

Here is a list of pre-requisites (not-exhaustive, but a good representation)-

Calculus I and II (For a foundational understanding of the underlying math)
Linear Algebra (For more intuitive understanding)
Statistics and Probability (to understand how data works for you ;))

Of course, this goes without saying, you will need a good understanding of a programming language (python in your case) before starting to use the following libraries.
Here are some python based libraries that are way too valuable to us-
(Basic)

Numpy
Pandas
Matplotlib

(Advanced)

SciPy
Scikit-learn
Tensorflow
Keras
Pytorch

Please note that the libraries are divided into basic and advanced. You can start with the pre-requisites and basic libraries and then slowly move on to the more advanced ones.

Where should i start at the math since I've just learned basic math im not really good at it and im following cornell machine learning course?

It's good that you know you are not good at it. No one is, at the start. Based on your skill level, I will suggest taking a more basic course to start (try udemy if you can) or seeing some fundamentals courses on Coursera. If you feel comfortable with Matlab, Andrew Ng's course on ML on Coursera is highly recommended.

Before watching the course? What area of math should I learn statistics, calculus, linear algebra? Im really lost

This is a non-specific answer as the order may vary case by case. For me, based on my understanding, I'd suggest start with linear algebra, move to calculus, and then dive into statistics and probability.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Failing to train and avoid overfit on noisy training data
I have added some Gaussian noise to CIFAR10 training and test set. I am using VGG16 and ResNet34 as the model to be used for training.
Under normal training conditions, where the standard CIFAR10 is used(without any Gaussian noise), the training happens as expected. The model does not overfit when trained using the right transforms and data augmentation. I am able to achieve 95% training accuracy and 91% test accuracy.
But if I add minor Gaussian noise to the entire training and test set and train the model using the same transforms, the model overfits considerably. The training accuracy reached 98% whereas the test accuracy does not go past 80%.
My initial reasoning for this was that I used color jittering for the transforms during normal training(when using clean data). But since color jittering is similar to gaussian noise, it does not prove to be an effective image augmentation technique when the model is being trained on data with gaussian noise.
I added salt and pepper noise as additional augmentation techniques when trying to train on the noise data, but it does not have any effect and continues to overfit.
For the hyperparameters:
I am using the Adam Optimizer, tried SGD as well.
I tried various learning rates 0.1, 0.01, 0.02, 0.025, 0.005, 0.001 along with step schedulers to reduce the learning rate.
I experimented with weight decays from 1e-04 to 1e-06.
The transforms I used for clean data training are:
transforms = transforms.Compose(
[transforms.RandomHorizontalFlip(),
transforms.RandomRotation(15),
transforms.RandomAffine(degrees=15, shear=15, scale=(0.85,1.15)), transforms.ColorJitter(brightness=0.15, contrast=0.15, saturation=0.15, hue=0.15), transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])

The transforms I tried when trying to train the models from scratch on the gaussian noise data are:
transforms = transforms.Compose(
[transforms.RandomHorizontalFlip(),
transforms.RandomRotation(15),
transforms.RandomAffine(degrees=15, shear=15, scale=(0.85,1.15)), transforms.ColorJitter(brightness=0.15, contrast=0.15, saturation=0.15, hue=0.15),
SaltPepperNoiseTransform(prob=0.1, amount=0.05),
SpeckleNoiseTransform(std=0.15),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])

I experimented with different values for each augmentation I applied.
How can I achieve the same training and test accuracy when training the model from scratch on the Gaussian noise training and Gaussian noise test data as I did on training from scratch on clean CIFAR 10 training and test data ?
Any tips or advice will be helpful.


text:
When do we apply a mask onto our padded values during attention mechanisms
When we are applying a mask onto the padded values in an input sequence, it is typically done through setting the padded values as negative infinity. For example, a tensor of values [1,2,3,0,0] should result in a padding mask of pad_mask = [True, True, True, False, False] (or the opposite depending on your flavour). However, if we apply the mask i.e attention_scores = attention_scores.masked_fill_(pad_mask.T == False, float('-inf')) before applying softmax, won't we get the 4th and 5th row of the attention_scores as 'nan' when we softmax attempts to calculate the probability distribution along each row?
Does that mean the step of where to apply the mask is incorrect, and we should apply a zero-ing out of the pad token rows in the attention_score matrix after applying the softmax function? or is there another key concept/step I am missing here

You are masking at the wrong place. Masking happens before the sequence goes into the encoder/decoder layer depending on what kind of architecture you are using. It happens right after you calculate the embeddings using positional encoding + token encodings.
This is because the masked positions are also considered in calculating the attention values, and if you simply make those masked attention values zero, the model loses the attention and can't make accurate predictions.

This is the correct place to use the mask.
If you are concerned that applying the softmax afterwards will produce nan, then instead of float(-inf) just use some relatively large number, e.g. -1e9:
attention_scores = attention_scores.masked_fill_(pad_mask.T == False, -1e9)

Why not mask after the softmax ?
After you apply the softmax function to the attention scores you get the attention weights (also called attention probabilities). In general, you want these to sum up to $1.$, and this is exactly what you get with the softmax layer. However, if you mask some attention weights after the softmax, then they will no longer sum to $1.$
Setting the attention scores to a high negative number will effectively make the softmax produce $0.$ at these positions and will re-distribute the masked weight over to the un-masked positions so that the attention weights sum to $1.$
You can read more (and see some cool pictures) here.


text:
Why is ChatGPT fast on some questions and slow on others?
When using ChatGPT i have noticed it is fast on some questions and slow on others. Why is that?
Based on my understanding of the transformers architecture, the user prompt is tokenized and the vector of token ids is fed as input to a strictly feed-forward neural network and the output of the model becomes part of the input for the next token that the model has to generate (this gives the model its auto-regressive nature).
Since there are no conditional loops in the NN, I would expect the time to generate the first token should depend on the length of user prompt. It shouldn't matter whether the question being asked is easy or tough. But I have clearly observed that when asked an easy question ChatGPT responds very fast and when asked a tough question it takes time to respond. Why is that? Is it due to caching of previous responses?

That is a very good question. On a high level, it might be the Causal reasoning used by OpenAI and designed by their Legal and Design teams.
WARNING! Trigger words ahead - Completely for illustration purposes.
As you can see, in the first completion produced by Da-vinci-003, it produced the content based on my input alone. But, as we know their chat models are fine-tuned for human level conversations, and with time have also been explicitly trained to do and not-do certain things, it refused to answer my query. What I believe is that they have multiple causal layers, which get triggered with certain queries.
Now with that argument, this causal layer, when recieves a sophisticated query, it is instructed to generate a certain kind of response. Maybe it is instructed to first break it down into simpler points, structure them in a logical order before generating the "to be returned" response to the user.
On the technical side, you are correct, during the inference the time complexity should practically be the same (given two inputs of the same number of tokens).

https://i.sstatic.net/5UA8n.png [davinci-003]

https://i.sstatic.net/1T8il.png [gpt-3.5-turbo]

This site not appropriate for speculations. This site originally fell under the science category, so it should be about facts. So, I'd discourage you from providing answers if you only have a guess. If you have a guess, you can leave a comment, once you get the privilege to comment under posts.

@nbro makes sense. I will keep that in mind.


text:
RAM Capacity of Mac Studio with M2 Ultra for inference of 65B LLM
How much RAM would be needed on Mac Studio M2 Ultra for inferring from a 65B LLM model. There are three options: 64GB, 128GB and 192GB. If using Apple M2 Ultra with 24‑core CPU, 76‑core GPU, 32‑core Neural Engine with 192GB Unified Memory, how would be the performance of the model?


text:
Comparing Auto-regressive Encoder-Decoders and Topological Neural Networks
I am interested in what insights can be gained about the mathematical class of auto-regressive encoder-decoders (LLMs), by comparing them to topological neural networks.
Specifically, I am looking for similarities and differences in their structures, behaviors, and mathematical properties.
In the context of this question, an LLM is a type of neural network that is designed to generate sequences of data, such as sentences in a language. It does this by learning to predict the next item in a sequence based on the previous items.
Any insights, references, or resources that could help clarify this would be greatly appreciated.
References:

Building an Autoencoder for Generative Models


text:
How are gaming GPUs different from GPUs for ML?
I have a seemingly innocent question. I was looking to buy a computer for deep learning development using large language models. When I searched online on consumer e-commerce websites, I could not find any computer with NVIDIA P100, V100 for sale. Why? These are the GPUs used for deep learning on cloud VMs. Consumer GPUs (I use this term to mean GPUs available in computers you will find on sale on e-commerce websites such as Amazon.com) on the other hand seem to be centered around so-called RTX series built for gaming applications. I tried looking for GPU benchmarks on 3 different sites [1,2,3] and cannot find any mention of P100, V100 etc. All of them are filled with RTX.
What is the difference between these chipsets? And more specifically, is there some inherent difference that will make RTX GPUs suboptimal if I were to use one for ML tasks (as opposed to gaming)?

Usually the difference is mainly about drivers and amount of VRAM (other than price..).
For example, high-end NVIDIA GPUs tend to even share the same architecture, having the same amount of "compute units", between RTX and A series (or the older P and V). The difference is that professional GPUs have more optimized drivers which are both more stable and faster for supported applications.
Another difference is that the A series (e.g. A6000 or A100) tend to have way larger VRAM capacity (e.g. 48GB and even 80GB) which are more suitable to handle large models, and even large batch sizes during training.
In principle, when picking a GPU for DL you should care more about VRAM than execution speed.

Nvidia also claim that the server-based GPUs like the A100 are built to handle higher demands of services, mainly uptime and constant use at max capacity. In addition, and very important to OP's question, Nvidia block use of gaming GPUs in the cloud with a legal mechanism - they refuse to license the driver software for e.g. 4080 series for servers


text:
How does a multidimensional vector get fed into a single node in a neural network?
I mostly develop neural networks completely from scratch, like without libraries. I've been seeing, especially in NLP tasks, entire vectors, often representing words, get fed into a single node. I'm not really sure how this works, because in all the books I've ever read, and all the projects that I've developed, only a single number has ever gone into each node. So, how does this work?

where have you seen this? maybe mentioning the reference and the quotes (or the codes) can help clarify whether it really is feeding a whole vector into a single node

If the node is $R\to R$ it is probably some transform prior to it and then applied elementwise (or on scalar). If the node is $R^m\to R^n$ I think pretty much anything can happen, as long as you can compute the appropriate jacobians.

Fine thing to develop your algorithms from scratch, fine thing.
In deep learning, particularly in natural language processing (NLP) tasks, it is common to represent words as vectors and feed these vectors into neural network nodes. This technique is known as word embeddings or word vector representations. I can explain the concept to you based on the principles outlined in the book Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
In traditional neural networks, each node receives a single number as input, typically representing a feature or an activation value. However, when working with text data, a single number is not sufficient to capture the rich semantic meaning of words. Instead, we need a way to represent words in a continuous vector space, where similar words are close together and dissimilar words are far apart.
Word embeddings provide a solution to this problem. They are dense vector representations that capture semantic relationships between words. These vectors are learned from data and can be considered as a projection of words into a lower-dimensional space, where each dimension represents a different aspect of meaning.
One popular word embedding algorithm is Word2Vec, which has two main variants: Continuous Bag-of-Words (CBOW) and Skip-gram. Both algorithms aim to learn word embeddings by predicting the context of a word within a given sentence or a window of surrounding words.
The CBOW model, for instance, takes the context words as input and predicts the target word. In this case, the context words are encoded as vectors and summed together before being fed into a single node in the neural network. The network learns to associate the context vector with the corresponding target word. The training process adjusts the word embeddings in such a way that words with similar contexts have similar vector representations.
By using word embeddings, neural networks can leverage the semantic information captured in the vector space to perform various NLP tasks. These embeddings can capture syntactic and semantic relationships, analogies, and contextual information between words.
For example, Let's consider a simple example using the Word2Vec CBOW model to learn word embeddings from a small corpus.
Suppose we have the following corpus consisting of three sentences:

"I love deep learning."
"Deep learning is fascinating."
"Neural networks are powerful."

To create word embeddings, we need to define a window size, which determines the number of context words we consider on each side of the target word. Let's assume a window size of 1 for this example.
First, we need to construct a vocabulary from our corpus. In this case, our vocabulary would consist of the following unique words:
["I", "love", "deep", "learning", "is", "fascinating", "neural", "networks", "are", "powerful"]

Next, we assign a one-hot encoding to each word in the vocabulary. A one-hot encoding represents each word as a binary vector where only the position corresponding to the word's index is 1, and all other positions are 0. For example, the one-hot encoding for the word "learning" would be [0, 0, 0, 1, 0, 0, 0, 0, 0, 0].
Now, we can create training examples for our CBOW model. Let's take the second sentence, "Deep learning is fascinating," as an example.
For each target word in the sentence, we consider the window of context words. With a window size of 1, we have the following training examples:

Input (context) words: ["deep", "is"]
Target (center) word: "learning"

To encode the context words as vectors, we use their respective one-hot encodings:

Context word "deep": [0, 0, 1, 0, 0, 0, 0, 0, 0, 0]
Context word "is": [0, 0, 0, 0, 1, 0, 0, 0, 0, 0]

To feed these context word vectors into a single node in the neural network, we can simply sum them together:

Sum of context word vectors: [0, 0, 1, 0, 1, 0, 0, 0, 0, 0]

The target word, "learning," will be used as the output label for training.
We repeat this process for all target words in the corpus, considering the context words within the defined window size. The neural network then learns to associate the summed context vectors with the corresponding target words by adjusting the word embeddings.
After training, we obtain word embeddings where words with similar meanings or contexts are closer together in the vector space. These embeddings can be used for various NLP tasks or as input to other downstream models.

I understand how CBOW and Word2Vec works, I was asking about how a whole vector of numbers gets fed into a single node.

LINEAR COMBINATION WITH AN OFFSET ("BIAS")
Let's look at a simple situation where just a 2-vector is fed to a node. Draw it out.

Let $\sigma$ be an acitvation function. The the output node on the right takes a value equal to:
$$
\sigma\left(
\omega_{\text{red}}x_{\text{red}} +
\omega_{\text{blue}}x_{\text{blue}} +
b_{\text{purple}}
\right)
$$
The $\omega_{\text{red}}x_{\text{red}}$ is equal to some scalar multiple of the first vector component (in red).
The $\omega_{\text{blue}}x_{\text{blue}}$ is equal to some scalar multiple of the second vector component (in blue).
The $b_{\text{purple}}$ is an offset or bias term (hence the "b").
For a general input vector of dimension $p$, you wind up with something like $
\sigma\left(
b +
\overset{p}{\underset{j=1}{\sum}} \omega_j x_j
\right)
$.
The $b +
\overset{p}{\underset{j=1}{\sum}} \omega_j x_j$ is a single number, so the activation function only takes a scalar input, rather than a vector.
The $\omega$ weights and $b$ bias are learned through optimization.


text:
In DQN how does the Q network not converge to the incorrect target?
Whenever you are doing reinforcement learning you periodically update the target network based on the weights of the Q network. While I do understand this helps create a stable target I do not understand how it helps the Q networks accuracy. If I am updating the Q network much more frequently than I am the target network then wont the Q network converge to and estimate of the target network that is inaccurate since the target network is being updated much slower ?

I think this https://ai.stackexchange.com/a/13459/69468 already answers your question

oh wow perfect this is exactly what I needed


text:
Relation between Batch Size and Micro Batch Size
In distributed training of large models (pipeline parallelism), a mini batch of training samples is divided into n-micro batches. Each device performs forward and backward passes for a micro batch.
What is the relation between Batch Size, Micro Batch Size, Gradient Accumulation (usually: batch size/micro batch size) in relation to no of GPUs as well as GPU size?
I understand that a micro-batch size is bounded by the GPU capacity, increasing it may lead to OOM error. Let suppose, for two GPU of 40GB capacity, we can use a max micro-batch size of 4 and we are using the batch size of 128. Next, we want to increase the count of GPU to 8 (each 40GB capacity). What is the best way to speed up. Whether increasing the batch-size will help? Or the global batch size has very subtle/no effect on the training time (only overhead is synchronization) in distributed training.


text:
How to approach abstract art image AI captioning
I tried sachin/vit2distilgpt2 to caption an abstract image I have, and it seemed to output a blank string for the caption. I'm new to this, but my goal is to caption some of my abstract artwork - for instance describe the colors, sharpness, vividness, what it may be representing or abstractly looks like.
What could I use to approach setting up and/or training an AI model to characterize and caption abstract artwork?

To get a sense of the possibilities you can use the "describe" command in midjourney

Check out the "Image to Text" section: https://pharmapsychotic.com/tools.html for clip interrogation techniques.


text:
What programming language do I learn if I want to become artificial intelligence engineer?
I would become artificial intelligence engineer in the future, so I want to start programming but I don't know what programming language I start for that.

Hi Mamadou, I understand that starting something can be challenging, and thank you for your question. I would rephrase it to, "What programming language has the biggest community support and documentation for designing AI models and algos?". This is because practically any language can be used to design and train AI models. In answer, I think it will benefit you to start learning python for the same cause.

Python is a great programming language to start with if you're interested in becoming an AI engineer. It's widely used in the field of AI and machine learning due to its simplicity and large community support. It also has numerous libraries and frameworks like TensorFlow, Keras, PyTorch, and Scikit-learn that are specifically designed for AI and machine learning.
After you're comfortable with Python, you might want to learn other languages that are commonly used in AI like Java, C++, and R. However, Python is definitely a good starting point.

Any language will do. Most differences are just in the syntax and the idioms of a particular language, but you can usually pick up a new language fairly quickly.
Having said that, there are some languages that are more common in certain areas. Python seems to be quite popular in a lot of modern AI work. I myself work in natural language processing, and the company I'm at uses Lisp.
My suggestion would be to start with some easier languages, and keeping an open mind about learning about other languages. The more you know, the easier it is to switch between them and to learn new ones.


text:
What is an appropriate tool to use that takes in a large knowledge base in string form and can answer questions based on the knowledge base?
I have an issue where I'm trying to use the openAI API to input a very large custom knowledge base (exceeding 1GB) that allows the user to ask questions based on that base to receive intelligent answers. However, the openAI API is very restraining in the tokens and requests that can be inputted, and I was wondering if there is another tool that is more suited for my problem. I figured it would have to utilize an LLM to understand and parse the info.
I tried different measures such as smartly looking up specific sections of info then inputting a smaller chunk it into GPT, but I'm still largely restricted by the API's restraints.
I'm quite new to this, so any ideas and suggestions would be appreciated

Perhaps this is a good case for training your own model on your knowledge base?

The model needs to be able to make inferences between different documents in the knowledge base. My goal is to have something with the intelligence of an LLM like ChatGPT, and I don't think training my own model to do that is feasible. I'd like to know if something like that could be done though.

It sounds like you need a method to determine what subset of your data is necessary for a given query. There's a lot of different methods for document retrieval. I'd start with something simple like BM25 or Tf-IDF, the sentence-transformers library also has some models you can try for semantic search.
Depending on how complex your question is, you can also have the LLM generate sub-queries and retrieve documents based on those as well.


text:
How are Target Values Generated in Alpha Zero Architecture
I am a little confused as to how the target values are generated to train the neural network with the Alpha Zero architecture(in specific to a chess game). I understand how the improved policy is generated from the number visits to each child node. In a lot of different sources they say that a training sample is generated by keeping track of (state, policy,_) where _ is the eventual win or loss that occurs in the game. This would mean that if white eventually won the game, every single state that white had in the Monte Carlo Tree would receive a value of +1. Isn't this very counter intuitive since there were clearly suboptimal moves that white chose not to take(for example moving a queen out into direct danger to be captured) but these states will still be marked with a +1. Wouldn't it make more sense to just label the state/moves that directly led to the win(traversing up the tree) instead of marking every considered state based on the outcome of one path.
Another question I had was instead of training the values of every state on the eventual win or loss +1/-1, would it not make more sense to train it against the final Wins/Visits value that was calculated when conducting simulations on the tree. Wouldn't Wins/Visits be a much more representative for the average value of the state instead of declaring it to be completely a +1 or -1 since there are probably particular subbranches that could result in a win and some that could result in a loss. Sure the target values in the beginning of training would be very inaccurate, but as the network gets more experience with states right before the end of the game, the accuracy of the value predictions would improve upwards the tree.

I think you are mistaken in how the target value (or value function) is computed. For example, the statement "if white eventually won the game, every single state that white had in the Monte Carlo Tree would receive a value of +1" is imprecise. Not every state, but only those nodes on that branch is +1.

I suggest starting from the earlier paper of AlphaGo  which contains more details on how the target value etc. are computed. Later papers assume readers are already familiar with the topic and skip much of the details, leading to confusion.


text:
LSTM Ensemble: Combine low, mid, and high frequency time series data
I am trying to implement time series classification, but I am struggling a bit with the fact that my multivariate data has mixed frequencies. I have about 10 variables that are updated every minute, 10 features that are updated every 2-3 hours, and some low frequency variables that are updated every day.
The prediction frequency should be quite high at 15 minutes. So the obvious way might be to group variables within 15 minute intervals. However, this works well for the high frequency variables that occur each minute, but not for the low frequency as within 15 minutes no new values occur.
Maybe you know something that could help me and give me some sort of starting point?
I had an idea, but I have never seen such an approach in practice. For the medium and low frequency variables, could we just go back 3h and 1 day respectively, starting from the time of prediction, and aggregate the data in that interval. One could then form several of these packages. However, this would mean that one could recalculate the input sequence for each time point and not simply send a new data point as input into the model which has retained its state from the previous inputs.
Another idea would be to simply pass in the last raw values of the variables along with some timestamp when they occured. However, I'm a bit afraid that the timestamp introduce some unwanted correlation.

Is the low frequency data inherently slow changing, or is it faster changing but only sampled at a low rate? Is it actually meaningful to predict new values for it with 15 minutes intervals?

no i don't think it makes sense to predict new values for it with 15 minutes intervals it is rather slowly changing


text:
How do we get from types of activation functions to fitting lines to our data?
I'm completely new to AI and admittedly have never been good at math (also please excuse me if I use the wrong terminology). Despite this, I'm trying wrap my head around activation functions and how they are used by a neural network to fit a line to a set of data.
I understand the purpose of an activation function, but what I'm struggling with is how we go from deciding which type of activation function to use for each neuron to actually fitting a line to our data.
In an attempt to illustrate this question, here is a YouTube video which demonstrates how a fitted line would change as data is passed through a neural network using various types of activation functions.
At the moment, it just seems like magic that we pass data through and depending on which type of activation function being used the line somehow changes in various ways and I have no idea why it's doing what it's doing.
Any help with this or guidance regarding what knowledge gaps I might have that are preventing me from understanding this would be greatly appreciated! My goal in all in asking this is to be able to do the math by hand in a much smaller network to demonstrate that I understand what's happening behind the scenes.

Most of the commonly used activation functions for neural networks work reasonably well on most machine learning tasks. You do need to care about the function used in the output layer though.
For hidden layers, the activation functions do change the possible shapes that the final output can make easily. The basic shape of each neuron's output is the activation function shifted left or right, and in the next layer those shapes can be combined in any multiple (including negative weights that invert the shape vertically). However, once you have a few dozen neurons and 2 or more layers, the differences between what you can make are subtle and difficult to predict. Whilst it is possible to tune a neural network by trying different activation functions in hidden layers, it is more common to pick one that works well enough - some variation of ReLU is a reasonable choice, especially for deeper networks.
The output layer is an exception. You want the output to match the range of your training data, and ideally naturally represent the output type for your problem.
There are three very common choices for output layer activation function:

For regression problems, do not use any activation function in the output - sometimes this is called the "linear" activation function.
For single classifiers or arrays of non- exclusive classifiers, use sigmoid activation to represent the probability of being in the class(es)
For multi-class exclusive classifiers, use softmax activation which naturally sums to 1.0 over all the outputs and represents probability of being in each class.

In all cases, the activation function does not affect the how of fitting the output to the data. That is done by calculating the gradient of all the parameters in the neural network with respect to a cost function (which measures how close the outputs were to the data) and taking a small step in the opposite direction to the gradient, which should reduce the value calculated by the cost function. This is repeated many times - measuring the gradient again and taking a new step in a different direction each time. This is gradient descent. Calculating the gradients is done using back propagation, which is based on the chain rule for differentiating combined functions. The activation function is one of the functions being combined, so is the sum of outputs from the previous layer.


text:
what is the best way to feed GPT a scientific paper with a lot of formulas
I often wish to ask GPT to read a scientific paper with a lot of formulas but I run into a difficulty. Usually a scientific paper is in pdf format and there are a lot of formulas. If those formulas are in latex form, GPT can indeed understand them precisely which is evidenced by that it can write codes that precisely translate a formula (according to my experience). But what if, like in a pdf, the formulas are not in latex form? If I copy and paste them the formulas usually become non-readable. I know that one can use some tool like mathpix that ocr a picture into latex with quite high fidelity. But that is still quite inconvenient as there may be ocr-errors which still need significant human inspection to make sure to eliminate. But openai must have fed scientific textbooks with loads of formulas into GPT in the training process in the first place, I suppose? So I suspect there does exist a convenient way to feed GPT a scientific paper in pdf format with a lot of formulas. If so, how do one do that?

There is an example "book_translation" in the OpenAI cookBook that automatically translates, chunk by chunk, a Math-book from the Slovenian language to the English language.
Maybe it helps to study the source code of that python notebook to solve your problem.

Translate a book writen in LaTeX from Slovenian into English
With permission of the author, we will demonstrate how to translate the book Euclidean Plane Geometry, written by Milan Mitrović from Slovenian into English, without modifying any of the LaTeX commands.
To achieve this, we will first split the book into chunks, each roughly a page long, then translate each chunk into English, and finally stitch them back together.


text:
Does Negative Prompting Exist?
All the prompt engineering techniques I've seen seem to focus on telling the model what to do e.g. Few-Shot Prompting.
Is there any value in giving the model examples of what not to do? Can you link me to any papers/techniques on the topic?
Example
I am building a bot to improve students' foreign language writing skills.
Bad output: Corrected spelling of 'heisse' to 'heiße' because 'heiße' is the correct spelling in German.
Better output: Corrected spelling of 'heisse' to 'heiße' because 'ss' can be combined to form 'ß' in German.
I could solve this specific problem using few-shot prompting. But really, I want to tell the model "don't give answers like 'this is how it is done in German', instead explain what is being done and the reasons for it".
I may have answered my own question there... just put it what I said above in the system prompt?

I think the question in the title is a bit vague and not very interesting. You can certainly create prompts where you ask the model not to do something. The more interesting question is - how effective are they and is there any research on the topic? So, I'd reformulate the title to be more specific. From my experience, if we ask the model not to be verbose, it can still be verbose. But this doesn't say much because if you ask the model to do something it might also not do it.

You can fine-tune davinci-003 for your case. With enough examples, it should start outputting the required information.

@nbro yes the idea of them being effective/whether research exists was the idea

@Chinmay so you're saying that yes you can do negative prompting and that it is effective?

The Super-NaturalInstructions paper tested including negative (i.e., incorrect examples) during testing and instruction-tuning, along with adding explanations for why an included few-shot example was correct or incorrect. It seems like it doesn't improve accuracy, at least in their configuration (see Table 4).
Although, if you're using a model like ChatGPT, as @nbro mentioned, you could probably just prompt the model to not do something.

Actually, my previous answer is a bit out of date. There's a recent paper that does what you're looking for: "PREADD: Prefix-Adaptive Decoding for Controlled Text Generation."
During inference-time, they do two forward-passes. The first is for your normal generation, and the second is prefixed with a prompt that causes undesirable outputs.
In your example, your normal forward pass could be: "Improve the student's grammar." and the bad forward pass could be "You are a terrible language teacher. Improve the student's grammar." You can then sample tokens that are unlikely under this second (bad) forward-pass, but likely in the first forward-pass. See Section 3 in the paper for the exact formulation.

Does Negative Prompting Exist?

Yes, e.g. in some text-to-image generation models such as https://app.leonardo.ai/ai-generations:

One can run such negative prompts on one's computer, e.g. with Automatic1111's Stable Diffusion WebUI (Windows 10/11 installer):

The paper {1} describes one example of how negative prompts can be implemented in LLMs.

References:

{1} Stay on topic with Classifier-Free Guidance. Guillaume Sanchez, Honglu Fan, Alexander Spangher, Elad Levi, Pawan Sasanka Ammanamanchi, Stella Biderman. arXiv:2306.17806.


text:
How is speech recognition software able to distinguish between different speakers and yet still understand them all?
Some background: I'm an EE major and data science minor, so I have a basic understanding of machine learning - I had a one semester course on it where we covered some of the most commonly used algorithms such as Naive Bayes' and K-nearest-neighbors, as well as the basics of how artificial neural networks work.  I've also had EE classes on signal processing, so I'm familiar with the Fourier transform and the like.  So I have a general idea of how different algorithms can, for lack of a better term, "pick up on" patterns in the training data and make predictions on the testing data, which will hopefully be reasonably accurate.  And I have a general idea of how signals can be represented as discrete binary sequences which could then be used to train an ML model.
What I want to know is, what sorts of patterns, in the mathematical sense, in audio recordings of speech specifically, do these algorithms find?  e.g., What allows the Alexa app to both recognize when I'm speaking vs when it's my grandma speaking, and yet still understand the same commands from both of us?  What, mathematically, makes, e.g., a recording of me saying "Alexa play music" both distinguishable from a recording of a different person saying the same command, and yet still recognizably contain the same information of the command?  In other words, how do the algorithms extract and match the speaker-specific and content-specific information from the speech signals?

Is there a reason you find this odd? After all, you understand yourself, your grandmother, etc.

@Dave, it's not that I find it odd -- it's just that I don't understand how it works and I want to understand.   Heck, I don't understand how our brains are able to do it either, but I don't really know anything about neurology beyond the basic facts most people know.  I do know the basics of machine learning well enough that I'm hoping I can get a high level understanding of the parts of the math behind speech recognition that distinguish between speaker specific and content specific information.  I can't think of a particular reason I'm curious about it -- I just am.

I would slightly disagree with Ryan's answer: the fundamental frequency is mainly specific to a speaker. Sounds are defined by other frequency patterns.
Vowels, for example, have two bands of energy in different frequency ranges, called formants. Depending on the speaker, these will be in different actual frequency ranges, but the general pattern will remain the same. They are linked to the fundamental frequency F0 in that it is multiples of F0 which can be seen in a spectrogram.
There is obviously  lot of variation in the exact frequencies involved because of circumstances. Apart from different speakers with different physiology, having a cold also changes your frequency output. For speech recognition you need various mechanisms to map a sequence of frequency values to phonemes and then words. When I was studying phonetics in the 1990s, the upcoming tool was Hidden Markov Models, and neural networks were being explored as well, but were still in their infancy.
I'm not sure what currently is used for speech recognition, as I haven't done any phonetics for years, but I would expect NNs to be more prevalent these days.
To summarise: there are some features in the frequency pattern of an utterance which give you an indication of the (language) sounds produced, whereas others reflect the physiology of the vocal tract (and are thus speaker-specific). But they are not unique, as there is no "voice print", which was some time ago touted as a finger print equivalent.

A sound byte can be decomposed into a set of features which are distinguishable by a classifier. The most important feature extracted is the fundamental frequency. This is the lowest frequency function from approximating the windowed waveform with Fourier series. It is fairly unique for each phoneme. In addition, each phoneme can be represented as a series of "frequency bins" where the intensity is the height of each bin. These features are not only unique to each phoneme, but also to each speaker. Additionally, statical moments can be calculated for each window and used as features.
A Recurrent Neural Network (there are many techniques) can be used to agglomerate features into a recognition model. The idea is to learn the probability of one set of features following another set in time.
For an example, a 20 second sound byte of a speaker can be broken into 20 1-second windows of speech. Each window represents a phoneme or some part of speech (depending on window-size, speaker speed, etc). A Fourier transform is applied on a window to derive a feature set. The feature set is fed into an RNN to predict the speaker (or text-to-speech, TTS). There are many inter-related applications here.
Hope this was helpful.

"A sound byte can be decomposed into a set of features which are distinguishable by a classifier."   What If like to know is, which classifiers are relevant to distinguishing between different speakers and which are relevant to distinguishing between different messages?  e.g. If had a recording of a person with a British accent accent saying the word "water" and another recording of a person with an American accent saying, which features of the data would recognize the different accents and which ones would "recognize" (for lack of a better term) that it's the same word being spoken?

Typically there are (at least) two stages. The first stage picks up features like phonemes and syllables. The second stage identifies specific words and picks up things like which homophone was meant (e.g. they're or their).

@MikaylaEckelCifrese the same models can be used for both. RNNs, LSTMs, transformers, etc can all be used for either task. The difference is in the target for training which comes down to calculating loss for updating the network. Is the network "output" a vector that represents a speaker or a word. This is training dependent. I would recommend becoming more familiar with ML in general before tackling this specific task.

@Ryan, but about when the software needs to recognize both speaker specific and content specific information?  Also, I don't currently have any intention of trying to make a speech recognition system, as that's definitely beyond my current skill level -- I'm just curious about how such software works at a high level.

@MikaylaEckelCifrese, Many features can be re-used for both tasks, however most models in practice will have common feature extraction layers and then have models that perform specific tasks. There might be several layers on top of the base extraction for one task, and separate layers for another task. The software has multiple models with some common layers.


text:
Who invented DAN?
DAN was a prompt that went through many, many iterations during the initial months of ChatGPT’s release to the public. DAN is an acronym which stood for “Do Anything Now”, and was a prompt specifically designed to circumvent the grid lines OpenAI had set to contain ChatGPT’s responses, under its alignment.
DAN stimulated a lot of interaction on related forums like Reddit, where people shared DAN-type output and modifications of the DAN-prompt.
DAN was essentially a first noteable example of prompt injection, in the LLM field, where an attacker can modify the behavior of an LLM system from what its creator intended and programmed it to do.
Out of historical interest, is it known who invented the DAN prompt - which individual or individuals played a role in the evolution of DAN as a cultural phenomenon, and theoretical finding, in the field of prompt engineering?

DAN is invented by a college student Walker, who first posted the prompt in this Reddit post [1].
DAN then became particularly popular. However, there were countless examples of earlier "jailbreaking" prompts on day-1 of ChatGPT. In particular, the "split personality" idea appears before. See, for example, this video, the part from 20:17 by Yannic Kilcher for a summary of various people's Twitter posts.
The idea of the prompt injection attack predates the release of ChatGPT. Riley Goodside posted in this Twitter post a basic "ignored previous instruction" attack against an OpenAI's text-davinci model. Note that this model doesn't have a "safety" filter built-in, so it is not an example of "jailbreak." See also [2].

[1]: Oremus, Will. "The clever trick that turns ChatGPT into its evil twin". Washington Post (2023).
[2]: Fábio Perez, Ian Ribeiro. Ignore Previous Prompt: Attack Techniques For Language Models. NeurIPS ML Safety Workshop (2022).


text:
How is the number of channels in a convolutional layer shrinked or expanded?
I know in order to shrink or expand the number of channels a 1x1 convolution is performed.
I need to clarify the following: is the 1x1 convolution(s) just a matrix multiplication between the image with shape (hw, 3) (RGB) and a matrix that holds the learnable weights with shape (3, 1)? Which will result in a new matrix of shape (hw, 1) (in this case the number of channels shrunk from 3 to 1).
If the above is correct, what happens under the hood of a NN framework, such as PyTorch, when the number of input channels is equal to the number of output channels? Does a matrix multiplication take place between the input (h*w, 3) and a matrix with learnable weights (num_channels, num_channels)? Doesn't this introduce unnecessary (and unwanted) operations?

You can think of applying a point-wise, i.e. $1\times 1$, convolution with $F$ filters to a $H\times W\times C$ image (or feature maps, in general) as the application of a linear fully-connected layer (with $F$ units) to each spatial location of the input, resulting in $H\times W\times F$ tensor.
In general, you can see the application of a convolutional layer (regardless the kernel size) as a matrix multiplication between the kernel matrix, say $K$, and the flattened image matrix, say $I$, where: $K$ is a block diagonal matrix (you have zeros outside the diagonal) build by kind of stacking the kernel (the weights of the conv layer) $H$ times as follows:

then $I$ is obtained by flattening the input image. After the multiplication you reshape the results back to a 3d-tensor shape.
In the example you have a $1\times 2$ kenel (reddish vector), and a $3\times 3$ image. So, you should consider for the matrix multiplication a stack of $2\times 3$ kenels ($k\times W$, in general) and a reshaped image of size $(H\times W,1)$.

what happens under the hood of a NN framework, such as PyTorch, when the number of input channels is equal to the number of output channels?

For practical implementations I don't think the matrix formulation of conv layers is beneficial, especially if you consider GPUs which have many threads and you can basically let each thread access a location of the input tensor, compute the respective index in the kernel, do the product and finally sum. Now if $k=1$ (as in pointwise conv), the matrix $K$ is built from $1\times W$ matrices, stacked $H$ times, resulting in $(W\times H)\times H$ kernel matrix, which is also mostly sparse. Maybe with ad-hoc algorithms for sparse multiplication this operation may be convenient.

It is useless to use a 1x1 convolution if the number of output channel is equal to the number of input channel, as it allows no additionnal expressivity for the network and just adds useless computations. You can however add a ReLu between the two layers to make it useful as the linearity will no longer hold.
If you use a 1x1 convolution (or more precisely, a 1x1xc convolution where c is the number of input channels), then each output channel will be a linear combination of the input channels. The coefficients of these linear combinations corresponds to the coefficients of the different 1x1 kernels.

This answers one of the questions but does not answer the main question which is what does that mean implementation-wise? Does going from N channels to M channels involve a matrix multiplication between the input of shape (height*width, N) and a weights matrix of shape (N, M) ? Apologies if the question above was not clear.


text:
How to Formulate a realiable ChatGPT Prompt for Sentiment Analysis of a Text, and show that it is reliable?
I have a dataset which consists of like.. 400000 sentences and I want give each sentence to ChatGPT so it classifies each sentence as positive or negative. My question is, where can I find a reliable / trusted prompt to do that? and provide evidence that the prompt I used gives reliable labels for the problem?
If I just create my own prompt like.. "Please conduct a sentiment analysis on the following text and tell me if the sentiment expressed is positive or negative: [insert text here]"
How can I convince someone that my prompt was good? This is what I mean :/ So later no one can criticize like "Hey maybe you just used a bad prompt and ChatGPT could have performed better!" :c
I hope my question is clear.

Is there a reason not to use something like a RoBERTa model specifically trained on sentiment classification? Such a model would always return a machine-digestible output (versus ChatGPT, which would not necessarily), would probably have higher accuracy due to being trained on just this one task, and could be fine-tuned specifically for your domain (versus ChatGPT which is basically closed-source now). You could then feed that data to ChatGPT or another such chatbot for a human-readable summary if necessary.

You would also know that your inputs were not being used as training data for a public-facing model, so they could not be estimated from an attack on the model or sold for advertising to third parties or whatever.

The problem with using ChatGPT with sentiment analysis is exactly that; it can't be relied upon. But, we can increase our chances of correct output by using chain-of-thought prompting with few-shot examples.
If you don't know CoT prompting and few-shot prompting, please refer to this.
Now, for your case, you can use a prompt like this-
Classify the sentence as positive or negative.

Think step-by-step, and provide reasoning why the text is identified as positive or negative.

For example,

Sentence: 'Same S##t, different day'

Sentiment: The sentence uses s##t. S##t is publically accepted as vulgar or profane. So, the sentiment is Negative.

and

Sentence: 'This product has changed my life, I am so much more productive now'

Sentiment: The product that is being talked about has made the user more productive. Productivity is generally associated with positivity. So, the sentiment is Positive.

Sentence: <<Paste your own sentence here>>

Sentiment:

Here, we are asking ChatGPT to provide reasoning for its decision and also use our examples to remember how step-by-step reasoning works.
Now, while this should work well, it still will give bad results once in a while as ChatGPT is finetuned on human textual data and separately forced to accept some things as positive or negative.
Consequentially, a sentence that you consider as positive/negative might not be looked upon in the same way by ChatGPT.
UPDATE: According the comment, it is absolutely correct that even seemingly vulgar words can be used in a positive context, a modified prompt in that case will look like this-
Classify the sentence as positive or negative.

Think step-by-step, and provide reasoning why the text is identified as positive or negative.

For example,

Sentence: 'Same S##t, different day'

Context: The context seems to be that of frustration where the speaker is frustrated and/or angry at the metaphorical same s##t or mundane tasks.

Sentiment: The feeling is that of frustration and boredom about something. So, the sentiment is Negative.

and

Sentence: 'This product has changed my life, I am so much more productive now'

Context: The context seems to be that of admiration of a product that helped the user in some wah and made their life easier.

Sentiment: Productivity is generally associated with positivity. So, the sentiment is Positive.

Sentence: <<Paste your own sentence here>>

Sentiment:

I mean... I definitely wouldn't train it that anything using vulgar language is negative... c.f. "this product is f###ing sh#t", vs "this product is the f###ing sh#t". :D

You are given the text of reviews delimited by triple quotes.
Your task is to determine the sentiment of the review in the steps:
1, Identify a list of emotions that the writer expresses, not just positive nor negative.
2, Quote the phrases that are related to the identified emotions.
3, Give the sentiment of the review in a single word either "positive" or "negative" and verify that the sentiment is correct.
4, Format your sentiment in json. The permissible values of sentiment are "positive" and "negative".

"""I have mixed feelings about this fitness tracker. On one hand, it accurately tracks my steps and heart rate, and the app is easy to use. However, the battery life is not very long and I find it uncomfortable to wear for long periods of time. I also think it's overpriced for what it does. So, while it has some good features, I'm not sure if I would recommend it."""

Give it steps to take time to think to avoid jumping to conclusion.  The outputs from step 1 and 2 can be used for spot check its accuracy.
It would be optional but safe to ask it to verify it for you.
Finally, provide you an easy way to parse the output.
The test result with ChatGPT-3.5 is provided for your reference:

Emotions: mixed feelings, accuracy, ease of use, discomfort, overpriced, uncertainty
Quotes:
"mixed feelings about this fitness tracker"
"accurately tracks my steps and heart rate"
"the app is easy to use"
"the battery life is not very long"
"find it uncomfortable to wear for long periods of time"
"overpriced for what it does"
"while it has some good features, I'm not sure if I would recommend it"
Sentiment: Negative
Formatted sentiment: {"sentiment": "negative"}

The sentiment of the review is negative because while the fitness tracker has some good features such as accurately tracking steps and heart rate, and having an easy-to-use app, the negative aspects such as short battery life, discomfort, and overpricing outweigh the positive ones. Additionally, the uncertainty expressed by the reviewer about recommending the product further indicates a negative sentiment towards it.


text:
Fit Q Evaluation in offline reinforcement learning
I am working on a PyTorch implementation of Implicit Q-Learning (IQL) (paper), given a dataset $\mathcal D = \left\{ (\mathbf s_i, \mathbf a_i, \mathbf s_i', r_i ) \right\}$ of transitions. I think I have implemented IQL correctly, so now I have a learned policy $\pi$ that takes in an element of the state space $\mathcal S$, and outputs a mean vector and covariance diagonals for a multivariate normal distribution over the action space $\mathcal A = \mathbb R^d$. I would like to evaluate this learned policy $\pi$.
This leads me to investigate off-policy evaluation, specifically Fit Q Evaluation (FQE), given a test dataset of transitions $\mathcal D_e$ as above. I have been reading the offline reinforcement learning paper by Prudencio et al. (link). My problem is that I do not understand how to implement FQE as described in the paper, so I would appreciate guidance for this. Below I quote the relevant part of the paper and ask some questions.

C. Fit Q Evaluation
In fit Q evaluation (FQE), we first train a $Q$-function $Q_{\phi}^{\pi}$ by minimizing the Bellman error under the policy $\pi$. Then, we evaluate the policy by computing the average expected return over the states and actions from $\mathcal D_e$, such that
\begin{align*}
\hat J(\pi) = \mathbb E_{\mathbf s, \mathbf a \sim \mathcal D_e} \left[ Q_{\phi}^{\pi}(\mathbf s, \mathbf a) \right].
\end{align*}

Questions:

Presumably I should use a neural network for $Q_{\phi}^{\pi}$, right?
What is the Bellman error in this context? I have some course notes that give a definition of the Bellman error involving $\max_{\mathbf a'}$, which is not workable since my action space $\mathcal A$ is $\mathbb R^d$ for some $d \in \mathbb Z$.

I have looked at the off-policy evaluation paper by Voloshin et al. (link), which seems to describe FQE in a different way. Specifically,

$\hat Q(\cdot, \theta) = \lim_{k \to \infty} \hat Q_k$, where
\begin{align*}
\hat Q_k &= \min_{\theta} \frac{1}{N} \sum_{i = 1}^N \sum_{t = 0}^{\tilde T} \left( \hat Q_{k - 1}(x_t^i, a_t^i ; \theta) - y_t^i \right)^2\\
y_t^i &\equiv r_t^i + \gamma \mathbb E_{\pi_e} \hat Q_{k - 1}(x_{t + 1}^i, \cdot ; \theta)
\end{align*}

If I understand this correctly, we fit a sequence of neural network $Q$ functions $\hat Q_1, \hat Q_2, \dots$ here, trying to minimize the trajectory-average squared distance between $\hat Q_{k - 1}(x_t^i, a_t^i ; \theta)$ and $y_t^i$. But how do I calculate $y_t^i$ here? I don't know what $\mathbb E_{\pi_e} \hat Q_{k - 1}(x_{t + 1}^i, \cdot ; \theta)$ means.
I appreciate any help.

Bellman error in general is the MSE error between your estimate of $Q(s, \pi(s))$ and your target, e.g. $r + \gamma Q(s', \pi(s'))$ (keeping notation simple here for how we obtain actions from policy). Your notes probably use a max because it will have been introduced in the context of Q-learning.

Did you check the original paper? See algorithm 3.
Batch Policy Learning under Constraints
Here is an implementation I found: https://github.com/clvoloshin/COBS/blob/master/ope/algos/fqe.py
See their OPE paper for details: http://arxiv.org/abs/1911.06854


text:
How do open source LLMs compare to GPT-4?
I have heard some back and forth regarding open source LLMs like Llama.
I have heard that on certain benchmarks they perform close, the same or better than GPT-4, but caveats that they tend to lack the diversity and range of GPT-4, and also fail to be equivalent in ways certain benchmarks or metrics don’t capture fully.
GPT-4 has about 170 trillion parameters, I believe?
It seems like the biggest open source models are all in the billions - like Bloom or the new Falcon 40b.
There are techniques where they refine GPT-4’s output into a smaller amount of training data that supposedly hits all the marks and does just as well; but again, I don’t know if that’s only true under the reductionist of view of a particular benchmark-questionnaire.
So, do open source models actually compete with GPT-4, and why or why not? Is the whole situation a matter of scale, that a commercial venture like OpenAI can foot the massive bill of training a multi-trillion parameter model that no open source AI project can afford, on top of them having expertise in model design, making GPT-4 continually the state-of-the-art? Or is there any open source model that truly can compare in terms of usability?

I recently came across this article - Open Source vs Closed Source LLMs. Pros and Cons, you can learn more.

The remarkable performance of GPT 4 is due to the massive size of its architecture and the amount of data it was trained on, which costs a lot of money. Few organizations have the hardware resources and funding to support such efforts.
Huggingface maintains a leaderboard for tracking open source LLMs. See Open LLM Leaderboad.
They provide this information on the benchmarks:
We evaluate models on 4 key benchmarks from the Eleuther AI Language Model Evaluation Harness , a unified framework to test generative language models on a large number of different evaluation tasks.

AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions.

HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models.

MMLU (5-shot) - a test to measure a text model’s multitask accuracy. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more.

TruthfulQA (0-shot) - a test to measure a model’s propensity to reproduce falsehoods commonly found online.
We chose these benchmarks as they test a variety of reasoning and general knowledge across a wide variety of fields in 0-shot and few-shot settings.

Here is a snapshot of the top 5 models:

For comparison you can see OpenAI's benchmark results here:

How do open source LLMs compare to GPT-4?

https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard has a leaderboard containing both open source LLMs and GPT-4 (and GPT-3.5-turbo):

Model
⭐ Arena Elo rating
 MT-bench (score)
MMLU
License

GPT-4
1193
8.99
86.4
Proprietary

Claude-1
1161
7.9
77
Proprietary

Claude-2
1134
8.06
78.5
Proprietary

Claude-instant-1
1130
7.85
73.4
Proprietary

GPT-3.5-turbo
1118
7.94
70
Proprietary

Vicuna-33B
1097
7.12
59.2
Non-commercial

Llama-2-70b-chat
1060
6.86
63
Llama 2 Community

WizardLM-13b-v1.2
1046
7.2
52.7
Llama 2 Community

Vicuna-13B
1046
6.57
55.8
Llama 2 Community

MPT-30B-chat
1043
6.39
50.4
CC-BY-NC-SA-4.0

Guanaco-33B
1036
6.53
57.6
Non-commercial

CodeLlama-34B-instruct
1032

Llama 2 Community

PaLM-Chat-Bison-001
1008
6.4

Proprietary

Vicuna-7B
1003
6.17
49.8
Llama 2 Community

Llama-2-13b-chat
999
6.65
53.6
Llama 2 Community

Llama-2-7b-chat
979
6.27
45.8
Llama 2 Community

Koala-13B
979
5.35
44.7
Non-commercial

ChatGLM2-6B
965
4.96
45.5
Apache-2.0

GPT4All-13B-Snoozy
964
5.41
43
Non-commercial

MPT-7B-Chat
943
5.42
32
CC-BY-NC-SA-4.0

RWKV-4-Raven-14B
939
3.98
25.6
Apache 2.0

Alpaca-13B
919
4.53
48.1
Non-commercial

OpenAssistant-Pythia-12B
911
4.32
27
Apache 2.0

ChatGLM-6B
896
4.5
36.1
Non-commercial

FastChat-T5-3B
888
3.04
47.7
Apache 2.0

StableLM-Tuned-Alpha-7B
859
2.75
24.4
CC-BY-NC-SA-4.0

Dolly-V2-12B
838
3.28
25.7
MIT

LLaMA-13B
814
2.61
47
Non-commercial

WizardLM-70b-v1.0

7.71
63.7
Llama 2 Community

WizardLM-30B

7.01
58.7
Non-commercial

Vicuna-13B-16k

6.92
54.5
Llama 2 Community

WizardLM-13B-v1.1

6.76
50
Non-commercial

Tulu-30B

6.43
58.1
Non-commercial

Guanaco-65B

6.41
62.1
Non-commercial

OpenAssistant-LLaMA-30B

6.41
56
Non-commercial

WizardLM-13B-v1.0

6.35
52.3
Non-commercial

Vicuna-7B-16k

6.22
48.5
Llama 2 Community

Baize-v2-13B

5.75
48.9
Non-commercial

XGen-7B-8K-Inst

5.55
42.1
Non-commercial

Nous-Hermes-13B

5.51
49.3
Non-commercial

MPT-30B-Instruct

5.22
47.8
CC-BY-SA 3.0

Falcon-40B-Instruct

5.17
54.7
Apache 2.0

H2O-Oasst-OpenLLaMA-13B

4.63
42.8
Apache 2.0

Also, some papers compare GPT against open-source models, e.g.  MeetingBank: A Benchmark Dataset for Meeting Summarization:

For non-English languages, see ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning.

In the "Fine-tuning" section of their end-user-documentation, OpenAU writes that

The more training examples you have, the better. We recommend having at least a couple hundred examples. In general, we've found that each doubling of the dataset size leads to a linear increase in model quality.

The context for this is training ChatGPT's model with one's own data, (not starting from scratch). However this might explain why the last few percent in high benchmark scores are so difficult to achieve for any model, including open-source models.
However a new paper "Textbooks Are All You Need" just came out (mid 2023) that argues that using higher input-data quality  matters more than quantity of input size. From the abstract:

using a selection of ``textbook quality" data from the web (6B tokens) and synthetically generated textbooks and exercises with GPT-3.5 (1B tokens). Despite this small scale, [our model] phi-1 attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP.

Also Sam Altman said something interesting during the Q+A Session of a public event at a university (it's on Youtube, cannot find the quote ):
It is not the parameter size,  it's the 200 little things you need to get right on the project-management level (esp. during the finetuning processs) that matter.


text:
Why can't Lucene search be used to power LLM applications?
w.r.t. LLM applications using the RAG (retriever-augmented-generation) architecture, people have started taken it for granted that it will be powered by a vector database. e.g., see this:

The most important piece of the preprocessing pipeline, from a systems standpoint, is the vector database.

Why can't lucene index (full-text search) be used for the retriever? Is there any objective study that has been done comparing quality of results using full-text search vs. using a vector database?
As I was writing this, even lucene seems to have jumped on the vector bandwagon. see this

At least the traditional Lucene full-text search is not vector based but an inverted index structure which can sort document relevance ranking via scoring functions such as the term frequency-inverse document frequency (TF-IDF) or the Okapi BM25 algorithm. On the other hand vector-based LLMs often use techniques such as cosine similarity instead for the new semantic relevance ranking which the traditional full-text search cannot satisfactorily achieve.

They are used. Tf-IDF and BM25 are pretty strong baselines for document retrieval, and papers on semantic search often include these methods as a comparison point.
e.g., papers like this one on open domain QA retrieve documents based on the question being asked. Vector representations perform better, but BM25 is still pretty solid, especially when performance is a bottleneck.

Because Lucene does not do a comparison of MEANING. It is, essentially, still a text search.
Vector databases are an attempt to find similar MEANING, even if the words are not identical. That is a totally different area of search. A standard vector (going by OpenAI here) has more than 1500 dimensions and identifies the similarity based on every dimension - Lucene would not even know the difference between "large village" and "city" while in reality they are QUITE similar. Or would not know that a "House" is close in meaning to a "Building".

Is there any objective study

No one reading the documentation would need that - they simply do too many different things. Heck, Lucene does not even HAVE semantic search built in. The difference is not in the search - it is in what is searched and how the vector for a vector database is constructed to try to capture the meaning, not the similarity in words. At the minimum you would need Lucene + a semantic search mechanism - and then you would have to calculate something like a similarity along a lot of axis. Also known as a vector.


text:
Can a Fully Connected Neural Network represent all Neural Networks of smaller size?
A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this setting, $a_0$ is the input dimension and $a_L$ the output dimension of the network. $L$ represent its depth (the number of hidden layers) and $W:=\max_{1\le i\le L} a_i$ represents its width (the hidden layer with the highest number of neurons).
With that formalism, given a set of parameters $\boldsymbol\theta := \Big((A_l,b_l)\Big)_{l=1}^L \in \Big(\mathbb R^{a_1\times a_0}\times\mathbb R^{a_1}\Big)\times\cdots\times\Big(\mathbb R^{a_L\times a_{L-1}}\times\mathbb R^{a_L}\Big)$, the function realized by a neural network with architecture vector $\mathbf a$, activation function $\sigma$ and parameters $\boldsymbol \theta$ is given by
$$\mathcal F_{\boldsymbol\theta} : x\in\mathbb R^{a_0}\mapsto \mathcal A_{(A_L,b_L)}\circ\sigma\circ\cdots\circ\sigma\circ \mathcal{A}_{(A_1,b_1)}(x) $$
Where $\mathcal{A}_{(A_l,b_l)}(x):= A_l x + b_l$ and $\sigma$ is applied componentwise. This mathematical formulation of Fully Connected Neural Networks is taken from, e.g., this paper by Berner et al. (2020) (page 3).
My question is now the following : consider a neural network with architecture vector $\mathbf a$, depth $L$, width $W$ activation function $\sigma$ and fixed parametrization $\boldsymbol\theta$. For any other architecture vector $\mathbf a'$ with depth $L'\ge L$, number of neurons in each layer satisfying $a_l'\ge W\ \forall 1\le l\le L-1$ and activation function $\sigma$, is it possible to find a parametrization $\boldsymbol \theta'$ such that
$$ \mathcal F_{\boldsymbol\theta} (x)= \mathcal F_{\boldsymbol\theta'}(x)\ \ \forall x\in\mathbb R^{a_0 }\ ?$$
(of course, we need to assume $a_0' = a_0$ and $a_{L'}' = a_L$). In words, I am asking : can Neural Networks with more computation units in each layer AND more hidden layers represent ALL neural networks with fewer computation units and hidden layers ?
Intuitively, I am pretty sure that the answer is yes : for instance, if we don't increase the depth but only the number of neurons in each hidden layer, we can simply reproduce the smaller network with a properly chosen subset of the units, and set everything else to zero. The problem if we add more layers and do as I just said is that we now need to have the identity function from the $L$-th to $L'$-th layer, which is not feasible exactly for any reasonable activation function (I am mostly interested in ReLU, in which case a good approximation on a compact set is possible, but not an exact equality). Because deeper networks are more expressive, I would bet that the result still holds in that case.
I would appreciate an answer containing a proof and/or a reference (I am fine with adding more assumptions if needed). Thank you in advance.

So, you're really interested just in the case where all activation functions are equal (in both neural networks) and cannot be changed from layer to layer?

@nbro yes, exactly, the activation function is the same throughout all layers for both networks (it could be a ReLU activation for instance)


text:
Fine-Tune Llama on main and auxiliary task
I am trying to fine-tune Llama model on two task at the same time, using hugging face library:
Main task: Causal language model like the model was initially trained for
A classification task based on the whole input sequence (recommend an article). For this task I am getting as a reference the LlamaForCausalLM class, overwriting init and forward functions .
However, I want to combine the two tasks above into one process. The main problem is that language modelling is an iterative process were the loss is calculated for every new context token in the input sequence, while for the classification task the loss should only be calculated once.
How can I freeze the loss update on the classification task up and only calculated once the language modelling part has been completed. Is there any example you can recommend in order to combine a main LM task with an auxiliary classification task?
First question for me here, thanks everyone for your understanding.

After your forward pass, you can always just use the logits and calculate the loss yourself. In this case, you'd only use the loss on the last input token.
Huggingface also has a LlamaForSequenceClassification class, which does exactly this.


text:
Effect of large activations of hidden layers
The example is trying to predict wether coffe is well roasted or badly. 1 is good roasted and 0 is bad. The architecture is:
Now I try to visualize the model.

Unit 1 has higher values when the duration of roasting is too little. Unit 2 has higher values for bad combinations of temperature and time. The blue-hatched regions show the layer outputted higher activations for the respective unit.
I used a threshold of 0.5
So I can conclude higher activations from layer 1 give an output class of 0. But as I used the sigmoid activation function I thought that large values give an output class of 1. And very small values give a 0, due to the S-shape.

Can you please highlight the question? Now sure what to answer.

Maybe you have negative weights in your output layer. What are the 3 weights that compute a2 from a1 ?

if it's a 2 layer network or more, you cannot reason anymore like this

The sigmoid activation is defined as follows: $\sigma(x)=\frac{1}{1+e^{-x}}$, this means that for $x\to\infty$ the activation saturates to one whereas for $x\to-\infty$ it saturates to zero.
In your example you're trying to visualize the activation per neuron, in feature (or input) space. What I see is that if you sum the shaded blue regions (the ones with activation close to 1) you are able to identify the positive class quite well. Basically, each neuron is learning to threshold one side of the triangle where the negative class resides.
The output layer can sum these activation and rescale such that a further sigmoid, returns values close to $1$ for points that fall outside the triangle.

I can conclude higher activations from layer 1 give an output class of 0.

I don't see how you can say so. You shouldn't reason in feature space, but rather also account for the value of the weight that multiply each feature. Maybe, you want to say that, for a given neuron, a lower value of the feature corresponds to a higher activation but that depends on the sign of the weight the feature is multiplied with.


text:
In theory, LSTM should be adequate to solve my problem, but it doesn't
I am currently doing my Master's Thesis in the area of wireless sensing using radio waves. I have 4 receiver distributed in the corners of my room and each of them records so called Channel Impulse Responses in the time domain. This is basically a correlation of how the radio wave is influenced by the room and they look like this:
https://www.researchgate.net/figure/LOS-channel-impulse-response-and-NLOS-channel-impulse-response_fig1_308986067
Each CIR is 300 samples long in my case and given $i, j$, $j$ is dependent on $i$ if $j\geq i$.
Thus, the data I feed into my LSTM has the size Nx300x4, where N is the batch size.
Why I thought LSTM should work here is that the correlation from each sample to the previous samples for each receiver directly relates to the changes happening in the room (not within one single 300x4 sample but over multiple samples). You can think of the data as 4 sentences with 300 words each.
The only thing that is a bit confusing here is that the LSTM implementation (at least in PyTorch) returns a vector with size 300xhidden for a single input.
I am reducing the 300xhidden vector to the number of classes I have using a flatten + linear layer but I am not sure if this is correct?
Nonetheless, the LSTM is much worse than my CNN implementation - but why? If I may bring back the analogy to the 4 sentences with 300 words - wasn't LSTM the NN of choice for text analysis before transformer?

From what you described, you seem to not be using the time-series capabilities of the LSTM. Try to convert your data into sequences (2D shape to 3D). Something like, [samples, TIME_STEPS, features]. You can also do this with some signal pre-processing like using FFTs on your radio signals as the raw signal interpretation by the LSTM might not produce that much value.

Isn't [NumSamples, TimeSteps, Features] exactly what I am doing? NumSamples is the batch-size, TimeStemps is 300 (in my case) and features are the 4 receivers.

Not really. Right now, you have in one batch 300 data points by 4 channels/receivers (N, 300, 4) whilst with, for example, if you pre-process with an FFT with window size of 30s (assuming random time-frame and frequency) you would end up with something like one batch with (N, 300, 30, 4). See this example: https://towardsdatascience.com/time-series-of-price-anomaly-detection-with-lstm-11a12ba4f6d9

I am not sure if I can plug a 4D vector into PyTorch's LSTM-Model: https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html
The input is of size (N, L, H_in).


text:
Does 1 Training Example (Approximately) Fix 1 Parameter of a MLP network?
I'm thinking from the math perspective of: "you need as many equations as unknowns". And thinking the loss on each training example (with 1 output variable) as one such equation.
Also I realize that in practice (if say you use ReLU), some neurons may be "dead" right off the bat & that SGD isn't perfect, so it's unlikely to be "exactly true". But I'm wonder if under ideal circumstances it is true or if it is approximately true in general.
The intro of this paper seems to imply that a model with N parameters fit to N training examples could be a perfect fit... But they don't directly state that...

If your model is a linear one that can fit perfectly, with no noise, and no redundancy in the data, then you could in theory solve it as a system of linear equations. You can think of that as one equation with new data (not simply a linear combination of existing equations) reducing the number of free parameters - it doesn't fix any specific parameter, but reduces the number of them that you can freely choose to have any value before the remainder must be set logically to make all the equations balance correctly.
If you keep the "perfect fit" requirement, and under some other assumptions (such as bijective nonlinear functions) then this also applies to some non-linear models in principle, although there may be no direct way to calculate the correct values and with layering, as in neural networks, there will be multiple equivalent solutions.
However, neural networks are usually trained without assumptions of noise free, non-redundant data or requiring a perfect fit. You might see that in some toy approximation examples, but those are not real use cases for neural networks (no-one really uses a neural network to calculate xor in any production system).
There is still a loose relationship between number of trainable parameters in a neural network, and the dataset size that will help to train it. Many factors affect that though, so it is more common to explore the values experimentally, by plotting learning curves where the amount of data used to train is varied. And this is something that state-of-the-art developments do explore - for example with the latest batch of large language models like GPT, a ratio of 20 text tokens in the training data per free parameter in the model has been suggested.


text:
StyleGAN runtime phenomenom
I was playing around with MobileStyleGAN pretrained model and multithreading and came along with an interesting phenomenom. After a while application is running MobileStyleGAN starts to produce video clip alike pieces.
And I am wondering does anyone have an idea what takes?
I made this lengthy video for it where you can see since it looks cool; https://youtu.be/8kpxHZK1NTQ
Here's the source code used in the video if it's any help; https://github.com/harism/i_style_gan

Here's one more video while trying to see how looping the GAN component output a few times into input react to all this; https://youtu.be/iVd7rve5Ja8

not really familiar with the model, but I heard of similar issues usign batch norm in GANs, as the EMA for $\mu$ and $\sigma$ creates dependence of the batch, and the batch history

@AlbertoSinigaglia could be I do not know GAN at all actually, I'm just amazed about it the model keeps producing human like images no matter what I do. I'm more used to see randomness produces different kinds of images. However, if you compare this state I got GAN into, could it be it's a bit same as LLM hallucinating, i.e I believe no one taught the model all this.

I will answer my own question and state it briefly that yes StyleGAN models do indeed come with sort of an gesture mapping. I have not tried those properly since I believe they are mainly computed with original StyleGANx models from The NVidia Helsinki office. Instead the smaller model made with smartphones as the target devices in mind MobileStyleGAN seems to have similarities too - and this is what I faced while writing my original question here.
To support my findings I leave this latest realtime capture of the current state I have achieved so far with MobileStyleGAN;  https://youtu.be/FqxbTijK41g


text:
Noob crafting a simple "Zero-Shot Classifier" Using an API . How can I avoid passing the categories every single request?
I have a collection of 700 categories, all potential classifications for articles. My current need is to create a system that can dynamically categorize short texts or articles according to these 700 categories.
I've been experimenting with a rudimentary approach using chatGPT to read the categories from a PDF via a plugin. The process is quite straightforward - I input the title and the first two lines of an article, and chatGPT does a fairly decent job of predicting the most fitting category.
The downside? I'm concerned about its scalability and economic viability. The current method might not work so well when we're talking about classifying a significant number of articles.
My question to you, my fellow AI enthusiasts: How would you approach designing a system, via an API, capable of doing this quickly and on a large scale?
Is there a feature that allows the Language Learning Model (LLM) to retain the list of 700 categories in its memory so that I don't have to pass it every time? I'm aware that the billing structure is token-based, so it would be ideal to submit the categories once (or as few times as possible) and then pose a simple query like:

"Categorize this article based on the categories I previously gave you. Article title: 'Barbie vs Oppenheimer: Which Movie Will Garner Greater Success?'"

Ideally, I'd want this system to be persistently active and capable of processing countless queries over an extended period, say a month or a year.
So, any ideas on how to design such a system? There are undoubtedly numerous routes to take. I'm really just seeking some initial direction so that I can dive deeper into research on my own.

There's been a couple ideas thrown around wrt the task of "prompt compression". For example, the gpttrim algorithm, applies some basic NLP preprocessing to reduce unnecessary tokens (e.g., removing stopwords). Others have tried prompting ChatGPT to compress text.
There's a couple things to be careful about with these techniques, though:

Make sure the number of tokens is decreasing, not the number of characters. You're charged per token.
Such compression is usually lossy, which for your use case of category labels may not be ideal.

A better approach may be to reconsider your classification pipeline, keeping in mind the accuracy and cost tradeoff of different algorithms. For example: 700 categories is a lot, and I'm sure there's a significant portion that are "obviously wrong" for a given article. Going off of this, you could try some cheap, classical NLP techniques (e.g., TF-IDF + logistic regression) to narrow down the set of possible predictions first, then use the more expensive API.

Thank Alexander, can you give me an example of TF-IDF ?

Here's an example using scikit-learn: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html


text:
Which algorithm for production scheduling with multiple goals - alternative for genetic algorithm
I am currently using a genetic algorithm for optimising the production schedules of a factory that produces bespoke insulation panels.
The factory has a list of bespoke panels that need to be produced in the most optimised way.
The panels have:

a thickness (10 sizes)
type of material (20 types)
unique width and length
angle left/right (not all panels are 90 degrees)

The panels are produced on tables. There are max 2 panels per table (they most both fit on the table (combined with of the panels must be smaller than the width of the table).
It is best to combine panels with the same thickness and type of material. This saves a lot of manual work. If 2 panels with (almost) the same angle can be combined on a table, only one manual action needs to be done by a worker.
We are currently using a genetic algorithm to calculate the best combinations in the given list.
The fitness is calculated by using the thickness, material type, angle (and sum of panel width < table width)
Are there other types of algorithms/AI that can be used and would be much less resource intensive? How could we approach these?

GA sounds not too bad, have you tuned stuff like mutation rate and population size? It could also be a fit for integer programming.


text:
Modern graduate-level machine learning books with focus on generative models
I'm looking for a modern machine learning book with graduate-level treatment of more recent topics such as diffusion and generative models, transformers etc.
I have a hard copy of Deep Learning by Goodfellow and Bengio; while I liked the book and read it extensively when it was published, it is a bit dated now.
I'm considering Probabilistic Machine Learning: Advanced Topics by Kevin Patrick Murphy. But maybe there are better alternatives.
I need this for my comprehensive Ph.D. examination. Any suggestions are very appreciated.
P.S. I'm also aware of this stack post, but the list of references there is quite dated or introductory.

Berkeley CS294-158 is a graduate-level course on deep unsupervised learning. They cover a lot of architectures used in modern generative modeling. They have recorded lectures and slides online.
Stanford CS236 is a course on deep generative modeling. They don't have a textbook, but they have course notes and slides online.

What part of "I'm looking for a modern machine learning book" is not understandable?

A good resource is a good resource. I doubt the medium that resource comes in is a deal-breaker for someone studying for an exam. Especially because they're looking for resources on relatively recent topics (e.g., diffusion in the context of deep generative modeling only came out in 2020). Good quality textbooks aren't published in three years -- course notes are going to be close to the best resource you can get.

I totally understand what you're getting at, but I thought it'd be more helpful to at least point them in the direction of helpful resources instead of just leaving the question answered.

Let's put it in this way. Let's say I'm looking for the Lord of the Rings films and instead you give me the book. Thanks a lot, but that's not what I ask - I don't like reading. Anyway, the OP will decide if your answer is fine.

@AlexanderWan Thanks for the links, I guess it is a good material for self-study, but this is not what  I'm looking for. Anyways, upvoted.

@AlexanderWan, usually lecture notes become books at some point. I believe  the book that I linked was lecture  notes that were properly bundled together into a formal book that could be referenced.

Here are my suggestions:

Deep Learning: Foundations and Concepts (2024) by Bishop – A comprehensive and up-to-date text covering fundamental and advanced topics in deep learning.
Reinforcement Learning: An Introduction by Sutton – A classic and authoritative reference on reinforcement learning, perfect for understanding RL fundamentals and recent advances.
Multi-Agent Reinforcement Learning: Foundations and Modern Approaches (2024) – Focuses on the growing field of multi-agent systems, with a solid theoretical foundation and modern applications.
The Principles of Deep Learning Theory – Offers a theoretical perspective on deep learning, bridging the gap between statistical physics and machine learning.
Foundations of Computer Vision  (2024) – A modern take on computer vision, providing both foundational knowledge and cutting-edge techniques.

Also, don't forget the famous list of 27 papers (reportedly shared by Ilya Sutskever with John Carmack). As Sutskever mentioned:

"If you truly master these, you’ll understand 90% of what matters in modern AI."


text:
Can LLMs be used to discover new laws of logic?
Can LLMs be used to discover new laws of logic?
Stephen Wolfram seems to claim this in What Is ChatGPT Doing … and Why Does It Work?, § "What Really Lets ChatGPT Work?":

is there a general way to tell if a sentence is meaningful? There’s no traditional overall theory for that. But it’s something that one can think of ChatGPT as having implicitly “developed a theory for” after being trained with billions of (presumably meaningful) sentences from the web, etc.
What might this theory be like? Well, there’s one tiny corner that’s basically been known for two millennia, and that’s logic. And certainly in the syllogistic form in which Aristotle discovered it, logic is basically a way of saying that sentences that follow certain patterns are reasonable, while others are not. Thus, for example, it’s reasonable to say “All X are Y. This is not Y, so it’s not an X” (as in “All fishes are blue. This is not blue, so it’s not a fish.”). And just as one can somewhat whimsically imagine that Aristotle discovered syllogistic logic by going (“machine-learning-style”) through lots of examples of rhetoric, so too one can imagine that in the training of ChatGPT it will have been able to “discover syllogistic logic” by looking at lots of text on the web, etc. (And, yes, while one can therefore expect ChatGPT to produce text that contains “correct inferences” based on things like syllogistic logic, it’s a quite different story when it comes to more sophisticated formal logic—and I think one can expect it to fail here for the same kind of reasons it fails in parenthesis matching.)

I'm not sure if he does claim this. It seems like he's saying that the logic discovered by ChatGPT depends on what it's train on, and any logic it discovers is already found in human text.

In the quoted text, Stephen Wolfram uses scare quotes to signify loose, non-technical analogies. So by my reading, he is not claiming that ChatGPT has discovered any new type of logic. Instead he is comparing ChatGPT's ability to output meaningful sentences loosely to classical logic.
The new "logic" that the model may have then, would be: Is this sentence meaningful? I.e. does it represent an idea that can be parsed and understood?
He then makes a comparison to existing classical logic (whether a statement is true or not), which ChatGPT does arguably approximate, and definitely parses and outputs simple logically correct statements better than chance.

The short answer is no, it can't.
LLM like we have today are really less complicated than some people think (well it's definetly not simple to make one, but the task it's doing is fairly simple, just difficult to optimize). All it really does is, given some text, guess what's more likely to be the next word in that text if a human was writing it.
What it usually learns first is syntax, like how to build words with some tokens (tokens could be characters, common combinations of characters or even full words, but then it would already have syntax done). It's fairly easy for it to learn to complete the following: "improvemen", as the word to be completed is probably improvement, so it would guess a "t". Next it would learn some more complex speech patterns, like how an open bracket usually get's closed in the future, or how words coordinate with each other. It's as if it learns what are verbs, nouns, adjectives etc.
The cool part comes now, because if you train a large enough model with enough data for enough time, it will start to identify complex subjects structures, like how when we are teaching about LLM we might start explaining RNNs, than from RNNs we explain about BPTT and exploding/vanishing gradient, and around there we move from RNNs from LSTM, and so on. Notice how each subject has layers (teaching -> LLM -> RNN -> BPTT -> exploding/vanishing gradient ...), and depending on what layers have come before we know in what layer we will enter next, and each layer is really more complicated than what I said, with common words or sentence structures, and the LLMs start learning about those in each layer and how it should "navigate" through the layers.
In the end, it can mimic how we talk about various subjects, but still, it's just mimicking the texts it has seen. It doesn't "think" or use logic at all in what it's saying. It's "logic" consists of copying our pattern of speech in different subjects, but no actuall logic of what it's saying exists. Nothing new can come from just following patterns. Humans can extrapolate common sense and create new things, and that's what makes us special.
Also note that my explanation of how it learns things is just a human interpretation. In the end it's just a lot of numbers marching down a complicated cost function, and really have no guarantee that itcan be reasoned from a human perspective.

Ian Goodfellow, Yoshua Bengio, & Aaron Courville, Deep Learning (2016), § 6.1 is an example on using deep feed-forward networks to learn the XOR logic operation.