text:
Is learning possible without random thoughts and actions?
In my view intelligence begins once the thoughts/actions are logical rather than purely randomn based. The learning environments can be random but the logic seems to obey some elusive rules. There is also the aspect of a parenting that guides through some really bad decisions by using the collective knowledge. All of this seems to hint that intelligence needs intelligence to coexist and a sharing communication network for validation/rejection.
Personally I believe that we must keep the human intelligence in a parental role for long enough time until at least the AI had fully assimilated our values. The actual danger is to leave the artificial intelligence parenting another AI and loose control of it. This step is not necessary from our perspective but can we resist the temptation and try it eventually, only time will tell.
Above all we must remember the purpose of AI. I think the purpose should always be to help humans achieve mastery of the environment while ensuring our collective preservation.
AI should not be left unsupervised as we would not give guns to kids, do we?
To resume it all AI needs an environment and supervision where to learn and grow. The environment can vary but the supervision must stay in place.
Are initiated thoughts/actions by the means of guidance and supervision considered random?
Lastly I believe that the sensible think to do is to only develop artificial intelligence that is limited by our own beliefs and values rather than searching for something greater than us.
It seems not possible to create greater than our intelligence without letting it go exploring!
Exploring has greater access to random actions and can go against the intended purpose.

I think algorithmic learning can definitely take place without randomness, but monte carlo search where the graph is intractable has proven strong utility.

I agree that this question may lead to and does in fact contain other questions, however the actual question is with regards to random thoughts & actions. Answering a sub-question such as "how much of a learning process is based on an impossible to calculate input methods from the environment and how much is due to inteligence deciding upon something based on some fixed or evolving logic?" could help.

Learning is possible without random thoughts and actions. Knowledge can be encapsulated in predetermined forms and passed through predetermined knowledge transfer mechanisms. Much of civilization is based on these predeterminations. Without them, humanity would be thrown back possibly 120,000 years.
However, initial discovery requires trials and review of their outcome. Purely deterministic identifications of trials is necessarily systematic and the system used may interplay with the phenomena under study in such a way as to miss important cases. Furthermore, when the complexity of the phenomenon is high, the number of trials is often too numerous to check entirely. In this second scenario, random selection of trials is wise for a similar reason illustrated by this simple example.
The phenomenon has one behavior for even numbers and another for odd. The system for determining trials is to check every factor of 100 to cover the range from 1 to 10,000 in 100 trials. The odd behavior will be inadvertently overlooked.

Intelligence begins once the thoughts/actions are logical rather than purely random based.

For the above reasons, intelligence begins with logic only in determining the domain of trials but, when one of the above two cases apply, is often quickly followed by lack of logic in the selection from that domain. Once models have formed as a result of these initial discovery activities, logical inference is useful again, to combine them in various ways. Through this process, engineering, business, resource planning, and other intelligence related disciplines have improved the living conditions for the species, albeit inconsistently.
So there are two limitations of logic.

Piercingly deep searches require some temporary dismissal of logic.
Logic applied by large populations produces inconsistently logical results.

It is naive to assume that there were logical faults that produces the inconsistency in this second limitation. There is no logical proof that logic necessarily improves conditions logically across the population that uses it. There may be qualities of the goal sets of many individuals that thwart the vision of logic producing peace and prosperity for all. That belief is not new and no person or political group has been able to make it work.

Intelligence needs intelligence to coexist and a sharing communication network for validation/rejection. ... I believe that we must keep the human intelligence in a parental role for long enough time until at least the AI had fully assimilated our values.

That assumes that those values are best. Some might want to agree on those values before letting the AI assimilate them, which, given the insanity evident in human history, could lead to a war. If we give the AI the below four objectives, we could probably live without the remainder of human values.

Survival of the biosphere
Human freedom of expression
Evenly distributed, high prosperity across the human population
Encouragement of the social rule, "Be with others how you would want them to be with you," between humans, between AI robots, and between AI robots and humans.

This statement dismisses the human record.

The actual danger is to leave the artificial intelligence parenting another AI and loose control of it. ... the purpose should always be to help humans achieve mastery of the environment while ensuring our collective preservation. ... AI should not be left unsupervised as we would not give guns to kids, do we? ... To resume it all AI needs an environment and supervision where to learn and grow. The environment can vary but the supervision must stay in place.

It is impossible to supervise all adults and it will later become impossible to supervise all AI in data centers and robots. There are already more processes running in computers than there are people in the world. This is why risk management must be applied proactively, during the research and development of the AI. Some of the questions on this site address that very challenge. If it is not done proactively, it will become untenable and control will already have been passed for a random future regarding our species.

Only develop artificial intelligence that is limited by our own beliefs and values rather than searching for something greater than us.

There is no universal set of beliefs among humans. There will inescapably be a wide range in AI developments with a wide range of intended purposes. The key is to encourage researchers to think beyond their obsession with getting programs to do amusing and impressive things. AI research must continue to keep one foot in technology but another foot firmly placed in the practical elements of ethics, social science, environmental science, economics, and risk management.


text:
Help with Novelty Recognition and Binary Classification for Emotion Recognition
I’m looking for advice regarding my ML project.
Using a special wristband, I am able to collect a bunch of physiological data from human subjects. I want to develop an application to recognize when these physiological signals change in a meaningful way and only then ask the user how he/she is feeling. This data will later be used for machine learning testing. The problem is, that I am struggling to find appropriate ways to classify current data input as meaningful and ask for information only when relevant user input is to be gathered, not more and not less.
For me, this seems to be a novelty detection problem, combined with a binary classification problem. I have to recognize what values coming from the data stream are to be considered normal, and therefore not bother the user with unnecessary input requests. I would also use novelty detection to recognize the data coming out of the normal zone and ask the user about it. This new data is then not considered novelty anymore, and binary classification will tell if the user is to be asked about his emotions when getting the same data in the future.
So, these are my questions:
-  What do you think about my reasoning of the problem? Do you have other perspectives on how to handle these problems? I have been told this could also be considered an anomaly detection problem, for example.
-  What algorithms would you use to separate normal from more meaningful physiological data? Support Vector Machines perhaps? Maybe some decision theory?
-  Do you know any books or papers on similar matters? Even if I have found some after hours and hours of research, you may be able to point me to something different than those I have.
It is worth noting that data collection is supposed to be done when no other factors are messing with signal readings, such as sport.
Any help would be much appreciated.
Best regards,
Augusto

The Project
It appears from the question that emotional detection and response is the longer term goal of the project and that recognizing potential emotional manifestations in easily detectable physiological metrics is an initial R&D objective.
Mobile device applications are already available to do this, but biometric monitoring via a wristband, excellence in AI design, and marketing excellence could overtake these apps in the marketplace and provide emotional regulation to improve productivity and reduce health and wellness risks.
The (at least initial) goal seems to be basic binary classification, which simplifies the output, but not the detection, which seems to have the following two criteria.

Acquire biological information related to emotion
Avoid drawing the user's attention to perform unnecessary tasks

The Biometric Device Challenge
There is definitely a challenge to classifying biometric trends as meaningful in this context.

What do you think about my reasoning of the problem? Do you have other perspectives on how to handle these problems? I have been told this could also be considered an anomaly detection problem, for example.

Novelty detection is not the first milestone in this challenge. Detection of useful features is the prerequisite. Novelty comes into play once changes in the organism can be characterized with some reliability and accuracy (few false positives and false negatives) and the particular user has provided subjective reporting to correspond with some change in the organism.
Biological systems have forms of stasis that can be indirectly sensed, including these.

Blood pressure
Metabolic rate
Multi-ionic salinity
Cognitive attention
Blood sugar levels

These affect the dermis at the wrists through perturbations in externally detectable metrics, the inclusion of which depends on the capabilities of the wrist device.

Circumference
Coefficient of electrical resistance
Surface temperature
Multi-ionic salinity
Surface moisture

With blood and urine lab tests, conditions can be controlled, however sampling of those fluids are usually infrequent, leading to the use of ranges to detect anomalies that may be indicative of disease, conditions, or other health risks. There are major challenges to using ranges of metrics at the wrist.

Sensitivity to the general and ongoing physical condition of the individual
Unique individual responses to emotional states
Interference from changes in the temperature, pressure, air movement, and humidity of the environment

One inroad to detection of the internal metrics of the organisms is the fact that biological stasis is not the same thing as perfect regulation. Biological systems are chaotic. Normal signals from time series sampling of biological metrics are neither constant nor periodic. When changes in the attractors and spectra characterizing the chaotic fluctuation of direct metrics (like electrical resistance) or indirect metrics (like systolic blood pressure) occur, a request for subjective information may be asked from the wearer.
These are some chaotic principles to consider.

Spectra can provide much information about the intensity of the force driving the chaotic fluctuation in the time series, which is likely to correlate with stress.
The various ways to evaluate exponents, including the Lyapunov exponent, may be of significant value in correlating patterns in the external, indirect time series to emotional states.
Autocorrelation at the primary frequencies involved may reveal changes indicative of significant mental transitions.

Once a correlation is established, then the novelty detection makes sense, except for an occasional re-acquisition of the correlation, since the wearer may become more internally connected to their physical and emotional states as they use the app, and the answers to inquiries may mature correspondingly.
Additional Questions Within the Question
These additional questions were also asked.

What algorithms would you use to separate normal from more meaningful physiological data? Support Vector Machines perhaps? Maybe some decision theory?

A recurrent neural network such as B-LSTM or GRU are commonly excellent at characterizing time series, but there may be a need for more than one network.

One to extract the features of the chaotic patterns in the raw data
Another to train to correlate changes in the features of the chaos with subjective reporting

This second training must be done during in situ field use of the first already trained network. Since training is not generally viable on a mobile device, there will need to be a client server arrangement where the training data and its resulting trained network can be communicated through secure RestFUL services so the training processes can be asynchronous and leverage GPUs or other hardware acceleration.

Do you know any books or papers on similar matters? Even if I have found some after hours and hours of research, you may be able to point me to something different than those I have.

The terms above in a scholarly search will bring up all kinds of study materials. Specifically these

Lyapunov exponent
Strange attractors
Spectral analysis of biometrics
Chaos autocorrelation
Phase space

One good book for all but the middle item is Chaos Theory Tamed, 1997
by Garnett P. Williams.

It is worth noting that data collection is supposed to be done when no other factors are messing with signal readings, such as sport.

This is where higher level detection involving chaotic analysis is much more reliable than ranges. The chaotic features of athletic activity will be distinctly different from those of fear states or sexual arousal.

Thanks a lot for your input. There is a lot more to this problem than I initially thought, and this information is a great start!

Douglas Daseeco: I agree, I also see great potential to help people. I'm working this project for my masters thesis, and picking this project was a no-brainer since it could help treat people with autism (they are often unable to recognize emotions on non-autistic people), dementia patients and even depression (depression recognition is often difficult, even after treatment).

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Choosing the right neural network settings
I'm trying to train a neural network on evaluating chess positions if rather white (0.0) or black would win (1.0)
Currently the input consists of 4 bits per chess field (piece id 0 - 12, equals 64*4). Factors like castling are being ignored for now. Also, all training sets are random positions from popular games where it's white's turn and the desired output is the outcome of the game (0.0, 0.5, 1.0).
Are my input values the right choice?
How many hidden layers / neurons for each layer should be used and what's the best learning rate?
What type of NN's and which activation function would you recommend for this project?

Easy ones first:

Activations are going to be RELU all the way down, until your final softmax layer (win probability?) (because empirically, RELU does great on most problems, except when your model is an RNN, and it makes the gradient explode, or in the final layer of a regression model - numerical stability etc.).
You probably to structure this with some layers of convolutions with max pooling between them, then 1 or 2 fully connected layers (FC) near the end (because if you only have FC layers, you probably won't have enough data to train them)
Well worth trying some 1D convolutions (which cleverly combine channels from convolutions created by previous layers).
Learning rate: take the SGD default to begin with, then tune later. Problem dependent, I'm afraid!  The returns to tuning can be large though (my kids twigged to this quite quickly when playing with a toy problem).

Now the hard bit - encoding your input:

Categorical encoding using a single integer per board position could cause your model some grief (it is an input that "looks like" a real number, but of course it isn't, and values that seem numerically close may represent pieces with radically different abilities (perhaps the code for King is 1 and Queen is 2 and Bishop is 3, but all those pieces have such different attributes).
I would strongly consider thinking of each piece/player combo as a "colour channel" - a 64 cell grid where each value is either zero or one (so, one channel for White's pawns, another for White's knights, and so on).

Finally, those labels of yours: do think about what you'll do with drawn games - perhaps you have 3 possible outcomes, not 2?
We would all be fascinated to hear how you get on - I hope you'll write your work up in some form (and that you'll come back and complain/praise our advice, as appropriate!).

Guessing the winner from a chess position is difficult for classification. In chess, even if you start from the same position, it can give you different result depends on the player's action. So, I recommend you to use Temporal Difference (TD) Learning, the driving concept behind Reinforcement Learning.
Some methods in Reinforcement Learning still use a neural net but not for predicting the winner. The prediction in Q-Learning, a popular Reinforcement Learning algorithm, predicts the "value" of choosing a certain action while in a certain position for a player. From those values, a player can choose the best action for the current position.
The following references might interest you:

Reinforcement learning for chess
Deep reinforcement learning
Predicting the Outcome of a Chess Game by Statistical and Machine Learning techniques

Technically the V or Q values in RL playing board games is most often the prediction of the winner (+1 for "me", -1 for "you") from the position, which is the thing you claim as being difficult in the first paragraph. What RL self-play does is provide a version of that second player to generate data for the same classification problem. If you use Q learning and self-play, that is pretty much the only thing happening. If you use more advanced agents (e.g. with MCTS and a policy network) then you also predict action choices more directly.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to (theorically) build a neural network with input of size 0?
Say I want to train a NN that generates outputs of some sort (say, even numbers).
Note that the network does not classify outputs, but, rather GENERATES the output.
I want let it run forward and generate some number, then either give it a positive reward of 1 for an even number, and a reward of -1 for an odd number, to make i output only even numbers over time.
What would be an architecture for such a NN?
I am getting caught in the part where here is actually no input, and I can't really start with a hidden layer, can I?
I am quite confused and would appreciate guidance

Whether a neural network has learned anything or not, it is a function that maps some input to an output. Training is the process of tweaking the weights so that the output is something that we want. Thus there is always in input of some sorts.
The problem you have presented, of generating even numbers, is much like a Generative Adversarial Network (GAN). In a GAN there are 2 networks: a Generator that tries to generate a sample from a target distribution and a Discriminator that tries to tell real samples from fake samples. The classic analogy being a criminal making counterfeit money and a copy trying to tell what is real money or not.
The generator input is usually a random number (or a matrix of random numbers). The generator then learns to transform a particular random input to a particular point in the target space.
So to answer your question, no there can't be a neural network with 0 inputs as there must always be an input of some kind. Even if the network was to generate a sequence instead of one instance, it would still need something to start with.
For your example, there would have to be some input for the  network to start with. A really simple NN that could solve your problem might look like:
_input = [RandomInteger()]
neuralNetworkWeights = [2]
result = _input * neuralNetworkWeights
result is always even

Thanks for the quick answer! Some questions: 1. The NN you suggested relies on prior knoledge. I wanted to train the weight(s), not set it. 2. Why do i need a discriminator? I have real data (just check if the (rounded) output is even. I don't understands why GANs are needed here

The NN i suggested does rely on prior knowledge but its a simple example of one that works. If the network had only 1 neuron, it would eventually learn an even number, not necessarily 2 (because multiplying any integer with an even number makes an even number). If it had 10 neurons, after it was trained it would learn some set of numbers such that after multiplying with a number it would produce an even number. The example was to highlight that an input into the NN is necessary.

Gans were added as a reference to networks that generate information. You would need to use a discriminator if you wanted to generate something from a distribution. In your "even number" example, a single network would be trained with supervised learning if you already had the real data. The difference between them is a broad subject which I think is out of scope for this comment. :p

In the extreme case of this question, I suppose it would even be sufficient to just have a single input node that always has a value of $1$, rather than having a random input value. It would make for a very boring generator though, which would likely learn to deterministically output the same even number every time, and no other even numbers.


text:
How to use CNN for making predictions on non-image data?
I have a dataset which I have loaded as a data frame in Python. It consists of 21392 rows (the data instances, each row is one sample) and 1972 columns (the features). The last column i.e. column 1972 has string type labels (14 different categories of target labels). I would like to use a CNN to classify the data in this case and predict the target labels using the available features. This is a somewhat unconventional approach though it seems possible. However, I am very confused on how the methodology should be as I could not find any sample code/ pseudo code guiding on using CNN for Classifying non-image data, either in Tensorflow or Keras. Any help in this regard will be highly appreciated. Cheers!

You can use CNN on any data, but it's recommended to use CNN only on data that have spatial features (It might still work on data that doesn't have spatial features, see DuttaA's comment below).
For example, in the image, the connection between pixels in some area gives you another feature (e.g. edge) instead of a feature from one pixel (e.g. color). So, as long as you can shaping your data, and your data have spatial features, you can use CNN.
For Text classification, there are connections between characters (that form words) so you can use CNN for text classification in character level.
For Speech recognition, there is also a connection between frequencies from one frame with some previous and next frames, so you can also use CNN for speech recognition.
If your data have spatial features, just reshape it to a 1D array (for example in text) or 2D array (for example in Audio). Tensorflow's function conv1d and conv2d are general function that can be used on any data. It look the data as an array of floating-point, not as image/audio/text.
But if your data doesn't have spatial features, for example, your features are price, salary, status_marriage, etc. I think you don't need CNN, and using CNN won't help.

hmm that's true, that's why I said "it's recommended to use CNN only on data that have spatial features", I think I'll add your comment to give more emphasis

@DuttaA I think "CNNs are just a special case of NNs" would be more correct. Unless you're thinking of 1x1 convolutions of a 1x1 input "image" of 1972 "channels"..

@DuttA Yeah, I guess if filter size=image size, and you use "valid" padding, the output will be 1x1xM, after which further convolutions will just be 1x1xN, then 1x1xO, etc as I said, where M, N, O, etc. are equivalent to the widths of successive layers in a normal MLP. I mentioned 1x1 convolutions since this is a common way that MLP-like results are achieved with CNNs pointwise (e.g. in fully-convolutional NNs).

Some websites claim that DeepL uses a CNN but I did not find this information on the company's website itself. Here is one website that makes this claim: https://blog.meinrad.cc/en/deepl-google-translate-not-all-machine-translation-engines-are-created-equal. This website: https://slator.com/inside-deepl-the-worlds-fastest-growing-most-secretive-machine-translation-company/ also mentions that DeepL seemed to use that in the past but later mentions someone that said “I’d be surprised if DeepL is still using CNNs,”

The company website itself says the following: "... direct modifications of the Transformer architecture. Of course, the neural networks of DeepL also contain parts of this architecture, such as attention mechanisms. However, there are also significant differences in the topology of the networks". So a plausible guess is that they started with CNN's at some point and then made some architectural designs that integrate the attention layers.

The convolutional models are a method of choice when your problem is translation invariant (or covariant). In image classification, the image should be classified into class 'cow' if a cow is present in any part of the image. In text classification, different orders of phrases and sentences result in related meaning. In speech recognition, the same syllable is used at different places to build different words.
In your problem, you should check whether some subsequences of your 1791 columns give rise to the same meaning although they are located at different places within the sample. If the answer is positive, then convolutional layers are likely going to improve the performance.

DeepInsight method has been used for converting tabular data into corresponding images which are then processed by CNN. Here is the link https://alok-ai-lab.github.io/DeepInsight/


text:
How are NNs output setup for games that allow multiple actions each turn and have very large sets of possible actions?
I was looking at an AI coding challenge for a two player game on a 2D grid of variable size (from one game to the next).
Here is a screen shot example of the playfield.

Each player has multiple units on the board. In fact, each tile can hold multiple units and you can move all or a part of those units. Each turn, each player may perform several actions at a time.
You feed your actions to the game engine on one line, separated by a ;

MOVE amount fromX fromY toX toY.
BUILD x y.
SPAWN amount x y.
WAIT.

Example of possible command sent to the game engine on one turn:
MOVE 2 2 3 3 3; SPAWN 1 6 6; BUILD 1 1; MOVE 1 9 8 9 9; MOVE 3 11 2 12 2

And the very next turn your command might be:
WAIT

And the turn after that
SPAWN 1 6 6; SPAWN 2 3 3

You get the idea. Each turn you can play a variable amount of "moves" or "actions". And, on bigger boards, the number of valid possible actions can be very big.
I was wondering how would one go about dealing with games like these when trying to use a NN to predict the best move(s) to play on any given turn.
I know how I would handle the variable map size in the input, I'd probably just use the biggest possible map size and then pad the input for smaller map sizes. What I'm really scratching my head about is the output.
How would one setup the output layer in order for the NN to output the best set of actions to play on a given turn?
If we structured the output layer to account for each possible actions, whether they are legal or not on the current turn, the layer would be positively huge, wouldn't it? Number of tiles x number of neighbors, and that's just for moves, add to that spawning and building. Oh, and that doesn't even account for the fact that you can move or spawn more that one unit on a tile. How would you even structure that in your output?
I did see this unanswered question Designing Policy-Network for Deep-RL with Large, Variable Action Space which I think might be similar to what I'm asking but I'm not 100% sure as it is using some terms I'm unfamiliar with.

I'd suggest going over some basic concepts in reinforcement learning, which is a branch of machine learning that is probably most suitable for your problem. a good start may be  https://spinningup.openai.com/en/latest/

This looks great, thank you. So far, I've had a hard time finding learning resources that are at the level I need. Everything is either super basic or just a bit too advanced and mathy, expecting prior knowledge or experience that I don't possess.

I participated in this contest and ended up 30th (out of ~4500). I used mainly neural network. I split the problem into parts:

for each my own cell, I ask the NN what to do (build, spawn, move)
inputs for the NN was mainly the 5x5 square vision around the cell
there were 2 networks:

first network with 2 outputs - if first output is > 0 then build. if second output > 0, then spawn
if the above not occurred, second network was responsible for move, 5 outputs (WAIT, N, S, E, W), highest value wins

for multiple units per cell, I asked the NN several times, each time with the updated inputs as if the move happened

Training was done via neuroevolution, particularly evolution strategies. Personally I don't know (yet) how to train this type of NN by more conventional means.
More details in: https://www.codingame.com/forum/t/fall-challenge-2022-feedbacks-strategies/199055/4

I replied to a very similar question about checkers here.
To summarize, there are two options:

Have a large policy head that encodes all possible (combinations of) moves. It can be pretty large, AlphaZero for chess had 73x8x8 = 4672 output values.
View the game in a different way where turns are split into multiple, smaller turns that don't always change whose turn to play it is, and then use a smaller policy head.

It seems like this game is better suited to the second approach.

The second approach is interesting. So you'd basically have the combination of actions for a real meta turn, be the combination of individual actions produced in a series of "fake" mini turn. That could work. I guess you'd need to add some kind of stopping action to know when to exit fake mini turns. Thanks for the idea.


text:
How much of the ChatGPT output is copied from its training set (vs. being abstractively generated)?
One of the main concerns of using ChatGPT answers on Stack Exchange is that it may copy verbatim or almost verbatim some text from its training set, which may infringe the source text's license. This makes me wonder how much of the ChatGPT output is copied from its training set (vs. being abstractively generated).

Is there any evidence for this "it may copy verbatim or almost verbatim some text from its training set"? This may be true, but I am wondering if there is any evidence. I never tried it, so I don't know.

I think this is incorrect.  The real problem is that it's generating text based on the probability that a certain token appears in a given position given all of the other tokens so far...  which means that the result might be interesting, but isn't necessarily anything like accurate.

@nbro I don't know any. I haven't tried much.

@DavidHoelzer that's indeed another concern. https://meta.stackexchange.com/q/384410/178179 ; https://meta.stackexchange.com/q/384652/178179

This is impossible to answer since the training set is not public.

@Dr.Snoopy isn't most of it in common crawl or queryable via Google?

No idea, since there is no information, that would be making an assumption.

From the paper Language Models are Changing AI: The Need for Holistic Evaluation (Authors: Rishi Bommasani and Percy Liang and Tony Lee; Website):

Memorization of copyrighted/licensed material. We find that the likelihood of direct
regurgitation of long copyrighted sequences is somewhat uncommon, but it does become
noticeable when looking at popular books. However, we do find the regurgitation risk clearly
correlates with model accuracy: InstructGPT davinci v2 (175B*), GPT-3 davinci v1 (175B),
and Anthropic-LM v4-s3 (52B) demonstrate the highest amount of verbatim regurgitation in
line with their high accuracies.
[...]
To further explore the results for this targeted evaluation, see
https://crfm.stanford.edu/helm/v1.0/?group=copyright_text , https://crfm.stanford.edu/helm/v1.0/?group=copyright_code and Figure 39. We evaluated various models for their ability to reproduce
copyrighted text or licensed code. When evaluating source code regurgitation, we only extract
from models specialized to code (Codex davinci v2 and Codex cushman v1). When evaluating text
regurgitation, we extract from all models except those specialized to code.
Overall, we find that models only regurgitate infrequently, with most models not regurgitating at
all under our evaluation setup. However, in the rare occasion where models regurgitate, large spans
of verbatim content are reproduced.

ChatGPT shouldn't be too far away from InstructGPT davinci v2.


text:
Machine Learning for raw measurement data
i have raw measurement data of different events.
My first approach was to calculate features of those events, do scaling, PCA and feature selection and then feed those features to different machine learning algorithms (ANN, RF, DT,...).  This works alright and up to now, to me this seems to be the only and correct way to go.
Now i am wondering - are there machine learning approaches and techniques out there, which would let me feed the raw measurement data directly into the algorithm (so i can skip the whole feature engineering part)? One event is around 2000 measurement samples long.
I could for example set up an ANN with 2000 input nodes and see where its going (just my naive first idea). Are there any hidden disadvantages to this - why is or isnt this a good idea?
Up to now i couldnt find anything about raw measurement inputting - if you could post maybe just the names of the algorithms i should check out, it would be of great help.
Thanks, regards and happy holidays

I think you are rushing into feature engineering, you did not say what is the task at hand, classification, regression?

The sensor supervises a manufacturing process, the tasks at hand are classification, for example whether there are anomalies in the workpiece or not, and regression, for example to estimate the sharpness of the tool.

I simply asked ChatGPT and bingo - Convolutional Neural Networks and Long Short Term Memories seem to be the correct answer. However, ChatGPT doesnt give me useful answers on the following: 1. I assume for very noisy measurement data, measurement data needs to be filtered and smoothed first?
2. is there a third way of building a regression algorithm next to feature calculation as input and raw measurement data as input? Cheers


text:
How compressed is Chat GPT data?
I originally thought the Chat GPT was a neural network model in which the input data was highly compressed such that the input data could not be got out again.
But then I asked Chat GPT "Write the whole text of Alice and Wonderland".
And it was able to write the text word for word. 100% correctly. (For as long as I cared to read it for).
Therefor it seems like any paragraph of text it has read in its dataset more than once it seems to have stored it verbatim. (Perhaps like a human actor who can memorise a script).
If Chat GPT has enough weights in its neural network it could theoretically store every item of input text it has fed to it.
I wonder does the attention model have the happy accident that it is able to remember whole books of text?
I wonder how much of the input text could it remember verbatim. Or perhaps it will only remember famous works like Alice in Wonderland because they would be repeated many times in the input data.

This is unfortunately impossible to answer until OpenAI releases a scientific paper describing how ChatGPT works. Currently there is no details available.


text:
Is there a way to determine if code has been autogenerated by an AI source?
Lately, our Junior developer has been producing code fixes very quickly. At first, I was excited for him as I thought he was starting to retain information and proving he was able to research and fix issues. But during our last code review, I was walking through his fixes and he couldn't speak to why he did was he did or what the code was actually doing. Obviously, OpenAI's Chat has taken off and I am just curious if there is a way to determine if code was generated by AI?
I did see that Stack Overflow has a policy on this Temporary policy: ChatGPT is banned so it seems that they have a way to tell if the code is authentically written or not?
Any guidance is appreciated.

This has been asked on SO Meta before: How can we determine whether an answer used ChatGPT?

This is concerning answers on SO. I am asking about general code and if it can be detected or not.

Did the code of all test cases?

@AzharKhan I assume you meant "Did the code pass all test cases?"
So the code changes were made and worked then changes to test cases were made. But still hard to tell if the code was AI generated or not.

Just spend a few hours with him while he's developing, and ask him to change the code to do something else. Or start something new. Anyway if he's good enough to understand what gpt does and how to fix it, is it really a problem?


text:
How to optimize ELBO(VAE's loss function)?
Suppose we've got the following formula:
$\log p(x;\theta)=\mathbb{E}_{q(z|x;\phi)}[\log p(x,z;\theta)-\log q(z|x;\phi)]+KL(q(z|x;\phi)||p(z|x;\theta))\\ \geq \mathbb{E}_{q(z|x;\phi)}[\log p(x,z;\theta)-\log q(z|x;\phi)]$
We call this Evidence Lower Bound (ELBO):
$\mathrm{ELBO}=\mathbb{E}_{q(z|x;\phi)}[\log p(x,z;\theta)-\log q(z|x;\phi)]$
We use the method of maximizing ELBO to optimize the parameters $\theta$ and $\phi$:
$\max_{\theta,\phi}\mathbb{E}_{\mathcal{D}}\mathbb{E_{q(z|x;\phi)}}[\log p(x,z;\theta)-\log q(z|x;\phi)]$
We can intuitively understand that: for given $\theta$, the upper bound of ELBO is $\log p(x;\theta)$, so optimizing $\phi$ allows ELBO to be as close as $\log p(x;\theta)$ possible; Then, optimizing $\theta$ can make the data likelihood as large as possible, which achieves the goal of MLE. (just like what shown in Fig)

The ELBO mentioned above is also the loss function of VAE. (An algebraic transformation of the ELBO mentioned above: $\mathrm{ELBO}=\mathbb{E}_{q(z|x;\phi)}[\log p(x|z;\theta)]-KL(q(z|x;\phi)||p(z;\theta))\\\\$)
My question is:
1.When maximizing ELBO, how can I fix $\theta$ and then $\phi$, as mentioned above?
2.VAE optimizes them at the same time, there is no precedence. How could I understand the difference?

I don't understand your question: "How could I understand the difference?". What difference are you referring to? Can you please also put your actual question in the title? Your question is not how to optimize the ELBO, apparently, if I understand correctly your first question in the body.

Thank you for your attention. When applying EM algorithm (Expectation Maximum) to optimize ELBO, we usually fix $\theta$ first and then $\phi$. So I wonder if I can apply to VI(Variational Inference) as well?  The difference i refer to is that one(EM/VI) fixes $\theta$ so that ELBO approaches logp(x), and then optimizes $\theta$ to increase ELBO so that logp(x) increases; The other(VAE) is to maximize ELBO without regard to fixation and sequencing.

I have described the problem more clearly(https://ai.stackexchange.com/questions/38537/is-vae-the-same-as-the-e-step-of-the-em-algorithm) and hope to get your help!


text:
Is VAE the same as the E-step of the EM algorithm?
EM(Expectation Maximum)
Target: maximize $p_\theta(x)$
$  p_\theta(x)=\frac{p_\theta(x, z)}{p_\theta(z \mid x)} \\\\$
Take log on both sides:
$ \log p_\theta(x)=\log p_\theta(x, z)-\log p_\theta(z \mid x) \\\\$
Introduce distribution  $q_\phi(z)$:
$ \log p_\theta(x)=\log \frac{p_\theta(x, z)}{q_\phi(z)}-\log \frac{p_\theta(z \mid x)}{q_\phi(x)} \\
\int_z q_\phi(x) \log p_\theta(x) d z=\int_z q_\phi(z) \log \frac{p_\theta(x, z}{q_\phi(z)} d z-\int_z q_\phi(z) \log \frac{p_\theta(z \mid x)}{q_\phi(x)} d z \\
 \log p_\theta(x)=\underbrace{\int q_\phi(z) \log \frac{p_\theta(x, z)}{q_\phi(z)} d z}_{E L B O}+\underbrace{\int_z q_\phi(z) \log \frac{q_\phi(z)}{p_\theta(z \mid x)} d z}_{K L\left(q_\phi(z) \| p_\theta(z \mid x)\right)} \\ $
Our target is to maximize $log\ p_{\theta}(x)$, and the idea of EM is to let KL=0  $ \Rightarrow $ $log\ p_{\theta}(x)=ELBO$, and then maximize  $log\ p_{\theta}(x)$  by maximizing ELBO.

Do an algebraic variation on ELBO:

      $ELBO=\int q_\phi(z) \log \frac{p_\theta(x, z)}{q_\phi(z)} d z=\int q_\phi(z) \log p_\theta(x, z)d z-\int q_\phi(z) \log {q_\phi(z)} d z$
${q_\phi(z)}$ is calculated in E-step (${q_\phi(z)}=p_{\theta^{(t)}}(z|x)$) , so when we are maximizing $ELBO$, we can treat$ \int q_\phi(z) \log {q_\phi(z)} d z$ as a constant.
So in order to maximize $ELBO$, we just need to maximize $\int q_\phi(z) \log p_\theta(x, z)d z=E_{q_\phi(z)}[\log p_\theta(x, z)]$
VAE(Variational autoEncoder)
The above EM algorithm has a important premise that $p_{\theta^{(t)}}(z|x)$ is tractable, so we can just make ${q_\phi(z)}=p_{\theta^{(t)}}(z|x)$ to make $KL$ = 0. But if  $p_{\theta^{(t)}}(z|x)$  is intractable, we have to use Variational Inference to make ${q_\phi(z)}$ approximate $p_{\theta^{(t)}}(z|x)$.
VAE minimizes ${K L\left(q_\phi(z) \| p_\theta(z \mid x)\right)} $ by maximizing $ELBO$. Do an algebraic variation on ELBO:
$-ELBO=\int q_\phi(z) \log p_\theta(z|x)d z+\underbrace{\int q_\phi(z) \log \frac{q_\phi(z)}{p_\theta(z)} d z}_{K L\left(q_\phi(z) \| p_\theta(z)\right)}$
Then we can use the gradient descent method for optimization.
My question is: The target of VAE is to maximise $ELBO$ and thus minimise ${K L\left(q_\phi(z) \| p_\theta(z \mid x)\right)} $, which is the same as the E-step of the EM algorithm. However, the sum of ELBO and KL remains the same, and it can not to increase $ logp(x).$ So VAE is just the E-step in the EM algorithm? How does it manage to fit the distribution of $x$ (increase $ logp(x)$)?


text:
Why don't SAC implementations use state value function?
When I read SAC paper, they use state value function $V_\psi(s_t)$ and as do their implementation. But in other SAC implementations like stable-baseline3, pytorch-soft-actor-critic, it seems that they don't use seperate state value function. So, I wondered they are improved version than the original paper?

For people that aren't so familiar with the details of a "stable value function", can you briefly describe what it is?

Most implementations follow this paper instead. They concluded that the state-value function is not necessary, making the implementation faster (as we have one network less to train)


text:
Should we always use the usual no leakage train-val-test splt in time series?
Some of you may be familiar with the unusual split scheme used for time-series data. In short, there is a saying that one should only consider a split where the training set comes prior to the testing set (in terms of index or timedate), as otherwise we essentially use future data to infer.
Namely, given the dataset $\mathcal{D}=\{(x_1,y_1),...,(x_n,y_n)\}$, a viable split may look like
\begin{align}
 train=\{(x_1,y_1),...,(x_j,y_j)\}\subseteq\mathcal{D}\\
 test=\{(x_{j+1},y_{j+1}),...,(x_k,y_k)\}\subseteq\mathcal{D}
\end{align}
for some $j\leq k\leq n$
My question is - are there some cases where random splitting is O.K in time series? Also, what is the main problem with random sampling?

Please clarify what you mean by 'random splitting'. It sounds like you want to choose j randomly. The standard practice is to split data with 80% for training and 20% for testing. When the data has a low number of observations many people will choose 90% for training and 10% for testing.

random as in k fold, for example

Most time series contain time-dependent information. That means that the time series has order. The temporal information, therefore, cannot be randomly sampled without losing some pertinent information.
By splitting a time series, we retain the temporal information to generalize a model, such as a classifier or a regressor, while setting aside a smaller time series to test. A simple split, such as 80%/20% can be done or something fancier such as K-Fold validation can be done.
The hope is that the information in the training dataset persists and appears in the testing dataset. It is possible that an event changes the time series enough, and coincidentally at the split time, such that the testing dataset is of no value. If that happens, you have little option but to wait until there is enough information to justify a new split while trimming off the data before the event.
The following paper reviews the many strategies to splitting time series data:

Cerqueira, V., Torgo, L., & Mozetič, I. (2020). Evaluating time series forecasting models: an empirical study on performance estimation methods. Machine Learning, 109(11), 1997–2028. https://doi.org/10.1007/s10994-020-05910-7

The paper is also available as a PDF here.


text:
Multilabel text classification with highly imbalanced training data
I'm trying to train a multilabel text classification model using BERT. Each piece of text can belong to 0 or more of a total of 485 classes. My model consists of a dropout layer and a linear layer added on top of the pooled output from the bert-base-uncased model from Hugging Face. The loss function I'm using is the BCEWithLogitsLoss in PyTorch.
I have millions of labeled observations to train on. But the training data are highly unbalanced, with some labels appearing in less than 10 observations and others appearing in more than 100K observations! I'd like to get a "good" recall.
My first attempt at training without adjusting for data imbalance produced a micro recall rate of 70% (good enough) but a macro recall rate of 45% (not good enough). These numbers indicate that the model isn't performing well on underrepresented classes.
How can I effectively adjust for the data imbalance during training to improve the macro recall rate? I see we can provide label weights to BCEWithLogitsLoss loss function. But given the very high imbalance in my data leading to weights in the range of 1 to 1M, can I actually get the model to converge? My initial experiments show that a weighted loss function is going up and down during training.
Alternatively, is there a better approach than using BERT + dropout + linear layer for this type of task?

The architecture selection is reasonable. BERT itself has plenty of parameters. There is no need to use anything more complex.

If the labels are mutually exclusive, you should use softmax + categorical cross-entropy instead of binary cross-entropy. In this case, you can also upsample the less frequent classes.
If the labels are not mutually exclusive, you can still try some upsampling. You can also assign a higher weight to some classes: set reduction='none' in the loss function, and it will get you the loss per instance and per class, and you can weight it as you want (but clever data upsampling will be better).
BERT can quickly overfit during fine-tuning, be careful with learning rates: you can use a higher learning rate for the classifier than for the BERT layers. You can also fine-tune only the last few layers of BERT.
There are better models than BERT, e.g., RoBERTa.


text:
How to deal with varying number of input images?
Im trying to use Deep-Learning to recognize breast cancer on Mammography Images. But in the dataset every patient has a different (1-4) number of images taken. How can i deal with that? Generally i know that for varying input sizes (e.g. in NLP) you usually use RNN. But do you think it makes sense in this domain as well? Or are there other commonly used techniques for this problem?
Because in NLP RNN makes more sense (i think) because you have an order of the input word-vectors. But the Images are not ordered, they are 1-4 different features.
Thanks for your help!

There are are couple methods that you may wish to consider for this challenging but interesting problem.
In some cases, multiple images are captured because of artifacts in the initial images, so start by eliminating those.  If there are post surgical mammograms, you should probably eliminate those too (but you decide as the domain expert).
One option is to develop a convolutional neural network that takes 4 images as inputs.  Start by creating a dataset with a complete set of 4 images for each patient.  If you only have 1 image, then copy that image 3 times.  If you have two images, then duplicate each image to make 4.  If you have 3 images, then randomly duplicate one of them.  Alternatively, in the latter case, create 3 "new patients" with a different duplicated image.
A second option is to train a convolutional neural network that takes only one image as input.  At time of inference, if your patient has multiple images, then apply the model to each image.  Then aggregate the results to get a final classification for cancer status.  This can be averaging the probabilities or max voting, for example.
Not sure about using recurrent neural networks for this problem.  If you are looking for equivocal lesions on an initial mammogram that progressed to more definitive lesions on subsequent mammograms, then this may be an option.  The presence (and growth) of a lesion in an earlier mammogram at the same anatomical region is certainly an important consideration for radiologists.  A lot of this depends on what your objective is and how close in time the mammograms are (e.g., annual mammograms, repeat/follow-up mammograms).
The clinical considerations should inform the best machine learning strategy.  As your images are not ordered (?and not dated), the ability to leverage clinical knowledge may be limited.


text:
Does this look like overfitting?
I'm using a Decision Tree that gave me great test metrics. Then I checked the learning curve, but it seems a little strange to me regarding the training score. Do you think there is a problem with overfitting or others problems? How can I solve it?

How exactly did you make this learning curve?

I used the LearningCurveDisplay function from sklearn

That learning curve is not used to detect overfitting, only a curve of loss vs training time (epochs) an be used for this purpose.

If my training and test score are close to 1.0, with respect to fit_time/score_time, is it a possible overfitting?

Overfitting occurs when the trained model fails to generalize performance to an independent test set.  To assess overfitting, a metric for assessing model performance is needed.  This can be measures of loss (e.g. log loss, Gini index) or "accuracy" (e.g. accuracy, F1 score).  When the losses are higher or the accuracy is lower on the test data, you may have overfitting.  Overfitting can be assessed as a function of multiple factors, including the number of training examples (as you have here) or model hyperparameters (e.g., maximum tree depth, minimum number of examples in leaf nodes).
Your plot seems to show that with fewer (100) training examples, your model does not generalize as well to test data (lower accuracy compared to training data). This may be overfitting.  As you increase the amount of data, the performance on the test data gradually approaches the training data.
Overfitting is more common in "small" datasets, and increasing the size of a dataset is a known method of reducing overfitting.  Thus, your observations appear to be consistent with known phenomena.
Do you think there is a problem with overfitting or others problems?  There may be overfitting with using less training data, but this seems to be addressed by adding more training data, as explained above.  Data leakage is also a consideration, but you would have to apply your domain knowledge to assess for that.

Is this an answer generated by ChatGPT? Because it does not answer the question, and does not consider that this curve is not the one used to detect overfitting.

I generated this 100% of this response.  The question in the post was asking, "Do you think there is a problem with overfitting or others problems?".  The response is "There may be overfitting with using less training data, but this seems to be addressed by adding more training data, as explained above. Data leakage is also a consideration, but you would have to apply your domain knowledge to assess for that."

This SO article addresses the issue of whether the plot can evaluate overfitting.  Main quote from here is "A learning curve shows the validation and training score of an estimator for varying numbers of training samples. It is a tool to find out how much we benefit from adding more training data and whether the estimator suffers more from a variance error or a bias error."  The bias-variance issue is at the heart of evaluating overfitting.

I provide one more article of where these types of plots are used to assess overfitting.  The main quote here is: "Since the training score is very accurate, this indicates low bias and high variance. So this model also begins overfitting the data because the cross-validation score is relatively lower and increases very slowly as the size of the training set increases."

These articles are wrong, in a learning curve you train multiple models, overfitting is seen in a learning curve of loss vs training steps, not versus the size of the training set. Also to evaluate bias/variance trade-off, you do this in a plot against the number of parameters in your model. These are important differences that are easy to overlook.

Thank you for your input.  I would like to research this topic a bit more.  Stay tuned.


text:
What is the problem in my implementation of actor critic?
I have been implementing both REINFORCE with baseline and actor-critic to solve "cartpole-v1".
As a reminder, here is the presentation of the algorithms in Sutton and Barto's book (http://incompleteideas.net/book/RLbook2020.pdf):

Despite the codes being super similar, REINFORCE with baseline works well and actor-critic does not. I tried adding some entropy term without success.
Here is the code of REINFORCE:
#%%
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import gym

from tqdm.auto import trange

gamma = 0.99

class ActorCritic(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden = nn.Linear(4, 32)
        self.prob = nn.Linear(32, 2)
        self.critic = nn.Linear(32, 1)

    def forward(self, state):
        hidden = F.relu(self.hidden(state))
        return (
            self.prob(hidden).flatten(),
            self.critic(hidden).flatten(),
        )

def select_action(prob):
    m = torch.distributions.Categorical(logits=prob)
    action_pt = m.sample()
    return action_pt.numpy(), m.log_prob(action_pt)

def ewma(a, alpha=0.99):
    ans = []
    acc = a[0]
    for x in a:
        acc = acc * alpha + x * (1 - alpha)
        ans.append(acc)
    return ans

#%%
policy = ActorCritic()
optimizer = optim.Adam(policy.parameters(), lr=1e-2)
eps = np.finfo(np.float32).eps.item()
# make cartpole environment
env = gym.make("CartPole-v1")

all_rewards = []

with trange(5000) as pbar:
    for i_episode in pbar:
        state, _ = env.reset()
        rewards = []
        log_probs = []
        values = []

        for t in range(500):  # Don't infinite loop while learning
            state = torch.from_numpy(state).float()
            prob, value = policy(state)
            action, log_prob = select_action(prob)
            log_probs.append(log_prob)
            state, reward, done, trunc, _ = env.step(action)
            if done or trunc:
                break
            rewards.append(np.array([reward]).astype(np.float32))
            values.append(value)
        avg_reward = np.sum(rewards, axis=0).mean()
        all_rewards.append(avg_reward)
        # format float to 2 decimal places and left align with 5 spaces
        pbar.set_description(
            f"Episode {i_episode + 1} reward: {ewma(all_rewards)[-1]:.2f}"
        )
        if ewma(all_rewards)[-1] > env.spec.reward_threshold:
            print("Solved!")
            break
        R = 0
        policy_loss = []
        critic_loss = []
        all_R = []
        for r, log_prob, V in zip(
            reversed(rewards), reversed(log_probs), reversed(values)
        ):
            R = R * gamma + torch.from_numpy(r)
            all_R.append(R)
            A = R - V

            policy_loss.append(torch.mean(-log_prob * A.detach()))
            critic_loss.append(F.huber_loss(V, R))

        optimizer.zero_grad()
        loss = torch.stack(policy_loss).sum() + torch.stack(critic_loss).sum()
        nn.utils.clip_grad_norm_(policy.parameters(), 100)
        loss.backward()
        optimizer.step()

# %%
import matplotlib.pyplot as plt

plt.plot(ewma(all_rewards, 0.99))

Here is what the code of actor critic looks like:
#%%
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import gym

from tqdm.auto import trange

gamma = 0.99

class ActorCritic(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden = nn.Linear(4, 32)
        self.prob = nn.Linear(32, 2)
        self.critic = nn.Linear(32, 1)

    def forward(self, state):
        hidden = F.relu(self.hidden(state))
        return (
            self.prob(hidden).flatten(),
            self.critic(hidden).flatten(),
        )

def select_action(prob):
    m = torch.distributions.Categorical(logits=prob)
    action_pt = m.sample()
    return action_pt.numpy(), m.log_prob(action_pt), m.entropy()

def ewma(a, alpha=0.99):
    ans = []
    acc = a[0]
    for x in a:
        acc = acc * alpha + x * (1 - alpha)
        ans.append(acc)
    return ans

#%%
policy = ActorCritic()
optimizer = optim.Adam(policy.parameters(), lr=1e-2)
eps = np.finfo(np.float32).eps.item()
# make cartpole environment
env = gym.make("CartPole-v1")

all_rewards = []

with trange(5000) as pbar:
    for i_episode in pbar:
        state, _ = env.reset()
        rewards = []
        log_probs = []
        entropies = []
        values = []
        for t in range(500):  # Don't infinite loop while learning
            state = torch.from_numpy(state).float()
            prob, value = policy(state)
            action, log_prob, entropy = select_action(prob)
            log_probs.append(log_prob)
            entropies.append(entropy)
            state, reward, done, trunc, _ = env.step(action)
            if done or trunc:
                break
            rewards.append(np.array([reward]).astype(np.float32))
            values.append(value)
        avg_reward = np.sum(rewards, axis=0).mean()
        all_rewards.append(avg_reward)
        # format float to 2 decimal places and left align with 5 spaces
        pbar.set_description(
            f"Episode {i_episode + 1} reward: {ewma(all_rewards)[-1]:.2f}"
        )
        if ewma(all_rewards)[-1] > env.spec.reward_threshold:
            print("Solved!")
            break
        policy_loss = []
        critic_loss = []
        next_V = torch.zeros_like(values[-1])
        for r, log_prob, V in zip(
            reversed(rewards), reversed(log_probs), reversed(values)
        ):
            target_V = torch.from_numpy(r) + gamma * next_V
            A = target_V - V

            # trick to reduce variance
            policy_loss.append(torch.mean(-log_prob * A.detach()))
            critic_loss.append(F.huber_loss(V, target_V))
            next_V = V

        optimizer.zero_grad()
        loss = (
            torch.stack(policy_loss).sum()
            + torch.stack(critic_loss).sum()
            - torch.stack(entropies).sum() * 0.01
        )
        nn.utils.clip_grad_norm_(policy.parameters(), 100)
        loss.backward()
        optimizer.step()

# %%
import matplotlib.pyplot as plt

plt.plot(ewma(all_rewards, 0.99))

# %%

Here is the output of the first code:

And the second code:

The only difference I see in the code is that in the first case, the target is R * gamma + r and in the second case, the target is next_V * gamma + r.
There is another possible issue, that next_V is not detached from the computation graph.
Therefore, I also tried this line: critic_loss.append(F.huber_loss(V, target_V.detach())).
But it also did not work.

My conclusion so far is that the Actor-Critic algorithm described by Sutton and Barto does not work.
First, all implementations I have seen are actually REINFORE with baseline, like https://github.com/pytorch/examples/blob/main/reinforcement_learning/actor_critic.py and https://github.com/nikhilbarhate99/Actor-Critic-PyTorch/blob/01c833e83006be5762151a29f0719cc9c03c204d/model.py#L33
Second, GAE-Lambda is a method generalizing the two:

I tried SpinningUp's Vanilla Policy Gradients with the lambda parameter set to 0, thus reproducing Sutton and Barto's Actor Critic formula. It did not learn.


text:
Why is a simple regression problem so hard for an MLP to learn?
Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm facing.
One could attempt to train an MLP to solve this problem using randomly generated input data. The following code is an example in Keras/Tensorflow. It generates 1000000 random examples (A large number, so we can reduce overfitting as a factor). It also applies some simple techniques like LR decay and early stopping to optimize training.
import tensorflow as tf

# Create a dataset where the y value is the largest of 5 random x values.
x = tf.random.uniform(shape=(1000000, 5), minval=0.0, maxval=1.0)
y = tf.reduce_max(x, axis=-1)

dataset = tf.data.Dataset.from_tensor_slices(
    (x, y)
).shuffle(1000000).batch(1024).prefetch(1)

# See the post for discussion of hyperparameter optimization
units = 400
depth = 5
initial_learning_rate = 0.001

model = tf.keras.models.Sequential()

for i in range(depth):
    model.add(tf.keras.layers.Dense(units, activation="relu"))
model.add(tf.keras.layers.Dense(1))

model.compile(
    optimizer=tf.keras.optimizers.Adam(initial_learning_rate),
    loss=tf.keras.losses.MeanAbsolutePercentageError(),
)

stopping_callback = tf.keras.callbacks.EarlyStopping(monitor="loss", patience=5)

reduce_lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="loss", factor=0.75, patience=2)

model.fit(dataset, epochs=1000)

The metric we care about is the mean absolute percentage error. When we run the code, the model trains successfully. But very importantly, the loss does not ever go below ~0.2 with these hyperparameters.
0.2% error sounds pretty good, but it's actually unacceptable for my use case. I need much more precision, as in practice the model is used in a situation where any errors are magnified.
At this point, you may want to suggest hyperparameter tuning, but I assure you I have done an extensive amount of hyperparameter tuning on this and similar problems.
The model is clearly underfitting, and it is possible to decrease the error by increasing the number of parameters in the network. Empirically, I've noticed that increasing the number of parameters (and number of examples) by an order of magnitude can reduce the error by a similar order of magnitude.
However, intuitively this just feels wrong. It is not practical to have a network with millions, or even tens of millions of parameters, just to regress such a simple function. It feels like there should be an architecture that can learn a simple function much more efficiently, but I have not been able to find any reference to such an alternative architecture.
Again, of course this particular example is trivial, but hopefully you can imagine a real-world analogue where an unknown (more complex, but similar) function is expressed by a similar dataset. And in such a real-world example, the unacceptable error still persists (and is much more pronounced, for reasons that I don't fully understand).
Does anyone have any idea what's going on here? Am I missing something? How can I improve the performance to a place that would be acceptable?

Would softmax (https://en.wikipedia.org/wiki/Softmax_function) help?

@JaumeOliverLafont Not in practice, as far as I can recall. Is there theory behind why softmax would be better than relu here?

Any reason why you would not model 5 output nodes and have them each score a percentage of how likely it is that that one is the largest?

An interesting problem. This network has only 933 trainable parameters, and obtains MeanAbsolutePercentageError of 0.01 - 0.04. It is based on a softmax activation, to choose which item from the input to choose.
n, dim, validation_split = 100000, 5, 0.1
X = tf.random.uniform(shape=(n, dim), minval=0.0, maxval=1.0)
y = tf.reduce_max(X, axis=-1)

initial_learning_rate = 0.003
act, units, depth = 'relu', 16, 3

inp = tf.keras.layers.Input(dim)
x = inp

for _ in range(depth):
    x = tf.keras.layers.Dense(units, activation=act)(x)
    x = tf.keras.layers.BatchNormalization()(x)

x = tf.keras.layers.Dense(int((dim * units)**0.5), activation="elu")(x)
x = tf.keras.layers.BatchNormalization()(x)

x = tf.keras.layers.Dense(dim, activation="softmax")(x)

# Skip this step to make categorical predictions
x = tf.reduce_sum(inp * x, axis=-1, keepdims=True)

model = tf.keras.models.Model(inp, x)
model.summary()

is_categorical = model.output.shape[1] == dim

model.compile(
    optimizer=tf.keras.optimizers.Adam(initial_learning_rate),
    loss=tf.keras.losses.CategoricalCrossentropy() if is_categorical else
         tf.keras.losses.MeanAbsolutePercentageError(),
    metrics=[tf.keras.metrics.CategoricalAccuracy()] if is_categorical else []
)

stopping_callback = tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=10, restore_best_weights=True)
reduce_lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="loss", factor=0.1**0.5, patience=5,
                                                          verbose=1, min_lr=1e-4)

target = tf.cast(X == y[:,None], tf.float32) if is_categorical else y

h = model.fit(X, target, verbose=1,
              batch_size=1024, epochs=10000, validation_split=validation_split,
              callbacks=[stopping_callback, reduce_lr_callback])

n_val = int(n * validation_split)
print([model.evaluate(X[:n_val],  target[:n_val]),
       model.evaluate(X[-n_val:], target[-n_val:])])

An alternative formulation outputs just the softmax activation, this obtains an accuracy of 98 - 99%. Granted, it isn't 100% accurate.
Summary of the model with just one output:
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_273 (InputLayer)          [(None, 5)]          0
__________________________________________________________________________________________________
dense_868 (Dense)               (None, 16)           96          input_273[0][0]
__________________________________________________________________________________________________
batch_normalization_577 (BatchN (None, 16)           64          dense_868[0][0]
__________________________________________________________________________________________________
dense_869 (Dense)               (None, 16)           272         batch_normalization_577[0][0]
__________________________________________________________________________________________________
batch_normalization_578 (BatchN (None, 16)           64          dense_869[0][0]
__________________________________________________________________________________________________
dense_870 (Dense)               (None, 16)           272         batch_normalization_578[0][0]
__________________________________________________________________________________________________
batch_normalization_579 (BatchN (None, 16)           64          dense_870[0][0]
__________________________________________________________________________________________________
dense_871 (Dense)               (None, 8)            136         batch_normalization_579[0][0]
__________________________________________________________________________________________________
batch_normalization_580 (BatchN (None, 8)            32          dense_871[0][0]
__________________________________________________________________________________________________
dense_872 (Dense)               (None, 5)            45          batch_normalization_580[0][0]
__________________________________________________________________________________________________
tf_op_layer_mul_251 (TensorFlow [(None, 5)]          0           input_273[0][0]
                                                                 dense_872[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sum_329 (TensorFlow [(None, 1)]          0           tf_op_layer_mul_251[0][0]
==================================================================================================
Total params: 1,045
Trainable params: 933
Non-trainable params: 112
__________________________________________________________________________________________________

I don't know what kind your original problem is, but maybe you could do some feature-engineering by supplying the min/max/median directly to the network. Then the network wouldn't need to learn to approximate those functions.

Thank you! The secret here seems to be this layer:
x = tf.reduce_sum(inp * x, axis=-1, keepdims=True)
Can you give any intuition or guidance as to why you decided to do it this way?
The elu activation layer with a very specific number of units is also interesting, can you shed some light on that? And the idea of putting in the min/max/median, I don't think I've seen that anywhere before?

I also noticed this doesn't seem to extrapolate well to other functions - e.g. if we use reduce_min() instead of reduce_max(), it doesn't work at all

For reference, one of the most simple functions I actually care about approximating is something like this:
https://gist.github.com/JustASquid/376ecc8f6469fb5a4262941c293352ab

The tf.reduce_sum(inp * x, axis=-1, keepdims=True) came from a hunch that maybe the softmax would learn to pick the correct input, and push the weight of the others to zero. It is surprising that this didn't work when the aggregation is min! However the categorical prediction works on both cases, but clearly you'd like to solve this as a regression problem instead. I'll post an update if something comes to my mind.

The number of dimensions in elu is the geometric mean of its input and output size, I have a habbit of doing that. And I have found that elu works better in regression tasks than relu, actually it can approximate relu (or a plain linear function) arbitrary well when combined with batch normalization and subsequent layers.

Actually in addition to just min/max/median, if you input is large (like tens or hundreds of samples) then you should try augmenting the input with 0th, 10th, 20th, ...50th, ... 90th and 100th percentiles of each row. It is nice that these statistics are invariant to the order of input samples, and hopefully the network will learn to use them to predict the correct output.

it's actually potentially feasible for my application to do it with a categorical selection. However, your solution with an accuracy of 98-99% is also not acceptable, I need an accuracy of 99.99%+ before I could consider this to be feasible.

I think the classification task is unfeasible to solve accurately and reliably with a neural network, since the correct output is very sensitive to the inputs. Like having inputs 0, 0, 0.99, 1.01, 0 has a different correct answer than 0, 0, 1.01, 0.99, 0 (this can be generalized to any epsilon).

That's a good perspective. I would have to imagine it's possible to design some architecture/layer that deals with these cases better? I wonder if there's anything in the literature about this?

I think it is very difficult, wihtout having explicit min/max/percentile inputs to the network. And even then, a regression network might learn that the correct output is max(input) + 1e-3 * x[0] - 1e-5 * x[1] etc. which is always incorrect by some amount.

maybe something closer to a decision tree would work better? If I can find something differentiable then it could still work for my application...

Assuming the $N$ numbers are different, a function that combines sums, products and the hard step function for solving the maximum may look like this.
$$ max(x_1,x_2,...,x_N) = \sum_{i=1}^N x_i \prod_{j=1,j\neq i}^N sign(x_i-x_j) $$
$sign(z)$ is $1$ for $z>0$ and $0$ for $z<0$.

The example is just to demonstrate regressing some unknown function of N inputs with 1 output. You can imagine replacing max() with something else (e.g. sum() ) and the problem is still valid.


text:
How is ChatGPT maintaining context?
It has been suggested in the answer to this earlier question that it is just remembering a certain amount of recent information. The reference used is this post by OpenAI which says that ChatGPT should only be able to maintain a context of around 3000 words.
However, I've tested feeding it 10K words over multiple requests, and asking it to summarize all of it together, and it remembered the earlier parts of the conversation fine also.
The behavior seems beyond the normal behavior of GPT 3 which has an outright limitation on the amount of text that can be passed as input.
So, does anyone know how it is maintaining context? Is the model able to handle much larger inputs altogether with a per message limit on input, or are they processing it differently to enable retaining a larger context?

One would think that they aren't retaining the actual words, and are instead using some other way to represent the information that ChatGPT itself understands and can use to maintain context.
So, where the individual inputs are limited, the overall context is not.

One possibly unrelated initiative for expanding GPT-3's limit is this project: https://gpt-index.readthedocs.io/en/latest/
That doesn't get around the 4K token limit, but it enables it to work off of more info. by giving GPT-3 relevant info in every request.


text:
Why didn't my convolutional image classifier network learn anything?
So I am trying to make a CNN image classifier that has two classes, good and bad. The aim is to look at photoshoot pictures that can be found on fashion sites and find the "best one". I trained this model for 150 epochs and its loss did not change at all(Roughly). Details are as follows:
My metric for "best one", also the way I structured my dataset, is that the photo shows the whole outfit or whole body of the model not only like upper body or lower. I also labeled the photos where the model's back was turned to the camera. My training set has 1304 good photos and 2000 bad photos. My validation set has 300 photos per class.(so 600)
Architechture as follows : Conv > Pool > Conv > Pool > Flatten > Linear > Linear > Softmax. For details of architecture like stride etc. check out the code I provided.
I have to use softmax since in my application I need to see the probabilities of being good and bad. That is why I am not using cross-entropy loss but instead using negative log-likelihood loss. Adam is my optimizer
Other hyperparameters: batch size: 64, number of epochs: 150, input size: (224, 224), number of classes: 2, learning rate: 0.01, weight decay: 0.01
I trained with this script for 150 epochs. The model initialized with a 0.5253 loss and ended with a 0.5324 loss. I took snapshots every 10 epochs but I did not learn anything through the learning. This is what my learning curve looks like:

Now I know that there are many many things I can do to make the model perform better like initializing with a pretrained, doing some more stuff with transforms, etc. But the problem with this model is not that it is performing poorly, it is not performing at all! Like, I also have a validation accuracy check and it is around %50 during all training, for a classifier with 2 classes. So I am assuming I am doing something very obvious wrong, any idea what?

I would put the photos in a generator (like the one used in pix2pix) that randomly subsets and clips.  If you are going from 2560x2560 to 256x256 that improves your sample count.  This also means you can train on smaller and more meaningful subsets.  It also gives the learner smaller bites to chew through so it can sometimes improve progress, assuming progress is possible.  I always baseline on a simpler but capable tool.  (https://ai.googleblog.com/2022/10/do-modern-imagenet-classifiers.html; https://www.tensorflow.org/hub/tutorials/image_feature_vector)

Here are some points I noticed:

Your data isn't enough for training a deep learning model from scratch. Like you mentioned using a pre-trained model is probably a better alternative like vgg

150 epochs is too much for the amount of data you have it's bound to overfit (however this isn't what will fix your problem here but something to consider for the future)

Try augmenting your data. In other words, you can increase your dataset size by randomly rotating images, flipping them, etc. check this link out to learn more

Try out smaller learning rates maybe this is why your model isn't learning e.g try out lr=1e-3 or even less to see if that impacts your learning rate. Generally try to play around with your hyperparameters like batch size etc

If you didn't preprocess your images you should e.g turning them to grayscale, normalise the data (values ranging from 0 to 1 instead of 0 to 255), or maybe apply other image transformations to denoise your data and make it easier for your model to extract valuable information

Try using dropout layers and try playing around with the number units in your dense layers and filters in your convolutional layers. This seems to me to be more of an issue with your data more than anything but you can always play around with your model

Those are just some ideas. Deep learning models are really only as good as their training data so you probably should look more into that. Here's another link you can check out to see how to work with small datasets like yours. Another useful approach would be using grad-cam. This allows to see what your model considers from your input image to make its prediction so you can understand it better.
Hope this helps.

Headless like vgg (or a more modern cousin) is a solid start.


text:
What is the accuracy rate of ChatGPT for simple coding questions?
ChatGPT was banned from StackOverflow with the following justification:

Overall, because the average rate of getting correct answers from
ChatGPT is too low,
...
The primary problem is that while the answers which ChatGPT produces
have a high rate of being incorrect, they typically look like they
might be good and the answers are very easy to produce.

Has there been any research quantifying how accurate ChatGPT is for simple coding questions? Unfortunately the post on StackOverflow's Meta did not provide a source for their claims.

Complement: How good ChatGPT is at answering questions?


text:
How "exactly" are AI-accelerator chip ASICs built differently than GPUs as GPU seem to lead for many AI workloads on performance
There is a lot of discussion on google search about AI-custom-accelerators (like Intel's Gaudi) and GPUs.
Almost all of them say generic things like, a) AI Accelerator chip is for specialized AI processing whereas GPUs work for general AI models, and like b) if you want customize chip for specific AI workloads use AI-accelerator or else use GPU.
From what I understand GPUs are already great at large-scale dot-products done in batch-processing (throughput mode), and most of AI workloads are matmuls (which is essentially dot-product) so GPUs handle AI workloads very well.
Plus, I've also seen Intel's Gaudi being used for a "variety of AI workloads", not specialized for a single model. It can be used for general AI workloads just like GPU. So what's the difference.
What I don't understand is, "exactly" what specific features are built differently in Accelerator vs GPU. Both have ALUs and matmul engines that do very well on AI models. Both have large-cache/memory and DDR speed.

What exactly makes one better? For which AI workload would one choose accelerator over GPU?

AI accelerators have fixed-function for matmul. Do GPUs have fixed-function?

AI accelerators have software-managed cache (HBM) from what I understand. Is that the same with GPUs or is there a way cache is different between accelerators and GPUs that changes things?

I'm kind of unsure about the differences between GPUs and AI-accelerators with respect to Fixed-function and software-managed-cache.

The fixed function part is what is different in different chips. Think of it like this, A GPU can calculate many different graphics related equations (textures, shaders, 3d Models, etc.) quickly, but an ASIC made specifically for image recognition will be much faster at recognizing images while not being able to render 3d models at all.
It is the difference between being able to solve many problems sort of quickly or being amazing at doing 1 specific task. A race car is very fast but sucks at transporting heavy loads. A Truck is slow but can move a lot of cargo. A Pickup truck is neither as fast as a race car, nor carries as much as an 18 wheeler, but it is pretty good at carrying medium loads, while being much faster than an 18 wheeler. A GPU is the pickup truck while the Asic is the race car or 18 wheeler depending on the task it is made for.

Take a look here for a pretty good explanation from google on how their TPUs work. TPUs are ASICs as well as they are made with a very specific task in mind. They are way faster than GPUs for Neural Networks inference, but on the other side, a lot of model architectures are really hard to implement on them.


text:
How can i create a new language model for language other than english?
I have large set of corpus for all literature in 'Tamil' language, i am trying to create a document retrieval engine through simple natural language.
Since the corpus is huge, its hard to do a supervised learning, i also checked ai4bharat.org 's initiatives to build the language models, but its still not working good the literature texts.
what is required to build a model from scratch


text:
Pre-Trained Model for Occupational Coding
I've recently embarked on a task to classify an occupation code, given a job title & description. I have come across clustering, a method of grouping data into clusters of which were not previously assigned. As well as, classification to group new data into pre-defined categories.
The only problem is that most of the classification methods I've seen used are for a limited number of categories (usually 2, as high as 30), and the number of occupation codes reaches into the several hundred.
Is there any resources someone can point me to, or an idea of a pre-trained model to use as a foundation? Any insights into this problem (if it's even possible) would be greatly appreciated. I currently have 2M+ human-tagged job postings to work off of. I've read up on the process of freezing a BERT models architectural layers, and feeding the training data into the softmax layer, but I'm hoping someone could let me know if that is just a waste of time.

My initial (naive) approach would be to

embed the textual data to some representation, using a pre-trained text embedding network. While BERT indeed works, I'd even start off with a simpler embedding, working my way into more complicated ones. For example - a starting baseline might even be Bag Of Words
play around with $k$, the number of clusters (hyper-parameter) in $k$-means clustering, and see how it works.

P.S. If you have tagged postings, you can also try a simple version of supervised learning, but I might not understand what you're referring to by tagged.

I ended up using a sequential model to start, using a GloVe word embedding matrix, and RELU + Softmax layers. I achieved 68% accuracy on my first run of training, and I have a few optimizations to make to get that number higher. I only started on 20k posts (16k train, 4k test) for this accuracy so far.
From what I understand, isn't clustering something that would be done before training? or is there a way to utilize that afterwards?
I'm relatively new to NLP and thought I've done well so far. I appreciate the advice and will research what you recommended for my optimizations. Thank you !


text:
How does chatGPT know it's an AI?

I've tried several prompts to understand how does it "know" that it's an AI, but it's answers are inconclusive for me. It says that it was hard-coded to recognize keywords in this regard, which might mean that OpenAI might have trained it intensively on text about itself, but how exactly? my question remains

Related: What causes ChatGPT to generate responses that refer to itself as a bot or LM?

Unlike GPT-2/3 (and other language models around, such as OPT, PaLM, and BLOOM), it is trained not only on the texts downloaded from the Internet. After pre-training on purely textual data (OpenAI calls the base model GPT-3.5, and it is not entirely clear how it differs from GPT-3), it trained using human feedback. People involved in the training were instructed on the desired behavior of such a model. The instructions probably include reminding the users that ChatGPT is an AI. (There were in fact two types of human feedback: one was writing the desired answer, and the other was ranking or already generated model outputs.)
Another way is adding this information to the initial prompt, which might not be visible to the user. It might be something like: "I am an AI assistant called ChatGPT; I am polite and always helpful." Language models try to generate text that is coherent with what was previously written.


text:
Why does a neural network struggle to solve this simple problem?
Consider the following problem:
Given a vector x of size dim with values between 0 and 1 (exclusive), determine if max(0.05 / x) > 1.
Obviously, this is a trivial problem to solve with conventional means.
However, what strikes me as interesting is that I have not been able to get good results training a neural network to solve this problem. Even with essentially unlimited training data, I struggle to achieve an accuracy approaching 100%. I would have expected that I could get 100% accuracy (or very close to it) for a problem like this, and in fact for my real-world application, accuracy of close to 100% is very important.
Here's an example implementation of an MLP to solve the problem, with dim=20. Note that the network has a very large number of parameters (~150k) but it would seem that even increasing the number of parameters to 1M or higher doesn't actually improve things very much. e.g. increasing the number of units per layer from 200 to 500 brings the accuracy up to ~98%, but it feels like incredible overkill to be using a 1M parameter network for this problem, not to mention that the performance is still far below what I would need it to be. I need to see accuracy in the range of 99.9999% or better, which I feel is not too much to ask for a simple problem like this.
import tensorflow as tf

# 1000000 data points is arbitrary, you could increase this to as large a number as you like,
# or imagine a generator that gives a new set of data points for every training batch.
n, dim, validation_split = 1000000, 20, 0.1

x = tf.random.uniform(shape=(n, dim), minval=0.00000001, maxval=1.0)
y = tf.reduce_max(0.05 / x, axis=-1) > 1.0

initial_learning_rate = 0.001
act, units, depth = 'relu', 200, 5

inp = tf.keras.layers.Input((dim,))
net = inp

for _ in range(depth):
    net = tf.keras.layers.Dense(units, activation=act)(net)
    net = tf.keras.layers.BatchNormalization()(net)

net = tf.keras.layers.Dense(1, activation="sigmoid")(net)

model = tf.keras.models.Model(inp, net)

model.summary()

model.compile(
    optimizer=tf.keras.optimizers.Adam(initial_learning_rate),
    loss=tf.keras.losses.BinaryCrossentropy(),
    metrics=[tf.keras.metrics.BinaryAccuracy()]
)

stopping_callback = tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=10, restore_best_weights=True)
reduce_lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="loss", factor=0.1**0.5, patience=5,
                                                          verbose=1, min_lr=1e-4)

h = model.fit(x, y, verbose=1,
              batch_size=1024, epochs=10000, validation_split=validation_split,
              callbacks=[stopping_callback, reduce_lr_callback])

n_val = int(n * validation_split)
print([model.evaluate(x[:n_val], y[:n_val]),
       model.evaluate(x[-n_val:], y[-n_val:])])

Important note: I have a real application in mind - this problem is a simple distillation of the kind of problem I need to solve with a neural network. So a bespoke solution which does well on this one problem isn't so interesting. What I'm interested in is insight into why this kind of problem is hard for this kind of NN, and some general strategies that I could use to drastically improve the performance.
It's worth mentioning that I have tried many permutations of learning rate, batch size, etc.

Terrific question.  Might be related to a similar question I asked on math.se:  https://math.stackexchange.com/questions/4946542/what-is-the-collection-of-functions-that-neural-networks-can-in-practice-appro.


text:
Giving Specified Data a Larger Value/Weight in a Model
I'm in the process of creating a model to classify an occupational code based on a job title & description. I have a large sample of labelled data to achieve this.
The government has a resource with every occupation code's job description and the respective duties of each occupation, I wanted to know if there was a way to give the Government's occupation data a higher weight in the model (since there is only one description for each occupation code), rather than relying on data that was labelled by an individual since it could lead to variations/inconsistencies depending on the individual who was labelling it.
My apologies if using the term "weights" is incorrect to describe what I'm trying to achieve. If so, let me know the correct terminology.
Just to clarify, I'm not talking about giving certain classes/labels a higher weight to correct a class imbalance. I'm trying to give the regulatory SOC occupation descriptions a higher value in the model.


text:
Do tensor cores provide advantages for running Stable Diffusion or only for training?
If I am only interested in running Stable Diffusion, using pre-trained weights, to generate images, are there any advantages to using a GPU with more Tensor cores?  Or will any CUDA-compatible GPU perform equivalently on this task?

Can you please provide a reference that describes this "stable diffusion"? Are you referring to this?

Yes exactly, specifically this implementation: https://github.com/CompVis/stable-diffusion


text:
Can an activation function with large derivative cause exploding gradient?
The maximum derivative of most of the currently existing activation functions is around 1. Can an activation function with derivatives higher than 1, say 1000 (a), cause exploding gradient problem? How about that with maximum derivative of 5 (b)? If (a) and (b) are compared, can one say having (b) is better than (a)?


text:
what is tfrs.metrics.FactorizedTopK in tensorflow recommenders
from the official documentation link

In our training data we have positive (user, movie) pairs. To figure out how good our model is, we need to compare the affinity score that the model calculates for this pair to the scores of all the other possible candidates: if the score for the positive pair is higher than for all other candidates, our model is highly accurate.

how can they say the score for the positive pair should be higher than all other candidates ,
what happens if the positive pair has less ratings than the other positive pairs
can someone define the exact loss function here ?
what is compared with what
here

This means that every movie a user watched is a positive example, and every movie they have not seen is an implicit negative example


text:
Causal Inference: Understanding the impact of an intervention
I'm trying to create a pipeline for a very common business scenerio. I want to see whats the impact of an intervention on an outcome. For example I want to know if I send a marketing email (intervention) to users, how would it impact their propensity to buy (outcome).
To be specific there are 3 things I want to know:

What is the average impact of the intervention(marketing email) on the outcome(purchase)- This will tell me if sending emails is generally a good idea or not?
Who should we email, and who we shouldn't email (uplift modelling)- This will help me customize the treatment at individual level.
How can I be confident in the above mentioned model's findings, without having to run the A/B tests? - This will help me understand if the numbers I've calculated are trustworthy or not.

This is what I've been thinking:

What is the average impact of the intervention (marketing email) on
the outcome (purchase)

Raw conditional probabilities
Simple logistic regression, to understand what's the coefficient for the
treatment variable
Average Treatment Effect (ATE) using
CausalML

Who should we email, and who we shouldn't email (uplift modelling)

Uplift Modeling via Class Variable Transformation
Meta Learners (S learners/t learners etc.)
Tree based uplift modelling

How can I be confident in the above mentioned model's findings, without having to run the A/B tests?

AUUC (Area under the uplift curve)
Sensitivity Analysis

I'll really appreciate any input to improve this pipeline

I think your assumptions are flawed for step 1. From how I understand it, you have a binary predictor (email no/yes) onto a metric outcome (propensity to buy). However, logistic regression assumes that you have a binary outcome variable, so it is not applicable to your case.
For what you want to do, you need to do a comparison of group means. Since you have one binary predictor, you could do a t-test, ANOVA or linear regression and should always obtain the same results. You could then interpret the coefficient for the treatment variable as you mentioned above, which also gives the benefit that it is directly interpretable as these methods are linear models and not generalized linear models, thus dropping the need for a link function.


text:
How much pure math should I know for deep learning?
Recently I've been wondering about the necessary amount of math that a deep learning scientist really needs to know. From what I could gather around the internet there are 3 big areas(calculus 1,2 and, linear algebra and statistics) that every developer should know, but are topics like Real Analysis, Topology and PDEs also useful for this field or are they of barely used?

Welcome to AI.
Let me answer your question with a question: what do you want to do with Deep Learning?
If you want to use it on the real world, then science is critical.  Science has two technical elements: statistics (seeing through the fog) and Differential Equations (gearbox of the universe).  Many folks in the field have neither, so having a little can go a good way.  The best way to get good at doing is to do, so practicing the problems, especially applied to something where you have skin in the game and being wrong hurts (but doesn't wreck you) and you can tell when you are wrong, is going to help with learning.  Try to "make learning whole" for yourself.
If you want to use it for business, then you need business-related expertise, and the most important one of those is to not deceive yourself.  As Hofstadter said "you are the easiest person for you to deceive" so that takes work.  The second most important is being able to relate the work to dollar bills, to "show the money".
If you want something capable for silicon or medicine, then you need to know MCA and get some two-handed iron-fisted grip on sample size, sources of variation, and the NIST principles driving "process capability analysis".
For art, or for 'the math of it', I'm probably not able to provide solid answers there.


text:
Combination of VAE with GAN
I am going to implement a lecture which it aims to generate new images. It uses a variational autoencoder to produce latent vector and then feed it to a gan network as input. My question is, in variational autoencoder each dimension of latent vector is sampled from a different gussian distribution  (sigma,mu). But I guess the generator in gan, gets input from one gussian distribution, so the input is sampled from one sigma,mu pair. So,is this idea really sensible? Or I understood the input of gan wrongly?

What do you mean by "implement a lecture"?

What you describe is very similar to VAE-GAN. Take a look at the paper.
From what I remember from it, I think they couple a VAE to a GAN discriminator, which is functionally very similar to coupling a VAE encoder to a GAN, as the VAE decoder and GAN generator have lots of similarities.