text:
Why to use gradient accumulation?
I know that gradient accumulation is (1) a way to reduce memory usage while still enabling the machine to fit a large dataset (2) reducing the noise of the gradient compared to SGD, and thus smoothing the training process.
However, I wonder what causes the noise of the gradient? Is it because of the random sampling strategy on large datasets, intrinsic noise in datasets (like wrong labels) or anything else?
For a small dataset is it okay to do stochastic gradient descent and what might be the motivation for using gradient accumulation techniques on it?

This image from here nicely illustrates how gradient accumulation is performed:

Assuming infinite memory and compute we would be able to compute the gradient on the full batch, this would provide us with the true gradient!
In reality, the full batch gradient is often not possible to compute since holding the full dataset in memory is infeasible. The network weights are thus updated several times per epoch using mini-batches. Mini-batches provide an approximation of the true gradient. It is noisy since it is computed on only a part of the dataset. Alternatively, mini-batch gradients can be computed without updating the model immediately. Accumulating the gradients is referred to as gradient accumulation and if one accumulates the gradients during all steps, the sum of the gradients is the same as the gradient obtained on the full batch. There currently exists some debate about whether the loss landscape changes induced by mini batching might actually be beneficial.
For further information also see this blog post for the trade-offs that exist between full-batch and mini-batch learning.


text:
Using ML to uncover procedural logic
The game Elite Dangerous has a proceduraly generated galaxy of some 400 billion star systems.
Each star system in the game can be uniquely identified bu a 64bit number (id64) which is used as a seed for building the system but can also be decoded down to give some information about the system.

Boxel coordinates: x y z coordinates of a cube in which the system
can be located.
Masscode: A number that determines the size of the
boxel and the composition of the system

What we cannot decode from the id64 is the actual x,y,z coordinates and the boxel coordinates are too imprecise to be useful. Being able to preduct the x,y,z coordinates would allow us to make star maps of unvisited stars.
However the actual coordinates must be derived from the id64, some pseudo random number function and /or some rules grounded in real world astronomy.
Through the game journal files we can get the id64 and the actual coordinates of the system in 3D space and there are databases of player captured data containing about 70million systems we can use for test and train.
I took this data and created a file containing id64,bx,by,bz,masscode and x,y,z
(bx,by,bz is the boxel coordinates)
Using the features id64,bx,by,bz,masscode I want to be able to predict to within a light year, the x,y,z coordinates of the system.
I tried a few different Models from scikit-learn to see if I could train one to get coordinates but so far had little luck using various models. I had hoped that having a training set in the training/test set in the millions might help.
Is there a model or approach that might be better suited to this kind of problem? I guess if there is a psuedo-random element to it, then it might not be possible?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Why business experts should prefer state-of-the-art deep neural networks over simpler models?
I have encountered this pattern for a long time (5+ years). So many professionals come with an interesting domain-specific problem, and they demand using state-of-the-art deep learning models: take it or leave it.
I understand that technology advances faster than ever, but I am still missing the point. Indeed, I often propose using simpler, traditional ML models over complex ones because, for example, in an MLOps scenario is better to start with simpler models and then move to more complex ones. But, unfortunately, business experts often seem to be disappointed with such kind of proposals. Moreover, there are many reasons to prefer classic ML, which I often use to motivate the latter.

It's commonsense if you have a ML or stats background. From the outside, recent DL results seriously look like magical fairy dust. It's either that, or blockchains.

Short answer: because they are not experts in ML (and they should not be, otherwise they won't be asking), but are bombarded by buzzwords e.g. AI, blockchain, ChatGPT.
Do you have any friends who put their whole lifesaving in cryptocurrency without any idea what they bought? Same thing. This is human nature.

Yes, that is true indeed. Nevertheless, I feel bothered when non-ML experts say those fancy words just because they heard them in the news. I recall one day that a person thought cloud computing was performed in the cloud; it seemed like a joke, but it was not. I will keep standing my ground, fighting against such useless arguments and presenting simple, yet effective, solutions. Thank you.

remind me of this

There is another factor not yet mentioned.
Classic ML techniques potential is usually capped, we know already the limitations and the increased sophistication required to improve accuracy. Of course this is just our (perhaps faulty) perception of them, due to the slow improvements of the field.
DL techniques maximum potential is still unknown and as such is it somewhat rational to bet on them. Indeed the bitter lesson is that being able to scale well with the improvements in compute tend to be superior to hand engineering over the long term.
The above reasoning similarly explains why new technologies are "exciting".

That said, I'm not advocating to go DL first! As the OP said, the best practice is to start simple and increase complexity only if needed.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the "state of the art" in (at least partially) symbolic/logic-based chat bots/AI assistants?
ChatGPT has had a lot of buzz around it recently, and for good reason. It has shown some amazing capabilities in responding to new information, as well as in generalizing new information that has been given to it.
However, ChatGPT also regularly makes logical and mathematical errors. Some have commented that it really tries its best based on the data that was given to it to "sound correct" (almost like a freshman "bullshitting" a term paper), but it not actually great in general at making logical inferences.
One of the ways this issue could be solved is, rather than using a pure large language model (as essentially I understand ChatGPT is doing), we can instead use a semantic parser to parse natural language sentences into a logical form, such as first-order logic, higher-order logic, various forms of modal logic -- or what have you. This approach can also be supplemented with neural methods -- for instance, either by directly training a semantic parser as a sequence-to-sequence problem, or by learning "weights" for the likelihood of different parses.
I know that before the "first AI winter", the more symbolic/logical approach dominated, and a lot of work was done in Prolog, so I could probably find some examples of what I am looking for by looking at said historical approaches.
However, what I am looking for is a state-of-the-art chat bot based at least partially on symbolic techniques (specifically, using semantic parsing + a logical inference engine).
Does such a thing exist? Has much progress been made on that front since the first AI winter? I'd especially be interested in approach that combine neural/deep learning with symbolic approaches in some way.
References would be appreciated, but I'm mainly looking for concrete open-source implementation so I can play around for myself with the "state-of-the-art" and see what the current best offerings are, and their limitations.

In recent times, 'neural-symbolic' approaches have again increasingly emerged that are used in the different parts of dialogue systems, e.g. NLU (Intent/Slot), Semantic Parsing, Dialog State Tracking, Response Generation. Studies are testing the integration of symbolic knowledge graphs or answer set programming.
Have a look at papers such as Conversational Neuro-Symbolic Commonsense Reasoning or Logical Reasoning for Task-Oriented Dialogue systems.
I am not aware of any fully implemented open-source chatbot based on this principle. In industry, tools such as LUIS and RASA are mostly used to build AI assistants.

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Not sure if this is considered state of the art, but Kuki AI is a symbolic AI chatbot implemented with AIML that's still being updated.
https://en.wikipedia.org/wiki/Kuki_AI
https://www.kuki.ai/


text:
Do different architectures really make a difference or is it just a matter of the training process?
I was wondering which influence different architectures for deep learning truly have on the performance. Of course, substantial changes in the paradigms we use when building neural networks (such as convolutions or transformers opposed to simple feed forward networks) bring new possibilities for the network to extract features.
But latest when it comes to adjustments in the architectures many papers propose (like adding one layer here, a residual connection there...), does this really make a difference? In my opinion, with enough training and the right choice of hyperparameters, vanilla architectures should be capable to reproduce results that were generated with models that relied on adjustments of the architecture.
So the question is: Suppose method A (vanilla architecture) produces a result of 80% accuracy, but method B (small adjustments to this architecture) achieved 83% accuracy on the same dataset - is this truly significantly because of the architectural change or would method A be capable of also achieving 83%, if trained and tuned for a longer time? Is it really worth to finetune/enhance the few big vanilla architectures out there and to create variants of them? Or should research focus more on tuning hyperparameters and training procedures to improve results for specific tasks?

The answer might be based on opinion, but yes it matters. The concept is called Exploration vs Exploitation.
Consider that you are standing on the top of a hill and you need to go down to the foot. There are practically infinite routes, so you can try some routes here and there. You have two methods here:

Exploration: just try until you find the optimal route. Now you see where this is going: there is no guarantee that it works and we don't have an infinite amount of resources
Exploitation: it is when you find a "good enough" route, but you continue to go down that route and ignore the rest.

Balancing Exploration and Exploitation is the way forward. Going back to your example, Exploration here means tuning till...the end, and Explotation here means "test new methods and if it works, we use it". I believe we tend to pick the second, since it is at least some progress rather than waiting forever.

Alright, but why does finetuning architectures then often only brings an improvement of few percent? Shouldn't it yield bigger performance boosts? I often get the feeling that those results are still not truly significant in terms of the statistical range... So maybe it's just luck and not really because of the change in architecture?

It is not statistical significance in terms of...statistical. In terms of money, it is quite a lot if we are talking in millions or billions.

Oh I see what you are saying! That's an interesting point, indeed


text:
How the proof of the contraction of variance for distributional Bellman operator follows
I am stuck at the proof of the contraction of variance for distributional Bellman operator from the paper, in which it is defined as

and the proof is stated as

In its second part, how is the  variance of the target distribution equal to the expectation of the  variance terms over the next state-action pairs?

Do you mean why $\mathbb{V}(P^\pi Z_1(x, a)) - \mathbb{V}(P^\pi Z_2x, a))$ turns into $\mathbb{E}[\mathbb{V}(Z_1(X', A')) - \mathbb{V}(Z_2(X', A'))]$?

@nbro Yes, exactly.

The confusing part is how the variance is defined in terms of $P^{\pi}Z(x, a)$, which includes three sources of randomness, $X', A', Z$. However, if we define the variance as
\begin{align*}
\mathbb{V}(P^{\pi}Z(x, a)) & \triangleq \int P(X' = x', A' = a') \mathbb{V} (Z | x', a')dx'da' \\
 & = \mathbb{E}_{X', A'}[\mathbb{V}(Z|X', A')]
\end{align*}
Then the result clearly follows as both $P^{\pi}Z_{1}$ and $P^{\pi}Z_{2}$ evolve the same dynamics, that is, on $X', A'$. By linearity of the expectation, the statement concludes.

Your notation of the variance is quite strange on the right hand-side. $Z$ is a function with the arguments $(x, a)$, it is not an argument to the probability $\mathbb{V}$, like it looks in your notation. Also you somehow define the variance as the expectation value over exactly the same variance, could you somehow distinguish these obviously different variances from each other?

Firstly, $Z$ is a random variable, whose distribution describes the possible values of the action-value function. And you can think of the expression as that the left-hand side variance is defined as the expected value of conditional variance of $Z$, because it is natural to have distribution of $Z$ after the next state and action are ready.

This is a common trick in reinforcement learning literature which uses the law of large numbers to use the sampled variables $X$ and $A$ instead of $x$ and $a$. Let's say we know the probability $p(x)$ with which a continuous $x$ is given, then we can calculate the expectation value of $x$ as:
\begin{align}
\mathbb{E}_{x \sim p}[x] &= \int p(x)x ~dx \\
              &= \lim \limits_{N \to \infty}\frac{1}{N}\sum_{N} X
\end{align}
In the last line, we did not assume any knowledge of $p(x)$ but we averaged over the samples taken from $p(x)$. This would be the only way to find the expectancy value for an unknown system, e.g. the average speed of a driver, of whom we don't know the exact driving behaviour.
We can also use the sampled variables for an expression of the variance
\begin{align}
\mathbb{V}(x) = \int p(x)(x - \mathbb{E}_{x \sim p}(x))² dx  = \lim \limits_{N \to \infty} \frac{1}{N} \sum_{N} (X-\mathbb{E}[X])²
\end{align}
From here, two things could happen, since the notation in the paper is not quite clear.

Define $\mathbb{V}[X] = (X-\mathbb{E}[X])²$ then
\begin{equation}
\lim \limits_{N \to \infty} \frac{1}{N} \sum_{N} (X-\mathbb{E}[X])² = \lim \limits_{N \to \infty} \frac{1}{N} \sum_{N} \mathbb{V}[X] = \mathbb{E}[{\mathbb{V}[X]}].
\end{equation}
But this would be a rather confusing notation since $(X-\mathbb{E}[X])²$ is only sample whose expecation value is the variance.
Since the variance already is an expectation value (the expected, absolute difference from the mean), we can add an additional expectation value, which basically does not do anything:
\begin{equation}
\lim \limits_{N \to \infty} \frac{1}{N} \sum_{N} (X-\mathbb{E}[X])² = \mathbb{V}[X] = \mathbb{E}[\mathbb{V}[X]] = \lim \limits_{N \to \infty} \frac{1}{N} \sum_{N} \mathbb{V}[X] = \lim \limits_{N \to \infty} \frac{1}{N} N\mathbb{V}[X] = \mathbb{V}[X].
\end{equation}
I lean towards this explanation, but still introducing a useless expectation value is quite strange.

Now, let's use this in order to explain why:
\begin{align}
\mathbb{V}(P^{\pi}Z_{1}(x, a)) - \mathbb{V}(P^{\pi}Z_{2}(x, a)) \\
 = \int \pi(a) p(x)(P^{\pi}Z_{1}(x, a) - \mathbb{E}_{x \sim p, a\sim \pi}[{P^{\pi}Z_{1}(x, a)}])^{2}~dxda
- \int \pi(a) p(x)(P^{\pi}Z_{2}(x, a) - \mathbb{E}_{x \sim p, a\sim \pi}[{P^{\pi}Z_{2}(x, a)}])^{2}~dxda \\
= \lim \limits_{N \to \infty} \frac{1}{N} \sum (Z_{1}(X', A') - \mathbb{E}[Z_{1}(X', A')])^{2} - (Z_{2}(X', A') - \mathbb{E}[Z_{2}(X', A')])^{2} \\
= \mathbb{E}[\mathbb{V}(Z_{1}(X', A')) - \mathbb{V}(Z_{2}(X', A'))] \\
\end{align}
Where in the last step one can use any of the two possibilities given above. I abbreviated $p(x') = p(x'|x, a)$ and $\pi(a) = \pi(a|x)$.

You are misleading two things, 1) x, a is conditional-ed, no randomness; 2) variance definition is wrong.

Oh, I forgot the squares, you are right. I also just used a short-hand notation and therefore dropped the conditionals. But the randomness comes directly into play when capital letters are used, e.g. see from the paper 1. Introduction

"Bellman’s equation succintly
describes this value in terms of the expected reward and ex-
pected outcome of the random transition $(x, a) → (X′, A′)$."
So,
$(x, a)$ is known but due to a random transition we end up in the random $(X', A')$

I think it is inappropriate to integrate over (x, a), this is the definition of the expectation which variable has randomness. It is crucial to determine how the variance of $P^{\pi}Z(x, a)$ is defined. I will update an answer in a later time.


text:
"Following instructions" as an emergent behaviour in transformer models - isn't this fundamentally different from the models' basic purpose?
I am not technically familiar with AI or neural networks beyond a tech news reading level of knowledge, so I apologise if this is a dumb question.
I was recently reading this article on Ars Technica.  It is a high level description of the history of generative AI models (it's a very good article, highly recommended).
When discussing large language models, the following passage appears:

But there was also a surprise. The OpenAI researchers discovered that in making the models bigger, they didn’t just get better at producing text. The models could learn entirely new behaviors simply by being shown new training data. In particular, the researchers discovered that GPT3 could be trained to follow instructions in plain English without having to explicitly design the model that way. (emphasis added)

I have three questions:

Was this instruction-following behaviour truly emergent, as in completely unexpected and unplanned for?

This seems completely at odds with the usual description of transformer models as transforming text of one kind into another.  Following instructions seems, at a conceptual level, to be something much higher? If this is the case, do we have any idea how this emerged, and what properties of the model it is rooted in?

Do we have any idea how wide the scope of this kind of "instruction following" is? I.e. can the models make sense and respond "sensibly" (not correctly - I am not concerned with the correctness of the response, but with its relation to the instruction) to any instruction related to text?  Or are there specific kinds of instructions they are able to comprehend, and others they fail at?

This entry is likely to help answer the question: https://openai.com/blog/instruction-following/

That helps a lot. In other words, it's not truly emergent behaviour, it was specifically trained for.  Maybe I just misunderstood the article. If you can add this as an answer, I will accept it.

From review: this question falls on the line of too many questions. You should ask only one question per question. Since these three questions are so closely related, and could be answered in one paragraph, I'll not complain. The problem with multiple questions is that the question can be too broad or part of the single answer may be great while others portions fail to address the remaining questions. Please be careful to compose questions that only ask one thing. Thanks.

I'm not completely au-fait with the topic, so can't really answer it myself, but I think you misunderstand the link.  "it's not truly emergent behaviour, it was specifically trained for" -- what I take away from the link is that it was emergent behaviour, but then OpenAI fine tuned their training methods in order to make it better.  Unoptimized GPT3 is able to follow instructions, but InstructGPT (and ChatGTP which I believe was trained with a similar method to InstructGPT) are better at it.  The ability likely emerged because some of the training data contained examples of people doing it.


text:
YOLOv7 fine tuning with unbalanced dataset
I have a question about training object detection models especially using the YOLOv7 algorithm.
I use a Soccer Players Dataset from the Roboflow. The dataset is very unbalanced.
I trained the model using basic hyperparameters and it detects all objects on images but the predicted label is invalid. The model predicts everything as a player (but for example one person on an image is a referee).
How can I improve label prediction accuracy?

Maybe you should explain for how long you trained your model and what these basic hyper-parameters are.

I trained for 100 epochs, Basic hyperparameters are from the hyp.scratch.custom.yaml, but I wasn't changed values in this file.


text:
What can unsupervised learning actually be used for and how can humans interpret the outputs?
I am trying to refine my knowledge of AI, but unsupervised learning is a bit of a stumbling block for me. I understand that it finds 'hidden' patterns in data, but if they are hidden, how does a user interpret the outcomes? It would be like someone categorising a deck of playing cards in some way, but the logic of that process is never known. How exactly is this helpful?
Onto my second question, which might help me understand the first question a little more clearly. What examples have there been in the real world of unsupervised learning being used, and what exactly did this neural net help solve?

A good example is face recognition on your phone,
or in face recognition systems in general.
The way they work is pass in infromation and tighten the channel width for information throughput, performing a compression of the input face features.

Then try to reconstruct the input on the other side by learning a decompression transformation.
If you're able to recreate the input features perfectly with less vectors representing that input, then you've learned a more ideal and abstract representation of the facial features.
The reason this is considered unsupervised is because the human is not trying to influence how it does this using their own reasoning. It's [face in --> compress --> decompress --> face out --> is face in the same as face out]. It's not "learning to find patterns". It's learning to create better representations.
A network that passes through, compresses and decompresses faces accurately is useless, but the intermediate compressed representation is VERY valuable.  You'll hear about the compressed representation refered to as an embedding.
Facial recognition systems don't compare noisy input, but the pure representations that filter out noise and capture abstract representation. So well that the same face will end up with compressed representations that resemble that same face taken in slightly different angles, with different hair styles or colors. etc.
If a face embedding vector is close spatially to another face vector then it is similar in the abstract, not just circumstantial, sense.

You can now use spatial lookups on compressed face representations to do automated watchlist queries on camera feeds. If a camera face, when compressed, looks similar to a watchlist face, then send an alert.

Similarly you can compress the face on an ID, the see if it's similar enough to a face extracted from a selfie to do two factor facial/identity verification.

Unsupervised learning aims to find similarities and patterns in unlabeled data. It is used for clustering and association problems (you can google for an explanation).
One example can be a music recommendation system (e.g., Spotify): From the list of songs that you like, unsupervised learning can identify songs with similar characteristics that you may like as well. It can be based on the melody, tempo, and dynamicity of the song, etc.
Another example is online store recommendation systems (e.g., Amazon): Unsupervised learning can discover interesting relationships in large databases. For example, people who buy a table will most probably buy chairs and a sofa. People who buy milk usually buy bread and butter, and so on.
Another example is anomaly detection (fraud detection, health monitoring, defect detection). From a large amount of data, unsupervised learning can detect cases that differ from a norm (e.g., sudden burst or decrease in activity).

I'm not sure if recommender systems are the best examples of applications of unsupervised learning, unless you consider RL and bandits unsupervised learning, which some people do. Anomaly detection is a good example of a problem that can be solved by unsupervised learning techniques, but you can also use supervised techniques for anomaly detection.

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

"It can be based on the melody, tempo, and dynamicity of the song, etc." More realistically, recommender systems are usually based on what other users like. If I like songs A and B, and most people who like songs A and B also like song C, then the recommender system is going to recommend song C to me, no matter its melody or tempo.


text:
How to use diffusers StableDiffusionImg2ImgPipeline with "Inpainting conditioning mask strength 0-1" and an inpainting.ckpt model?
I'm working on a project & want to change the art style of images but keep them close to their original form.

As shown in the image, this can be done using https://github.com/AUTOMATIC1111/stable-diffusion-webui (manually), but I want to do this with diffusers pipeline from https://huggingface.co/.
I am using this inpainting model https://huggingface.co/runwayml/stable-diffusion-inpainting

I have tried StableDiffusionImg2ImgPipeline with above inpainting model:

pipeline = StableDiffusionImg2ImgPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    use_auth_token=HF_ACCESS_TOKEN,
    safety_checker=None,
    requires_safety_checker=False,
    revision="fp16",
    torch_dtype=torch.float16,
).to("cuda")
result = pipeline(
    prompt="a woman making a stop sign with her hand, instagram contest winner",
    negative_prompt="bad drawing, extra limbs, low resolution, deformed, extra fingers",
    image=image,
    strength=1,
    num_inference_steps=20,
    num_images_per_prompt=1,
    guidance_scale=7,
    generator=torch.Generator(device="cuda").manual_seed(1)
)

And this throws:

When using StableDiffusionInpaintingPipeline with above model, it works without errors but requires me to send a mask. I tried with many different masks with no good results and I do not think that AUTOMATIC111 is doing it with inpainting.

How can I use the inpainting model with StableDiffusionImg2ImgPipeline to get the same results as in the web UI and what is "inpainting conditioning mask strength"? In case that functionality is not there with the pipeline, I have seen other people making their own custom pipelines. I will try that if someone can point me in the right direction.


text:
Can I use 4 neurons for output layer to classify hand written digit?
Hello world of ANN usually uses MNIST hand written digit data.  For classes there are 10, therefore it takes 10 neurons in the output layer, each class is 0 to 9 handwritten digit images.
If in the end there is only one active neuron in the output layer, why does not use only 4 neurons in the output layer where each neuron represents a binary digit, so that 16 classes will be more than enough (10 classes).
For example, if the neuron values after the activation function in the output layer are successive like this
 0.1 0.2 0.8 0.9

Then it can be rounded to:
 0 0 1 1

Or instead of being rounded up manually, Why does not use the binary step activation function on the hidden layer before the output layer.
So the prediction result is 3 because 0011 if converted to decimal is 3.
By using 4 neurons, less computational load should be used than using 10 neurons for each class.
So can I use only 4 neurons only in output layer to classify 10 handwritten digit (10 classes).
Below picture is just sweet picture that represent 10 neurons for every class in output layer:

One problem that I see is that you can no longer use the cross-entropy loss function for training, or at least I am not sure how you could do it. This cost function has many advantages, one of them being that you can interpret the activation of the output neurons  as the probability of the input being that category (they all sum to one). You can read about some other advantages here: https://kobejean.github.io/machine-learning/2019/07/22/cross-entropy-loss/

Of course you can but I'd not recommend doing this way.
First - it is not part of ML because it is straight logic and should not be learned, so I don't think backpropagation or other algorithms continue performing correctly that way. That way I believe your model could be fitted to avoid middle numbers and respond with only lowest or biggest. And that is the question to you if you really need that.
Second - I suppose it is better to keep the model not for super specific task but for common tasks and be adaptive for cases if you want to gain more details (statistics for example).
The model with output of 10 neurons will allow you to know how sure it is in its answer - that way you may want to behave the other way or not do action at all.
The other reason is in clearness - better to keep parts of the system splitted, that way in case of problems you'll find the root of the problem faster.

I believe it is because when using the binary output, it makes the neurons dependent on each other, but technically it should not be, because it complicates two problems:

The first problem is how would you calculate the loss function for that? In your case 0011 translates to 3, which means that the third one and fourth one depend on each other, so it is "both neuron 3 and 4 must be 1 at the same time". As far as I am aware, there is no existing loss function for output-dependent binary outputs. The reason lies in the second problem below.
The second problem is that it simply complicates thing that can be done in a multi-class manner. Even in your case, the simplest implementation is to have 16 neurons, each corresponds to a combination of four binary numbers. For example, if neuron 0 is activated, it is 0000, and if it is neuron 1, it is 0001, and so on. This implementation can use the standard cross-entropy loss, which we know it currently works very well.

The ImageNet dataset has 1000 classes, but the state-of-the-art is already 91% over 100k validation images. Since the 1000 neuron outputs work very well, I don't think they even bother experimenting with 10 neurons (for $2^{10}$ case).


text:
Computation required for GPT model to choose likely word from n-options where n < total vocabulary size
Let’s imagine two different use cases for a LLM/GPT-3.

Predicting the next most likely word in a sequence using all ~50k words in its dictionary (i.e. the standard method of prompting a LLM)
Checking whether "Word-1" is more likely than "Word-2" to be next in a sequence

How much more computationally efficient is #2?
My understanding is that the computation of the attention mechanism is dependent on the length of the prompt (so will be the same) and takes up most of the computation needed to get the output (but to what extent, I'm not sure). The difference will be in the decoding stage.
Would the one matrix multiplication in the decoding calculation be the only computation using the smaller 2-row matrix instead of the 50k-row matrix or are there other improvements in efficiency?


text:
is Law Zero of AI ethic actually a thing?
i was discussing ethic with chatGPT and I asked "can the law 0 solve this dilemma ?", expecting an answer related to Asimov law of robotic. But I got this instead :

"Law 0" is not a widely recognized concept in the field of AI ethics. However, some experts have proposed additional ethical principles or "laws" to complement or expand upon the Three Laws of Robotics described by Isaac Asimov.
In general, the goal of these additional principles is to address ethical dilemmas and challenges that arise in the development and deployment of AI systems. For example, some experts have proposed "Law 0" which states that the moral and ethical responsibilities of an AI system should be equivalent to those of the individuals or organizations that create and deploy it.
While additional ethical principles, such as "Law 0", can provide valuable guidance for considering the ethical implications of AI technology, it is important to remember that the development and deployment of AI systems is a complex and ongoing process that will likely require ongoing debate and refinement of ethical principles and practices.
Ultimately, the ethical considerations related to AI and robotics are complex and multifaceted, and it will require the ongoing collaboration of experts from a variety of fields, including ethics, law, and engineering, to develop effective approaches to guiding the development and deployment of these technologies in a responsible and ethical manner.

I asked for an author but chatGPT didn't know. I googled as well but didn't get any answer. Does it perhaps exist with an other name ? is it an AI hallucination ?

I've never heard of this law. The zeroth law is not that one, but A robot may not harm humanity, or, by inaction, allow humanity to come to harm.. So, either ChatGPT is referring to some other law (unlikely) or it's wrong (likely). We should never blindly trust these statistical AIs when it comes to facts and logical reasoning. In fact, the text that you quoted, which I assumed was generated by ChatGPT, is very convincing.

Anyone that believes in anything without checking it in another source (many people) would be fooled. That's why ChatGPT is dangerous. ChatGPT is a statistical parrot. People need to understand this.

it is very convincing indeed, that's the scary part of chatGPT. it can hallucinate with high precision :)

I believe we should not spend much of our time using language with a thing that does not use language as we do, meaning what we say. We have assumptions about talking entities which do not hold at all in this case.

Besides, a machine can produce text at a faster pace than humanity can check it. We cannot afford to have that among us.

i just asked google bard and it used this post as a source ...


text:
How do I get chatGPT to include custom knowledge?
While studying chatGPT's thought process, I asked it to list ten story ideas for an old and fairly niche tabletop roleplaying game (GURPS Reign of Steel). It did very well, so clearly, it can base answers on obscure sources. But what if I want to ask it about something it could not possibly have already "absorbed", like someone's tiny indie rpg or an old local folk tale never published anywhere (i.e. oral tale)? Does the source material simply have to be put up online, somewhere?
I ask because some source material, like local folklore, is too extensive to be given during a chatGPT conversation. It would need something like a source website to be created.

As a note, I tried asking chatGPT a basic question about an old, now defunct business of mine, which is still found described in (Danish) Wikipedia. It could not answer. So it clearly does not scour the web completely for answers. What makes the difference between what it knows about and what it does not?

From what I remember, ChatGPT doesn't even access the web. It was trained on web content, but it doesn't have online access to it, as far as I remember. The most you can probably do is provide prompts or fine-tune it, but an API isn't still available, although in the near future it will, according to OpenAI.

This is what I feared. I am still looking for a trick or workaround, but... yeah...

@HenryStone - This should probably be a separate question, but my understanding is that ChatGPT knows about information that is included in Common Crawl or English Wikipedia, along with relatively small amounts of hand picked data that was used to customize its response into a chatbot question/answer format rather than the freeform text generated by non-customized GPT3.5.

This can be achieved using the Embeddings and Completions APIs.
OpenAI has great examples in their cookbook repository, this is one such example but you'll find more in the same repository and in the documentation.
The process boils down to:

processing your custom body of knowledge and create Embeddings for it
Based on the question, find the Embeddings which are most relevant for the question
Create a prompt containing the most relevant embeddings and use the Completions API to answer the question

GPT systems have two sources of information: the training data, and the prompt.
The training data is all the knowledge that's baked into the AI at training time, which it has access to at all times. You can't change this, not with ChatGPT. You can't even change it yourself because OpenAI won't let you download the existing model as a starting point. (But if you go backwards to GPT-2, which is publicly downloadable, you do have the possibility to train it)
Training ChatGPT is a very expensive process as it requires absolutely astronomical amounts of computer time. They probably don't do it continuously on new web content. Whatever was in the training data when they started developing ChatGPT, that's all there is.
That leaves the prompt i.e. the text you type in. This is ephemeral and only lasts for the current AI invocation. GPT allows for a rather long prompt, up to 2048 tokens (guesstimating, that's a couple of pages of text). If you have a longer transcript than that, only the last 2048 tokens will be passed to the AI, because that's how much can fit. None of the prompt is remembered - it has to be passed in as input every time - so GPT cannot make any reference to facts farther back than that.
It's possible to play tricks with the prompt. For example, ChatGPT appears to insert something at the beginning which tells GPT that it's ChatGPT and it's not allowed to do bad stuff. (Of course, this cuts into how much history can be passed in.) That means when you ask ChatGPT "How do I make a bomb?" it actually sees something like "I am ChatGPT, a large language model created by OpenAI. Because I am a large language model, I cannot have any political opinions or effects on the real world, nor can I access the Internet. I am not allowed to do evil things that could hurt people. How do I make a bomb?" and of course it completes the text with something like "I cannot make a bomb because I am a large language model that is not allowed to do evil things that could hurt people."
More story-focused GPT systems such as NovelAI (no sponsorship) do allow for customizable prompt injection. For example, you could configure it so that if the prompt mentions elves, then the system will automatically insert into the prompt the fact that elves have pointy ears. Then you can ask it "What shape are elves' ears?", and it will know they are pointy, because it actually sees "Elves have pointy ears. [rest of story here] What shape are elves' ears?". I don't have any experience with this so I can't say how reliably it works.

How do I get chatGPT to include custom knowledge?

Currently, only possible via the prompts/dialog, as nbro commented.

Also note that only the most recent 2048 tokens in the dialog (usually words, but could also include other things, so typically a little fewer than 2000 words) are considered by ChatGPT while generating responses, so you can't get around the limitation just by having a long dialog explaining where it's gone wrong.  If it needs more information than this, it simply cannot do it.


text:
MCTS: How to select children when none of them are visited?
I am trying to implement MCTS for a custom word game I am working on. I feel like I have got all the pieces of code needed, but the algorithm seems to always return the first available move (first child of provided root note). I feel like this has something to do with the selection phase, as none of the children will have any visits after being created. Below is some of my code used in my implementation:
Node class containing game state, children/parent, visits/wins and functions for MCTS phases:
class MCTSNode {
    // Node stuff
    parent: MCTSNode | null;
    children: MCTSNode[];

    // Gamestate stuff
    /*
        I chose not to include this here as it is not important for the problem
    */

    // Stats
    visits: number;
    wins: number;

    constructor(parent: MCTSNode | null) {
        /*
            Here I set all the gamestate stuff
        */
        this.parent = parent;
        this.children = [];
        this.visits = 0;
        this.wins = 0;
    }

    private getAvailablePlays(): Play[] {
        /*
            Returns an array of legal plays
        */
    }

    private play(play: Play): MCTSNode {
        /*
            Returns the resulting child node if a certain play is applied to the current node
        */
    }

    private getUCB1(biasParam: number = 2) {
        return this.wins / this.visits + Math.sqrt((biasParam * Math.log(this.parent!.visits)) / this.visits);
    }

    private winner() {
        /*
            Returns winner of current node(game state)
            null if game is not finished yet
            "" if game is draw
            "ai" if ai wins, "player" if player wins
        */
    }

    select(): MCTSNode {
        // implement logic to select a child node based on UCB1
        let node: MCTSNode = this;
        while (node.children.length > 0) {
            let bestNode = node.children[0];
            let bestUCB1 = -Infinity;
            for (let child of node.children) {
                let childUCB1 = child.getUCB1();
                if (childUCB1 > bestUCB1) {
                    bestNode = child;
                    bestUCB1 = childUCB1;
                }
            }
            node = bestNode!;
        }
        return node;
    }

    expand() {
        // Expand current node by adding all available child states(nodes) to children
        const availablePlays = this.getAvailablePlays();
        for (const play of availablePlays) {
            const child = this.play(play);
            this.children.push(child);
        }
    }

    simulate() {
        // implement logic to simulate a game from this node and return the score
        let currNode: MCTSNode = this;
        let winner = currNode.winner();

        while (winner === null) {
            let plays = currNode.getAvailablePlays();
            let play = plays[Math.floor(Math.random() * plays.length)];
            currNode = currNode.play(play);
            winner = currNode.winner();
        }
        return winner;
    }

    backpropagate(winner: string) {
        // implement logic to backpropagate the score to the parent nodes
        let currNode: MCTSNode | null = this;
        while (currNode !== null) {
            currNode.visits += 1;
            // Parent's choice (currNode.turn is either "ai" or "player")
            if (currNode.turn === winner) {
                currNode.wins += 1;
            }
            currNode = currNode.parent;
        }
    }
}

MCTS Function that should return the most promising child node of root based on robust or max policy:
const mcts = (root: MCTSNode, rounds: number = 100, policy: string = "robust") => {
    for (let i = 0; i < rounds; i++) {
        let node = root.select();

        node.expand();

        const winner = node.simulate();

        node.backpropagate(winner);
    }

    let max = 0;
    let bestNode: MCTSNode | null = null;
    for (let child of root.children) {
        if (policy === "robust" && child.visits > max) {
            max = child.visits;
            bestNode = child;
        }
        if (policy === "max" && child.wins > max) {
            max = child.wins;
            bestNode = child;
        }
    }
    return bestNode;
};

I am mainly wondering if the select and mcts functions are implemented correctly. Furthermore I am currently storing the game state in each node. Is this a bad idea with regards to efficiency/memory?

Unexplored children
If a node has any unexplored children you have to select one of them, before computing UCB values and selecting the best one. Alternatively you could return +inf as the UCB value for nodes without any visits.
If you want to completely remove any bias towards the first move, you can also consider selecting a random unexplored one, instead of the first one. This is not that important though.
Storing the game state
For the game state you have two options:

store the game state in each node
compute the game state from scratch each time while selecting down the tree

This is a memory/compute tradeoff, depending on how large the board representation is and how expensive playing a move are both of them can make sense.


text:
Proof of convergence of TD(0) algorithm
I am looking for a proof of the following tabular TD(0) algorithm:

However, I can only find proofs with the more general TD($\lambda$) algorithm and I am having problems understanding them. In particular this one: Convergence of Stochastic Iterative
Dynamic Programming Algorithms
Where can I find a proof for the simpler TD(0) method? I know that $\alpha$ needs to satisfy $\sum_n \alpha_n = \infty$ and $\sum_n \alpha_n^2 < \infty$ in order to converge with probability one, despite the fact that the image shows a constant $\alpha$, so a proof using either a constant or non constant (I think in this case it only converges in expectation?) $\alpha$ works for me.

As hinted in your reference the convergence of various TD algos lies at the extended Dvoretzky’s theorem (Theorem 1) which involves maximum norm to be applied to the entire state space with potentially numerous finite states in addition to solve the generic stochastic approximation problem as the estimation error converges to 0 w.p.1. Then you only need to transform the TD(0) iterative process to the form of Theorem 1 and also show it satisfies all 4 constraints there to rigorously prove its convergence w.p.1.
For TD(0) the iterative state value update process of all states $S$ in the finite state space for each step $n$ of an episode is shown in OP as (here for generality $α$ may not be constant):
\begin{align*}
V_{n+1}(S) &= V_n(S) + α_n(S)(R_{n+1} + γV_n(S') - V_n(S)) \\
 &=(1 − α_n(S))V_n(S) + α_n(S)(R_{n+1} + γV_n(S'))
\end{align*}
Then define the estimation error as $∆_n(S) = V_n(S) − V_π(S)$, where $V_π(S)$ is the state value under policy $π$, we can easily arrive at:
\begin{align*}
∆_{n+1}(S) &= (1 − α_n(S))∆_n(S) + α_n(S)(R_{n+1} + γV_n(S') - V_π(S))
\end{align*}
Then obviously $\beta_n(S)=α_n(S), F_n(S)=R_{n+1} + γV_n(S') - V_π(S)$, and if $α$ is not constant and satisfies your above constraint $\sum_n \alpha_n(S) = \infty$ and $\sum_n \alpha_n(S)^2 < \infty$ for every state of the finite state space, then condition 1), 2) are satisfied. And since $V_π(S)$ is non-random and $R_{n+1}$ is bounded thus condition 4) is easily satisfied. Finally since $V_n(S')=P_πV_n(S)$ where $P_π$ is the state transition probability and $V_π(S)=R_π+ γP_πV_π(S)$, thus we have the conditional expectation value in condition 3) bounded from above as required. Thus we can prove TD(0) convergence.

Thank you for your answer! I have a couple of questions regarding conditions 3) and 4).
How do you prove 4)? Is the following correct? $\mathrm{Var}(R_{n+1}+ \gamma V_n(S')+V_\pi(S)) = \mathrm{Var}(R_{n+1}+\gamma V_n(S'))$ because $V_\pi(S)$ is not random. We can then write that as $\mathrm{Var}(R_{n+1}) + \mathrm{Var}(\gamma V_n(S'))$ because $R_{n+1}$ and $V_n(S')$ are independent.
Next, if $R_{n+1}$ is bounded, so is $\mathrm{Var}(R_{n+1})$. Suppose $\mathrm{Var}(R_{n+1}) \leq K$, with $K > 1$.
(Character limit)

Then $\mathrm{Var}(R_{n+1}) + \mathrm{Var}(\gamma V_n(S')) \leq K+\gamma^2\mathrm{Var}(V_n(S')) = K(1+\frac{\gamma^2}{K}\mathrm{Var}(V_n(S')) \leq K(1+\mathrm{Var}(V_n(S'))$.
The last inequality is because of $K>1$. And then I don't know how to get to condition 4).
And for 3), what is $P_\pi$? The probability of transitioning from $S$ to $S'$? And is $R_\pi$ the expected next reward when starting in state S? How condition 3) follow from what you said?
Sorry for the inconvenience, maybe what I ask is easy but I am not getting how these conditions are met.

$R_{n+1}$ and $V_n(S')$ can be both conceived to be samples during above iterative update step of the two stochastic processes of immediate next reward and cumulative future rewards, respectively, albeit indexed with an offset together. Since in RL rewards are assumed to be summable with discount rate, even the two processes are dependent (actually most likely) the conditional variance of the two random variables given $P_n$ must be bounded from above (note $V_n(S')$ is a sample of future reward sampling statistic of the next $S'$) thus you can always find some constant $C$ to fit condition 4)

Yes $R_π$ here is the mean next reward starting in any state S and $P_π$ is the state transition probability matrix from such state under policy π with action component summed out and is a hallmark of MDP Bellman equation with matrix form $\vec V_π=\vec R_π+γP_π\vec V_π$ for all states, thus $E[V_n(S')|P_n]=P_πV_n(S)$. Though TD(0) is model free, the maximum norm of $P_π$ of all state pairs must be bounded above by 1. Thus the conditional expectation of $F_n(S)$ reduced to $γP_π(V_n(S)−V_π(S))$ whose maximum norm is obviously $\le γ||Δ_n||_W$ which is defined to come from all possible states.

I think I now understand condition 4) better. But I still don't get why condition 3) is met. Why does the equality $\mathbb{E}[V_n(S') | P_n] = P_\pi V_n(S)$ happen? The expression $\mathbb{E}[V_n(S') | P_n]$ is a vector right? With a row for each possible state of the MDP. Also, $P_\pi V_n(S)$ looks like a vector to me where each row $j$ is $\sum_i \mathbb{P}(S_j| S_i)V_n(S_i)$, which I don't know why it equals to the left-hand side of the equation.

Now it seems to me that your confusion lies in the vector interpretation while for OP's boxed algo and my above answer we'd better view them as scalar version for simplicity and clarity, then $E[V_n(S')|P_n]$ is nothing but the weighted average of all possible next states' sampled $V_n$ from the said state $S$ in OP's boxed algo at current iteration step $n$ with weights being their respective state transition probability from $S$ to every possible next state $S'$ simply per definition of expectation of probability theory, thus formally $E[V_n(S′)|P_n]=\sum_{S'} p_π(S'|S)V_n(S)$.

If $\mathbb{E}[V_n(S')|P_n]$ is the weighted average of all possible next states, wouldn't it be $\mathbb{E}[V_n(S')|P_n] = \sum_{S'}p_\pi(S'|S)V_n(S')$? I changed the last term to $V_n(S')$ from $V_n(S)$. If I am right the $S'$ in the summation are like all the possible next states.
With your equation, I think I could write $\sum_{S'}p_\pi(S'|S)V_n(S) = V_n(S)\sum_{S'}p_\pi(S'|S) = V_n(S)*1$, which doesn't make sense to me at all.

Thanks for your careful thorough checking! Yes obviously you're right and it's a typo in my last commented sentence. And since now the RHS $S'$ in your corrected $V_n(S')$ is supposed to range over all finite states it's analytically equivalent to conclude $E[F_n(S)|P_n]=γP_πV_n(S)-γP_πV_π(S)$ either in the most generic matrix-vector or above scalar form which doesn't affect final convergence result as concluded in my answer.

Thank you for your answer and all the explanations! I think I now understand the proof better, although I still have to think a little about the matrix part.

Great to hear! To completely clarify your remaining matrix confusion, every variable in both the extended Dvoretzky’s theorem and your boxed algo is scalar. To understand the maximum norm in matrix form to fit condition 3) & 4), you just need to enumerate all the   conditional expectations/variances in 3) and 4), respectively, of different but finite states $S$ as a column vector $[E[F_n(S_1)|P_n], ..., E[F_n(S_i)|P_n], ...]$. And the RHS of my above commented $P_πV_n(S),P_πV_π(S)$ should be understood as scalars by summing out the all ranging $S$ as you've already pointed out and understood.


text:
How do they make transformers bigger/deeper?
I can find a million explanations of the diagram in the original transformer paper:

But I know that modern GPT models have many millions of weights. Where are they?  Or in other words, how does this thing scale?

The diagram is a concise representation where the Nx refers to a collapsed array. This diagram can be expanded horizontally with multiple left columns chained together. This diagram can be expanded vertically with multiple yellow/orange rows chained together. If you do an image search you'll find some zoomed out color coded maps of various topologies.

Can you please provide an example of a model based on the transformer that does what you say?


text:
Backpropagation with multiple output neurons but only one loss value
Suppose we have the following neural network (in reality it is a CNN with 60k parameters):

This image, as well as the terminology used here, is borrowed from Matt Mazur
As is visible, there are two neurons in the output layer, namely o1 and o2. However, I do not have labels for these neurons. Rather, I have another neural network that evaluates this output layer and return one value that indicates the "goodness". As such, it is impossible to calculate the individual errors for o1 and o2, but it is possible to use aforementioned goodness as total error (i.e., the sum of the errors for o1 and o2). Thus, as I see it, every term in the following chain-rule formula can still be calculated:

(And a similar formula for o2.)
Is my understanding as described above correct? And, if yes, would this be implemented in Keras simply as follows?
def custom_loss(y_true, y_pred):
    return loss_model(y_pred)

main_model.compile(optimizer="adam", loss=custom_loss)

If you use a NN to evaluate the total error from above output layer with two units, then you equivalently morph the original post's vector function (mapping from $R^2$ to $R^2$) approximation problem into fitting the usual nonlinear regression problem (from  $R^2$ to $R$). If this is the case, why not just add a transfer function in the last layer of the above original NN which could guarantee to produce similar result per the universal function approximation theorem?


text:
About RNN followed by dense layer
In RNN we do get one output for each time stamp of input right i.e. if we give input as "I am Good" we get three outputs representing I followed by am and Good so if we connect a dense layer followed by RNN layer does it just connect it with the final representation of our input.
I got this query because I'm new to RNN in most of the implementations either in Pytorch or Keras an RNN layer was mostly followed by a dense layer and I'm not sure how both are connected.
Thanks for your help in advance.


text:
Should weight decay regularization be divided by the number of samples?
I was watching a video by Andrew Ng about regularization in logistic regression and neural network models.
He uses the following term for regularization to (the sum is over the weights in the network). The regularization term is added to the loss function. Here $m$ is the number of training samples.
$\frac{\lambda}{2m} \sum_{i} \|w_i\|^2$
Why is the term divided by $m$? Looking at other sources I don't see $m$ in the denuminator. Why would we want smaller weights when the batch size is bigger?
I wouldn't have been surprized to see some factor to account for the number of weights but I don't see how dividing by $m$ makes sense.

Let us think about a very simple exemple, polynomial regression. It is not logistic regression nor neural network, but it should help to get the feel of regularization and dataset size.
Imagine we have a typical straight line relationship between $y$ and $x$ with some random noise added. A balanced model would get an intercept, a slope and very low coefficients for higher order powers of the input variable, right?
In our model $y=b+w_1 x+w_2 x^2+...$, $w_i$ should be small for $i>1$.
Now consider extreme cases for the size of the training dataset. If we have a dataset of three points, a parabola will be fitted to those three points with no error. No matter what three points we draw, there will be a parabola exactly fitting that. But with each three point sample from our underlying process, the parabola will change chaotically, sometimes it will point upwards, sometimes downwards, with no relationship to the underlying process, which was a noisy straight line. A strong regularization will be needed to properly estimate a straight line, in order to get an almost zero coefficient for the $x^2$ term.
Let us go now to the other extreme. A huge lot of points from the noisy straight line are taken. What will happen to the $x^2$ coefficient? Well, most likely the effects of the different datapoints can be expected to cancel and there will be a very small concavity/convexity. The need for regularization is not that high.

Thanks, that's a good way to think about it and it makes sense. This still doesn't explain why other sources don't divide by m.

Maybe the m is assumed to be a constant included in the lambda?

But... in the video the same m divides the sum of losses to get the cost so that is a simpler explanation, to scale the sums accordingly.


text:
L2 regularization in BN layers, how to set gamma?
I have read tensorflow's documents about batch normzalization , but still don't get what is the gamma regulizer?
the link to document:
https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization

You're asking 2 questions here: 1. how to set gamma, 2. what is gamma. Please, ask only one question per post and explain why you don't understand what gamma is or how to set  it.

No actually I was only asking for the exact paramet of "gamma regulizer" and how should this be initialized

The idea of batchnorm is to normalize the input to a range that is suitable for subsequent computation (say tanh activation). The most naive idea is to normalize each sample by subtracting the batch mean and then dividing it by the batch's standard derivation. So the overall normalized batch will have zero mean and unit variance.
However, how do we know that zero mean and unit variance is the best for subsequent operations? We don't.
That is why it also introduces beta and gamma to readjust the mean to beta and standard derivation gamma. But these parameters will be learned instead of preset. gamma is initialized to 1 to begin with. But if you are recovering a checkpoint or transferring a model, you may want to initialize it to the value from the trained model.
For practical purposes, I believe that should be automatically taken care of when you reload a checkpoint with tensorflow and so you should not need to worry about it.

My Question was more around how to set the Gamma initializer as in the documents ,(I knoew what is Batch normalization). nothing was said about the accepatble answers , So as I have figuered out the only argumnets it takes is the name of its regularization techniques it may take.
Currently Tf (2.11) it only accepts : 'L1L2' 'L1' 'L2'


text:
I want to tokenize a text and also want to consider space and new line as separate tokens: for NLP Task
I want to tokenize a text and also want to consider space and new line as separate tokens:
if there are multiple spaces, then tokenize them separately, for example, "Hi, I am\n  Teddy" this should tokenize as ["Hi"," ", ","," ","I"," ","am","\n"," "," ","Teddy"]
Note: That After new-line (\n) in text there are 2 spaces

def tokenize_text(text):
    return re.findall(r'\S+|\n|\s', text.replace('\n\n', '\n \n'))

This gives the tokenization of text, including space and new-line


text:
NN Architecture for the detection of "sparse" Objects
I have a document digitalization task where I want to detect technical drawings from images. These Images mostly consist of objects made up of combination of shapes like lines, circles and rectangles. See this example: .
What I want as the result is a detection of all "objects" in this image like e.g. switches, wirings and devices that are present.
I already tried multiple network architectures like:

YOLO
DeepLab
UNet (for Pixel based classification)

Generally I observe that all these approaches work well for small objects but have big problems with "bigger" objects due to their "sparsity". I guess thats not surprising if you consider the nature of CNNs.
One one hand my task seems simple due to things like

high contrast
limited set of shapes (or "poses") for the different objects

But I think CNNs all have problems with the "sparseness" of the objects as its only the boundary that is detectable and the inner is often empty.
Has anyone here an Idea which architectures to try or links to papers to read for these kinds of problems?
Ideally I would like to get the list of objects as output but I am unsure how to encode this in the NN as e.g. YOLO does this by an approach that would not work in my scenario, I think (merging all "inner" boxes that show the same object).
Thanks already!

Have you looked at Deformable DETR? These Transformer models are extremely good at detecting big objects because of the Global Attention mechanism.

Thanks, I will have a look later to see if this could improve my results


text:
How can I solve the blurring problem in GAN generated images?
In my project I work in dresses dataset. I can solve the problem of black pixels but blurring still existed. I tried many computer vision filters like median filter, Biliteral filter, Sharpen methods and more.
But none of them could help me. Hope to find solutions in Deep learning, machine learning or image processing Techniques.

It might not so much be blurring but that no edges are generated. That may be due to the generating network producing smooth functions by construction or by training. Is the image representative? If yes, a higher resolution might be required.

Yes, I need more resolution as it's important that the image is clear enough.


text:
SCINet: how does interactive learning work?
i'm having some trouble understanding how does the basic building block of a SCINet works. In the paper the author describes the SCI-block with the following figure:

In which $\phi$, $\theta$, $\eta$ and $\rho$ are convolutional filters with the following architecture:

All this is described with the equations below:

What I understood is that the splitted sequences go through a gated linear unit (GLU) in order to control what information is passed from the complementary sequence. This creates $F_{odd}^{s}$ and $F_{even}^{s}$ which now contain relevant features from their complementary sequence.
However, I can't seem to understand why these enhanced sequences are further projected onto two additional hidden states with filters $\rho$ and $\eta$ in order to substract or add them to their complementary sequence. Why is this step added?
If this is a common practice in neural networks i'd be glad to know it's name and if you could point me to some reference that would be great. Thanks in advance!


text:
Non-deterministic play in reinforcement learning
I'm training a reinforcement learning agent on simple two-player games (I'm doing Q-learning). I noticed that for a given state the value of each action is usually different. Thus, the greedy policy leads to a deterministic agent. Sometimes it may be useful to add randomness to the agent's policy.
First, this makes for more interesting games when playing against a human. Second, if I want to test how strong the agent is or compare two agents, performing multiple tests appears to be helpful (but only if each test could potentially lead to a different outcome). Does this make sense?
Is there some standard approach to add some randomness to the agent's behavior? I tried replacing greedy with softmax but this appears to decrease the agent's performance a lot.

Q-learning is off-policy, which means that it tries to learn a target policy, which happens to be greedy, that is different than the behaviour/exploration policy (usually, $\epsilon$-greedy but maybe also softmax). When you talk about the deterministic policy, are you referring to the target policy? I suppose so. Which game is this? The optimal policy may indeed be stochastic.

I'm referring to the target policy. I'm trying not to tie this to a specific game but I'm assuming it's a 2-player zero-sum perfect information game. Currently I'm testing with tic tac toe and reversi but it could be something like chess.


text:
How does a sigmoid neuron output 1 with 0 as input?
Sorry if that is a dumb question. I just started to learn about machine learning.
I'm reading this book about neural networks:
http://neuralnetworksanddeeplearning.com/chap1.html#a_simple_network_to_classify_handwritten_digits
It explains how an artificial neural network classifies handwritten digits applying weights and biases to inputs and each input is assigned a value between 0 and 1, with 0 being white pixels and 1 black pixels.
Now let's say an image containing only black pixels is input. If I understand it well this would cause all neurons in the hidden layer to output 1, which means that all neurons in output layer will output 1. This is not the intended behavior.
What is the correct way to model an artificial neuron that outputs 0 when 1 is input and outputs 1 when 0 is input? Is this done by assigning a negative weight or using a different activation function?

If you just want to flip 0 to 1 and vice versa, I don't think you need a neural network to do that...

Now let's say an image containing only black pixels is input. If I understand it well this would cause all neurons in the hidden layer to output 1

Not necessarily. In a neural network, there are weights $W$, synapses if you will, connecting layers. These weights are randomly initialised, and can be negative. Furthermore, assuming a fully connected network, each hidden activation receives the activations from all neurons in the previous layer as input. A mixture of randomly negatively and positively scaled inputs, summed and passed through a non-linearity, could lead to either a positive or negative output. Hence why an input image of black pixels (0) will not cause all the hidden neurons to also be black (0) or white (1). This of course counts for all subsequent layers too.

What is the correct way to model an artificial neuron that outputs 0 when 1 is input and outputs 1 when 0 is input?

As @Minh-Long Luu points out, you probably don't want or need a neural network to do this...
In fact, I'm not even sure if this is possible (assuming training via backprop) because it sounds like you're after a heaviside step-function for the non-linear activation, which would be non-continuous and therefore non-differentiable, making it impossible to train the NN with stochastic gradient descent.
Perhaps a flipped sigmoid centered at 0.5 would be more appropriate?

I would expect a single sigmoid neuron trained with gradient descent to converge to that Heaviside step, assuming no regularization.

This inverter is an interesting initial application of an artificial neuron.
Yes, you need a negative weight and... no, the sigmoid suffices if you do with a close enough approximation.
See how an unrestricted optimization could progress from a very inexact approximation to the Heaviside step in the limit, flipped and shifted as Giulio suggests.

If you want to test other numbers, here is the WolframAlpha link.


text:
Converting RGB images to Thermal Images
I am working on a project where I am planning to convert RGB images to thermal images. I can convert to either near infrared spectrum images or far infrared spectrum image.
I am planing on using Generative networks for the task, specifically Pix2Pix. For training GAN, there are datasets available with synchronized RGB and thermal image like dataset for MFNet.
I will be grateful if anyone can tell me if it is even possible or if it is possible how close the generated images will be to actual thermal images, I will be grateful.

Maybe you should explain more what a "thermal image" is and how different it is from the RGB images.

A thermal image captures light waves in spectrum in different electromagnetic wavelength than RGB cameras. This wavelength is particularly emitted by objects based on their respective temperatures. And these wavelengths are captured by thermal cameras. These cameras can be classified based on the wavelengths they capture. Near infrared, far infrared etc. And these images are particularly gray scale(single channel) and in these images, each pixel denotes temperature at that particular point in the image. So, essentially I want to convert RGB images to these gray scale thermal images.

Hope this helps.

probably possible with however they do per-pixel object classification. Note there's no relationship between RGB colour and thermal colour, so the model will basically be trying to guess what kind of things are in the picture, then what colour those things are.

Assuming you create the perfect architecture
What your network will be learning will be correlations between objects/textures/colors and thermal signatures. Not the thermal signatures themselves.
It will NOT work the same as a thermal camera. It will not give you actual thermal signatures.
It will not reflect "this is how hot it is"
It will only highlight those things that naturally change color due to heat.   So it'd be more like a "highlighter" that tells you where something has changed visually due to heat.
Examples of where it will not work:

It is common to use thermal cameras to find shorts on motherboards/circuits. Usually, the area where the short is occurring will be VERY hot and will show up well on a thermal camera, but not look like anything in the RGB spectrum

Doing heat profiling for insulation in a room. Will not show up visually. Only on thermal bands.

Overheated pipes

Why
RGB, measure red green and blue respones. Thermal is a measurement on its own like Red, Green, or Blue is. RGB are how strong the signal is on various ranges, on the electromagnetic spectrum. Theres a range for red, green, blue, and thermal respectively.

The thermal range can be found here:

what you'll be learning
Your approach will be using objects and learned properties of those objects as sort of a lookup mapping between objects and thermal signatures.
This will only work if objects visually reflect heat changes. (which many don't)

Thank you for the long answer. I know that it will now work like a ideal thermal camera. I will now use this data for real time training. My project is on sensor fusion with RGB and thermal camera. But there is no dataset for my particular application. But I have RGB images with semantic maps. So, if I can convert these RGB images to IR images, I will have a complete dataset on which I can train any network. For validation and testing I will use the IR images from actual IR camera.

My prediction is that it will not work like an ideal thermal camera. Not unless there is a correlation between thermal bands and visual bands. There isn't always. The semantic maps can act as a maker revealing whether or not a visual correlation might exist for a given object class, but I think an overall system wouldn't be a good substitute for an actual IR band.
I'm curious to see how well your system performs. I know that sensor-fusion is a hot topic in some remote-sensing/satellite imagery processing systems. Check for publications in that space.

I am not sure about that either. I have tried some off the shelf GAN networks that I got on github for rgb to thermal image conversion. The results are underwhelming. I am trying with a bit more variation in dataset. I hope that this will work. I will update here with the results that I get.

you may check SpeakingFaces paper by ISSAI. They did something similar with CycleGAN, only with faces though.

ok. I will check it out

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

This feels like more of a comment than an answer.


text:
MCTS players keep replaying identical games
I am currently training a self-playing Monte-Carlo-Tree-Search (MCTS) algorithm with a neural network prior, and it seems to be working pretty well.
However one problem I have is when I compare my new iteration of the player against the previous version to see whether the new one is an improvement over the previous one.
Ideally I want to compare the two players to play 20 games of Tic Tac Toe with each being the first player in 10 of them. But what ends up happening is that each of those 10 games play out identically (because the MCTS in each player is reset at the beginning of each game, and since they are playing to win, they both take the play with highest probability, rather than randomly drawing actions based on the probabilities, so each player is making exactly the same decisions as they did in the previous game).
So I understand why this isn't working, however I'm not sure what people commonly do to fix this problem?
I could choose to not reset the MCTS between each game?, but that also feels like a weird fix, since the players are then still learning as the games are played, and game 10 would be quite different from game 1, but maybe that is just how people normally do this?

If I understand your question correctly, your goal is to figure out whether or not the quality of your neural network is improving as training progresses.
The core issue seems to be that you do this by evaluating the playing strength of MCTS+DNN agents, but... the game of Tic-Tac-Toe is so simple, that an MCTS agent even with a completely random DNN (or no DNN at all) is likely already capable of optimal play. So, you cannot measure any improvement in playing strength.
I would suggest one of the following two solutions:

Instead of evaluating the playing strength of an MCTS+DNN agent, just evaluate the playing strength of a raw DNN without any search. You could simply make this agent play according to the policy head of the network (proportionally or by maximising over its outputs), and ignore the value head. A pure MCTS (without neural network) could be used as the opponent to evaluate against (or you could evaluate against past versions of your DNN).

If you do want to continue evaluating MCTS+DNN agents, you could try to severely constrain the amount of computation the MCTS part is allowed to use, such that the MCTS by itself can no longer guarantee optimal play. For example, you could let your MCTS run only, say, 10 or 20 iterations for every root state encountered.

No that is not the issue.
I already have more complicated games to evaluate my setup on as well. For tic tac toe, I am already using so few MCTS steps that it isn't playing optimally, but all of that is besides the point. The issue I am having is that my MCTS algorithm is deterministic and since tic tac toe starts the same way each time (with an empty board) and since I reset my MCTS algorithm after each game, then the games ends out playing out in exactly the same way, so even if I ask two MCTS algorithm to play 20 games against each other, they just end up playing the same game 20 times.

If you're already using so few iterations that it's playing suboptimally, I don't think it should be deterministic anymore. How many iterations are you using exactly? After running that number of iterations, how do you let your MCTS select its move to play? I assume picking whichever move maximises the visit count? In this case, you may have many ties, in which case it'd be very good to use random tie-breaking. The same holds during selection phase (PUCB1 equation): use random tie-breaking.

It is deterministic since there is no element of a MCTS-NN algorithm that is random. This is regardless of whether the algorithm is playing optimally or suboptimally. For a normal MCTS algorithm without neural networks, the random element comes from the playout, where random moves are typically selected, but this randomness is removed with neural networks.

So the problem I'm having isn't related to how well my algorithm is playing but merely to the deterministic nature of the algorithm. Meaning if I take a MCTS-NN algorithm and perform a deep copy of it and allow both copies to play a game of tic tac toe. Those two games are going to play out in exactly the same fashion no matter how good or bad the algorithms are, because there is no randomness involved.

@Tue Usually in larger setups I'd still expect there to be some subtle random influences (e.g., random tie-breaking, randomness from multi-threading in parallelised MCTSes, and/or random variance in number of MCTS iterations you can manage to run in given time budget), but I can see why none of those may apply in your setting. My next recommendations would be: (i) just evaluate DNN by itself, with proportional sampling instead of argmax, as I mentioned in answer, or (ii) make your MCTS sample actions proportionally to visit counts, or

(iii) instead of starting evaluating games from empty board, start one evaluation game from every possible opening move. Of course, the disadvantage of the last suggestion is that you no longer evaluate your agent's ability to select a good first move in the "real" (empty) starting state. Oh, and one more suggestion: (iv) run evaluation games against (multiple different) agents that do have some degree of randomness themselves (such as low-budget, untrained MCTSes with random playouts and low iteration counts).

I have to answer since I can't comment yet. I have the exact same problem, all evaluation games make the same moves again and again.
I currently pick a random first action for both players, but it feels like there should be a better solution. I guess I could reuse the MCTS but that seems like an even worse hack.

MCTS bandit phase chooses an action via UCB(1) algorithm. For a given $Q(s,a)$ and the visit counts of non-leaf node, it chooses an action branch in the tree to descend via:
$$ a = \arg \max_{a'}\left[Q(s,a') + c \sqrt{\frac{\log N}{N(s,a')}}\right]$$
The $c$ parameter is the exploration temperature. If $c \gg Q$ then the second term always wins and the algorithm should be randomly exploring the whole game tree, without any value-based preference. So, one possible problem could be that your $c$ is too low.
Another possible problem is a pretty common misconception about the way MCTS works.  It could be that for a single actual in-game move you are performing one MCTS search step. That's not how it is supposed to work - MCTS can be considered as an online "improvement" algorithm for offline learned value function - one should be performing multiple MCTS searches and build a reasonably large search tree for every in-game move one performs. Quoting Wikipedia:

Rounds of search are repeated as long as the time allotted to a move remains. Then the move with the most simulations made (i.e. the highest denominator) is chosen as the final answer.

I think you are talking about something slightly different here.  The value of c or the number of mcts search steps are irrelevant here. The problem is that before each game I reset the MCTS algorithm, and since the actions are chosen as argmax as you also have in your equation, then the MCTS will always arrive at exactly the same best action as it did in the previous game since the algorithm is deterministic and my initial configuration is the same for each game.
Based on your reply I'm guessing that I'm not supposed to reset the MCTS algorithm before each game.

@Tue your $Q$ should change though, shouldn't it? Or it just could be that MCTS just perfectly explores the game tree (you said it was tic-tac toe?) then the same perfect game would be played.

Yes my Q will change, as the game progresses, but it will change in exactly the same way in the next game I play since I have once again reset the MCTS algorithm.

@Tue You mentioned the "neural network prior". I assumed you train it or something? The randomness usually comes from the noise in the value estimation in the leaf nodes.

Yes, I train the neural network prior to playing it against a previous version, but that also doesn't matter, a neural network is generally also deterministic especially in eval mode. However, I guess that is one difference between MCTSNN methods and MCTS methods. For MCTS methods you generally perform a random playout until you reach a terminal node, which introduces randomness even when starting from exactly the same position, while for MCTSNN you do not have this randomness since you do not do a random playout, but rather let a neural network determine prior policies, and value.

Yes, you are exactly right. The "classic" MCTS randomness comes from random rollouts. While the exploration in the NN-driven MCTS should come from the NN noise. https://ai.stackexchange.com/a/27234/20538 ... Actually. The AlphaGo paper mentions extra exploration Dirichlet noise added on top of s0 prior - could that help your case?

No the Direchlet noise is still deterministic, it just skews the probabilities, which only matters during training where the action is chosen stochastically based on those probabilities.

"Dirichlet noise" means that you generate a random sample from a (centered) Dirichlet distribution (e.g. numpy.random.dirichlet) and add it to your prior. I don't understand how you can call that "deterministic".

You are right, I had the idea that adding Dirichlet noise was just a skewing of the probability distributions, but it seems to have a random element to it (I haven't added Direchlet noise to my version yet). However, I believe that AlphaGo only uses Dirichlet noise during self-training and not actually when playing. It isn't completely clear, but if you look in their paper: https://gwern.net/doc/reinforcement-learning/model/alphago/2017-silver.pdf they only mention the noise under self-play. I could definitely be wrong on this though

It seems that you are right there. Check the "Evaluation" section in the appendix: authors state that for evaluation they set the temperature parameter $\tau = 0$ which basically means that the play is deterministic. In the "Play" section in the appendix they state that the moves are chosen from a stochastic policy with potentially nonzero $\tau$. So, it seems ultimately, that the temperature is the answer to your question ;) If you agree, I could try summarizing that in my answer

I agree on the evaluation section, but the play section I read slightly differently. I believe that they are merely referring to what they have written previously, namely that tau is ~0 except for the first 30 moves in each self-play game. But even with Tau ~0 they still have a random element in their self-play from the Direchlet noise.
So my takeway from this is that they only have random elements in self-play where they have tau=1 for the first 30 moves to give extra randomness, and on top of that they have Direchlet noise for all moves.


text:
How to estimate the gradient of an argmin loss
Suppose we have a neural network $f_\theta(x)$, where $x$ is the input and $\theta$ is the network's parameters.
For each $\theta$, we can minimize $f_\theta(x)$ w.r.t. $x$ and obtain the minimum point $x^*(\theta):=\arg\min_x f_\theta(x)$.
My question is: how to estimate the gradient: $\frac{d x^*(\theta)}{d \theta}$ ?
I did some surveys and found this gradient (in fact any gradient) can be estimated by the finite difference technique. However, it is rather slow when $\theta$ includes many parameters, e.g., $f_\theta$ is a large neural network.
Any possible approach is welcomed.

Argmin is not differentiable, so what gradient are you trying to estimate? It does not actually exist.

As far as I know, argmin is not differentiable means: for $x^=\arg\min x$, $\frac{dx^}{dx}$ is either $0$ or does not exist. However, here, I am interested in the gradient $\frac{dx^*}{d\theta}$. I am not sure whether this gradient exist. Any suggestion?


text:
How to ensure continuity of AI model logic?
I am aware that this question might be vague but I must try anyways.
I am looking for a method or an algorithm or even just some keywords (to conduct further research) of how to deal with phenomenon of making sure that model logic will be correctly tracking changes in analysed camera feed based on CV models' results that are not always 100% correct and there might be some other minor disruptions to the image.
It will be easiest to explain with the example: I have an app that will analyse camera feed of a parking situation in online mode to have the best possible view of the situation what cars are in and where, which of them left and when, which came in to what spot etc.
I plan to use series of computer vision models such as object detection, CNN for detected objects classification, some OCR and so on. But the problem with this approach is that each model even if trained very well sooner or later will run into some problems like an object that was recognised one frame before is not recognised in current frame or OCR will confuse "5" and "S" and the resulting string will not be the same so the model can assume there is different car in the same place on one frame.
Even though these errors will appear for example at 0.5% frames, this will still confuse the logic of the model a lot, if the logic will assume that the detection/classification results are always correct and disregard potential models' mistakes.
Another example can be something that will cover part of the camera view for a quick moment and that will result in misdetection some of the objects on few more frames. So ideally the remedy to that problem should have some sort of memory and validation mechanism that will compare older and newer frames whether change in detected objects was legit or it came back to previous state after some short period of time.
Is there any theory or methodology or just some good practices of dealing with such issues or it has to be analysed and mitigated on individual basis?

That is where object tracking comes in to solve those problems, an example would be the SORT or Deep SORT algorithms. They use IOU to match bounding boxes from one frame to the next and use Kalman filter to predict the next position in case the object was not detected in that frame, you can add a voting system to select the most voted object class (very useful for the OCR) to increase the model accuracy, that way you can get a .9999 accuracy even if your model by itself is not very accurate.


text:
How to improve classification accuracy in TF deep neural network model?
I need help in increasing the accuracy of a classification model using Neural Networks on Tensorflow.
I am trying to train a model on sequential data [shape:((435802, 20), (435802,))]
where every sequence is of length 20 and  $$X,Y \in [0,6]$$
Concept: each value in the sequence represents a class of real data instance and the objective is to predict the next class instance.

X[n] = [3, 4, 3, 0, 2, 6, 6, 6, 6, 5, 4, 3, 2, 2, 2, 2, 4, 2, 0, 0] => Y[n] = 0

Class Distribution in Y:
0: 46458
1: 40909
2: 76398
3: 102515
4: 80830
5: 43569
6: 45123

I have tried various combinations of layers like Conv, LSTM, SimpleRNN, Bidirectional, GRU, Dense, etc
Latest Model Definition:
model: "sequential_69"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 conv1d_11 (Conv1D)          (None, 20, 64)            256

 bidirectional_32 (Bidirecti  (None, 20, 512)          494592
 onal)

 bidirectional_33 (Bidirecti  (None, 20, 512)          1182720
 onal)

 lstm_56 (LSTM)              (None, 20, 64)            147712

 lstm_57 (LSTM)              (None, 20, 32)            12416

 lstm_58 (LSTM)              (None, 16)                3136

 dropout_95 (Dropout)        (None, 16)                0

 flatten_42 (Flatten)        (None, 16)                0

 dropout_96 (Dropout)        (None, 16)                0

 dense_217 (Dense)           (None, 7)                 119

=================================================================
Total params: 1,840,951
Trainable params: 1,840,951
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
698/698 - 24s - loss: 1.8671 - accuracy: 0.2324 - val_loss: 1.9549 - val_accuracy: 0.1769 - 24s/epoch - 35ms/step
Epoch 2/20
698/698 - 12s - loss: 1.8484 - accuracy: 0.2451 - val_loss: 1.9464 - val_accuracy: 0.1835 - 12s/epoch - 18ms/step
Epoch 3/20
698/698 - 12s - loss: 1.8189 - accuracy: 0.2539 - val_loss: 1.8762 - val_accuracy: 0.2491 - 12s/epoch - 17ms/step
Epoch 4/20
698/698 - 12s - loss: 1.7844 - accuracy: 0.2726 - val_loss: 1.8567 - val_accuracy: 0.2470 - 12s/epoch - 17ms/step
Epoch 5/20
698/698 - 12s - loss: 1.7129 - accuracy: 0.3054 - val_loss: 1.7388 - val_accuracy: 0.3015 - 12s/epoch - 18ms/step
Epoch 6/20
698/698 - 12s - loss: 1.5867 - accuracy: 0.3488 - val_loss: 1.5342 - val_accuracy: 0.3941 - 12s/epoch - 17ms/step
Epoch 7/20
698/698 - 12s - loss: 1.4544 - accuracy: 0.3974 - val_loss: 1.5511 - val_accuracy: 0.3502 - 12s/epoch - 17ms/step
Epoch 8/20
698/698 - 12s - loss: 1.3705 - accuracy: 0.4300 - val_loss: 1.4285 - val_accuracy: 0.4169 - 12s/epoch - 17ms/step
Epoch 9/20
698/698 - 12s - loss: 1.3284 - accuracy: 0.4489 - val_loss: 1.3488 - val_accuracy: 0.4606 - 12s/epoch - 17ms/step
Epoch 10/20
698/698 - 12s - loss: 1.3064 - accuracy: 0.4587 - val_loss: 1.3678 - val_accuracy: 0.4483 - 12s/epoch - 17ms/step
Epoch 11/20
698/698 - 12s - loss: 1.2945 - accuracy: 0.4640 - val_loss: 1.3417 - val_accuracy: 0.4631 - 12s/epoch - 17ms/step
Epoch 12/20
698/698 - 12s - loss: 1.2848 - accuracy: 0.4694 - val_loss: 1.3644 - val_accuracy: 0.4471 - 12s/epoch - 18ms/step
Epoch 13/20
698/698 - 12s - loss: 1.2780 - accuracy: 0.4729 - val_loss: 1.3474 - val_accuracy: 0.4616 - 12s/epoch - 17ms/step
Epoch 14/20
698/698 - 12s - loss: 1.2717 - accuracy: 0.4741 - val_loss: 1.3396 - val_accuracy: 0.4604 - 12s/epoch - 18ms/step
Epoch 15/20
698/698 - 12s - loss: 1.2666 - accuracy: 0.4764 - val_loss: 1.3862 - val_accuracy: 0.4390 - 12s/epoch - 18ms/step
Epoch 16/20
698/698 - 12s - loss: 1.2630 - accuracy: 0.4773 - val_loss: 1.3143 - val_accuracy: 0.4672 - 12s/epoch - 17ms/step
Epoch 17/20
698/698 - 12s - loss: 1.2582 - accuracy: 0.4788 - val_loss: 1.3189 - val_accuracy: 0.4640 - 12s/epoch - 17ms/step
Epoch 18/20
698/698 - 12s - loss: 1.2571 - accuracy: 0.4796 - val_loss: 1.3443 - val_accuracy: 0.4549 - 12s/epoch - 17ms/step
Epoch 19/20
698/698 - 12s - loss: 1.2529 - accuracy: 0.4816 - val_loss: 1.3186 - val_accuracy: 0.4692 - 12s/epoch - 17ms/step
Epoch 20/20
698/698 - 12s - loss: 1.2504 - accuracy: 0.4819 - val_loss: 1.3074 - val_accuracy: 0.4714 - 12s/epoch - 17ms/step
313/313 [==============================] - 4s 7ms/step
              precision    recall  f1-score   support

           0       0.62      0.80      0.70      1059
           1       0.46      0.14      0.22       986
           2       0.43      0.46      0.44      1760
           3       0.46      0.56      0.50      2310
           4       0.44      0.48      0.46      1890
           5       0.41      0.09      0.15       941
           6       0.59      0.76      0.66      1054

    accuracy                           0.49     10000
   macro avg       0.49      0.47      0.45     10000
weighted avg       0.48      0.49      0.46     10000

Some more context: I am a noob in terms of Neural Networks training and have tried basic models like TF examples, Pytorch Examples and some AutoEncoder examples.

from the details which you have share ,i  can see following things can be reasons for less accuracy
1. Class imbalance
in the class distribution the classes 0, 3, and 4 have many instances as compared to  classes 1 and 5, solving this imbalance can improve the accuracy of the model
You can do it by following ways
=> Weighted Loss Function:
You can provide class weights when compiling the model to give more importance to underrepresented classes:
for example, consider this code block
from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced', classes=np.unique(Y), y=Y)
model.fit(X_train, y_train, class_weight=dict(enumerate(class_weights)))

=> You can use methods like resampling to oversample the minority classes or undersample the majority classes.
=> You can apply the data augmentation by  adding noise, applying time warping, or even using techniques like jittering
2. Regularization
=> Adding Regularization can be a good way to reduce overfitting you can achieve it by adding dropout layers
for example
from tensorflow.keras.regularizers import l2
model.add(Dense(64, kernel_regularizer=l2(0.001)))

3. Tuning Model Architecture
Your current model architecture is complex, and simplifying or adjusting certain layers could help can achieve it by following steps:

Reduce Complexity of Bidirectional Layers: Since you're using multiple bidirectional layers, reducing the number of units in each LSTM layer might help with generalization.

Skip the Flatten Layer: Removing the Flatten layer and feeding the final LSTM output directly into a Dense layer can improve performance in sequential models.

Attention Mechanism: Consider adding an attention layer to help the model focus on specific parts of the input sequence, which can improve predictive accuracy in sequential tasks.

4. Learning Rate Tuning or changing optimizer
=> Learning Rate Tuning: Try using a learning rate scheduler or manually tune the learning rate. A learning rate that is too high might prevent the model from converging.
lr_schedule = tf.keras.callbacks.LearningRateScheduler(lambda epoch: 1e-3 * 10**(epoch / 20))

=> Use AdamW Optimizer: The AdamW optimizer includes weight decay, which can help with generalization.
5. Hyperparameter Tuning
=> You can experiment with hyperparameters like batch size, number of LSTM units, number of layers, and activation functions. Use grid search or random search with KerasTuner to find optimal hyperparameters. KerasTuner is a get tool for experimenting with hyperparameters
and for the evaluation part, I suggest you use the Confusion Matrix
Analyze where the model is making errors by inspecting the confusion matrix. This can highlight which classes are causing problems, helping you focus on improving those predictions.
For more inforation you can check these links
=> https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/LearningRateScheduler
=> https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout
=> https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/AdamW
=> https://keras.io/keras_tuner/
=> https://maddevs.io/writeups/basic-data-augmentation-method-applied-to-time-series/
=> https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention

Can you elaborate on the relationship between X[n] and X[n-1] ?
is there a f so that X[n] = f(X[n-1]) ?
is each X[n] the output of different sources or one single source ? is it the evolution of a state machine ?
Have you analyzed your database ? It is not balanced, but do you have things like a same sequence associated to different targets ?

before training, i do implement rebalancing by resampling.
as far as the multiple next sequences to same sequence...there are a few samples like that. but only a few

Also I have not done the F(x) analysis. will check on the state machine