text:
How to approach a blackjack-like card game with the possibility of cards being counted?
Consider a single-player card game which shares many characteristics to "unprofessional" (not being played in casino, refer point 2) Blackjack, i.e.:

You're playing against a dealer with fixed rules.
You have one card deck which is played completely through.
etc. An exact description of the game isn't needed for my question, thus I remain with these simple bullet points.

Especially the second point bears an important implication. The more cards that have been seen, the higher your odds of predicting the subsequent card - up to a 100% probability for the last card. Obviously, this rule allows for precise exploitation of said game.
As far as the action and state space is concerned: The action space is discrete, the player only has a fixed amount of actions (in this case five - due to the missing explanation of the rules, I won't go in-depth about this). Way more important is the state space. In my case, I decided to structure it as follows:

A
2
3
4
5
6
7
8
9
10
J

4
4
4
4
4
4
4
4
4
14
2

First part of my state space describes each card value being left in the stack, thus on the first move all 52 cards are still in the deck. This part alone allows for about 7 million possible variations.
Second part of the state space describes various decks in the game (once again without the rules it's hard to explain in detail). Basically  five integers ranging from 0-21, depending on previous actions. Another 200k distinct situations.
Third part are two details - some known cards and information, though they only account for a small factor, but still bring in a considerable amount of variation into my state space.
Thus a complete state space might look something like this: Example one would be the start setup: 444444444142;00000;00. Another example in midst of the game: 42431443081;1704520;013. Semicolons have been added for readability purposes.
So now arises the question: From my understanding my state space is definitely finite and discrete, but too big to be solved by SARSA, Q-learning, Monte Carlo or alike. How can I approach projects with a big state space without loosing a huge chunk of predictability (which I might fear with DQN, DDPQ or TD3)? Peculiarly due to the fact that only one deck is being used - and it's played through in this game - it seems like a more precise solution would be possible.

If there are any questions left, feel free to post them below.

The explanation of the state/action space is unclear. You mention "rules" with no explanation as to what these are or how they affect the game. It seems to me like the state space should be 12 dimensional (number of each of the 11 cards in deck, + value of your current hand). I don't understand this second part "various decks in the game" or the third part "details". Are the 5 actions corresponding to https://en.wikipedia.org/wiki/Blackjack#Player_decisions?

What do you mean by "losing a huge chunk of predictability" when using something like DQN/DDPG? Also I should point out that you don't have to use neural network function approximators - one popular option is using linear combinations of hand picked features as your function approximator.

Also, as far as I'm aware, blackjack is a solved game, so that offers a precise solution to the optimal blackjack policy.

@Taw Thanks for your replies. To clear things up a bit: The game resembles Blackjack in some rudimentary characteristics, i.e. cards can be counted - but it’s not Blackjack. Should have mentioned that more clearly. My simple problem is the sheer size of the underlying state space (it can’t be shortened further or else considerable losses in performance would arise). As far as your second question is concerned, I was referring to the approximating of my state space - which, if Implemented poorly, could end up with a horrible performance. Nonetheless, I’d be curious to how you’d approach that.

How many active turns do you need to look ahead in this game? If it's played in short, separate rounds with the only thing preserved between them being state, then you might be able to do a tree search in limited memory.

i would believe that blackjack in the settings you describe can be solved analytically as well........ the expected value of each next draw can be computed very cleanly, and so i don't see any problem with an analytic solution.

@k.c.sayz'k.csayz' Problem is computational capacity.

How can I approach projects with a big state space without loosing a huge chunk of predictability (which I might fear with DQN, DDPQ or TD3)?

You can impact this by choosing a combination of function approximator and engineered features which are a good match to predicting either the value functions or policy function that an agent will need to produce.
It is hard to tell a priori how much work you would need in this regard. Given that you play as many training games as you have time for, use a deep neural network with hardware acceleration, then one approach is to simply normalise/scale your feature vector as it is, and train heavily. This approach in RL has repeatedly shown new state-of-the-art results for e.g. AlphaZero, Open AI's DoTA agent and others. This appears to work provided the compute resource that you can throw at the problem is large enough. As you mention, card-counted blackjack is a solved problem, so doing this may be within your reach on consumer hardware.
Some smart feature engineering may help though, by making the core problem easier for the agent. As this is a hobby project, what you do will depend on what you want the agent to learn. Is the purpose of your project to teach the agent how to card-count from scratch? Given that you have told the agent the count of remaining cards in the state, it does not appear so.
Next question: Do you need the agent to learn to sum up all remaining cards in order to calculate the raw probability of revealing a card of each type amongst cards so far unseen? That's currently what your main state feature is doing. If you don't need the agent to learn that, you could help by using the probability of each card being seen as a feature instead of the remaining count. This feature is likely to reduce the complexity of value and policy functions, thus it will reduce compute time, and may also improve accuracy.
With a game that can be solved analytically, you could take this up to the point of deriving the optimal policy directly and not using RL at all (i.e the input to your NN would be the correct action or the action values!). The question for your project is then this: What precisely are you hoping to demonstrate that your agent can learn? Or maybe: What do you want to gain by applying RL to this problem?

Yes, you’re right, I could parse the probability of each card appearing as an input - though that wouldn’t condense my state space, would it? I’d still input an integer value for each card type into my action space, except this time it represents a probability (which could be calculated by simple dividing the amount of said card left and the total amount of cards left). I don’t see any advantage of deriving this value beforehand? For my specific problem, it’s important to consider the probability of each card appearing in the next turn.

My goal is to programmatically determine the best choice for each stage in the game. My first approach was to backcalculate any decision tree within the MDP, though this ended up to be ineffective due to sheer amount of calculation required (even with alpha beta pruning and other shorting techniques). After that I went ahead and tried to implement an agent playing the game, but as mentioned ended up with monstrous state spaces, therefore making it ineffective.

"I don’t see any advantage of deriving this value beforehand?" Although the calculation is trivial, it is specific, very likely to be useful when calulating expected return, and a learning algorithm could take some time to stumble across an approximation to it statistically from the data.

tho the size of a single digit in comparison to multiple might be valuable in a huge q-table - as you’re apparently proposing.

I am not proposing any Q table at all. You will need to use an approximator such as a neural network. Your question is about making that neural network more accurate. My first paragraph explains in general wht you might need to do tyo make it more accurate. The second para explains why you might not need that. Third para onwards suggest some specific things you may want to consider.


text:
Validation accuracy less than training accuracy (with no sigh of overtraining)
I am working with a deep CNN with over 100k sample data. I divided it up into 75% training, 12.5% validation and 12.5% for testing. As I train my network, the training accuracy approaches near 100% accuracy. The validation accuracy approaches 70-90% accuracy. The validation accuracy is always increasing and never decreases so I do not believe that the network is over-training.
The training accuracy is similar to the validation accuracy but both are less than the training accuracy.
My question is, what is causing my validation/training data to trail the training data? Is it because my validation/training sets contains sample types which are not found in the training set? What else might be causing this?
Additionally, between epochs, I see this 'stair case' in learning in that I see a huge jump in accuracy as soon as a new epoch starts. I am shuffling my data between epochs. What might be causing this jump in accuracy?
Also, if there are more technical terms for the events that I am describing please let me know so that I can further research these.
Thank you!
blue = training,
black = validation

The validation trend doesn't inform you much about real overfitting, cause the model hyperparameters are optimized based on the validation set. Reason why usually the validation scores are better than the test ones. So to truly check overfitting you should constantly look at the test scores.
The jumps make me think that you're using all training instances each epoch, so no matter if you shuffle or not, the model performs better cause it starts going trough already seen examples.
In general, don't expect to reach a test accuracy similar to the training one, it happens only with perfect toy datasets. And if you have reasons to think the test accuracy should be much closer than what you are observing then inspect more closely your dataset and the splitting you're performing. Specifically check if you have imbalanced classes shown more in validation/test than training, larger variance in data features for validation/test than training or any other form of imbalance you might check depending on the type of data you're using.

Thank you for your response.
Your answer makes sense. I will look more into my training and test/val data to see how they might differ from the training set.
Yes, I am using the whole datasets for each epoch so that does explain the 'stair case' jumps.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How does graph Fourier transform work when multiple signals present on each node?
Context: I was reading the following set of notes (page 83): here and it says:

Thus, the Fourier transform of signal (or function) $ \mathbf{f} \in R^{|V|} $ on a
graph can be computed as $$ \mathbf{s} = \mathbf{U}^T \mathbf{f} $$

Question: What happens if each node has multiple 'signals'? Are the Fourier transforms on each signal independent of one another?
Attempt: I assume that each signal is denoted as a column vector, and thus multiple signals may be written as a matrix $\mathbf{F} = [\mathbf{f_1}, \mathbf{f_2}, ..., \mathbf{f_n}]$ (where $ \mathbf{f_i} \in R^{|V|} $) for a graph with $n$ signals. Thus, the graph Fourier transform would be $$ \mathbf{S} = \mathbf{U}^T \mathbf{F} $$ and thus each of the Fourier transforms would be independent of one another. Is this the correct way to think about this.
Many thanks in advance!

Single node can't have multiple signals they have $\mathbb{R}^n$ dimensional feature vector, so in this context each node have $n$ number of channel. The signal is define over the entire graph. Take the first channel of each node, you'll get a graph signal, then take second channel you'll get another signal(Think of RGB image where we have 3 channel). Now when we perform GFT, we perform it over individual graph signal.

Thanks @SwaksharDeb! Okay cool, yes I was thinking exactly along the lines of RGB (but didn't know how to word it). Okay so each nodes can have $n$-dimensional feature vectors and when taking the GFT, we just consider one layer at a time (over the whole graph).? That makes more sense now

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What does it mean when accuracy of regularized model is higher for training set than for validation set?
Accuracy of my regularized model is higher for training set than for validation set.

The situation improves when regularization coeefficient is reduced:

What does this really imply?
From my understanding, this seems to suggest that regularization is actually resulting in the model overfitting training set, which is the opposite of the intended outcome

It implies that your regularization effects are too much, and prevent the model from learning from data. Also, at such a low accuracy (~10%), we can't really talk about overfitting.


text:
Identify whether two companies are the same
I am trying to solve a problem where I need to map multiple variations of a company name to a single name. For example: say I have a company named Super Idea Corporation Limited.
I need to resolve the following to Super Idea Corporation Limited

SICL
Super Idea Corp Ltd
SIC Ltd
SIC Limited

Is there a non regex way of doing this? The reason I am averse to using regex is that there are a lot of business names that can be represented in many different ways. I want something that is more flexible and adaptive.

Can you explain how this question is related to Artificial Intelligence, as defined in our on-topic page (https://ai.stackexchange.com/help/on-topic)?

@nbro, I am not very well researched on NLP. Asked here to see if there is any work done NLP-wise on this particular kind of problem. Hence the tag.

Have a look at Named Entity Recognition (NER); these algorithms are mainly concerned with recognising that there is an entity, but often also include normalising the name to a canonical form for information retrieval -- this is what you would need.
In a previous job I actually implemented this, using a fuzzy match with variable word order. You would still need to map Corp and Corporation onto each other as exchangable, and deal with acronyms, but that should be tractable.


text:
Discrepancy of backpropagation formula between Andrew Ngs ML Course and those derived by neuralnetworksanddeeplearning.com
I'm currently working through Week 5 of Andrew Ngs Machine Learning course on Coursera, which goes through the backprop algorithm for basic neural networks. Whilst trying to derive the formulae he gave in the lectures, I noticed that the formula for $\delta^L$, "error" of last activation layer, is slightly different to that derived in http://neuralnetworksanddeeplearning.com/chap2.html.
In Andrew's, it seems like there is no inclusion of the partial derivative da/dz, or $\sigma'(z)$, only the dC/da part.

However Michael Nielson does include that term:

Is this difference significant and why does it arise? Is it because the derivation Nielson goes through defines the Cost using the mean square errors, whereas Andrew Ng defines the cost using the -ylog(h(x))... one? Also will Nielson's equations score full marks on the Ng's assignment?
Thank you for reading.

It's a matter of different loss functions used. In Andrew Ng's C1W2L09 video he derives the loss gradient as
$$\frac{d}{dz} = \frac{d}{da}*\frac{da}{dz}$$
using the loss (cost) function binary cross entropy whose derivative is $\frac{d}{da} = -\frac{y}{a} + \frac{1 - y}{1 - a}$
and the activation function is sigmoid whose derivative is
$\frac{da}{dz} = a(1-a)$.
The product of the two is
$$a - y$$ which is consistent with the vectorized form you showed.
Nielsen uses the quadratic loss function $L = \frac{1}{2}||y - a^L||^2$ whose derivative is as you've shown: $\frac{d}{da} = a^L - y$. His formula $\delta^L = (a^L - y) \odot \sigma'(z^L)$ is specific to quadratic loss.
So Andrew's $\frac{d}{dz} = \frac{d}{da}*\frac{da}{dz}$ is the same as Nielsen's matrix based $\delta^L=\nabla_aC \odot \sigma'(z^L)$, here are the equivalent terms (Andrew's on the left, Nielsen's on the right)
$$\frac{dL}{da} = \nabla_aC $$ and $$\frac{da}{dz} = \sigma'(z^L)$$


text:
Continue teaching pre-trained network without forgetting previous data set
I have a rather interesting problem here; I work in the field of image classification for quality assurance.
For this I have a dataset of about 1 million images, which I have used to train different defect classes.
Now one of these defect types has additional properties (new features of an image class), I would like to teach these new features to the previously trained network, and without re-training the whole previous dataset.
In short: new features of an image class should be taught without affecting the performance of the network on the previous training set too much. Is this possible, and if so, what are some strategies for doing this?
Thanks in advance!


text:
Where can I read about upsampling methods in detail?
In deep learning, we encounter the upsample blocks several times, especially when we deal with images.
Consider the following statements from description regarding UPSAMPLE in PyTorch

The algorithms available for upsampling are nearest neighbor and
linear, bilinear, bicubic and trilinear for 3D, 4D and 5D input
Tensor, respectively.

Where can I read about these upsampling techniques in detail, especially in the context of deep learning?

They are just standard upsampling or interpolation techniques in computer graphics. Have you tried Google? For example, nearest-neighbour interpolation just copies the same pixel a bunch of times.

I tried googling for upsampling techniques and found no single material that contains the information. @user253751

really? https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation

@user253751 It contains only one right? I want all of them together like a chapter or unit. Is there no such material?

https://en.wikipedia.org/wiki/Interpolation ?

https://graphics.stanford.edu/courses/cs205a-13-fall/assets/notes/chapter11.pdf

@user253751 yeah, apt for intermediates.

https://www.ijert.org/research/a-comparison-study-on-different-interpolation-methods-based-on-satellite-images-IJERTV2IS60011.pdf

https://sisu.ut.ee/sites/default/files/imageprocessing/files/interpolation3.pdf

Tricky question. In my experience is better to just look for math resources on classic upsampling method, since deep learning papers and books tend to give them for granted, or not something related to AI (they are after all analytic methods). Another reason is probably that the math is not that hard, and already the wikipedia pages offer a good description of the basic methods
(nearest-neighbours, bilinear interpolation, bicubic interpolation).
For some others interesting and more advance techniques I found useful this paper: Mathematical Techniques for Image Interpolation
even though it's not exhaustive since it doesn't mention well known alternatives to bilinear or bicubic upsampling like Lanczos, so for completeness I would read also A Study of Image Upsampling and Downsampling Filters

Regarding the deep learning part, it's more valuable to search for papers that studies not the upsampling methods themselves, but the artifact they introduce, especially related to tasks like super-resolution. As a starting point I would dig into this blogpost and its references.


text:
How to measure Deep RL algorithms in terms of safety?
I applied for a Ph.D. in AI, my advisor told me that my thesis is about safe applications of deep RL algorithms in healthcare. So I decided to do as the first paper, a comparison of Deep RL algorithms in terms of their inherent safety. However, after lots of research, I could not find an answer to my question, that is: How to measure Deep RL algorithms in terms of safety?

I think you should first start with definition of software safety in health domain. For example, you should start with Therac-25 accident.
Then look at the current scientific articles and standards about software safety in medical domain. Then think about how your algorithm will be tested.
You are thinking Deep RL algorithms as a blackbox but they are software in the end.
If Deep RL algorithms will be used in hospitals, they will have to be tested. The benchmarks, conditions and restrictions of normal software must apply to RL algorithms too.

You are correct, in fact, I have been looking for a (healthcare) task to contextualize the safety definition. It's not that I didn't find, there are lots, but the probleme is, these tasks are not available/accessible for me to measure. For example, if I chose the task of sepsis treatment, How Can I measure the safety-related properties of the algorithms. There is no simulation environment or dataset available. This what makes RL widely applicable and measures only on games (given the available benchmarks such as OpenAI gym environments)

Then you need to start with simulator

There is this paper talking specifically about the safety of ML/DL
Algorithm but in industrial application. Since the algorithm is purely learning function you have to define the safety in terms of the application you are using it for example diagnosis or surgery.


text:
Can a Reinforcement Learning problem with multiple simultaneous actions be formalized as a Multiagent Partially Observable Markov Decision Process?
Consider the following decision making problem. We have a controller that selects locations from a grid of coordinates and captures an image (observation $o_t$) with a camera at each location (action $a_t$). We try to find an optimal sequence of locations for a specific goal. This decision making problem can be formalized as a Partially Observable Markov Decision Process (POMDP). Here, we seek an optimal stochastic policy $\pi^{*}_{\theta}(a_t|h_t)$ that maps the history $h_t= \langle o_1, a_1, ..., o_{t-1},a_{t-1},o_t \rangle$ of actions and observations up to the current time $t$ to action probabilities. The history $h_t$ can be summarized by the hidden state of a RNN and we can use a policy gradient method, e.g. REINFORCE, to update the policy parameters $\theta$.
Suppose now that we want to select multiple locations, i.e. actions, simultaneously. According to my understanding, we could formalize the problem as a Mutliagent POMDP (MPOMDP) [1]. In this formalism, we would replace the single action of the previous problem by joint actions $\vec{a}_t = \langle a^1_t, ..., a^N_t \rangle$, the single observation by joint observations $\vec{o}_t = \langle o^1_t, ..., o^N_t \rangle$ and the history by $h_t= \langle \vec{o}_1, \vec{a}_1, ..., \vec{o}_{t-1},\vec{a}_{t-1},\vec{o}_t \rangle$, where $N$ is the number of agents. We would now try to find an optimal joint policy $\vec{\pi}^{*} = \langle \pi^{1*}, ...,\pi^{N*} \rangle$ consisting of sub-policies $\pi_{\theta_n}(a^n_t|h_t)$ that map the history $h_t$ to the action probability of each agent $n$. This would mean that the RNN would have $N$ output nodes and each sub-policy $\pi^n$ would be parametrized by $\theta_n$, a sub-set of weights of the output layer [2]. Would it be correct to assume that an optimal or near-optimal joint policy $\vec{\pi}^{*}$ can be obtained by simply applying the policy gradient method used above to each sub-policy $\pi^n$?
I would be curious to hear what you think about the MPOMDP formalism applied to the latter decision making problem or whether you would suggest something else.
[1] Oliehoek, Frans A., et al. "A concise introduction to decentralized POMDPs." Springer, 2016.
[2] Gupta, Jayesh K., et al. "Cooperative multi-agent control using deep reinforcement learning." International Conference on Autonomous Agents and Multiagent Systems. Springer, Cham, 2017.

I guess it depends on what the goal is. If the goal is a general reward function, this formulation as an MPOMDP could make sense. One way to think about this, is as a way of modeling a general (centralized) POMDP with factored actions and observation spaces.
However, it seems that what you are describing might be an active perception problem, where the goal is to minimize uncertainty? In that case, there could be better formulations. I personally did some work in this direction with Yash Satsangi and others. E.g., these papers might be of interest:
Yash Satsangi, Shimon Whiteson, Frans A. Oliehoek, and Henri Bouma. Real-Time Resource Allocation for Tracking Systems. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence (UAI), August 2017.
Yash Satsangi, Shimon Whiteson, Frans A. Oliehoek, and Matthijs T. J. Spaan. Exploiting submodular value functions for scaling up active perception. Autonomous Robots, 42(2):209–233, February 2018.
(and there is a large literature on this topic as well - look at the references in the papers).
Hope that is useful.


text:
Is it true that channels always represent colours of an image?
Convolutional neural networks are widely used in image-related tasks in artificial intelligence.
The input of a conventional neural network is generally an image. The output of a convolutional neural network can also be an image. But the output of the hidden/ intermediate layers of a convolutional neural network is generally the feature maps of the input image.
In general, channels of an image, represent colors used. If the input and output of a convolutional neural network are RGB images. Then it is true that three channels of input and output images are representations for red green and blue colors. Is it true for feature maps also? Do the channels in feature maps are also the representations of colours?
I got this doubt because of the reason that I remember channels as colours and feature maps may contain an arbitrary number of channels. If they are also represented under then it is difficult for me to understand.

No, channels do not have to only represent colours. It is common for them to represent other things, even without considering feature maps. For instance RGBD images, where D is a depth measurement or distance from a sensor. Or when CNNs are applied to grid-based games, such as chess or go with AlphaZero, where the input channels are information about game pieces on a board.
Mathematically, there is little to differentiate between a channel or a feature map. Both are numerical values stored in some muti-dimensional array, most often with the following assumptions:

All values in a single channel or single feature map represent a measurment of the same concept. That might be how much blue light was detected at a sensor at a point in space, or it may be the degree to which pixels close to that point match patterns associated with the centre of a certain type of cat's nose.

The values are considered co-located with other feature maps or channels within the same system, such that values at index $i,j$ (or just $i$ of 1D or $i,j,k$ for 3D etc) in one channel are considered to be at same location as values in related channels or feature maps, at least within the same layer.

You will tend to find channels used to describe inputs and outputs that can be directly visualised, whilst feature maps tend to be used to describe the more abstract pattern matching that occurs in the outputs of a CNNs hidden convolutional layers. However, the two terms can be used loosely, and sometimes interchangeably.
The feature maps within a CNN typically do not carry separate colour channels. Although it is possible to design architectures that keep colour information separate, this is very rarely used - normal CNN architectures allow mixing of all layer channels/features with each new layer, through the mechanism of having weights that connect every input channel/feature to every output channel/feature between layers.
You will sometimes see colour channel information extracted from the neural network weights of the first convolutional layer, in order to visualise what that layer is matching to. That is because the first layer's weights (and only the first layer's weights) can be interpretted as template matching to the input channel for each output feature map. This is not the same as visualising the output feature maps - whilst those maps are influenced by the input colour channels, thus do in a general sense carry colour information, they do not measure colour intensity in the same sense as an image channel used for input.
More generally, because human perception is strongly tied to RGB colour channels, and because computer displays and image formats are designed around this, whenever you see any representation of what a CNN layer is doing, you will see one of:

A greyscale representation of feature map values. This is the closest to "true" representation, but sometimes it is not very informative.

A heat map of feature map values. Using colour may help with visualisation, but it is false colour in the sense that the same colours are not actually in the feature map.

A representative input that would cause the feature map to activate. This can be informative about the feature map, but it is not showing what the feature map is doing directly, and the channels defined in the input are used for colour.


text:
How general is generalization?
I am sorry but I have to explain my question using an example, I do not know how to ask it in proper scientific terms.
Let's assume, I have trained a deep learning model on classifying hand gestures, but training and testing datasets' images are shot only in one lighting conditions and I achieved certain accuracy, let's assume 85%. As far as I understand, adding more data of the same hand gestures images but shot with different lightning should increase my model's "generalization" capabilities, right?
So the question is, if I take this model, trained in two lightning conditions, and test it only on the dataset of the first lightning conditions, would that increase it's accuracy (the 85%) or maybe this "generalization" would only mean that it can now also classify correctly images with different lightning, but not increase the accuracy on the first set?

I think there's a crucial point missed in the question, touched by jros answer but without further elaboration.
If you train a model on domain A: single lightning condition and test it on domain B: two lightning condition then you're not evaluating generalization but transfer learning capabilities. Or to phrase it differently you're evaluating how close domain A and B are for the model you trained.
The test set as you said is truly made of instances never seen by the model during training, but it should nevertheless be representative, i.e. correctly sampled, from the training domain, or from the same distribution as jros wrote. So the generalization of your model, trained on single lightning condition, should be evaluated on single lighting condition as well.
A final remark about the rest that have been said:

everything holds only under the assumptions that the initial training dataset is not only unbiased but also balanced. In a real case scenario changing the training distribution from something specific (single light condition) to another distribution (multiple light conditions) might well be lead to a worse model, simply cause the problem is now inherently harder to solve.

So the answer to your question (regarding both, true generalization on same distribution and what you describe, transfer learning) is actually just empirical.

Although you say this in other parts of your answer, when you write "So the generalization of your model, trained on single lightning condition, should be evaluated on single lighting condition as well.", I think you should also emphasize that this is true only if you're trying to learn a target function where the inputs are assumed to have the same lighting condition and you assume that your training data is representative of your target function or probability distribution (you say that in other parts of the answer, but that sentence read alone may be misleading).

I dont exactly need my model trained on domain A to test on domain B. My situation is that my model performs not as expected on domain A only images, I am waiting to get new data from other domains, and i wonder whether training model again on set with samples from additional domains would also help the performance on set A, or do I have to fix something within the model first.

Amd this has led me to a bit more general theoretical question that I asked, cus I was hoping that since most of the features from domain B and C and D would  be same as in A (cus general shapes of gestures are always the same), maybe it will also help domain A.

Generalization
In machine learning, generalization describes a model's ability to properly correct its algorithms to predict new data from the same distribution as the data used to train the model.
By providing additional training for your model (on data with varying lighting conditions), you are correct that you would be increasing the capabilities of your model.
Does better generalization equal better performance?
Consider two models and assume they have the same number of coefficients/ layers & nodes:

Model 1, which was trained solely on a single lighting condition
Model 2, which was trained on multiple lighting conditions

Let's say we have two test sets as well:

Test Set 1 : data with the same lighting condition as the data used to train Model 1
Test Set 2 : data with the multiple lighting conditions

Model 1 would be expected to have an equal or better performance than Model 2 on Test Set 1, due to "over" fitting on that lighting condition, but as you noted, Model 1 would not perform as well on Test Set 2 as on Test Set 1. We also would not expect Model 2 to perform better on Test Set 1 than Model 1 due to the generalizability achieved during training.
Simply put, you're probably sacrificing some lighting condition-specific accuracy for better accuracy across multiple lighting conditions.
However
By allowing your Model 2 to increase the layers & nodes, or coefficients (including interactions), Model 2 may well be capable of performing just as well. All this depends on the size of training sets as well. For instance, if Model 1 is trained on 1,000 data points from the single lighting condition, and Model 2 is trained on 500 data points, Model 1 is generally expected to perform better on Test Set 1.


text:
Should I train my network for classification on samples whose ground truth label is ambiguous?
Imagine that I am training a model to classify handwritten digits. Suppose there are some bad quality images that could be classified by a human as either 0 or 8, 1 or 7 or other commonly misclassified pair of digits.
My question is, should I simply remove such ambiguous samples? Should I annotate it as the most similar digit, even though there are other similar answers? Should I present it repeat the sample, presenting it once per each 'acceptable' answer?

This depends on the behaviour you want. If the ambiguous sample's ground truth is classified by a range of people, your network will get an average* based on that group. If it's only by one person, your network will be biased to how that one person classifies these samples.
Alternatively, depending on your loss function, you could train the network to classify ambiguous samples with an ambiguous label. For example, if 40 people label the digit as 1, and 60 as 7, you could have the desired output for the network be 0.4 for 1 and 0.6 for 7 (assuming it's a probability).
This doesn't have an exact answer, it's whatever behaviour you deem best for your scenario. If you want to keep it simple, you can remove the samples and then see how the network performs on these ambiguous samples at testing time. Assuming your dataset is good, you'll probably find the network performs about the same on these samples anyway.


text:
Is it possible to have different channel dimensions in a CNN?
Let's say I have two channels that I wish to feed into a CNN. One of the channel contains 4 traces and has a width of 512. Stacking them on top of each other therefore yields an image with dimensions (4, 512). The other channel is just 1 trace, so its dimensions would be (1, 512).
I then have convolutional filters that are of dimension (1, 5) as an example. That means that the filters run over each trace separately. The first channel (containing the 4 traces) will then have a set of filter weights, shared among the 4 traces. The second channel (containing the 1 trace) will have a completely different set of weights (as per this SE question).
TLDR: Can convolutional layers in a CNN have different dimensions? Putting this in the context of images: Could we have a CNN that takes an image that has dimensions (100, 100) for the red channel, (100, 100) for the green channel, and (50, 100) for the blue channel?

Hi. Can you clarify what are "traces" exactly ? Are they some kind of features over "time" ?

You can do whatever the heck you want.
Of course you will have to design the data flow through the network so that it can make whatever inferences you intend it to make.

The first channel (containing the 4 traces) will then have a set of filter weights, shared among the 4 traces. The second channel (containing the 1 trace) will have a completely different set of weights (as per this SE question).

Sure, you can do that. No reason why you couldn't. Will it work well? Who knows. Have to try it and see.
The best way to combine them will depend on what the NN is supposed to actually do. With the architecture you've described, at least this layer is unable to relate traces to each other. This would be bad when processing, for example, colour images - you don't want to treat red, green and blue the same way as each other, and you want to detect certain combinations of red, green and blue. If you also want the network to have this ability, then maybe you should treat each trace as a channel so the network can see all of them at once.
At some point you will obviously have to combine the results together.

Could we have a CNN that takes an image that has dimensions (100, 100) for the red channel, (100, 100) for the green channel, and (50, 100) for the blue channel?

As I said, it depends on what the network is supposed to do. Are these channels totally separate? Then you can process them with different sized CNNs - or even the same CNN until the dense layers - and combine the results in the dense layers at the end. But if these are the RGB components of one image, you'd be better off just stretching the blue channel so the CNN can recognize colours like yellow.


text:
When to use Multi-class CNN vs. one-class CNN
I'm building an object detection model with convolutional neural networks (CNN) and I started to wonder when should one use either multi-class CNN or a single-class CNN. That is, if I'm making e.g. a human detector from image data and a cat detector also from image data, then when should I have a specific model for each task, and when should I just combine all the data into one and use just one general multi-class CNN?
I've understood from the No-Free-Lunch-Theorem and generally from estimation theory, that there there does not, in theory, exist a model which is simultaneously optimal for every problem. In other words, case specific models should, in general, beat the "all-purpose"-models in the same task.
I have a difference in opinion with a colleague of mine whether to use one-class of a multi-class CNN and I would like to hear the communities opinion on this.

I am not really a fan of the One vs All approach.
From my experience it is never convenient to transform a multi-class classification problem with, say, $N$ possible classes to a bunch of binary classification problems.
Reason #1
The number of binary classifiers you need to train scales linearly with the number of classes. Hence, you can easily find yourselves training lots of binary classifiers. What if each one of them has a huge number of neurons? As you can understand, the computational burden here is quite a problem.
Reason #2
With a small $N$, the computation is less of a problem, but still.. why would you do that? By doing things like this, you can easily end up in awkward situations such as two or more of your binary classifiers give a positive outcome, or none activates. How do you handle these issues?

However, there exists a very specific setup where you might want to use a set of binary classifiers, and this is when you're facing a Continual Learning(CL) problem. In a Continual Learning setting you don't have access to all the classes at training time, therefore, sometimes you might want to act at a architectural level to control catastrophic forgetting, by adding new classifiers to train. However, even in CL there exist other methods that work better.
To conclude, I wouldn't recommend anyone go for this option. You can train a multi-class classifier much more easily and avoid all the aforementioned issues.

Hi @danin thank you for your reply. If I've understood correctly from your answer, this seems to be a bit of an engineering and computational complexity issue? Not really about the theoretical perfomance?

We would be interested in selecting the approach which has the best performance. I would suspect, a general multi-class classifier cannot reach the same performance as a specialized one-class model, even though computationally more easier approach, unless we have a huge amount of data. Still, I would suspect case specific models should, in theory always beat the general purpose models. Or am I mistaken? Anyway, I'm in no big favor of either approaches, it depends on the problem we are solving.

It is always difficult to speak about theoretical performance for machine learning models. The reason is that there's no well established theory on the choice of the model architecture. Thus, we can't really know whether one model would perform better than the other. Still, I would expect multi-class models to generalise better to unknown samples. From my experience with one-class models (not very experienced to be honest), I've always seen that they tend to overfit more than general-purpose models. I'd recommend you try training both models and see which one gives the best test result :-)

Okay, thank you for your comment! :) Good point, yeah it's not that trivial always. Have to give this some more thought.


text:
FCNs: Questions about the filter rarefaction in the CVPR paper [Long et al., 2015]
I am reading the paper about the fully convolutional network (FCN).
I had some questions about the part where the authors discuss the filter rarefaction technique (I guess this is roughly equivalent to dilated convolution) as a trick to compensate for the cost of implementing a shift-and-stich method.

Consider a layer (convolution or pooling) with input stride $s$, and a subsequent convolution layer with filter weights $f_{i,j}$ (eliding the irrelevant feature dimensions). Setting the lower layer’s input stride to 1 upsamples its output by a
factor of s.

How does setting the input stride of the lower layer to 1 leads to upsampling (and not in the reduction of output dimension)? I am confused about what the terminologies lower/higher layer and input/output stride refer to here.

To reproduce the trick, rarefy the filter by enlarging it as
$f'_{i,j} = \begin{align} \begin{cases} f_{i/s,j/s} & \text{if $s$ divides both $i$ and $j$} \\ 0 & \text{otherwise} \end{cases} \end{align}$
(with $i$ and $j$ zero-based). Reproducing the full net output of the trick involves repeating this filter enlargement layer-by-layer until all subsampling is removed.

I was wondering how the rarefaction defined here leads to the filter enlargement. Based on the equation, it seems that $f$ and $f'$ has the same size, with $f'$ having different filter weights based on $s$.

I am also unclear on the meaning of this. For (1) my understanding is simply that the stride is reduced from s to 1 which increases the output. I.e., if your stride was originally 4 and you reduce it to 1 then you will increase the output by a factor of 4. I hope someone else answers this.


text:
Which method can accurately detect circular/angular shapes? (attached example)
Is there a method to detect shapes like these accurately and efficiently? I have tried the OpenCv Haar Casacade Classifier which does not work well. These shapes should all be the same class object and can be of different sizes and a little differently shaped (more circular or more angular). In the attached picture there are 4 separate shapes, of which 2 overlap each other.

Is the size or area consistent (no really large ones covering 25% of screen, or really small ones only a few pixels across)? These look like outlines of cells from a microscope slide?

@NeilSlater there are langer ones as well but not covering 25% of the screen, but lets say 4 or 5 times as big. These are the smallest ones though. Its an image of a game where I used an Hsv filter to get only what I need to make it easier to detect, these are outlines of the objects

If it is outlines of cells, several researches are done to count "bad" red blood cells from images, and improve the count accuracy by considering the overlapped cells (I think it's for malaria). You might want to look that up for insights.


text:
2-stage model overfitting
I'm trying to build an entity matching model. There are 2 kinds of features - binary (0/1) and text features. Initially I made a deep learning model that uses character level embeddings of some of the text features, word level embeddings of the other text features and also uses the binary features as inputs.
The output is through a softmax for 3 classes, so basically a $n\times 3$ array (where $n$ is the input data length).
I've done three splits - train, val and test, and for training the DL model through Keras, I've specified train as the training split and val as the validation split. I measured the performance on the test split to get DL model metrics. The softmax outputs for all three splits were obtained using model_DL.predict.
Next, I used a Random Forest model as a second stage. Inputs: all the binary features PLUS the softmax outputs as inputs. e.g. I took the train split, removed the text features and added in the columns of the predicted array as separate features. To be even more specific, if predtrain was obtained by using model_DL.predict on the train split, then the additional features were added using train['class1prob'] = predtrain[:,0], train['class2prob'] = predtrain[:,1], train['class3prob'] = predtrain[:,2].
Similarly I did for test and val splits. Now I trained the RF on the augmented train split and measured its performance on the val and test splits. The F-score for the train and val splits was around 0.85, 0.74, 0.73 for the 3 classes respectively (i.e. performance was similar on both splits).
BUT for the train split the predictions were near perfect - 0.98, 0.99, 0.98 F-scores for the three classes. My intuition is that overfitting of the 2nd stage RF is understandable for train, since the softmax outputs were predicted using the 1st stage DL model, which in turn was already trained on train. Also, there's some data leakage for the val split since val was used as a validation set to finetune the DL model by Keras, so maybe even the val metrics aren't so reliable. But there is no leakage for the test set.
My question is, in this scenario have I made an error, or is this blatant overfitting normal for 2 stage models? If there's a glaring error, any way or best practice to fix that?


text:
Monocular depth estimation
I am currently reading the paper towards robust monocular depth estimation and I have 2 doubts about it.
First of all the paper stated that there are 2 types of depth annotated, dense and sparse. What are they and what are their differences?
Secondly, the paper predicts the relative depth given an input, how do we calculate the loss when a relative depth map is predicted? I know we could simply use MSE if the model predicts an absolute depth map. If I were to train a model myself to predict relative depth maps as well, how should I calculate the loss or other evaluation metrics?
Any help would be greatly appreciated, thanks in advance!


text:
What is the best open source python repo for facial recognition?
I am looking for best open source python repo for facial recognition. Best if it uses tensorflow backend. I know you can train images to recognize. Yolo can be used if trained on face. To name the person.
But I wonder if there is any code where you can add new faces to database without training or minimum training. As new faces are added I don't want to train the network repeatedly. Also the less amount of face picture needed the better.
If code is not available any guide or research paper will also be helpful. For example what approach can I take to make an app for a person who has difficulty remembering peoples name. So the app can take a small video or few photos with name and will be able to tell the persons name in the future. Neural network should not be retrained while adding new face to database if possible.

It seems your problem is more related to Face Identification than Face Recognition.
I understand you are looking for the implementation using a NN based approach, but if you're open to giving it a try to other approaches you could consider using Eigenfaces, which is based in PCA.
For that, you can find some references and code implementations.
Datasets you can use for testing face identification:

AT&T Face Database
Extended Yale Face Database B
BioID Face Database

References you can look at for additional implementation details:
https://www.researchgate.net/publication/333367462_Improving_face_recognition_of_artificial_social_companions_for_smart_working_and_living_environments
A forked repository on GitHub for this approach:
https://github.com/joaoquintas/facerec
Original repository for reference https://github.com/bytefish/facerec


text:
How matrix factorization helps with recommendations when it converges to the initial user-items matrix?
We can say that matrix factorization of a matrix $R$, in general, is finding two matrices $P$ and $Q$ such that $R \approx P.Q^{T}$ with some constraints on $P$ and $Q$. Looking at some matrix factorization algorithms on the internet like Scikit-Learn's Non-Negative Matrix Factorization I come to wonder how this works for recommendation systems. Generally with recommendation systems we have a user-item ratings matrix, let's denote it $R$, which is really sparse so when we look at datasets we find missing values, $NaN$. When I look at examples of using matrix factorization for recommender systems I find that the missing values are replaces with $0$. My question is, how do we get actual predictions on the items non rated by users when the dot product $P.Q^{T}$ is supposed to converge to $R$?
I have tried with this simple matrix that I found here
R = [
     [5,3,0,1],
     [4,0,0,1],
     [1,1,0,5],
     [1,0,0,4],
     [0,1,5,4],
    ]
R = np.array(R)

The algorithm I used is Scikit-Learn's and no matter how I change the parameters, I can't seem to find a matrix that has actual values in place of $0$s. It always finds a really good approximation of $R$. Maybe all the hyperparameter tuning I'm doing is leading to overfitting, and let's suppose there is a set of combination of parameters for which we don't have $0$s and still we minimize $||R-P.Q^{T}||$ with regard to some norm to a decent level, how can we be sure that the predictions are accurate? I mean, there must be many different combinations of parameters that ensure both prediction different values for the $0$s and minimizing $||R-P.Q^{T}||$ to a decent level.
Thank you!


text:
Improving validation losses and accuracy for 3D CNN
I have used a 3D CNN architecture, for detecting the presence of a particular promoter (MGMT), by using FLAIR brain scans. (64 slices per patient). The output is supposed to be binary (0/1).
I have gone through the pre-processing properly, and used stratification after splitting the "train" dataset into train and validation sets, (80-20 ratio). My model initialisation and training kernels look like this:
def get_model(width=128, height=128, depth=64):
"""Build a 3D convolutional neural network model."""

inputs = keras.Input((width, height, depth, 1))

x = layers.Conv3D(filters=64, kernel_size=3, activation="relu")(inputs)
x = layers.MaxPool3D(pool_size=2)(x)
x = layers.BatchNormalization()(x)

x = layers.Conv3D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPool3D(pool_size=2)(x)
x = layers.BatchNormalization()(x)

x = layers.Conv3D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPool3D(pool_size=2)(x)
x = layers.BatchNormalization()(x)

x = layers.Conv3D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPool3D(pool_size=2)(x)
x = layers.BatchNormalization()(x)

x = layers.GlobalAveragePooling3D()(x)
x = layers.Dense(units=512, activation="relu")(x)
x = layers.Dropout(0.3)(x)

outputs = layers.Dense(units=1, activation="sigmoid")(x)

# Define the model.
model = keras.Model(inputs, outputs, name="3dcnn")
return model

# Build model.
model = get_model(width=128, height=128, depth=64)
model.summary()

Compile model:
initial_learning_rate = 0.0001
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
    initial_learning_rate, decay_steps=100000, decay_rate=0.96, staircase=True
)
model.compile(
    loss="binary_crossentropy",
    optimizer=keras.optimizers.Adam(learning_rate=lr_schedule),
    metrics=["acc"],
)

# Define callbacks.
checkpoint_cb = keras.callbacks.ModelCheckpoint(
    "Brain_3d_classification.h5", save_best_only=True,monitor = 'val_acc',
                             mode = 'max', verbose = 1
)
early_stopping_cb = keras.callbacks.EarlyStopping(monitor="val_acc", patience=20,mode = 'max', verbose = 1,
                           restore_best_weights = True)

# Train the model, doing validation at the end of each epoch
epochs = 60
model.fit(
    train_dataset,
    validation_data=valid_dataset,
    epochs=epochs,
    shuffle=True,
    verbose=2,
    callbacks=[checkpoint_cb, early_stopping_cb],
)

This is my first time ever working with a 3D CNN, and I used this keras webpage for the format:https://keras.io/examples/vision/3D_image_classification/
The (max) validation accuracy in my case was about 54%. I tried reducing the initial learning rate , and for 0.00001 I got to a max of 66.7%. For learning rates of 0.00005, 0.00002, I got max accuracy of about 60 and 62%.
Accuracy vs epoch plots for learning rates 0.0001, 0.00005,0.00002 and 0.00001:

It does seem like reducing the initial learning rate has a positive effect on accuracy, although the accuracy is still very low.
What other parameters can I tune to expect a better accuracy? And is it okay to just keep reducing the initial learning rate until we achieve a targeted accuracy?
I know this is a rather broad question, but I am quite confused as to how we should approach increasing the accuracy in the case of CNNs, (that too 3D), where there just seems to be a lot of stuff going on. Do I change something in my initialisations? Add more layers? Or change the parameters? Do I decrease or increase them? With so many things going on, I don't think trying every combination and just keep repeating the training process is an efficient idea...
Full notebook (including pre-processing steps): https://www.kaggle.com/shivamee682003/3d-image-preprocessing-17cd03/edit

link you provide to the notebook is not working.

What is the No Information Rate (NIR)? I.e. what are the percentages of positive and negative labels? Have you looked at the predictions of your model? If it's all 0's or all 1's then it probably learned nothing, other than predicting the majority class.
When it comes to architectural choices and hyperparameters, especially if you start working with NNs, then Andrej Karpathy's blog post called A Recipe for Training Neural Networks is a really good starting point. It gives a good reference on how to approach things in the beginning when you have not much intuition. Simply reducing the learning rate will not help much if your model is way too small. You may also find it useful to add ResNet-like skip connections to improve performance for very deep models (i.e. many layer).

Try removing the dropout before the prediction layer. I couldn't find the paper or article I read about this (will update the post once I find it), just found a Cross Validated post which does not add much information. As you are
If you are lowering the learning rate, you should also lower the batch size accordingly.
As for Batch Normalization layers, they probably should be applied after the convolutional layers.


text:
A recommender system based on millions of fields including text and number
I want to train a model based on millions of fields, including text and number, that are stored in a SQL database and recommend a perfect match based on some inputs. Now, which algorithm is the best for this problem?
For instance, consider this database pattern:

Title
Content
Volume
Count

First
row1
5.36
34

Second
row2
36.1
239

...
...
...
...

The first step
You need to decide if you want to hold each string column or not. Then you must encode your text fields into numbers which you need to use some embedding algorithms like word2Vec. Check here.
Second step
Probably, you will have a lot of columns. Now, you need to reduce the space dimension. PCA, manifold transforms, partial least squares regression, etc., may help you in this way.
Third step
Here, you will have nice and tidy tabular data which you can feed into any Recommender System you want to.

I presumed that you mean millions of columns when you say "millions of fields".

If you mean millions of rows, thenprobably need to use methods thatdealingh Big Data.


text:
What is the purpose of hard distillation?
In order to get a smaller model, one often uses larger model, that performs reasonably well on the data as a teacher, and uses the information from large model to train the smaller one.
There are several strategies to do this:

Soft distillation
Given logits of the teacher one adds the KL-divergence between the student logits and teacher logits to  the loss:
$$
\mathcal{L}_{loss} = (1 - \alpha) \mathcal{L}_{BCE} (y_{student}, y_{true}) +
\lambda  \mathcal{L}_{KL} (y_{student}, y_{teacher})
$$
Intuition behind this approach is clear - logits are more informative than a single target label and seemingly allow for faster training.

Hard distillation
One adds the BCE between student logits and teacher model outputs as if they were true labels.
$$
\mathcal{L}_{loss} = (1 - \alpha) \mathcal{L}_{BCE} (y_{student}, y_{true}) +
\lambda  \mathcal{L}_{BCE} (y_{student}, y_{teacher})
$$

And the benefit of the last approach is unclear to me. For the perfect model, one will have no difference with the vanilla training procedure, and for the case, where the teacher makes mistakes, we will optimize the wrong objective.
Despite these concerns, it was shown experimentally in several papers, and in
the Deit, that this objective can improve performance. Even more, it is better, than soft distillation.

Why can this be the case?

There are multiple confounding factors here. Firstly, learning from the mistakes is not necessarily bad if encouraging the student model to make these mistakes improves its generalisation†. This can be a result of there being an insufficient sampling around these incorrectly predicted data points, incorrect human-labelling, or these points are just too hard to classify anyway.
If you train a student network solely on the incorrect predictions of the teacher (no ground truth) it will generalise to some extent. This is because the distillation process is doing some form of function matching. Interpolating between these incorrect predictions is then a reasonable function for classifying the training data, hence the student will non-trivially generalises.
Another perspective is that regardless of these two methods (hard or soft), the teacher is still transferring the translational equivariance to the student, which is a strong inductive bias for generalisation.
In summary, although the correlation between classes of the teacher predictions is one important component attributed to the success of distillation, there are a lot more factors to consider which then explain why hard distillation works.
†: Alternatively, we can say the student may accurately predict more of the training data on average by making these few mistakes anyway.


text:
What are the defining moments that make community realise the potential of deep learning?
Consider the following paragraph from the chapter named pre-trained models from the textbook titled Deep Learning with PyTorch by Eli Stevens et al.

The AlexNet architecture won the 2012 ILSVRC by a large margin, with a
top-5 test error rate (that is, the correct label must be in the top 5
predictions) of 15.4%. By comparison, the second-best submission,
which wasn’t based on a deep network, trailed at 26.2%. This was a
defining moment in the history of computer vision: the moment when the
community started to realize the potential of deep learning for vision
tasks. That leap was followed by constant improvement, with more
modern architectures and training methods getting top-5 error rates as
low as 3%.

This paragraph is mentioning a moment that makes the community realize the potential of deep learning. Are there any other similar defining moments in the history of deep learning?


text:
Multi label classification on non binary labels with pytorch
I am working on a project consisting of medical images and a huge dataset of multi-label and non-binary labels/outcomes ( sex, blood pressure, age and 40 more ).
Would be the best approach to hard code all of them or is there some better approach? If this is the best way, does anyone have a similar PyTorch notebook on which I could orientate myself? Or some smart solution how to hard code them automatically?
Any help is welcome!

Is you network supposed to perfrom multilabel-multiclass classification for each training sample?

Hey! Yes, that is the plan.

If your goal is to predict given an image multiple labels (each of them can be binary or multi-class) you could consider two strategies:

Create for each classification task a separate model, which predicts solves only one problem
Create a single model with multiple heads

The first option seems to be more straightforward, but it would most likely need to consume more memory and computational resources, and the gradient signal from the prediction of one model is independent of other models. In case the labels are uncorrelated or weakly correlated this is not a problem.
The second option  when you have a joint backbone for all classification problems and only, in the end, the computation graph splits into branches solving each of the classification problems seems to be more efficient, and in the case, where these tasks are related to each other, improvement of accuracy in one task is very likely to be beneficial for the other task.
Overall, the resulting architecture resembles the Inception architecture:

You can try to put all classification heads in the very end, or some can disentagle from the other part of network a bit earlier.


text:
Can people without a background in STEM go into the field of AI?
I've recently become very interested in the potential AI holds for the future of society. I believe it has the potential to truly alter the way we live our lives in the not-too-distant future. I've read around three dozen books by leading scholars and I've written my undergraduate thesis on the implications of AI for modern warfare and the international balance of power. With that in mind, I've started to think about a career related to AI and machine learning.
However, I have a bit of a problem considering my background as a humanities person. I'm about to finish my undergraduate studies in International Relations and I plan to get a master in a similar field. As someone with little background in Comp Sci/ Math/ Physics/ etc, how could someone go about working in a field like AI? I struggle to see what utility non-STEM people could offer to firms.
I hope I'm asking this in the right place considering this forum is mostly for technical questions.

You will end up gaining a background in math and comp sci anyway as you try to understand what's going on. Physics is entirely optional. BUT: AI research doesn't just have to mean the technical side. If I want to know something about ethical vs unethical AI, a philosopher is likely to have a better answer for me than a programmer. What exactly do you hope to accomplish?

If you want to invent better AI techniques then of course you will have to delve into the maths and computer science. If you want to think about whether we should allow corporations to use our personal data for AI training, then, well, you won't have to. In any case it will help to have a bit of hands-on experience with the technology so you understand what it can and can't do


text:
Is there any way to remove background of an image fully with the help of post-processor techniques(like edge detector) after deep learning based model
I'm using a deep learning-based model (deep lab v3+ with xception as the backbone) for image segmentation and removing the background. The subject of the image will be a person. And my target is to extract the person from the image. But only with the machine learning model, the output is not satisfactory. I'm thinking if somehow I can detect the edge of the person with "Canny Edge detector" and use this as post processor to get exact output. I also try the blur effect for smoothing the object's edge and getting a pleasant-looking output. I saw Adobe Photoshop's "select subject" feature. It is quite accurate.
Main image -> segmented mask from ml-model -> adding blur effect on the edge
Canny Edge Detector as Post Processor:
I tried the following steps:

Run Canny Edge on the main image. canny edge image
Remove the background noise(of the canny edge image) using the image segmentation mask(I got from the ml-model). Here I use bitwise-and of the canny edge image and segmentation mask. background noise removed canny edge image
In the 2nd step, some of the real edges of the person also get removed, and to restore those, I have used the breadth-first-search algorithm and run it up to some limit(suppose, up to 10 neighbors). after BFS
I shrink the segmentation mask and using this, removed the interior noise of the canny edge image. removing the inside noise

Now if I could somehow connect the edge line of the canny edge image, we would get a perfect segmentation. But I failed to do so.
Any suggestion will be helpful for me.


text:
Demonstration of AI-powered Mario collecting lots of coins?
As part of a talk I'm giving, I'd like to show one of the many videos on YouTube where an AI is playing Mario, such as this one. What bothers me though is that the AI is trying to complete the level as quickly as possible, without trying to collect lots of coins. I believe it's known as "speed run" in the gaming world, which is fine and well, but I think most people expect Mario to be collecting lots of coins and mushrooms.
Are you familiar with a video of an AI-powered Mario that does collect a lot of coins and mushrooms?
If not, maybe you know a different video of a similarly popular video game where the AI does try to get lots of points and not just do a speed run?


text:
What is the sensible amount of augmentation?
I am playing with the transforms from Torchvision.
There are plenty of different kinds of these like:

Resize
RandomCrop
ColorJitter
Blurring
...

These are some cases of Resize for a given image:

ColorJitter

RandomAffine

The main purpose of the augmentation procedure is to prevent overfitting, extend
the training dataset in a certain way.
Some transformed images still look like the image from the original dataset, since a small change in contrast or brightness still makes the image look real.
For the other, there can be a significant change in color, or the original image occupies only a small fraction of the resulting image.
In all cases, one can still classify this object as a dog. However, many of these would not lie on the manifold of real images, and the classification for these may not make sense.
Are there some papers or research discussing the issue of the augmentation choice, and the correspondence of these to the directions along the manifold of real images?


text:
What is meant by sub-region of an image?
Consider the following sentences from the research paper titled PatternNet: Visual Pattern Mining with Deep Neural Network by Hongzhi Li et al.

The value of each pixel in a feature map is the response of a filter
with respect to a sub-region of an input image. A high value for a
pixel means that the filter is activated by the content of the
sub-region of the input image.

The sentences mention the sub-regions of an image. Is there any formal definition for a sub-region of an image?

It seems that they are informally using the term "sub-region" to refer to the section of the image with which you multiply the kernel to produce a scalar value of the feature map (which they call pixel, but I would use the term pixel only to refer to the scalar values of the image). So, it seems that a sub-region is a synonym for receptive field. We have a question on the difference between receptive fields and feature maps here.


text:
CNN: Difficulties understanding backward pass derivatives
I have really quite hard difficulties to understand what is actually going on in the backward pass of a CNN.
I am currently focusing on these references:

https://towardsdatascience.com/forward-and-backward-propagations-for-2d-convolutional-layers-ed970f8bf602
https://leonardoaraujosantos.gitbook.io/artificial-inteligence/machine_learning/deep_learning/convolution_layer

At the moment, I only want to compute the gradient which is then passed to the next lower layer.
I tried to implement the last equation on this image (from [1]):
https://miro.medium.com/max/2400/1*K2K0tfxmAlyRlqqbj4z0Rg@2x.png
For me, this formula looks like 6 nested for-loops.
The first for the channel c, the second for the output height i, the third for the output height j, the fourth for the number of kernels f, the fith for the kernel height m and the sixth for the kernel width n.
Am I wrong? I tried to implement this but I always get an out of bounds error.
Someone here who doesn't mind going a bit more into detail?
Any tips are greatly appreciated

Hello. This post seems to be either off-topic (because it may be a programming issue) or too broad. I recommend that you take a look at our on-topic page and How do I ask a good question?.


text:
Decreasing number of neurons in CNN
the conventional way of creating a CNN is using increasing number of neurons:
model = models.Sequential([
  layers.Conv2D(32,(3,3),activation='relu',input_shape=input_size),
  layers.MaxPooling2D((2,2)),
  layers.Conv2D(64,(3,3),activation='relu'),
  layers.MaxPooling2D((2,2)),
  layers.Conv2D(128,(3,3),activation='relu'),
  layers.MaxPooling2D((2,2)),
  layers.Conv2D(128,(3,3),activation='relu'),
  layers.MaxPooling2D((2,2)),
  layers.Flatten(),
  layers.Dense(128,activation='relu'),
  layers.Dense(64,activation='relu'),
  layers.Dense(1,activation='sigmoid')
])

where in this case, the number of neurons increase from 32, 64, to 128. However, i have also found a paper https://pubmed.ncbi.nlm.nih.gov/33532975/ that uses decreasing number of neurons i.e. 128, 64, 32 , as the network goes deeper. but in this paper, not much explanation was given on how the NN work in decreasing number of neurons. Does it mean decreasing number of neurons assumes that "there are less number of important features to be captured as the network goes deeper" ?
Question:
Can someone explain to me

how does the increasing number of neurons work
how does the decreasing number of neurons work and why this is not the common practice
referring to 2, what keyword should i find, in order to get articles or writing related to 2?


text:
How are partial derivatives calculated in a computational graph?
I am trying to understand how are partial derivatives calculated in a computational graph. I understand reasoning behind computational graphs and I am bold enough to say I understand how they work, at least on high level of understanding.
But what I don't know is how are partial derivatives itself computed or better said, how are they implemented in code.
I have checked few resources like this lecture slides from CS231N, this blog post or this blog post on TowardsDatascience. They explain graphs and how are expressions calculated in graph, but they don't explain how are partial derivatives derived (or I didn't understand from those explanations). For example, blog post from TowardsDatascience says:

Next, we need to calculate the partial derivatives of each connection between operations, represented by the edges. These are the calculations of the partials of each edge:

And then they show image with values of partial derivatives but they newer explain how are these equations actually calculated in implementation of graph.
Yes, okay, I know how to calculate these partial derivatives on paper and then hardcode them in my code, but I don't know how are they actually automatically computed and implemented in code of libraries like Torch or Theano.
Do they have some basic rules implemented in code, like, for example:
$$ \frac{\partial (a + b)}{\partial a} = \frac{\partial a}{\partial a} + \frac{\partial b}{\partial a} = 1 $$
and then decompose expressions until they reach basic elements/rules or is there another way that libraries like Torch, Theano or TF do it?
Or to put it in another way, if I have this code in Torch:
from torch import Tensor
from torch.autograd import Variable

def element(val):
    return Variable(Tensor([val]), requires_grad=True)

# Input nodes
i = element(2)
j = element(3)
k = element(5)
l = element(7)

# Middle and output layers
m = i*j
n = m+k
y = n*l

# Calculate the partial derivative
y.backward()
dj = j.grad
print(dj)

how does Torch know, that is, how does it compute internally that $ \frac{\partial y}{\partial j} = l \cdot 1 \cdot i = l \cdot i $?

I'm not sure if that's the actual implementation, but you just need to find all the paths from the input node to the output node in the computational graph (which is directed), compute the gradients along them by multiplying the edge gradients along each path, then sum them to the get the total gradient. The hard part is finding all the paths, but there's well-known algorithms for that.


text:
model and trained model parameters on CIFAR-10
I'm looking for different models (specifically ResNet18/20, ResNet32/34, VGG16, MobileNet and SqueezeNet) and their parameters after training (i.e., .pth file) that were trained on CIFAR-10 or CIFAR-100. I tried looking for them for hours and couldn't find anything. perhaps someone could refer me to a site which have trained models for CIFAR-x?
Thank you.

You can find pretrained models on CIFAR-10 in this GitHub repository.
Also, for fun you can take any backbone trained on ImageNet from TorchVision models.
Just replace the classification part from the Linear layer, outputting 1000 classes to 10 or 100. Then you can freeze most of the layers of the network and fit just some bottom layers with the classification head (last nn.Linear).
Modern papers often use CIFAR-10 for transfer learning and you can find the parameters in the paper or discussion on GitHub like here.


text:
Can I flip a video to generate more data for action recognition?
There are 8 distinct action classes and around 50+ videos per class. I was wondering if flipping videos from the training set can be a good option to generate additional data. Is it?

Probably flipping a video left/right will be OK and useful for your case.
When considering data augmentation approaches, you should think about two things that may prevent it working:

Could the augentation change the label? E.g. would a human looking at the augmented data still label it the same way?

Does the augmentation create data that is too different from expected later use?

So in your case for action recognition you should only be concerned (and maybe not add the left-right flipped videos) if:

The activity label would change depending if tasks were performed left-handed or right-handed.

There is lots of content in the videos (e.g. writing) that is unrealistic when flipped. This is why you should only flip left/right, not top/bottom, in most cases. However, top/bottom and even arbitrary rotation might be fine if your videos are a top-down view, so it depends on your specific case.

As an aside, it is also important to keep the original and augmented copy in the same part of the dataset (training, cross-validation or testing), because they are correlated - not doing so may cause a data leak that will prevent you measuring performance correctly. To play this safe, you should only augment training data, so that you don't risk measuring generalisation of your model against imaginary production data that could not occur in reality.
Other augmentations you might consider for video could be small rotations, random crops, and minor colour, contrast and brightness adjustments.

Thanks @Neil for the detailed response.
My dataset consists road accidents as once class which remains unaffected to the flipping. Also I was using augmentation to flip images horizontally and rotate slightly by 10 degrees. But then, I still was wondering if could chuck the horizontal flip data augmentation and create more data just by flipping them and then add other augmentations for more ruobustness. What do you think?

@UdaySai I cannot see your images, even if I could, I don't know what your production scenario is like. I would say your biggest concern is my second point - at what point do your augmentations get too far away from the real dataset that you will be predicting against. One way to play that safe is to augment training data only, not validation or test data.

It is possible that road accidents are impacted by horizontal flipping, since the road system is handed. I would still do the augmentation, because it may not be a big effect compared to other detectable features of accidents. But definitely do not augment cross-validation or test sets.


text:
Is there any simple example for volumetric data except from physics and medicine?
Recently I heard about the term volumetric data. The definition for volumetric data is as follows
#1: Definition

Volumetric data is typically a set S of samples $(x, y, z, v)$,
representing the value v of some property of the data, at a 3D
location $(x, y, z)$. If the value is simply a 0 or a 1, with a value
of 0 indicating background and a value of 1 indicating the object,
then the data is referred to as binary data. The data may instead be
multivalued, with the value representing some measurable property of
the data, including, for example, color, density, heat, or pressure.

#2: Some more details

A volumetric dataset consists of information at sample locations in
some space. The information may be a scalar (such as density in a
computed tomography (CT) scan), a vector (such as velocity in a flow
field), or a higher-order tensor (such as energy, density, and
momentum in computational fluid dynamics (CFD)). The space is usually
3D, consisting of either three spatial dimensions or another
combination of spatial and frequency dimensions.

In simple words, I can say that volumetric data is nothing but a three-dimensional collection of tensors.
The articles linked above contain some examples of volumetric data in the medical (and probably physics) domain.
Are there any other simple real-world examples for volumetric data other than the medical and physics domain and physics?

"Physics" is quite broad... That eliminates several engineering applications I can think of. However, you mention medicine, where it's mostly used by biophysicists, so by physics, do you mean pure/theoretical physics?

@Avatrin Yeah, I do mean pure/theoretical physics.

Whenever a system is being modelled in three dimensions, you can be sure that tensors either can or are being used. Most of the systems I can think of are either simulated volumetric data, or a combination of real measurements and interpolated values.
CAD and BIM
Computer-Aided Design, or CAD, if quite commonly used in engineering. It's quite natural that, if they are modelling a mechanical system, that they need information regarding the materials they are considering to use to make the device; A little bit of googling showed that Autodesks Inventor computes inertia tensors and their mechanical simulation software computes stress tensors.
BIM is essentially CAD for the construction industry, but again, a lot of the same kind of information regarding the materials used is needed; Else, the building may collapse.
Geology
Modelling the stability of soil to construct a skyscraper may require 3D data. There is an old question at gis.stackexchange discussing software used to model geological systems. Of those, this one and this other one still seems to be up.
Meteorology
Most meteorological data, to the best of my knowledge, is two-dimensional. However, weather radar can be used to extract 3D data with information about precipitation and motion.


text:
Alternatives to neural networks for function approximation in Q learning?
I want to know if there is anything other than neural networks (or Deep NNs) that I can effectively use to perform function approximation? I am asking this w.r.t to the use of approximators in Q learning with large state space.

Any parameterisable function does the trick.

@DavidIreland Any examples?
Actually, I am going to post my scenario specific question after 20 min (as I have to wait before posting another). I will give the link here. I request you to please also take a look on that, if possible. I have been asking small small questions and still stuck. So I decided to ask my scenario specific question finally.

@DavidIreland This is the link I said in the above comment. https://ai.stackexchange.com/questions/32472/scenario-specific-question-w-r-t-q-learning-and-deep-q-networks

It's a good idea to do a little bit of research before asking a question. Please, take a look at How do I ask a good question?.

it looks like you’re using a graph as the state, in which case I’d recommend looking at graph neural networks.

Any series approximation technique will regress on training data. Taylor Series approximation is a popular technique.
https://en.wikipedia.org/wiki/Taylor_series
Fourier series is also important in DSP and computing analog signals but  can be used to approximate functions where a higher term count yields more accuracy.
https://en.wikipedia.org/wiki/Fourier_series#:~:text=A%20Fourier%20series%20(%2F%CB%88f,be%20determined%20using%20harmonic%20analysis.
you can pick any degree of a polynomial and start regressing on data points for the coefficients but these term approximations show the calculus of choosing the degree of the approximator with respect to accuracy and other considerations such as if the data is even able to be modeled by a function (vertical line test) or if it needs to be transformed/mapped first.
To my understanding, Neural Networks have powerful generalization ability meaning they do well at modelling the function over a larger domain (including outside the trainning dataset as seen in cross-validation). To learn more on this I would suggest reading on the difference between neural networks and linear approximators.
I say this since any black box optimizer such as neural networks are equivalent to function approximators. Deep Learning is a subset of the study of function approximation but choosing what approximator is needed needs to be researched for each application.

One note of caution on the last paragraph: Deep Learning involves much more than function approximation. Topics like scalable system design, deployment, computational complexity and more don't fall cleanly into a "function approximator" perspective. This isn't a pedantic critique but a warning to anyone that might take that too seriously and then make reductive claims about the "nature of deep learning being nothing but function approximation."


text:
Feature Extraction for printer classification
I need some advice. I am currently trying to do a printer classification with ML/DL.
What do I have?
11 colored-images with high resolution from 8 different inkjet-printers (in total 88 images)
I have 8 classes (printers)
All images are scanned with 2.400 dpi, so you are able to see the halftone of the images and the matrix dots
I know each printers are different in terms of size of matrix dots, dot pattern etc.
Based on that I need to do a feature extraction and train a ML model which can classify the correct printer. There is a previous work which has been done with Wavelet-Transformation for feature extraction and SVM for classification. The goal now is to find another approach of feature extraction and training.
My question here is, what do you think is the best solution?
My idea is:
Isolate the dots into binary color (black/white)
do an edge detection with opencv (using filters like sobel, canny etc.)
But I am not sure if this is a good approach. After reading a lot of papers on related work I found out many used Transfer Learning (e.g. VGG, Resnet) where features are learned in the training process.
So basically I have images and when you can zoom in you see for each printers the pattern are different. So instead of doing Wavelet-Transformation I need to do another approach.
In the litarature common feature extractor for this are Gray-levcel Co-Occurences, Wavelet-Transformation, Spartial filters which will be used in SVM or AdaBoost. Another approach is as said above with pre-trained CNN (transfer learning).
So, what do you think I should tackle next?

How about you make a dataset using patches from your image, and train a CNN model with that dataset?
That is, if you want to train a neural net, a dataset with 11 images for each class is too small and thus is prone to overfitting.
However, since your image is high resolution and the printers can be classified by just using the zoomed in images, you can split each of your images into hundreds to thousands of patches and use that as a dataset. That way each of your class (printer) will have data samples equal to 11 * num_of_patches_per_image.
After you make your dataset, train a cnn classifier with 8 classes with it. You can do transfer learning and finetune a pretrained cnn, or I think depending on the amount of data you end up with, you can train a whole new cnn from scratch.
During inference time, you can pass your cnn model a patch from your test image (or you can pass a dozen of patches and ensemble the results for increased accuracy and robustness).
P.S.
With deep learning, you don't really have to worry too much about feature extraction as traditional machine learning. Especially with image-like data. I have a strong feeling that you can probably make a very high-performance model just by doing the above.

Yes I know that this is a way to go. But the goal of the task is to find a manual way of feature extraction. We know that each printer are different in terms of their printing algorithm, thus the dot size are different, distances of dots etc. We want to find a way to extract those features with manual approach, if it's possible. So the idea is to to work with classic Image Processing tools like opencv.. is there such a way?

Does this feature have to be human interpretable? You suggested pretrained cnns above, and they also don’t extract any interpretable features.

Yes. I think with pretrained CNN we don't know which features are relevant or not, right? I somehow need to extract the dots and compute the average size (mean) and standard deviation.. these are possible features in my opinion


text:
General approaches in text encoding and labelling for NLP
What are the approaches of encoding text data? I would be glad to hear some summarization from experienced persons.
And are there any solutions accepting words outside the vocabulary and including them to the results (online machine learning)?
Data input
So my basic understanding is that if we want predict some value (linear regression) or say what is the probability of occuring some event (logistic regression) we have to gather some features as our input and encode them as number. But this is not necessarily true when working with continuous data like sentences.
The most naive aproach, which comes to my mind is just to assign some natural numbers to each word in the vocabulary. But this number does not contain any meaningful data about the word itself. On the other hand what seems to be important in NLP is just the order of the words. This is where I think about n-grams so we feed network with more than just one word. Or attention like in the Transformer.
Another idea, which cames to my mind is to vectorize the word using one of the Word Embedding technique. Here we have some context about the word so the input is not just a dumb number. But does it have any value when we want to predict the next word? Can Word Embedding be used in that way or it's purpose is completely different.
Last thing I was reading of was to encode characters rather than words but it feels pointless in such basic example as next word prediction. I would think about it more for sub-word tasks like inflection generating.
Labelling
Again based on my knowledge when we want to solve yes/no problem we're using sigmoid function. If we have more classes we can use one-hot encoding. But sometimes the output of the network might give us ambiguous meaning so we're using the softmax function so all output sum to 1.
How this looks in NLP area? When having a vocabulary consisting of 600k words do we really need 600k softmaxed outputs? I'm also thinking there about Word Embedding solutions where we can reduce the number of outputs to let's say 300 numbers and then find the closest word matching the output without using softmax.

What do you want to do? NLP is a vast area, and there are lots of options to deal with it, but this depends on what you want to achieve. Without knowing that it's impossible to answer.

I wasn't thinking about any specific use case. I thought it will be possible to get some general understanding of that topic. But if I had to make something then I have two examples:

Conversion of text to phonetic representation in a language having some general rules but also many exceptions
Part-of-speech tagging

Hello. Welcome to Artificial Intelligence Stack Exchange. After having quickly read several parts of this post, it seems to me that you're asking many distinct (although related) questions. Please, ask only one question per post. If you have more than one question/issue/problem, ask each question in a separate post. If you think this question was wrongly closed and you're actually asking the same question but in different ways, let me know.

Text encoding depends very much on the purpose of your application. Here are some examples:
Text-to-speech: You would start with the word form itself, and probably look it up in a table that gives you a mapping to the phonological structure. Or you work through it (eg with a finite state transducer) and look at combinations of letters and check for the most common realistion of these letters (depending on the language and the writing system you are using). In the latter approach you can also deal with unknown words.
Part-of-speech tagging: You would look up the string in a dictionary, returning you a list of all the possible tags for the word (possibly augmented by frequency information). From then on you would operate on a sequence of tags, until you have them disambiguated, and you end up with a list that maps each word in the sentence (token) with a particular tag. For unknown words you can either assign them all possible tags (and let the disambiguation sort them out), or choose only the open class ones (you won't find many unknown prepositions or determiners).
Information retrieval/storage: This is one situation where you would want to replace word by numbers. You'd have a look-up table that maps word to integer values, and these can be very efficiently stored. For example, in the Bank of English one of the biggest early language corpora (several 100s of millions of tokens in the early 1990s), each token was encoded as a 4 byte integer. It was then very fast to go from the index positions of a word (from an inveretd index) to the actual position in the file, as each token is 4 bytes long, so the token at index $n$ is at file position $4n$. It also somewhat compresses the text if you assume the average word length is greater than 4 (and that you don't encode the spaces between the tokens). Unknown words: they are by definition not in your text, so you can just return "not found".
Machine learning: ML approaches don't really work on strings, so you need to encode word numerically. Ideally this should not be a random mapping (such as alphabetical order), so one approach that is used is to represent each word by a vector that encodes which other words occur in its neighbourhood. It is a well-known principle in corpus linguistics (since the 1930s) that words are not randomly distributed in a text, but that some words co-occur with regularity. For example, rasher and bacon usually go together. So the vector which describes rasher will have a positive number in the slot which  relates to does it occur with 'bacon'?, whereas the vector describing bookshelf will not. With a large enough vector, you can approximate similarities in meaning between words (based on their lexcial environments). Unknown words: you could use a random or average vector -- I don't know what the standard way is to deal with that.


text:
Use soft-max post-training for a ReLU trained network?
For a project, I've trained multiple networks for multiclass classification all ending with a ReLU activation at the output.
Now the output logits are not probabilities.
Is it valid to get the probability of each class by applying a softmax function at the end, after training?

Hello. Welcome to Artificial Intelligence Stack Exchange! Can you please explain 1. why you trained more than one neural network to solve the same problem, 2. why you used the ReLU at the last layers of these neural nets, 3. how exactly you trained the neural nets? Did you compare the (vector) labels with the output produced by the last ReLU, 3.1 you got good accuracy with this approach?

Hi, thank you.

It's not the exact same problem, it's for slightly differing datasets and slight differing architectures.
This is a constraint set for research purposes.
As usual, using backprop - output decision was decided using argmax over the last relu output vector; and yes test/validation accuracy and loss are satisfying

My main and only question, is if we can convert post-hoc some relu-output to some 'propabilities' in a valid way


text:
tfp.Distributions.Categorical.sample() is picking the same action everytime after certain iterations
I have written a code for an RL agent such that at each state the model calculates the probabilities of all possible actions and samples one action randomly to proceed further. To acheive this, I have written the following code
act_prob_dist = tfp.distributions.Categorical(probs=act_probs)
action = act_prob_dist.sample()

It is working fine in the initial stages of training. Once the model has learnt about the particular state really well and the probability of one particular action has increased significantly than the others, the sample() call is picking is the same action every time. For example, when the action probabilities of a particular state are
tf.Tensor(
[[0.05213022 0.06613996 0.4933109  0.02918373 0.04188393 0.04100212
  0.03228914 0.00716161 0.08877521 0.02158365 0.04645196 0.07092285
  0.00916469]], shape=(1, 13), dtype=float32)

The model is sampling an action randomly. After decent iterations of learning the action probabilities became
tf.Tensor(
[[1.12852089e-12 1.54888698e-06 6.40413802e-08 1.03480375e-11
  2.05246806e-08 2.17290430e-09 1.04494591e-09 5.20959872e-11
  9.99995708e-01 1.26053008e-08 6.85156265e-10 1.70332885e-06
  9.99039457e-07]], shape=(1, 13), dtype=float32)

The model started picking index with high probability every time(index 8 in this case). The documentation reads Generate samples of the specified shape. I'm assuming it implies the choosing happens randomly. Can someone please explain why same action is being chosen in my case?
PS: tf.version is returning tensorflow._api.v2.version

Hello. Welcome to Artificial Intelligence Stack Exchange. It seems to me that this is or could be a programming issue. Let me know if I am wrong or right. Generally, questions about programming problems are off-topic here and should probably be asked on Stack Overflow. Here, we focus on the theoretical, philosophical, social and historical aspects of Artificial Intelligence. Please, take the time to read our on-topic page. So, if you think this is not a programming problem, you should edit your post to clarify that.

I guess you are right. I posted it on stack overflow but got no responses. So I thought there might be something wrong with the way I'm structuring my agent and this platform might be the right place for me to ask