text:
What is the high-level algorithm followed by contemporary packages for the calculation of gradient?
Most of the neural network models in contemporary deep learning packages are trained based on gradients.
Let $f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ be a function for which we want to find a gradient, then the gradient is generally represented by a Jacobian matrix that looks like below
$$J = \begin{bmatrix}
    \dfrac{\partial y_1}{\partial x_1}       &     \dfrac{\partial y_1}{\partial x_2} &     \dfrac{\partial y_1}{\partial x_3} &\dots &     \dfrac{\partial y_1}{\partial x_m} \\
     \dfrac{\partial y_2}{\partial x_1}       &     \dfrac{\partial y_2}{\partial x_2} &     \dfrac{\partial y_2}{\partial x_3} &\dots &     \dfrac{\partial y_2}{\partial x_m} \\
    \cdots & \cdots & \cdots & \cdots & \cdots \\
        \dfrac{\partial y_n}{\partial x_1}       &     \dfrac{\partial y_n}{\partial x_2} &     \dfrac{\partial y_n}{\partial x_3} &\dots &     \dfrac{\partial y_n}{\partial x_m} \\
\end{bmatrix}
$$
For example: If $f(x_1, x_2) = \begin{bmatrix}
           x_1 + x_2 \\
           x_1x_2
         \end{bmatrix}$ then $J = \begin{bmatrix}
           1 & 1 \\
           x_2 & x_1
         \end{bmatrix}$
After calculating the Jacobian matrix, we can substitute the co-ordinate values of a particular point so that we can obtain a real matrix which is a gradient at a particular point.
$$
J_{(4, 5)} = \begin{bmatrix}
           1 & 1 \\
           5 & 4
         \end{bmatrix}
$$
In order to perform the gradient of a function at a point, the algorithm I know is as follows:

Write each output of the function in the analytical form in terms of input;
Apply partial derivative on each output w.r.t each input;
Substitute the values of the input point at which we want to find the gradient.

Thus, finally, we will get the gradient.
Do the popular packages like PyTorch, Tensorflow, Keras, etc., use this or a variant of this algorithm to find the gradients at a particular point?
If yes, will those packages be able to write the analytical forms of all the output variables in terms of input variables?
If not, what is the high-level algorithm for calculating gradients? Is it based on geometrical slope version of gradient?

Does the popular packages like PyTorch, Tensorflow, Keras, etc., use this or a variant of this algorithm to find the gradients at a particular point?

Yes. This is effectively what back-propagation is. However, there are a couple of important details:

Using a loss function flattens the matrix form you have to a vector, because with a loss function the dimension of the output $n=1$. This is important, since it is not possible to directly search for a maximum or minimum value for multi-dimensional output.

Back-propagation is resolved as follows:

Set up by constructing a computation graph that composes multiple functions. Some libraries set this graph up directly e.g. TensorFlow, whilst others like PyTorch can resolve it dynamically at run time from requested computations (this is convenient and easier to write, but there is a performance cost).
Gradients are calculated numerically by resolving function composition in reverse using the chain rule.
There is an implied analytic form for the forward calculation, but it never used directly.

If yes, will those packages be able to write the analytical forms of all the output variables in terms of input variables?

In theory, yes, as the data to do so is in the computational graphs that are used to run neural networks forward, and is the same data that is used to resolve back propagation. In practice I have not seen this done. There may be an add-on or library that could print out the analytic forms of neural network outputs at any layer, or for the loss function. If there is not one, it would not be too hard to write one.
However, other than as a teaching aid for very small networks, then the analytical view of the outputs or loss function are not a practical or useful description of what is going on. The graph view showing how the linear algebra and non-linear activations are composed into a sequence of computations (and reversed step-by-step for gradient calculations) is much easier to comprehend. Many neural network libraries can generate views of this compuational graph.
One important take-away though, is that despite the apparent complexity of back-propagation, it is doing exactly what you expect in the question: Calculating the gradient components in a Jacobian.


text:
Explainable AI for complex input features
I have a model for binary classification that includes 2 linear layers with RELU activation function and Sigmoid in the last layer. The input features are FastText word embedding, frequency, and statistical signals.
This model has a 93% f1-score and I want to add an explanation to this model but I don't know how can I start.
My question is which models or papers good for these complex input features?
I appreciate any advice to achieve this goal.

Hello. What do you mean by "i want to add explanation to this model"? You need to clarify what exactly do you expect the model to do. Moreover, I would recommend that you take a look at this post before proceeding. Once you have done that, please, edit the post to ask a more specific question.

@nbro tnx for your response, yes i read about this articles & areas but in my model, input features are in high dimensions (word embedding) and these articles not good in this situation my question is: which Explainable AI techniques good for such models that i said.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Proximal Policy Optimization for continuous control problem
I am using clipped PPO to train a neural network to act as the controller for steering an aircraft, and am finding that my networks aren't learning. The goal is to keep the aircraft flying to cover the distance, and the code is implemented using Pytorch. I am wondering if anyone who is more experienced would be willing to take a look at my implementation.
Aircraft and actor model
I have a flight simulation environment with the dynamical system of an aircraft, where the simulation is propagated through time in 0.1s increments. The states of the aircraft serve as inputs to the actor network (airspeed, pitch, heading, height), and the controls include pitch and roll angles.
The actor network uses a multivariate normal distribution to output two values that serve as the means for the pitch and roll angle controls. The variances are fixed. This way, the policy is stochastic and allows for exploration. I am using two hidden layers of 256 neurons each for both actor and critic networks, with learning rates of 1e-4. ReLu activation on the hidden layers and sigmoid on the actor's output, with the critic's output being linear.
Reward structure
I have tried two different reward structures:

A reward of 1 for every time step that the aircraft is in the air (with a reward of 0 in the terminal state, which is when the aircraft crashes to the ground or if it achieves the maximum flight time)
A reward of 0 for every time step, but a single reward computed at the terminal state that is proportional to the displacement of the aircraft over the simulation, and an extremely large penalty that is subtracted from this reward if the aircraft crashes.

Neither of these reward structures seemed to allow learning. I am wondering if simply having a reward of 1 at every time step, much like the gym cartpole environment is insufficient in getting the model to learn how to fly. I was expecting the network to figure out ways of controlling the aircraft to fly longer to obtain a greater total reward. For the second reward structure, I expected the advantage estimation to carry the large reward/penalty backward through the trajectory so that the actor would learn to avoid flying in certain ways that end up in crashing the plane.
PPO implementation
For the clipped PPO implementation, I am:

Resetting the flight simulation environment
Propagating the simulation until the aircraft crashes or reaches maximum flight time
Training the actor and critic networks every 20 time steps (2s of simulation)
Repeating this for a maximum number of simulations

For training, I take the 20 time steps, shuffle them, and divide them into 5 batches. Then I train the network on these batches, reshuffle the 20 time steps and create 5 new batches, and train again.
I repeat this process for a total of 4 epochs per learning cycle.
The fastest that the aircraft can reach the ground (crash) is 1.7s of simulation time (17 time steps), whereas I want the aircraft to fly ultimately for 10m (6000 time steps). I am thinking that it is too difficult to train a network to fly for such a long period of time because it would have to learn all the states leading up to that point.
Results
I found that training the algorithm typically resulted in extreme volatility in the test simulation's score (sum of all rewards at every time step). The score history would look like: (moving average of 100 scores in orange).
What I've tried

Changing the standard deviation of the normal distribution of the actor output so that there is less exploration (stability of the aircraft is quite sensitive to the particular control values).
Advantage normalization
Increasing training time to 10 000 simulations (~40 000 learning iterations since there are 4 epochs per simulation)
Testing the algorithm on the gym's cartpole environment (discrete actions). I was able to successfully train the actor in 200ish games.

If anyone has any other suggestions of things to try, it would be greatly appreciated.

Training using only 20 timesteps at a time is far too small, especially when the goal will ultimately consist of episodes of length 6000. You definitely need to increase that substantially and that will probably solve your problem immediately. You might try something like simulate 5 episodes and then train on all timesteps in those 5 episodes.
If that still doesn't work, another thing you can do is try training the value function (critic) differently, e.g. use monte carlo returns or more steps in the temporal difference, and you can keep a memory replay for the value function as well.

Thanks for your suggestion. I've tried it out, and it doesn't seem to be doing too much better. I'm wondering if PPO isn't the right algorithm for this? Wouldn't it be quite unlikely that a series of stochastic actions would keep the plane flying for so long? And even if that were possible, the clipping and learning rate almost guarantees that the network wouldn't learn immediately from a single successful run no?

@james Have you tried adjusting the learning rate? If it works on cartpole it should probably work on your problem, unless your problem is significantly more difficult.

Yes I've tried tuning the learning rate, but I get a similar score history, where the network appears to be learning sporadically for a few episodes at a time before performance once again plummets. I'm thinking at this point that the stochastic nature of the action selection is hurting performance. The variance of the normal distribution from which the action is sampled is fixed, so if it is too large, it will explore but not consistently perform the "correct" action, while if too small, it will never explore.

Have you tried tuning the variance? You can also predict the variance the same way you predict the mean.

Ah I see. You mean make the variance an output as well. I think that might be a good idea, because as I understand, the exploratory part of the model comes from the variance, and I'm finding that the model isn't exploring enough and becomes stuck at local minima.

A different, variable reward structure might help. You could try a combination of airspeed, pitch, roll and whether it is hovering in the air or not in each timestep as a representation for the reward.
Maybe airspeed should, in expectation, contribute up to 30% of the reward, pitch up to 15%, roll up to 15% and being in the air up to 40%. This would explicitly reward motion, trying new movements as well as being reasonable, i.e. hovering in the air. You can create a new formula for the reward around this premise, even use some logarithms or other fancy functions that you see fit.
The important thing is that this way, the reward in each timestep is varying based on the four aforementioned variables and the chosen formula. It is not 0 or 1 like your previous structures. There is also feedback in each timestep on what it is doing right and how well it is doing, not just the last timestep like the second structure.

I see. I'll try implementing a more detailed reward structure then. I've been running the algorithm for 10 000 - 100 000 simulations. Would that typically be enough for a control problem like this?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?
I have a database that contains healthy persons and lung cancer patients. I need to design a deep neural network for the binary classification problem (cancer/no cancer). I need to split the dataset into 70% train and 30% test.
How can I do the splitting? According to persons?
I think that splitting according to persons is correct since this will ensure that the same person will not exist simultaneously in both the training and the test subsets. This is reasonable since we are recognizing the disease, not the person. If images from the same person exist in both subsets, the problem will be easy, and not reasonable, from a practical point of view. Do you agree?

it is not clear to me what other alternatives you have. From the way you describe the data it seems you have information per each person, e.g. [age:25, weight:60, height: 1.80, blood_pressure:120, ..., cancer: False]. By splitting into train test, since you have one record per person, it would be impossible to have the same person into train and test sets at the same time. Am I missing something?

I have many samples for the same person (10 samples for examples)). The samples represent X rays images from the person.

we are recognizing the disease, not the person.
If you're training a computer vision model with only images and no auxiliary information then a randomized sampling should be enough to prevent the model from over fitting on x-ray scans taken on the same person.
If images from the same person exist in both subsets, the problem will be easy, and not reasonable, from a practical point of view. Do you agree?
Only partially. The aim of a test set is to allow you to quantify how well your model will perform on a real use case. Which is close but not the same as making the problem as hard as possible in testing phase.
Also, it is not necessarily true that having 20 examples from the same person in the training set will lead to high accuracy on even 1 single test instance coming again from the same person. This is because those training instances might contain bias (like many tumors on the left side of the scan rather than the right side).
This is why I personally would go for the following splitting approach:

sample from the whole dataset a small group of people (for example top n people with fewest amount of scans). Let's call this set between groups test set.
sample per person (from the remaining data) 70% training instances and 30% test instances. Let's call it within groups test set.

During validation I would then perform the following checks:

metrics calculation (precision, recall, f-score, i.e the whole package) on the between group test performed as usual.
metrics calculation per person (or clusters of people, paying attention in keeping the clusters the same at every validation step) on the within groups test set.

The second check is the most interesting. Of course there will be some random variation in the performance, but if the model will start performing better on some specific clusters of people that might be a hint that the model is learning some confounding variables probably related to the presence of really similar scans or other bias. testing only on the between groups will not let you to draw that conclusion.


text:
Can a GIoU loss (generalized intersection over union) be used after an STN module (spatial transformer network)?
I have a model that uses an STN module for number detection and Mean Squared Error loss. But I would like to replace it for GIoU, because MSE doesn't take into account how much of the target area has been detected, only how close individual coordinates are close to the target. But I wonder if this makes sense. Has anyone tried it, or has some insight?


text:
2D models on 3D tasks (convolutions): simple replace?
2D tasks enjoy a vast backing of successful models that can be reused.
For convolutions, can one simply replace 2D operations with 3D counterparts and inherit their benefits? Any 'extra steps' to improve the transition? Not interested in unrolling the 3D input along channels.
Publication/repository references help.

Details
The input is an STFT-like transform of multi-channel EEG timeseries, except it's 3D. There's spatial dependency to exploit across all three dimensions. The full transform is 4D, but one of the dimensions is unrolled along channels, so data and transform channels are mixed. The transform itself has stacked complex convolutions and modulus nonlinearities (output is real).
I seek to reuse models like SEResNet, InceptionNet, and EfficientNet. The "benefits" of interest are mainly train viability (convergence speed, amount of required tuning) and generalization (test accuracy) - or alternative to latter, that blocks don't interact harmfully by e.g. assuming an inherently 2D structure.

Could you give a more concrete example of what you would intend to do, and what "benefits" you are hoping to preserve? Some things related to CNNs may translate 2D to 3D, but not everything.

@NeilSlater Updated.


text:
What is the analytical formula for "Kaiming He" probability density function?
A probability density function is a real-valued function that roughly gives the density of probability at a particular value of a random variable.
For example, the probability density function of a normal random variable is given by
$$f(x) = \dfrac{1}{2\sigma \sqrt{2\pi}} e^{-{\LARGE(}\dfrac{x-\mu}{\sigma}{\LARGE)}^2}$$
Uniform Kaiming He probability distribution function is used for initialization of weights in Convolutional neural networks in PyTorch and the distribution function was initially mentioned in the research paper titled Delving Deep into Rectifiers:
Surpassing Human-Level Performance on ImageNet Classification by Kaiming He et al. I think.
What is the analytical formula for the Kaiming He probability density function?

The activation function proposed by He et al. is not a new probability function of its own kind. It's an improvement over a previously proposed activation function now called Xavier or Glorot (even though it was named by the authors normalized activation in the original paper). The Xavier activation is also simply an activation function and not a new kind of probability distribution.
To answer the question, both functions simply sample values from either a uniform or normal distribution. What change is how the parameters that define the distributions are estimated.
Focusing on the Kaiming activation, since that's the question asked, the formulas used to determine the correct parameters of the uniform and normal distributions to sample values from are:

Uniform: $$\boldsymbol{\mathit{U}}(-\sqrt{6 / n_{j}}, +\sqrt{6 / n_{j}})   $$

Normal:
$$\boldsymbol{\mathbf{\mathit{N}}}(0, \sqrt{2/n_{j}})$$

nj is the generic notation used in the original paper, in other sources like the pytorch documentation the notation used is fan_in or fan_out. Without explaining the details, fan_in means number of hidden units of the input layers for the weight matrix being initialized (or number of rows of the weight matrix to initialize), while fan_out means number of hidden units of the output layers for the weight matrix being initialized (or number of columns of the weight matrix to initialize).


text:
Multi-class classification but a single feature sometimes boils it down to a binary-classification
I have a three-class classification problem for a large dataset. Classes are 0, 1, and 2. There's a categorical variable in my feature vectors such that when a sample point has this variable positive, it can only belong to either class 1 or class 2. On the other hand, if that categorical variable is negative for another sample point, then this sample can be from all three classes. I was wondering how I can make a neural network use that information during training? I guess I need a custom loss function however I could not figure out how to exactly create that.

I think a custom loss function would be an overkill in this situation. A linear pattern like this would be easily learned by any loss desinged for multi class classification.
If I were you I would try 3 roads, in this order:

train a classifier based on decision trees (random forest & xgboost in primis). The rule you described would most likely be learned as first split to perform, if the remaining features are easily separable then such classifier would perform better than a neural network, plus it would be interpretable.
train a neural network without any fancy loss function, again the rule you describe is linear and really simple, no reason to think any loss function would miss it. BUT, if you have specific reasons to think that or if you want to try to get the best of the best out of your data there is still another possibility.
train an ensambled model. More precisely, split the data yourself into 2 subsets, one with positive feature "x" and one with negative feature x, and train 2 separate models on each subset of training data. It is reasonable to think that the model trained on the subset associated with positive feature x would perform better, since the problem will turn from multi class classification to binary classification, but with pros comes cons as well, specifically higher risk of over fitting or class unbalance depending on the stats of your overall dataset.


text:
How do the scale of an embedding affects a downstream task?
I am currently training a neural network in a self-supervised fashion, using Contrastive Loss and I want to use that network then to fine-tune it in a classification task with a small fraction of the data with labels. I'm basing this project on the paper titled A Simple Framework for Contrastive Learning of Visual Representations
by Ting Chen et al.
My questions go oriented on a very specific thing that I think is causing me some problems. After finishing the self-supervised training I extract the embeddings of a bunch of data and I get the following. These stats are calculated using a flattening of all the embedding of all the bunch of data.
Mean: -23.090446
Std: 91.78753
Min: -710.24493
Max: 651.8682

What bugs me is that when I extract the embeddings in a ResNet, for example, I get much much lower values (the highest value in absolute terms don't usually go beyond 15), but I'm getting much larger values.
When looking for similarities with the use of these embeddings, it is something that looks like it works, but my question is that if it is something that might affect when adding another layer and using it for a classification task.
I have the feeling that something here is wrong but I can not spot exactly what it is.
Could you give me any advice on this?


text:
Are the existential dangers of AI exaggerated?
I understand some of the inherent dangers involved with AGI and advanced machine learning. While I can see some of the more low-level risks associated with AI coming to fruition (deep-fakes, biased algorithms, etc.), some of the more existential dangers seem far-fetched and lack empirical examples. As someone outside of AI development circles and without a background in comp-sci, I would like to know how realistic those working in the field of AI find these fears. Are there any convincing hypotheticals about the risks of a superintelligence?
I've read some Stuart Russel and have recently been watching Robert Miles videos on the topic. Both of these observers seem worried. Is this opinion widely held or is it seen as a bit extreme among developers?
Provably Beneficial Artificial Intelligence
by Stuart Russell
Robert Miles YouTube channel

We have had similar questions in the past. For example, this, this, this, etc. Your question is very related to (or a duplicate of) this, which has some good answers, in my view, but it focuses on "the emergence of superintelligence" rather than "existential dangers". Having said that, I think we can all agree that Stuart Russell is an expert in AI (or, at least, on some AI topics). He's the co-author of the AIMA book.


text:
How to label unsupervised data for deep learning multi-classification
I have unlabeled credit card transaction data that has the following columns:
Transaction_ID     Frequency       Amount    Fees
   192831             21            829       23
   382912             14            920       24
   483921            839           24059      87

Eventually, I'd like to build a deep learning model(e.g. LSTM) that can tell me whether a transaction(row) has a "high", "moderate", or "low" risk. However, since the data is unlabeled, I believe I need to label the data first before I feed the data into the deep learning model.
For example, transactions that have small frequency and amount values like the first two rows need to be labeled as "low (0)" while transactions that have large frequency and amount like the last row should be labeled as "high (2)". If both frequency and amount have moderate values, the row will be labeled as "moderate(1)".
I wonder if it is okay to use other machine learning techniques such as K-Means clustering to label the data before I feed the data into the deep learning model. Is it okay to use one Machine Learning algorithm (K-means) to label the data and feed the same labeled data into another Deep Learning model (LSTM)? Or is it a bad practice? For example, if the first model (K-means) is biased, will that bias(error) be carried over from the first model to the second model (LSTM)?
If it is a bad practice to use two different ML technologies, what else can I do to label the data?

"is it okay to use another machine learning technology such as K-Means clustering to label the data?"
In computer vision there's an entire branch called automatic image annotation dedicated to this topic. And after a 2 sec search online I found a tutorial that suggest precisely what you want to try. So yes, on the paper it's ok to try, the real question though should be:
Will it work?
And the unfortunate answer is: no. Or to be less harsh: only with toy datasets, not real ones.
Moreover take a step back and think about the possible outcomes of the approach and their implications. If you train a k-mean clustering on your data you have 2 possible scenarios:

it works, then you you have an unsupervised model that does the job, why bothering training a supervise model that at best will perform as good as the one you already have?
it doesn't work, you made your life harder and you're at the same point where you started.

what else can I do to label the data?
Even though it's not the answer you want to hear:
Labeling is like removing a plaster from a wounded cut, you spend hours thinking how to do it without suffering to just realize at the end that the best way is to just do it.
Answers to the most predictable counter argument, the data are too many:

Rely on online crowdsourcing platforms: it seems that you already have rules to use as an annotation guideline. But it costs money.
Label just a part of the data, train, and repeat till you get a good model. You can also leverage active learning to optimize and make the labeling process smarter.
Dig the web till you find an annotated dataset.
Move to unsupervised learning, but for the training itself. At the very least you'll rely on theories and model designed precisely for unlabelled data without having to train several models with a cascade of performance drops.

As a final note, there are several reasons to just start labeling the data yourself, the main one being that you'll have control over the quality of the annotations. Many people underestimate or don't even know about the importance of inter annotators agreement metrics like Cohen's Kappa that estimate not only the quality of a dataset but also how hard for a human the task you want to automatize is. Which is extremely relevant, cause if for humans the best agreement score is 80% instead of 100% then there's no way you'll obtain more than that from a model, due to the personal biases introduced by the annotators in the data. And by knowing that, you'll be happy about your 0.8 f-score without wasting hours thinking what you did wrong and why the model is not performing better.
Hope this will be be somehow helpful and good luck with your data!


text:
Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?
Consider the following paragraph from the chapter named Vector Calculus from the textbook titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

Central to this chapter is the concept of a function. A function $f$
is a quantity that relates two quantities to each other. In this book,
these quantities are typically inputs $x \in \mathbb{R}^D$ and targets
(function values) $f(x)$, which we assume are real-valued if not
stated otherwise. Here $\mathbb{R}^D$ is the domain of $f$, and the
function values $f(x)$ are the image/codomain of $f$.

we can notice that the textbook is taking $\mathbb{R}^D$ as the domain for objective functions. I want to know whether it is valid in general cases.
Do the objective functions that we generally use in artificial intelligence have $\mathbb{R}^D$ as the domain?
I am guessing it would not be since the loss functions are generally defined on the datasets which can also have discrete attributes and hence the objective function cannot be defined on every point in $\mathbb{R}^D$. So, I am guessing that the correct form of the bold statement from the quoted paragraph is "Here the domain of $f$ should be a subset of $\mathbb{R}^D$" if we are intended to deal with a general case. Am I correct or is there any arrangement such as defining $f$ as zero where the function is not defined?

I think, that $\mathbb{R}^{D}$ is most natural choice in practical situations since many kinds of data can be described in a this way:

Image is 2d array $H \times W$ with each pixel taking value in $\mathbb{R}^{c}$ (say, $c=3$) or a continuous subset of $\mathbb{R}^{c} = [-1, 1]$
In the sequence modeling problems one gives an embedding vector to each token in some $\mathbb{R}^{k}$. There is no reason priori to put constraints on the embedding vector (probably one may want to clip norm of these vectors)

However, for the applications in physics, or any other fields, by the virtue of the task arbitrary input in $\mathbb{R}^{D}$ may not make sense, and one can be restricted to consider functions only on a certain manifold $M$ (sphere $S^d$, hyperbolic spaces)
In applications of Quantum Machine Learning input is a vector in a Hilbert space - system of quits, whatever.
You may be interested to consult this review.


text:
What is the rigorous and formal definition for the direction pointed by a gradient?
Consider the following definition of derivative from the chapter named Vector Calculus from the test book titled Mathematics for Machine Learning by Marc Peter Deisenroth et al.

Definition 5.2 (Derivative). More formally, for $h>0$ the derivative
of $f$ derivative at $x$ is defined as the limit
$$\dfrac{df}{dx} := \lim\limits_{h \rightarrow 0}^{} \dfrac{f(x + h) − f(x)}{h}$$
The derivative of $f$ points in the direction of the steepest ascent of $f$.

You can observe that the derivate of a function is another function. If we consider derivative at a single point then it will be a real number that quantifies the rate of change of the output of the function with respect to the input.
There are two kinds of directions we need to focus on that are related to gradients. One is the direction pointed by a gradient and another one is the direction for moving our input parameters using a gradient. This question is restricted to the direction of the first kind.
We can treat the sign of the derivative at a particular point as the direction to move our input parameters. And I am not sure about the rigorous definition for the direction pointed by a derivative. I have thus doubts about the direction pointed by a gradient.
What exactly is the direction pointed by a gradient? and I want to know the formal definition for the direction of a gradient
I know about the direction that is given by gradient to move our parameters. But, I am not sure about the rigorous definition for the direction of a gradient vector.

I am having trouble understanding what the sticking point is here, since you are demonstrating a reasonably advanced understanding of maths used in ML (elsewhere, as well as here). How much do you already know about vectors? Are you aware that a vector can be described by a direction and a maginitude? Your 1D example is a standard simplified version of that, applied to a 1D gradient. What is it, about vectors, or about gradients specifically, that is blocking you from understanding the same concept when handling gradients?

@NeilSlater I am exactly asking for a rigorous definition for the direction that we obtain from a gradient vector. Is it an angle wrt some axis? Is it another (unit) vector? Is it some other quantity? Is it a result of the vector addition of directions of wrt several axes?.....

@NeilSlater Please tell me in case of any ambiguity.

AFAIK, both the angles wrt axes, and unit normalised vector are valid and equivalent measures of a vector's direction, and apply to gradients (with no difference in this concept because the vector being described is a gradient). However, the request for formalism means I cannot really answer. A truly formal and robust definition may be more involved.

@NeilSlater Ha. I am updating the question for more clarity. Thanks for the suggestions.

If $u$ is a vector, the direction pointed by the vector is defined as $\dfrac{u}{\lVert {u}\rVert}$ where $\lVert \cdot \rVert$ is the 2 norm (euclidean norm).

So, it may be a good idea to emphasize 2 things. 1. The gradient vector is not special. If a vector of $n$ real numbers, then it's a vector in $n$-d space and you cannot really visualize it if $n > 3$. 2. $\frac{u}{|u|}$ is the unit vector that has the same direction as $u$. Why do you compute the unit vector? Because you can have vectors of different magnitude that point in the same direction, and they are not the same vector because they have a different magnitude: e.g, in physics, you can have 2 vectors that represent 2 forces in the same direction but different magnitude.

I think the definition for direction is becoming circular here. So, the direction pointed by a vector is the same as the direction pointed by the unit vector of that vector. But, what is meant by "direction pointed by a vector" mathematically?

@hanugm A vector is defined by a magnitude and direction. So, a vector cannot just be a line (i.e. without a direction) or a direction (i.e. without a magnitude). The definition may look circular indeed, because you use a vector to define the direction of another vector, but we can view this unit vector as representing the directions of infinitely many other vectors (if $u$ is a unit vector, then $c u$ all have the same direction, for all $c \in \mathbb{R}$), so this unit vector is not specific to the original vector you want to find the direction of.

You could also view the direction as the angle that the vector makes with the axes. In that way, you might think that you don't need to define it with respect to a unit vector, but you would still need to define it with respect to a basis vector, so a vector. I'm not sure if there's any other "better" way to represent the direction of a vector.


text:
How will MLOps and lifelong learning be complementary?
According to [1], in MLOps, continuous training is

a new property, unique to ML systems, that's concerned with automatically retraining and serving the models.

While lifelong/incremental learning mainly studies how to incrementally learn rather than retrain. [2]

Lifelong Machine Learning or Lifelong Learning (LL) is an advanced machine learning (ML) paradigm that learns continuously, accumulates the knowledge learned in the past, and uses/adapts it to help future learning and problem-solving.

I can see some links or conflicts between the two but cannot explicitly explain, and I asked an author in the second link about this issue and he said that the two are complementary. I wonder how will the two help each other? Or will this kill that?

Lifelong learning and MLOps are indeed complementary.
Lifelong learning (LL) can be defined as the set of learning algorithms and models that can deal with more and more data and/or tasks without forgetting (completely) the previous one and (usually) without fully retraining the model with all data that you have available now. So, in LL, we attempt to mimic the way humans continually learn (different tasks) throughout their lives and transfer knowledge/skills acquired in one task to other tasks (e.g. from the task of walking to the task of running) [1].
MLOps is the application of DevOps to the machine learning or data science context. In other words, it is more a software engineering practice, where you define a pipeline (or sequence) of tasks that need to be done (semi-)automatically until and after you deploy your software. So, MLOps is just DevOps, but, in addition to the common practices in DevOps, like continuous integration and continuous deployment, you also have ways to deal with the ML part of the software, like automatic retraining as more data is available, and automatic evaluation of the (new) trained model on a test dataset.
They are complementary, because MLOps is concerned with the automatization of the data science process (i.e. collection and cleansing of data, training of the model with the new data, evaluation, deployment of the new trained and evaluated model to a production server), while LL is (only/usually) concerned with the development of learning algorithms and models that can cope with new tasks and data (without being completely reconfigured or retrained), so it has nothing or little to do with continuous integration and continuous deployment. In MLOps, given that you may (automatically) retrain the model as more data is available, you may think that this is a form of lifelong learning, but you completely retrain the model and you may not be able to cope with different tasks, but you could indeed use LL techniques in MLOps.

I read Chip's book and wrote an answer. Could you please help have a check? Thanks.

To combat data distribution shifts, especially sudden shifts, or to adapt to rare events, a deployed model should be retrained continually.
Incremental learning is one of the two manners of how a model is retrained(continual learning): 1) stateless retraining; and 2) stateful retraining(fine-tuning or incremental learning), both of which are largely an infrastructural problem, that is, MLOps.
Most companies do stateless retraining—the model is trained from scratch each time.
As of stateful retraining:

As of today, stateful training is mostly applied for data iteration, as changing your model architecture or adding a new feature still requires training the resulting model from scratch. There has been research showing that it might be possible to bypass training from scratch for model iteration by using techniques such as knowledge transfer (Google, 2015) and model surgery (OpenAI, 2019). According to OpenAI, “Surgery transfers trained weights from one network to another after a selection process to determine which sections of the model are unchanged and which must be re- initialized.” Several large research labs have experimented with this;
however, I’m not aware of any clear results in the industry.

Since modeling is only one part of the MLOps pipeline:

Incremental learning also heavily relied on MLOps, such as for data labeling, evaluation, etc.
References:

Designing Machine Learning Systems
CRISP-DM


text:
What are the different possible usages of the word "i.i.d" in machine learning?
The acronym "iid" stands for "independent and identically distributed". It is a property of a sequence of random variables. You can read here for more details. This question is just about the usage of the word "iid" in contemporary machine learning and is not about the feasibility of checking iid based on either associated joint distribution or dataset.
In the formal and strict sense, the word "iid" should be used only as a property for a sequence of random variables based on the underlying joint probability distribution function. But, I noticed that there is another (maybe less-strict) usage for the word 'iid' based on the context.
Consider the following statements compiled from different answers to my questions 1,2
From this answer

The term i.i.d. is a property of a dataset. A dataset can be created
that is i.i.d. with respect to a particular probability distribution.
It doesn't matter what that distribution is, it just has to exist, and
be relevant to the purpose the ML is being put to.

From this answer

The point is even you know the distribution, sometimes you can't prove
that the sampled data is i.i.d. or not!....

From this answer

....A table of results of dice throws is likely iid...... (there are some issues with this answer, but the bolded excerpt is true)

So, the usage of the word iid, in this sense, is somewhat different. Although I think, iid is a property of a sequence of random variables in this sense also, it is okay to use the word 'iid' for a dataset (collection of samples) since the dataset represents some underlying probability distribution.
Thus, the two usages I am aware of up to now are

iid for a sequence of random variables based on joint distribution.

iid for a sequence of random variables based on the collection of samples.

Is my understanding of the two usages of the word "iid" correct? and are there any other usages for the word "iid"?

@NeilSlater what about the bolded portion?

"[A] table of results of dice throws is likely iid" seems correct to me (with reasonable assumptions). However, the non-bolded "it is because the dice roll itself is iid" is not correct

Why do you get hung up on definitions of words so much? The important thing is the concept, not the association between words and concepts. Different people may use the same word with slightly different definitions, and that's fine.

@user253751 It may be due to self-study. I am trying to understand underlying concepts. Unfortunately, these terminological issues are always popping up for me.

consider that a sample is a random variable, in a way .When you sample random variable X 5 times it is the same as sampling from 5 random variables whose distribution is identical to the distribution of X and which are independent.

Oh! seems to be intuitive. I will think on it... Then, it may be similar to unbiased sampling. @user253751


text:
Discard irrelavant states from a MDP
I came across this question about MDP.
From the look of it, it seems the full MDP is reducible if the discarded state only have 1 way in and out but is it really so if we change the discounted factor? I think there is some tricky part regarding this problem...

What is the question? The question from the book?

Can you also clarify where you took this screenshot from?

In terms of meaningful decisions that an agent might make, then states $S_1$ and $S_2$ do seem redundant.
However, if you remove those states and the actions from them, you also remove the time step tracking that goes along with them, and this fundamentally affects the MDP.
The question gives some hints about what will be affected at the end to help you answer it. This is a non-episodic (continuing) MDP without a terminal state, so you cannot use discount factor $\gamma = 1$, because the expected return will be infinite. Any valid approach for value functions in continuing MDPs - using a discount factor, finite horizon or average reward - will give you different results in the reduced MDP due to the difference in counting time steps.


text:
Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?
Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled GENERATING MULTIPLE OBJECTS AT SPATIALLY DISTINCT LOCATIONS by Tobias Hinz et al.

In the global pathway of the generator we first obtain the layout
encoding. For this we create a tensor of shape (10, 16, 16) (CLEVR:
(13, 16, 16)) that contains the one-hot labels at the location of the
bounding boxes and is zero everywhere else. We then apply three
convolutional layers, each followed by batch normalization and a leaky
ReLU activation. We reshape the output to shape (1, 64) and
concatenate it with the noise tensor of shape (1, 100) (sampled from a
random normal distribution) to form a tensor of shape (1, 164). This
tensor is then fed into a dense layer, followed by batch normalization
and a ReLU activation and the output is reshaped to (−1, 4, 4). We
then apply two upsampling blocks to obtain a tensor of shape (−1, 16,
16).

The paragraph is saying that a tensor of shape (1, 164) is reshaped to (-1, 4, 4). What is  the reason behind using negative number -1? Is it representing axis? Can't we represent it with $a \times x \times y$, where $a, x, y$  are natural number s and dimensions of the tensor?
$\dfrac{164}{4 \times 4}$ is not a natural number, so what is the shape of the reshaped tensor using only the natural numbers?

This seems to be just a programming question, so I will close it, unless you clarify how this is not just a programming question.

@nbro my primary question is what -1 represents: dimension or axis.

In my view, this is still a programming question, because -1 may not exist in all libraries, and its meaning may depend on the specific library. The other linked question that I answered could also be considered a programming question for the same reason. I would avoid asking these questions here, if you don't want them to be closed. I will leave this open for now, but to me this seems to be off-topic.

@nbro But, authors started using them in their research papers. What would you say about this? i mean, what is your opinion on it?

In Tensorflow, -1 means to calculate automatically. Only one dimension can be -1

The 164 tensor is not reshaped to (-1,4,4). It is fed to a dense layer and the output of the dense layer is reshaped to (-1,4,4)

@hanugm I didn't read the paper, but they are probably assuming that the user is familiar with the most common libraries for deep learning, i.e. TensorFlow, Keras, and PyTorch, where this notation/implementation is used (not sure in PyTorch, as I've not used it for a long time, but it should be the same). I've just quickly checked and they mention PyTorch or a link to some implementation there, so they are probably using this notation because they assume people that will read this paper are or should be familiar with PyTorch.

It definitely is. If you check the code (line 145) you'll see that in the forward definition of the Stage1 Generator they do:
class STAGE1_G(nn.Module):
    def __init__(self):
...
    def forward(self, text_embedding, noise):
    c_code, mu, logvar = self.ca_net(text_embedding)
    z_c_code = torch.cat((noise, c_code), 1)
    h_code = self.fc(z_c_code)

    h_code = h_code.view(-1, self.gf_dim, 4, 4)

Where self.gf_dim is a parameter defined in the configuration file of the gan (most likely number of feature maps, but check cause they didn't write documentation about the config settings)


text:
How to understand the common practices followed for writing a "bounding box" for an image in datasets?
For the image datasets, there may be a bounding box for each image at the dataset. It is an annotation for an image. It is a rectangular box intended for focusing on something inside the image.
I read about the following two types of representations for a bounding box.

using two points $(x_1, y_1)$ and $(x_2, y_2)$.

$$<x_1><y_1><x_2><y_2>$$

Using a point $(x_1, y_1)$, width, and height.
$$<x_1><y_1><width><height>$$

How do understand both the representations, Specifically, does the point $(x_1, y_1)$ used to denote the top right or top left or bottom right or bottom left in both cases?

In case 2 it could also be the centre, ie the box starts at (x - (w/2)) and end at (x+(w/2)) etc.

With Bounding box people usually mean literally the box which bounds something. So, more intuitive way to use the points in the first case as bottom left and top right points and in the second case bottom left point and sizes. This corresponds to the logic of points on the Cartesian grid.
Of course, you may reverse sometimes against Y axis in case your task requires so. For example, some software has X origin at top left corner of the monitor. In this case, sometimes y subscripts can reverse.

$(x_1, y_1)$ or $(x_{min}, y_{min})$ generally refers to the coordinate of the top-left of the bounding box, as the x-coordinate goes from left to right and the y-coordinate goes from top to bottom in computer-vision.
However, note that the bounding box coordinate labeling may be different depending on what convention the dataset/model is following.
For pascal_voc convention, this is $(x_{min}, y_{min}, x_{max}, y_{max})$
For coco convention, this is $(x_{min}, y_{min}, \text{width}, \text{height})$
For yolo, this is $(x_{center}, y_{center}, \text{width}, \text{height})$ normalized
You may refer to the below link for more details
https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/


text:
What does it mean by "low-level" and "high-level" in features generated by CNN?
Across the literature, the terms "high-level" and "low-level" are generally used as an adjective to the features generated by the convolution neural network as intermediate representations.
Should I understand the level to be either high or low based on

the position of feature maps considering the architecture of the convolutional neural network i.e., the size of the feature maps generated  at different layers of the convolutional neural network

(For example, the feature maps after bottleneck layers are "high-level" and are after the wide layers are "low-level" )
or

the content of the feature maps based on the task

(For example, the feature maps that learned eyebrows of a cat are "high-level" and are just containing pixel intensities are "low-level" )
or

any other property?

Like every question about the use of a word: please link to the paper where you read the word.

@user253751 I think it is widely known one.

consider that "layer" and "level" can have similar meanings in English. Is it possible the high-level features are just the features from the high levels (layers)?

@user253751 I don't think layer, level are ambiguous here. Level is entirely different from layer. Layer is particular to neural networks and I never heard layer as level.

as a comment: its just a saying, and there is no real rigour definition. it is used to denote that, for example, certain "neurons" in CNNs activate higher (and seemingly uniquely) when showed certain edges, and certain "neurons" in CNNs activate higher (and seemingly uniquely) to more "difficult concepts" like faces. there is no rigour definition because we are not certain how this works


text:
Why identity function is generally treated as an activation function?
It is known that the primary purpose of activation functions, used in neural networks, is to introduce non-linearity.
Then how can the linear activation function, especially the identity function, be treated as an activation function?
Are there any special applications/advantages in using an identity function as I cannot see any such use theoretically?

The identity function can be useful in some cases. For example, if you are doing regression, the output of your neural network needs to be a real (or floating-point) number, so you use the identity function. (If you were doing logistic regression or classification, that wouldn't probably be the case). The identity function is also used in the residual networks (see figure 1). There are probably other examples of its usage and usefulness.
Some people may not consider the identity an activation function because it does nothing to the input. However, whether you consider the identity an activation function or not is a matter of convention, in the same way that considering a model with only identity (aka linear) functions a real neural network or a perceptron a neural network is a matter of convention.
I don't think there's still a consensus on this subject. In fact, you will often hear (or see) people say (or write) that they use "no activation (function)" or "linear activation" rather than saying that they use the "identity" (for example, see the documentation for the parameter activation here).


text:
How to understand the results of a generator that switches, for metric evaluation?
I am running a code on generative adversarial networks. The code is designed in such a way that it outputs a fake image after every 5 epochs. The total number of epochs is 800 in number.
After the completion of the program, when I check the images generated by the generator of the generative adversarial network while training, I am so confused about the results.
The phenomenon is as follows:

Image after epoch n is very realistic, while the images n -5 and n + 5 are
not highly realistic. I can see many such n's. And vice-versa sometimes.

Although I am interested to know why it happens. My question here is not directly about why it happens. My doubt is about the decision regarding the generator on which I need to evaluate some metrics.
It is a general practice to evaluate the metric on the generator after the last epoch, that is, the 800th generator. But if the phenomena I told holds, then it may be true that the last generator is not capable of generating realistic images, and the generator 795 or the generator 805 may be good.
So, if I am correct, I need to check the fake sample generated by the generator and then apply my metric to the generator which generated high quality and realistic image. Am I correct?

You are generating random images, right? Perhaps the images it generates are sometimes realistic and sometimes unrealistic.

@user253751 So, you are saying that it is a normal phenomenon?

Well, that depends on what you count as normal. Have you tried generating several images at the same time, to see whether all of them are unrealistic, or just some of them?

@user253751 Generating several images. Every of the following happens: All are good, All are bad, Some are good.


text:
What is the role of the word sampling in upsampling and downsampling?
Upsampling and downsampling are highly used in deep learning algorithms that involve convolutional neural networks.
Upsampling increases the size downsampling decreases the size of tensors.
What is the role of the word sampling in the words upsampling and downsampling?
Does it always have a connection with the sampling techniques that are generally used in statistics? Or is it true that sometimes it does not have any such connection and the only task of upsampling and downsampling is to increase and decrease respectively the size in any way?

This is the type of question where providing a few examples of the usage of these words that you have found could be useful. Are you sure that "upsampling" and "downsampling" are widely used in deep learning? Pooling is sometimes called downsampling. So, I suppose that these words occurred in the context of convolutional neural networks, so you may want to use that tag for this post.

Having said that, pooling, indeed can be thought of a way of sampling, i.e. selecting one out of something, but often you do it deterministically (e.g. you take the maximum out of 4 numbers). So, I am not sure if it's really "sampling" if "sampling" implies that it's a stochastic selection, but maybe the definition of "sampling" does not imply that it's a stochastic process. Another thing to consider is if the word "sampling" should only be used when we are talking about the input and output data rather than other parts of the models.

I think that this terminology originates from digital signal processing:

https://en.wikipedia.org/wiki/Upsampling
https://en.wikipedia.org/wiki/Downsampling_(signal_processing)
https://en.wikipedia.org/wiki/Sample-rate_conversion

Given a signal at a given frequency, one would like to get an approximation of the signal, which would be obtained by sampling with a frequency $n$ times higher ($n$ times smaller) than for the original signal. The resulting signal should be close to the original in a certain sense - in the Fourier domain, for instance.


text:
What is meant by "shorter connections" in the case of deep convolutional neural networks?
Consider the following two excerpts from the research paper titled Densely Connected Convolutional Networks by Gao Huang et al.
#1: From abstract

Recent work has shown that convolutional networks can be substantially
deeper, more accurate, and efficient to train if they contain shorter
connections between layers close to the input and those close to the
output.

#2: From discussion

One explanation for the improved accuracy of dense convolutional
networks may be that individual layers receive additional supervision
from the loss function through the shorter connections. One can
interpret DenseNets to perform a kind of “deep supervision”.

Both excerpts mention the type of connections called shorter connections, especially to the layers that are close to the input and the output layers of the deep convolutional neural network. What does it mean by shorter connections here?

I think this is one of those vague terminologies used in the context of skip (long-range connections). In a standard feedforward network, say if the information needs to propagate from layer 1 to 4, it has to go through two intermediate layers, namely $1 \to 2\to 3\to 4$. In a densely connected convolution network, every layer is connected to every layer downstream, meaning that you have a direct connection from layer 1 to 4, or $1\to 4$, hence a "shorter connection".
Note you also have other "shorter connections" such as $1\to 3\to 4$ and $1 \to 2 \to 4$

Shorter connections are nothing but concatenating output tensors along the channel axis. Having shorter connections between layers, will help in gradient flow throughout the network easily and hence better training.


text:
In layman terms, what does "attention" do in a transformer?
I heard from many people about the paper titled Attention Is All You Need by Ashish Vaswani et al.
What actually does the "attention" do in simple terms? Is it a function, property, or some other thing?

Let's start by stressing out that in the literature unfortunately the term attention is still used widely without any precise consensus around the technical details, the only constant across papers is that attention should be used when a model is capable of learning, or focusing on local vs global patterns in the data we use for training. And with "should be used" I simply refer to the fact that everyone like to feel eligible to write "Hey, we used attention!" simply because of the hype generated by the introduction of transformers by Vaswani et al.
Said that, I think up to this point the best expression to describe attention is:
A specific type of architecture
What do I mean by this: Vaswani et al. introduced the expression attention in the paper you cite with a new whole machine learning architecture, namely the transformers. In the paper, attention is used to refer to a specific set of layers, similarly we call residual blocks or dense blocks specific type of layers combinations that were introduced for convolutional neural networks. For me the is no difference at all between attention and the two above mentioned examples. The confusion around the use of this expression in my opinion arose from the fact that Vaswani et al. put a lot of emphasis on the final purpose of the new proposed model, i.e. capturing local similarities within sentences in machine translation.
One last consideration why I think that architecture is the best label for attention is that it include also type of attentions that are completely different from the multi-head attention module introduced by Vaswani et al, like architectures that leverage attention maps. Mathematically, attention maps and the multi-head attention module share nothing but the name, still, because conceptually they seems to fulfill the same purpose, we call them both attention, with the consequence that to avoid confusion, one should always refer to a specific paper when talking about attention.

This answer is good (I think, although I am not an expert in this topic), but there's one thing that may not be clear from your explanation. If "attention" is best described as a "layer" or "specific architecture" (submodule of a neural network), then, conceptually, what exactly does it do? (I think this was the question). You say "because conceptually they seems to fulfill the same purpose". What is the purpose then? That's one thing that is not emphasized enough in your answer.

I think that, before the transformer, attention was already gaining "attention" (sorry for the pun) within the machine translation community (see neural machine translation, for instance), from what I remember. The transformer paper certainly made the name "attention" and all related ideas even more famous, but I think it's important to note/emphasize that the idea of attention was already in the air before the transformer.

You should probably also clarify this sentence "For me the is no difference at all between attention and the two above mentioned examples" by saying that there's no difference in terms of role in a neural network (I think that's what you mean), i.e. that "attention" is just a layer or a module of a neural network, in the same way that "skip connections" are just specific types of layers or submodules.

The answer above is very concise but I will try to give an ELI5 example. I also agree with @nbro that attention does not exclusively mean transformer architecture.

Before attention
What is the height of the youngest female child of the father of your mother's first cousin? That query is convoluted, depends on your good memory of your family tree's relationships. It looks like this:
$Q_{RNN} = f(Mother(first\_cousin(father(yongest\_female(height))))\ \ (1)$
What you see in (1) is how Recurrent Neural Networks function, and it is obvious that their performance is inversely correlated with the length of the input.
After attention
A more efficient way to query the height of that person is by giving it a name - Aligna. Now, whenever you need to get characteristics of that person you don't have to remember it's "chained" relationships backwards from the person that is closest to you. This new approach would look like this:
$Q_{attentive_RNN} = f(Aligna(height))\ \ (2)$
The latter is what essentially attention pooling does. Attention establishes a direct link with all input steps (e.g. words in an input sentence) so that it can independently pay attention to the input (or set of inputs) that will generate a successful output/prediction.


text:
When is an object detection approach over a CNN approach appropriate?
I understand that CNNs are for image classification while object detection is for localization + classification of the objects detected. However, in particular, AI for chest radiographs, why is object detection used? If a CNN has 99% accuracy, should object detection still be considered? I see a lot of research papers on object detection with x-ray data but they don't explain why object detection is better than CNNs. While object detection allows users to see "where" the object is located, does this even matter if we get such high accuracy already? Also, if the location really does matter, can't we just use the heat maps from CNN?

I think you might have misunderstood 2 concepts here: CNNs and Object Detection.
Object Detection is an AI approach to solve problems where you are interested in both the location and the classification of key elements in the image. On the other hand Image Classification is another approach where you are interested in classify the whole image with a tag.
Those are very well known approaches to solve computer vision problems through AI. There lots of approaches, you select the one that outputs the information you are most interested in.
CNNs is network type. You can build Object Detectors or Image Classifiers with CNNs, but you can also build it with Transformers, with Multi Layer Perceptrons, with LSTM, even there are some approaches with Reinforcement Learning.
Going back to your problem of AI for chest radiographs, when you see 99% accuracy it is probably in Image Classification (predict probability of bone broken in the image). On the other hand Object Detection is a more informative approach because it locates the places where a bone could be broken and the probability of that bone being broken. It is more informative because the doctor now knows where to look in the chest radio image.


text:
Hand Landmark Detector Not Converging
I'm currently trying to train a custom model with TensorFlow to detect 17 landmarks/keypoints on each of 2 hands shown in an image (fingertips, first knuckles, bottom knuckles, wrist, and palm), for 34 points (and therefore 68 total values to predict for x & y). However, I cannot get the model to converge, with the output instead of being an array of points that are pretty much the same for every prediction.
I started off with a dataset that has images like this:

each annotated to have the red dots correlate to each keypoint. To expand the dataset to try to get a more robust model, I took photos of the hands with various backgrounds, angles, positions, poses, lighting conditions, reflectivity, etc, as exemplified by these further images:

I have about 3000 images created now, with the landmarks stored inside a CSV as such:

I have a train-test split of .67 train .33 test, with the images randomly selected to each. I load the images with all 3 color channels and scale both the color values & keypoint coordinates between 0 & 1.
I've tried a couple of different approaches, each involving a CNN. The first keeps the images as they are, and uses a neural network model built as such:
model = Sequential()

model.add(Conv2D(filters = 64, kernel_size = (3,3), padding = 'same', activation = 'relu', input_shape = (225,400,3)))
model.add(Conv2D(filters = 64, kernel_size = (3,3), padding = 'same', activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2,2), strides = 2))

filters_convs = [(128, 2), (256, 3), (512, 3), (512,3)]

for n_filters, n_convs in filters_convs:
  for _ in np.arange(n_convs):
    model.add(Conv2D(filters = n_filters, kernel_size = (3,3), padding = 'same', activation = 'relu'))
  model.add(MaxPooling2D(pool_size = (2,2), strides = 2))

model.add(Flatten())
model.add(Dense(128, activation="relu"))
model.add(Dense(96, activation="relu"))
model.add(Dense(72, activation="relu"))
model.add(Dense(68, activation="sigmoid"))

opt = Adam(learning_rate=.0001)
model.compile(loss="mse", optimizer=opt, metrics=['mae'])
print(model.summary())

I've modified the various hyperparameters, yet nothing seems to make any noticeable difference.
The other thing I've tried is resizing the images to fit within a 224x224x3 array to use with a VGG-16 network, as such:
vgg = VGG16(weights="imagenet", include_top=False,
    input_tensor=Input(shape=(224, 224, 3)))
vgg.trainable = False

flatten = vgg.output
flatten = Flatten()(flatten)

points = Dense(256, activation="relu")(flatten)
points = Dense(128, activation="relu")(points)
points = Dense(96, activation="relu")(points)
points = Dense(68, activation="sigmoid")(points)

model = Model(inputs=vgg.input, outputs=points)

opt = Adam(learning_rate=.0001)
model.compile(loss="mse", optimizer=opt, metrics=['mae'])
print(model.summary())

This model has similar results to the first. No matter what I seem to do, I seem to get the same results, in that my mse loss minimizes around .009, with an mae around .07, no matter how many epochs I run:

Furthermore, when I run predictions based off the model it seems that the predicted output is basically the same for every image, with only a slight variation between each. It seems the model predicts an array of coordinates that looks somewhat like what a splayed hand might, in the general areas hands might be most likely to be found. A catch-all solution to minimize deviation as opposed to a custom solution for each image. These images illustrate this, with the green being predicted points, and the red being the actual points for the left hand:

So, I was wondering what might be causing this, be it the model, the data, or both, because nothing I've tried with either modifying the model or augmenting the data seems to have done any good. I've even tried reducing the complexity to predict for one hand only, to predict a bounding box for each hand, and to predict a single keypoint, but no matter what I try, the results are pretty inaccurate.
Thus, any suggestions for what I could do to help the model converge to create more accurate & custom predictions for each image of hands it sees would be very greatly appreciated.

Unless I am missing something it looks like your network is predicting the exact same thing. Look at face landmark detection examples.

@FourierFlux Thanks for the reply! You are not missing anything. You're right on the money. That's the issue. It seems to be predicting more of an average of all the values it's trained on rather than a custom solution for each image. And the models I built are actually based on facial landmark detection models repurposed for my data, so I've already spent a lot of time looking through such examples.

Look up the pytorch one which doesn't use dense and instead stays completely convolutional. It kind of makes sense to me that your dense layers are having issues.

I see. I'm not very familiar with pytorch, so I'll have to look into that. Do you have a link to or recommendation of any particular, model, guide, or paper in particular, or should it be easy enough to find & build?


text:
What is uncentered variance and how it becomes equal to mean square in Adam?
I have been reading about Adam and AdamW (Here). The author mentioned that in "uncentered variance" we don't consider subtracting mean

In this statement, the author is talking about uncentered variance and how it becomes equal to the square of the mean.

I want to understand what exactly is uncentered variance? (because if I consider the general equation of variance $\dfrac{(\text{obs-mean})^2}{(n-1)}$ then it is not making sense to me the removing mean will lead to the definition of uncentered variance as we need a point around which we are calculating variance and here mean is that point)
Also if we are making mean=0(not subtracting mean from obs) then if I consider this as
uncentered mean (For me it is variance around 0) then it is becoming hard to understand how this will lead to uncentered $\text{variance = mean}^2$

Hello. Rather than providing a screenshot of the excerpt that you're quoting, it would be better if you can copy and paste the text directly into your post.

Thanks for the suggestion. I'll follow it next time. Just curious to know what difference text makes in comparison to screenshot?

Let me try to explain here.
Usually, we calculate the variance by subtracting the mean term and then square it. But here mean (first-moment m_t) is fluctuating like anything at each time "t" and is getting calculated with the influence of past mean as well, also with the influence of beta_1.
So when the 2nd-moment term v_t is getting calculated with a similar approach, when variance is mentioned in the Adam context is understood that we are considering the gradient mean to be equal to zero. This makes sense, since there is no reason to think positive or negative gradients would be more probable than one-another. we are only considering g_t^2 here, so this is called Uncentred variance in the context of calculating 2nd momentum for ADAM.


text:
Simple Image-based example for not utilising the variable-sized input handling capability of a Convolutional neural network
Convolutional neural networks are capable of handling inputs of varying sizes. It is one of the key benefits of convolutional neural networks. But I am unsure about the cases when we should not utilize this advantage of the convolutional neural network.
Although the following example has been provided in the chapter named Convolutional Networks of the textbook titled Deep Learning by Ian Goodfellow et al.

Convolution does not make sense if the input has variable size because
it can optionally include diﬀerent kinds of observations. For example,
if we are processing college applications, and our features consist of
both grades and standardized test scores, but not every applicant took
the standardized test, then it does not make sense to convolve the
same weights over features corresponding to the grades as well as the
features corresponding to the test scores.

It is not clear for me to understand the above example since I am habituated with images in the case of convolutional neural networks.
Can anyone provide a possible example of images where we cannot utilize the benefit of passing variable-size images to the convolutional neural network?

Well, I think, this statement sounds somehow misleading.
The main statement is actually in the first passage of this statement:

Note that the use of convolution for processing variably sized inputs
makes sense only for inputs that have variable size because they
contain varying amounts of observation of the same kind of
thing—different lengths of recordings over time, diﬀerent widths of
observations over space, and so forth.

Convolution is applicable only provided the input is the 1D, 2D, 3D array, graph, any collection of the data of the same kind. Each pixel is the feature vector of the same shape.
This statement means, that one cannot apply convolutions if the inputs are not-structured, like the data tables, which involve boolean, categorical, continuous features. There is no notion of locality and adjacency for this kind of data.


text:
Multi-armed Bandit in optimization on graph edges selection
I have the problem, which I described below. I wonder if there exists a class of multi-armed bandit approaches that is related to it.
I am working on computer networking optimization.
In the simplest scenario, we model the network as a graph with a circular node topology, similar to that seen in Chord (attached photo). Each node(vertex) can have a maximum number of $X$ active links (tunnels or edge) to other nodes at any given time. Then it can open, maintain, or close links (each operation has a cost associated with it). If there isn't a direct edge, traffic must be routed through neighboring nodes. What is the best link structure(optimal set of edges in the graph connecting nodes) in the underlying graph given the predicted traffic intensity matrix between the nodes?
Note: the optimal link structure should be recalculated on a regular basis to account for history (for example, it is worthwhile to keep a connection between two nodes open even though there is no traffic at the current time because it was generally a busy link in the past and the chance of using this link is high in future).

Estimation: Can multi-armed bandit be useful here?


text:
In mini-batch gradient descent, are the weights updated after each batch or after all the batches have gone through an epoch?
Say I have a mini-batch of size 32, and I have 10 such batches. Assuming I only run it for one epoch (just for the sake of understanding it), Will the weights be updated using the gradients of one mini-batch, or will it be done after all the 10 mini batches have passed through?
Intuitively for me, it ought to be the first one because otherwise, the only difference between Batch-GD and mini-batch GD will be the size of the batch.

In one epoch, weights will be updated 10 times for the above case.

Thanks, such a silly thing but it was bothering a lot.

What @SwaksharDeb says is correct. There are 10 batches, so 10 updates. There are 32 data points in each batch but each batch produces a single aggregate (or mean) error on the 32 data points so a single gradient is produced making a single update to the parameter values.

Are both the the weights and the gradients updated at each iteration or at each epoch?
Or at each minibatch iteration do you only calculate the loss and later add it to the other losses and calculate the weights?


text:
NEAT Speciation distance: How does one treat disabled connections?
When calculating the distance between two genomes, how does one treat disabled connections?
For example, consider the following genome:

[1, 0.2, E]
[2, 0.1, D]
[3, 0.2, E]
[4, 0.15, E]
[5, 0.3, D]

[7, 0.25, D]
[8, 0.25, E]
[9, 0.1, E]

[1, 0.2, E]
[2, 0.2, E]
[3, 0.1, E]

[6, 0.15, E]

For $\overline{W}$, the weighted sum of common genes, do I only sum the genes that are enabled in both (1 and 3), or do I sum all of the common genes (1, 2, and 3)?
For $D$, the disjoint genes, do I only count the disjoint enabled genes (4 and 6), or do I count all of them (4, 5, and 6)?
For $E$, the excess genes, do I only count the enabled genes (8 and 9), or do I count all of them (7, 8, and 9)?
And finally, for $N$, do I count all of the genes in the larger genome or just the enabled ones?
Oh, one last question. Is gene 2 now considered disjoint since it is disabled in the first genome?


text:
What are the recommended ways to change shape of feature maps channel wise other than using Convolutional neural networks?
Suppose I have a feature map with size $C_1 \times H \times W$. And I need to convert it into a feature map of size $C_2 \times H \times W$.
One way to do this is to use convolutional neural networks as Conv2d($C_1, C_2$)
I want to know whether there are any other ways in Literature to perform the desired operation?

any specific goal in mind when thinking about resizing these feature maps? from the way the question is phrased now it seems you're asking for a generic list that potentially include any matrix operation

@EdoardoGuerriero Goal is to down-sample only, but without learning parameters.


text:
Is there any subtle difference between kernel and filter in the context of neural netowrks?
Consider the following excerpt from a paragraph, taken from the topic Detecting features with convolutions of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding convolutional neural networks.

From this angle, the job of a convolutional neural network is to
estimate the kernel of a set of filter banks in successive layers
that will transform a multichannel image into another multichannel
image, where different channels correspond to different features (such
as one channel for the average, another channel for vertical edges,
and so on).

The usage of words kernel and filter bank in this excerpt confused me as i generally treat both to be the same.
What is the subtle difference between a filter and a kernel in the context of CNN?

In the context of CNNs, filter and kernel are the same thing, although I am not saying that you couldn't make a conceptual distinction between the two. I've never seen "filter" being used to refer to something that is different than the "kernel". It's just the tensor of weights/parameters. Filter banks, which I am not familiar with, are something specific, which is used in other contexts other than CNNs. The author is probably trying to make an analogy with filter banks. Without knowing the details of how "filter banks" are mathematically defined and implemented, I cannot say more than this.

As Neil says below, you could say that the kernel is used to implement the filtering operation. For example, in image processing, you could say that you use a Gaussian kernel to filter/blur an image. So, in this sense, there's a conceptual difference. However, in the context of CNNs, this difference is never made, because we usually don't think of CNNs as neural networks that filter the image, but that extract features from the image.

Keep in mind that you can also have a neural network that "filters" the image. For example, you can implement some autoencoder that deblurs/filters the image. However, you could well also say that this autoencoder uses the convolution to deblur the image and never mention the word "filter" or use the word filter to refer to the matrix of weights (i.e. kernel).

The kernel is the matrix of numbers that implements the filter. The filter is the effect  or function that is applied to the input signal.
It is possible to conceive of filters in the general sense that work other than applying convolutions, but of course those mainly exist outside of CNNs, and a complex enough CNN can approximate any filter that is implemented differently.


text:
How to visualize the input and output of the GRU cell?
GRU belongs to the family of recurrent neural networks. This family of neural networks works on sequence data.
But, it is taking time for me to understand the differences between sequence length and input in the case of a GRU cell.
In the case of a CNN, the input tensor is of the form $B \times C \times H \times W$.  Here $B$ is the batch size, $C$ is the number of channels, $H$ is the height of the image and $W$ is the width of the image.
We can visualize the input and outputs of CNN here.
Similarly, in the case of a GRU layer, the input tensor is of the form $B  \times L \times I$. Here $B$ is the batch size, $L$ is the length of the sequence and $I$ is the number of input features.
I want to know what exactly the input and sequence lengths are. If there are any visualizations, please provide them.

If we're using embeddings, $I$ might be the embedding size. If you're using the GRU for machine translation, $L$ is probably the maximum number of words that you expect in a sentence. I suppose this is the answer to your question, but I am not sure. Can you please provide a source that describes the input of the GRU unit as a tensor of shape $B \times L \times I$?

@nbro I think you missed the link from the comment.

@nbro I updated the link.


text:
What does it mean by "dynamics of a sequence" mathematically?
Consider the following paragraph from the topic named sequential models from the textbook titled Dive into Deep Learning

Both cases raise the obvious question of how to generate training
data. One typically uses historical observations to predict the next
observation given the ones up to right now. Obviously we do not expect
time to stand still. However, a common assumption is that while the
specific values of might change, at least the dynamics of the
sequence itself will not. This is reasonable, since novel dynamics
are just that, novel and thus not predictable using data that we have
so far. Statisticians call dynamics that do not change stationary.

Here sequence refers to $x_1, x_2, x_3, \cdots, x_t$. Say the stock price of  a company.
What does it mean rigorously by the dynamics of a sequence in statistics?

I don't think there is any rigorous mathematical definition of dynamic, or to phrase it better, the definition is embedded in the definition of dynamical system itself (which would be probably a better term to use in place of the generic "sequence"):

A dynamical system is formally defined as a state space $X$, a set of
time points $T$, and a rule $R$ that specifies how the state evolves with time.
The rule $R$ is a function whose domain is $X×T$ and whose
codomain is $X$, i.e., $R:X×T→X$. The rule function $R$ means
that the $R$ takes two inputs, $R=R(x,t)$, where $x∈X$ is an
initial state and $t∈T$ is a specific time step.

I think that what the paragraph states is that we assume the rule funcion R to remain the same, cause that's what we're trying to learn.
As a final note, there are out there also classifications of different non-linear dynamical systems, based on the property of $R$.


text:
Should I need to interpret the word "metric" in "performance metric" rigorously?
Consider the following abstract from the research paper titled A Note on the Inception Score for instance

Deep generative models are powerful tools that have produced
impressive results in recent years. These advances have been for the
most part empirically driven, making it essential that we use
high-quality evaluation metrics. In this paper, we provide new
insights into the Inception Score, a recently proposed and widely used
evaluation metric for generative models, and demonstrate that it fails
to provide useful guidance when comparing models. We discuss both
suboptimalities of the metric itself and issues with its application.
Finally, we call for researchers to be more systematic and careful
when evaluating and comparing generative models, as the advancement of
the field depends upon it.

Here we can observe the usage of word metric several times. In mathematics, the word metric is used only in the context of metric spaces afaik. The definition for metric space and metric is defined as follows

A metric space is a set $X$ together with a function $d$ (called a
metric or "distance function") which assigns a real number $d(x, y)$
to every  $x, y, z$ belongs $X$ satisfying the properties (or axioms):

$d(x, y) \ge 0$ and $d(x, y) = 0$ iff $x = y$,
$d(x, y) = d(y, x)$,
$d(x, y) + d(y, z) \ge d(x, z).$

Do research papers generally use the word metric in the sense of the metric defined above? Or do we need to interpret the word metric less rigorously, just as a measure, like an accuracy?
Note: Although I provided the abstract from a research paper containing the word metric, the question is not restricted to this particular context. This question can be applied to all AI-related research papers that used the word metric, especially in the context of performance or evaluation metrics.

I don't think you should interpret it rigorously. Most of the "metrics" used turn out to satisfy those constraints, but I think I've seen KL divergence-based "metrics" being called, well, metrics, while they do not satisfy property 2

@Ant That looks like the beginning of an answer. In addition to KL divergence, maybe you can provide more examples of "metrics" that are or not really metrics by showing that they satisfy or not the properties of a mathematical metric.

"Metric" should be understood as "a function of the trained model and of a dataset which returns a number".
For example, in reinforcement learning, one can use as an evaluation metric the total cumulative discounted reward that a policy gets on a set of episodes for a given task, starting from the same initial state $S_0$.
That's a function $f(\tau_1, \tau_2, ... | \pi, S_0)$, where $\tau_1$ is the first trajectory obtained by running policy $\pi$ on the task, $\tau_2$ the second one, etc. (we assume that $\pi$ is stochastic, otherwise it doesn't make sense to run multiple episodes). $f$ is an "evaluation metric" but is certainly not a distance function.


text:
Transforming Moving Position Data to an Inputvector for Neural Networks
Imagine a car is driving on a long street (= x-axis). The car can go in both directions and it will arbitrarily change its direction.
I'm trying to formulate an Inputvector to tell a neural Network which area of the street was driven on and how often that area was driven on.
What is known are the exact coordinates (x - coordinates) of the car with corresponding time stamps.
The neural network needs to be updated every minute, so I have to come up with an intelligent way how to "summarize" one minute of movement of the car.
My ideas:
I could calculate the following from the available position data with timestamps:

weighted mean of the coordinates of the car
the range (i.e. xmax - xmin)
the travelled distance within one minute

these three inputs already give the neural network a good idea on how the street was being used. However, what if the car drives two times like depicted in this picture:

Both cases would result in the same mean position 0.5, the same range 1 and the same travelled distance left: 5*1 = 5, right: 6*0.33 + 1 + 6*0.33 = 5 However, the middle part of the street was driven on less on the right than on the left. How can I differentiate between these two cases?
A different approach to this problem as a whole would be to instead of using mean + range + travelled distance, I just divide the street into 10 parts and tell the neural network which part was being used how intensely. E.G. for the case of the picture on the right where mostly the first and the last third of the axis was being used the input would be something like:
[0.15, 0.15, 0.15, 0.1, 0, 0, 0.1, 0.15, 0.15, 0.15]
However, what I don't like about this approach is that first of all it's not very precise cause I'm classifying the street into 10 parts, I would prefer to have a continuous solution. Also, It will generate too many inputs (I have to do it for 3 axes x,y,z) so I will have 30 inputs and the problem is that I will also need some other inputs for different reasons so the directional inputs will be too "heavy" altogether.
Maybe someone of you knows a different approach how to deal with that kind of problem. Or maybe I have to use some special kind of neural network architecture (something like a time delayed neural network with 60 time delayed one-second inputs instead of a single one-minute input) Help would be greatly appreciated.


text:
Are RNN a good approach to solve this type of problem?
I have a problem that can be optimized by taking five actions, and finally, after a series of steps to achieve a solution.
The actions (1 to 5) are picked randomly. A time-step (epoch) is concluded every time five actions are taken. The outcome is influenced by the sequence of actions and the time-step.
E.g Action sequence: 5,4,2,1,3 may be better for time 3 than time 9.
Time 1 Action   A1  A5  A3  A4  A2
Time 1 Outcome  0   0   5   122 6

Time 2 Action   A5  A3  A2  A1  A4
Time 2 Outcome  42  0   9   80  7

Time 3 Action   A3  A2  A1  A4  A5
Time 3 Outcome  13  43  1   0   10

...

Time N Action   A2  A3  A4  A5  A4
Time N Outcome  15  100 120 42  75

The goal is to collect as many points as possible by the Nth round.
Should I approach this problem with RNN, or is there a more suitable theory/algorithm?


text:
Is there any advantage in viewing weights of a neural network as random variables?
In artificial intelligence, especially in machine learning, the inputs and outputs of neurons in a neural network can be viewed as random variables.
And this view is highly useful in many ways. The usefulness of this view manifests in applying various mathematical results of random variables in understanding outputs and other aspects of neural networks. Other aspects include loss functions also.
I want to know whether it is useful to view weights of neural networks as random variables? What does the literature say about this view?

In Bayesian statistics, as opposed to frequentist statistics, you can model the parameters as random variables. Bayesian machine learning is the application of Bayesian statistics in the context of machine learning. The specific application of Bayesian statistics to learning in neural networks is denoted as Bayesian deep learning.
Even just in the simple example of maximum a posteriori (MAP) estimation, you model the weights as random variables. For example, you can view weight decay in neural networks as MAP estimation when you put a prior on the weights, i.e. you assume that the weights are random variables or samples from a distribution. Here you can find more details about MAP estimation and how it's different from MLE.
In MAP, we only find a point estimate of the weights. However, we could also learn a probability distribution over the weights. If you do this with neural networks, we have Bayesian neural networks. Two of the first people that worked on this were David J. C. MacKay and Radford M. Neal, but only very recently did Bayesian neural networks really become popular, with Alex Graves and Blundel et al. Anyway, there are different approaches to Bayesian learning in neural networks. There are techniques based on Monte Carlo methods (e.g. Neal) or variational inference (e.g. Graves and Blundell).
So, what are the advantages of viewing weights as random variables? Well, you can derive MAP and Bayesian neural networks. Basically, all the advantages of Bayesian statistics.

Nice answer, thanks! Do you have any practical examples of where BNNs are used nowadays, maybe even more successful than other alternatives?

@Mathy Honestly, no. It's been a while since I worked with them, so I forgot a few things. Maybe that could be a good question for our site, so I recommend that you ask it ;) Make sure to you use the tags [tag:state-of-the-art] and [tag:reference-request] and/or to provide some context (e.g. a brief description of BNNs).

Thank you for the idea; just done so ;-) https://ai.stackexchange.com/questions/36040/todays-practicality-of-bayesian-neural-networks


text:
What can I infer if my model is converging extremely fast?
I am running a model with fixed hyperparameters. To my surprise/shock, the model converged extremely fast with the least loss possible.
I want to know the causes of this phenomenon. I have the following guesses:

Underlying mapping is so simple.

Hyperparameters are apt.

Both.

Are there any other reasons for this phenomenon?

What you mean by "Hyperparameters are apt." ?

@pasabaporaqui I mean, correct hyperparameters for the model

send us your loss function plot over epochs ( or steps ). this will help to get a better guidance(use log scale for loss axis). sending more details of your learning process may help too.
but in this situation, i think you should decrease the learning rate and using the learning rate decrease method. this method helps you to see stepwise decrease of loss to the best losses. you can see more details of this method in this link.

There might be several reasons for that:

the data is easily understood by the model you are using
the model you use is fitted to the problem
the problem complexity is low

There are a lot more reasons to explain the convergence of an algorithm.

If a NN converges in a few steps to the absolute minimum of the loss function it means the loss function has a gradient (in the domain that the inputs defines) very regular, pointing to the absolute minimum.


text:
What does "statistical strength" mean in this context?
Consider the following excerpt from a paragraph taken from chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named Deep Learning by Ian Goodfellow et al regarding the advantages of RNN over full traditional MLP.

To go from multilayer networks to recurrent networks, we need to take
advantage of one of the early ideas found in machine learning and
statistical models of the 1980s: sharing parameters across diﬀerent
parts of a model. Parameter sharing makes it possible to extend and
apply the model to examples of diﬀerent forms(diﬀerent lengths, here)
and generalize across them. If we had separate parameters for each
value of the time index, we could not generalize to sequence lengths
not seen during training, nor share statistical strength across
diﬀerent sequence lengths and across diﬀerent positions in time. Such
sharing is particularly important when a speciﬁc piece of information
can occur at multiple positions within the sequence.

The authors used the phrase "statistical strength". Do they mean the strength of RNN in learning the embeddings of a word based on its context rather than its position in input, if it occurs in several inputs? Or does it mean that RNN uses fewer parameters to generalize in a better way compared to a traditional MLP? Or do they mean something else?