text:
Cover a surface with smaller predefined objects
I'm trying to make a program that takes a surface designed by the user, and different 3D geometries from a dataset as inputs and gives a good approximation of the surface using only the objects found in the dataset. This program shouldn't do any warping, and should avoid geometries to collide, even though cutting them could be acceptable, but again with as little loss as possible.
I thought about hardcoding this, but I can't find any good way to optimize the surface coverage without brute-forcing it. I'm wondering what ML techniques would be best for this, and how to find a good balance between precision and speed.

a subset of your problem is pallet-loading (you can find a survey here about it). Yet, there is not any polynomial algorithm for an even simpler case of your problem such as pallet-loading. However, different planners in automated planning contexts can be helpful and give you some heuristics to solve the problem, some planners like Fast Forward.


text:
Is there a systematic way of conducting deep learning experiments?
I have been working on a computer vision problem with the use of cnns, but quite frustratingly I'm often in the situation of not knowing what to do to improve my results. It seems to me that most of the time I am mostly making random changes and experimenting in hope that this change will bring some improvement.
I notice how this is different from non-AI software development where debugging can be performed by trying to pinpoint where exactly in the code lies an unexpected behaviour.
I wonder if there is a technique that could better orient the research effort.

There are definitely things that you can do. However, I doubt there is a top-level system which starts with "not good enough results" and works you through all possible deep learning scenarios to discover what the most likely issue is. So, I would suggest that as well as this question, you open a new question with more specific details of what is currently blocking your progress (maybe on Data Science SE, but heremay also be OK). That may produce some useful leads, even if it does not give you some kind of DL cheat sheet for troubleshooting

I guess that's why machine deep learning is a research topic and not something any 10-year-old can do reliably.

If there is really one systematic & pretty reliable way to improve results and performance without human intervention, then we, the AI-Engineers, would lose our jobs.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How can I weight each point in one-class SVM?
I want to give weights to some data points
Specifically, these are points related to anomalies
(I'm implementing one-class SVM for anomaly detection)
Exactly, I want to consider some data points that are likely to be anomalies as more important data points
Is it possible in one-class SVM ?

Hello, please keep all the necessary details in the question as it seems difficult to understand without related context.

@hanugm Hi, one-class SVM is known as a good anomaly detector (imbalanced binary classifier) However, I want to give weights to some data point in advance, so that the points can be anomalies in one-class SVM classifier

This might help

@Yahya Thk you!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Creating DQN Learning Agent without Gym environment for a custom project
In a project for college I created a simple turn based game, with up to 4 players that can either move or attack the opponents. The players are playing over the network, meaning the clients are supposed to be programmed AIs. The client itself is fully functional, meaning it has all the game logic and can simulate complete games.
Now my task is to create a RL-Agent with a Deeq-Q-Network that learns to play the game. However, I don't really find any source to how that should be done. I was able to create an Agent with a DQN for the CartPole environment of OpenAI gym with PyTorch. Now my guess would be to create my own environment with the gym framework, but since the game itself is already implemented I was thinking if it was possible to feed data in the DQN without having to create the gym environment. As a state it would get the gamestate (which is a 2d grid, with information about the players and their remaining hitpoints) and all the possible moves in the current state as the action space. And since the game is network based, it would save the network after each game and reload it when the next starts during the training. For the training I would start the games over and over with a script and let it train for a while.
As I'm quite new to Machine Learning it seems really blurry as of how to tackle this problem and was hoping to get led in some direction on how to start.

Most OpenAI gym environments are thin wrappers around existing games and libraries. You could do the same with your game. See e.g. https://towardsdatascience.com/beginners-guide-to-custom-environments-in-openai-s-gym-989371673952 for a tutorial. There are many others, you can search "open ai gym custom environment" for more

Thanks for the reply! I've read it and tried creating my environment but I'm still stuck when it comes to how to model the custom environment as it is somewhat complex.
Would it be possible to use my PyTorch DQN and prepare the data it needs on my own and calculate a reward? And not use the Gym Framework at all? Since my code already does simulate games.

You don't have to use the provided wrappers, or follow Gym's API exactly, but you take more risks with incompatibility or making a mistake. If you know what you are doing, it's fine to connect your environment direct to the DQN agent though.


text:
Is there any difference between conditional batch normalization and batch normalization except the usage of MLPs for predicting $\beta$ and $\gamma$?
Batch normalization in neural networks uses $\beta$ and $\gamma$ for scaling. The analytical formula is given by
$$\dfrac{x - \mathbb{E}[x]}{\sqrt{Var(X)}}* \gamma + \beta$$
Conditional batch normalization uses multi-layer perceptrons to calculate the values of $\gamma$ and $\beta$ instead of giving fixed values to them.
Is it only the difference between them or is there any other fundamental difference between them in terms of functionality?

Is this the paper that talks about this conditional BN you're referring to?

@nbro Yes. I think the same is found in the PyTorch option for normalization with affine = True option. But... Yeah... The same paper...


text:
Train separate AutoEncoder's on each class or one AE for all classes to learn features?
I'm working on a project where the dataset contains time series of three classes, depending on the shape of the series. I want to learn the representations of these series as vectors, so naturally I use AutoEncoder for the task (precisely, I use LSTM-AutoEncoder to better handle the sequential data).
My question is: should I train one model for all classes or one model for each class? If possible, could you also point out what are the pros and cons of each approach? One thing that worries me about the latter approach is that the AE will simply memorize the data without any learning (again, would that be a concern?)
Thank you very much in advance!
Sincerely,

Without your data this is tough to answer.  The very best thing to do is to try it both ways and see how well each solution fits your problem.

@DavidHoelzer I tested both approaches and they yield the same observation: if I perform a quick clustering on the feature vectors (either learned by separate models or by the same model), the clustering result is consistent in both approaches. I wonder if it may affect the "correctness" of the learning task, or is it just a matter of managing the pipeline?


text:
Training and Evaluating BERT and XLNET
I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.
In the project, I have been given a data set with (rather small) 30.000 text documents, which are labeled with 0 and 1. I want to train and evaluate (with respect to accuracy) a BERT and XLNet model.
Can you give me some rough estimates for the following questions?:

How much computing power do I need for this task, i.e. can I simply use my private laptop for this or do I need a special CPU/GPU for it?
So far, I just worked with classical machine learning models (e.g. random forests, SVMs, etc.). I am not experienced deep learning architectures yet. How difficult would it be to implement a BERT oder XLNet model with my own data set, having no experience with BERT oder XLNet yet? I.e. how much code would it be that I have to develop by myself? And would I need a deep understanding for it or would be sufficient to follow an online tutorial and basically copy the code from there?
Many thanks.

You’ll want a reasonable GPU (probably 8GB+), but otherwise no special hardware needed.* You may need to tune down sequence length and batch size to fit your GPU; RAM will be the limiting factor. Don’t try it on a CPU. It will “work” but you’re gonna have a bad time.
Try the Huggingface Transformers library as your implementation. It’s well documented and straightforward and includes both models.

*assuming an Nvida GPU or something compatible with CUDA. Things are rather hairier on Apple hardware. But you can always grab a cloud VM for a few hours


text:
Is seq2seq the best model when input/output sequences have fixed length?
I understand that seq2seq models are perfectly suitable when the input and/or the output have variable lengths. However, if we know exactly the input/output sequence lengths of the neural network. Is this the best approach?

If you would classify a transformer as Seq2Seq, then it is arguably the best. This is only arguably due to accuracy.
Shallow neural networks or even decision trees and forests may be better in production due to lower training time, lower inference time and smaller size in memory.
Overall, it requires a bit of a compromise, and you need to do what suits your use case. For example you wouldn't run GPT-3 on a raspberry pi pico (for example). So it depends on what you mean by "best".


text:
Why is BatchNormalization causing severe overfitting to my data?
So I've been making a mini version of VGGNet, trying to tweak the hyperparameters to match the CIFAR-100 dataset.
It was running slow at first but I was able to get decent accuracy after 60 epochs or so.
However, when I added BatchNormalization layers to my two fully-connected hidden layers, it started learning at like 20% accuracy immediately, but began overfitting my data so badly that after 7 epochs my validation didn't improve from 0.01, compared to 20+ testing accuracy.
Why would adding these layers which are supposed to act as a regularizer, actually cause severe overfitting instead? I'm confused.

How big are the batches? I had a similar symptom when I accidentally applied BN to batches of size 1. BN relies on the batches being big enough that the batch average matches the training/test/validation set average.

Oh like 256. I just read the ResNet paper. Do you think it's the degradation problem that He et Al addressed on their paper?


text:
Transforming a complex if-else decision-making to ML
I have a time series classification problem that uses a series of if-else statements to arrive at a particular label. I am attempting to use ML/DL to make the system simpler.
So far, I have tried using a tabular data approach where I take a snapshot of information up to a particular point. For example, this will be the rolling sum of certain columns and so on. I have also tried LSTM and CNN. All these approaches have failed to give me F1 scores significantly above 50 %.
Are there other ML/DL approaches that I should try before giving up? The models were built using AutoKeras and PyCaret.

Are you sure, that there is a need for ML? If there is a set of rules that allows to solve this problem without ML, it gives already 100% accuracy, whereas ML/DL will do it up to a certain accuracy, and this task may be even tough for supervised algorithm. Sorting problem is very difficult for neural networks.
Anyway, check whether your algorithm can express these if-else statements. For example such branch, given that the output is of the same size for both conditional branches, can be approximated by the following:
sigmoid(beta * condition) * output_1 + sigmoid(-beta * condition) * output_2

Here beta controls the slope of the sigmoid, condition is the conditional statement of the form $f(x) >(\geqslant) 0$, and output_1 and output_2 are the outcomes of both options.


text:
How can equivariance to translation be a benefit of a CNN?
I just learnt about the properties of equivariance and invariance to translation and other transformations.
Being invariant to translation is clearly an advantage, as even if the input gets shifted, the network will still learn the same features, and work fine.
But how is equivariance useful?

I think your question is mostly about what the difference between invariance and equivariance is. If invariance is clearly an advantage, imo equivariance is also clear. Can you explain what isn't clear to you?
Invariance means output does NOT change with inputs. Equivariance means output changes equivalently with the input. Depending on your application (image classification versus image segmentation) you will want one or the other.

Equivariance is useful because the neural network can learn to detect common image components - edges, corners, curves in specific orientations - in a general way that is then applied across a whole image evenly. These components typically do exist and can appear in multiple places within an image, and may be parts of larger-scale features in turn. Identifying all the edges in an image can be useful before any kind of pooling that adds invariance is applied.
Without equivariance, an edge oriented in one way in one part of the image would be completely different to the neural network to the same kind of edge elsewhere. It would only get made part of a relevant filter and used if a specific training example had an important edge in that one place.
It is hard to separate this usefulness of equivariance from the associated reduction in number of free parameters, thanks to using convolutional filters with small amount of local connection as opposed to fully connected neural networks. CNNs have to be equivariant due to the architecture, whilst the invariance requires a little more effort (pooling and/or strided convolutions).
The fact that CNNs are so successful using this approach probably says something about natural images. It should be possible to construct non-natural images where equivariance would be of limited use e.g. where local features don't exist, or where they vary over the image in a way that makes detecting them in more than a few places pointless.

As explained here, both properties are useful depending on your application and expected result.

For an image classifier, you'll expect a invariance (in-variance = not change) result, meaning all results are the same, no matter how you translate the image.
For an image segmentation, or an object detector, on the other hand, you'll expect the output to shift together as the input varies. In other words, an equivariance (equi-variance = same change) is expected.


text:
What is the difference between “AI Methods” and “AI Techniques”?
These are words that we frequently come upon. What can be said about the differences? Would these two words' subheadings be different?

To answer this question, we need more context. Where did you find these terms? It's possible that they are synonymous, but it's also possible that they are not being used interchangeably. In other words, I don't think that this is standard terminology.


text:
Why not undefined expression is different from numerical underflow?
Consider an architecture or programming language that uses $n$ bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a given range, say $[lf, uf]$.
If there is a need to store any floating point number less than $lf$ then we generally treat such phenomenon as underflow. Consider the following from Numerical Computation chapter of Deep Learning book.

One form of rounding error that is particularly devastating is
underﬂow . Underﬂow occurs when numbers near zero are rounded to zero.
Many functions behave qualitatively diﬀerently when their argument is
zero rather than a small positive number. For example, we usually want
to avoid division by zero (some software environments will raise
exceptions when this occurs, others will return a result with a
placeholder not-a-number value) or taking the logarithm of zero
(this is usually treated as $-\infty$, which then becomes not-a-number
if it is used for many further arithmetic operations).

You can observe that two examples has been given while explaining underflow: division by zero and logarithm of zero. If we treat mathematically, both are undefined. It should not be an issue of storage, especially underflow.
Is there any reason behind proving such examples, which are mathematically undefined, under the umbrella term underflow and using the term "not-a-number"?

Yes, they can be related to underflow. Mathematically, we do not expect facing with division by zero when we have an expression $\frac{1}{\varepsilon}$ when $\varepsilon > 0$. However, in numerical softwares, it can happen if $\varepsilon < lf$.
Moreover, you can think about the same scenario for taking the logarithm of zero.


text:
What does a value of -1.000 mean in MS COCO Metrics for Object Detection
I am training some Object-Detection-Models from the TensorFlow Object Detection API and got from the evaluation with MS COCO metrics the following results for Average Precision:
IoU = 0.5;0.9
maxDets = 100
area = small
AP = -1.000
The other values all make sense to me. But I don´t know what the -1.000 stands for. Does it mean that there are no small objects in my dataset to be detected?

APsmall stands for Average Precision for small objects.

This website contains all the descriptions about the metrics:
https://cocodataset.org/#detection-eval
If you want to know more about the metrics evaluation (how to calculate this metric), you can find the metric's source code here: https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py

Does it mean that there are no small objects in my dataset to be detected?

Yes.
From the evaluation source code as mentioned by the other answer, L334:

-1 for the precision of absent categories

From the coco dataset website, Detection Evaluation:

$AP_{small}$: AP for small objects: area < $32^{2}$
$AP_{medium}$: AP for medium objects: $32^{2}$ < area < $96^{2}$
$AP_{large}$: AP for large objects: area > $96^{2}$


text:
Types of decoder parametrizations in VAE for continuous data
I'm wondering what are the different choices of parametrizations available for the decoder in a variational autoencoder. If the data is discrete, you can just output probabilities for each class, so that's straightforward. If the data is a) continuous and not bounded, then the only parametrization I know of is the (multivariate) normal distribution. If the data is b) continuous and bounded, again the only parametrization I know of is a (multivariate) normal distribution, mapped to [0,1] using a sigmoid function.
Are there other formulations that are commonly used? If there are, what are the advantages and any disadvantages compared to using a normal distribution?
To ask the question in a different way: the normal distribution seems like the "go to" choice in problems with continuous output since it's simple and allows gradients through the reparametrization trick. Are there other reparametrizable continuous distributions which are also commonly used in machine learning? I know the exponential distribution (and related distributions such as weibull, gamma) are reparametrizable, but would you ever use those instead of a normal distribution to model continuous data?

In first paragraph, you are asking about the data distribution "for the decoder", i.e. p(x|z) while in the last paragraph, you are asking a very different question about p(z|x) and its reparameterizability. Kindly clarify which one you want to know, or you want both ?

The question is about when (if ever) you would ever use something other than a normal distribution to model continuous data.


text:
Is there any way to train a neural network without using gradients?
The only algorithm I know for updation of weights of a neural network is based on gradients. The update equation can be roughly written as
$$w \leftarrow w - \nabla_{w}L$$
where $\nabla_{w}L$ is the gradient of loss function with respect to weights.
Are there any learning algorithms for updating weights in neural networks that does not use gradients?

Yes.
A prominent class of "gradient-free" algorithms in ML world is known as Evolution Strategies (ES). Evolutionary Algorithms, although existed for a long time, only a few have shown to scale well.
Recently, the research group OpenAI managed to train Deep RL models with a specific variant of ES (with careful engineering). You can read this paper. This blog by David Ha provides a starting point if you want to learn about ES and its modern derivatives.


text:
Predict a part of the input based of the output
I'm working on a fun project where I have a dataset of input and output data, both having a fixed size of characters.
I would like to predict a part of the input based on a known output as follows:
$$Input = A+B$$
$$Output = X+Y$$
A, B, X, Y are strings that will be concatenated and they have a fixed size
Knowing A, X and Y; I want to predict B (even if it takes a lot of tries). The output is split in 2 because I don't really care what value Y has (so maybe I can delete it, depending how is more easy).
Is it possible? I'm new to ML and AI and first I want to know if it possible before starting to work on the project (I am time limited). And if it is possible then could you tell me what exactly to study/learn or how I can do it?

could you give us a more concrete example of these inputs/outputs? It would help to understand better the whole set up you described in the question. Also, if you're trying to predict something, that's the output by definition, and what you use to predict it is the input, they can be partially or literally the same thing, but "predicting an input" is not really an expression that makes sense. Not a criticism, but they way you put it make things a little bit confusing.

Yes, I can see how's confusing... I guess I can swap the input and output then. An example would be: "aaaaa" = "abcde" + predict_value (that would be X = A + B). So now I'll need to predict a part of the output.


text:
The results are not correct when predicting the future for a very long period of time with LSTM
I am currently using LSTM to try to predict future data in AirPassengers.csv.
This is current code op my Colab (sorry for the comments are Japanese)
https://colab.research.google.com/drive/16Ntg3dA5ywZvm35PEeMUjHtKhNdsSgbc?usp=sharing
I wanted to use this code to make predictions for a much longer period of time with different data in the future, so I changed the prediction period in the last code block from the original code I referenced from 3 years to 20 years as follows:
#from
pred_time_length = 12*3
#to
pred_time_length = 12*20

When I do this, I get values like this damped oscillation, and I think that the prediction is probably not working well.
What is the cause of this? Also, what should I change in the code to make it work?

thank you in advance!


text:
How to classify two very similar images using Deep Learning?
I am a newbie in Computer Vision.
I have a scenario in which I have a stationary camera in a factory. I want to detect whether the technician is working on the machine or not.
Images are like the following:
Technician working:

Technician absent:

Technician not working:

I am confused whether is it a Image classification issue or an Object Detection/Pose Detection problem.
As per my knowledge this should be a classification problem, I should take multiple images of a condition in which the machine is unattended, and a condition in which the technician is working on the machine.
I will train the model if with different individual technicians on different days with different clothes.
Now if I am in the right direction, how much images do I need to have a good accuracy?
I see there are different models on Tensorflow Hub on image classification like EfficientNet, etc.
Which model/architecture will work for me?
I am sorry if I sound noobish.
I can train the model using simple classifiers' code (like Cat vs Dog), but I want the my architecture to understand that there is an area in the image which should only be checked if it is occupied or not to classify properly.
OR
Shall I cut out the middle area (where technician stands) simply using opencv. And then feed that cutout image to some classifier to detect if there is a human standing there?
Thanks in advance!

This is a bit old question, but I'll answer anyway. Naturally the more you have data the better, but rather than capturing an image (for example) every second, I would rather capture an image every 5 minutes for a period of at least 30 days. This would give you 8 * 60 / 5 = 96 pictures per work day. But naturally you can start developing the algorithm at day one.
Generating test & validation sets on "time series" data can be tricky, I would assign complete work days to the validation set rather than taking random samples. This way you can test that the code works even when people are wearing different clothing, and the weather could be different as well.
For supervised learning you must manyally classify all of the images, since there aren't that many images I would go this router rather using a more difficult semisupervised learning. As a human you have some domain expertice on the image content, I would crop the image quite tightly around the area of interest so the network doesn't need to learn to ignore certain areas by itself.
You could have some edge-cases to classify, for example if a technician is just standing in front of the machine, is he considered working? How about if the is facing the other way? Depending on how you want to handle these, your network can either be very simple or a bit more advanced.
Maybe you don't need a neural network at all, and you can just pass a low-res image through LDA / Fisher Linear Analysis. Or create pixel-wise histograms of the colors for working and non-working image classes separately, and create heuristics based on that.
Once you have images from different days, pay attention on what kind of variance there is in colors, brightness, camera noise etc. Then create a data augmentation pipeline which mimics the observed deviations, this way you can make the model much more robust and you don't need to wait several months to start getting good results.

What you are asking about can be treated as a classification problem, indeed. But I would treat it rather as a detection problem.
Given an image, the goal is to draw the bounding box or any other geometric shape around the object of interest. In other words, you would like to localize and identify the object.

I recommend reading this link https://machinelearningmastery.com/object-recognition-with-deep-learning/.
Good repository for detection - https://github.com/ultralytics/yolov5.


text:
How is Google Translate able to translate texts of arbitrarily large length?
Sequence-to-sequence models with attention are known to be limited by a maximum sequence length. So how can we handle sequences of arbitrarily large size? Do we just set a very large maximum sequence length?

You simply split the sequence into smaller sequences; while there are some long-distance dependencies in language, that is generally not a problem for this.
A sentence would typically be short enough, and very long sentences are composed of shorter clauses which would form independent units (albeit connected with each other).

How would this "splitting" be done in practice? How are these clauses found? Would this require another model or is it just a matter of defining a set of segmentation rules?

I don't know, but I'd guess it would just be by length. I doubt that there would much linguistic knowledge behind it, as it would generally be language independent. Maybe on commas or other punctuation marks.


text:
What does it mean "having Lipschitz continuous derivatives"?
We can enforce some constraints on functions used in deep learning in order to guarantee optimizations. You can find it in Numerical Computation of the deep learning book.

In the context of deep learning, we sometimes gain some guarantees by
restricting ourselves to functions that are either Lipschitz
continuous or have Lipschitz continuous derivatives.

They include

Lipschitz continuous functions
Having Lipschitz continuous derivatives

The definition given for Lipschitz continuous function is as follows

A Lipschitz continuous function is a function $f$ whose rate of
change is bounded by a Lipschitz constant $\mathcal{L}$:
$$\forall x, \forall y, |f(x)-f(y)| \le \mathcal{L} \|x-y\|_2 $$

Now, what is meant by having Lipschitz continuous derivatives?
Does they refer to the derivatives of Lipschitz continuous functions?  If yes, then why do they mention it as a separate option?

Lipschitzness doesn't imply differentiability, e.g. $f(x)=|x|$ is Lipschitz but not differentiable at $0$. On the other hand, having a Lipschitz continuous derivative doesn't imply Lipschitz continuity, e.g. $f(x)=(-1)^{\mathbb{1}_{x\ge 0}}x^2$. So they're separated.

Consider a function $f(x) : \mathcal{R}^m\rightarrow\mathcal{R}^n$ defined for $x \in X$. If $f$ is Lipschitz continuous, it has three main properties:

$f(x)$ is continuous for all $x \in X$
$\frac{d f(x)}{d x}$ exists almost everywhere. Meaning, if the derivative is not defined for $x \in \mathcal{B}$, where the set $\mathcal{B} \subset X$, then $\mathcal{B}$ has measure zero.

$\underset{x \in X}{\sup} \left\lVert \frac{d f(x)}{d x} \right\rVert_2 \leq L$, where $L$ is the Lipschitz constant, and the norm indicates the induced matrix norm (or if $f$ is scalar, just the regular 2 norm).

So it follows that a Lipschitz continuous function is continuous and has a bounded jacobian.
Now if $f$ has a lipschitz continuous derivative, then it means $\frac{d f}{d x}$ is Lipschitz continious, i.e.
\begin{align*}
\left\lVert \dfrac{d f}{d x}\big|_{x = s} - \dfrac{d f}{d x}\big|_{x = t} \right\rVert_2 \leq M \left\lVert s - t \right\rVert_2 \quad s, t \in X
\end{align*}
where $M$ is the lipschitz constant. So a function with Lipschitz continuous gradient is continuously differentiable and has a bounded hessian.

Answer:
A function $f$ 'Having a Lipschitz continuous derivative' is simply when the derivative of $f$ is Lipschitz continuous. So it is not the derivative of a Lipschitz continuous function.
Your book states that 'Lipschitz continuous derivatives' as plural, meaning that all derivatives of $f$ should be Lipschitz continuous.
Intuition on Lipschitz continuity:
Lipschitz continuity is a property of functions that measures how much the output of a function can change for a given change in the input. A function is Lipschitz continuous if there exists a constant, called the Lipschitz constant, such that the difference between the output of the function for two input points is bounded by the product of the Lipschitz constant and the distance between the two input points.
$$d_{Y}(f(x_{1}),f(x_{2}))\leq Kd_{X}(x_{1},x_{2})$$
where $K$ is the Lipschitz constant.
The wiki on Lipschitz continuity gives a nice visualization that might help understand the concept.


text:
Do solving system of linear equations required anywhere in contemporarty deep learning?
Consider the following from Numerical Computation chapter of Deep Learning book

Machine learning algorithms usually require a high amount of numerical computation. This typically refers to algorithms that solve
mathematical problems by methods that update estimates of the solution
via an iterative process, rather than analytically deriving a formula
to provide a symbolic expression for the correct solution. Common
operations include optimization (ﬁnding the value of an argument that
minimizes or maximizes a function) and solving systems of linear
equations. Even just evaluating a mathematical function on a digital
computer can be diﬃcult when the function involves real numbers, which
cannot be represented precisely using a ﬁnite amount of memory.

The paragraph clearly mentions that solving system of linear equations is a common operation in machine learning. I just know that solving system of linear equations is useful in reinforcement learning and some basic algorithms of machine learning including regression.
Is solving system of linear equations useful anywhere in deep learning?
I think that we use them nowhere since optimization is the only algorithm generally used in deep learning.

I guess a first distinction should be made between deep learning as a whole or deep learning as architecture.
I think the paragraph you quote refers to solving systems of linear equations as a simple operation involved in deep learning generically. And this is definitely the case, when training a deep model we're always solving systems of linear equations, think about the way weights and biases are applied to an input:
$W*x + b$
this is in itself a system of linear equations.
Then, moving away from training and model architectures, in deep learning there is still a massive use of dimensionality reduction techniques involved in pre or post processing of the data like SVD or PCA that also consists in solving systems of linear equations (an we could add any matrix factorization technique, relevant especially in the early methods for word embedding training before the advent of transformers).

Well the paragraph you quote says "Machine learning algorithms ...", so it's not restricted to deep learning.
When considering Machine Learning in general there is the case of Gaussian Process Regression where to fit the models a matrix needs to be inverted (which is equivalent to  solving a  linear system of equations), check equations 2.7 and 2.8 of chapter 2 of the standard GPR book


text:
Role of confidence or classification score in object detection mAP metrics
I know that mAP (mean Average Precision) is the common evaluation metric for the object detection tasks. It uses IoU (Intersection over Union) threshold such as mAP@0.5 to evaluate whether the predicted box is TP (True Positive), FP (False Positive), or FN (False Negative).
But I am confused about the role of classification score in this metric since the positive and negative is determined by the IoU, not the classification score. So, what is the role of classification scores in mAP evaluation?
Let's describe it by example, suppose there is a single object in an image with the ground-truth as follows:

Bounding boxes: [[100, 100, 200, 200]]
Class Index: [0]

Then the prediction of the object detection model resulting as follows:

Bounding boxes: [[100, 100, 200, 200], [100, 100, 200, 200], [100, 100, 200, 200]]
Class Indexes: [3, 2, 0]
Class Scores: [0.9, 0.75, 0.25]

When I tried to calculate the mAP using this library: https://pypi.org/project/mapcalc/
The mAP score is 1.0. So I am confused in the mAP point of view, this prediction is calculated as the correct prediction? So what is the role of classification score in this case? Should we also define the classification score threshold when using mAP?

Confidence Calibration for Object Detection and Segmentation

Nice question got me thinking, hope these help you understand better Source

MS COCO — uses 101-Recall-points on PR-chart mAP (mean Average Precision) for Object Detection
PascalVOC2007 — uses 11-Recall-points on PR-chart
PascalVOC2010–2012 — uses Area-Under-Curve on PR-chart
ImageNet — uses Area-Under-Curve on PR-chart

I too have to understand it further


text:
How to localize and classify objects in video
What methods are used to localize an object in an video and classify that object?
Example: I have a camera which detects an pickup truck driving into a garage of three (1,2,3). In need to know if the truck was loaded or not (classification) and which garage it picked (localization). How would a schematic workflow of this problem look like?
It is assumed that the camera is mounted in a fixe position.


text:
When to activate batch normalization and dropout in deep Q-learning?
In the vanilla version of deep Q-learning, there are three places where the Q-network is queried:

When exploring.

When training:
a. When calculating the optimal value of the state reached by an action (so as to compute a target discounted reward).
b. When calculating the optimal Q-value for a given state, during training (so as to nudge the network weights and better reproduce the observed reward).

Now, during which steps should batch normalization and dropout be activated?
I couldn't find anything through a Google search.
Here are my guesses, for each step:

When exploring: activate batch normalization and dropout: this lets the normalizations to be learned, and gives a chance to uncertain Q-values to be selected even if they are relatively low (because the dropout can result in a Q-value prediction higher than its average).

When training:
a. Do not activate batch normalization and dropout for calculating the optimal state value of the state reached by an action, because we want the Bellman equation to converge faster and therefore prefer stable (optimal state value) targets.
b. Activate batch normalization and dropout when calculating the Q-value of a chosen action, as this is the whole idea of dropout (we use it during training).

What is the common wisdom on this?

Batch Normalization should be applied between all layers and their activation functions excluding the output layer. This squishes the ranges of numbers in a better range for neural network to build appropriate sized gradients.
I've not seen much use of Dropout in Deep RL because the networks are usually small and overfitting isn't as much of a problem as in supervised learning.

This is quite true, but this doesn't seem to answer the question of when to let the batch normalization layers learn their normalization, during reinforcement learning.

I would always try it. As far as I know there is no one size fits all solution. So in general just treat using BatchNorm as a hyperparameter and test your agent with and without.

Hmm, I should have written "where" should batch norm be used (not "when"): the algorithm itself uses the network in 3 different places (the the original question): the question is in which of these places should batch normalization be activated.

If you use BatchNorm, then use it during all phases. Only Dropout is something you turn off during evaluation.

Batch norm is indeed always used, but the question is really about the activation of its updates (i.e. training vs evaluation mode): which of the three different parts of the basic Reinforcement Learning algorithm should update the running measurement of the input's statistics? I'm not sure it matters much, but maybe this is incorrect.

I'm not really sure I understand your question. It will be always updated by backprop whenever your network is updated.

… and also possibly when calling it during exploration and when computing the state value of the state reached by an action (for Bellman's equation). Hence the question.

What exactly do you mean by 'during exploration'? In general your get_action method called in rollout should not calculate a gradient. For calculating state value you should use a target net for stability, so it's use of batchnorm depends on if your state value function uses it.

Indeed, for exploring we do not calculate a gradient, but there is the choice of whether we want to accumulate statistics for the batch normalization, hence the question (the same question also holds for training, when calculating the expected discounted reward of the state reached by an action).

There is no reason to accumulate statistics during interaction with environment (this is what you call exploration phase?). This is because your experienced timesteps will be stored in some kind of memory to train on. When our agents then trains on them, batchnorm will train on the data.

Since the agent constantly interact with the environment, what I mean by "exploration phase" is simply the early part of training, where the agent accumulates experiences (in the "memory" you refer to--the replay buffer). When the learning takes place, batch norm will indeed learn statistics. But then why not accumulate batch norm statistics when accumulating experiences too? Since no sampling takes place, this even looks like a good idea, no?

Because you would do it twice then.

… or more or less, depending on how the replay buffer is sampled. All in all, it looks like when batch norm accumulates statistics might not matter too much. One question could be whether we want the batch norm statistics to be based on the exact samples on which we learn, or be skewed towards all the samples encountered during exploration episodes. If anybody has any opinion on this, that would be interesting.


text:
How does a VGG-based Style-Loss incorporate color information?
I've recently been reading a lot about style transfer, its applications and implications. I understand what the Gram matrix is and does. I can program it. But one thing that has been boggling me is: how does the VGG style loss incorporate color information into the style?
In the paper "Texture Synthesis by CNNs", Gatys et al. show that minimizing the MSE between the Gram matrices of a random white noise image and a "target texture" yields new instances of that texture, with stochastic variation. I understand that this must work, as the Gram matrix measures the correlation between features detected by the VGG activations across channels, without spatial relation. So if we optimize the white noise image to have the same Gram matrix, it will exhibit the same statistics, and hence look like an instance of the original texture.
But how does this work with color? Of course, the VGG could learn something like a mean filter, with all ones, whose output would be the avg. color over that filter kernel. After all, "color" is just another statistic. But then when using that in conjunction with the Gram loss, wouldn't this information be lost, as it's all just correlation and hence "relative" to each other?
While writing this question, I'm starting to think of it like this: Maybe the feature correlation expresses these color constraints in some form like: "if one part is red, there must be a green part close to it" (for the radish), or "if there is a rounded edge, one side of it must be in shadow (=darker)" in case of the stone texture. This would tie color to the surrounding statistics (e.g., edges, other colors) and is the only reason I can think of why this works at all.
Can somebody confirm/refute this, and share their thoughts? Happy to discuss!

Image Source: Texture Synthesis by Convolutional Neural Networks, Gatys et al.

My two cents on this topic:
"After all, "color" is just another statistic", I think this is the simple (and correct) answer to the question. To go a bit deeper, you can check this paper, which shows how minimizing a loss based on the Gram matrix is mathematically equivalent to minimizing the Maximum Mean Discrepancy between the inputs and targets distribution. The two distributions inevitably contains information about colors, so while disentangle spatial features is rather simple (you could simply show one pixel at the time instead of an image),  disentangling colors is much more tricky, cause it's an intrinsic characteristic of each point.
A last remark from my side is that the main problem when working with style transfer is precisely that "style" mean everything. This is not a problem for papers that simply try to achieve it per se, i.e. without a real use case in mind, but it becomes fundamental in real applications. A concrete example of this is super resolution. Many papers try to achieve it with style transfer, coupling low resolution and high resolution images. Ideally the features you would like to transfer are enhanced sharpness and maybe texture injection for details generation. Problem is that along with them there are always side features that hinder the quality of the resulting image, among which noise specific to the target domain, and also colors.

If you want to put color channel value for the loss function in neural network training, I have seen Structural Similarity Index Measure, SSIM, being used at Super Resolution neural network. Here is one webpage about SSIM and example code from Sci-Kit Image.
Your intention looks like not just one pixel but surrounding, so you may somehow putting moving window or Region of Interest in SSIM equation. Or you have to write own equation using Region of Interest with pixel value(s).


text:
Why is the prior on the latent variable standard gaussian in VAE?
While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. Why not assume any other gaussian as the prior? What are the intuitive reasons for this?

A good reason is we normalize data before the training process!

Simply, it is just a design choice.
Isotropic gaussian is one of the easiest density to work with. It has an easy-to-compute likelihood and easily reparameterizable.
You are free to use other distribution, but might face computational or implementation hurdles.


text:
Residual Blocks - why do they work?
I've learnt that idea that the residual block was invented to solve the vanishing gradient problem due to the deep layer to layer multiplication.
I understand that for example if I have 10 layers, and I add another 5 layers, that the output of the 10th layer will 'skip' the 5 layers. Although, the output of the 10th layer will also pass through the 5 layers as well. Just before the 15th layer Relu, the output from the 10th layer is element-wise summed with the 15th layer, just prior to the final Relu.
I have some confusion with this.

Identity mapping/function. I keep reading that it creates an identiy function or it learns an identity function. What exactly is this? Isn't is just F(x) = 5 added layers, and x =output of 10th layer and thus it is just F(X) + X?

By summing the output of the 10th layer to the 15th layer, will this not affect what was learnt in the 5 layers? I.e. from 11th -15th layer.

I believe it also helps with backpropagation so that it doesn't have to update all the weights layer by layer and it can skip back to shallow layers. Therefore, are the weights inside the residual block, i.e layers 11-15 not updated? If not, then what is the point of the 11-15th layer if they are not designed to "do anything".

An identity function is f(x)=x so F(X)+X is the same as F(X)+f(X) if f is the identity function

@user253751 Yes I know that, but why do we train the residual block to output 0? Because we need it to be 0 such that x the identity, can be the same as the input to the residual block.

The purpose of a residual block is not to output zero, I am not sure where you got that from.

@Dr.Snoopy I keep reading that the output should equal to input. And the equation F(x) = f(x) + x keeps getting brought up and they say the point of identity is the input should be the same as the output, so in my head that means the residual block needs to be 0, or learn to make it 0. because F(x) = 0 + x which is F(x) = x

Keep reading, where? That is not the correct interpretation, I think you are reading about a different kind of residual.

@Dr.Snoopy https://towardsdatascience.com/resnets-residual-blocks-deep-residual-learning-a231a0ee73d2
"In the above equation in figure 3, let g be the ReLU activation function. If the activations for the layer l+2 tends to 0"  - This is what confuses me

First a little intro, skip to the end for the straight answers: residual networks were proposed after observing that deeper models tend to perform worse than their shallow counterpart if we just keep adding hidden layers without applying any other change to the architecture, as we can see in the very first picture of the original paper.

The reason of this phenomena is indeed gradient vanishing. The more the hidden layers, the more the information of the original input get lost, due to the fact that a hidden layer receives only information from the previous hidden layer. How to solve this? Using residual connections. A residual connection is just an identity function that map an input or hidden state forward in the network, so not to the immediate next layers, that's why these connections are also called skip connections. The only purpose they serve is to force deep layers to retain information learned in the early layers of the network.
From a numerical perspective you can think about information getting lost as weights becoming smaller and smaller. By brutally summing the hidden states of previous layers you make sure to avoid this problem, giving the weights a broader range of adjustment even in very deep layers.
In conclusion, to answer your questions:
1 Residual connection don't create or learn an identity function, they simply use it. The formulation of such connections in the paper is:
$y = F(x, W_{i}) + x$
where x could be rewritten as $I(x)$, $I$ being the identity function.
2 No, we don't loose any information by summing the residuals, on the contrary, they are designed to retaining information also in very deep layers, for the above mentioned reasons.
3 All layers are updated, there's no frozen layers in a residual network. The "skip" term refers to the fact that a hidden layers is copied in forward layers, which is a legit operation, but it doesn't refer at all to skip in training or weight updates.

Thanks for the detailed description. I have a quick question. I'm reading that the goal of the residual block so that before the last 'ReLU' activation, the summation of the input and the output of the residual block, should equal. So that is why I am confused that to make this happen you need the residual block to output 0. Also, does summating the input with the output of the residual block not alter the true learning of the residual block? or is the purpose to just mix in information previously learnt with information learnt in the residual block. Thanks.

"the goal of the residual block so that before the last 'ReLU' activation, the summation of the input and the output of the residual block, should equal ". The summation is not suppose to be zero, the 'skip connection' before the relu actually prevent everything going to zero, due to vanishing gradient.  "is the purpose to just mix in information previously learnt with information learnt in the residual block" Yes, precisely. Thanks to the skip connections a deep layer would at the very least output the same as previous layers, but never loose information.

The residual layer was not invented to solve the vanishing gradient problem. Citing from the official ResNet paper:

An obstacle to answering this question was the notorious problem of vanishing/exploding gradients [1, 9], which hamper convergence from the beginning. This problem, however, has been largely addressed by normalized initialization [23, 9, 37, 13] and intermediate normalization layers [16], which enable networks with tens of layers to start converging for stochastic gradient descent (SGD) with backpropagation [22].

The problem that this layer solves is: How to create deeper models so that they perform definitely better than shallower models?
By adding the residual connection we modify the output of the layer so that instead of $F(x) = f(x)$ it now outputs $F(x) = f(x) + x$, where $f$ is the function represented by the layer around which we add the skip-connection.

(technically the skip-connection is considered the main flow, and the NN layer is called the residual)
Now if you have two identical models, but one has $N$ layers and the other $N+1$ layers, and if the shallower model was the better choice, then the last layer of the deeper model is actually not needed (or one of the layers, no need to be the last). In order for the deeper model to simulate the shallower model it simply has to learn that the function of the last layer has to be $f(x) = 0$, i.e. all of the weights of the last layer have to be $0$. Without the skip-connection the model has to learn that the last layer function has to be $f(x) = x$, which is much more difficult.
So now if your model is not performing well you can simply add more layers and you expect that the performance will not degrade. It may not improve, but it should not degrade.
See this if you want to read more: https://pi-tau.github.io/posts/res-nets/

For the Third question,

I believe it also helps with backpropagation so that it doesn't have to update all the weights layer by layer and it can skip back to shallow layers. Therefore, are the weights inside the residual block, i.e layers 11-15 not updated? If not, then what is the point of the 11-15th layer if they are not designed to "do anything".

Model with many layers, it is easy to get a phenomenon- gradient vanishing. In short, model does not learning in some layers(as layers 11-15 in your example). And Residual Block can help to solve this phenomenon.
So, we can clarify it. These layers are designed and we hope they could work well. But in some situation, not well --> adding Residual Block is a solution.


text:
Is knowing the class of probability density function mandatory for explicit density estimation?
In deep learning, models may learn the probability distribution that generated the dataset. Observe the following paragraph from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.)

Unsupervised learning algorithms experience a dataset containing many
features, then learn useful properties of the structure of this
dataset. In the context of deep learning, we usually want to learn the
entire probability distribution that generated a dataset, whether
explicitly, as in density estimation, or implicitly, for tasks like
synthesis or denoising. Some other unsupervised learning algorithms
perform other roles, like clustering, which consists of dividing the
dataset into clusters of similar examples.

I read about density estimation in the same chapter, as given below

In the density estimation problem, the machine learning algorithm is
asked to learn a function $p_{model} : R^n \rightarrow R$, where
$p_{model}$(x) can be interpreted as a probability density function
(if $x$ is continuous) or a probability mass function (if $x$ is
discrete) on the space that the examples were drawn from.

This question is focused on explicit probability density estimation in continuous case i.e., learning density function $p_{model}$  directly.
Suppose I have a dataset $D$ with $n$ continuous random variables (features) $X_1, X_2, X_3,\cdots, X_n$. And I don't know anything about the probability density function of individual random variables. That is, I don't know about any information about any $X_i$ , such as, whether $X_i$ follows normal distribution or any other distribution. Then, is it possible to learn density function explicitly? Or do I need to provide some necessary information such as the class of probability distribution function to be learned?
I am thinking as follows:
If I have some information about $X_i$, such as: $X_i$ falls to a well known distribution, then I can learn the parameters of the underlying density function from $D$. So, is it mandatory to know some information about the underlying probability density function.

Neural Networks can approximate any function. Quoting the essence in case the article is removed in the future.

The key to neural networks’ ability to approximate any function is that they incorporate non-linearity into their architecture. Each layer is associated with an activation function that applies a non-linear transformation to the output of that layer.

So no, knowing the class of the probability density function is not required to approximate it via Deep Learning. With a large enough number of samples you could construct an approximation.


text:
Is discriminator a regressor or classifier in implementations?
GAN has two components: generator and discriminator.
Discriminator in the original GAN is a regressor and always gives value in $[0, 1]$. You can read it in original paper

$D(x)$ represents the probability that $x$ came from the data rather
than $p_g$

Is it true with most of the (advanced or) contemporary GANs? Or the do nature of discriminator, either as a regressor or as a classifier entirely depends on the context?

Discriminator in the original GAN is a regressor

No, it is a classifier. It classifies an image as "real" or "fake", with the output usually being probability that the image is "real" (you could reverse this and use generated images as the target class, provided you change the generator training to match).

Is it true with most of the (advanced or) contemporary GANs?

In WGANs the W stands for Wasserstein, and these GANs use Wassersten loss, which measures the distance in "realness" between real and fake images. This measure of realness is a regression problem, with the caveat that there is no true measure of realness for any images that you can train with separately. The architecture of the critic, which plays the same role as discriminator for classic GANs, is the same as a neural network used for regression.
In general, if you see the term "discriminator", you can assume a classifier is being used. If you see the term "critic", you can assume a regressor. This may not be true for everything published about GANs, as some authors may use the terms loosely, but it is reasonable to expect if you are reading original papers or learning from a course.
As far as I can tell, StyleGAN2, which produces state-of-the-art results, uses a standard discriminator/classifier setup. There are plenty of other architectural details in the discriminator that contribute to the performance of the GAN. There is a link to the paper describing these from the linked Github implementation.


text:
Get the name of a merchant from records
I have a bunch of bank transaction records from which I want to extract merchants' names. In a few subsets of these records, the structure of the string is the same within the subset with only the merchant name changing. For example
subset 1

XXXXX_ID_TIME_STAMP MERCHANT1 CREDIT
XXXXX_ID_TIME_STAMP MERCHANT2 CREDIT

subset 2

BILL PAYMENT BANK_NAME MERCHANT NAME 3
BILL PAYMENT BANK_NAME MERCHANT NAME 4

In the above two subsets, the structure of the string is the same, only the merchant names changes
and so on ...
Using NLP, I want to extract merchant names in such cases. How should I approach this?
Using regex is not feasible because I'd have to manually go through the complete data, identify all such patterns and create regex strings that'll extract the name. I would also have to do this for every new pattern.
Is there a way where I can train a model that can identify/extract merchants in such cases?

The problem:
You are facing a Natural Language problem called Named Entity Recognition (that's the key word you are looking for).
But before you dive deep into it, have in mind it's best suited for user input data (where users are absolutely chaotic) and it looks like you have a system data.
The right way:
You should have some kind of tabular (structured) data, instead of a string. So:

Triple check if you have some white-space separator (like tabs).
Review you content retrieval source and method.
If you have influence over the data source, ask them to generate it the right way.

If none of them help...
The programmatic way:
I can't see a specific Machine Learning model to solve that, but you can combine several techniques (some of them NLP) to help. It's an analytical and explorative process. Here are some insights:

the structure of the string is the same within the subset with only the merchant name changing

If you have some absolutely identical records, except for the data you are looking for, you could simply look for the diff between them.

Dictionary retrieval: If you have the list of Merchants, it's as simple as checking if a record contains any of the listed words. If you don't, you can build it as you run other methods. So maybe one subset can help you solving the other, like a Sudoku puzzle.

Track special characters:

If some columns (XXXXX_ID_TIME_STAMP CREDIT BILL PAYMENT) contain numbers (or any special character), you can eliminate them right away.

Tokenization: you can convert every word to a unique token. If the Merchant Name is composed by 2 or more words, you can use n-grams.

Frequency Analysis: The names (tokens / n-grams) will probably have similar frequency inter and intra subsets. For example:

The bank name might be much more frequent than a merchant name.
A merchant might be frequent in one subset but rare in others.

Divide each record into smaller substrings.

If you can eliminate some n-grams inside a record (using methods like special characters or frequency analysis) you'll have smaller (and more structured) problems to handle. For example: [_][_]MERCHANT[_]BANK[_]


text:
Why use sin/cos to give periodicity in time series prediction
In this tutorial https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering (scroll down a bit to "Time" heading), they take the sin/cos of the time index, and give this as an input so that the model can see the periodicity.
Why use sin and cos (which map to a circle)? Why not map to a square, or a diamond?
What about if you just mapped time to 1d instead of 2d, so e.g. 23:59 would be 1 and 00:00 (1 minute later) would be 0. Would that "jump" actually cause problems?
Any actual research or experiments which look at this issue?

I can imagine that one reason is that the author has the properties of the Fourier series and transformations in mind.  Every periodic behavior can be decomposed as a sine expansion.

Why use sin and cos (which map to a circle)?

Specifically from the tutorial's point of view, this is the most usual piece of feature engineering applied to periodic input data.
It offers the following advantages - roughly in order of importance (in my opinion):

Removes discontinuities (e.g. as you suggest the "jump" from 1 to 0 when time of day wraps around). Non-linear learning algorithms - such as neural networks - can learn to approximate discontinuities from inputs or outputs, but why force the learning algorithm to learn about them, when you can make the necessary transform for it? After all, that is a core purpose of feature engineering, to inject knowledge about the problem that assists the learning algorithm.

Creates a consistent distance measure. The vector distance between two points on the circle is always the same if they are the same distance away linearly on the pre-transformed feature (including the periodicity). This is not the case with other mappings such as squares or diamonds.

Sin and cos are standard library library functions in most programming languages. Mapping to a square or diamond involves slightly more custom coding.

Why not map to a square, or a diamond?

Squares or diamonds are close enough to circles, that in a lot of cases you would not notice the difference in practice. There may even be cases where some kind of closed polygon is a more natural mapping for a given problem. So feel free to use them and experiment when performing feature engineering.

Any actual research or experiments which look at this issue?

This kind of low-level feature engineering is simple enough that you probably won't find many papers that pull out just solutions to mapping periodic variables. You will find using sin and cos to transform a periodic variable used as a standard transform in many situations though.
If you are interested to compare different options for mapping periodic variables, you will likely need to perform the experiments yourself. For any specific problem where you are not sure, this is good practice if you have the time. There is a chance that you will find a useful generic mapping for periodic data that is not based on circles, but my prediction is that you will find other shapes perform similarly to circles a lot of the time, but are sometimes worse and rarely if ever better.


text:
Why labeling facades?
In Pix2Pix by Isola et al. they translate images from different pairs of image categories to one another. While most other example applications for the algorithm make sense to me, I'm having difficulties understanding why one would translate facade labels to facade images. As the title says, I already don't see how labeling a facade would help solving any real world problem.
I skimmed the related work of the paper and found little about what "facade parsing" could be used for, except maybe reconstruction from images. Where are facades reconstructed from facade labels? Can anyone tell me other example applications for facade labels and translating them to images?

Does it have to be facade labels? They are just labels into images. The same technique could be used to convert other labels into other images.

although maybe facades are useful for a computer game in a city!

I guess computer games is the best possible use case. However, I don't know any game that was made using facade labels. The same technique can in fact be used for autonomous driving. Labeling traffic scenes is another example in the paper. But facades? It's probably because labeling facades is a comparably easy computer vision task.

Welcome to SE:AI!  This seems like a great question on theory.

It is enlightening to look at how the training data is created. For the purses and shoes they took real purses and shoes and then performed some thing like a county edge on them. They trained the learner using the simpler image to infer texture and color.
There are plenty of folks in the fields like “visual design“ that want to turn a sketch into something more realistic, and the elements of form, and lighting, and space, are important to how they go about doing that. They would use something like this trained pix2pix in their explorations of the transition between the architectural sketch and the real world.


text:
Total number of states reachable from the initial state in 8-puzzle problem
I know it's a simple question but the book Artificial Intelligence by Russel says that the number of reachable states from any initial state in the 8-puzzle problem is $\frac{9!}{2}$. However, I think it should be $9!$. Note that we can't say if we rotate the grid horizontally then state we get is the same so as to divide the total number of states by $2$. So why do we divide the total number of (initial) states by $2$? What extra states are we counting?

IIRC, swapping any two puzzle pieces in any state results in a state which is not reachable from the initial state. There is some kind of "parity"

@user253751 Just one thing: What does that llRC mean?

"If I remember correctly"

@user253751 Could you elaborate on that please? In permutations, we can also think of swapping any two states, resulting in a different state, yet the number of unique possible states is still n!

@OmarAlSuwaidi But only half of them are reachable

@user253751 ahhh I see, thanks for the clarification, it makes sense now


text:
Is the following a typo or am I understanding wrongly regarding discriminator?
Consider the following paragraph from the section 3: Background of the research paper titled Generative Adversarial Text to Image Synthesis by Scott Reed et al.

Goodfellow et al. (2014) prove that this minimax game has a global
optimium precisely when $p_g = p_{data}$, and that under mild
conditions (e.g. G and D have enough capacity) $p_g$ converges to
$p_{data}$. In practice, in the start of training samples from D are
extremely poor and rejected by D with high confidence. It has been
found to work better in practice for the generator to maximize
$\log(D(G(z)))$ instead of minimizing $\log(1 −D(G(z)))$

I am guessing the bolded portion should be replaced by  from G. Am I correct? If not, where am I going wrong?

I think it's a typo (but given that I am not an expert in GANs, I will let someone else provide a formal answer). There's another (less severe) typo in that same paragraph, i.e. "optimium", it should have been "optimum". Not sure why this kind of typo goes through the scrutiny of the reviewers or even automatic correction software (like Grammarly or even the editor(s) they used to write the paper).


text:
When would you use Evolutionary Strategies over Step-Based Reinforcement Learning
In Salimans et al, 2016, the authors argue that ES should be considered a competitive alternative to MDP-based RL algorithms like Q-Learning, TRPO.
However, in practice, I notice that more often than not ES takes far more episodes to converge than MDP-based algorithms. So what would still be a reason to consider those, apart from pure academic interest?
The authors mention that ES will show less variance in long-horizon tasks, but didn't give an example. Is this aspect crucial ?

Great question! I did some research and found out that Generally capable agents emerge from open ended play by Deepmind is using ES in form of population based training:

We also explored the question, what distribution of training tasks will produce the best possible agent, especially in such a vast environment? The dynamic task generation we use allows for continual changes to the distribution of the agent’s training tasks: every task is generated to be neither too hard nor too easy, but just right for training. We then use population based training (PBT) to adjust the parameters of the dynamic task generation based on a fitness that aims to improve agents’ general capability. And finally we chain together multiple training runs so each generation of agents can bootstrap off the previous generation.

But this didn't really answer on their reasoning, so I dug some deeper and found a great article on lesswrong.com about the use of PBT. I will quote the essence, but highly recommend to read the linked Chapter on PBT:

What does the evolutionary selection give us that we don't already
have? What problem does this let us avoid?
There are several answers to this question.
The more narrow answer is that this allows the dynamic task generation
hyper parameters themselves to shift in a direction that promotes
general competence. Neither of the optimization levels beneath us
include any way of changing these parameters. But the ideal filtering
parameters for the production of general competence might be different
at the beginning, or at the middle of training. Or they might be
different from agent to agent. Without something like
population-based-training, they would have no way of changing and this
would hurt performance.
The less narrow answer, I think, is that this ensures that agents are
developing broad competence in a way the innermost loop cannot do.
[...]  each agent in our population of agents will learn to get better at
some distribution of tasks, then, but without
population-based-training they might not spread themselves broadly
across the entire span of this distribution. Like a student who
advances wildly at subjects she prefers, while ignoring subjects she
is not good at, our agents might not approach our desired ideal of
general competence. Population-based-training helps prevent the
scenario by multiplying agent / teacher pairs that do well generally
and non-narrowly.


text:
What is meant by correlation structure?
I know only about the Pearson's correlation coefficient in literature.
Covariance between two random variables $X$ and $Y$ is defined as
$$Cov[X, Y] = \mathbb{E}[(X - \mathbb{E}[X])(Y-\mathbb{E}[Y])]$$
(Linear) Correlation between two random variables $X$ and $Y$ is defined as
$$Corr[X, Y] = \dfrac{Cov[X, Y]}{\sigma(X)\sigma(Y)}$$
Covariance is a measure of association between two random variables whereas correlation measures how much dependent they both are on each other.
Consider the following excerpt mentioning "correlation structure" from section 2: General Design Principles of the research paper titled Rethinking the Inception Architecture for Computer Vision

Theoretically, information content cannot be assessed merely by the
dimensionality of the representation as it discards important factors
like correlation structure; the dimensionality merely provides a rough
estimate of information content.

What is meant by the "correlation structure" mentioned here? Is it a graph on input random variables? Is it in any way related to the aforementioned correlation?


text:
How to embed game grid state with walls as an input to neural network
I've read most of the posts on here regarding this subject, however most of them deal with gameboards where there are two different categories of single pieces on a board without walls etc.
My game board has walls, and multiple instances of food. There are 8 different categories,
Walls, enemy food, my food, enemy powerup, my powerup, attackable enemies, threatening enemies, and current teammate.
I have one hot encoded all of this data into a tensor of size (8, 16, 32) where (16, 32) are the sizes of the game grid. However I'm not sure whether this is appropriate since many of the categories have multiple occurrences of each category in a single  (walls, food). Is it appropriate to use one hot encoding to represent categories in spatial data, where multiple one's may be present?
The alternative I was considering was to use a CNN, however many posts have said it is inappropriate for one hot data. My reasoning was that since the data is a abstract Boolean grid representing the RGB frames, it might be appropriate.
Does anyone have any suggestions as to the best way to represent a spatial Boolean grid representing multiple categories for input to a network?

The way you describe with one hot encoding is correct.
Note that how the state is encoded is a separate question from the neural network, so I'm not sure what convolutional neural networks have to do with the question. In the famous atari game example, the input is a sequence of RGB images; a cnn is used to process the images. In your example you probably just want to use a regular Dense network, as your input is just the one hot encoding and not images.

Ahh, i've gone with a Dense network now. Do you think I should use activation functions? at the moment my network is outputting values between -1 and 1. I'm guessing it'd be better to use the probabilities as my reward function is a sum of percentages

Q tables/functions are attempting to predict the rewards for a given action assuming all of the perfect decisions are made from there forward until the end of the episode.  It is not only the present reward but a discounted value of all future rewards.  So no matter the value of the rewards, an output neuron for the Q-Value of an action should be able to represent (-inf, inf), this is typically linear activation.

If there can be multiple items on a square something more complex than one-hot is needed.  As the name implies, only one category can be hot (set to true) in  one-hot.  You need multi-hot.  Further, if multiple instances of items can be on a grid square (2 food, 1 wall) it might be beneficial to encode it proportionality-wise (.66 food, .33 wall).
CNN's inductive bias is locality.  It's looking for items near each other in the grid space with certain values.  It identifies groups of items that are similar to some pattern (ie looking for the ball or paddle in pong).  I'll leave it to you to decide if this is interesting in your environment.


text:
Image recognition neural network: scaling and rotation
Are there some effective and robust solutions for scaling and rotation for image recognition with the neural networks (NN)?
I see tons of sources on the Web with explanation how neural network is used for image recognition, but all of them avoiding the topic of scaled or rotated images. The network trained for patterns won’t recognize it if the pattern scaled or rotated.
Of course, there are some intuitive/naive workarounds/approaches:

Brute force – you can rotate and scale image until NN recognizes it. Too much expensive.
You may teach NN for all cases of rotated and scaled image, could be hard and will lead to NN weakening.
You may teach NN that some images (which are rotation and scale of the original image) are clusters and teach to recognize clusters and interpolate/extrapolate them. A little bit tricky in coding and debugging.
For rotation you can move to polar coordinates, this gives a kind of invariant both for recognizing patterns and building histograms for specific portions of the image. But for this you need to recognized the pivot point and again this is quite expensive.

Are there any better solutions, ideas, hints, references?
(I read some answers there to the rotational problem, but what I saw doesn't cover the topic).

@norbertk, of course. You even don't need to generate them, it is just the way the algorithm (some layer of it) converts the information before processing. Anyway, this is quite costly, you will need lots of cycles with different rotations and what is worse, you won't have an idea what number of scales to use, you can only set some reasonable thresholds. Applying both rotations and scales multiplies the number of processing cycles, so the amount of work will be tremendous.

@norbertk, I understood this. I just say this this can easily be optimized. Some intermediate layer will give the NN rotated and scaled results just based on the angle and scale and original image. No need to create images and waste memory and throughput. This is done in the intermediate layer, NN won't see it. For NN it is the same either to read from memory (file) or from some stream (function). Anyway, this doesn't matter. Even with your case, just imagine how many files you will have to create and process, there will be tons of them. This is a strike on performance.

@norbertk, well, so let's consider your option. We generate images on fly. Let's say we recognize real-life not self-transforming objects like cars, airplanes, etc. How many images will we need? What is the appropriate step for rotation? One degree, ten degrees? How many scales will we need? We will have a crazy amount of their combinations. All will take time, scaling, rotation, NN learning, etc. For each image we will have to pass through all these operations. This works, but really slow. We should have a way to do this faster.


text:
What is the optimal score for Tic Tac Toe for a reinforcement learning agent against a random opponent?
I guess this problem is encountered by everyone trying to solve Tic Tac Toe with various flavors of reinforcement learning.
The answer is not "always win" because the random opponent may sometimes be able to draw the game.  So it is slightly less than the always-win score.
I wrote a little Python program to calculate that.  Please help verify its correctness and inform me if it has bugs or errors.

Who moves first? Agent always moves first?

@Taw Yes, currently.  I may make it an option in the program.

This blog post suggests that when playing against a random opponent, if the agent goes first, the win rate is 97.8%, and if they go second, the win rate is 79.6% (and the rest are draws).

Your numbers may be compatible with my answer.  I will modify my program to calculate the win/lose rate also....

Using your numbers, if agent goes first, 20 * 97.8% + 10 * 2.2% = 19.78 which is different from my answer = 19.9479...

@YanKingYin There may be a difference between perfect play and maximising play against a specific opponent, especially as you have incentivised a draw differently to the usually assumed zero-sum game in your code. Perhaps re-run your code with draw scoring 0?

@NeilSlater:  That would be 20 * 97.8% + 0 = 19.56, still different from my program's result = 19.8958333...

By the way, that blog article's numbers are from "experimental" statistics, so it may be inaccurate.

@YanKingYin: You need to re-run the experiment if the rewards change, because the optimal policy is different. Again it is also important whether this anwswer references perfect play or optimal play, because they can be different. Your experiment approximates optimal play for a given reward structure and given opponent. The blog appears to be using perfect play for its "expert" player.

@NeilSlater:  Yes, I reran the experiment with new scoring scheme.  What is the difference between "perfect" and "optimal" play in your opinion?  Can you explain with an example?

@YanKingYin Too much for a comment section. If it is of enough interest to you, then please ask a new question on the site.

@NeilSlater:  I believe that choosing the maximum expectation value is the same as "perfect play".  I have never heard of anything to the contrary.

@YanKingYin They are guaranteed the same when your opponent also plays perfectly (related fact - this is a Nash equilibrium, neither player can benefit from changing their policy). However, against a random opponent, the maximum reward policy and perfect play policy will be different in some states. that is because perfect play is always defensive against assumed perfect play by the opponent, it is about "not making mistakes". If you want to see proof, ask on the site.

As a simpler example, you could construct an opponent P2 which was almost perfect, but reliably made a losing mistake in a specific reachable state S, that would normally be a losing state for P1. It is trivial to see that maximising play from P1 would attempt to reach this state, whilst perfect play would do the opposite and avoid it. The perfect player would expect a draw at best, but the maximising player may expect to win.

Hmm.. very interesting.  I'm starting to see your point.  So if optimal play is maximizing expectation value, how would you define "perfect" play?  What kind of objective function does it maximize?

@NeilSlater:  Yes, I should add a question on this site.  Hold on....

Those numbers are consistent with the results of playing the Nash Equilibrium strategy I got with MiniMax against a random agent.

Here I asssume:

Both players avoid illegal moves perfectly
Player X always chooses the move with maximum expectation value
Player O chooses all available moves with equal probability

Result depends on the scoring scheme:

(This scheme is used in one version of Gym Tic Tac Toe)
For win=20, draw=10, lose=-20:
Optimal expectation value =

X plays first: 19.94791666666666...

O plays first: 19.164021164021...

For win=20, draw=0, lose=-20:
Optimal expectation value =

X plays first: 19.89583333333333...

O plays first: 18.497354497354....

It also helps to verify the program with some pre-played board positions, included in the code.
Here is the program:
import math     # for math.inf = infinity

print("Calculate optimal expectation value of TicTacToe")
print("from the perspective of 'X' = first player.")
print("Assume both players perfectly avoid illegal moves.")
print("Player 'X' always chooses the move with maximum expectation value.")
print("Player 'O' always plays all available moves with equal probability.")
print("You may modify the initial board position in the code.")

# Empty board
test_board = 9 * [0]

# Pre-moves, if any are desired:
# X|O|
# O|O|X
# X| |
#test_board[0] = -1
#test_board[3] = 1
#test_board[6] = -1
#test_board[4] = 1
#test_board[5] = -1
#test_board[1] = 1

def show_board(board):
    for i in [0, 3, 6]:
        for j in range(3):
            x = board[i + j]
            if x == -1:
                c = '❌'
            elif x == 1:
                c = '⭕'
            else:
                c = '  '
            print(c, end='')
        print(end='\n')

if test_board != 9 * [0]:
    print("\nInitial board position:")
    show_board(test_board)

# **** Calculate expectation value of input board position
def expectation(board, player):

    if player == -1:
        # **** Find all possible next moves for player 'X'
        moves = possible_moves(board)

        # Calculate expectation of these moves;
        # Player 'X' will only choose the one of maximum value.
        max_v = - math.inf
        for m in moves:
            new_board = board.copy()
            new_board[m] = -1       # Player 'X'

            # If this an ending move?
            r = game_over(new_board, -1)
            if r is not None:
                if r > max_v:
                    max_v = r
            else:
                v = expectation(new_board, 1)
                if v > max_v:
                    max_v = v
        # show_board(board)
        print("X's turn.  Expectation w.r.t. Player X =", max_v, end='\r')
        return max_v

    elif player == 1:
        # **** Find all possible next moves for player 'O'
        moves = possible_moves(board)
        # These moves have equal probability
        # print(board, moves)
        p = 1.0 / len(moves)

        # Calculate expectation of these moves;
        # Player 'O' chooses one of them randomly.
        Rx = 0.0        # reward from the perspective of 'X'
        for m in moves:
            new_board = board.copy()
            new_board[m] = 1        # Player 'O'

            # If this an ending move?
            r = game_over(new_board, 1)
            if r is not None:
                if r == 10:             # draw is +10 for either player
                    Rx += r * p
                else:
                    Rx += - r * p       # sign of reward is reversed
            else:
                v = expectation(new_board, -1)
                Rx += v * p
        # show_board(board)
        print("O's turn.  Expectation w.r.t. Player X =", Rx, end='\r')
        return Rx

def possible_moves(board):
    moves = []
    for i in range(9):
        if board[i] == 0:
            moves.append(i)
    return moves

# Check only for the given player.
# Return reward w.r.t. the specific player.
def game_over(board, player):
    # check horizontal
    for i in [0, 3, 6]:     # for each row
        if board[i + 0] == player and \
           board[i + 1] == player and \
           board[i + 2] == player:
            return 20

    # check vertical
    for j in [0, 1, 2]:     # for each column
        if board[3 * 0 + j] == player and \
           board[3 * 1 + j] == player and \
           board[3 * 2 + j] == player:
            return 20

    # check diagonal
    if board[0 + 0] == player and \
       board[3 * 1 + 1] == player and \
       board[3 * 2 + 2] == player:
        return 20

    # check backward diagonal
    if board[3 * 0 + 2] == player and \
       board[3 * 1 + 1] == player and \
       board[3 * 2 + 0] == player:
        return 20

    # return None if game still open
    for i in [0, 3, 6]:
        for j in [0, 1, 2]:
            if board[i + j] == 0:
                return None

    # For one version of gym TicTacToe, draw = 10 regardless of player;
    # Another way is to assign draw = 0.
    return 10

print("\u001b[2K\nOptimal value =", expectation(test_board, -1) )

Example output (for X's turn to play):

There is either a typo in your post, or bug in your code, because if win = 10, draw = 0, lose = -10, it's impossible that the expectation is 19.90.

Actually, you can directly calculate the optimal odds to win in TicTacToe against a random opponent and it turns out that you would win 191 out of 192 games and draw the last one when beginning.
For that -unintuitively- you have to start in a corner, lets say top left:
O . .
. . .
. . .

Then one can see that the only way to draw for the second player is to put the
X in the middle (so chance for a random opponent: 1/8). If he doesn't, O can force a win, e.g. after
O X .
. . .
. . .

O can put the marker in the middle, forcing X on the bottom right and then putting an O on the middle left, creating 2 possibilities for a tic-tac-toe.
One can easily also check the other options for X to go wrong and will confirm that only an X in the middle will hold a draw, but to keep the answer brief I will not proof it here.
Then, we can put our next "O" in the top middle, leading to
O O .
. X .
. . .

where the second player needs to put an "X" in the top right (chance 1/6 for a random opponent)
Then we place our "O" in the bottom left to simultaneously deny "X"'s thread and make a thread on our own:
O O X
. X .
O . .

So at this point a random opponent has a 1/4 chance to save the game. After that, we have to play middle right to draw, as we have no more chance to get any Tic-Tac-Toe even with cooperative play from X.
The chance to win is therefore 1 - 1/8*1/6*1/4 = 191/192.
Remark: One can easily see that this is the best strategy for a win. The total number of moves for "X" are 384 (8 on turn one, then 6, then 4, then 2), out of which with this strategy only 2 ways draw. When "O" starts in the middle, with perfect play all 4 corners can draw for "X". Analogously one can justify every subsequent decision for "O".


text:
Can I treat "experience" in reinforcement learning as "training data" in statistical learning?
Statistics is a branch of mathematics that extracts useful information from data. The data is generally called as "training data" in statistical (machine) learning.
Consider the following paragraph from the section 1.1 Reinforcement Learning of CHAPTER 1. THE REINFORCEMENT LEARNING PROBLEM of the textbook Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto.

Reinforcement learning
is different from supervised learning, the kind of learning studied in
most current research in field of machine learning. Supervised
learning is learning from a training set of labeled examples provided
by a knowledgable external supervisor. Each example is a description
of a situation together with a specification—the label—of the correct
action the system should take to that situation, which is often to
identify a category to which the situation belongs. The object of this
kind of learning is for the system to extrapolate, or generalize, its
responses so that it acts correctly in situations not present in the
training set. This is an important kind of learning, but alone it is
not adequate for learning from interaction. In interactive problems it
is often impractical to obtain examples of desired behavior that are
both correct and representative of all the situations in which the
agent has to act. In uncharted territory—where one would expect
learning to be most beneficial—an agent must be able to learn from its
own experience.

You can observe that, training data in machine learning, if we model it in a proper format, can be for reinforcement learning. But, may not be complete and practical.
I am asking this question in the view of statistical learning rather than machine learning alone. training data in statistical learning can be understood to any data, at any time instant, useful in learning.
Then, is it perfectly fine to always interpret experience in reinforcement learning as training data in statistical learning?

When you say "I am asking this question in the view of statistical learning rather than machine learning alone.", you need to clarify what you mean by both, because, from my perspective, statistical learning is just another name for ML, although you may put emphasis on statistics, or, put in another way, to me it's just ML with a statistical perspective, but given that ML and statistics overlap a lot, then the difference is really minimal if none and this is just a problem of consensus and/or terminology.

Having said that, I am not sure I understand the type of answer you're looking for. My answer would be "Yes, experience would be some type of data. We call it "experience" because we can collect more and more, as opposed to SL, where you typically have a fixed dataset, while in RL, at least in simulation, this is not the case, although with things like experience replay, one may think of it as collecting a dataset, but this dataset is not fixed and can actually change anyway." I will not post this as a formal answer until you clarify the type of answer you're looking for.

@nbro Although they are same, I am guessing that there may be some differences if we go too deep in to formal aspects. So. to keep them intact at all levels. I used two different names.

If I always start interpreting experience in RL as the data in statistical AI, then will I get contradiction somewhere or not is the question.

I do not understand this question, I think because you do not clarify anywahere what you mean by "treat like" from the title. Or "interpret" in the question body. Assuming there are some useful parallels between the two terms (and I think there are), what do you intend to do with that information? Are you looking for ways to transform between supervised learning and RL, or trying to understand the differences between RL experience and a supervised learning dataset?

@NeilSlater trying to understand difference between RL experience and SL data.

The main similarity between reinforcement learning experience and supervised learning datasets, is that both consist of a set of records. These records are commonly expressed as vectors of numbers for use in the algorithms. In addition, reinforcement learning that uses neural networks (or other function approximation) will typically implement some variant of supervised learning internally.
There are a few key differences between a prepared dataset for supervised learning and the experience in reinforcement learning. There are exceptions to these, but these are the usual case:

A supervised learning dataset has a fixed target value to learn by association, for each entry, e.g. a class or regression value. An individual reinforcement learning experience does not - the raw tuple of state, action, reward, next state $(s,a,r,s')$ must be processed in some way to obtain a useful training target value, and this processing is not static. When training in reinforcement learning for optimal control, even with historical experience, the target values must constantly be re-assessed.

Reinforcement learning is in part a design for actively collecting experience. There is no equivalent in supervised learning where the dataset is a given.

Reinforcement learning experience arrives in groups of $(s, a, r, s')$ - state, action, reward, next state - such groups of related data within a record are called tuples. The RL records are often correlated with each other, at least initially because each time step changes things only slightly, and that can be bad when combined with supervised learning which usually assumes uncorrelated data. In supervised learning you will often have a deliberate shuffling or randomisation of dataset order to protect against this. Experience replay in deep RL is a related idea to protect internal neural networks from being exposed to training samples in sequences with correlated values.

It is possible to apply reinforcement learning to supervised learning problems in theory. You can do this by making the agent guess each labelled value as an action, and reward it with negative the loss from the supervised learning. This is generally a bad idea because it is very inefficient, and there is no matching concept of state transitions in the supervised learning problem (the agent cannot impact what state is next due to its guess). However, the fact that this is possible with very little modification to the reinforcement learning agent shows how general reinforcement learning is as a learning approach.
The inverse is not really true, you cannot normally adjust a supervised learning algorithm so that it can solve a reinforcement learning problem from the given experience. There are some edge cases, such as when learning only from previous experience to assess an existing policy, or to learn a control algorithm which only cares about immediate reward. In which case you could use reinforcement learning theory to help construct a fixed dataset and give that to a supervised learning algorithm. However, by far the most common approach is to use supervised learning approaches (e.g. a neural network) as components of the agent, and rely on reinforcement learning to generate data for them on the fly.


text:
Is it impossible to evaluate the generator distribution directly?
The following excerpt is taken from 3. The Inception Score for Image Generation from the paper titled A Note on the Inception Score.

Suppose we are trying to evaluate a trained generative model $G$ that
encodes a distribution $p_g$ over images $\hat{x}$. We can sample from
$p_g$ as many times as we would like, but do not assume that we can
directly evaluate $p_g$. The Inception Score is one way to evaluate
such a model.

The excerpt is saying that we are not directly evaluating the $p_g$, the generator distribution but trying to evaluate the model $G$.
Does the excerpt intend to say that it is practically impossible to evaluate $p_g$?

Does the excerpt intend to say that it is practically impossible to evaluate $p_g$?

No, this statement says that they assume that they are not going to directly evaluate $p_g$ for a given problem, and offer other evaluation criteria which does not require doing so, which is the point of the paper. I think the authors have chosen the wording carefully and accurately.
They have probably been motivated to publish about this approach, because it is not possible to evaluate $p_g$ in most use cases for GANs. Image generators output highly complex functions with very high dimensionality and a lot of correlation between pixel values. So the paper is of interest because it is hard (usually a practical impossibility) to define $p_g$ directly, or use something like KL divergence to assess the generator's match to the real distribution.