text:
Computer vision - Can you put more weight on a specific part of the object?
Let's say I'm looking for any item that has a certain shape (outline) in a photo. but I can further classify it only according to particular features, that most of them are expected to be shown only in a smaller area of the the object itself.
How may I give more weight, in the model, to that particular area, in order to avoid wrong classification issues?
What is the flow, and are there specific tools that should be used for that purpose?
Example:
I want to detect all triangles in the image, and try to classify them like this:
If triangle has 3 lines in its corner, it's A type. if only two lines, it's B type.
So the triangles outline composes 100% of the object, but we can see that the area where the red lines are present, is only about 10% of the object area. How may I give more weight and tell the model to carefully look for the details in that area, so it doesn't confuse A with B or vice versa, just because the other 90% of the shape is similar.
And Of course, I want the certainty level to be as close as possible to 100%, for both A and B, and to be distinguished from the other option.
So my goal is the get this output:
Purple Triangle ==> Type A, certainty, 99%. Type B: certainty: 50%
Green Triangle ==> Type B, certainty, 99%. Type A: certainty: 50%

Well I am new in the computer vision,depending, you could try to train a convolutional neural network:
you should train classifier for lines and triangles. get the values in the before the las layer, kind of "the probability that with will be triangle and probablity this could be a two lines"

So I don't know this for sure, but I think you could trains classifier that attempts to increase the classification gradients around these edges.

@jaljalvarezalvarez in my real use case there won't be actual lines, the idea is that I want to give more weight to particular elements that are supposed to be found at the base of the triangle, but those elements do not have a particular shape that I can pre-train.

Its very interesting question..

One of the main strengths of deep neural networks is that they can learn representation of the data. It should be able to figure out on it's own what are the relevant features without you telling it what is relevant. Alternatively, you can try hierarchical approach - First, detect your main object (e.g triangle), extract relevant part of that object and then do classification on that extracted part only.

@Brale What kind of mechanisms are available to use in order to instruct the tool which part of the triangle is "relevant"?


text:
Bechmark models for Text Classification / Sentiment Classification
I am currently working on a novel application in NLP where I try to classify empathic and non-empathic texts. I would like to compare the performance of my model to some benchmark models. As I am working with models based on Word2Vec embeddings, the benchmark models should also be based on Word2Vec, however I am looking for some relatively easy, quick to implement models.
Do you have any suggestions?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Training while predicting on dataset
I have been trying to figure out whether if I train a model and then while predicting is it possible to train images too just like humans
Somehow converting valid images to the dataset by asking us when an object is shown
Like the Google Photos somehow they ask us if they predicted a face correctly and then reinforces on it

Yes, this method of training a model is commonly known as online learning and specific learning algorithms have been designed for this purpose, such as, Stochastic Gradient Descent(SGD). As opposed to Batch Gradient descent, which computes gradients over the entire training set at each step, the SGD algorithm computes gradients for individual samples and updates the model's parameters.
The online learning strategy is not specific to Reinforcement Learning(RL) methods but is also suitable for the broader classification of Machine Learning algorithms. Suppose, we are given a simple model pretrained on a large Face Recognition dataset, and wish the model to continuously adapt to new faces, we may keep feeding in the new input images with their correct labels (obtained from feedback), as they appear, and apply SGD to incrementally update the model. To go about this using an RL based approach, we may penalize the model with negative rewards for every label misclassified and update it.

During prediction, if we are using the camera to detect the person instead of uploading picture to predict. Can we still make the algorithm learn? And what if the model would be detecting incorrectly

Yes, a camera can be thought of as a device capturing a sequence of images. As long as there is some mechanism to tell that the model detected incorrectly, you should be able to make it learn.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to create a Q-Learning agent when we have a matrix as an action space?
I have a 2-dimentional matrix as an action space, the rows being a resource to be allocated, and the columns are the users that we will allocate the resources to. (I built my own RL environment)
The possible actions are 'Zero' or 'One'. One if the resource was allocated to the user, Zero if not.
I have a constraint related to the resource allocation, which states that each resource can be allocated to one user only, and the resource should only be allocated to users who have requested a resource to be allocated to them, and that would be the state space which is another matrix.
A penalty would be applied if the agent violates the constraints and the episode would end and the reward would equal the penalty. Otherwise, the reward would equal the sum of all the users that were satisfied with the allocation.
I am struggling with the implementation. The agent starts by exploring, then little by little it starts exploiting. When it gets to be more exploitative, I've noticed that the action matrix's values are all set to 'One', and the penalty always has the same value from episode to episode.

How does the state space evolve? If it is fixed for an episode, or changes independenty of actions taken, then this seems more like a contextual bandit scenario than full RL. It may also be better resolved as a constraint optimisation problem using a different tool to RL, so could you clairfy whether the point here is to solve the problem or learn RL?

could you give an example ? Initial data and optimal solution.

I was thinking this strategy may work.
So, Q-learning takes vector input as state representation let's say your vector has n dimensions i.e. [$n_0$, $n_1$, $n_2$,..., $n_{n-1}$]
Now, from my interpretation you want to populate a matrix with 0 and 1's given the state vector but action-space has a high complexity e.g. an 8*8 matrix has 64 cells i.e. $2^{64}$ possible actions if you want the action to be a matrix.
I suggest this:
Fill each cell, one at a time. i.e. Your agent has only two possible action 0 and 1.
To indicate to your agent that you are at a specific cell, concatenate the row and column number to the state vector before passing it as input to the Q-learning agent.
Example:
If you original state-vector is [55, 22, 100, 4] and you have to fill cell at position (10, 30) of the matrix, the state-vector should be modified as follows:
[55, 22, 100, 4, 10, 30].
I'm not sure of the sample-efficiency of this approach though.


text:
Dynamically adapting activation function
I am training a network through reinforcement learning. The policy network learns rotations, but depending on the actual input (state), the output of the network should be restricted to be in certain bounds otherwise it mostly fails to reach these bounds. I am using tanh as last activation function. So, I wonder if there could be a way to modify this last activation function s.th. it can adaptively change bounds depending on input? Or would this have a negative impact in learning?
I would also be open for papers or publications tackling these kind of problems. Thank you for your help!

It sounds like the tanh activation function is simply wrong for your purposes, you may like to consider just the identity activation function. If your model has access to these bounds then you could either feed them into the model as extra input, or truncate your model's output, or both!

@CameronChandler I have access to the bounds for every input. I could manually truncate the output of the identity activation function, but that would be 'hard coded' in a sense... Besides feeding the bounds as input, I would rather want to tell the network: at this input state, consider only these bounds.

@pasabaporaqui depending on which state the agent is, the bounds it has to consider will change. And the (-1,1) bounds of tanh allow rotations in the whole 3D space, that's why I need to restrict them dynamically.


text:
how to handle highly imbalanced multilabel classification?
I am working on a multilabel classification in which I am having 206 labels. When I saw the percentage of the number of 1's in each label they are way less than 0.1% for each label. The maximum percentage of ones in labels is 0.034%.
Below is the distribution of percentage of one's in each labels

If I simply build a multilabel classification single model. The score it gives may be high but it got biased towards zeros very much so, it doesn't give probability of a label to be one very high. And if I want to build for each label different model, I can treat it as a bunch of imbalanced data and apply smote algorithm to each model, But I have a doubt whether can smote produce a good amount of data to balance because we know how imbalance my data is. Now, doubt is can I gave a try to autoencoders, which I heard good at fraud detection when the data is having a percentage of one's less than 1% or such. Will it perform better in my case? because if it can work well, then I will study autoencoders.


text:
How do I find the data-point with respect to a given frame?
I've been reading this paper that formulates invariant task-parametrized HSMMs. In section 3.1 (Model Learning), the task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{j=1}^F$, where $A_j$ denotes the rotation of the frame as an orientation matrix and $b_j$ represents the origin of the frame. Each datapoint $\xi_t$ is observed from the viewpoint of $F$ different experts/frames, with $\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$ denoting the datapoint w.r.t. frame $j$.
How is $\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$ derived? I understand that we must subtract $b_j$, but I'm not sure if I should pre-multiply by $A_j$ or $A_j^{-1}$, so it'd be great if someone could help me understand this better. Since $A_j$ is an orientation matrix, I'd guess that it's orthogonal, and so $A_j^{-1} = A_j^T$ - and it may just be a matter of convention (i.e. depending on how $A_j$ is defined). The details aren't clear from the paper though, and I'd appreciate any help!

You should be able to find the answer to your question in some book that describes coordinate frames and transformations in the context of robotics or related fields (such as computer vision or computer graphics). Be prepared to deal with homogenous coordinates and stuff like that. Maybe this article http://wiki.ros.org/tf/Overview/Transformations will also "prove" to you that these concepts are widely used in robotics. Now, I can't help you further, but, hopefully, you will find a good resource that describes these concepts more clearly (btw, I completely understand your doubts).

authors: Софья Моргачева


text:
How do multiple coordinate systems help in capturing invariant features?
I've been reading this paper that formulates invariant task-parametrized HSMMs. The task parameters are represented in $F$ coordinate systems defined by $\{A_j,b_j\}_{j=1}^F$, where $A_j$ denotes the rotation of the frame as an orientation matrix and $b_j$ represents the origin of the frame. Each datapoint $\xi_t$ is observed from the viewpoint of $F$ different experts/frames, with $\xi_t^{(j)} = A_j^{-1}(\xi_t - b_j)$ denoting the datapoint w.r.t. frame $j$. I quote from the abstract:

"Generalizing manipulation skills to new situations requires extracting invariant patterns from demonstrations. For example, the robot needs to understand the demonstrations at a higher level while being invariant to the appearance of the objects, geometric aspects of objects such as its position, size, orientation and viewpoint of the observer in the demonstrations."

"The algorithm takes as input the demonstrations with respect to different coordinate systems describing virtual landmarks or objects of interest with a task-parameterized formulation, and adapt the segments according to the environmental changes in a systematic manner."

Though it makes some intuitive sense, I'm not fully convinced why working with multiple coordinate systems would help us capture invariant patterns in demonstrations, and leave aside the scene-specific details. That is the goal, right? On a very high level, I see that having access to more "viewpoints" may help the robot understand the environment better, and neglect viewpoint-specific biases to focus on invariant patterns across different frames. However, this is very handwavy - and I'd love to know specific details about why using multiple viewpoints is a good idea in this case.
Thanks!


text:
XOR problem with bipolar representation
I am taking a course in Machine Learning and the Professor introduced us to the XOR problem.
I understand the XOR problem is not linearly separable and we need to employ Neural Network for this problem.
However, he mentioned XOR works better with Bipolar representation(-1, +1) which I have not really understand.
I am wondering what Bipolar representation would be better than Binary Representation? Whats the rationale for saying so?

If bipolar means values +1 and -1 only, this scheme is totally equivalent to {0, +1}, being the mapping (x+1)/2. The NN can handle both, even doing the mapping at a first layer with bias. Some activation functions could be more valid in one case or another thinking in learning performance.

xor is not separable with a single hyperplane, but it is with two. There are lots of methods different to NN able to handle this subject.

But why Bipolar representation works better for this problem?


text:
Has "deep vs. wide" been resolved?
All else being equal, including total neuron count, I give the following definitions:

wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared and they have different outputs.
deep is a series ensemble, where for the most part neurons have as input the output of other neurons and few inputs are shared.

For CART ensembles the parallel (wide) ensemble is a random forest while the series (deep) ensemble is a gradient boosted machine.  For several years the GBM was the "winningest" on kaggle.
Is there a parallel of that applied to Neural networks?  Is there some reasonable measure that indicates whether deep outperforms wide when it comes to neural networks?  If I had the same count of weights to throw at a tough problem, all else being equal should they be applied more strongly in parallel or in series?

Deep performs better because of multilevel feature extraction. At least the prevailing thought is that are hierarchy of features is implicitly extracted.

I am not sure what you are really looking for but I leave here this paper here, where some intuition into that direction is provided. This paper compares the performance of a deep learning model scaling in 3 dimensions: resolution, width and depth. As depicted in their definition:

If you go to section 3.2 you will see how scaling the different dimensions independently (resolution, width and depth) affects performance and how by performing a compound scaling the performance of the model is maximized (so there is a close relation). It is a very thorough ablation study. For me this was the paper where I understood how these Width, Depth and Resolution parameters come together.
Rethinking Model Scaling for Convolutional Neural Network: https://arxiv.org/abs/1905.11946


text:
How does vanish gradient restrict RNN to not work for long range dependencies?
I am really trying to understand deep learning models like RNN, LSTMs etc. I have gone through many tutorials of RNN and have learned that RNN cannot work for long Range dependencies, like:
Consider trying to predict the last word in the text “I grew up in France… I speak fluent French.” Recent information suggests that the next word is probably the name of a language, but if we want to narrow down which language, we need the context of France, from further back. It’s entirely possible for the gap between the relevant information and the point where it is needed to become very large. Unfortunately, as that gap grows, RNNs become unable to learn to connect the information.
it comes due to vanish gradient problem. However, I could not understand that how to vanish gradient creates an issue for RNN to not work for long-range dependencies.
Since, as I know that vanish gradient usually comes when we have many hidden layers and the gradient for the first layer usually produced too low and that affects the training process. However, everyone connects this issue with vanish gradient, technically what is the relationship RNN (long-range dependencies) with vanish gradient?
I am really sorry if it is a weird question

Vanishing gradient is: as the gradient starts to flow from the end of the network (right side of the network) to the start of the network (left side of the network), it will be multiplied by numbers less than 1 and gradually it will become weaker and weaker and when it arrives to the first layers, it's so weak that makes almost no change in initial layers parameters.
Now in case of RNNs you can unroll the network, and now you can see that it is like a deep network. For clarity look at the image below (taken from a course by Andrew NG and edited):

Red arrows show the way of gradient backpropagation and you can see that in each step they are multiplied by a number (actually a matrix is multiplied in another matrix). If this number is less than one, then it results in vanishing gradient. If this number is greater than one it will result in exposure (which can be controlled by simply clipping it to a max value). The higher the steps, the more of vanishing or exploding effect.
But they have found a solution for this problem, LSTM or GRU. What these units do is that they will make a highway for gradient to backpropagate through (in each state it will be multiplied by 1). So gradient can travel for longer distance (from French into France). Even though, the problem still exists for very long relations.
If you know what is a ResNet and how it works, you can find the same concept behind LSTM or GRU and ResNets as both make a highway for gradient to flow back.
You can have an intuition of how LSTM or GRU works by following the forward pass and assuming what they are doing as locking the concepts in memory cells. Like when the network sees the word France it will understand that it's an important word and maybe it will become handy later. So it will put the France word in one of it's memory cells and keeps it there for several steps and when it wants to guess the language it will use this memory cell to predict French not English or Persian. Then it can release that memory cell and use this cell for something else.
If you want to learn more and get better intuition, I highly recommend you to look at this link: CS231n of MIT - lecture 10

Thank you so much for your answer. I have got somehow, but still, there are some fogs/confusions in my mind. Do you have any resource for RNN, LSTM, GRU where I can find step by step calculation (Calculation on each neuron and weight calculation) on an example, such as this link [https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/], but it is a simple example on traditional NN and I need for LSTM, RNN and GRU.

Secondly, I have got the following lines from this [https://towardsdatascience.com/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21], which are as follow as, So in recurrent neural networks, layers that get a small gradient update stops learning. Those are usually the earlier layers. So because these layers don’t learn, RNN’s can forget what it has seen in longer sequences (what is the logic in between this), thus having a short-term memory.

To your first comment: You can find a simple RNN implementation here. If you have problem to understand the code in the link, make sure you understand what is RNNs by looking the video in the answer above, and if you have time you can see the related videos on coursera, taught by Andrew NG (understanding the concept worth the time). I have no link for LSTM and GRU but you can implement them if you understand them.

To your second comment: Long dependency means the (RNN) network should keep the information for longer time (the network expands) and It's impossible for a simple RNN to learn these long dependencies as it's impossible for a very deep network to learn. Maybe you are still confused, but unfortunately I don't know how to explain it in a few lines

Thank you so much for your effort and comments. I would do more effort on this then will tick "Accept the answer".

@amin If we unfold the computational graph of an RNN the weights still get updates (since they are the are reused at every timestep). The problem is not that weights don't get update, the problem is that the contributions to the updates from the earlier layers is negligible.


text:
Support Vector Machine Convert optimisation problem from argmax to argmin
I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here could help another student learning about Support Vector Machines some day.
I'm currently learning about Support Vector Machines at university and came across a weird step I could not understand. We were talking about basic SVMs and formulated the optimisation problem $\max_{w,b} \{ \frac{1}{||w||} \min_n(y^{(n)}f(x^{(n)}))\}$ which we then simplified down to $\max_{w,b} \{ \frac{1}{||w||}\}$ by introducing $\kappa$ as a scaling factor for $w$ and $b$ according to the margin of the SVM. Now our lecturer converted it without explanation into a quadratic optimisation problem as $\min_{w,b}\{\frac{1}{2} ||w||^2\}$ which I could not explain myself. I hope someone with context can help me how this is possible and what math or trick is behind this approach?

Notation information:

$w$ - weight matrix
$b$ - bias (sometimes denoted $w_0$ I believe?)
$x^{(n)}$ - Independent variable (vector)
$y^{(n)}$ - Dependent variable (scalar classifying the input in a binary classifcation as $y=1$ or $y=-1$)

Thank you very much!

Welcome to SE:AI!  I'm glad you asked here as this involves supervised learning.

So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.
Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{||w||}$ decreases, the overall value increases, i.e. we maximise it. The reason for choosing $\frac{1}{2}||w||^2$ turns out to be a less mathematic and more practical one as our optimisation algorithms used perform better on quadratic functions and the $\frac{1}{2}$ seems to be a rather arbitrary scaling choice.
If anyone has something to add, I'd love to hear any details though!


text:
What is the justification for Kaiming He initialization?
I've been trying to understand where the formulas for Xavier and Kaiming He initialization come from. My understanding is that these initialization schemes come from a desire to keep the gradients stable during back-propagation (avoiding vanishing/exploding gradients).
I think I can understand the justification for Xavier initialization, and I'll sketch it below. For He initialization, what the original paper actually shows is that that initialization scheme keeps the pre-activation values (the weighted sums) stable throughout the network. Most sources I've found explaining Kaiming He initialization seem to just take it as "obvious" that stable pre-activation values will somehow lead to stable gradients, and don't even mention the apparent mismatch between what the math shows and what we're actually trying to accomplish.
The justification for Xavier initialization (introduced here) is as follows, as I understand it:

As an approximation, pretend the activation functions don't exist and we have a linear network. The actual paper says we're assuming the network starts out in the "linear regime", which for the sigmoid activations they're interested in would mean we're assuming the pre-activations at every layer will be close to zero. I don't see how this could be justified, so I prefer to just say we're disregarding the activation functions entirely, but in any case that's not what I'm confused about here.

Zoom in on one edge in the network. It looks like $x\to_{w} y$, connecting the input or activation value $x$ to the activation value $y$, with the weight $w$. When we do gradient descent we consider $\frac{\partial C}{\partial w}$, and we have:
$$\frac{\partial C}{\partial w}=x\frac{\partial C}{\partial y}$$
So if we want to avoid unstable $\frac{\partial C}{\partial w}$-s, a sufficient (not necessary, but that's fine) condition is to keep both those factors stable - the activations and the gradients with respect to activations. So we try to do that.

To measure the "size" of an activation, let's look at its mean and variance (where the randomness comes from the random weights). If we use zero-mean random weights all i.i.d. on each layer, then we can show that all of the activation values in our network are zero-mean, too. So controlling the size comes down to controlling the variance (big variance means it tends to have large absolute value and vice versa). Since the gradients with respect to activations are calculated by basically running the neural network backwards, we can show that they're all zero-mean too, so controlling their size comes down to controlling their variance as well.

We can show that all the activations on a given layer are identically distributed, and ditto for the gradients with respect to activations on a given layer. If $v_n$ is the variance of the activations on layer $n$, and if $v'_n$ is the variance of the gradients, we have
$$v_{n+1}=v_n k_n \sigma^2$$
$$v'_n=v_{n+1} k_{n+1} \sigma^2$$

where $k_i$ is the number of neurons on the $i$-th layer, and $\sigma^2$ is the variance of the weights between the $n$-th and $n+1$-th layers. So to keep either of the growth factors from being too crazy, we would want $\sigma^2$ to be equal to both $1/k_n$ and $1/k_{n+1}$. We can compromise by setting it equal to the harmonic mean or the geometric mean or something like that.

This stops the activations from exploding out of control, and stops the gradients with respect to activations from exploding out of control, which by step (2) stops the gradients with respect to the weights (which at the end of the day are the only things we really care about) from growing out of control.

However, when I look at the paper on He initialiation, it seems like almost every step in this logic breaks down. First of all, the math, if I understand correctly, shows that He initialization can control the pre-activations, not the activations. Therefore, the logic from step (2) above that this tells us something about the gradients with respect to the weights fails. Second of all, the activation values in a ReLU network like the authors are considering are not zero-mean, as they point out themselves, but this means that even the reasoning as to why we should care about the variances, from step (3), fails. The variance is only relevant for Xavier initialization because in that setting the mean is always zero, so the variance is a reasonable proxy for "bigness".
So while I can see how the authors show that He initialization controls the variances of the pre-activations in a ReLU network, for me the entire reason as to why we should care about doing this has fallen apart.

In He's paper do you get how the justify the jump from Eq6 to Eq7? They say x is not zero mean, which means Eq7 is wrong. I don't get it.


text:
Why do we add additional axis in CNN autoencoder while denoising?
I am currently learning about autoencoders and I follow https://www.tensorflow.org/tutorials/generative/autoencoder
When denoising images, authors of tutorial add an additional axis to the data and I cannot find any explanation why... I would appreciate any answer or suggestion :)
x_train = x_train[..., tf.newaxis]
x_test = x_test[..., tf.newaxis]

Then the encoder is built from the following layers:
 self.encoder = tf.keras.Sequential([
      layers.Input(shape=(28, 28, 1)),
      layers.Conv2D(16, (3,3), activation='relu', padding='same', strides=2),
      layers.Conv2D(8, (3,3), activation='relu', padding='same', strides=2)])

 self.decoder = tf.keras.Sequential([
      layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),
      layers.Conv2D(1, kernel_size=(3,3), activation='sigmoid', padding='same')])

This question is really at the limit of being on-topic, but I will leave it open.

I hope this is still relevant. I was also recently taking this tutorial and after some searching it appears that this is required for the Conv2D layers used in the denoising autoencoder (specifically the encoder). This axis represents the number of channels present in the data.
For colour images, this would be 3 representing the red, green and blue channels. However, in this case, the images are black and white and hence only 1 channel is present.
Looking at the documentation we see the following:

When using this layer as the first layer in a model, provide the
keyword argument input_shape (tuple of integers or None, does not
include the sample axis), e.g. input_shape=(128, 128, 3) for 128x128
RGB pictures...

I hope this clears things up a bit. I am new to the world of tensor flow so my knowledge is very limited.


text:
How to choose the first action in a Monte Carlo Tree Search?
I'm working on reimplementing the MuZero paper. In the description of the MCTS (page 12), they indicate that a new node with associated state $s$ is to be initialized with $Q(s,a) = 0$, $N(s,a) = 0$ and $P(s,a) = p_a$. From this, I understand that the root node with state $s_0$ will have edges with zero visits each, zero value and policy evaluated on $s_0$ by the prediction network.
So far so good. Then they explain how actions are selected, according to the equation (also on page 12):

But for the very first action (from the root node) this will give a vector of zeros as argument to the argmax: $Q(s_0,a) = 0$ and $\sum_bN(s_0,b)=0$, so even though $P(s_0,a)$ is not zero, it will be multiplied by a zero weight.
Surely there is a mistake somewhere? Or is it that the very first action is uniformly random?

Yeah, it seems that you're right and based on the description of the paper it would indeed behave uniformly random at the very first iteration (or maybe just always deterministically pick whichever action happens to be the first one in the list). I can't find anything that would suggest otherwise in the paper, and also the pseudocode they put on arXiv suggests the same behaviour.
I can't really think of any reason why this would be better than playing according to $P(s, a)$ in the very first iteration though. Maybe it creates a tiny little bit more variety in the tree search results you get? I suppose all the stochasticity that is normally inherent to a vanilla MCTS is no longer present in MuZero (or AlphaZero), because they always run for exactly the same number of iterations, and don't have any sort of random rollouts anymore; this would at least introduce a tiny bit of variation.
On the other hand, I also can't imagine this choice really making a meaningful difference in practice, except maybe if you go down to extremely low iteration counts (like if you run fewer iterations than there are legal actions in the root node).

But as the policy network gets better and better, it would make a bit more difference no? When I think of a game of Go, where there are hundreds of actions stemming off the root node, I wouldn't want to waste time exploring bad ones from the start : /

@Ziofil I feel that it would probably not matter much in practice because this is only an issue the first time you ever visit a node, for any node. And nodes that are "good" nodes are likely to get many more than just a handful of visits anyway because the policy network will keep steering towards it if the network is good (even from nodes multiple levels higher up in the tree). But again, that's just my intuition, only way to be sure is to test it. And theoretically, I certainly don't see any advantage to doing it the way it is described either


text:
Get object's orientation or angle after object detection
I'm trying to get a detected car's orientation when object detection is applied. For instance, when we apply object detection on a car and get a bounding box, is there any ways or methods to calculate where the heading is or the orientation or direction of the car (just 2D plane is fine)?
Any thoughts or ideas would be helpful.

There is a paper face pose estimation
It uses a very straight forward technique, and very obvious augmentaions to achieve nice results.
You could use exactly the same if you have a tagged dataset for cars rather than for faces.
I was able to reproduce the results myself a while back.

I think the problem can be phrased (more generally) as a Pose Estimation Problem.  That term might help in obtaining better search results when searching for relevant papers.
One paper that I found on the given topic was this one. Even if it is maybe (for whatever reason) not what you are looking for precisely, it might still contain valuable references to different techniques that might be applicable to your problem at hand.


text:
Find object's location in an area using computer vision
I'm trying to see how to detect the location of a soccer ball in the field using the live camera. What are some ways to achieve this?
1- Assuming we have a fixed camera with a wide shot. How to find the ball location on the actuall field?
2- A camera is zooming into the ball. But we know the location of the camera and maybe it's turning angle. Can we estimate the ball location on the field using this info? Or maybe we need additional info? Can we do it with two cameras as reference points?
Any thoughts would be helpful.


text:
How can I predict an anomaly based on FFT of multiple signals?
I have 999 signals, each with separate day timestamp, each T=10s long, sampled with fs=25kHz. This gives N=250,000 samples in total.
My task was to obtain the averaged magnitude spectrum for each signal. For example, for k=100, the signal is divided into k-equal fragments, 0.1s and 2500 samples long. Then FFT is computed on all fragments and mean value is calculated for all spectral component (mean for each frequency from DC to Nyquist Frequency).
The averaged spectrum for each signal for k=100 contains 1251 values and 1251 frequency points (0-fs/2).
My question is, how can prepare the train dataset for multiple Machine Learning models based on that data, so I can predict when is the threshold time, before the failure of the machine occurs?
Do i treat each spectral component (frequency) as separate feature? Or there is a different approach ?

"mean value is calculated for all spectral component". Mean of all 100 FFTs from fragments of a signal for a specific frequency ?  You are not interested in the information from time evolution of the FFT ?

Instead of having 125,001 frequency components I want to firstly average the spectrum and have smaller dataset of (for k=100) 1251 spectral components. Maybe my approach is bad, I am a novice. If you have an advice, I will definitely listen to you.


text:
Is there a pretrained (NLP) transformer that uses subword n-gram embeddings for tokenization like fasttext?
I know that several tokenization methods that are used for tranformer models like WordPiece for Bert and BPE for Roberta and others. What I was wondering if there is also a transformer which uses a method for tokenization similarly to the embeddings that are used in the fasttext library, so based on the summations of embeddings for the n-grams the words are made of.
To me it seems weird that this way of creating word(piece) embeddings that can function as the input of a transformer isn't used in these new transformer architectures. Is there a reason why this is not tried yet? Or is this question just an result of my inability to find the right papers/repo's.

There is a pre-trained language model called ProphetNet for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.
https://github.com/microsoft/ProphetNet
Also, there are few variants on hugging face website as well
https://huggingface.co/models?search=ProphetNet


text:
What exactly are deep learning primitives?
I came across the concept of "deep learning primitives" from the Nvidia talk Jetson AGX Xavier New Era Autonomous Machines (on slide 44).
There doesn't seem to be a lot of articles in the community on this concept. I was able to find one definition from here, where it defined deep learning primitives as the "fundamental building blocks of deep networks" like fully connected layers, convolutions layers, etc.
I was curious to find out if a self-attention layer is a primitive, I came across this OpenDNN issue and one person explained that self-attention layers can be built by other primitives like inner product, concat, etc.
So my question is what exactly are primitives in deep learning? What makes a convolution layer a primitive and a self-attention layer not a primitive?

Slide 44 only says "DEEP VISION PRIMITIVES" and then they provide examples of computer vision tasks, so I am not sure where, in that slide, you saw the term "deep learning primitives".

I thought the concept of primitives would generalise to beyond just vision tasks? The second link gives a definition of deep learning primitives: https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html


text:
How to interpret the variance calculation in a Guassian process
I answered another question here about the mean prediction of a GP, but I have a hard time coming up with an intuitive explanation of the variance prediction of a GP.
Thew specific equation that I am speaking of is equation 2.26 in the Gaussian process book...
$$
\mathbb{V}[f_*] = k(x_*, x_*) - k_{x*}^TK_{xx}^{-1}k_{x*}
$$
I have a number of questions about this...

if $k(x_*, x_*)$ is the result of the kernel function with a single point $x_*$, then won't this value always be 1 (assuming an RBF kernel) since the kernel will give 1 for a covariance with itself ($k(x, x) =\exp\{-\frac{1}{2}|| x - x ||^2\}$)

If the kernel value $k(x_*, x_*)$ is indeed one for any single arbitrary point, then how can I interpret the last multiplication on the RHS? $K_{xx}^{-1}k_{x*}$ is the solution to $Ax = b$, which is the vector which $K_{xx}$ projects into $k_{x*}$, but then my intuition breaks down and I cannot explain anymore.

If the kernel value $k(x_*, x_*)$ is indeed one for any single arbitrary point, then can we view the whole term as the prior variance being reduced by the some sort of similarity between the test point and the training points?

Is it every possible for this variance to be greater than 1? Or is the prior variance of 1 seen as the maximum, which can only be reduced by observing more data?


text:
Multi-armed bandits: reducing stochastic multi-armed bandits to bernoulli bandits
Agrawal and Goyal (http://proceedings.mlr.press/v23/agrawal12/agrawal12.pdf page 3) discussed how we can extend Thompson sampling for bernoulli bandits to Thompson sampling for stochastic bandits in general by simply Bernoulli sampling with the received reward $r_t \in [0,1]$.
My question is whether such extension from Bernoulli bandits to general stochastic bandits hold in general and not only for Thompson sampling. E.g. can I prove properties such as lower bounds on regret for Bernoulli bandits and always transfer these results to general stochastic bandits?


text:
Is automated feature engineering a path to general AI?
I recently came across the featuretools package, which facilitates automated feature engineering.  Here's an explanation of the package:
https://towardsdatascience.com/automated-feature-engineering-in-python-99baf11cc219

Automated feature engineering aims to help the data scientist by
automatically creating many candidate features out of a dataset from
which the best can be selected and used for training.

I only have limited experience with ML/AI techniques, but general AI is something that I'd been thinking about for a while before exploring existing ML techniques.  One idea that kept popping up was the idea of analyzing not just raw data for patterns but derivatives of data, not unlike what featuretools can do.  Here's an example:

It's not especially difficult to see the figure above as two squares, one that is entirely green and one with a blue/green horizontal gradient.  This is true despite the fact that the gradient square is not any one color and its edge is the same color as the green square (i.e., there is no hard boundary).
However, let's say that we calculate the difference between each pixel and the pixel to its immediate left.  Ignoring for a moment that RGB is 3 separate values, let's call the difference between each pixel column in the gradient square X.  The original figure is then transformed into this, essentially two homogenous blocks of values.  We could take it one step further to identify a hard boundary (applying a similar left-to-right transformation again) between the two squares.

Once a transformation is performed, there should be some way to assess the significance of the transformation output.  This is a simple and clean example where there are two blocks of homogenous values (i.e., the output is clearly not random).  If it's true that our minds use any kind of similar transformation process, the number of transformations that we perform would likely be practically countless, even in brief instances of perception.
Ultimately, this transformation process could facilitate finding the existence of order in data.  Within this framework, perhaps "intelligence" could be defined simply as the ability to detect order, which could require applying many transformations in a row, a wide variety of types of transformations, an ability to apply transformations with a high probability of finding something significant, an ability to assess significance, etc.
Just curious if anyone has thoughts on this, if there are similar ideas out there beyond simple automated feature engineering, etc.

Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply exist with no utility, needs some guidance.
In machine learning, feature engineering is typically a search for features that improve performance at a specific task, such as classification or regression. The "intelligence" is partially in the manner of searching, and in setting the goals in the first place. Automated feature engineering typically uses fairly crude search algorithms to look for good features, such as random combination, and follows up with raw processing power in order to cover large numbers of options. Automated feature engineering also does not set the goals for any task, or feed back active behaviour from the output of ML to direct the search. An intelligent agent might actively search for data in its environment in order to test an idea (e.g. repeat an action in order to discover whether the same thing happens, or move in order to observe an interesting event better). Feature engineering is a separate issue to this.
There are some theories of general intelligence that are focused on smart pattern matching as a key component. For instance, Jürgen Schmidhuber has long been a proponent of the reward system for intelligence being compression of observations and predictions. In such a system, better pattern matching discovered by an agent is an intrinsic reward signal as it allows for better compression of world models used by the agent.
Marcus Hutter is another well-known AI researcher, who has proposed a framework called AIXI which incorporates similar ideas. An agent operating using AIXI will benefit from discovering features in its observations that improve its own predictions of what will happen next. Some form of automated feature engineering could well be core to such an agent.


text:
When training deep learning models for object detection in images, do you need a large number of images, or a large number of training samples?
I am training a deep learning model for object detection. The consensus is that the more images that you have, the better the results will be. All the tutorials that I have seen say that more images are key.
I am labeling objects in my images with Label-Img, which provides the algorithm with specific training samples on the images. For my images, I am using photos with dimensions of 1100 x 1100 pixels. In my case, I could generate anywhere between 50-100 high-quality training samples per image. For example:

In cases such as this where large numbers of training samples can be generated from a single image, do you really need several hundred images? Or can you lessen the number of images because of the number of training samples?

The short answer could be "yes, if a single image contains several instances of the object, that made thinks easier and number of training images can be reduced". However, for a good answer, could you describe what kind of NN network you use ?

@pasabaporaqui Convolutional Neural Networks (i.e. ResNet50) and RCNNs.

@ihb Do you already find the answer by yourself? If not, what is your input image dimension to the first layer of neural network and what is your kernel size? How about these suggestions,  https://datascience.stackexchange.com/questions/11390/any-rules-of-thumb-on-number-of-features-versus-number-of-instances-small-da and            https://stats.stackexchange.com/questions/10423/number-of-features-vs-number-of-observations


text:
DQN rgb input channels problem using pytorch
I've been trying to learn about CNN's and reinforcement learning and I found this project to play with: https://github.com/adityajn105/flappy-bird-deep-q-learning
I've been trying to change the code to work with RGB input instead of grayscale. Pre-precosessing part is fine, but I'm having a problem with state and next_state I guess because they're deque and when deque is appended shape is (4,H,W) because it's appended 4 times (4 frames). Problem I'm having is when I append frames to deque which are RGB it becomes something like (4,H,W,3). I tried some stuff that came to mind and that I googled and read about online, but I still had problems with dimensions. What should be done so that it works with RGB instead of grayscale?

Hello. Note that programming questions are off-topic here. We focus on theoretical aspects of AI. Take a look at https://ai.stackexchange.com/help/on-topic for more info.

You can reshape it to (12, H, W) using NumPy reshape function. By the way, this will only increase the complexity of this problem. If you want to practice RL then just get the idea from their code and try implementing on some other problem/game.


text:
Understanding neural network achitectures in policy gradient reinforcement learning for continuous state and action space
I am trying to train a neural network using reinforcement learning / policy gradient methods. The states, i.e. the inputs, as well as the actions I am trying to sample are vectors with each element being a real number like in this question: https://math.stackexchange.com/questions/3179912/policy-gradient-reinforcement-learning-for-continuous-state-and-action-space
The answer that was given there already helped me a lot. Also, I have been trying to understand Chapter 13: Policy Gradient Methods from "Reinforcement Learning: An Introduction" by Sutton et al. and in particular Section 13.7: Policy Parametrization for Continous Actions.
My current level of understanding that I can use the weights in the network to calculate the mean/means and the standard deviation/covariance matrix. I can then use them to define a multivariate Gaussian distribution and randomly sample an action from there.
For now, I have one main question: In the book it says, that I have to split the weights, i.e. the policy's parameter vector, into two parts: $\theta = [\theta_{\mu}, \theta_{\sigma}]$. I can then use each part together with a feature vector to calculate the means and the covariance matrix. However, I was wondering, how this is usually done? Do I train two separate networks? I am not sure how an architecture of this will look like. Also, I am not sure what the output nodes will be in this case. Do they have any meaning like for supervised learning?
So far, I have only found papers that talk about this issue rather theoretically like it is presented in the book. I would be very happy to understand how this is actually implemented. Thank you very much!

Welcome to SE:AI!


text:
In a face database containing multiple images per subject, how do we determine the face image which is most suited for face recognition?
Let us imagine a face database with several subjects, each subject having multiple face images. How do we determine which is the best face suitable for face recognition purposes?

This should not matter that much as long as you do enough preprocessing?
But one could use e.g. the Facenet architecture to extract the embedding vector for each face of the same subject.
Then you could compute the covariance matrix and perform PCA on it. This would give you the most significant features with which you can decide which face to take.
Alternatively, you could do something in the direction of the eigenfaces: https://en.wikipedia.org/wiki/Eigenface.

Could you elaborate on how to decide which face to take?


text:
Loss function decays linearly in segmentation MRI fascia
I am working on a segmentation of MRI images of the thigh. I am trying to segment the fascia, there is a slight imbalance between the background and the mask. I have about 1400 images from 30 patients for training and 200 for validation. I am working with keras. The loss function is combination of weighted cross entropy and dice loss (smooth factor of dice loss = 2)
def combo_loss(y_true, y_pred,alpha=0.6, beta=0.6): # beta before 0.4
    return alpha*tf.nn.weighted_cross_entropy_with_logits(y_true, y_pred, pos_weight=beta)+((1-alpha)*dice_coefficient_loss(y_true, y_pred))

When I use a value alpha greater thatn 0.5 (the weighted cross entropy) the loss rapidly decreases during the first epoch. Afterwards if slowly decreases in a linear manner.

Why is this happening? What would be a good approach reasonable approach to choose the values of alpha and beta?


text:
Can $Q$-learning or SARSA be thought of a Markov Chain?
I might just be overthinking a very simple question but nonetheless the following has been bugging me a lot.
Given an MDP with non-trivial state and action sets, we can implement the SARSA algorithm to find the optimal policy or the optimal state-action-value function $Q^*(s,a)$ by the following iteration:
$$Q(s_t,a_t)\leftarrow Q(s_t,a_t) + \alpha(r_t + \gamma Q(s_{t+1}, a_{t+1}) - Q(s_t,a_t)).$$
Assuming each state-action pair is visited infinitely often, fix one such pair $(s,a)$ and denote the time sequence of visiting the said pair as $t_1 < t_2 < t_3 < \dots t_n\dots.$ Also, let $Q_{t_n}(s,a) = X_n$ for ease of notation and consider the sequence of random variables:
$$X_0, X_1, \dots X_n,\dots $$
Can $\{X_n\}_{n\geq 0}$ be though of a discrete-time Markov Chain on $\mathbb{R}$ ? My intuition says no, because the recurrence equation will look like:
$$X_{n+1} = (1-\alpha)X_n + \alpha(r_{t_n} +\gamma Q_{t_n}(s',a'))$$
and that last term $Q_{t_n}(s',a')$ will be dependent on the path even if we condition on $X_n = x.$
However, I am not quite able to rigorously write an answer in the either direction. I will greatly appreciate if someone can resolve this issue that I am having in either direction.


text:
Why would a VAE train much better with batch sizes closer to 1 over batch size of 100+?
I've been training a VAE to reconstruct human names and when I train it on a batch size of 100+ after about 5 hours of training it tends to just output the same thing regardless of the input and I'm using teacher forcing as well. When I use a lower batch size for example 1 it super overfitted and a batch size of 16 tended to give a much better generalization. Is there something about VAEs that would make this happen? Or is it just my specific problem?

My response is based on my limited experience with VAE:
Given these networks generate random samples (z) conditioned to x, and then the decoder output (D(z)) is compared with x (||x-D(z)||), if xs in the batch does not replicate some randomness, the network will not be trained properly. In other word, there should not be a direct correspondence between x and D(z) and instead the encoder should present a probability distribution function. I hope it makes sense.


text:
Which models can I use for supervised learning with images?
I have to do a project that detects fabric surface errors and I will use machine learning methods to deal with it. I have a dataset that includes around six thousand fabric surface images with the size 256x256. This dataset is labeled, one thousand of it was labeled as NOK that means fabric surface with error, and the rest was labeled as OK which means fabric surface without an error.
I read a lot of papers about fabric surface error detection with machine learning methods, and I saw that "autoencoders" are used to do it. But as I saw that the autoencoders are used in unsupervised learning models without labels. I need to do it with supervised learning models. Is there any model that can I use for fabric surface error detection with images in the supervised learning? Can be autoencoders used for it or is there any better model to do it?

Have you already thought about Convolutional NNs? (CNNs) It sounds like you have a reasonable amount of data, so you even could start from scratch with a custom built CNN. If you are lucky maybe you can find a pretrained model for a similar problem, which you can build on.

Thanks a lot. I am working on the CNNs models right now. I think it can be useful for this project.

Papers With Code have a great summary of the tasks in computer vision and their respective State of the Art models. Also the tasks page from HuggingFace serves as a great reference.

Convolutional Neural Networks are mostly used for all kind of computer vision tasks.
Here you can find a tutorial on how to train a CNN for image classification from scratch.


text:
How to use unmodified input in neural network?
My question may be a bit hard to explain...
My neural network learns a categorical distribution, which serves as an index. This index will look up the value (= action_mean) in Input 2.
From this action_mean, I create a normal distribution where the network has to learn to adjust the standard deviation. The output of the network is a sample of this normal distribution.
Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives Nones, respectively, because the output of the net is not completely connected with the input.
Would there be a way to link my action_mean with the input value, without changing the input values itself? To describe my problem, I attached a simplified computational graph how tensorboard shows it.
I would be very thankful for any help!

This part "Since the value of action_mean is directly taken from the input, somehow the gradient can't be computed or gives Nones, respectively, because the output of the net is not completely connected with the input." is not clear to me. It's not clear to me why you can't directly connect the output of one NN to the input of another NN. What is this "action mean" supposed to represent for the second network? Are you training both NNs jointly?

@nbro It is only one network and action_mean should represent the mean value of a normal distribution. I can write it in pseudocode for better clarity:
def forward(inp1, inp2): x=softmax(layer1(inp1)) dist1 = Canonical(x) index = dist1.sample() action_mean = inp2[index] dist2 = Normal(action_mean, action_std)

@nbro where action_std is a parameter defined in init. I dont know why but at that point where action_mean is defined, the network seems to be disconnected somehow. Overall, I need a constrained Normal distribution for certain regions and these regions will be provided by the canonical distribution and input2.

When you say "I dont know why but at that point where action_mean is defined, the network seems to be disconnected somehow. ", what do you mean by "it seems to be disconnected somehow"? Are you getting an error that tells you that you cannot back-propagate? That may be because something is not differentiable in your model (don't know, just guessing). Which library are you using? PyTorch?

@nbro yes indeed, there is an error when back-propagating. I inspected the gradients and some of them are None. The problem lies in the definition of action_mean, which is just a value of input2 (depending on the index) without layers or any other operation in between. So, it cant propagate the gradient back to the input. And yes, I am using PyTorch. I hope you see my problem now...


text:
React on train-validation curve after trening
I have a regression task that I tray to solve with AI.
I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)
I understand basic principle: Look if model overfit or underfit - according change the parameters. In theory.
I would ask for help with two graphs:

If I understand correctly what is going on
How would you attack the situation.

1. Graph

I use LightGBM
learning_rate = 1
max_depth = 3
num_leaves = 2**15,
number of iterations = 4000

If I understand this model is Underfitting. The validation and training is falling, but not very much...
BUT: The number of iteration is now too large and place higher number is not ok. Learning rate is 1 (as hight as it gets). Only the max_depth is low, but if it is higher (I try 30) the graph is same, just the values are worse.
So, what to do, so that model would not underfit.

Graph

I use Neural Nets
epochs=200,
batch_size=64

The model
i = Input(shape=(100,), name='input')
x = Dense(128)(i)
x = Dense(64)(i)
o = Dense(1, activation='relu', name='output')(x)

Here I am not sure. This doesn't really looks like underfit, but more that doesn't converged.
Is this right?
So, should I create more complex model (more neurones or more layers?)?
And how much epochs do I need to see this behaviour? Because, in the beginning I use only 10 epochs, for faster development, and I thought that model is overfitting. Only when I use more epochs I see that I was wrong.
How would you start to "debug" this neural net? What would be the plan of attack?


text:
Semantic segmentation failing in small instance detection
I performed semantic segmentation with U-net. My dataset consists of grayscale images of defects. After training the dataset for I got an metric accuracy of only 0.3 - 0.4 IOU. Eventhough it is merely low it performs well enough to identify instances that are huge means the prediction performs well enough in places where there is a standard intensity change(color change) and they are bigger instances. There are many other instances where there is no color change and it occoupies only few pixels in image(smaller instances) and the prediction rate is almost 0 on these instances.
I also tried Resdiual connection in the downsampling part of U-net likewise in ResNet. But still its the same and for smaller instances I used dilated convolution blocks in between the skip connections for encoder and decoder of U-net based on some papers. But still I cannot have a higher accuracy in my network and prediction rate for smaller instances are really poor. Although I use only 350 images for training with Data Augmentations. My image size is also 256,256.
Is there any other method I can try to increase the accuracy and prediction rate for smaller instances?
Any suggestion would be helpful.

You may find it useful to categorize these smaller defects as a separate class, then introduce a class weights matrix to penalize incorrect classification of the smaller defects more heavily. If these small defects represent a very small portion of the total number of pixels in your training data, then the model may be stuck at a local minimum, where it just predicts them as zero because the loss is not heavily penalized for this. So you need to add extra penalty for the incorrect classification.
Assuming you create a new class for the small defects, the class weights matrix is implemented as a keras callback function.
If you do not want to create a multiclass problem, there is also this paper : https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5977596/#!po=32.4561
The multilevel unet architecture will not solve class imbalance if that is truly the underlying issue, but if may make the network more aware of smaller scale changes in the images you're working with.

Hi, I already use kernel_regularizer which also penalizes the weights. Then what would be the difference between class_weight and kernel_regularizer.

Edited. By class weights I mean a weight matrix applied as a callback during the final loss calculation, not a regularization layer.


text:
Multi-armed bandit problem without getting rewards
In a 2-armed-bandit problem, an agent has an opportunity to see n reward for each action. Now the agent should choose actions m times and maximize the expected reward in these m decisions. but it cant see the reward of them. what is the best approach for this?

Could you clarify - the agent is allowed $n$ time steps where it can see the rewards, followed by $m$ time steps over which it wants to maximise the reward, but gets no further feedback?

Yes, That's right!


text:
How do self-driving cars perform lane changes?
I am a bit stuck trying to understand how a lane change is performed from an operational point of view.
Let's assume a self-driving car uses an occupancy grid map for local planning, this map may even have the detected lane boundaries. It's following a slow car and decides to overtake, but how does it know where the centre of the adjacent lane is? Does it use a separate map, or is there a separate data structure that is used to keep the lane information which informs the car where the centre of the adjacent lane is?
Alternatively, does the car just decide to start drifting off to the side until it picks up the lane boundaries and then centres itself?

Note that the specific technologies or ways of achieving this may also depend on the self-driving car (or company that develops it), especially, now, where the technology isn't still very solid.


text:
Hierarchical reinforcement learning for combinatorial complexity
I want to try a hierarchical reinforcement learning (HRL) approach to hard logical problems with combinatorial complexity, i.e. games like chess or Rubik's cube. The majority of HRL papers I have found so far focus either on training a control policy or they tackle quite simple games.
By HRL I mean all methods that (among others):

split hard and complex problem into a series of simpler ones
create desired intermediate goals (or spaces of such goals)
somehow think in terms of 'what to achieve' rather than 'how to achieve'

Do you know any examples of solving logically hard problems with HRL or maybe just any promising approaches to such problems?

I am not sure if "hard logical problems with combinatorial complexity" is the right term/expression to describe games like chess or Rubik's cube, though I think I understand what you mean. Maybe the term "combinatorial optimization problems" is what you mean?

Check this paper Combinatorial Optimization by Graph Pointer Networks
and Hierarchical Reinforcement Learning.

Thanks a lot. It may be something I was looking for.

authors: ميمز - رياكشن .


text:
Minimax algorithm with only partial visibility
I'm trying to implement the minimax algorithm with alpha beta pruning on a game that works like this:

Player 1 plays (x1, y1).

Player 2 can only see the x-value (x1) that Player 1 played (and not the y-value y1). Player 2 plays (x2, y2).

An action event happens, which may change the heuristic of the current game state.

Player 2 plays (x3, y3).

Player 1 can only see the x-value (x3) that Player 2 played (and not the y-value y3). Player 1 plays (x4, y4).

Action event. The game continues with alternating starting players for a maximum depth of 10.

To do so, I have been treating each turn as you regularly would with the minimax algorithm, with each player making moves given the set of moves already played, including the possibly hidden move from the turn before. However, I've noticed that my algorithm will return moves for Player 2 that assume that Player 1 plays a certain way when, in a "real" game, it may be the case that Player 1 plays something else (and vice versa). For example, if Player 2 could guarantee a win on a given turn under all circumstances (when Player 1 plays first for that series), it might not play optimally when it assumes Player 1 will not play its maximum-strength move.
I believe it is doing this precisely because it assumes that all moves are visible (a fully visible game state). And indeed, if that were the case, the sequence of moves it returns would be optimal.
How can I remedy this?
I do not believe that a probability-based algorithm (e.g. Expectiminimax) is necessary, since the game is entirely deterministic. The partial visibility part is making things difficult, though.
Something tells me that changing the turn order in my algorithm might be a solution to this problem, since the action event is the only time the game heuristic is changed.


text:
Neural network architecture with inputs and outputs being an unkown function each
I am trying to set up a neural network architecture that is able to learn the points of one function (blue curves) from the points of an other one (red curves). I think that it could be somehow similar to the problem of learning a functional like it was described in this question here. I don't know at all what this (let's call it) functional looks like, I just see the 'blue' response of it to the 'red' input.

The inputs of the network would be the (e.g. 100) points of a red curve and the outputs would probably be the (e.g. 50) points of the blue curve. This is where my problem begins. It tried to implement a simple dense network with two hidden layers and around 200-300 neurons each. Obviously it didn't learn much.
I have the feeling that I somehow need to tell the network that the points next to each other (e.g. input points $x_0$ and $x_1$) are correlated and the function they belong to is differentiable. For the inputs this could be archieved by using convolutional layers, I suppose. But I don't really now how to specify, that the output nodes are correlated with each other as well.
At the beginning I had high hopes in the approach using Generalized Regression Neural Networks as presented here, where a lowpass filter is implemented with NNs. However, as I understood, only the filter coefficients are predicted. As I don't know anything about the general structure of my functional, this will not help me here ...
Do you have any other suggestions for NN architectures that could be helpful for this problem? Any hint is appreciated, thank you!


text:
Statistical method for selecting features for classification
I'm working on a classifier for the famous MNIST handwritten data set.
I want to create a few features on my own, and I want to be able to estimate which feature might perform better before actually training the classifier. Lets say that I create the feature which calculates the ratio of ink used between two halves of a digit. Note that with ink I mean how much white there is used (which ranges from 0-255 per pixel).

For example I would calculate the ratio between the total amount of white in the left and right halves (seperated by red line). I could also do the same with the op and bottom half, or seperate the digits diagonally. With this I can calculate the mean and standard deviation.

I imagine that the left / right ratio might give some differences for other numbers. But the ratios might all be closer to the average.
Is there some method for estimating which feature might perform better compared to others? I.e. is there a method which gives a numerical value on how "seperable" a data set is?


text:
Object Detection as a means of Anomaly Detection
Is it possible to train an Object Detector (e.g. SSD), to detect when something is not in the image. Imagine an assembly line that transports some objects. Each object needs to have 5 screws. If the Object Detector detects 4 screws, we know that one is missing, hence there is an anomaly.
Actually this is an Anomaly Detection task where there is something else than a screw (e.g. a hole), but unsupervised anomaly detectors are hard to train and not as stable as object detectors.
Is my assumption correct, that even though it is not really an object detection task, one can use such methods?

Your assumption is correct. You could use object detection. Perhaps the real question is more around whether object detection is a good way to do it vs your other alternatives. I don't have particular experience with this sort of problem but if you put it in front of me, the first thing I'll try is to train an autoencoder, and then use the embeddings from that in a binary classifier (defect/no-defect) which can be one of: SVM, MLP, use a distance metric with a hard-coded threshold.


text:
What do echo state networks give us over a generic RNN resevoir?
Slightly generalizing the definition in Jaeger 2001, let's define a reservoir system to be any system of the form
$$h_{t}=f(h_{t-1}, x_t)$$
$$y_t=g(Wh_t)$$
where $f$ and $g$ are fixed and $W$ is a learnable weight matrix. The idea is we feed a sequence of inputs $x_t$ into the system, which has some fixed initial state $h_0$, and thereby generate the sequence of outputs $y_t$. $f$ is fixed (for example, a randomly generated RNN) we can then attempt to learn $W$ in some way in order to get the system to have the behavior that we want.
Now we add the echo state condition: the system has the echo state condition iff for any left-infinite sequence $...x_{-3}, x_{-2}, x_{-1}, x_0$, there is only one sequence of states $h_t$ consistent with this input sequence.
Seen from this perspective, any training procedure that could be applied to an echo state system could be applied to a generic reservoir system. So what do we get out of the echo state condition? Is there some reason to think echo state systems will generalize better, or be more quickly trainable? Jaeger does not seem to attempt to argue in this direction, he just describes how to train an ESN, but as I've said, nothing about these training methods seems to require the echo state property.

The echo state condition states that differences in the input sequence results in separate trajectories of reservoir states.
Which means that when you make the reservoir large enough any difference in the input sequence result in a linearly separable difference in the state space.
This is quite similar to the Kerneltrick in e.g. Support Vector machines where the data becomes linearly separable in the feature space. In reservoir computing the reservoir is in a way a random feature space.
To sum up, the echo state condition ensures that the signal becomes linearly separable.

Can you state this intuition in the form of a theorem? It is not clear to me how you get linear separability just from echo state (or even what "linearly separable" really means in the context of sequence-to-sequence transformations).

In this articel (http://www.scholarpedia.org/article/Echo_state_network) Jaeger writes: "ESP states that the reservoir will asymptotically wash out any information from initial conditions." This "fading memory property" was formalized by Boyd et al (Fading Memory and the Problem of Approximating Nonlinear Operators with Volterra Series, 1985): for all epsilon there exists delta>0 and T >0 such that |F(v)(t=0)-F(u)(t=0)| < epsilon with |u(t) - v(t)| < delta for all t \in [-T,0].

where F is some time-invariant, causal Operator (the reservoir in our case).


text:
What is a good way of identifying volatile positions for a checkers game?
I am implementing an AI for a mobile checkers game, and have used alpha-beta pruning with Minimax.
Now I have the problem of horizontal effect, and need to do Quiesence search to avoid that.
Any advice on what makes a position volatile for a checkers game?
I want to consider the cases when player can take a piece, and also when any piece can be taken by opponent a volatile position, and continue searching for another depth.
Anything else?

When combatting the horizon affect, you want to consider any short term actions that will greatly affect your position evaluation. Thus, in addition to captures, you will also want to include:

When the opponent can make a king next move
When the current player can make a king next move
When the only legal moves left will lead to capture the turn after for the opponent
When the only legal moves left will lead to capture the turn after for the current player

3 and 4 are commonly called Zugzwang and play a very prominent role in high level checkers, but may be a bit more difficult to implement. However, they will contribute perhaps most of all when combatting the horizon affect.


text:
Image classification - Need method to classify "unknown" objects as "trash" (3D objects)
We have an image classifier that was built using CNN with faster R-CNN and Yolov5.
It is designated to run on 3D objects.
All of those objects have similar "features" structure, but the actual features of each object class are somewhat different one from another. Therefore, we strive to detect the classes based on those differences in features.
In theory there are thousands of different classes, but for now we have trained the model to detect 4 types of classes, by training it on data sets that includes many images from different angles for each of those 4 classes (1,000 images each).
The main problem we face is that whenever the model runs on an "unknown" object, it may still classify it as one of our 4 classes, and sometimes it will do it with a high probability score (0.95), which hinders the whole credibility of our model results.
We think it might be since we are using SoftMax, which seems to force the model to assign an unknown object to one of the 4 classes.
We want to know what will the best way to overcome this issue.
We tried adding a new, fifth "trash" class, with 1,000 images of "other" objects that do not belong to our four classes, but it significantly reduced the confidence level for our test images, so we are not sure if this is at all a progress.


text:
Looking for a good approach for building an automated director for a racing game spectator mode
I'm building a tool that should assist a director to broadcast a racing game. I want this tool to suggest the human director which car to focus on and with which camera (among the available ones). I can access quite a lot of data about the current race so I can extrapolate some parameters(like car positions, how many cars near to each other there are, how close they are, last time the camera was switched etc) to be used in the decision making process. I would like the AI to learn from the human director in order to suggest him according to his "direction style".
My idea is to split the problem in 2 sub problems: the first is the choice of the car to focus on and the second is the choice of the camera to use (or cameras, since is fairly common to switch cameras while following the same car). My plan was to use some sort of Q-learning, rewarding the AI whenever one of the generated suggestions is chosen by the director but I guess it would be really difficult to define a set of states and moreover it would probably take ages before it would start to give some useful suggestions.
Are there some other good approaches I could consider? I'm also thinking about using a neural network so maybe the learning process would be faster.


text:
Training a model to identify certain differences between images?
Newbie to CV here so sorry of this is basic. Here's the deal, I have a program that I run many times. and each run I produce a screenshot. I need to compare screenshots from N-1 and N runs and make sure they aren't different in any dramatic way. Of course there are some minor changes like logos and pictures getting updated, etc.
SO far I've used something as simple as absdiff from opencv to highlight the difference regions and then use some sort of threshold to determine whether something passes or not. But I want to make it slightly intelligent but I'm not 100% sure how to proceed. Google hasn't yielded ghe best answers.
Essentially, I want to train the model on many different pairs of images and have the output be binary, yes or no depending on whether it should pass or not. In theory, I should be able to plug in 2 images and based on previous training, it should be able to tell me whether there is significant difference or not. What are some ways I might approach this, particularly with regards to what kinds of models to use.
The requirements here might seem amorphous but that's kinda the nature of the problem. the differences could be, in theory, anything. I am hoping that there will be patterns between different images and that a model would pick up on that. Things like the name of a document is 045 instead of 056 or a logo is slightly updated.

Cross-posted: https://ai.stackexchange.com/q/24957/1794, https://cs.stackexchange.com/q/132831/755.  Please do not post the same question on multiple sites.

If your image just slightly changes, all you need is the simple algorithm, just find the keyword "k-Nearest Neighbor" or take a look at this link.
To locate where is the difference, you can find the difference by subtracting two images by this script:
def extract_diff(imageA, imageB):
    '''
    Find the different between two image:
        + Input: two RGB image
        + Output: binary image show different between two image
    Assume the different between two image in each channel will be bigger or equal 30
    '''
    subtract = imageB.astype(np.float32) - imageA.astype(np.float32)
    mask_motion = cv2.inRange(np.abs(subtract),(30,30,30),(255,255,255))
    # mask_motion[mask_motion==255] = 1 # scale to 1 to reduce computation
    return mask_motion

And locate where the position with value 255 in the image. Next time you should this kind of question in the stack overflow community, there is a more active community, they can give you a better solution than mine.


text:
How to build a DQN agent with state and action being arrays?
I have a Reinforcement-Learning environment where the state is an array of 0s and 1s with length equals to the number of users the agent must satisfy (11 users).
The agent must choose one of 12 resources for the 11 users according to the state array. If state[0] == 1, that means that user0 needs a resource, so the agent must choose a resource out of the 12 resources it has. So, the action array's first element would be, for example: action[0] = 10, which means that resource 10 was allocated to user0.
If the next user (user1) is asking for a resource as well, then the number of resources to choose from is 12 - 1, in other words, because resource10 was already allocated to user0, it cannot be allocated to another user.
If state[X] == 0, it means that userX is not asking for a resource, therefore it must not be allocated any resource.
An example of a state array:
[1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0]

An example of an action array according to the state array example: (resource count starts at 0 | -1 indicates no resource was allocated)
[10, 2, -1, -1, -1, 3, 11, 5, -1, -1, -1]

I'm new to Reinforcement Learning and Deep Learning, and I have no idea how to translate that into a neural network.

Let's assume you would like to work with a classic DQN. You need to train the Q-network where inputs are the states and actions. The DQN is a function of Q(states, actions). The network is supposed to predict Q-value. The agent must pick up the action that produces the highest Q by giving the all possible actions to the network, in your case. Let's assume the current state is
[1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0]

You must pick up an item for the user which state is 1? you need to generate the all possible actions, for example,
[1, 2, -1, -1, -1, 3, 4, 5, -1, -1, -1]
[12, 1, -1, -1, -1, 2, 3, 4, -1, -1, -1]
[11, 12, -1, -1, -1, 1, 2, 3, -1, -1, -1]
...

You will need to generate all of these possible actions into the Q-network. Your worst case would be around 479 million possible actions to calculate in DQN.
If you would like to implement DQNs for this problem, I do recommend you to check out REINFORCE -> DQN -> Advantage Actor-Critic (A2C), which is the combination of policy network and DQN. With A2C, you can produce continuous actions with a policy network, action in range a[i]∈(0,1)*12, then probably round the generated actions. Then, the next problem is the policy network may produce an impossible action. So, you might skip to the nearest possible item.
For further reading on continuous action domains, I recommend to check out DDPG, PPO

I'm struggling a bit with building the neural network itself, I get that I should use CNN, but I'm a bit confused as to what layers to use and how the input and output layers would look like.

If your inputs are not images, You don't need CNN. the main reason of using CNN is to extract the features, like in Atari paper where inputs are pixels of the game. I see your states and actions are raws, you probably use need to build FCC (Dense layers). here is a nice explanation.


text:
How should I use deep learning to find the rotation of an object from its 2D image?
I have 6600 images and I am supposed to know the rotation of the object in each image. So, given an image, I want to regress to a single value.
My attempt: I use Resnet-18 to extract a feature vector of length 1000 from an image. This is then passed to three fully-connected layers: fc(1000, 512) -> fc(512, 64) -> fc(64, 1)
The problem I am facing right now is that my training loss and validation loss immediately go down after the first 5 epochs and then they barely change. But my training and validation accuracy fluctuates wildly throughout.
I understand that I am experiencing over-fitting and I have done the following to deal with it:

data augmentation (Gaussian noise and color jittering)
L1 regularization
dropout

So far, nothing seems to be changing the results much. The next thing I haven't tried is reducing the size of my neural net. Will that help? If so, how should I reduce the size?

Try going directly from the 1000 feature vector to an output of 1. What's your loss function? It's also not unreasonable to treat this as a classification problem, but a resolution of 1 degree (ie, just have 360 output nodes), but this depends on how diverse your data is. Even so, it's worth a try because a lower resolution but accurate network is better than one that hardly fits at all.

Also, it's super super important when you augment you aren't flipping the image, as this could really mess with the true rotation