text:
A2C Critic Loss Interpretation
I'm working on an Advantage A2C implementation, and I just finished creating the value network $\hat{V_{\phi}}$. I train this network with the standard MSE loss of discounted rewards-to-go:$$\|\hat{V_{\phi}}(s_{t'}) - \sum_{t=t'}^{T}\gamma^{t-t'} r(s_t, a_t)\|^2$$
I would like to be able to evaluate and assess the performance and ability of the value network as I train, especially to see how that relates and interacts with the changes and improvements in the policy, however I'm not sure how to do this.
My first instinct was to track the loss after each batch of experiences, but this doesn't work. As the policy improves, episodes last longer, and the value loss increases. I understand why this happens, as it is harder to predict the rewards-to-go when the length of the future is more undetermined.
To fix this, I tried dividing the loss that I'm computing by the total number of steps in the batch of episodes. However, the loss is still increasing as the policy improves (as the episodes get longer). Why is this still happening? Then, is there anything I can do to get a better assessment of the quality of the value network?

Unfortunately no, the way to go is track the total reward and see if it's increasing and converging eventually. Value loss isn't a useful metric as the loss can be 0 when the value network always predicts 0 and the agent doesn't collect any reward, meaning very poor behavior.


text:
How to know when a Environment will yield a deterministic model
Given enough experiment data on time taken for objects to fall to earth from different heights, one can create various models that will accurately predict the time it will take for an object falling at any height (in the inner the atmosphere, this is a toy example).
In this simple example the model is deterministic, it will always produce an output given an input regardless of the amount of data over a small threshold—something akin to Newton’s gravity equation.
Try modelling something like the stock-market (the other extreme end of the scale) and the predictions will never reliably converge on a predictable accurate model.
Is there any way of knowing whether your domain will yield a deterministic or non-deterministic model or not?

I guess I want to know two things: is there ever a way to reduce a complex model into a quick equation given my example above where within a tolerance it can be reduced to an equation. And two whether there are ever deterministic models and how to know: e.g. if I had a load of data for the Fibonacci sequence, would there be a way of knowing that there is a simply representation rather than the complex model I could have produced?

Just wonder if there if you are expecting a determinsitic outcome (e.g. like a squence on numbers with an underlying pattern) whether there would be a good enough method of reducing the problem to a simplified equation using a ML method, or whether this is not really a thing?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
One end to end Neural network or many task-specific ones?
Is it better to train one neural network for a dispersed labeled data with large number of classes or first classify data by unsupervised learning then train each part by a separate NN?
I mean by unsupervised learning we help each NN to classify in lower dispersed data with lower number of labels.
So for test data  the class of data is found by unsupervised learning then the final label is found by the network associated with that class.
Does this question generally have an answer or it depends on data and needs to be answered in practice?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Training a reinforcement learning model with multiple images
I am tentatively trying to train a deep reinforcement learning model the maze escaping task, and each time it takes one image as the input (e.g., a different "maze").
Suppose I have about $10K$ different maze images, and the ideal case is that after training $N$ mazes, my model would do a good job to quickly solve the puzzle in the rest $10K$ - $N$ images.
I am writing to inquire some good idea/empirical evidences on how to select a good $N$ for the training task.
And in general, how should I estimate and enhance the ability of "transfer learning" of my reinforcement model? Make it more generalized?
Any advice or suggestions would be appreciate it very much. Thanks.


text:
Object size identification and maximum number of classes with convolutional neural networks
I am working on a project that involves using a ConvNet to identify screws. I am able to train from scratch a ConvNet based on the first version of the inception network, but shallower (only 3 inception modules), and at the moment classifying only 45 different screws (the goal is to cover a significant part of a catalog containing ~ 4000 different itens).
My training set consists of rectangular grayscale images of the screws (150 x 300 pixels), approx. 700 images for each class.
The prototype of this model has been working pretty well with 45 classes (test set accuracy ~98%), but I am starting to worry about two things:
1) Many screws in the catalog have similar shapes, but different sizes, so the production model will have to be able to infer the scale of the objects. This is important because future users will image screws with different smartphone cameras, yielding different screw sizes in the images fed to the ConvNet. I haven't been able to find much about this in the literature. And from what I have read about ConvNets, they are good at detecting shapes, which mean that two objects, the first 1 meter long and the other 1 centimeter long, would be considered "equal" by a ConvNet if they appeared similar in an image. One (not very elegant) solution I imagined would be to include a scale in the training images, by means either of a ruler or a common object (a coin, for example). Anyway, I wonder if this problem has a simple solution, since I believe many people might have faced it.
2) All of the notorious ConvNets I know of are trained with the ImageNet dataset, which comprises 1000 different classes. My screw dataset will ultimately have more than that. Is that an issue? Assuming I have the hardware resources to train very large fully connected layers and softmax output layers, is there an upper bound to the number of classes a ConvNet can identify?


text:
Are hadoop ecosystem tools main goal is to break up large data sets into fast readable files?
I am  new to big data theory, and during the past 3 days, I took an official big data course with some of the best instructors available in my country in this domain. The things was little bit obscure for me as I have an engineering background but with no knowledge in AI techniques and domains.
After getting an intro on big data and the 5Vs (Volume, Velocity, ...), we got an intro on hadoop and hadoop ecosystem tools (Hive, Pig, ...). Then a simple example on how to run MapReduce Java script on small data file.
So to make things clear with me, are hive, pig and other hadoop ecosystem tools, are tools to break up my large data files from different sources and servers into fast-readable files, by which we create new tables with our required fields to use them later on in machine learning scripts and feature extractions ?
P.S. by fast readable files I mean, using a scripting tools that normal relational database tools like SQL and oracle don't have it on huge data sets (1 terrabytes and above) to manage and get info from it as fast as possible ?


text:
How do GANs create an image with specific characteristics?
I've seen GANs that do things like convert an image to a painting or this GAN here https://make.girls.moe/#/ that takes in a set of characteristics and generates a waifu with those characteristics.
My understanding of a GAN is that the generator upsamples random noise and the discriminator detects if an image is within the real or fake. So if the generator say, generates a waifu with the wrong hair color, how would the discriminator know?


text:
Will balancing dataset of images for object detection for a single-shot OD (Yolov3-spp) by cropping lower the quality of the model?
I'm trying to work out an approach to balancing my dataset, which is a subset of a google openimages - some classes are represented orders of magnitude more than others and I am hesistant to simply throw away data. The approach I want to try next is roughly:
Add a single instance of every image to the training set.
Crop out instances of the majority class from entire dataset and then add these cropped images to the dataset. keep adding these cropped images to training set until number of instances of 2nd majority class is close to number of instances of 1st majority class.
Crop out instances of second majority class and repeat (2)
keep going until all images have been cropped down to the minority class and dataset is roughly balanced.

This approach would mean that no instance of any class would be repeated more often than any other instance of the same class. My concern is that the minority classes will end up being predominantly represented in training by heavily cropped images. I'm not sure but I don't think this would matter in a two-stage object detector but I'm concerned that it might be a problem for a one-stage detector if, for some classes, it was predominantly trained on images that were cropped to various degrees.
The alternative to doing this would be undersampling ie. just throwing away most of the data which I'm hesistant to do.
Anyway, it would be really great to get some opinions on this in the context of Yolov3? Is yolov3 sufficiently robust at scale-invariance that training on many examples that are cropped/larger than what they are likely to be at test time?
(I'm fairly commited to some sort of oversampling because of the results in this paper which found best results generally from oversampling with CNN classifiers as opposed to undersampling, threshold adjustment etc. https://arxiv.org/pdf/1710.05381.pdf)


text:
OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)
I'm about to create an OpenAI Gym environment for a flight simulator. I'm wondering, how to cope with the fact, that the result and reward for some action needs a considerable time to advance through the system due to the inherent time constants.
In the easy example Gym-environments (e. g. cartpole, or some games) the step can anstantly be executed and the resulting reard can be calculated.
In my continuous control system (aka flight simulator), There is some reaction time needed, until I can calculate the result from my action. E. g. When I pull the stick, it takes some time, until the aircraft lifts it's nose. So there is a considerable delay (maybe in the seconds ballpark) from commanding the action to the environment, and the earliest observation of that result and it's corresponding reward.
How can I cope with that. As far as I understood, the env.step(action) function blocks till it comes back with a new observation and a corresponding reward.

How can I cope with long lasting reward caclulations?
Is it possible to have overlapping actions somehow? E. g. command a new action every 100ms, but get the reward for that action only 1 second later. In this case there would be always 10 rewards pending.

I hope I made my point clear. Don't hesitate to ask for further details in the comments.
Any hints are welcome. Is there anything to read out in the wild dealing with a similar issue?
Cheers,
Felix

It's not on your end, as a creator of flight simulator, to worry about what action should get the credit for the reward that happened some time after the action was taken. You should return the reward when the actual event happens not when the action that caused it happened. It's the job of the reinforcement learning agent to figure that out. For example if you want to give a reward when airplane nose is at 45 degrees from the horizontal axis, you should return the reward when that event actually happens, RL agent should figure out that the crucial action happened some time ago. This may be difficult for the agent but its up to the user to use a proper algorithm and proper exploration strategy to solve the problem.

I think you got me wrong: I'm not the creator of the flight simulator, but the creator of the agent and of the gym-interface. What you said in the second half of your response is the interesting part for me: When I get you right, you propose, that the agent sees the reward corresponding to the current situation, no matter when the action happend, that led to this situation.
So the delay between action and reward shall be figured out by a proper algorithm in the agent. Will we need some special treatment in the agent, or will this turn out in the course of learning?

Strong feature of RL is that it deals well with temporal credit assignment problem, in general you shouldn't need any specialized techniques, standard deep RL algorithms like PPO, ACER, DQN, etc. should be able to work fine, some better, some worse, you should try it out. Your case isn't unique, currently in OpenAI Gym there are environments with same challenges, especially Atari games. For example Pong, you get a reward when you score against opponent, but the action that "scored" happened when you touched the ball with paddle and between those 2 moments several frames and actions happened.


text:
How to learn to sample?
Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (size around $10^{9}$), and $\hat{\pi}(s)$ is a sample from this distribution.
A detail is important : $\pi$ is not a distribution right, it's $\pi(s)$ who is one.
My question is simple : how can an algorithm learn to, given a state s, sample from $\pi(s)$ ?


text:
Does the human brain use beam search for text generation?
As far as I understand, beam search is the most widely used algorithm for text generation in NLP. So I was wondering: does the human brain also use beam search for text generation? If not, then what?

i dont think neuroscience is far enough to the point it can answer that....
beamsearch is an effective and memory constrained method of approximating the sequence that maximizes the joint probability

I think text generation uses for more different approaches than just this one algorithm. There is a lot of NLP happening which does not involve neural networks/deep learning approaches.

@OliverMason for example ... ?

Any work in text generation in the past 60 years.

@OliverMason would you mind being more specific? For example, any state of the art method which does not use beam search? Some concrete references/links will be appreciated.

I thought the answer might be no.

In this 2020 ICLR paper: The Curious Case of Neural Text Degeneration, researchers found that beam search text is less surprising compared to human natural language. And they proposed a nucleus sampling method which generates more human like text.


text:
Online normalization of database for DQN
I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: I divided it into consecutive batches of 96 steps (the window size that my NN gets as input) and I normalized each batch respectively with Z-score. However, I am unable to extend these results to an online setting, as online I only have access to the last 96 elements, and thus I can only normalize according to the last 96. This small difference actually causes a sharp decrease in the performance of my DQN, as the weights of the NN were perfectly tuned for the first normalization but are not great with the online normalized database. In a nutshell, the problem is that only every 96 steps the first normalized database and the online one are the same, for all steps in between this is not happening. I have the weights for the first one, but I cannot find a way to exploit them for the online one.
What I have tried so far with the online database:

If I normalize every last 96 steps, and act for every new step (as it should be), the performances are quite bad.
If I normalize every last 96 steps, and act just every 96 steps (repeating the same action in between), the agent is actually picking the optimal action every 96 steps (like in the offline setting), so the results are somewhat decent but far from optimal for the long period between the actions. If I try with shorter periods, like 48, performances decrease sharply as it only acts optimally every 2 actions.

I don't know if there is a way to tune the optimal weights for the online database, acting directly on them without going through training again. It would be nice to understand why the NN picks its actions at each step in the optimal setting, so that I would be able to follow its strategy, but I'm not aware if it's possible to actually deduct this from the analysis of weights and features, especially for a multi-layer LSTM network.
Otherwise, I was thinking about something like normalizing the online database directly through similarities with the old batches of 96 (using their mean and std) or something like that. Anything that would help reducing the time between optimal actions to around 50-60 steps instead of 96 would be enough to provide a nearly optimal strategy, so at this point, I would consider any kind of (unelegant) method to get what I want.
I don't know if any of these is feasible, but retraining the agent is very difficult as every single time but once the agent got stuck in suboptimal strategies, this is why I am trying to get around this problem using the optimal weights I have instead of retraining.


text:
How can I train a deep learning model to predict a matrix?
I am trying to train a deep learning model to predict an 8*2 matrix. The predicted matrix would have complex values and the input matrix would be real numbers. Can it be done? Thank you for your time.

Do you have access to a large training dataset? (Or can you cheaply create one?)

I am in the process of creating one. The input would be a 6*1 vector.

You could use a CNN or Fully Connected Network and output a matrix of size 8*2*2. The first 8*2 matrix is the real number and the second is the imaginary number. Example code below uses keras.
'''
input: numpy array of shape(batch_size,input_dim_0,input_dim_1,1)
Y: keras tensor of shape(batch_size,8,2,2)

'''

model = Sequential()
model.add(Conv2d(64,3,input_dim =(input_dim_0,input_dim_1,1))
model.add(Conv2d(64,3)
model.add(Conv2d(2,3)

model.compile(loss='MSE',metrixs=['accuracy'])
model.fit(input,Y)

Thank you! I will give this a shot. Will 6*1 vector as an input create issues?

If you use keras , you should resize the input to 611. The last 1 is the channels. Also, the model should be a bit different, as the 61 need to be convolve to 82. Perhaps you could increase padding or repeat the 61 vector twice to make the input 621. Then you can make padding to 1 on the first axis and convolve the input. This will make the output 82*1.


text:
Convolutional neural network debugging
Im trying to implement CNN for small images classification (36x36x1) (grayscale). I've checked every forward/backward pass function on small example, and still my cnn is not doin any progress on training. Tests were done on learning rate [0.001 - 0.01].
Network structure:
Conv -> Relu -> Conv -> Relu -> MaxPooling -> Conv -> Relu -> Conv -> Relu -> MaxPooling -> Flattening -> sigmoid -> Fully connected -> sigmoid ->Fully connected -> softmax
Is there a mistake in forwardPass/ backwardPass function?
    def forwardPass(self, X):
    """
        :param X:   batch of images (Input value of entire convolutional neural network)
                    image.shape = (m,i,i,c) - c is number of channels
                    for current task, first input c = 1 (grayscale)
                    example: for RGB c = 3
                    m - batch size
                    X.shape M x I x I x C

        :return :   touple(Z, inValues)
                    Z - estimated probability of every class
                    Z.shape M x K x 1

    """

    W = self.weights

    inValues = {
        'conv': [],
        'fullyconnect': [],
        'mask' : [],
        'pooling' : [],
        'flatten' : [],
        'sigmoid' : [],
        'relu' : []
    }

    """
        Current structure:
        Conv -> Relu -> Conv -> Relu -> MaxPooling -> Conv -> Relu -> Conv -> Relu -> MaxPooling -> Flattening ->
        -> sigmoid -> Fully connected -> sigmoid ->Fully connected -> softmax
    """

    inValues['conv'].append(X)
    Z = self.convolution_layer(X, W['conv'][0]);z = Z

    inValues['relu'].append(z)
    Z = self.relu(z);z =Z

    inValues['conv'].append(z)
    Z = self.convolution_layer(z, W['conv'][1]);z = Z

    inValues['relu'].append(z)
    Z = self.relu(z);z =Z

    inValues['pooling'].append(z)
    Z, mask = self.max_pooling(z);z = Z
    inValues['mask'].append(mask)

    inValues['conv'].append(z)
    Z = self.convolution_layer(z, W['conv'][2]);z = Z

    inValues['relu'].append(z)
    Z = self.relu(z);z = Z

    inValues['conv'].append(z)
    Z = self.convolution_layer(z, W['conv'][3]);z = Z

    inValues['relu'].append(z)
    Z = self.relu(z);z = Z

    inValues['pooling'].append(z)
    Z, mask = self.max_pooling(z);z = Z
    inValues['mask'].append(mask)

    inValues['flatten'].append(z)
    Z = self.flattening(z);z = Z

    inValues['sigmoid'].append(z)
    Z = self.sigmoid(z); z = Z

    inValues['fullyconnect'].append(z)
    Z = self.fullyConnected_layer(z, W['fullyconnect'][0]); z = Z

    #dropout here later

    inValues['sigmoid'].append(z)
    Z = self.sigmoid(z); z = Z

    inValues['fullyconnect'].append(z)
    Z = self.fullyConnected_layer(z, W['fullyconnect'][1]);z = Z

    Z = self.softmax(z)

    return Z, inValues

Backpropagation:
 def backwardPass(self, y, Y, inValues):

    """

    :param Y: estimated probability of all K classes
                ( Y.shape = M x K x 1 )
    :param y: True labels for current
                M x K x 1
    :param inValues: Dictionary with input values of conv/ff layers
                     example: inValues['conv'][1] - Values encountered during feedForward on input of Conv layer with index 1
    :return:  Gradient of weights in respect to L
    """

    np.set_printoptions(suppress=True)
    W = self.weights

    G = {
        'conv' : [],
        'fullyconnect' : []
    }

    Z = self.softmax_backward(Y, y); z = Z

    Z, dW, dB = self.fullyConnected_layer_backward(z, W['fullyconnect'][1],inValues['fullyconnect'][1]);z = Z
    weight = {
        'W': dW,
        'B': dB
    }
    G['fullyconnect'].append(weight)

    Z = self.sigmoid_deriv(z, inValues['sigmoid'][1]); z = Z

    Z, dW, dB = self.fullyConnected_layer_backward(z, W['fullyconnect'][0],inValues['fullyconnect'][0]);z = Z;
    weight = {
        'W': dW,
        'B': dB
    }
    G['fullyconnect'].append(weight)

    Z = self.sigmoid_deriv(z, inValues['sigmoid'][0]);z=Z

    Z = self.flattening_backward(z, inValues['flatten'][0]); z = Z

    Z = self.max_pooling_backward(z,inValues['mask'][1]); z = Z

    Z = z * self.relu(inValues['relu'][3], deriv=True); z = Z

    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][3],inValues['conv'][3]); z = Z
    weight = {
        'W': dW,
        'B': dB
    }
    G['conv'].append(weight)

    Z = z * self.relu(inValues['relu'][2], deriv=True);z = Z

    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][2],inValues['conv'][2]); z = Z
    weight = {
        'W': dW,
        'B': dB
    }
    G['conv'].append(weight)

    Z = self.max_pooling_backward(z,inValues['mask'][0]);z = Z

    Z = z * self.relu(inValues['relu'][1], deriv=True);z = Z

    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][1],inValues['conv'][1]); z = Z
    weight = {
        'W': dW,
        'B': dB
    }
    G['conv'].append(weight)

    Z = z * self.relu(inValues['relu'][0], deriv=True);z = Z

    Z, dW, dB = self.convolution_layer_backward(z, W['conv'][0],inValues['conv'][0]); z = Z
    weight = {
        'W': dW,
        'B': dB
    }
    G['conv'].append(weight)

    G['conv'].reverse()
    G['fullyconnect'].reverse()

    return G

update:
    def update(self, alfa, W, G):

    W['fullyconnect'][0]['W'] -= alfa * np.sum(G['fullyconnect'][0]['W'],axis=0)
    W['fullyconnect'][1]['W'] -= alfa * np.sum(G['fullyconnect'][1]['W'],axis=0)
    W['fullyconnect'][0]['B'] -= alfa * np.sum(G['fullyconnect'][0]['B'],axis=0)
    W['fullyconnect'][1]['B'] -= alfa * np.sum(G['fullyconnect'][1]['B'],axis=0)

    W['conv'][0]['W'] -= alfa * np.sum(G['conv'][0]['W'],axis=0)
    W['conv'][1]['W'] -= alfa * np.sum(G['conv'][1]['W'],axis=0)
    W['conv'][2]['W'] -= alfa * np.sum(G['conv'][2]['W'],axis=0)
    W['conv'][3]['W'] -= alfa * np.sum(G['conv'][3]['W'],axis=0)
    W['conv'][0]['B'] -= alfa * np.sum(G['conv'][0]['B'],axis=0)
    W['conv'][1]['B'] -= alfa * np.sum(G['conv'][1]['B'],axis=0)
    W['conv'][2]['B'] -= alfa * np.sum(G['conv'][2]['B'],axis=0)
    W['conv'][3]['B'] -= alfa * np.sum(G['conv'][3]['B'],axis=0)

    return W

Can you show what error you're facing? Does the model compile? I mean, does it even 'start' training, or do you face matrix dimensionality issues?


text:
What qualifies as 'fitness' for a genetic algorithm that minimizes an error function?
Suppose I have a set of data that I want to apply a segmented regression to, fitting linearly across the breakpoint. I aim to find the offsets and slopes of either line and the position of the breakpoint that minimizes an error function given the data I have, and then use them as sufficiently close initial guesses to find the exact solutions using a curve fit. I'll elect to choose the bounds as the mins and maxes of $x$ and $y$ of my data, and an arbitrary bound for a slope with $slope = a * \frac{y_{max}-y_{min}}{x_{max}-x_{min}}$ for a suitable $a$ where I can safely assume the magnitude of $a$ is greater than any possible slope that realistically represents the data. Let's suppose I define a function (in Python):
 def generate_genetic_Parameters():
            initial_parameters=[]
            x_max=np.max(xData)
            x_min=np.min(xData)
            y_max=np.max(yData)
            y_min=np.min(yData)
            slope=10*(y_max-y_min)/(x_max-x_min)

            initial_parameters.append([x_max,x_min]) #Bounds for module break point
            initial_parameters.append([-slope,slope]) #Bounds for slopeA
            initial_parameters.append([-slope,slope]) #Bounds for slopeB
            initial_parameters.append([y_max,y_min]) #Bounds for offset A
            initial_parameters.append([y_max,y_min]) #Bounds for offset B

            result=differential_evolution(sumSquaredError,initial_parameters,seed=3)

            return result.x

        geneticParameters = generate_genetic_Parameters() #Generates genetic parameters

fittedParameters, pcov= curve_fit(func, xData, yData, geneticParameters)

This will do the trick, but what is the implicit standard of fitness that the differential evolution here deals with?


text:
What is wrong with this CNN network, why are there hot pixels?
I'm building a CNN decoder, which mirrors (in reverse) the VGG network structure from Conv-4-1 layer.
The net seems to be working fine, however, the output looks broken. Please note that the colour distortion is fine, it's the the [255/0 RGB pixels] e.g. green that I'm worrying about.
I tried to overfit a single image, but even then I get these hot pixels. Does anyone know why they appear?

My net:
    activation = 'elu'

    input_ = Input((None, None, 512))
    x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', bias_initializer='zeros', activation=activation)(input_)

    x = UpSampling2D()(x)
    for _ in range(3):
        x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)

    x = UpSampling2D()(x)
    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)

    x = UpSampling2D()(x)
    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=3, kernel_size=self.kernel_size, padding='same')(x)

    model = Model(inputs=input_, outputs=x)

I've seen this too many times - it's not a problem with your network, it's a problem with matplotlib and how it displays the image. You are probably trying to display a float with range $<0, 255>$. When matplotlib sees float type as input, it assumes a range of $<0, 1>$, and thresholds everything outside of that range, and the results you can see.

Before printing, I do y = np.clip(y, 0, 255) and assert y.dtype == np.uint8, 'Must be uint8', so these pixels correspond to [0, 0, 0] to [255, 255, 255] where 0/255 are sometimes interchanged to produce yellow/cian etc.

Mind sharing the full code?

Unfortunately I can't as it would turn into a debugging exercise. I thought it's a common problem that has a known reason for happening. I guess the issue is somewhere in the code. I'm partially afraid that I'm stuck in local minima, but I doubt it.

Trust me, I'll spot your mistake in under 60 second. There's no way a network-related issue results in something like this. It's something with how you process the data.

Here is the network -> https://github.com/skiler07/mst-tf/blob/master/trainer/MST.py I am implementing a paper. The preprocessing is quite complex. If you wish to reproduce, you can clone master and try running locally.


text:
How does adding a small change to an neuron's weighted input affect the overall cost?
I was reading the following book: http://neuralnetworksanddeeplearning.com/chap2.html
and towards the end of equation 29, there is a paragraph that explains this:

However I am unsure how the equation below is derived:

The expression you give in your final statement is the definition of a math derivative. Partial, because cost depends on all other network parameters, and using $\Delta$ instead of differential, but it is the concept of derivative:change in result due to infinitesimal change in input.

@pasabaporaqui It's not the definition of derivative. The problem with that notation is that if you change a parameter by a certain additive amount the derivative will not change by the same multiplicative factor (in general).

I think for state functions where you can write down the total differential, using Euler's Rule and the Chain Rule, C depends upon z. So, the infinitesimal change in C, dC in some arbitrary dimension is given by:
dC = (delta C/delta z)*dz

@nbro: $ df = f' dx$, $ \Delta f = f' \Delta x$, $f' = lim ( \Delta x -> 0 ) \Delta f / \Delta x$, ... lots of ways to write same concept: change in result due to small change in input.

@pasabaporaqui Ok. However, you have to look at the context. Nielsen states that if you add $\Delta z$ to $z$, then the derivative changes by the same multiplicative factor, but this is not, in general, true. Furthemore, the text is really ambigous. He states "the overall cost". The overall cost of what?

I think it works for this case because C is only dependent on z. So if you have a graph of cost vs. z, then a small change in z results in a small change in C. Thus, the total (overall) derivative of C wrt z is equivalent to the partial derivative of C wrt z. So dC/dz = delta C/delta z. Hence, dC = (delta C/delta z)*dz if you multiply both sides by dz.

C does not only depend on z. The network has a lot of nodes in that example.

C in this example is a function of a single variable (z)

@flying_costa No, it is not. Why do you think so? $C$ is the cost function and it is a function of all variables of the neural network.

@flying_costa: there are several usual ways to generalize same concept to functions of several arguments: partial derivative, gradient ( $\nabla $) ... In the text you quote, partial derivative is used for all parameters $z_i^j$ with $i$ in ... and $j$ in ....

I believe he's just saying that:
$$
\frac{\partial C}{\partial z_j^l} \Delta z_j^l \approx \frac{\partial C}{\partial z_j^l} \partial z_j^l \approx \partial C
$$
so that the change in cost function can be arrived at simply for a small enough perturbation $\Delta z_j^l$.
Or, taking that line of approximations backwards, the change in the cost function for a given perturbation is just:
$$
\partial C \approx \frac{\partial C}{\partial z_j^l} \partial z_j^l \approx  \frac{\partial C}{\partial z_j^l} \Delta z_j^l
$$

The $\approx$ in $\frac{\partial C}{\partial z_j^l} \partial z_j^l \approx \partial C$ can be replaced with an $=$ (it is not just an approximation, it is an exact equation).

How does this answer the question? You're just approximating $\partial C$. It doesn't not explain why, if you add $\Delta z$ to $z$, then the derivative of $C$ w.r.t. $z$ needs to be multiplied by $\Delta z$.

@nbro To respond to your first comment: Well, I wrote it like that because it's not a very mathematically rigorous thing to do to write it with strict equality. It's probably OK here, but I haven't dug into it.

@nbro I'm confused by your second comment: It doesn't not explain why ??? Not sure what you're saying.

Why isn't it a mathematically rigorous thing? It is just another way of writing $\frac{\partial C}{\partial z} = \frac{\partial C}{\partial z}$.

@nbro See this question and its answers over on SE.ME.

If you look at the context of the question, you will that the deamon adds $\Delta z$ to $z$, so Nielsen (the author of that article) states that "the overall cost changes by $\frac{\Delta C}{\partial z}\Delta z$". First of all, it is not clear what he means by "overall cost". Second, I think that your answer does not explain the relation between the addition of $\Delta z$ to $z$ and the fact that the "overall cost" is now $\frac{\Delta C}{\partial z}\Delta z$.

Ok, thanks for pointing me to that question. I was not aware of this. Anyway, a lot of people do it. For example, in integration by substitution, one manipulates infenitesimals.

I think that Nielsen just wanted to convey the idea of the back-propagation algorithm using that formula, as you can read from the next paragraph "Now, this demon is a good demon...", so I don't think that that partial derivative is mathematically correct, provided the partial derivative is still with respect to $z_j^l$.
$C$ is the cost (or loss) function. $z_j^l$ is the linear output of neuron $j$ in layer $l$, which is followed by a non-linear function (e.g. sigmoid), denoted by $\sigma$. So, the actual output of neuron $j$ in layer $l$ is $\sigma(z_j^l)$.
The partial derivative of the cost function $C$ with respect to this neuron's linear output, $z_j^l$, is $$\frac{\partial C}{\partial z_j^l} = \frac{\partial C}{\partial z_j^l} 1 = \frac{\partial C}{\partial z_j^l} \frac{\partial z_j^l}{\partial z_j^l}.$$
If the the linear output of node $j$ in layer $l$ is now $z_j^l + \Delta z_j^l$, then the partial derivative with respect to $z_j^l$ becomes
\begin{align}
\frac{\partial C}{\partial z_j^l}
&= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)}\frac{\partial (z_j^l + \Delta z_j^l)}{\partial z_j^l}  \\
&= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} \left( \frac{\partial z_j^l}{\partial z_j^l} +  \frac{\partial \Delta z_j^l}{\partial z_j^l} \right) \\
&= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} \left( 1 +  \frac{\partial \Delta z_j^l}{\partial z_j^l} \right) \\
&= \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)} +  \frac{\partial C}{\partial (z_j^l + \Delta z_j^l)}\frac{\partial \Delta z_j^l}{\partial z_j^l} \\
\end{align}
$\Delta z_j^l$ depends on $z_j^l$, but it is not specified how.


text:
Inverse Reinforcement Learning for Markov Games
This is an Inverse Reinforcement Learning (IRL) problem. I have data (observations) on actions taken by a (real) agent. Given this data I want to estimate the likelihood of the observed actions in a Q-learning agent. Rewards are given by a linear function on a parameter, say alpha.
Thus, I want to estimate the alpha that makes the observed actions more likely to be taken by a Q-agent. I read some papers (i.e. Ng & Russel 2004), but I found them rather generalistic.

Hi and welcome to this community! This is an interesting post, but what is your question exactly?

HI, thanks! My question is how could I implement an IRL algorithm, like [Ng and Russell, 2000], to a Markov game where multi-agents have opposite interests. The game is simple though, as it is a  matrix game. I read some papers on IRL, but neither of them accounts for a multi-agent framework. Actually what I ask is for any ideas on how to approach to a problem of this nature.

Have you looked into the work done by Deepmind on AlphaStar for StarCraft?
They use a combination of different methods to achieve their goal - one of which is IRL. You can read more about their outline of implementation here: https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii


text:
How is regression machine learning?
In regression, in order to minimize an error function, a functional form of hypothesis $h$ must be decided upon, and it must be assumed (as far as I'm concerned) that $f$, the true mapping of instance space to target space, must have the same form as $h$ (if $h$ is linear, $f$ should be linear. If $h$ is sinusoidal, $f$ should be sinusoidal. Otherwise the choice of $h$ was poor).
However, doesn't this require a priori knowledge of datasets that we are wanting to let computers do on their own in the first place? I thought machine learning was letting machines do the work and have minimal input from the human. Are we not telling the machine what general form $f$ will take and letting the machine using such things as error minimization do the rest? That seems to me to forsake the whole point of machine learning. I thought we were supposed to have the machine work for us by analyzing data after providing a training set. But it seems we're doing a lot of the work for it, looking at the data too and saying "This will be linear. Find the coefficients $m, b$ that fit the data."

I mean that the machine uses the training set to minimize a cost function to obtain parameters necessary to fit the data.

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy.
but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper models come into play: You allow a learn-able featurization and classification/regression. Think about it this way, the result of your regression is most likely linearly associated with some set of features, they just may not be the ones you are interested in (you can prove this actually with any infinitely wide network :: Universal approx Thm). Unfortunately we cant use an infinitely dimensional model, so we run with these giant over-parametrized models where we hope the a good function can be described by a sub-structure (only recently are we starting to pay attention, to how these sub-structures form -- look at this paper)
But the way you bring about thinking about it is a large pit fall for many trying to move forward. Alot of ML people now of days gain success by throwing a function without alot of parameters on a big data problem, but youll see the largest advancements in the field come from a theoretical understanding of the featurization and optimization.
I hope this helped

Actually regression comes under the statistical analysis.  As you know many business activity(decision making) relies in the previous trends that can be grabbed from the organizations transaction data. When regression is performed on those organizational data. One can understand what decision can be made. One could even simulate the different conditions when the regression line is generated and to predict the unknown cases, decision maker can pass the numerical values corresponding to the certain phenomena in the operation of the organization.
How regression is machine learning?
Let's start from the definition of machine learning.

Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves.

Source: https://www.expertsystem.com/machine-learning-definition/
As from the definition it becomes clear that machine learning is to know the inner insight about the data without being explicitly programmed. Doesn't it fills great to know about what my previous trends in the business related transaction data is trying to convey me.

Please note that, in machine learning algorithms like Regression, one is trying to built some relation between the transnational data.

So how relation between the data is being built?
Consider you are in the business of selling and buying the house and you want to predict the house price according to the latest trend. So what you have got is the data of house price and the feature of the house.

Feature : house_area, no_of_rooms
  Target (what you want to predict): Price

Now, you perform regression on those data and you want to find out what would be best price for the house with the feature that is not mentioned in the latest trend's data.
Suppose general regression becomes like:

price = a * hourse_area + b * no_of_rooms + some_constant

So in some sense. We're just trying to find the best fit line of the latest trend data with some variables like a, b and some_constant. Isn't it great to find such higher level of details from those trends data to know what would be the house price for non-mentioned data in the so called 'training data'.
Choosing the objective function for best mapping?
Suppose relation is sometimes is non-linear. But how my algorithms would know that. In such case, one can use Artificial-Neural Network as it can learn to hypothesize the non-linear training data too.

Note: You can learn to simulate the non linear data at: https://playground.tensorflow.org

It is just a statistical technique that is used in machine learning and it depends on the nature of the machine learning problem. I think you should be referred to the relation of the statistics and machine learning. These are no the same, but you can see the statistical methods in machine learning methods.
For your specific problem, there are a lot of optimization techniques in AI (not specifically in machine learning). So, I think you should scrutinize more on the problem to find the relation of machine learning, AI, and statistics in this regression example.

What you are asking touches upon two very different approaches to machine learning:

The empirical approach (many people just call this 'machine learning', and some people like to call it 'algorithmic machine learning')
The statistical approach (some people like to call this 'statistical machine learning')

The purely empirical approach is very goal-oriented - think discriminative models that are only used for prediction. You really only care about whether the data fits the training + test data well according to whichever metric you've selected.
The statistical approach is very process-oriented - you would want to identify the processes that generate the data, the distributions they follow, whether your results are statistically significant, etc.
Along this spectrum, most folks fall somewhere into the middle.
What you've described is closer to statistical machine learning - to practitioners of the other approach, regression only means that you are trying to predict for a continuous target variable (whereas classification would be for a discrete target variable). Then you might poke around the data a bit, fiddle around with features & hyperparameters, and try out a lot of different regression algorithms, going from OLS, SVMs, nearest neighbour regressors, random forests, gradient boosted trees, and maybe even RNNs, etc. In the extreme case, a purist of this approach wouldn't care about the statistics or whatever the underlying distributions are at all, but only care if the final regression gives good results in practice.
While there are clear risks with this approach (especially when the underlying assumptions of the models fall apart), it can give good results, especially when the practitioner is a good coder and can try out lots of possibilities very quickly, and even produce novel algorithms. The fact is that maths does sometimes lag development of other fields - Fourier analysis for example, and deep neural networks.
Another very approximate analogy would be science vs engineering.


text:
In addition to matrix algebra, can GPU's also handle the various Kernel functions for Neural Networks?
I've read a number of articles on how GPUs can speed up matrix algebra calculations, but I'm wondering how calculations are performed when one uses various kernel functions in a neural network.
If I use Sigmoid functions in my neural network, does the computer use the CPU for the Sigmoid calculation, and then the GPU for the subsequent matrix calculations?
Alternatively, is the GPU capable of doing nonlinear calculations in addition to the linear algebra calculations?  If not, how about a simple Kernel function like ReLU?  Can a GPU do the Relu calculation, or does it defer to the CPU?
Specifically, I'm using Keras with a Tensorflow backend, and would like to know what TensorFlow can and cannot use the GPU for, but I'm also interested in the general case.


text:
Training Haar Cascade model with grey vs color images
Most examples if not all, are models that have been trained with images that are turned grey. Does this mean that models only detect edges? Why wouldnt you want to keep color so that model could learn that as well?


text:
Unit integral condition on the output layer
I want to train a neural network on some input data from a probability distribution (say a Gaussian). The loss function would normally be $-\sum\log(f(x_i))$, where the sum is over the whole data (or in this case a mini batch) and $f$ is the NN function. However I need to enforce the fact that $\int_0^\infty f(x)dx=1$, in order for $f$ to be a real probability distribution. How can I add that to the loss function? Thank you!

I am given data points generated from a function (distribution) as input, and I want to train a NN to approximate that function. I am not sure if it is classification or regression. I just want to train the network to approximate f.

Both the input and the output are one single node, yes. It can be shown mathematically that given data generated by a probability distribution, the function that generated the data is the one minimizing the loss function I wrote there. I just need to impose that normalization (otherwise the constant function equal to 1 would give the smallest loss). But I am not sure how to do that.

The inputs are generated randomly from the distribution. They are not continuous, unless you generate infinitely many, which is not possible.

@AlexMarshall Your description of the function $f$ is confusing. ...f  is the NN function.... and then, ...f  to be a real probability distribution.... And, previously you desribe the loss: The loss function would normally be −∑log(f(xi)),.... I think you need to clear up the question and method a bit more.

@naive So I have an unknown probability distribution, let's say it's a gaussian distribution (this is also a function, as I can write it as f(x, mu, sigma)). I have some samples from the distribution and I want to build a NN such that given these sampled numbers as input, the NN learns to approximate the real (unknown) distribution. So if I pass a new number to the NN, it will give me something close to the value of the gaussian at that number.

@naive It can be shown mathematically that the loss function that I mentioned there is minimized only when the NN approximates the real function (intuitively, you want to find the function/distribution for which the numbers that you sampled are the most likely to appear). However, a probability distribution (such as a gaussian) must be normalized. My question is how do I impose the normalization condition to the NN such that it won't give me, for example numbers bigger than one, or numbers that would add up to more than one.

I think your first comment is clear. But, the second conflates the idea of a probability distribution function ( pdf) and probability.

From what I understood, your problem is of approximating a probability distribution function. This is not an easy task. You might find different probability distribution functions with similar losses for the training set that you provide. And, for the matter of normal distribution -- the normalizing constant can be used which is a function of the variance -- that I assume your function would have to approximate.

@naive oh sorry, you are right, I mean pdf. But yes, there could be many pdf that could produce the samples that I have with a high probability. But that's ok. I don't need the exact pdf, just something approximate. But that is mainly a second worry. the first one is how can I define a loss function itself that will force my NN the learn a good enough pdf? And in the case of the gaussian, yes, the NN should discover on its own that normalization constant. I am just not sure how to approach the problem.

For defining a loss function you need to have the actual output and predicted output. According to your description there does not seem to have an actual output. It seems this problem can be handled better through unsupervised learning techniques.

@naive But what is wrong with the loss I suggested? Shouldn't the network get pretty close to the tru pdf by using that loss?


text:
Prediction of values with an unsupervised model
Given a set of historical data points, I am trying to predict a continuous output of which I have no historical record of, therefore the problem is of an unsupervised nature.
I am wondering if there is any method or approach I should take to tackle this problem? Essentially, how to build a model that will provide an output that is not clustered?

I would add details about your available data as well as the way you are planning to frame the problem(i.e unsupervised anomaly detection).


text:
Make 9 AIs to replace Supreme Court justices
Since the supreme court is always political, why not program 9 AI robots that use different methods to determine whether a law is constitutional and the outcome of cases. How would engineers go about building this? Would it work?

AI is sending people to jail—and getting it wrong

This is how AI bias really happens—and why it’s so hard to fix

Politics are driven by a combination of things such as culture, ideology, religion, ethics, morality, desire for money, fame, and power, etc.  Although God has one truth, most people don't know this truth, a few do, and the rest either disagree about it or just don't believe in Him. The Constitution was written by man.  It does not tell us what to do in every situation. It is open to interpretation.  A person's interpretation of the Constitution is driven by his morality, culture, and all the things I mentioned above. AI systems are designed by people so they are driven by their biases.

It isn't my assumption but the definition of God.  God is perfect and therefore has a perfect truth.  For how can there be truth that conflicts with itself?  Can you have 0=1?  Yes=no?  Science was born from philosophy but they were originally one and the same.  This is why scientists who earn a doctorate, typically earn a Ph.D. (doctor of philosophy).

To have an unbiased AI system means you must know the truth and societies in general can't agree on the truth.

Explain the conflict.


text:
How can I interpret the following error graph?
I am training a neural network which produces the following errors (epoch number on the x axis). I have some questions regrading interpreting it.

When I say model.predict is it giving me the result based on the final state (that is epoch 5000)?
Towards the end (and some places in the middle) there are places where the training error and validation error are farther apart. Does this mean that the model was over-fitting on those epochs?
Based on the graph, can one determine that the model was best at a certain epoch?
Does Keras have API methods to retrieve the model at a specific epoch so that I can retrieve the best model?

@DuttaA thank you for your comment. Can you please elaborate? Also looking at the train error,there is a wave like pattern. Would you not say that the wave like pattern indicates that the model is learning (that is it was performing best when the wave was at a minimum?)

@DuttaA loss, as drawn in this example https://www.tensorflow.org/tutorials/keras/basic_regression, although the data is not the same. I just took the graph code from there

do you do measures agains over-fitting or maybe reduce leraning rate with the time?

@user8426627 I use Adam optimizer and I use Dropout

thats weird because even train error dont converge. Make learning rate smaller and try to run over small amount of epochs

@user8426627 thanks for the suggestion. I will try that


text:
Exploding population size in neat-python
I am trying to make my AI win the board game "Catan" against my friends.
Therefore i am using the python implementation of NEAT.
As I changed the values of weight_mutate_power, response_mutate_power, bias_mutate_power and compatibility_threshold in the config the number of individuals and species exploded (~Doubled every generation and exceeded pop_size).
weight_mutate_power = 12
response_mutate_power = 0
bias_mutate_power = 0.5
compatibility_threshold = 3

Playing with those values I discovered that the rate of the explosion changes in relation to those values (Everything worked fine with standard Values from the documentation).
Any idea how to control this behavior?
My cluster is drowning in genomes...

I had this same problem, and downloaded this better-implemented version of neat-python from GitHub and it fixed my problem: https://github.com/CodeReclaimers/neat-python


text:
RNN weights when varying the input size
I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes.
Say we have a set of natural positive integers
$$
\Gamma=\{1,2,\dots,F\},
$$
where $F=100$ for the sake of the example.
A valid observation vector of my agent at time $t$ might be
$$
\gamma_t=[1,3,5,1].
$$
Thus, at time $t$ a set of weights will be produced by my RNN, according to $\gamma_t$. Sat that at time $t+1$, my observation vector changes as
$$
\gamma_{t+1}=[3,5,8],
$$
and there is my problem. If I now continue training my RNN with the previous weights, the output would be inevitably affected. Also, Which weight shall I remove? I see RNN can face the issue but how shall I deal with the previously computed weights? Which one to remove? How to initialize a new one in case the cardinality of $\gamma_{t+1}$ is higher than that of $\gamma_t$?

is each $\gamma_t$ a new input to the RNN? or is the temporal component that it feeds in all $\gamma$'s at once? Also what weight are you referring to, when asking about "removing"?

$\gamma_t$ is the input of the RNN at time $t$. Let's say, $\gamma_t$ is a vector of requests that have to be satisfied by the agent. It might be that a request (num. $1$ in my example) is satisfied earlier and so the vector $\gamma_t$ will not present that element at time $t+1$. At the same time, a new request comes in (num. $8$ in my example) and so $\gamma_{t+1}$ will present this new value.

About the weights, I'm wondering if this change on the input layer would create some confusion to the RNN which will have a set of weights for the linear combination that refer to the previous condition. Have I been clear for what I intend as 'weights'? Thanks for your answer

i see, so youre problem is like one long temporal vector, but for the rnn you only let it see parts that fall into some form of bin (where bins overlap?). Can you clarify your problem a bit more specifically?


text:
Can next state and action be same in Deep Deterministic Policy Gradient?
I am trying to apply deep deterministic policy gradient (DDPG) on a robotic application. My states consist of the joint angle positions of the robot and my actions are also its joint angle positions. Since, DDPG produces a continuous policy output where states are directly mapped onto the actions, can I say that my next state and action will be same? Simplistically, the input of the policy network will be the current state and the output will be the next state?

Hi and welcome to this community! What do you mean by "my actions are also its joint angle positions"?

Thanks,  what I mean is that my actions are the next joint angle positions. For example: the current position (state) is 90 degrees and the action (next state) could be 91 degrees and so on. Actually, I am very confused with the design of action map for my problem, and the idea of continuous action spaces.


text:
How important is architectural similarity between the discriminator and generator of a GAN?
Shouldn't the discriminator and generator work fine even if they don't process data symmetrically? I mean, they don't only receive the final layer results of each other, they don't use data that from hidden layers.

Youre completely right, they dont need to be.  Intuitively though, it makes sense for them to be (to me atleast), since the descriminator decomposes the images into features and makes a final classification based on this, and you want the generator to learn how to replicate these features properly, so providing the network the same power to generate as compared to discriminate may provide to be useful in the back and fourth game.


text:
Neural Network for Error Prediction of a Physics Model?
I have physical model prediction data as well as actual data. From this I can calculate the error of each prediction data point through simple subtraction. I am hoping to train a neural network to be able to assign an error to the input of the physical model.
My current plan is to normalize the error of each data point and assign it as a label to each model input. So the NN would be trained (and validated)on a 1000 data points with the associated error as a label. Once the model is trained I would be able to input one data point and the output of the neural network would be a single class, that is the error. The purpose this would serve would be to tune the physical prediction model. Would this kind of architecture work? If so, would you recommend a feedforward or RNN? Thank you.

Correct, I am not attempting to find errors in the NN. The errors are coming from the physical model's predicted output and the "real" values. These errors are assigned to the corresponding physical model's input as classes and this serves as the input to the neural network.

pls explain in detail what kind of data do you have on input or output.

I have position and velocity data for a satellite. This is going in as an input into an orbit propagation model. The output of this physics model is the position and velocity at some future point in time (2 weeks). This is subtracted from the actual position and velocity at the future point to obtain the error. The error is normalized and assigned to the original input to the orbit propagation model. My goal is to train a model to be able to assign error values (the output) to orbit parameters (the input) and predict what the error would be for any position and velocity vector.

Autoencoders are unsupervised. I'm not sure how I could use that in my case.


text:
Questions regarding rrn-writer by Robin Sloane?
https://github.com/robinsloan/rnn-writer
I preface this by saying I do not know much about this topic, only that I have an intense interest in it, so I'm hoping I can make my questions as clear as possible.
This writing assistant was released with full code and instructions on how to make it work.  I was halfway through this process when I was told, basically, that this could not work on a Windows PC.  Torch, specifically, either doesn't work on PC, or doesn't work very well, and I am hesitant to continue.
First of all, does anyone know if that's true?
If it is true, is it theoretically possible to recreate this in a different way that will work on Windows PCs?
If so, has anyone ever done it before, or know how to do it?
If there is a way to make Torch work on PCs, is someone willing to tell me how?
I apologize if this isn't meant for this specific area of discussion.  I just don't know where else to go with my questions that I will get any kind of helpful responses.  Even if it's just telling me where I can go for more pertinent responses, that would still be appreciated.
Thank you for any help you might be willing to give.  Please let me know if I need to clarify anything.


text:
Limits for a bottleneck
I have some 64x64 pixels frames from a (simulated) video, with a spaceship moving on a fixed background. The spaceship moves in a straight line with constant velocity from left to right (along the x-axis), and the frames are from equal time intervals.  I can also place the ship at different y positions and let it move. In total I have 8 y positions and 64 frames for each y position (the details don't matter that much). Intuitively, as the background is fixed, and the shape of the ship is the same, all the information to reconstruct the image is found in the x and y position of the spaceship. What I am trying to do is to have a NN with an encoded and a decoder and a bottleneck in the middle and I want that bottleneck to have just 2 neurons. Ideally, the network would learn in these 2 neurons some function of x and y in the encoder, and the decoder would invert that function to give the original image. Here is my NN architecture (in Pytorch):
class Rocket_E_NN(nn.Module):
    def __init__(self):
        super().__init__()

        self.encoder = nn.Sequential(
            nn.Conv2d(3, 32, 4, 2, 1),          # B,  32, 32, 32
            nn.ReLU(True),
            nn.Conv2d(32, 32, 4, 2, 1),          # B,  32, 16, 16
            nn.ReLU(True),
            nn.Conv2d(32, 64, 4, 2, 1),          # B,  64,  8,  8
            nn.ReLU(True),
            nn.Conv2d(64, 64, 4, 2, 1),          # B,  64,  4,  4
            nn.ReLU(True),
            nn.Conv2d(64, 256, 4, 1),            # B, 256,  1,  1
            nn.ReLU(True),
            View((-1, 256*1*1)),                 # B, 256
            nn.Linear(256, 2),             # B, 1
        )

    def forward(self, x):
        z = self.encoder(x)
        return z

class Rocket_D_NN(nn.Module):
    def __init__(self):
        super().__init__()
        self.decoder = nn.Sequential(
            nn.Linear(2, 256),               # B, 256
            View((-1, 256, 1, 1)),               # B, 256,  1,  1
            nn.ReLU(True),
            nn.ConvTranspose2d(256, 64, 4),      # B,  64,  4,  4
            nn.ReLU(True),
            nn.ConvTranspose2d(64, 64, 4, 2, 1), # B,  64,  8,  8
            nn.ReLU(True),
            nn.ConvTranspose2d(64, 32, 4, 2, 1), # B,  32, 16, 16
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 32, 4, 2, 1), # B,  32, 32, 32
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 3, 4, 2, 1),  # B, 3, 64, 64
        )

    def forward(self, z):
        x = self.decoder(z)
        return x

And this is the example of one of the images that I have (it was much higher resolution but I brought it down to 64x64):

So after training it for around 2000 epoch with a bs of 128, with Adam, trying several LR schedules (going from 1e-3 to 1e-6) I can't get the loss below an RMSE of 0.010-0.015 (the pixel values are between 0 and 1). The reconstructed image looks ok by eye, but I would need a better loss for the purpose of my project. Is there any way I can push the loss lower, or am I asking too much from the NN to distill all the information in these 2 numbers?


text:
How to handle proper names or variable names in word2vec?
The input in word2vec is known word (spellings), each tagged by its ID.

But if you process real text, there can be not only dictionary words but also proper nouns like human names, trade marks, file names , etc, how to make an input for that?
Is you consider some input where items are variables, like the meaning of input would be x = something, and after some time you acces to x value and define some other stuff with it. That would be format for this input, and will this approach work at all?

I wasn't able to understand the second part completely, can you please explain with some sample code?

Word2vec works on the concept of  typical word co-occurrences. This means that it will work well only for words that occur frequently in the dataset. So proper nouns will not play any role in training the model. You can keep the proper nouns as they are or use only the words the occur more frequently than some threshold value based on the size of your dataset.
Once you use the value stored in variable x for something, and then change the stored value, it will not reflect anywhere unless you us use the variable x again somewhere in the program.
# Example
x = "something something"
print(x + "...")

# Result
something something ...

# Changing x
x = "new value"

# This new value of x will not reflect anywhere in the program
# Unless you use the variable x again.

i am thinking about interpreting knoledge from the text. For example, let me quote some article : "We propose a new network architecture for learning on graphs Unlike the traditional multi-head attention mechanism" Interpreting its information is sort of structure (in pseudo-code) : x = Subject (first_person_plural); main = Fact(propose_info_action, x, y) ; y =  info_method(of : neural_network, creation_time : new, tool_function = z) , z = Learning(info, learning_source : graph_information(plural)),

so items have kind of relation among each object and the link between sentences where an object occures. Word2vec seems completely not capable to do that since the info it extracts is more primitive?


text:
Convolutional Neural Networks for different-sized Source and Target
CNNs are often used in one of the following scenarios:

A known-sized image is encoded to an intermediate format for later use
An intermediate or precursor format is decoded into a known-sized image
An image is converted into a same-size image

(Usually 3 is done by sticking together 1 and 2.)
Are there any papers dealing with convolutional techniques where the image sizes vary?  Not only would the size of input X differ from input Y, but also input X may differ from output Y. The total amount of variation can probably be constrained by the statistics of the dataset, but knowledge of input X does not grant a priori knowledge of the size of output Y
(Masking is an obvious solution, but I am hoping for something more elegant if research already exists.  The problem domain need not be images.)

might want into anchoring. Most object detectors can use any size image because the featurize portions of the image. But you can extend this and just pool the features.

Yes actually. There have been quite a few different adaptations to convolutional neural networks to do precisely what you are describing.
Here is an earlier one. See section 3.2.5
Here He et al. create a method known as Spatial Pyramid Pooling(SPP). In this method, you are able construct a fixed length representation regardless of input size by pooling them into "spatial bins" which are proportional to the input size, and thusly do not need to modify input dimensions.
There are a few newer methods that improve on this in various capacities, as usual, your solution will be dependent on the problem and other situation-dependent constraints. I suggest you dig deeper into the literature to find the optimal solution in your case.

Ah, excellent!  This might be too big a gun for my particular target, but it is an excellent starting point.  Thank you.

@Novak I believe I remember reading about some lighter weight solutions but they are not coming to mind currently, just search something along the lines of "convolutional neural networks variant/different image sizes" and you should find some good material.


text:
How to approach a problem with infinite solutions
Think Angry Birds kind of game. You need to hit a target at some point by adjusting angle and power. There is infinite number of parabolas that will solve this problem.
My problem is not exactly that but similar, it also has infinite number of solutions. Could anyone please suggest how do I approach this kind of problems using Machine Learning?

Best way is to apply regularization to make it one solution. For example add requirements of minimum energy and/or minimum time, length etc. or make solution maximally close to initial approximation

well doesnt matter if it has million solutions, you just need to find one. If youre asking how to optimize in your domain, you need to be more specific so people can help you out. (there is no general answer to the problem you posed)

Functionally, though, it's not infinite, in that the number arcs are limited by the finite limits of the system.  So there may be a large number of parabolas that can be expressed in-game, but not an infinite number.


text:
How High and Low frequency filters effect activation in the next layer?
Generally, we come across terms such as High Frequency and Low frequency filters in Convolutional Neural Networks (CNN). In regards to this highlighted statement, in 'S1' section of this paper by Jason Yosinski (ref 1),  I thought, in order for high and low-frequency filters to produce a similar effect, weight of low-frequency filters should be greater than high-frequency filters. I would like to understand why I am wrong and I will be grateful if anyone can elaborate about High and Low frequency filters in CNN, in general, or in this context. Thank you.
Ref 1: Yosinski, Jason, et al. "Understanding neural networks through deep visualization." arXiv preprint arXiv:1506.06579 (2015).


text:
Evolving Machine Learning
It seems to me that, right now, the key to making a good Machine Learning model is in choosing the right combination of hyper-parameters.
Firstly: Am I right in saying, if a model is able to tune it's own hyper-parameters, we have in some sense achieved a general intelligence system? Or a glimpse of an actual artificial intelligence system?
I feel the answer lies in what one means by " tune it's own hyper-parameters". If it means to be able to reach Bayesian levels of performance on that task then theoretically, after the tuning, the model is able to perform at par or better than humans and so it seems the answer would be yes.
Secondly: I understand that hyper-parameter tuning is done intuitively. But there are a  set of general directions that is discernable looking at results.  Here I am talking about a heuristic approach to perfect a learning model.
Consider an example:
Say I hardcode a model to, while training, observe gradient values. If the gradient is too large or the cost is highly oscillatory, then restart training with a smaller learning rate.
Then obtain metrics on a test set. If it is poor, then again restart training with regularisation or increased regularisation.
It can also observe various plot behaviours, etc.
The point is maybe not every trick up a researcher's sleeve can be hardcoded. But a decent level of basic tuning can be done.
Thirdly: Let us say, we have a reinforcement learning system on top of a supervised learning system. That is an RL network sets some hyper-parameters. The action then is to train with these hyper-parameters. The reward would be the accuracy on the test set.
Is it possible that such a system could solve the problem of hyper-parameter tuning?

I don't know if that's what you're asking for, but neuroevolution may be something of your interest

It's not clear exactly what you're asking. I know that this is an old post, but, please, edit your post to ask a clear question. Also, remove the first question, which seems to be completely unrelated.

Such a system can and does solve the problem of hyperparameter tuning. Google's AutoML does this. Here is another example that uses a Genetic Algorithm to breed new neural network structures.
AutoML has been shown to outperform humans in the rate that it improves network designs. It seems to favour Residual Network style topologies.


text:
Designing state representation for board game
I am trying to write self-play RL (NN + MCTS http://web.stanford.edu/~surag/posts/alphazero.html) to "solve" a board game. However, I got stuck in designing boardgame same (input layer for NN).
1) What would be the best way to represent each cell, if there are ~10-100 cells in a game which could be occupied by any playing card.  Do one-hot-encoding and get 52 nodes for single cell ([0, 0, 1, ..., 0]) or just divide card_id by total number of cards and get single node for each cell ([0.0576...])?
2) Is it a good/bad practice to help NN by adding additional input that could be derived from the other nodes? For instance, imagine the game where whoever has most red cards wins. Input is 10 cards, and I am adding new input node (number of red cards) to emphasize it. Would that lead to a positive result or doing something like that is bad?
3) Would it help to reduce the number of illegal moves and increase the performance of NN by creating additional input stating which cards are available now and which are not?


text:
why my regression model predict every datapoint to the same value
I am trying to train a SVR but I found that with some combination of features, the trained SVR predict every point in test set to the same value. this problem occurs much more when I use linear kernel than other kernels. The parameters are: C=1, gamma=0.5.
My question is what leads to this kind of problem. Is there a name for this phenomenon? Thank you!

Please provide minimal reproducible code along with your expected output.


text:
How to choose the suitable Neural Network Architecture for Regression Tasks
so I'm working on a Project where I want to predict the Vehicle Position from the Vehicle Data like speed, acceleration etc.. now the data that I have comes also with a timestamp for each sample ( I mean that I have also a timestamp feature).
at first I thought that I should get rid of that timestamp feature because it is not relevant to my Project, I mean logically, I will not need a timestamp feature to predict the vehicle position, that didn't make sense to me when I first took a look at the dataset. I thought other features like speed, acceleration, braking pressure etc.. are more important and I thought also that the solution for this Problem would be to use a normal Deep NN or RBFNN for making this Prediction. recently, I read some papers that shows how a Convolutional NN can be also used for regression and that confused me to choose the Architecture needed for my Project. this Week I also watched a Tutorial where a RNN/ LSTM was implemented for regression Tasks.
Now I'm very confused which architecture should I use for my Project. I also noticed that maybe if I used that timestamp feature, I can maybe use an RNN/LSTM Network for this Task but I don't know if my dataset can be seen as time-series dataset, actually the vehicle position doesn't depend on the time as far as I can tell.
Hopefully can someone answer me based on Experience. It would be also great to have some Papers or references where I can look for more.


text:
How to train a model to extract custom and unknown entities
I'm trying to figure out how to extract specific text from an utterance by a user.
I need to extract "unknown" text from a short and simple text.  In this case, the user wants to create a list.  everything in the {} is unknown text. As it doesn't belong to a specific entity such as food, athletes, movies, etc.

create a new {groceries} list
create a list {movies}
create a new list {movies}
create a list and call it {books}
create a new list and give it the name {stamps}
create a list with the title {red ketchup}
create another list called {rotten food}

the above list is but a small sample of all the different ways that a user can say he wants to create a list.
In everything that I have seen, it's all based on existing entities for the NER and when someone says that it's custom, I found that it just means we have to train a specific set of words and hope for the best.  If I add one more word that isn't trained, it fails to get the data.
But in this case, the user can say anything such as "old shoes", "schools I want to go to", "Keanu Reeves movies".  So I cannot see how I could possibly train it.
With Spacy, I followed this example (https://raw.githubusercontent.com/explosion/spaCy/master/examples/training/train_intent_parser.py) and it mostly works in getting the proper titles. However, I have to train it for every different phrase to work.
For example, if a user says

create a beautiful new list and give it the name {stamps}

the word beautiful causes it to fail and now I have to train for that as well. At this rate, we are looking at millions of phrases to train.
before Spacy, we tried Dialogflow and Rasa.  At each point, it's about training phrases but the more we train, the more one thing worked and another broke.
At this point, we have tried and overall had good intent detection success but when it comes to extracting data such as this, I'm starting to look like a deer in a headlight.
We are new to NLP and while we've had a lot of good progress and over the past few weeks, we cannot seem to find any articles written on this specific problem and whether it can be solved.  Dialogflow has the concept of any entity but they recommend avoiding it and it works 2 out of 3 times when things get complicated.
The goal is to detect which of these words is the title based training. Can it be done? and if so, what's the approach?
Any code, hints or articles that might get us started would be appreciated.


text:
Why hasn't deep learning been used for word level alignment?
I've been exploring word-level alignments tools such as MGIZA and it seems to me that there hasn't been any new tool for this problem. Are neural networks not suitable to solve this problem or simply no interest in the area to build new tools?


text:
Are there deep networks that can differentiate object class from individual object?
We usually categorize objects in a hierarchy of classes. Let us say crow vs bird. In addition, classes can be "messy", for instance a crow can be also a predator, but not all birds are predators.
My question is, can deep networks represent these hierarchies easily? Has anybody studied that? (I could not find anything at all).

In this case, you have an ontology and want to learn the ontology. There are many researches in this topic that you can find. However, the data could be the most challenging part. Some of the researches:

Ontology learning for the Semantic Web
Ontology Learning

Also, as these are some frameworks to ontology learning, you can use deep networks such as RNN to learn the task.

+1 but it does not seem related to my question though. My questions was it is implementable in deep convolutional networks


text:
Finetuning GPT-2 twice for particular style of writing on a particular topic
Sorry if this is a stupid question. I'm just starting out in ML and am working with gpt-2 for text generation.
My situation is that I have to generate text in a particular field for eg. family businesses, which pretrained gpt-2 is unlikely to have much "training" with. Besides the topic, I also need to generate the text in the style of one particular writer (for eg. incorporating their turns of phrase etc) This particular writer hasn't written much about the family business topic unfortunately, but has written about other topics.
It occurred to me that I can take gpt-2, finetune it on a large corpus of material on family businesses, and then finetune the new model on the written material of the particular writer.
Would this be the right way to achieve my objective of creating content on family businesses in the style of this particular writer?
Any suggestions on what sort of stuff I should keep in mind while doing this?
Any help is much appreciated.


text:
Resizing effects on image recognition
I have been building a multilabel image classification model using inception v3, which uses images of size 299x299, I have been wondering what are the effects of feeding images of rectangular shapes for example (or arbitrary resolutions) on the performance of the model, and if I can define requirements for how the data should be to ensure optimal performance, what would those requirements be ?
Intuitively, I think that square images would perform better than rectangular images, is this true?


text:
How to show Sauer's Lemma when the inequalities are strict or they are equalities?
I have the following homework.

We proved Sauer's lemma by proving that for every class $H$ of ﬁnite VC-dimension $d$, and every subset $A$ of the domain,
$$
\left|\mathcal{H}_{A}\right| \leq |\{B \subseteq A: \mathcal{H} \text { shatters } B\} | \leq \sum_{i=0}^{d}\left(\begin{array}{c}{|A|} \\ {i}\end{array}\right)
$$
Show that there are cases in which the previous two inequalities are strict (namely, the $\leq$ can be replaced by $<$) and cases in which they can be replaced by equalities. Demonstrate all four combinations of $=$ and $<$.

How can I solve this problem?


text:
Assigning Weighting Factors
I have a hypothetical example that closes to my research problem:
Assume you are a boss and you have different types of tasks that you need to assign to your employee. Sensitive task (very classified), and task that requires high skills. So you need to assign a sensitive task (government document) to the trusted employee. While the other task (e.g. statistical analysis ) can be assigned to employee who is more creative and smart. Now every day you have many tasks that need to be done and have a large number of employees with a number of crowdsources (freelancers).
You have an outcome and history of trust and performance along of failure rate of assigned task on that day of these employees as:

As you can see here on day 1: the trust of emp 111 is good, so on that day, he had a low failure rate of the sensitive task. While his performance is low, and that made other task failed a lot.
So now assume you have a sensitive task coming, and you have a pool of workers.
The basic equation might not good here: Trust + Performance. I need to weigh each factor based on the type of tasks.
Trust x w1  +  performance x w2  ::::: w1 is high coefficient when sensitive is coming.
Any idea of how I model these issues.

Can I use the analytic hierarchy process (AHP)?


text:
Maximize loss on non-target variable
I have a neural network that should be able to classify documents to target label A. The problem is that the network is actually classifying label B, which is an easier task.
To make the problem more clear: I need to classify documents from different sources. In the training data each source occurs repeatedly, but the network should be able to work on unknown sources. All documents from a single source have the same class. In this case, it is easier to identify sources than the target label so in practice the network is not really identifying the target label, but the source.
The solution to this problem is making sure that the model is bad at identifying the sources in the training data, while still attaching the right target labels.
I think the first step is to get two output layers, one for the target label and one for identifying which source it is from. My approach fails however at the training procedure: I want to minimize the loss on the target output, but maximize the loss on the non-target output. But if I maximize the loss on that non-target output, that does not mean that the network 'unlearns' the non-target labels. So the main question for the non-target output is:
TLDR; How do I define a training procedure that minimizes the loss on a non-target output layer, and then maximizes that loss on all layers before it. My goal is to have a network that is good at classifying label A, but bad at a related label B. If anyone wants to give a code example, my prefered framework is PyTorch.

You can maximize the loss by doing 1-loss (if the range of the loss is between 1 and 0. However, I am not sure if that would help. The network can just propagate the weights of the source classifier to always output the wrong answer. a better approach is to hand craft features of the document to make it less identifiable as the source. If the document have stated the source, remove it. This may help. Also, another way is to use documents from more sources so the network cannot use the source to classify the label. Hope I can help you.

A little more information about the documents will be helpful. I am guessing that your scenario has webpages from different websites, you're feeding html pages to the network and the page contains the website name or url, which the network is picking up on and using it to label. I am assuming you're using a RNN or similar network for the classification task.
Your question is to identify when your network is using the source identifying feature (name or url), and instruct it to not use this feature. If you can do some preprocessing and identify where in the document the source name/url is, you may tag each word/character in the input with this information. You may use this tag to add a large regularizing loss term which brings down the weights connected to the tagged input words/characters in the first layer.
However, a better strategy will be to remove all instance of the header / url which contains website info, as @clement-hui suggested. This is cleaner and easier to implement than above.
Perhaps your network is identifying the document source slightly more indirectly, and above preprocessing is not sufficient/useful. In this case, you may want to make random chunks of your document, and give each chunk the label corresponding to that document. Here it is more likely that some chunks will not have the source identifying information, and the network will be forced to pick a strategy which labels using the content rather than the source.
At test time you may either feed the whole document or random chunks from the document and use majority voting to get the final answer.


text:
What's a good generative model for creating valid formats of a person's name?
I'm trying to come up with a generative model that can input a name and output all valid formats of it.
For example, "Bob Dylan" could be an input and the gen model will output "Dylan, Bob", "B Dylan", "Bob D" and any other type of valid formatting of a person's name.  So given my example the gen model doesn't seem that complicated to build, but it also has to handle stuff like "Dylan, Bob" and "B Dylan", but obviously the 2nd one it shouldn't output "Bob Dylan" as a potential output cause inferring that requires more than just "B Dylan". Any ideas for a good Generative Model for this?

Do you really need a generative model to do this?

mmm well not if you can think of a better way?

@nbro if your looking some fun why not

@DylanY I would use an encoder decoder network like in translation systems and I’d induce a prior based on inputs (maybe a secondary step). Seems like a fun problem

like a VAE? What did you have in mind for doing it?

Why some rules should not work?

@OmG sorry what rules are you referring to?

@DylanY I mean grammatical rules.

well I'm more interested in how people would actually format names when writing them. I guess that is partially grammatical rules, but I think it's more useful for a gen model that can find the latent variables first name, middle name and last name.