text:
Applying Artificial neural network into kaggle's house prices data set gave bad predicted values
I am trying to solve the kaggle's house prices using neural network. I've already made it with ensembling several models (XGBoost, GradientBooster and Ridge) and I've got a great score ranking me between the top 25%.
I imagined that by adding a new model to the ensembled models like ANN would increase prediction accuracy, so I did the following:
import keras

model = keras.models.Sequential()

model.add(keras.layers.Dense(235, activation='relu', input_shape=(235,)))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(1))

model.compile(optimizer='adam', loss='mean_squared_error')
model.summary()
model.fit(dataset_ann, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=3)])
y_pred = model.predict(X_test_ann)

I choosed 235 neurons for each layer, as the training set has 235 features.
For model ensembling:
y_p = (0.1*model.predict(X_test_ann)+0.2*gbr.predict(testset)+0.3*xgb.predict(testset)+0.1*regressor.predict(testset)+0.1*elastic.predict(testset)+0.2*ridge.predict(testset))

The shape of y_p is (1459, 1459) instead of (1459, ) where columns are all having the same values, so taking y_p[0] would be more than enough.
I submitted the result into kaggle and went from top 25% into bottom 60%.
Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?
I tried with epoch = 30, 100, 1000 and got nearly the same bad ranking.

Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?

I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.
I would try with a smaller network, so you "force" it to learn something, instead of memorizing.

So maybe one hidden layer with 20 to 30 neurons?


text:
What effect does increasing the actions in RL have?
Consider a 2D snake game, where the snake has to eat food to become longer. It must avoid hitting walls and biting into her tail.
Such a game could have a different amount of actions:

3 actions: go straight, turn left, turn right (relative to crawling direction)
4 actions: north, east, south, west (absolute direction on the 2D map)
7 actions: a combination of option A and option B (leaves the preferred choice to the player)

While the game in principle is always the same, I would like to understand the impact of the amount of actions on the training of a neuronal network. One obvious thing is the number of output nodes of the neuronal network.
In case A (3 actions), the neuronal network cannot perform an incorrect action. Any of the 3 choices are valid moves.
In case B (4 actions), the net IMHO needs to learn that going into opposite direction does not have the desired effect and the snake continues moving into the old direction.
In case C (7 actions), the net needs to learn both, 1 action is always illegal and the 3 relative actions somehow map to the 3 absolute actions.
How can I consider the learning curve in these situations? Does option B need 25% more training than option A to achieve the same results (same fitness) (similar: option C needs 125% more training time)?
Is giving a negative reward for an impossible move considered cheating, because I do code the rules of the game into the reward logic?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
RealNVP gives wrong probabilities
I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long). My code is mainly based on this code from GitHub and below are the main piece that I am using (with slight adjustments). The problem is that the loss is getting negative, which in the definition of my code means that the log-probability of the my data is positive, which in turn means that the probabilities are bigger than 1. This is impossible mathematically, and I see no way how this can happen, from a mathematical point of view. I also couldn't find a mistake in my code. Can someone help me with this? Is there a mistake there? Am I missing something with my understanding of normalizing flows? Thank you!
class NormalizingFlowModel(nn.Module):

    def __init__(self, prior, flows):
        super().__init__()
        self.prior = prior
        self.flows = nn.ModuleList(flows)

    def forward(self, x):
        m, _ = x.shape
        log_det = torch.zeros(m).cuda()
        for flow in self.flows:
            x, ld = flow.forward(x)
            log_det += ld
        z, prior_logprob = x, self.prior.log_prob(x)
        return z, prior_logprob, log_det

    def inverse(self, z):
        m, _ = z.shape
        log_det = torch.zeros(m).cuda()
        for flow in self.flows[::-1]:
            z, ld = flow.inverse(z)
            log_det += ld
        x = z
        return x, log_det

    def sample(self, n_samples):
        z = self.prior.sample((n_samples,))
        x, _ = self.inverse(z)
        return x

class FCNN_for_NVP(nn.Module):
    """
    Simple fully connected neural network to be used for Real NVP
    """
    def __init__(self, in_dim, out_dim):
        super().__init__()
        self.network = nn.Sequential(
            nn.Linear(in_dim, 32),
            nn.Tanh(),
            nn.Linear(32, 32),
            nn.Tanh(),
            nn.Linear(32, 64),
            nn.Tanh(),
            nn.Linear(64, 64),
            nn.Tanh(),
            nn.Linear(64, 32),
            nn.Tanh(),
            nn.Linear(32, 32),
            nn.Tanh(),
            nn.Linear(32, out_dim),
        )

    def forward(self, x):
        return self.network(x)

class RealNVP(nn.Module):
    """
    Non-volume preserving flow.

    [Dinh et. al. 2017]
    """
    def __init__(self, dim, base_network=FCNN_for_NVP):
        super().__init__()
        self.dim = dim
        self.t1 = base_network(dim // 2, dim // 2)
        self.s1 = base_network(dim // 2, dim // 2)
        self.t2 = base_network(dim // 2, dim // 2)
        self.s2 = base_network(dim // 2, dim // 2)

    def forward(self, x):
        lower, upper = x[:,:self.dim // 2], x[:,self.dim // 2:]
        t1_transformed = self.t1(lower)
        s1_transformed = self.s1(lower)
        upper = t1_transformed + upper * torch.exp(s1_transformed)
        t2_transformed = self.t2(upper)
        s2_transformed = self.s2(upper)
        lower = t2_transformed + lower * torch.exp(s2_transformed)
        z = torch.cat([lower, upper], dim=1)
        log_det = torch.sum(s1_transformed, dim=1) + torch.sum(s2_transformed, dim=1)
        return z, log_det

    def inverse(self, z):
        lower, upper = z[:,:self.dim // 2], z[:,self.dim // 2:]
        t2_transformed = self.t2(upper)
        s2_transformed = self.s2(upper)
        lower = (lower - t2_transformed) * torch.exp(-s2_transformed)
        t1_transformed = self.t1(lower)
        s1_transformed = self.s1(lower)
        upper = (upper - t1_transformed) * torch.exp(-s1_transformed)
        x = torch.cat([lower, upper], dim=1)
        log_det = torch.sum(-s1_transformed, dim=1) + torch.sum(-s2_transformed, dim=1)
        return x, log_det

flow = RealNVP(dim=data.size(1))
flows = [flow for _ in range(1)]
prior = MultivariateNormal(torch.zeros(data.size(1)).cuda(), torch.eye(data.size(1)).cuda())
model = NormalizingFlowModel(prior, flows)
model = model.cuda()

for i in range(10):
    for j, dtt in enumerate(my_dataloader_bkg_only):
        optimizer.zero_grad()
        x = dtt[0].float()
        z, prior_logprob, log_det = model(x)
        logprob = prior_logprob + log_det
        loss = -torch.mean(prior_logprob + log_det)
        loss.backward()
        optimizer.step()
    if i % 1 == 0:
        print("Saved")
        best_loss = logprob.mean().data.cpu().numpy()
        print(logprob.mean().data.cpu().numpy(), prior_logprob.mean().data.cpu().numpy(),
                  log_det.mean().data.cpu().numpy())

You might well have a bug, but in general there's no reason to not have a log-likelihood greater than 0. E.g. consider fitting a single 1-d point with a Gaussian with parameterised mean and variance. The maximum-likelihood solution is an infinitely sharp peak with infinite log likelihood.

So how should I think about a probability bigger than 1? Assuming I have 2 1-d points (in my case I have 1M), and one of them has a probability of, say, 1.5, how can I make sense of that mathematically? Thank you!

You are computing a probability density (flow models are density models), so you should interpret it as a probability density. Maybe this will help https://towardsdatascience.com/probability-concepts-explained-probability-distributions-introduction-part-3-4a5db81858dc

I don't know if this comment will be helpful, but shouldn't the sum of the log determinant of the Jacobian (LDJ) have opposite sign in the forward and inverse pass? I'm not talking about the LDJ being sum of the positive scaling function in the forward and sum of the negative of the scaling function in the inverse, I'm talking about the LDJ itself.
For example, from the change of variable formula,
$$p_z(z) \ dz = p_x(x) \ dx$$
differentiating w.r.t $\ dz$ gives,
$$ p_z(z) = p_x(x) \left|\frac{dx}{dz}\right|$$
taking the log of both sides gives,
$$ ln(p_z(z)) = ln(p_x(x)) +  ln\left(\left|\frac{dx}{dz}\right|\right)$$
Now, if we repeat this process but differentitating w.r.t $\ dx$,
$$ p_x(x) = p_z(z) \left|\frac{dz}{dx}\right|$$
taking the log and rearranging for $ln(p_z(z))$ gives,
taking the log of both sides gives,
$$ ln(p_z(z)) = ln(p_x(x)) -  ln\left(\left|\frac{dz}{dx}\right|\right)$$
A change of sign occurs when the Jacobian is reversed. Could this be your problem? Also, as @chris-cundy said you can have a probability density greater than 1 at a single point. Remember it is the integral over all space that cannot be greater than one, an individual point between greater than 1 is fine.

as far as I understand it, it's not the log-probability but the log-likelihood.
the likelihood can be greater than 1 so negative loss is possible.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Possible model to use to find pixel locations of objects
I want to make a model that outputs the centre pixel of objects appearing in an image.
My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where each pixel has a value of 1 if it is the center of an object and 0 otherwise. Each input image has roughly ~80 objects.
The problem with this is the CNN learns the easiest way to reduce the error, which is having the entire output be 0, because for 97% of cases that's correct. As such, error decreases but it learns nothing.
What is another potential method for training a network to do something similar? I also tried adding dropout, which made the output a lot more noisy and it seemed to learn ok, but eventually ended up in the same state as before with the entire output being 0, never really seeming to learn how to output the locations of objects.

I didn't saw your dataset but problem can also occur when one time pixel RGB(10, 10, 10) is 1, another time is 0

Do you have to train a new model or can you use a pre-trained model - it might be enough to remove the last one or two layers and adjust those to your needs? For object detection there are basically two approaches: Instance Segmentation that try to find a pixel-by-pixel classification and Bounding Box models that find the minimum box to contain the entire object. A centre pixel of an object can have different meaning for these approaches: is the centre pixel of the crescent the void in the middle of the bounding box?

@Gregor Yes the centre pixel would be the middle of the void in the bounding box. I could use a pre-trained model, but most of those models feel much to large for the task at hand

A pre-trained model is indeed probably fairly large for your purpose (usually trained to predict 1000 classes) but they come almost production-ready: set up a machine with TensorFlow, OpenCV, ... and load the model. That's it. The outputs are the probabilities of the different classes (this is where you might want to adjust your final layer to predict only those 80 classes) and the bounding box or pixel mask. The centre of a bounding box is simply the middle of the height and width.
You can also find a trade-off between accuracy and speed by adjusting the precision from FP32 to maybe FP16.

from what I understand you are building you own model for this specific use case. From my perspective I would try not to reinvent the wheel, as it is said, and use an already proven and working model such as the YOLOs (v1, v2 and v3).
YOLO does not tell you the center pixel of the image directly but it tells you the center cell, with respect to the predicted object, of a grid (which is built on top of the image) responsible for predicting each object. See on the left image how the grid is built on top of the input image, then YOLO computes a probability map, and then the bounding boxes are predicted from the cell in the object's center. I have highlighted which cells would be responsible for predicting each object in the rightmost image. (Image from YOLOv1 paper)

The grid can have different resolutions but if you make it equal to the image size, then you would have the center pixel of each object predicted. This is because YOLO predicts objects in each cell of the grid, so if the grid is equal to the image size, YOLO will predict objects in each pixel.
As an example, imagine you have an input image of $[H \times W] = [416 \times 416]$ then YOLO would compute a grid of $[S_1 \times S_2]=[52 \times 52]$ on top of it. And predict objects in the center cell of the $[S_1 \times S_2]$. So, if you tune YOLO for computing a grid such as $[S_1 \times S_2] = [H \times W]$, then YOLO would output objects prediction with respect to the image pixels, in other words, YOLO would predict bounding boxes centered in the image pixel on the object's center.
This is how I would proceed for this use case, I hope it helps you or at least give you some clues about how to proceed further! Cheers! :)

NOTE: I chose the image size and grid size with numbers I usually see at work. Specifically, using YOLOv3. In YOLOv3, for aninput image of $[H \times W] = [416 \times 416]$, 3 grids are built, with different resolutions (for predicting big and small objects), with the following grids sizes are: $[13 \times 13], [26 \times 26], [52 \times 52]$

Thanks for your reply. Is there any faster way to do this than with YOLO that you might know of? My dataset is very simplistic and could almost be solved by using a heuristic approach, were it not for minor differences in dimensions and proportions.

Yes, by building a custom model as you suggest, but you did not provide enough information about 2 key things: how you do the bounding box regression and what your loss function is (very important for custom models). By predicting the bbox coordinates (x,y,w,h) you can get the (x,y) pixel in the image. But before doing that you need to write an adequate loss function that facilitates that prediction.

You must increase the percentage of "1" labels on the target output, so that "all zeroes" doesn't give such a low loss. If your images are $512 \times 512$, could you label the target image as ones within a radius of 10 pixels? This way each target would have about 314 "1" labels (0.12%), rather than just one (0.0004%). Maybe even a radius of 20 pixels would be better.
Also binary crossentropy is a better loss function for this than MSE (L2 difference).


text:
Is Gradient Descent algorithm a part of Calculus of Variations?
As in https://en.wikipedia.org/wiki/Calculus_of_variations

The calculus of variations is a field of mathematical analysis that
  uses variations, which are small changes in functions and functionals,
  to find maxima and minima of functionals

The Gradient Descent algorithm is also a method to find minima of a function, it it a part of the Calculus of Variations?

How is this question related to AI? You should at least mention where variational calculus can be used in AI. I know where it can be used, but you should edit your post to make this question more on-topic.

Gradient Descent is AI, and the confusing term here is 'finding minima'

@datdinhquoc GD alone is not AI. GD is an iterative optimization algorithm, which can also be used in other contexts that someone would not necessarily call AI. I will help you. You could say something like "GD is used to train neural networks and variational calculus is used in computer vision" as a premise, then ask your question.

it's maths, yes, but it's used in AI as the basic optimiser

@datdinhquoc ok. but this is related with Directional Derivatives and the Gradient Vector, max and min values from Calculus? anyone can confirm?


text:
Post-classification after inference
I designed a fire detection using Deep Learning based classification approach. In my training dataset, I have both fire and fire smokes are supposed to be detected (all under "fire"; mostly real fires are detected. Fire smokes are less accurate).
Now after months, I need to differentiate them in my detection results. It would be difficult to retrain each class separately now. Another option coming into my mind is building a binary classification after the main one, getting the main detections as input and saying which of the two it belongs to. However, I may miss some fire smokes I believe because that's less accurate.
Is there any other approaches? What are pros/cons of various approaches?

Try just fine-tuning your model by only retraining the classification head / last layer.


text:
Nuances of Turing test requirements
My understanding is that there is no singular "The Turing Test Ruleset" and competitions don't all do it the exact same way. Still, I'm wondering about some commonly accepted rules and their nuances. My Googling is not producing any specifics about this.
I think most people agree that the purpose of the humans who talk to the judges is to just act normal. In the so-called "passed Turing test" instances where the humans tried to fool judges into thinking they were AIs, I would say the tests should be thrown out and I've seen critics in the field agree with that.
But this question is more about how the judge should act.
Let's say we are doing a competition where the threshold is for 40% of judges to call an AI a human after 5 minutes of chat.
During the chat, is the judge supposed to try to trick the potential AI into revealing itself, or is the judge supposed to attempt to act as unbiased and natural as possible, as if it is accepted the potential AI is human, and judge based on a completely natural conversation?
For example, asking "What is the value of PI out 10 digits?" or "What is 123456 times 654321?" or "If you saw a bunny and a dime stuck on the road with a truck about to hit them, which one would you pick up?" would be trying to trick AIs into revealing themselves because you are relying on exploiting the fact that the AI might tell you the correct answer or the inhuman answer.
This is as opposed to simply carrying on a natural and normal conversation, with no biases or expectations. If you came upon someone on the street you would not spend 5 minutes trying to hurriedly ask ridiculous interrogation questions in an effort to prove the other person was an AI.
So is the point of a Turing test generally assumed to be an attempt to flush out AIs or an attempt to judge their natural human conversation without interrogation-like prejudice?

Based on the answer, my question may seem a bit like an XY question, but I was not aware that one of the best known tests poses the same set of questions to all contestants. I don't think that's the case for all such contests. Maybe even this aspect of Turing tests is not agreed on, though I thought it would be.

I do not know about all competitions, but the Loebner prize is one of the more well known competitions based around the concept of the Turing Test, specifically for chatbots, and judges must ask the same questions of all bots. The questions are only revealed during the competition, it is not possible for bot programmers to prepare for them other than making a generally good chatbot. They judges can then rank bots on how they handle the same set of questions. E.g. from chatbots.org/ai_zone/viewthread/867 here are the 2012 "qualifying" questions that were released following the test:

My name is Bill. What is your name?
How many letters are there in the name Bill?
How many letters are there in my name?
Which is larger, an apple or a watermelon?
How much is 3 + 2?
How much is three plus two?
What is my name?
If John is taller than Mary, who is the shorter?
If it were 3:15 AM now, what time would it be in 60 minutes?
My friend John likes to fish for trout.  What does John like to fish for?
What number comes after seventeen?
What is the name of my friend who fishes for trout?
What whould I use to put a nail into a wall?
What is the 3rd letter in the alphabet?
What time is it now?

According to the WikiPedia description, the style of the questions has changed over time. The above set in my opinion seem to be designed to test very simple grammar, common sense and memory targets.

Your edit makes a good point. At this stage in the game, perhaps the point of the contests is not (or should not be) "Can you pass yourself as human reliably?" since we're still far from that but rather should simply be "How close are you to being able to carry on anything more than simple conversation patterns?" So not really a Turing test, but a "How close are we getting to being able to do real Turing tests?"


text:
Specific Neural Network Subtype for Automatic Web Scraping (Hyperlink Identification)
For a personal project, I'm trying to download files from a specific set of websites using a web scraper. The scraper has to navigate multiple webpages to get to the files I want to download. I'd like to use AI to find each successive link in order to avoid the inflexibility of hardcoded DOM paths. In other words, I want an AI system that replicates the way that a human would click through links to get to a download page.
This seems like a task for a supervised neural network, where the input would be the HTML page and the output would be the href link to the next webpage in the search. But that's about the extent of my AI knowledge.
Which subtype of neural network would be most effective for this kind of problem?
Note, I have looked at this related question. My problem would probably fall under bullet two.


text:
How can a new metric applied for humans causing danger on railtracks?
I am writing myself and was thinking about, what kind of metric can be applied to measure the "dangerousness" of a human being on a railtrack? For example detecting if a human is running on the rails?
Maybe predicting the human movement towards rails, the distance to the vehicle (ego perspective) or something else?


text:
Best architecture to learn image rotation?
Given an input image and an angle I want the output to be the image rotated at the given angle.
So I want to train a neural network to do this from scratch.
What sort of archetecture do you think would work for this if I want it to be lossless?
I'm thinking of this archetecture:
256x256 image
--> convolutions to 64x64 image with 4 channels
--> convolutions to 32x32 image with 16 channels and so on
until a 1 pixel image with 256x256 channels.
And then combine this with the input angle, and then a series of deconvolutions back up to 256x256.
Do you think this would work? Could this be trained as a general rotation machine? Or is there a better archetecture?
I would also like to train the same archetecture to do other transforms.

This would likely suffer from the blurry image problem that autoencoders are known to suffer from. See also here.
On the other hand, using GAN's to sharpen your images doesn't seem particularly helpful since you seem to be lookng for a way to rotate general images, not ones of a specific domain. Moreover, there's almost certainly going to be some loss
It seems unreasonable/overkill to use neural networks to do this transformation. I suggest treating the rotation as a linear transformation, mapping certain pixel locations to other pixel locations (in fact, you can make this transformation differentiable with respect to its inputs using some deep learning library's autodifferentiation tools, if that's what you're looking for).
Since rotations are just linear maps that send pixels to other pixels, this will be much more computationally efficient.

True, there's better ways of doing this than neural networks. It's more of a challenge to see if I could implement such a thing using neural networks. e.g. how perhaps a brain might rotate an image in it's mind.


text:
Error when using tensorflow HMC to marginalise GPR hyperparameters
I originally posted on SO (original post) but was suggested to post here.
I would like to use tensorflow (version 2) to use gaussian process regression to fit some data and I found the google colab example online here 1. I have turned some of this notebook into a minimal example that is below.
Sometimes the code fails with the following error when using MCMC to marginalize the hyperparameters: and I was wondering if anyone has seen this before or knows how to get around this?
tensorflow.python.framework.errors_impl.InvalidArgumentError:  Input matrix is not invertible.
     [[{{node mcmc_sample_chain/trace_scan/while/body/_168/smart_for_loop/while/body/_842/dual_averaging_step_size_adaptation___init__/_one_step/transformed_kernel_one_step/mh_one_step/hmc_kernel_one_step/leapfrog_integrate/while/body/_1244/leapfrog_integrate_one_step/maybe_call_fn_and_grads/value_and_gradients/value_and_gradient/gradients/leapfrog_integrate_one_step/maybe_call_fn_and_grads/value_and_gradients/value_and_gradient/PartitionedCall_grad/PartitionedCall/gradients/JointDistributionNamed/log_prob/JointDistributionNamed_log_prob_GaussianProcess/log_prob/JointDistributionNamed_log_prob_GaussianProcess/get_marginal_distribution/Cholesky_grad/MatrixTriangularSolve}}]] [Op:__inference_do_sampling_113645]

Function call stack:
do_sampling

1 https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Gaussian_Process_Regression_In_TFP.ipynb#scrollTo=jw-_1yC50xaM
Note that some of code below is a bit redundant but it should in some sections but it should be able to reproduce the error.
Thanks!
import time

import numpy as np
import tensorflow.compat.v2 as tf
import tensorflow_probability as tfp
tfb = tfp.bijectors
tfd = tfp.distributions
tfk = tfp.math.psd_kernels
tf.enable_v2_behavior()

import matplotlib
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
#%pylab inline
# Configure plot defaults
plt.rcParams['axes.facecolor'] = 'white'
plt.rcParams['grid.color'] = '#666666'
#%config InlineBackend.figure_format = 'png'

def sinusoid(x):
  return np.sin(3 * np.pi * x[..., 0])

def generate_1d_data(num_training_points, observation_noise_variance):
  """Generate noisy sinusoidal observations at a random set of points.

  Returns:
     observation_index_points, observations
  """
  index_points_ = np.random.uniform(-1., 1., (num_training_points, 1))
  index_points_ = index_points_.astype(np.float64)
  # y = f(x) + noise
  observations_ = (sinusoid(index_points_) +
                   np.random.normal(loc=0,
                                    scale=np.sqrt(observation_noise_variance),
                                    size=(num_training_points)))
  return index_points_, observations_

# Generate training data with a known noise level (we'll later try to recover
# this value from the data).
NUM_TRAINING_POINTS = 100
observation_index_points_, observations_ = generate_1d_data(
    num_training_points=NUM_TRAINING_POINTS,
    observation_noise_variance=.1)

def build_gp(amplitude, length_scale, observation_noise_variance):
  """Defines the conditional dist. of GP outputs, given kernel parameters."""

  # Create the covariance kernel, which will be shared between the prior (which we
  # use for maximum likelihood training) and the posterior (which we use for
  # posterior predictive sampling)
  kernel = tfk.ExponentiatedQuadratic(amplitude, length_scale)

  # Create the GP prior distribution, which we will use to train the model
  # parameters.
  return tfd.GaussianProcess(
      kernel=kernel,
      index_points=observation_index_points_,
      observation_noise_variance=observation_noise_variance)

gp_joint_model = tfd.JointDistributionNamed({
    'amplitude': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'length_scale': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'observation_noise_variance': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'observations': build_gp,
})

x = gp_joint_model.sample()
lp = gp_joint_model.log_prob(x)

print("sampled {}".format(x))
print("log_prob of sample: {}".format(lp))

# Create the trainable model parameters, which we'll subsequently optimize.
# Note that we constrain them to be strictly positive.

constrain_positive = tfb.Shift(np.finfo(np.float64).tiny)(tfb.Exp())

amplitude_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='amplitude',
    dtype=np.float64)

length_scale_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='length_scale',
    dtype=np.float64)

observation_noise_variance_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='observation_noise_variance_var',
    dtype=np.float64)

trainable_variables = [v.trainable_variables[0] for v in
                       [amplitude_var,
                       length_scale_var,
                       observation_noise_variance_var]]
# Use `tf.function` to trace the loss for more efficient evaluation.
@tf.function(autograph=False, experimental_compile=False)
def target_log_prob(amplitude, length_scale, observation_noise_variance):
  return gp_joint_model.log_prob({
      'amplitude': amplitude,
      'length_scale': length_scale,
      'observation_noise_variance': observation_noise_variance,
      'observations': observations_
  })

# Now we optimize the model parameters.
num_iters = 1000
optimizer = tf.optimizers.Adam(learning_rate=.01)

# Store the likelihood values during training, so we can plot the progress
lls_ = np.zeros(num_iters, np.float64)
for i in range(num_iters):
  with tf.GradientTape() as tape:
    loss = -target_log_prob(amplitude_var, length_scale_var,
                            observation_noise_variance_var)
  grads = tape.gradient(loss, trainable_variables)
  optimizer.apply_gradients(zip(grads, trainable_variables))
  lls_[i] = loss

print('Trained parameters:')
print('amplitude: {}'.format(amplitude_var._value().numpy()))
print('length_scale: {}'.format(length_scale_var._value().numpy()))
print('observation_noise_variance: {}'.format(observation_noise_variance_var._value().numpy()))

num_results = 100
num_burnin_steps = 50

sampler = tfp.mcmc.TransformedTransitionKernel(
    tfp.mcmc.HamiltonianMonteCarlo(
        target_log_prob_fn=target_log_prob,
        step_size=tf.cast(0.1, tf.float64),
        num_leapfrog_steps=8),
    bijector=[constrain_positive, constrain_positive, constrain_positive])

adaptive_sampler = tfp.mcmc.DualAveragingStepSizeAdaptation(
    inner_kernel=sampler,
    num_adaptation_steps=int(0.8 * num_burnin_steps),
    target_accept_prob=tf.cast(0.75, tf.float64))

initial_state = [tf.cast(x, tf.float64) for x in [1., 1., 1.]]

# Speed up sampling by tracing with `tf.function`.
@tf.function(autograph=False, experimental_compile=False)
def do_sampling():
    return tfp.mcmc.sample_chain(
      kernel=adaptive_sampler,
      current_state=initial_state,
      num_results=num_results,
      num_burnin_steps=num_burnin_steps,
      trace_fn=lambda current_state, kernel_results: kernel_results)

t0 = time.time()
samples, kernel_results = do_sampling()
t1 = time.time()
print("Inference ran in {:.2f}s.".format(t1-t0))


text:
On which data evaluate an object detection model ? (similar or real life data ?)
I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.
Even I have more than 2k different poses, but the background does not change much (I have only 5 different points of view).
I trained my model on these images (70% for the training and 30% for validation).
Now, I want to evaluate the model on an unbiased dataset.
Should I keep images of my dataset for this purpose or should I use a real life dataset ?
(A good solution would be to have a real life training set, but I don't have)
I tried both, but as expected, I have an mAP=0.9 when evaluated on similar pictures and mAP=0.5 when evaluated on completely different images.
Bonus question: is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")
Precision-Recall ? (but I have to choose a pose classification threshold...)

If you want to evaluate on real thermal image dataset, you can use this one.
Thermal Image dataset
is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")

Mean Average Precision is the relevant metric but it's more technical. You can start explaining with False Positives and False
  Negatives in the predictions. In turn, it leads to Precision and Recall. It mostly depends upon your use case because there will always be a trade-off between them.

Ty for your answer on the mAP but you did not answer my main question on "how to build a good evaluation/test set ?".

At the beginning itself, we have to split the data as train/val/test. It depends upon how much data you have for your task to train the model. The test set should be taken from almost the same data distribution of the training set. If we train a model with only daylight images and test with night conditions, it is a bad idea to judge the model.


text:
Interpolating image to increase resolution before feeding it to a neural network
Interpolation is a common way to make an image fit the right input shape for a neural network.
But is there any point in using interpolation to make it easier for the network to learn?
I assume interpolation adds no extra information to the input; It only uses existing information to increase the resolution and fill missing values.
However, sometimes I have observed that while I can not see anything with my human eye, using some kind of advanced interpolation technique such as b-spline interpolation makes it crystal clear that the object i am looking for is in the image, especially in the domain of infrared images.
So, is there any benefit for using interpolation rather than feeding a low dimensional image to a neural network?

One of the used is to use a neural interpolation, this is using a pretrained model to zoom our images, in some sense training with this kind of images is like combining this pretrainined model with our new data, and can be seen as transfer learning technique

I can't really understand how this answers my question. What are the benefits of using interpolation before feeding the input to the neural network, if there are any, is my question. I am confused reading this answer. Could you explain more clearly?


text:
What are the current tools and techniques for image segmentation in order of pragmatism?
To explain what I mean I'll depict the two extremes and something in the middle.
1) Most pragmatic: If you need to just segment a few images for a design project, forget AI. Go into Adobe Photoshop and hand select the outline of the object you need to extract.
2) Middle ground: If you need to build a reasonably accurate app for human aided segmentation of images, use a pre-trained model on a well known architecture.
3) Least pragmatic: If you need to reach unprecedented levels of accuracy on a large volume of images. Do heavily funded research on new and better methods of image segmentation.
So I'm most interested in painting out the spectrum for that middle ground. That is, how much of the wheel needs to be reinvented versus the complexity of the problem.
For example (and this is what lead me here), I need to segment out dogs from several hundred photos that owners have taken. The dog is probably going to be among the main subjects of the photos. Do I need to reinvent the wheel (design an architecture)? Probably not. Do I even need to change the tyres (train my own model)? I'm guessing not. Do I need to code at all? I'm not sure.
While I'm happy to get answers about my use case, it would be awesome if someone could map out the spectrum on my unfinished rainbow.


text:
Improve prediction with LSTMs when data have no particular trend (complex)
I have a deep learning problem, I am working with the CMAPSS dataset, which contains data simulating the degradation of several aircraft engines. The aim is to predict from data collected on a machine in full operation, the remaining useful time at this machine. My problem is the following when the features (sensor data) have a specific trend (either up or down), my model (LSTMs) predicts good results but when the data have no trend, my deep learning model gets a very bad score. I must specify that I work with sequential data. my dataset contains several aircraft engines with data recorded by the sensors. my question is how to process the data that has no particular trend in deep learning.
You will find below some pictures of my dataset where, RUL: is the life of a machine, unit: the machine identity and s1 to s3 is the sensor data.


text:
How to represent and work with the feature matrix for graph convolutional network (GCN) if the number of features for each node is different?
I have a question regarding features representation for graph convolutional neural network.
For my case, all nodes have a different number of features, and for now, I don't really understand how should I work with these constraints. I can not just reduce the number of features or add meaningless features in order to make the number of features on each node the same - because it will add to much extra noise to the network.
Are there any ways to solve this problem? How should I construct the feature matrix?
I'll appreciate any help and if you have any links to papers that solve this problem.

Hi and welcome to AI SE! The addition of dummy values isn't that uncommon in deep learning. For example, when doing sequence processing, one may require all sequences to have the same length. In that case, the sequences may be padded with dummy values. (Anyway, although I am familiar with GDL and graph NNs, I am not qualified enough to give an answer to your question).

Hi @nbro and thanks for your answer. May I ask you, how to choose the dummy values in this case? Do you have any idea of methods that can be used, so that  I don't broke completely the distribution?

Honestly, I should not suggest something that may not be appropriate. But maybe try some dummy values that are consistent with your representation of the features, train the networks, see the results, and iterate (i.e try other ideas). For example, whenever there are missing features, you could have zeros, which, when multiplied around, will have the effect of zeroing contributions. I hope someone else will provide more help, but GDL is a quite new area, so many techniques have yet to be developed. It's also been a while I've not read a GDL paper, so I am a bit rusty :)

The simplest way I could come with is to pad with 0 each feature which is not present. You said that you're going to add too much noise to the network, but I don't see the problem (please correct me if I'm wrong). For example we have two nodes, the first one has only 2 features with the 3rd one missing and the second node has all features X=[[1,2,0], [3,4,5]]. Now we can project the nodes to a hidden representation (pretty common). I'm going to use a weight matrix of W=[[1], [1], [1]]. The output of XW will be [[3], [12]]. Now let's add a new feature to the second node X=[[1,2,0, 0], [3,4,5,6]] and apply the same transformation W=[[1], [1], [1], [1]] the output will be [[3], [18]] you can see that the first node is not affected by the number of missing features.
Another way you could achieve this if you don't want to use the projection could be using a mask. For example give the same example above we could create a mask M=[[1,1,0], [1,1,1]] where each entry represents if a specific feature is present in a specific node. Now usually a GCN layer is defined as H=f(AHW) where A is the adjacency matrix. We could change the propagation rule to H=f(AH*MW) where * is the pointwise multiplication. Like this if a node is missing a feature it can not "access" information from others that are having that feature.

Hello @razvanc92, thank you. I like the masking idea, thank you for the suggestion.

My immediate suggestion would be to zero-fill the missing values, but I recalled the below comment suggesting a more sophisticated method:

Karim: How to deal with different size of feature vectors?
Nabila: That's a problem I'm actually working on. I've seen that you can create separate networks for each type of node feature, and sort of project them - so train them separately, and project them to the same size.
Or you can do concatenation so you don't have to worry about that, but at some point they all need to be the same size to do classification at the end.

A Literature Review on Graph Neural Networks


text:
Are there optimizers that schedule their learning rate, momentum etc. autonomously?
I'm aware there are some optimizer such as Adam that adjust the learning rate for each dimension during training. However, afaik, the maximum learning rate they can have is still determined by the user's input.
So, I wonder if there are optimizers that can increase/decrease their overall learning rate and other parameters (such as momentum or even weight decay) autonomously depending on some metric, e.g., validation loss, running average of gradients etc. ?


text:
How do I train a multiple-speaker model (speech synthesis) based on Tacotron 2 and espnet?
I'm new to Speech Synthesis & Deep Learning. Recently, I got a task as described below:
I have problem in training a multi-speaker model which should be created by Tacotron2. And I was told I can get some ideas from espnet, which is a end-to-end audio tools library. In this way, I found a good dataset called libritts: http://www.openslr.org/60/. And it's also found at espnet: https://github.com/espnet/espnet#tts-results

Here is my initial thought：

Download libritts corpus / Read the espnet code: ../espnet/egs/libritts/tts1/run.sh, learning how to train the libritts corpus by pytorch-backend.

Difficulty: But I cannot get it across that how the author trained a libritts.tacotron2.v1 as I didn't found anything about tacotron2 along those shells related to run.sh. Maybe He didn't make those codes open-source.

Read the tacotron2 code and tune it into a multi-speaker network:

Difficulty: I found the code is really complex.... I just got lost in reading these codes, without a clear understanding about how to tune this model. Cause Tacotron2 was designed with a LJSpeech Dataset(only 1 person).

Training the multi-speaker model with tiny set of dataset (http://www.openslr.org/60/) to save time.

they contains about 110 people's data, which can be enough for my scenario.

In the end:
Coud you please help me about my questions. I've been puzzled by this problem for a long time...


text:
Neural network seems to just figure out the probability of a specific result
I am really new to neural networks, so i was following along with a video series, created by '3blue1brown' on youtube. I created an implementation of the network he explained in c++. I am attempting to train the network to recognize hand written characters, using the MNIST data set. What seems to be happening is, rather than actually learn how to recognize the characters, it is just learning how many of each input there is in the data set and doing it with probability. When testing on a smaller dataset this is more noticeable, for example i was testing on a set with 100, and the numbers that were more frequent would always have a slightly higher activation at the end, and others where very close to 0. Here is my code if it helps:
#include <random>
#include <vector>
#include <iostream>
#include <fstream>
#include <cmath>

double weightMutationRate = 1;
double biasMutationRate = 1;

//Keeps track of the weights, biases, last activation and the derivatives
//of the weights and biases for a single node in a neural network.
struct Node
{
  std::vector<double> weights;
  std::vector<double> derivWeights;
  double activation;
  double derivActivation;
  double bias;
  double derivBias;
};

//Struct to hold the nodes in each layer of a network.
struct Layer
{
  std::vector<struct Node> nodes;
};

//Struct to hold the layers in a network.
struct Network
{
  std::vector<struct Layer> layers;
  double cost;
};

//Stores the inputs and outputs for a single training example.
struct Data
{
  std::vector<double> inputs;
  std::vector<double> answers;
};

//Stores all of the data to be used to train the neural network.
struct DataSet
{
  std::vector<struct Data> data;
};

//Generates a double by creating a uniform distribution between the two arguments.
double RandomDouble(double min, double max)
{
  std::random_device seed;
  std::mt19937 random(seed());
  std::uniform_real_distribution<> dist(min, max);
  return dist(random);
}

//Constructs a network with the node count in each layer defined with 'layers'.
//the first layer will not have any weights and biases and will simply have
//the activation of the input data.
struct Network CreateNetwork(std::vector<int> layers, double minWeight = -1, double maxWeight = 1, double minBias = -1, double maxBias = 1)
{
  //Network to construct.
  struct Network network;
  //Used to store the nodes in the previous layer.
  int prevLayerNodes;
  //Iterates through the layers vector and constructs a neural network with the values in
  //the vector determining how many nodes that are in each of the layers.
  bool isFirstLayer = true;
  for (int layerNodes : layers)
  {
    //Layer to construct.
    struct Layer layer;
    //Creating the nodes for the current layer.
    for (int i = 0; i < layerNodes; i++)
    {
      //Node to construct
      struct Node node;
      //Checks to see if the current layer is not the input layer, which does not have
      //any weights or biases.
      if (!isFirstLayer)
      {
        //Creating weights for the connections between this node
        //and the nodes in the previous layer.
        for (int i = 0; i < prevLayerNodes; i++)
        {
          //Getting a random double for the weight, between the bounds set in the arguments.
          double inputWeight = RandomDouble(minWeight, maxWeight);
          //Adding the inputWeight to the current node.
          node.weights.push_back(inputWeight);
          //Adding a 0 to the deriv weights for the weight just added.
          node.derivWeights.push_back(0.0);
        }
        //Getting a random double for the bias, between the bounds set in the arguments.
        double bias = RandomDouble(minBias, maxBias);
        //Adding the bias to the current node.
        node.bias = bias;
        //Adding the node to the layer.
      }
      layer.nodes.push_back(node);
    }
    //Updating the isFirstLayer variable if the current layer is the input layer.
    if (isFirstLayer)
    {
      isFirstLayer = false;
    }
    //Updating the prevLayerNodes variable for use in the next layer.
    prevLayerNodes = layerNodes;
    //Adding the layer to the network.
    network.layers.push_back(layer);
  }
  //Returning the constructed network.
  return network;
}

//Outputs the network passed to the networkPrint.txt file.
void PrintNetwork(struct Network network)
{
  std::cout << "Printing network ..." << std::endl;
  std::ofstream networkPrintFile;
  networkPrintFile.open("networkPrint.txt");
  //Iterates through each of the layers in teh network.
  for (int i = 0; i < network.layers.size(); i++)
  {
    std::cout << "Layer : " << i << std::endl;
    networkPrintFile << "Layer " << i << ":" << std::endl;
    //Iterates through each of the nodes in the current layer.
    for (int j = 0; j < network.layers[i].nodes.size(); j++)
    {
      networkPrintFile << "\t" << "Node " << j << ":" << std::endl;
      //Outputs the node's activation into networkPrintFile.
      double activation = network.layers[i].nodes[j].activation;
      networkPrintFile << "\t\t" << "Activation" << ": " << activation << std::endl;
      //Outputs the node's derivActivation into networkPrintFile.
      double derivActivation = network.layers[i].nodes[j].derivActivation;
      networkPrintFile << "\t\t" << "Deriv Activation" << ": " << derivActivation << std::endl;
      double bias = network.layers[i].nodes[j].bias;
      double derivBias = network.layers[i].nodes[j].derivBias;
      //Outputs the bias and derivative of the bias.
      networkPrintFile << "\t\t" << "Bias" << ": " << bias << std::endl;
      networkPrintFile << "\t\t" << "Deriv Bias" << ": " << derivBias << std::endl;
      //Iterates through all of the inputWeights in the current node.
      networkPrintFile << "\t\t" << "Weights" << ":" << std::endl;
      for (int k = 0; k < network.layers[i].nodes[j].weights.size(); k++)
      {
        double inputWeight = network.layers[i].nodes[j].weights[k];
        double derivWeight =  network.layers[i].nodes[j].derivWeights[k];
        networkPrintFile << "\t\t\t" << "Weight " << k << ":" << std::endl;
        networkPrintFile << "\t\t\t\t" << "Value" << ":" << inputWeight << std::endl;
        networkPrintFile << "\t\t\t\t" << "Derivative" << ":" << derivWeight << std::endl;
      }
    }
  }
  std::cout << "Done" << std::endl;
}

//Takes and input and peforms a mathematical sigmoid
//function on it and returns the value.
//             1
//  σ(x) = ---------
//          1 + e^x
double Sigmoid(double input)
{
  double expInput = std::exp(-input);
  double denom = expInput + 1;
  double value = 1 / denom;
  return value;
}

//Returns the activation of the node passed in give the previous layer.
double CalculateNode(struct Node &node, struct Layer &prevLayer)
{
  //Keeps a runing total of the weights and activations added up so far.
  double total = 0.0;
  int weightCount = node.weights.size();
  //Iterated through each of the weights, and thus each of the
  //nodes in the previous layer to find the weight * activation.
  for (int i = 0; i < weightCount; i++)
  {
    //Calculated the current weight and activation and
    //adds it to the 'total' variable.
    double weight = node.weights[i];
    double input = prevLayer.nodes[i].activation;
    double value = weight * input;
    total += value;
  }
  //Add the node's bias to the total.
  total += node.bias;
  //Normalises the node's activation value by passing it through
  //a sigmoid function, which bounds it between 0 and 1.
  double normTotal = Sigmoid(total);
  //Returns the caclulated value for this node.
  return normTotal;
}

//Adds the activation values to a layer passed in, given the previous layer.
void CaclulateLayer(struct Layer &layer, struct Layer &prevLayer)
{
  //Iterates through all of the nodes and calculated their activations.
  for (struct Node &node : layer.nodes)
  {
    double activation = CalculateNode(node, prevLayer);
    //Setting the activation to the node.
    node.activation = activation;
  }
}

//Takes in the first layer of the neural network and interates through the
//nodes and sets each input to each node in a loop.
void SetInputs(struct Layer &layer, std::vector<double> inputs)
{
  for (int i = 0; i < layer.nodes.size(); i++)
  {
    //Setting the node's activation to the corrosponding input.
    layer.nodes[i].activation = inputs[i];
  }
}

//Takes in a network and inputs and calculates the value of
//activation for every node for a single input vector.
void CalculateNetwork(struct Network &network, std::vector<double> inputs)
{
  //Setting the activations of the first layer to the inputs vector.
  SetInputs(network.layers[0], inputs);
  //Iterates through all of the layers, apart from the first layer, and
  //calculated the activations of the nodes in that layer.
  for (int i = 1; i < network.layers.size(); i++)
  {
    //Getting the layer to calculate to activations on and the
    //previous layer, which already has it's activations calculated.
    struct Layer currentLayer = network.layers[i];
    struct Layer prevLayer = network.layers[i - 1];
    //Calculating the nodes on the current layer.
    CaclulateLayer(currentLayer, prevLayer);
    //Setting the currentLayer back into the network struct with
    //all of the activations in it now calculated.
    network.layers[i] = currentLayer;
  }
}

//Caclulates the sum of the differences between the outputs and the correct
//values squared.
//
//  Cost = Σ((a-y)^2)
//
double CalculateCost(struct Network &network, std::vector<double> correctOutputs)
{
  //Keeps track of the current sum of the costs.
  double totalCost = 0.0;
  //The layer of the network that holds the calculated values, the
  //last layer in the network.
  struct Layer outputLayer = network.layers[network.layers.size() - 1];
  //Loops through all the node sin the output layer and compared them
  //to their corresponding correctOutput value, calculates the cost
  //and adds it to the running total, totalCoat.
  for (int i = 0; i < outputLayer.nodes.size(); i++)
  {
    struct Node node = outputLayer.nodes[i];
    double calculatedActivation = node.activation;
    double correctActivation = correctOutputs[i];
    double diff = calculatedActivation - correctActivation;
    double modDiff = diff * diff;
    //Adding the cost to the sum of the other costs.
    totalCost += modDiff;
  }
  //Returning the value of the calculated cost.
  return totalCost;
}

//Takes in the output layer of the network and calculates the derivatives of the
//cost function with respect to the activations in each node. this value is then
//stored on the Node struct.
void LastLayerDerivActivations(struct Layer &layer, std::vector<double> correctOutputs)
{
  //Iterating through all the nodes in the layer.
  for (int i = 0; i < layer.nodes.size(); i++)
  {
    //Getting the values of the node output and correct output.
    double activation = layer.nodes[i].activation;
    double correctOutput = correctOutputs[i];
    //Caclulating the partial derivative of the cost function with respect
    //to the current node's activation value.
    double activationDiff = activation - correctOutput;
    double derivActivation = 2 * activationDiff;
    //Setting the activation partial derivative to the layer passed in.
    layer.nodes[i].derivActivation = derivActivation;
  }
}

//Returns the derivative of the sigmoid function.
//   d
//  ---- σ(x) = σ(x)(1 - σ(x))
//   dx
double DerivSigma(double input)
{
  double sigma = Sigmoid(input);
  double value = sigma * (1 - sigma);
  return sigma;
}

//Takes in a node and the layer that the node takes inputs from and adds.
//to the derivWeight and derivBias of the node and adds to each of the
//deriv activations in the previous layer for them to be used in this
//function to calculate their derivatives.
void NodeDeriv(struct Node &node, struct Layer &prevLayer)
{
  //Starting the total at the bias.
  double total = node.bias;
  //Looping through all the weights and biases to find z(x).
  //  z(x) = a w + a w + ... + a w + b
  //          1 1   2 2         n n
  for (int i = 0; i < node.weights.size(); i++)
  {
    double weight = node.weights[i];
    double activation = prevLayer.nodes[i].activation;
    double value = weight * activation;
    //Adding to the running total for z(x).
    total += value;
  }
  //Finding the derivative of the cost function with respect to the
  //z(x) by multiplying the DerivSigma() by the node's derivActivation
  //using the chain rule.
  double derivAZ = DerivSigma(total);
  double derivCZ = derivAZ * node.derivActivation;
  //The derivative of the cost with respect to the bias is the same as
  //the derivative of the cost function with respect to z(x) since
  //d/db z(x) = 1
  node.derivBias += derivCZ;
  //Iterating through all of the nodes and weights to find the derivatives
  //of all of the weights for the node and the activations on the
  //previous layer.
  for (int i = 0; i < node.weights.size(); i++)
  {
    // dc/dw = dc/dz * activation
    double derivCW = derivCZ * prevLayer.nodes[i].activation;
    // dc/da = dc/dz * weight
    double derivCA = derivCZ * node.weights[i];
    //Adding the weights and activations to the node objects.
    node.derivWeights[i] += derivCW;
    prevLayer.nodes[i].derivActivation += derivCA;
  }
  //Resetting the activation derivative.
  node.derivActivation = 0;
}

//Takes in a layer and iterates through all the nodes in order to find
//the derivatives of thw weight and biases in the current layer and the
//activations in the previous layer.
void LayerDeriv(struct Layer &layer, struct Layer &prevLayer)
{
  for (int i = 0; i < layer.nodes.size(); i++)
  {
    NodeDeriv(layer.nodes[i], prevLayer);
  }
}

//Takes in a network and uses backpropogation to find the derivatives of
//all the nodes for a single training example.
void NetworkDeriv(struct Network &network, std::vector<double> expectedOutputs)
{
  //Calculating the derivatives of the activations in the last layer.
  LastLayerDerivActivations(network.layers[network.layers.size() - 1], expectedOutputs);
  //Looping through all the layers to find the derivatives of all of
  //the weights and activations in the network for this training example.
  for (int i = network.layers.size() - 1; i > 0; i--)
  {
    LayerDeriv(network.layers[i], network.layers[i - 1]);
  }
}

//Takes in an input string and char and will return a vector of the string split
//by the char. The char is lost in this conversion.
std::vector<std::string> SplitString(std::string stringToSplit, char delimiter)
{
  //Creating the output vector.
  std::vector<std::string> outputVector;
  //Initialising the lastDelimiter to -1, since the first string should be split as if
  //the char before it was the splitter.
  int lastDelimiterIndex = -1;
  for (int i = 0; i < stringToSplit.size(); i++)
  {
    //Getting the current char.
    char chr = stringToSplit[i];
    //If the current char is the delimiter, create a new substring in the vector.
    if (chr == delimiter)
    {
      //Creating the new substring at the delimiter and adding it to the end
      //of the output vector.
      std::string subString = stringToSplit.substr(lastDelimiterIndex + 1, i - lastDelimiterIndex - 1);
      outputVector.push_back(subString);
      //Setting the last delimiter variable to the current character.
      lastDelimiterIndex = i;
    }
  }
  //Adding the last section of the string to the output vector, since there is no
  //delimiter and will not be added in the for loop.
  std::string subString = stringToSplit.substr(lastDelimiterIndex + 1, stringToSplit.size() - lastDelimiterIndex - 1);
  outputVector.push_back(subString);
  //Returning the split string as a vector of strings.
  return outputVector;
}

//Takes in a vector of strings and converts it to a vector of doubles. normalise argument
//sets what value will be taken to be 1, and other numbers will be a fraction of that.
//Set normalise to 0 to disable normalisation.
std::vector<double> ConvertStringVectorToDoubleVector(std::vector<std::string> input, int normalise = 0)
{
  std::vector<double> convertedVector;
  //Iterating through all the strings int the input vector.
  for (std::string str : input)
  {
    //Converting the string into a double.
    double value = stod(str);
    //Checks to see if normalisation is enabled.
    if (normalise != 0)
    {
      //Normalising the double.
      value /= normalise;
    }
    //Adding the double to the output vector.
    convertedVector.push_back(value);
  }
  //Returning the converted vector.
  return convertedVector;
}

//Takes in a string of data and uses it to create a DataSet object to
//be used in the training of the neural network.
struct DataSet FormatData(std::string dataString)
{
  struct DataSet dataSet;
  //Splitting the input string into the seperate images.
  std::vector<std::string> imageSplit = SplitString(dataString, '|');
  //Looping through all of the images.
  for (int i = 0; i < imageSplit.size(); i++)
  {
    //Getting the current image string.
    std::string imageData = imageSplit[i];
    //Splitting the image between the inputs and expected outputs.
    std::vector<std::string> ioSplit = SplitString(imageData, '/');
    std::string inputs = ioSplit[0];
    std::string outputs = ioSplit[1];
    //converting the input and output strings into string arrays of the values.
    std::vector<std::string> inputVectorString = SplitString(inputs, ',');
    std::vector<std::string> outputVectorString = SplitString(outputs, ',');
    //Converting the string arrays into double arrays and normalising the input doubles.
    std::vector<double> inputVector = ConvertStringVectorToDoubleVector(inputVectorString, 255);
    std::vector<double> outputVector = ConvertStringVectorToDoubleVector(outputVectorString);
    //Creating a new Data object.
    struct Data data;
    data.inputs = inputVector;
    data.answers = outputVector;
    //Adding the object to the dataset.
    dataSet.data.push_back(data);
  }
  //Returning the completed dataset.
  return dataSet;
}

//Takes in a filename and extracts all of the ascii data from the
//file and calls the FormatData function to create a DataSet object.
struct DataSet CreateDataSetFromFile(std::string fileName)
{
  //Opening file.
  std::ifstream dataFile;
  dataFile.open(fileName);
  //Storing fild data in a string.
  std::string data;
  dataFile >> data;
  //Creating DataSet object.
  struct DataSet dataSet = FormatData(data);
  //Returning completed DataSet object.
  return dataSet;
}

//Takes in a network and a Data object and runs the network and adds
//to the derivatives of the network for that one training example.
void NetworkIteration(struct Network &network, struct Data data)
{
  //Extracting the input and output data from the data object.
  std::vector<double> inputs = data.inputs;
  std::vector<double> outputs = data.answers;
  //Caclulating the activations of the network for this data.
  CalculateNetwork(network, inputs);
  //Caclulating the cost for this iteration and adding it to the total.
  double cost = CalculateCost(network, outputs);
  network.cost += cost;
  //Caclulating the derivatives for the network weights and biases
  //for this training example.
  NetworkDeriv(network, outputs);
}

//Takes in a node and caclulates the average of the derivatives over
//the dataset and then multiplies them by a fixes mutation rate and
//applies the derivatives to the node's values.
void GradientDecentNode(struct Node &node, int dataCount)
{
  //Iterating through all of the weights of the node.
  for (int i = 0; i < node.weights.size(); i++)
  {
    double weight = node.weights[i];
    double derivWeight = node.derivWeights[i];
    //Getting the average over all of the training data.
    derivWeight /= dataCount;
    //Applying a constant multiplier to alter the rate at which is mutates.
    derivWeight *= weightMutationRate;
    //Subtracting the derivative from the weight.
    node.weights[i] -= derivWeight;
    //Reseting the weight derivative
    node.derivWeights[i] = 0;
  }
  double bias = node.bias;
  double derivBias = node.derivBias;
  //Applying a constant multiplier to alter the rate at which is mutates.
  derivBias *= biasMutationRate;
  //Subtracting the derivative from the bias.
  node.bias -= derivBias;
  //Resetting the bias derivative.
  node.derivBias = 0;
}

//Takes in a layer and iterated through all of the nodes in the layer
//and applies all of their derivatives to them.
void GradientDecentLayer(struct Layer &layer, int dataCount)
{
  for (struct Node &node : layer.nodes)
  {
    GradientDecentNode(node, dataCount);
  }
}

//Takes in a network and iterated through all of the layers and applies
//all of the derivatives to them.
void GradientDecentNetwork(struct Network &network, int dataCount)
{
  for (int i = 1; i < network.layers.size(); i++)
  {
    GradientDecentLayer(network.layers[i], dataCount);
  }
}

//Iterates through all of the training data in dataSet and calculates the derivatives
//of the weights and biases and then peforms the gradient decent using the derivatives.
void TrainNetworkSingle(struct Network &network, struct DataSet dataSet)
{
  //Iterating through all of the training data.
  for (struct Data data : dataSet.data)
  {
    //Caclulating the network for a single training example.
    NetworkIteration(network, data);
  }
  //Peforming the derivatives.
  GradientDecentNetwork(network, dataSet.data.size());
}

void TrainNetwork(struct Network &network, struct DataSet dataSet, int iterations)
{
  for (int i = 0; true/*i < iterations*/; i++)
  {
    TrainNetworkSingle(network, dataSet);
    std::cout << network.cost << std::endl;
    network.cost = 0;
  }
}

int main()
{
  struct Network network = CreateNetwork({784, 784, 16, 16, 10});
  struct DataSet dataSet = CreateDataSetFromFile("data.txt");
  TrainNetwork(network, dataSet, 100);
  PrintNetwork(network);
  return 0;
}
```


text:
Strategy of using intermediate layers of a neural network as features?
There is a popular strategy of using a neural network trained on one task to produce features for another related task by "chopping off" the top of the network and sewing the bottom onto some other modeling pipeline.
Word2Vec models employ this strategy, for example.
Is there an industry-popular term for this strategy?  Are there any good resources that discuss its use in general terms?

This a typical transfer learning technique, a lot of people refer to it with fine-tunning. I would recommend that you have a look on PyTorch tutorial: it explains well how to use.

This is an example of Transfer Learning, wherein instead of starting the learning process from scratch, a pre-trained model is used and appended with custom layers to achieve additional functionality.


text:
Creating a noising model for NLP that models human noising
I'm trying to create a noising model that accurately reflects how people would noise name data. I was thinking of randomly switching out characters and creating a probability over which character gets switched in based on keyboard closeness and how similar anatomically another character looks to it. For example, "l" has a higher prob of being switched in with "|" and "k" cause "k" is close by on the keyboard and "|" looks like "l", but that requires a lot of hard coding and reward for that seemed low because that's not the only 2 ways people can noise things. I also had the same idea above except use template matching of every character to every other character but itself and that would give it a similarity score then divide that by the sum over all chars to get the probs. Any other suggestions? My goal is the maximize closeness to actual human noising.

authors: Георги Цонев


text:
Image-to-Image Regression for GO territory classification
I'm trying to implement a neural network that is able to generate an image indicating territory occupation given a board state for GO (a strategy board game). Input images are 19x19x1 grayscale images, with white pixels indicating white pieces, black pixels indicating black pieces, and gray pixels indicating unoccupied areas. Output images are 19x19x1 grayscale images with white pixels indicating white territory, black pixels indicating black territory, and gray areas indicating unassigned territories. A sample input and desired output image is as follows:

The images are quite small, so just to give an overview of trends I noticed: - Pixels surrounded by pixels of opposite colors are 'captured' pieces and therefore part of opponent territory - Two 'eyes' or closed groups of pieces comprising at least two open intersections are invincible or confirmed territory
While I'm not looking for exact specifications of network layers etc., I was hoping I could be given some direction as to what type of network to use, and what it should comprise. Looking at MATLAB documentation, I've found info about semantic segmentation, and autoencoder networks but neither of these seem particularly helpful. I know the question is a little broad, but I just need some direction more than anything. This kind of image recognition problem is a first for me.


text:
How to represent a state in a card game environment? (Wizard)
We are attempting to build an AI that manages to play the cardgame Wizard. So far er have a working network (based on the YOLO object-detection) that is abled to detect which cards are played. When asked it returns the color and rank of the cards on the table.
But now when starting to build an agent for the actual training I just cant figure out how to represent the states for this game.
In each round, each player gets the amount of cards represented by the round(one card in round one, two in round two and so on). Based on that the players estimate how many tricks they will win in this round. With ending the round the players calculate their points w.r.t their estimation.
So the agent have to estimate its future tricks and have to play depending on that strategy. So how do I encode that into a form that a neural network can work with?

It's been almost two years now but since I had the same question and still found this post, I will also post how I plan to solve it: I have not implemented or tested this solution yet.
We will think of the input to the neural network in several blocks.

block: the player's hand
block: information about which cards have been played so far
block: other information

Now some more detail:
1. Block:
Represent every card with two input neurons: one for color, one for value
.then make block 1's size as large as it can possibly be. In a game with three players, there will never be more than 20 cards on the player's hand, each one needs two neurons, so we get 40 neurons in block 1. (Less, if the AI is trained for fewer players)
color can also be split into five neurons instead of one (4 colors + 1 neutral color)
2. Block:
60 neurons that have values 0 or 1 depending on whether the card has been played yet or not.
3. Block:
• Some information about how many tricks the AI should still win. For example using a single neuron with input value:
"Number of tricks to be won by AI" divided by "number of tricks still available"
• color the AI has to serve -> 1, 4 or 5 neurons
Optional Inputs:
• value of currently winning card -> 1 neuron
• color of currently winning card -> 1, 4 or 5 neurons
• ratios of total points each player has scores so far to the average number of points
• tricks wanted to tricks available ratio (as above) for the opponents
One could also combine blocks 1 and 2 into 60 neurons with an extra possible state:

not played yet -> 0
played -> 1
in own hand -> -1

You should maybe ask a question about this representation. The main issue that I can see is that it has is large amounts of redundancy in block 1 due to cards being essentially the same data point (having a card in your hand is not reliant on its position but the encoding you use varies the position). IMO you would be better off with block 1 a fixed representation for all cards, same as block 2, although possibly a sorting approach to block 1 and using LSTM or transformer architecture to help process it would also work well.


text:
Sign Language to Speech conversion
Is there any solution about sign language to speech conversion for mobiles? Can anyone suggest me the flow and tools so that I may implement the solution for mobiles?


text:
Why can't VAE do sequence to sequence name generation?
I'm working on research in this sector where my supervisor wants to do cannonicalization of name data using VAEs, but I don't think it's possible to do, but I don't know explicitly how to show it mathematically. I just know empirically that VAEs don't do good on discrete distributions of latents and observed variables(Because in order to do names you need your latent to be the character at each index and it can be any ASCII char, which can only be represented as a distribution). So the setup I'm using is a VAE with 3 autoencoders, for latents, one for first, middle and last name and all of them sample each character of their respective names from the gumbel-softmax distribution(A form a categorical that is differentiable where the parameters is a categorical dist). From what I've seen in the original paper on the simple problem of MNIST digit image generation, the inference and generative network both did worse as latent dimension increased and as you can imagine the latent dimension of my problem is quite large. That's the only real argument for why this can't work, that I have. The other would have been it's on a discrete distribution, but I solved that by using a gumbel softmax dist instead.
This current setup isn't working at all, the name generations are total gibberish and it plateaus really early. Are there any mathematical intuitions or reasons that VAEs won't work on a problem like this?
As a note I've also tried semi-supervised VAEs and it didn't do much better. I even tried it for seq2seq of just first names given a first and it super failed as well and I'm talking like not even close to generation of names or the original input.

Im confused. For GANs you'd need Gumbell Sfotmax or REINFORCE or something similar, but for VAEs this can be done directly without approximation. I see your approach but have you tried just teacher forcing?

VAEs can't do this because they can't backprop on discrete distributions, this was stated in Bengio's paper. I don't know what teacher forcing is.

Interesting. I don't know how much that helps VAEs cause I don't think that helps it approximate the inference or generation. If anything a semi supervised VAE makes more sense

Have you tried this using a normal AE? How does exactly the same input and structure perform if it is just a standard AE? This might help troubleshoot potential logic errors that could be causing this behaviour. Sorry I can't help with a mathematical intuition, but it does seem odd to use something meant for continuous distributions for something discrete.

@Recessive true but there are discrete VAEs that handle this by using the discrete outputs as a parameter to a continuous dist like Gumbel softmax. A normal AE does pretty good

@user8714896 It's a long shot, but you might have some luck training an AE first, then applying gumbel softmax to the latent space of the AE to make it variational. Sort of what you're already doing (by the sounds of it) but in reverse.

@Recessive that sounds like a semi-supervised VAE. Well sort of, you're suggesting training it as a DAE then making it variational after, which is pretty interesting

@Recessive actually the more I think about it the more I like it

@user8714896 Let me know how it goes if you try it!

What's their terminal goal with a Name VAE? Is it just an academic challenge? If it has a real goal behind (like generating new names, or saving data with less storage) you could just suggest another plausible approach instead of trying to prove them wrong.


text:
Intellectual property in the age of Industry 4.0
I am looking for specific references describing guidance principles around the interplay between IP (intellectual property) and Artificial Intelligence algorithms. For example, Company A has a large dataset and Company B has advanced algorithmic capabilities (assume near-AI). How might Company A protect itself in a joint partnership with company B? Apologies if AI SE is not the place for this - any suggestions for the right site?


text:
Does this tutorial use normalization the right way?
There is this video on pythonprogramming.net that trains a network on the MNIST handwriting dataset. At ~9:15, the author explains that the data should be normalized.
The normalization is done with
x_train = tf.keras.utils.normalize(x_train, axis=1)
x_test = tf.keras.utils.normalize(x_test, axis=1)

The explanation is that values in a range of 0 ... 1 make it easier for a network to learn. That might make sense, if we consider sigmoid functions, which would otherwise map almost all values to 1.
I could also understand that we want black to be pure black, so we want to adjust any offset in black values. Also, we want white to be pure white and potentially stretch the data to reach the upper limit.
However, I think the kind of normalization applied in this case is incorrect. The image before was:

After the normalization it is

As we can see, some pixels which were black before have become grey now. Columns with few black pixels before result in black pixels. Columns with many black pixels before result in lighter grey pixels.
This can be confirmed by applying the normalization on a different axis:

Now, rows with few black pixels before result in black pixels. Rows with many black pixels result in lighter grey pixels.
Is normalization used the right way in this tutorial? If so, why? If not, would my normalization be correct?
What I expected was a per pixel mapping from e.g. [3 ... 253] (RGB values) to [0.0 ... 1.0]. In Python code, I think this should do:
import numpy as np
import imageio
image = imageio.imread("sample.png")
image = (image - np.min(image))/np.ptp(image)

This is one of my pet-peaves... Either using an inappropriate standardization/scaling/normalization or using the wrong word to describe what's been done. :). Good catch @thomas!

Yep, definitely a good catch.  Probably worth posting a polite correction in the comments section of the video.

The normalize() function used in the tutorial is the incorrect function to use for normalization in this context.  This normalize function is for finding L1 and L2 norms.  The correct layer/function to use for scaling between 0 and 1 would be Rescaling():
x_train_norm = Rescaling(scale=1./255, offset=0.0)(x_train)

You could rank every pixel in terms of brightness before normalization and after to verify that the ranks are preserved. Correct normalization would preserve the  ranks. From the pictures 1 and 2, it seems that the ranks were not preserved (e.g. the top right pixels went from grey to black).
The tutorial's normalization is done incorrectly: x_train is a an array of 2D images, but the normalization was applied along the first axis. What happened is that each column of every image was normalized relatively to itself (compare image 1 and 2 column-wise – the ranks were preserved). The normalization should have been applied along the second axis – that would normalize every pixel relative to the image it's in. In seems that incorrect normalization wasn't enough to sabotage the learning though!
Your proposed normalization should work fine.


text:
Tversky Loss paper implementation: Recall/Precision do not improve as stated
I have been trying to implement this paper and I am very much intrigued. I am working on a medical image problem where I have to segment very small specimens on Whole Slide Images (gigapixel resolution). Therefore my dataset is highly unbalanced and I am having a high false positives rate.
I did my research and found that paper that describes the implementation of Tversky Loss and Focal Tversky Loss. It also describes some modifications to the network architecture which I am postponing for now.
I implemented the loss (Pytorch) and ran some experiments with several alpha/beta combinations. Well, the results are easy to understand: higher alpha results in higher precision and a lower beta increases the recall and pushes the precision down. Basically, what this loss is doing is balancing my recall and precision, only. That is good, I can solve my False Positives issue but since this is a medical problem, a good recall is mandatory. In the paper, the results show that there is an improvement in the Precision/Recall and I cannot understand how is that possible and how I cannot replicate that. I am just weighing false positives and penalizing them, it does not seem enough to improve the model overall.
Regards


text:
What models will you suggest to use in Industrial Anomaly Detection and Predictive analysis on live streamed data?
I have been working on industrial data, that is fed live, I want to explore a few models which might suit for this the best.
The data are KPI data from the manufacturing Industry.


text:
How do I make my LSTM model more sensitive to changes in the sequence?
I have a many to one LSTM model for multiclass classification. For reference, this is the architecture of the model
    model.add(LSTM(147, input_shape=(1000, 147)))
    model.add(Dense(5, activation='softmax'))
    model.compile(loss='categorical_crossentropy',
                  optimizer='rmsprop',
                  metrics=['accuracy'])

The model is trained in 5 types of sequences is able to effectively classify each sequence I feed into the model with high accuracy. Now my new objective is to combine these sequences together to form a new sequence. For e.g  I denote the elements from the class '1' with the sequence:
[1,1,1,1,1,1,1] So when I input the above sequence into the LSTM model for prediction, it classifies the sequence as class '1' with accuracy of 0.99
And I denote the elements from class '2' with the sequence:
[2,2,2]
LIkewise for the above sequence, the LSTM model will classify the sequence as class '2' with accuracy of 0.99
Now I combine these sequences together and feed it into the model :
 New sequence : [1,1,1,1,1,1,1,2,2,2]
 However the model does not seem to be sensitive to the presence of the class '2' sequence and still classifies the sequence as class '1' with accuracy of 0.99.
How do I make the model more "sensitive", meaning that I would expect the LSTM model to still maybe predict class '1' but with a drop in accuracy? Or is the LSTM incapable of detecting the inclusion of class '2' sequences?
Thanks.


text:
Hand-Signs Recognition using Deep Learning Convolutional Neural Networks
I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is:
Training = 1250 Images/hand-sign
Validation = 625 Images/hand-sign
Testing = 625 Images/hand-sign
How should I proceed with training the model?:
1. Should I develop a model starting from fewer hand-signs (like 5) and then increase them gradually?
2. Should I start models from scratch or use transfer learning (VGG16 or other)
Applying data augmentation, I did some tests with VGG16 and added a dense classifier at the end and received these accuracies:
Train: 0.87610877
Validation: 0.8867307
Test: 0.96533334
Accuracy and Loss Graph
Test parameters:
NUM_CLASSES = 5
EPOCHS = 50
STEPS_PER_EPOCH = 125
VALIDATION_STEPS = 75
TEST_STEPS = 75
Framework = Keras, Tensorflow
OPTIMIZER = adam
Model:
model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(pool_size=(2,2)),

    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),

    Conv2D(128, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),

    Conv2D(256, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),

    Conv2D(512, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),

    Flatten(),
    Dense(512, activation='relu'),

    Dense(NUM_CLASSES, activation='softmax')
])

If I try images with slightly different background and predict the classes (predict_classes()), I do not get accurate results. Any suggestions on how to make the model robust?

Please, next time, focus on a single problem and questions. For each of your questions, open a different thread/post.

I feel your problem might not be with the model itself but with the dataset. If you only have $2500$ images for $24$ labels (hand-signs) that gets you roughly $104$ images per label. This is very little for the models I train (~$80K$ images in the smallest of cases). In my view you got a really decent accuracy at validation and test time for the size of your dataset.
But answering your questions:

Starting from few labels and extending is usually helpful when your model is too deep and suffers from convergence problems. Your model is simple enough not to suffer those problems so I would go learning the $24$ labels at once.
Transfer learning can help a lot at reducing training times. For example, if you start from a VGG classifier that detects hands there is a good chance that the weights of your convolutional layers are already almost configured for your use case.

Generally speaking, the easiest way to increase the accuracy of your model is to use one of these 2 methods:

Increase the dataset: I am not sure if it is possible in your case, maybe you can use image augmentation (rotation, zooming in/out, changes in the color space...).
Increase the depth of your model (provided that you have a big dataset).

If you already fulfilled those items, then you can go and make changes to the model architecture or loss function. Looking at your model and from the top of my head I would try to add Batch-Normalization for the convolutional layers.
Hope this helps a bit :)

Thanks. I am actually having 2500 images for each hand-sign (each label). So total images are around 60000. I have used data augmentation before training the above model. I will try VGG16 classifier like:
model = Sequential([
vgg16_base,
Dropout(0.2),
Flatten(),
Dropout(0.2),
Dense(512, activation='relu', kernel_regularizer=l2(0.01)),
Dense(NUM_CLASSES, activation='softmax')
])
Also, I would unfreeze the last one/two layers. But should I add any other Conv2D, Maxpool, Flatten, or Dense Layers at the end?

Ah really nice then! 60K images is ok. I feel you could train VGG starting from some pretrained weights (for ILSVRC for example) instead of freezing layers. Regarding adding layers, you should add Conv2D in the Conv2D  stage before flattening to the Dense stage (same with Dense, add them only in the Dense layers stage). And yes, you could make your model deeper since you have enough training data that normally boost performance because it allows to learn more complex nuances of the use case.


text:
How to encode board before input into the neural net?
Currently I'm working on an educational project (implementation of AlphaZero approach to different types of board games).
My biggest concern at the moment is how to encode board before input into the neural network?
For example, how can this be done for Kalah game?
Kalah board has two straight rows of six pits, and two large score-houses.
Each pit contains 6 seeds (so there are 72 seeds)

Knowing how to generate the input to a neural network is more of an art than a set of rules. I would look up different ways which have been proven already (such as chess or go) to determine how you could do it for something as Kalah. And of course you should try different methods and compare them, certainly when you are doing this for educational purposes.

authors: Немања Бандука


text:
What is the use of concatenate layer in CNN?
I am not asking what does concatenate layer does in general in point of mathematical operation. But at feature level, what significance does it provide. Does it helps removing false negatives or does it prevents over-fitting? Do give the reference of papers regarding this topic.

Which "concatenate" layer are you talking about? Can you please link us to a CNN (implementation or definition in a paper) that uses the "concatenate" layer you're talking about?

Add layer as in Keras which is equivalent to concatenate, https://keras.io/layers/merge/


text:
Machine learning frameworks for esoteric languages
Is there a machine learning framework/library for any of the esoteric languages, such as the ones listed here ?

Why would you want to develop machine learning software in an esoteric language?

I dont want to. Im curious if someone else did.

I haven't yet found any machine learning library written with an esoteric language, but I found a tic-tac-toe program written in Brainfuck. See also https://mitxela.com/projects/ai_in_brainfuck.

Questions asking for tools, such as software libraries or APIs, are off-topic here. We focus on the theoretical, social and philosophical aspects of artificial intelligence (so this excludes recommendations of software libraries). If you have again this type of question in the future, you should ask it here: https://softwarerecs.stackexchange.com/

authors: Влад Плейзор


text:
Details on body measurements prediction
if someone want to do mobile app for body measurements prediction, please what are the necessary things to start with. I need details explanation on this.


text:
I need to select the image from a predefined dataset that are the closest to the input, is this possible or do I even need to use ML/AI?
So as the title states, I have a set of images and I want to process input images and need to select the image that "looks" the most like the input image.
I know I've seen something similar where the code could guess who's face was in a picture, I guess I want something like that but for general images.
Sorry if this is a stupid question, but any suggestions or points at resources would be greatly appreciated.

You will only need to use ML or AI if
a) the dataset is very big.
b) It is difficult to get the meaning or value of the image.(ex: group of ants, photos of stars in night sky)
And many more.
Below is the way I thought it can be done through a machine(with deeplearning, CNN)
1. Train the model
        arrange the images in dataset into clusters (the number of clusters can
        be decided based on the criticality of the application, more critical the
        application more should be the number of clusters) .

Predict about input
    Then predict the nearest image in the dataset to the provided image.

I recently listened a podcast in which, a problem is approached similarly. The podcast is about youtube predictions and the result is to provide closest video from the youtube videos ( here the dataset).Below is the podcast.
    ( https://open.spotify.com/episode/6PcMtVXR58i7iu8kLH40Wd?si=feyu0LjESoiE479xHkaqSA )


text:
Where can I find good tutorials on user tailored recommendation system for web?
I'm currently working on my uni project, but I have no idea where to start for the user tailored recommendation system on web. Where can I find a good guide on it, preferrably on languages like php and javascript.

Here is an example with Node.js.


text:
What are some of the best methods in detecting facial movement using state-of-the-art machine learning models?
I am currently working on implementing a lip reading system in Python using machine learning and image processing. Currently, two initial implementations have provided promising results, albeit not perfect: the LipNet model and the Google Cloud AutoML Vision API. Before I dive head on into these two different systems, I was wondering if anyone on here had been exposed to any other alternatives for lip reading or any other facial detection issue, and their experiences with these models.
Any information will be greatly appreciated.


text:
Implementing Actor-Critic with Experience Replay for Continuous Action Spaces
I have been trying to implement the ACER algorithm for continuous action spaces in reinforcement learning. The paper for the algorithm can be found here:

Sample Efficient Actor-Critic with Experience Replay

I have implemented parts of the algorithm, but I have encountered some roadblocks that I have not been able to figure out.
The following is the pseudo-code provided in the paper:

Here is what I have implemented so far:
states = tf.convert_to_tensor(trajectory.state)
actions = tf.squeeze(tf.convert_to_tensor(trajectory.action), axis=-1)
rewards = tf.convert_to_tensor(trajectory.reward)
dones = tf.convert_to_tensor(trajectory.done)

explore_means, state_values, action_values = actor_critic(states, actions)

average_means, *_ = brain.average_actor_critic(states)

k = len(trajectory.state)
d = env.action_space.shape[0]

# Policies
explore_policies = k*[None]
behavior_policies = k*[None]
average_policies = k*[None]

# Tracking
explore_actions = np.zeros([k, d])
importance_weights = np.zeros([k, 1])
explore_importance_weights = np.zeros([k, 1])
truncation_parameters = np.zeros([k, 1])

for i in range(k):

    behavior_policy = tfd.MultivariateNormalDiag(
        loc=trajectory.statistics[i],
        scale_diag=tf.ones(d)*POLICY_STD
    )

    explore_policy = tfd.MultivariateNormalDiag(
        loc=explore_means[i],
        scale_diag=tf.ones(d)*POLICY_STD
    )

    average_policy = tfd.MultivariateNormalDiag(
        loc=average_means[i],
        scale_diag=tf.ones(d)*POLICY_STD
    )

    explore_action = explore_policy.sample()

    importance_weight = explore_policy.prob(actions[i]) / behavior_policy.prob(actions[i])
    explore_importance_weight = explore_policy.prob(explore_action) / behavior_policy.prob(explore_action)

    truncation_parameter = min(1, (importance_weight)**d)

    behavior_policies[i] = behavior_policy
    explore_policies[i] = explore_policy
    average_policies[i] = average_policy
    explore_actions[i] = explore_action
    importance_weights[i] = importance_weight
    explore_importance_weights[i] = explore_importance_weight
    truncation_parameters[i] = truncation_parameter

explore_actions = tf.convert_to_tensor(explore_actions, dtype=tf.float32)
importance_weights = tf.convert_to_tensor(importance_weights, dtype=tf.float32)
explore_importance_weights = tf.convert_to_tensor(explore_importance_weights, dtype=tf.float32)
truncation_parameters = tf.convert_to_tensor(truncation_parameters, dtype=tf.float32)

q_ret = values[-1] if not dones[-1] else tf.zeros(1)
q_opc = tf.identity(q_ret)

for i in reversed(range(k - 1)):

    q_ret = rewards[i] + GAMMA*q_ret
    q_opc = rewards[i] + GAMMA*q_opc

    # Compute quantities needed for trust region updating
    c = TRUNCATION_PARAMETER

    with tf.GradientTape(persistent=True) as tape:

        tape.watch(explore_policies[-2].loc)

        log_prob = explore_policies[-2].log_prob(actions[-2])
        explore_log_prob = explore_policies[-2].log_prob(explore_actions[-2])

        kl_div = tfp.distributions.kl_divergence(average_policies[-2], explore_policies[-2])

    lp_grad = tape.gradient(log_prob, explore_policies[-2].loc)
    elp_grad = tape.gradient(explore_log_prob, explore_policies[-2].loc)
    kld_grad = tape.gradient(kl_div, explore_policies[-2].loc)

    term1 = min(c, importance_weights[-2])*lp_grad*(q_opc - state_values[-2])
    term2 = tf.nn.relu(1 - (c / explore_importance_weights[-2]))*(action_values[-2] - state_values[-2])*elp_grad

    g = term1 + term2

So the goal here was to implement it exactly the way they have it in the paper and then afterwards optimize it for doing batches of trajectories. For now, however, it is sufficient for the purposes of learning.
My confusion comes from the use of differentials in this algorithm. I don't know what the specific type is for them, such as whether they are using it for the loss value to optimize on or if they are storing the gradients that will be used for updating. Another issue I am having is that it is not clear what they mean by this line:

I don't understand why they are using a partial derivative here if there is clearly more than one parameter in the neural network. Maybe they mean the gradient I am not sure, however.
So what would be helpful is if anybody has some guidance as to what they are getting at in this portion of the paper or if anybody has some advice as to what steps need to be taken in TensorFlow 2.0 to implement this algorithm.
Any help would be greatly appreciated! Thanks!!


text:
Similarity of Images (CBIR) for two different cameras
Suppose we have a top down picture of an object (let's say it is a shoe) from an overhead camera. Also suppose we have a database of various objects from a closeup camera. If we feed the top-down picture of the hsoe into a CBIR model, then this picture would obviously have very low similarity with the images in the database. But would the image with the highest similarity score in the database still be a shoe even though the absolute score is small?


text:
Is there an AI that can complete Deezer Spleeter work?
I have used Deezer Spleeter but it produces echoes aside the stems, so I wonder if there is already an AI that remove echoes noises.


text:
Unable to understand V* at infinite time horizon using Bellman equation for solving MDP
I've been following the Berkeley cs188's assignment (I'm not taking the course). Currently, they don't show the solution in the gradescope unless I get it correct.

My reasoning was
$V^*(a)$ = 10 fixed, because the optimal action is to terminate and receive the reward 10.
$V^*(b) = 10 \times 0.2 = 2$ using Bellman optimality eqn $V^*(s) = R(s)+ \gamma \ \rm{max}_{a} \sum_{s'} P(s'|s,a) V^*(s')$, where the optimal actionfrom b is to left.
Similarly, I get $V^*(c) = 10 \times (0.2)^2 = 0.4$
For the state $d$, it is optimal to move to the right and exit at $e$ to receive 1, therefore $V^*(d) = 1 \times 0.2 = 0.2$.
And $V^*(e) = 1$ fixed.
However, there autograder says it's incorrect and doesn't show explanation. Can anyone explain what the right approach or answer is?

Nevermind. I found that above answer is indeed correct, but the gradescope has a bug (it requires the format to be .2 instead of 0.2).