text:
How to include exploration in Gaussian policy
When dealing with continuous action spaces, a common choice when designing a policy in policy gradient methods is to learn mean and variance of actions for a specific state and then simply sample from the normal distribution defined by the learned mean and variance to get an action.
My first question is, is explicit exploration strategy even needed is such cases, because the dose of randomness in actions would come from the sampling itself, on the other hand there could probably be cases where we would be stuck in a local optimum just by sampling.
My second question is, in case that explicit exploration is needed, how would one approach this problem of exploration for this specific setup.


text:
Approach to classify a photo and extract text from it
I am trying to make a personal ML project where my objective is using a photo from an invoice, for instance, a Walmart invoice, classify it as being a Walmart invoice and extract the total amount spent. I would then save this information in a relational database and infer some statistics about my spendings. The goal would be to classify invoices not only from Walmart but from the most frequent shops where I spend money and then extract the total amount spent. I already do this process manually, I insert my spendings in a relational database. I have a bunch of photos from different invoices that I have recorded over the past year for this purpose (training a model).
What algorithms would you guys recommend? From my point of view, I think that I need some natural language processing to extract the total amount spent and maybe a convolutional neural network to classify the invoice as being from a specific store?
Thanks!

There are three functional aspects to this project, each of which could be a sub-project, even though they are interrelated. The decoupling of these aspects in the associated R&D will likely improve the rate of development.

Classify images — in this case, the vendor brand
Locate and orient the text
OCR the text — in this case, the total

These are certainly feasible in the particular case of invoices from vendors, so the entire project certainly is.
The goal is to produce two fields of information from input images.

Vendor code
Total in some specific monetary unit

That past invoices, the names of the vendors, and the totals of each invoice are available as training data may be helpful. Whether the volume of training data is sufficient can only be determined through a very complicated application of theory to some of the data metrics or by trial. It is recommended to do both.

What algorithms would you guys recommend?

It would be irresponsible to define an AI design in terms of what people recommend for algorithm. The algorithms that guys recommend are already covered by the path names for files in the algorithms directory paths, the example's directory paths, and the sections of the documentation of any good AI framework. Let's first talk about the design and then the algorithm options, so there is more of a basis for algorithm selection than the largely random reading selections of the members of a site.
The models involved and which can be parameterized such that they can be tuned through training are partly defined by the three sub-projects above.
Analysis
For this project, there is no reason for the AI to recognize which text on the invoice is the vendor name. If there are a hundred vendor invoice types, the speed in which a single person can identify the key boundaries of each invoice type is orders of magnitude faster than the work necessary to develop a completely general algorithmic approach for automating that work. It would only be resource thrifty to develop such automation if there were thousands of vendors and constantly varying templates for invoices, which is likely in the case of personal finance.
These are the key bounds of the form quadrilaterals (four sided polygons) within which the following objects lie.

Vendor name text box
Total amount text box
Rectangle around the entire form

Digitizing the twelve points through one of the labelling programs is a model much more likely to produce a reliable system than the representation of three rectangular bounding boxes and tilts. This is because tilt angle requires at least two adjacent points anyway and the aspect ratio cannot be held constant in real scanning scenarios when the scanner might be replaced or produce different ratios with wear or the smoothness or wear characteristics of the paper invoice.
Doing all three things using a network would require more than one person's invoices for a decade, unless the person is a billionaire compulsive buyer with a team of secretaries doing scanning and data entry.
The models are then these.

Digitization of a quadrilateral document with varying contrast, brightness, tilt, horizontal pixels per inch, vertical pixels per inch, relative location and size of vendor name, and relative location and size of invoice total
Numeric characters and other monetary characters in a rectangular box
Brand image for the vendor, which may include a logo and type of arbitrary and possibly unique font

Now we can talk about artificial network approaches.

Adjusting for contrast, brightness, tilt, resolution, locations, and sizes will require a customized input, loss function, and layer arrangement based on the geometry involved. A GRU network may gain some advantages if the documents can be ordered chronologically, since the collection then becomes a time series, the trends of which can be exploited.
Monetary values can best be done using OCR libraries.
Recognizing the brand is probably best done with CNN as a categorization machine.

The system must indicate if the vendor and total are not found, in which case a new template is indicated, and the twelve points must be digitized for this new type.
It is possible to do this with one single deep convolutional network, but, again, the data set would need to be augmented. The one other way to do this is to create a filled-out-invoice generator to produce the volume of stochastically variable data across the various variability dimensions listed above to train the deep CNN.

Since many of these problems have been tackled earlier and we have quite a few good tools to handle images and text, the task does not seem to be so very difficult. But then you would only find out after actually trying out the solutions suggested.
I suggest the following approach:

Use Tesseract and OpenCV to extract text from the images you have saved. You can refer to a good example of using tesseract with python here - https://www.pyimagesearch.com/2018/09/17/opencv-ocr-and-text-recognition-with-tesseract/
Evaluate the results to plan and consider your next steps. Refer to the publications linked to the tutorials such as the EAST algorithm which describes the challenges and point to other studies on this topic.  This will help you evaluate the other approaches which have been studied for different cases by other researchers.
Assuming the text detection has worked well, proceed forward by detecting parts of the document by using row detection (find continuous white-space extending horizontally) and tab stop detection (continuous white-space vertically). Use the Tesseract/openCV detected text blocks for this purpose.
Re-organize the detected text according to the layout identified by rows/columns. Next, recognize and discard the text you may not required, e.g. the logo, other associated details such as phone no of the store, tax breakup, change given, etc. These may be easily picked up by carefully crafted regular expressions.
Run a spell checker to remove errors in recognition (aspell, pyenchant, etc.)
Finally, from the results obtained, you could evaluate the remaining errors and fix them by either writing a set of regular expressions for easily identified OCR mistakes, or, else build a text model to learn and fix similar errors.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How to identify too small network in reinforcement learning?
I am using Open AI's code to do a RL task on an environment that I built myself.
I tried some network architectures, and they all converge, faster or slower on CartPole.
On my environment, the reward seems not to converge, and keeps flickering forever.
I suspect the neural network is too small, but I want to confirm my belief before going the route of researching the architecture.
How can I confirm that the architecture is the problem and not anything else in a neural network reinforcement learning task?

Check the function loss.
It might be that your environment is impossible to learn. However, most likely the network simply can't handle it. By measuring the loss during the learning stage, if you find it is always very high and does not decrease, it's a strong indication this might be the issue.
Because the network is too simple, when you optimize for some states, you ruin others. There is not formal way to find out if this is the case, but since the same algorithm works elsewhere, it's either a problem of your environment, or of the network.

@Gulzar it can. Not by virtue of being unable to learn, but by taking too long to do all the calculations do to its size. Ideally, we want the smallest and fastest network possible that can learn the function we are modelling

Loss not decreasing during the training does not signify that network don't train. It may mean network is in the exploration mode. I have observed that situation - accumulated reward growing steadily, which mean network is training well, but loss is not decreasing.
If you already know solution to some (simpler or other) version of your environment, you can train network in supervised manner to reproduce that solution. If network unable reproduce existing solution that is strong indication that network is too small or otherwise not good.
The other reason for accumulated reward oscillation could be network overfitting on latest training samples. In that case bigger replay buffer or more slow update of target network(if target network is used) may help.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Does leaky relu help learning if the final output needs negative values?
In an MLP with ReLU activation functions after each hidden layer (except the final),
Let's say the final layer should output positive and negative values.
With ReLU intermediary activations, this is still possible because the final layer, despite taking in positive inputs only, can combine them to be negative.
However, would using leaky ReLU allow faster convergence? Because you can pass in negative values as input to the final layer instead of waiting till the final layer to make things negative

In short, yes Leaky Relu helps in faster convergence if your output requires both positive and negative values. But the catch is that you need to tune the negative slope of Leaky Relu which is a hyperparameter to get better accuracy.

By "tuning the negative slope" I mean that there is a parameter to Leaky Relu which has to be tuned for better results.

Both the output and the the gradient at the negative part of leaky ReLU is 100 times lower than at the positive part. I doubt that they have any significant impact on training direction and/or on the final output of a trained model unless the model is severely underfitting.


text:
How does a neural network output text box location data?
I'm interested in creating a convolutional neural network or LSTM to locate text in an image. I don't want to OCR the text yet, just find the text regions. Yes, I know Tesseract and other systems can do this, but I want to learn how it works by building my own. All of the tutorials and articles I've seen so far have the CNN output to a classification - "image contains a cat", "image contains a dog". Okay that's nice, but it doesn't say anything about where it was found.
Can anyone point me to some information that describes the output layer of a NN that can give location information? Like, x-y co-ordinates of text boxes?


text:
How to design an ANN to give an answer that includes two different components?
As an experiment, I want to teach an ANN to play the game of Nim.

The normal game is between two players and played with three heaps of any number of objects. The two players alternate taking any number of objects from any single one of the heaps. The goal is to be the last to take an object.

The game is easily solvable and I already wrote a small bot that can play Nim perfectly to provide data sets for supervised learning.
Now I am struggling with the design question, how should I output the solution to a specific board state. The answer always consists of two components:

How many stones to take (a more or less arbitrary integer value)
Which heap to take the stones from (the index of the heap)

What are available design choices in this regard and is there a state-of-the-art design for this type of problem?

Nim is a simple game and it's really simple to build a bot that gives the optimal solution.

The correct move is to leave an odd number of piles of size 1

So when it comes to training an ANN to play a game, there are some this to keep in mind.

Fixed play area (This is taken care of as there are only 3 stacks)
Providing input to the model so it knows it's current state (You can use a simple array of integers [3, 4, 1] denoting the current state of the stacks)
Providing feedback. An important step to guide the network in the right direction (You already have an optimal bot to do this)

Now that you can easily cover all the requirements, it's pretty simple to teach the model.

Input - Current state of the model [3, 4, 5]
Output - Move that model will make [2, 0, 0]
Final output - Here you will have to add another layer with a custom function.
This is the important part, to direct the model. Check if the move made is GOOD, BAD or ILLEGAL and assign a custom output to the same. For example

GOOD return1 if it matches one of the optimal moves.
BAD return 0 if it's not the optimal move and
ILLEGAL return -1 if the move is not allowed.

Another thing to keep in mind is that this might require a slightly larger network as it as to learn complex functions that are not linearly separable.
You don't need to train that model by playing it against itself as you already have the optimal solution. Playing against itself is required only when you are trying to achieve something that better than the current best agent.
Check this out, it's pretty interesting Neural network to play snake

Good answer, just note the Nim can be played with N stacks.  There the winning strategy is to finish every move with a nim-sum of 0.  Nim-sum is the sum in binary, neglecting all carries from one digit to another.  see Wikipedia: https://en.wikipedia.org/wiki/Nim#Mathematical_theory


text:
How can VAE have near perfect reconstruction but still output junk when using random noise input
I am creating a VAE for time series data using CNNs. The data has 4800 timesteps and 4 features. It is standardized and normalized. The network I am using is implemented in Keras as follows. I have used a MSE reconstruction error:
# network parameters
(_, seq_len, feat_init) = X_train.shape
input_shape = (seq_len, feat_init)
intermediate_dim = 512
batch_size = 128
latent_dim = 10
epochs = 10
img_chns = 3
filters = 32
num_conv = (2, 2)
epsilon_std = 1

inputs = Input(shape=input_shape)
conv1 = Conv1D(16, 3, 2, padding='same', activation = 'relu', data_format = 'channels_last')(inputs)
conv2 = Conv1D(32, 2, 2, padding='same', activation = 'relu', data_format = 'channels_last')(conv1)
conv3 = Conv1D(64, 2, 2, padding='same', activation = 'relu', data_format = 'channels_last')(conv2)
flat = Flatten()(conv3)
hidden = Dense(intermediate_dim, activation='relu')(flat)
z_mean = Dense(latent_dim, name = 'z_mean')(hidden)
z_log_var = Dense(latent_dim, name = 'z_log_var')(hidden)

def sampling(args):
    z_mean, z_log_var = args
    epsilon = K.random_normal(shape=(K.shape(z_mean)[0], latent_dim),
                              mean=0., stddev=epsilon_std)
    return z_mean + K.exp(z_log_var) * epsilon

z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_var])

decoder_hid = Dense(intermediate_dim, activation='relu')(z)
decoder_upsample = Dense(38400, activation='relu')(decoder_hid)
decoder_reshape = Reshape((600,64))(decoder_upsample)

deconv1 = Conv1D(filters=32, kernel_size=2, strides=1,
             activation="relu", padding='same', name='conv-decode1')(decoder_reshape)
upsample1 = UpSampling1D(size=2, name='upsampling1')(deconv1)
deconv2 = Conv1D(filters=16, kernel_size=2, strides=1,
             activation="relu", padding='same', name='conv-decode2')(upsample1)
upsample2 = UpSampling1D(size=2, name='upsampling2')(deconv2)
deconv3 = Conv1D(filters=8
                 , kernel_size=2, strides=1,
             activation="relu", padding='same', name='conv-decode3')(upsample2)
upsample3 = UpSampling1D(size=2, name='upsampling3')(deconv3)
x_decoded_mean_squash = Conv1D(filters=4
                 , kernel_size=4, strides=1,
             activation="relu", padding='same', name='conv-decode4')(upsample3)

class CustomVariationalLayer(Layer):
    def __init__(self, **kwargs):
        self.is_placeholder = True
        super(CustomVariationalLayer, self).__init__(**kwargs)

    def vae_loss(self, x, x_decoded_mean_squash):
        x = K.flatten(x)
        x_decoded_mean_squash = K.flatten(x_decoded_mean_squash)
        xent_loss = mse(x, x_decoded_mean_squash)
        kl_loss = - 0.5 * K.mean(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)
        return K.mean(xent_loss + kl_loss)

    def call(self, inputs):
        x = inputs[0]
        x_decoded_mean_squash = inputs[1]
        loss = self.vae_loss(x, x_decoded_mean_squash)
        self.add_loss(loss, inputs=inputs)
        return x

outputs = CustomVariationalLayer()([inputs, x_decoded_mean_squash])

# entire model
vae = Model(inputs, outputs)
vae.compile(optimizer='adadelta', loss=None)
vae.summary()

I wanted to ask whether it is possible for the network to nearly perfectly reconstruct the test timeseries when passed through the entire VAE network, but still output junk when using a random Normal input. For further details, here is one of the inputs and outputs when passing a test signal through the network.

Here is a reconstruction generated purely from a random sample.

How can this be? Even if there was a posterior collapse, the VAE should still be able to generate a good output sample with a random input. To further test this I decided to split the network into two parts (encoder and decoder), and then pass the test image through it. The encoder and decoder networks were made by simply splitting the trained VAE network as follows:
idx = 9
input_shape = vae.layers[idx].get_input_shape_at(0)

layer_input = Input(shape=(input_shape[1],))

x = layer_input
for layer in vae.layers[idx:-1]:
    x = layer(x)

decoder = Model(layer_input, x)
decoder.summary()

idx = 0
input_shape = vae.layers[idx].get_input_shape_at(0)

layer_input = Input(shape=input_shape)

x = layer_input
for layer in vae.layers[idx + 1:7]:
     x = layer(x)

encoder = Model(layer_input, x)
encoder.summary()

Interestingly, I also got junk output here. I'm not sure how it is possible. If the model itself is getting a near perfect reconstruction, surely just passing an image through the encoder, extracting the latent mean, and then passing that latent mean through the decoder should also create a near perfect image?
Is there something I am missing here?


text:
How to shape the weights or nodes during gradient training of neural network? Training with constraints?
Gradient training changes indiscriminately all the weights and nodes of the neural network. But one can imagine the situations when the training should be shaped, e.g.:

One can put constraints on some of the weights. E.g. human brain contains regions whose inner connections are more dense that external connections with different regions. One can try to mimic this region-shaped structure in neural networks as well and hence one can require that inter-regional weights (in opposit to intra-regional weights) are close to zero (except, possibly, for some channels among regions);
One can put constraints on some of the weights in such manner that some layer of neurons have specific structure. E.g. consider the popular encoder-decoder architecture of neural machine translation e.g. https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html We can see that that the whole output of the encoder is expressed as a single layer of neurons which is forwarded to the input of the decoder. So - one can require that the set of all the possible outputs of the encoder (e.g. the possible values of this single layer of the neurons) forms some kind of structure, e.g. some grammar of some interlingua. This example is for illustration only, I have in mind more complex neural network which has one layer of neurons which indeed should output the encoded words of some interlingua. So, one is required to guid all the weights of the encoder in such manner that this single layer has only allowable values.

So - my question is - are there methods that guide the gradient descent training with additional information about the weights or about the nodes (i.e. about the whole subsets of weights that have some impact on specific layer of nodes)? E.g. about methods that impress the region structure on the neural network or that constrains the values of some nodes to be in specific range only?
Of course, it is quite easy to include such constraints in evolutionary neural networks - one can simply reject the neural networks with weights that violates the constraints. But is it possible to do this in gradient-like training?

I have a somewhat similar problem, there's this paper that I was supposed to extend (I can explain how): https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1984-2
What they do is that they have a connection network between input and second layer (giving some biological context to training) I don't know how you can maintain the zero edges during back propagation though.


text:
Tips for keeping the distribution of weights normal
I am working on a project where the Neural Network weights must be quantized on 8 or 16 bits for an embedded platform, thus I will lose some precision.
Since our platform does not have floating point arithmetic we need to quantize the weights. By quantizing i mean taking the max absolute value of the weights and divide it by the maximum signed number representable on 8 or 16 bits. this operation will give us a quantization factor $(qf)$.
the final quantized weights will be integer$(value * qf)$.
If my weights are very sparse and have a very bad distribution, I lose more precision.
For example, to the left here is the distribution of weights for one layer, and to the right is the distribution of weights after I added to the loss function the Kurtosis and skew measures of the weights, and it improved a bit the shape of the distribution while keeping the same accuracy, even a bit higher.
Does anybody have any other suggestions? Has anyone tackled this problem before?

Not sure if I understand your question correctly, but could you take the log of the values to compress the range?

I edited my description. Not sure if it would help me because I work with integer arithmetic. And another downside is that the log is an expensive operation that would have to be applied also to the input...I think

@DuttaA

For a convolutional filter, we find the absolute max value of that tensor, name it Vmax, then if we want to have 8 bit precision, we divide 127 / Vmax, or if we want 16 bit precision we divide 32767 / Vmax both of these values will be named the quantization factor (qf). The resulting quantized tensor will have each value computed as v * qf.
The weights are initialized by Pytorch which is either He or Xavier (can't remember exactly)
I am trying to achieve a minimum quality loss of the quantized version vs floating point version

@DuttaA
sorry for my bad explanation. I was referring to the newly quantized weight tensor of the convolutional filter. V is just a single weight in that tensor. I should have named it as w. Sometimes I look over badly written code and write the same bad naming in forums

Then work with quantized network it's good idea to normalize block of them them with  floating point scaling factor (16 bit float for example).
The weight channel and input channel will have represenation S_f * w_n where S_f - scalar float and w_n - low-bit fixed or integer tensor or vector. Then calculating dot product put it into high-bit fixed/integer variable or float and recalculate scaling factor and low-bit representation for output. That is only if you can implement floating point (or high bit fixed) operations of cause. For low-bit representation you should clamp values to S_f*[-2^n:2^n] and that could be non-trivial. Good idea to use statistics of weights/inputs for it.
There are dozen of papers on quantization of neural networks with different approaches, you may want to read some.


text:
How does DeepMind perform reinforcement learning on a TPU?
I've watched this video of the recent contest of AlphaStar Vs Pro players of StarCraft2, and during the discussion David Silver of DeepMind said that they train AlphaStar on TPUs.
My question is, how is it possible to utilise a GPU or TPU for reinforcement learning when the agent would need to interact with an environment, in this case is the StarCraft game engine?
At the moment with my training of a RL agent I need to run it on my CPU, but obviously I'd love to utilise the GPU to speed it up. Does anyone know how they did it?
Here's the part where they talk about it, if anyone is interested:
https://www.youtube.com/watch?v=cUTMhmVh1qs&t=7030s

My first guess would be that they run the Starcraft engine on that TPU.

As I understand it, a TPU isn't a self-contained computer to be able to load an environment like StarCraft onto it? Maybe you can, I don't know. I thought it was essentially like a GPU architecture but with an higher number of cores.  I just can't think how you could put a game environment on it.

I do know you can implement a full-blown raytracer in GPU code (specifically, CUDA). As for utilizing parallelization, Blender renders each pixel in a tile at the same time, and DeepMind should be able to use the same trick - each core running one instance of the game - if they use neural evolution rather than gradient descent - and I have no idea how you would gradient-descend a Starcraft AI. Just guesswork on my side though, I don't know how TPUs actually work.

TPUs are hardware accelerators. There is no way to "run" a game on an accelerator, which as @BigBadMe pointed out, is not a computer. The game would need to be heavily customised to make use of TPUs as a form of accelerator - this is most certainly not the case.

Starcraft is not differentiable, so you can't learn to play using gradient descent directly, which is why they use reinforcement learning.

In their blog post, they link to (among many other papers) their IMPALA paper. Now, the blog post only links to that paper with text implying that they're using the "off-policy actor-critic reinforcement learning" described in that paper, but one of the major points of the IMPALA paper is actually an efficient, large-scale, distributed RL setup.
So, until we get more details (for example in their paper that's currently under review), our best guess would be that they're also using a similar kind of distributed RL setup as described in the IMPALA paper. As depicted in Figures 1 and 2, they decouple actors (machines running code to generate experience, e.g. by playing StarCraft) and learners (machines running code to learn/train/update weights of neural network(s)).
I would assume that their TPUs are definitely being used by the Learner (or, likely, multiple Learners). StarCraft 2 itself won't benefit from running on TPUs (and probably would be impossible to even get to run on them in the first place), because the game logic likely doesn't depend on large-scale, dense matrix operations (the kinds of operations that TPUs are optimized for). So, the StarCraft 2 game itself (which only needs to run for the "actors", not for the "learners") is almost certainly running on CPUs.
The actors will still have to run forwards passes through Neural Networks in order to select actions. I would assume that their Actors are still equipped with either GPUs or TPUs to do this more quickly than a CPU would be capable of, but the more expensive backwards passes are not necessary here; only the Learners need to perform those.

I've had a read of the blog post, and it reminded me that they employed supervised learning to create their initial agent, which was based on thousands of human games. So I guess because they have the raw game data already to hand they can train the model on a TPU, and no interaction with an environment would be necessary.

@BigBadMe Sure, but that's only for the start of their training. They still ran hundreds of years worth of actual RL experience afterwards :)

Yes true. I suspect they had hundreds of virtual environments actually running the games which then hooked into the TPUs to present the game state and get the actions.  I'm looking forward to the paper being released to get the full scoop.


text:
predict customer visit
Suppose we have a data set consists of columns

TransactionId, CardNo, TransactionDate

then how can we calculate the customer purchase interval (means if customer A purchased on Jan 1st and after 10 days he again purchased, and then he again purchased after 15 days.) and how to predict the next visit of customer A by analyzing the purchasing intervals of customer A.
Any help will be appreciated.

What interests us in this problem are only the intervals for 1 person. Lets say that we want to train a neural network on recognizing the simple pattern in date differences. This would mean that we could train the neural network on series of purchase histories of multiple people. That means that one possible input is the previous intervals: in your case the previous 2 intervals (10 days, 15 days). That is a very small number and it will be hard to recognise a pattern for 3 consecutive purchases, but lets disregard that. After all, you can always experiment with different input sizes.
For training, you can take multiple customers, where of all of them you know when 4 purchases took place. You calculate the intervals, thus for each customer you get 3 intervals. As input you give the network 2, you let the network predict the 3rd one, and then you let the network learn off its mistake (you can do that because you know what the 3rd one was in reality, that's how you train the network).
Provided that the network isn't too complex or too simple (by that I mean that a hidden layer could be necessary, and you need to give it sufficient neurons) than the network might give a decent approximation. The only problem is, as Manuel Rodriguez said, that the network would indeed only learn to make predictions based on the corectness of its previous ones on a large group of people. Perfect predictions thus won't be possible, but you can minimize the error by making your dataset and input larger, so the network has more information to work on.

An 'AI'* is only as smart as the information you give it
You've got to add your own knowledge of the situation into this. Currently we have a transaction id which only really tells us that there is a transaction, a card number (identifying a user, I assume) and a date.
The date can probably tell you most - what day of the week was it? What season (most sales experience some seasonality)? What time of day?
Comparing several dates can then tell you things like the average gap - deviation on that average.
You can use machine learning models to tell you how good these variables are at predicting the next visit day but you have to create these variables first, the model won't know about seasonality or its effect on sales of ice cream, umbrellas or winter jumpers so you have to use your knowledge of your customer base to pass the right variables to the model.
You might also want to consider the product purchased - if you can see that information - someone who buys a pint of milk or a loaf of bread will probably return for the same goods on a weekly basis (or whenever they run out) but someone who bought a set of screw drivers and returned for a hammer a week later is unlikely to return for the same goods.
The vast majority of work done for most predictive systems is in creating your variables and providing something to train on which will hold the pertinent information.
*I'm assuming here that you're working with a machine learning model


text:
Which Openai Gym environment should I use to test a Temporal Difference RL Agent?
Up to now, I have been using (my version of) open AI's code, with the suggested CartPole.
I have been using Monte Carlo methods, which, for cartpole, seemed to work fine.
Trying to move to temporal difference, Cartpole seems to fail to learn (with simple TD method) (or I stopped it too soon, but still unacceptable).
I assume that is the case because in Cartpole, for every timestamp, we get a reward of 1, which has very little immediate information about weather or not the action was good or not.
Which gym environment is the simplest that would probably work with TD learning?
by simplest I mean that there is no need for a large NN to solve it. No conv nets, no RNNS. just a few small layers of a fully connected NN, just like in cartpole. something I can train on my home cpu, just to see it starting to converge.

Cartpole will work just fine with a single-step TD method - e.g. Q learning - and a simple neural network. Configured well, a simple DQN model should solve it in a few minutes. You can also forgo the neural network and use discretised states, or tile coding etc.
What you cannot do is plug a neural network estimator into basic Q learning and make no other adjustments. That is because the bootstrap process in TD learning will create a runaway feedback driven from the random and incorrect return estimates in the initialised NN. You have to use something like DQN to get it to work at all. This is different to Monte Carlo approach, where you can generally just plug in a NN in place of a Q table.
The basics of DQN are:

Use experience replay. Store $(S, A, R, S')$ values from each step, and when training the NN, draw a small random batch (e.g. maybe 16 samples) from this table, re-estimate their Q values to calculate TD target e.g.  $R + \gamma\text{max}_{a'} Q(S',a')$ and use that to create a minibatch to train the NN
Use a "delayed" target network for estimating TD target. This can just be a snapshot of your learning network, taken once every N steps, where N is typically anything from 100 to 100000.

Further details of what configured well means varies from problem to problem. With DQN you will want to play with hyper parameters of experience history size, time to keep frozen copy of network (for bootstrap estimates), batch size of experience table samples, amount of exploration.
In CartPole though, there will be a wide range of acceptable hyperparameter values. So I suspect you have just plugged in a NN in place of the Q table and wonder why it doesn't work.

I assume that is the case because in Cartpole, for every timestamp, we get a reward of 1, which has very little immediate information about weather or not the action was good or not.

Yes this is part of what makes some learning goals become harder challenges in sequential-decision control systems. However, this was also the case for your Monte Carlo method, and in general this relates to the credit assignment problem in RL.

Thanks :) I will go on to implementing what you suggested

Also, why does having the "delayed" target network help stability? after all, we take an action, and receive a reward. Why does it matter if that action and reward came from a (temporarily) constant distribution or not?

Also by the way, When drawing an experience whose s is terminal (meaning no s'), should I just plug in 0 (zero) in place of Q(s',a')?

@Guizar: Yes, plug in a zero, the value of $Q(s_T, \cdot)$ is 0 by definition when $s_T$ is a terminal state, because Q is predicting future rewards and there are none. The delayed network is a separate question, too much detail for a comment - I think I have answered a similar one here on the site, sorry cannot find it at the moment


text:
How sigmoid funtion helps us in reducing error in neural networks?
As in sigmoid function when input x is very large or very small the curve is flat that means low gradient descent but when it is in between the slope is more so,
My question is how this thing helps us in neural network.

There are several nice things about using the logistic function and a few drawbacks as well.  I'm not going to discuss all of them, but here is a small synopsis.
Nice Properties

For networks which calculate probabilities, it brings the input features into a nice 0 to 1 range.
Since the logistic function maps inputs to the range (0,1), it is commonly used as an activation function in neural networks which require some sort of classification.  For example, if we had a convolution neural network (CNN) that classified if an image given to it was a dog or not, it could have an output neuron that returned 0 if the image was not a dog or 1 if it was.  When the CNN is given an image with inputs of varying dimensions and ranges, it's more intuitive to deal with the input features in the realm of 0 to 1 instead of 0 to 255 or whatever input format you're dealing with.
It is smooth and differentiable.
When performing back propagation, it's much easier to deal with differential functions.  Otherwise you'd have to approximate the gradient which may lead to unfavorable results.
A Vanishing Issue
At extreme values, the gradient vanishes.
The OP was also hinting that at very positive and very negative values of X, derivative of the logistic function could become close to 0.  For large input values this may hinder learning for extreme values.  If you think of gradient decent, the jump that gradient descent makes is only as strong as its magnitude of the gradient that it requires.  Analytically speaking, this also hurts back propagation because learning is calculated using the chain rule, which ends up multiplying several gradients together.


text:
How to create meaningful multiple object detection evaluation comparison graph?
I have got multi-class object detector. One model accuracy evaluation of detection consists of: mAP, FP, FN, TP for each class divided to two graphs and looks like this (I've used this repo for evaluation).

Now, I've got many of these evaluations (multiple times these two graphs for different models) and I would like to easily compare all these trained models (results) and put them to one graph.
I've searched through the whole Internet, but wasn't able to find suitable method of placing all the values to one graph. Also, the values of these three classes can be put together (eg. result mAP for this evaluation would be (75+ 68+ 66) / 3 = ~70%), so I would have just single value of each mAP, FN, FP, TP for one whole model evaluation.
What comes to my mind is the following graph (or maybe some kind of plot):

Note: It may not make sense to place mAP together with TP, etc. into one graph, but I would like to have all these values together to easily compare all the model evaluations. Also I am not really looking for a script, I can do the graph manually from values, but script would be more helpful. What really matters is, how to create meaningful graph with all the data :). If the post is more suitable for different kind of site, please, let me know.

@DuttaA Thanks, I wasn't sure about that one.


text:
Sample from a distribution inside a NN layer
Is it possible to sample from a distribution inside a neural network forward function? Assume that there is a NN and a sample is needed to be derived from it at every forward-pass to randomly set a layer-specific hyper-parameter.
Is this operation differentiable


text:
Input for the Env.step() in the 'Pendulum-v0' environment
I want to customize the 'Pendulum-v0' environment such that the action (the torque) from previous time step as well as from the current timestep serve as the inputs in the Env.step() function.
My problem statement is that I want to generate torque from the controller which has a white Gaussian noise of magnitude 1 and then filter it with the torque generated in the previous timestep as follows:
tor_ = tor_c + a*WGN ;
tor(t) = lambda*tor_ + (1-lambda)*tor(t-1) ;
https://github.com/openai/gym/blob/master/gym/envs/classic_control/pendulum.py#L37
We can see that the 'u' is an array of some numbers as the input which is afterward clipped from -max_torque to max_torque and then only the first element is taken as the torque value to calculate the states and the reward function for the given time-step.
My question is what does the value of the other elements signify? Are they
torque values from the previous time steps or is it that the length of the 'u' array is just 1 and its value is restricted between -max_torque to max_torque?
In conclusion, I just wanna access the action (the torque value) from the previous time-step. Is it possible ? If yes, how?

there are no other elements, u is an iterable with a single element (action that we want to take) that we send to step function. They use np.clip function to clip the number in given range and the np.clip function returns np.array (with only 1 element). Then they retrieve the float from it with [0].

So how can I access the action from the previous time step for low pass filtering of the torque? Aren't actions from every time steps saved somewhere in the action space ?

You provide actions to the environment from your code, simply save the action from the previous loop iteration in your code and use it for the update.

Also, I wanna model it such that the pendulum is always in -pi/8 to pi/8 (with the vertical) and if it gets beyond that, the environment resets. So I will define the observation space such that there is bound on theta rather than using cosine or sine function as used.in 'Pendulum-v0'. Is it okay do it this way? (Sorry if this is a noob question)

I don't think they actually have termination condition, environment just keeps on going, so the only current termination condition would be to limit the number of timesteps. Regarding limiting the angle for reset condition, I think that would be fine, except that in this case pendulums starting position is at random angle so with your condition environment could immediately reset, assuming that you meant pi/8 offset from pi/2 angle ?

In my case there is no problem in using theta and theta dot as the observation space for the environment, right? And what do u mean by termination condition? And speaking of starting position, the pendulum will be initialized at random angle between -5 to +5 degrees with the pi/2 angle.

I think using theta and theta dot would be fine, best way to find out is to try out. I think that you might be confusing pendulum-v0 with cartpole-v0 environment. Pendulum starts at bottom and tries to swing up and stay at top, while cartpole starts at top and tries to stay there without falling down.

But I want it the way cartpole is designed to start not the way pendulum is..


text:
What are the aspects that most impact on the inference time for neural networks in embedded systems?
I work with neural networks for real-time image processing on embedded softwares and I tested different architectures (Googlenet, Mobilenet, Resnet, custom networks...) and different hardware solutions (boards, processors, AI accelerators...). I noticed that the performance of the system, in terms of inference time, does not depend only on the processor but also on other factors.
For example, I have two boards from different manifacturers, B1 (with a cheap processor) and B2 (with a better processor), and two neural networks, N1 (very light with regular convolutions and fully connected layers) and N2 (very large, with inception modules and many layers). The inference time for N1 is better on B1, while for N2 it is better on N2. Moreover, it happens that, as the software is executed, the inference time changes over time.
So my question is: in an embedded system, what are the aspects that impact on the inference time, and how? I am interested not only in the hardware features but also in the neural network architecture (convolutional filter size, types of layers and so on).

You can expect that the inference time will strongly depend on particular hardware and software present on your platform. First, GPU equipped devices (eg NVidia TX) will outperform non-GPU equipped devices (eg. Intel Movidius). Second, software support (eg. cudnn, TensorRT) will make dramatic further impact.
For instance, we have measured the inference time of two convolutional models. The model A requires 250% more floating point operations than the model B. Yet, the two models take roughly the same time to evaluate on our device, since the layers of model A are better optimized in software. Conclusion: algorithmic complexity and practical execution time on a particular computing platform are not bound to be proportional any more.


text:
GPFlow: Gaussian Process Uncertainty Quantification
I trained some Gaussian process model with the Python library GPFlow on a dataset consisting of $(X, Y)$, inputs and outputs, in a regression setting. This model gives me pretty good predictions in the sense that the relative error is small almost everywhere. I want to use the uncertainty as well, which is given in a GPFlow setting in the form of a standard deviation (STD) associated with every prediction. Here's my problem: I normalised both inputs and outputs before training (separately) using sklearn's StandardScaler (effectively making the data normally distributed with $0$ mean and unit STD). So the STD given by the model pertains to the scaled data. How do I "rescale" the uncertainty estimates of the GP to the actual data? Using the inverse_transform function of the output scaler makes little sense. This issue might be easier solvable if I scaled with a MinMaxScaler (squishing all data points into the unit interval) by dividing by the length of the range of the original output set (at least I think it works that way). But how about the case of the StandardScaler? Any insights will be appreciated!

I think it works by computing an interval of 1 STD around the predictions and then rescaling both interval boundaries, which should yield the corresponding interval on the original data set, irrespective of the way we scale.

Not sure about which function to call in GPFlow, but the principle is simple. If your $Y$ data was scaled:
$$Y_{scaled} = {(Y - mean(Y)) \over {sd(Y)}}$$
So now if you have any prediction (from the scaled model) $\hat{Y}_{scaled}$, with stand error $\sigma_{\hat {Y}_{scaled}}$, then it is a normal distribution ${\cal N}(\hat {Y}_{scaled}, \sigma^2_{\hat {Y}_{scaled}})$. Then you can very easily rescale it back to the original scale ${\cal N}(\hat {Y}, \sigma^2_{\hat {Y}})$, where:
$$\hat {Y} = \hat {Y}_{scaled}\cdot sd(Y) + mean(Y)$$
$$\sigma_{\hat {Y}} = \sigma_{\hat {Y}_{scaled}} \cdot sd(Y)$$


text:
Bubble Chamber Image Analysis Using Neural Network
I have a data analysis problem that I can reduce to one similar to analyzing the trajectories in the images below. These images show the tracks of subatomic particles interacting in a bubble chamber.
It's pretty obvious that by eye, easily discernible patterns can be seen. I want very much to know more about how classification and segmentation can be done using neural networks for this type of image.
These images are binary. The trajectory is either at a point in the image or it isn't. As can be seen, trajectories cross over one another, Some data appears to be missing in otherwise smooth curves, at arbitrary points along those curves. (My data may be more sparse in this respect.)
A typical paper on bubble chamber analysis that I would find deals with the analysis of the physics after trajectories have been classified and segmented.
Can anyone identify some papers that address this or something similar in the context of neural networks? I am not able to find anything recent on automated methods at all, but my google fu may not be up to the challenge. (By the way, I am less interested in some of the parametric methods like Hough Transforms. I'd like to focus on the neural approach.)
(I posted this previous question which wasn't quite as specific as this one. I hope there is some available research in this area related to physics that might give me some insights that are more directly related to my problem.)


text:
Is it a great misconception that the softmax is an activation function?
An activation function is a function from $R \rightarrow R$. It takes as input the inner products of weights and activations in the previous layer. It outputs the activation.
A softmax however, is a function that takes input from $R^p$, where $p$ is the number of possible outcomes that need to be classified. Therefore, strictly speaking, it cannot be an activation function.
Yet everywhere on the net it says the softmax is an activation function. Am I wrong or are they?

An activation function isn't necessarily defined as $\mathbb{R} \rightarrow \mathbb{R}$.

I would be interested to know where you are taking your definition of an activation function from in your first paragraph? I am not sure if it has a formal definition, and even if it did, this would maybe become an issue of mis-naming or mis-classifying something, but the engineering of it - what it is and does - would remain exactly the same.

Coming to think of it, I did not see a formal definition. Instead, I saw pictures such as those link, where it seems that they are functions that take from & $\mathbb{R}$.

I see no problem in regarding the softmax as a particular activation function which takes a vector input and produces a vector output. In fact, the sigmoid function can be viewed as a two-dimensional softmax in which one of the two inputs is hardwired to zero while the corresponding output is neglected.

Thanks for your answer. Indeed, there is no problem with regarding it as such. The question was mostly aimed at clarification.


text:
Feature visualization on neural networks which are not for classification
Feature visualization allows to better understand neural networks by generating images that maximize the activation of a specific neuron, and therefore understand what are the abstract features that produce a high activation.
The examples that I saw so far are related to classification tasks. So my question is: can these concepts be applied to other convolutional neural network tasks, like semantic segmentation or image embedding (triplet loss)? What can I expect if I apply visualization algorithms to these networks?


text:
How to make episode ending "good" in reinforcement learning?
TL;DR: read the bold. The rest are details
I am trying to implement Reinforcement Learning:An Introduction, section 13.5 myself:

on OpenAi's cartpole
The algorithm seems to be learning something useful (and not random), as shown in these graphs (different zoom on the same run):

Which show the reward per episode (y axis is the "time alive", x axis is episode number).
However, as can be seen,

The learning does not seem to stabilize.
It looks like every time the reward maxes out (200), it immediately drops.

My relevant code for reference (inspired by pytorch's actor critic)
note: in this question, xp_batch is ONLY THE VERY LAST (s, a, r, s'), meaning experience replay is not in use in this code!
The actor and critic are both distinct neural networks which
def learn(self, xp_batch):#in this question, xp_batch is ONLY THE VERY LAST (s, a, r, s')
    for s_t, a_t, r_t, s_t1 in xp_batch:
        expected_reward_from_t = self.critic_nn(s_t)
        probs_t = self.actor_nn(s_t)
        expected_reward_from_t1 = torch.tensor([[0]], dtype=torch.float)
        if s_t1 is not None:  # s_t is not a terminal state, s_t1 exists.
            expected_reward_from_t1 = self.critic_nn(s_t1)

        m = Categorical(probs_t)
        log_prob_t = m.log_prob(a_t)

        delta = r_t + self.args.gamma * expected_reward_from_t1 - expected_reward_from_t

        loss_critic = delta * expected_reward_from_t
        self.critic_optimizer.zero_grad()
        loss_critic.backward(retain_graph=True)
        self.critic_optimizer.step()

        delta.detach()
        loss_actor = delta * log_prob_t
        self.actor_optimizer.zero_grad()
        loss_actor.backward()
        self.actor_optimizer.step()

def select_action(self, state):
    probs = self.actor_nn(state)
    m = Categorical(probs)
    action  = m.sample()
    return action

My questions are:

Am I doing something wrong, or is this to be expected?
I know this can be improved with eligibility traces/experience replay+off policy learning. Before making those upgrades, I want to make sure the current results make sense.

I think that questions related to programming issues are better suited for https://datascience.stackexchange.com/ or Stack Overflow. Anyway, I've you tried to run the PyTorch example (which you based your implementation on) and see if it behaves similarly to yours?

The example I linked to uses full episodes discounted returns. Mine is (supposed to be) using immediate rewards, online, regardless of episodes. just the very last sample.
So, they aren't supposed to give similar results, which is why I come to the experts and ask if this makes any sense

@nbro If this is better in datascience.stackexchange.com, I don't mind it being moved, should I ask for it via flagging? Anyway, this question is more about the results (in the graphs) than the code. The code is there for reference.

You could simply copy the text of this question to another question on Data Science SE and then (if you want) delete the question from here. Hopefully, there you will find more help, but I can't guarantee that.

Anyway, I feel like you're attempting to implement some non-trivial algorithms and trying to combine different techniques without really knowing the details of these algorithms and techniques or fully understanding them. This will make your life harder. I would encourage you to first implement what is really known to work. Then try to formalise what you think might work. Only then you should attempt to implement it.


text:
Logic questions: reasoning pattern, Infer literals, unit resolution, and-elimination etc

Show which literals can be inferred from the following knowledge bases, using both reasoning patterns and truth tables. Show all steps in your reasoning and explain your answers.
1) P & Q
  2) Q →R v S
  3) P → ~R

This is from my reasoning pattern tutorial, my text book shows similar question except that there is a single literal with workings, so I can somehow read through but I'm not familiar with some terms. I don't understand how I can infer all literals with the above information. I also don't fully understand what is and-elimination, modus ponens and unit resolution.
Is there anyone who is kind enough to use the above question as an example so that I can have a clearer picture?

I would like to note that the problem is not to infer all literals, but to show which literals can be inferred.


text:
Azure ML studio pull directly from sharepoint
I am toying around with creating a probability of win calculator for proposals that we do. the information on each proposal is housed in our corporate SharePoint (which I am the admin)
Is there a way to pull directly from SharePoint as the data source rather than have to export to xls then upload each time the data updates?


text:
Is fuzzy logic connected to neural networks?
Fuzzy logic is typically used in control theory and engineering applications, but is it connected fundamentally to classification systems?
Once I have a trained neural network (multiple inputs, one output), I have a nonlinear function that will turn a set of inputs  into a number that will estimate how close my set of given inputs are to the trained set.
Since my output number characterizes "closeness" to the training set as a continuous number, isn't this kind of inherently some sort of fuzzy classifier?
Is there a deep connection here in the logic, or am I missing something?

They are unrelated.
There is a possibility of interpreting fuzzy values as probabilities, but strictly speaking they are different: fuzzy values are vague, while probabilities reflect likelihood (see Wikipedia entry for Fuzzy Logic)
While rolling a particular number on a six-side die has a probability of $1 \over 6$, a roll can actually only ever have one outcome.
A fuzzy value "quite old" can simultaneously be member of a number of fuzzy sets with different degrees of membership, eg "young" with 0.001, "adolescent" with 0.1, "old" with 0.4, "ancient" with 0.7. Unless it is "defuzzified", it is simultaneously contained in all the sets.
Defuzzyfication is a way of interpreting the result of a series of fuzzy operations and finding the set that best matches, but it is not a clearly defined process such as picking a random number according to a set of probabilities (or rolling the die).
I am not sure that the sum of all fuzzy set membership values of any given fuzzy value has to add up to 1.0; whereas this condition has to hold for probabilities.
[EDIT: to clarify - probabilities are not a set; I refer here to all possible outcomes of a random event which have a certain probability of being realised. The sum of all possible event probabilities has to be 1.0]
One alternative interpretation for your application could be the confidence that the input set is identical to the training set. Which could be a fuzzy value if you wanted to do something else with it, eg by combining it with other fuzzy variables.

Are the outputs of neural nets explicit probabilities? Could you maybe link me to something that makes clear? Also, intuitively if you trained a neural network on pictures of bald men, somehow it's hard for me to believe that the "baldness" output is actually a probability of being bald. (Relative frequency makes some sense though)

It's not a probability, it's a weight score. Basically, you put in some feature values, theses are combined, weighted, and filtered through the activation function, and then come out at the output nodes. So they are neither probabilities nor fuzzy values.

Okay but now I'm a bit lost as to what your argument is.  You say that fuzzy logic isnt connected to neural nets. But your reasoning is that it's because fuzzy logic isn't the same as probability. But you give no reason as to why or how probabilities are connected to neural networks. So what your point in talking about probabilities in the first place?

Right, I see; it's not very clearly formulated. Neither fuzzy values nor probability are related to the output values of neural networks. For some reason I must've misinterpreted/misread your original question.


text:
Why is the learning rate is already very small (1e-05) while the model convergences too fast?
I am training a video prediction model.
According to the loss plots, the model convergences very fast while the final loss is not small enough and the generation is not good.
Actually, I have test the lr=1e-04and lr=1e-05, the loss plots drop down a little more slowly, but it's still not ideal. But I think lr=1e-05should be small enough, isn't it?
How should I fix my model or the hyper parameters?


text:
Judging a genetic algorithm's priority-based schedules by how far ahead the higher priority things are done
I'm creating a schedule for a summer camp. Because of the high risk of rain, the higher priority activities need to be attempted first, so there is more time for later attempts if need be (temporarily ignoring the schedule in that situation).
Camp takes place over four days. My current idea is to map the days to a set of numbers (4, 3, 2, 1), and get a correlation between these numbers and the priorities of activities. But I'm not certain this is the best way to do it, nor what the best way of correlating the two are. I'm also not sure how I would factor this correlation in with the fitness function, along with the priorities themselves.
How should I proceed?

Can I ask why a simple sorting of activities by priority wouldn't suffice?


text:
Problem with Proposition 1 of Google Deepmind's 'Weight uncertainty in Neural Networks'
I'm going through the paper Weight Uncertainty in Neural Networks by Google Deepmind. In the final line of the proof of proposition 1, the integral and the derivative are swapped. Then the derivative is taken. But this somehow yields 2 derivatives of $f$ with respect to $\theta$. I thought that this was the result of a product rule applied to $q(\epsilon)$ and $f(w,\theta)$ and then the chain rule. But that does not yield the same outcome as $\frac{\partial q(\epsilon)}{\partial \theta} = 0 $. My question is: does anyone understand how the equation in the last line comes about?

we start with
$\frac{\partial}{\partial \theta} \mathbb E_{q(\mathbf w\mid\theta)}[f(\mathbf w, \theta)]$
using definition of expectation for continuous case:
$\mathbb E[X] = \int xp(x) dx$
for the first equation we get:
$\frac{\partial}{\partial \theta} \int f(\mathbf w, \theta)q(\mathbf w \mid \theta) d\mathbf w$
we swap $q(\mathbf w \mid \theta) d\mathbf w$ for $q(\epsilon)d\epsilon$ and we get:
$\frac{\partial}{\partial \theta} \int f(\mathbf w, \theta)q(\epsilon)d\epsilon$
using definition of expectation again we have:
$\frac{\partial}{\partial \theta} \mathbb E_{q(\epsilon)}[f(\mathbf w,\theta)]$
i believe dominant convergence theorem let's us interchange expectation and derivative and we have:
$\mathbb E_{q(\epsilon)}[\frac{\partial}{\partial \theta}f(\mathbf w,\theta)]$
we have a function $f$ that depends on two variables $\mathbf w$ and $\theta$. In order to get a full derivative of such function we need to take derivative of $f$ with respect to both variables. We have:
$\mathbb E_{q(\epsilon)}[\frac{\partial f(\mathbf w,\theta)}{\partial \theta} + \frac{\partial f(\mathbf w,\theta)}{\partial \mathbf w} \frac{\partial \mathbf w}{\partial \theta}]$
First term inside expectation is derivative of function $f$ with respect to $\theta$ directly, and in the second term $f$ is derivated with respect to $\mathbf w$, but we need to apply chain rule to get full derivative with respect to $\theta$

Thank you for your answer. This solves it. I stared at this for a while but did not take into account that $\textbf{w}$ is a function of $\theta$.


text:
Presence of object (highly occluded vehicle) in a scene
How to detect presence of object (highly occluded) in a scene?
There are specific features (small patterns, etc), which allow to say that object is present, but it is not enough for detection for YOLO or RPCNN.
How to detect small specific pattern in a whole image efficiently?

Do you have the entire image which is noisy?

YOLO is still close to state-of-the-art for object detection, and there are plenty of hyperparameters to try, so it would be anyone's guess without more details of why it is failing for you. It may help if you show an image or two, with some details of what is detected (does it even have some small confidence of any object centred in the right grid location?), and what hyperparameters you have tried.

One potential way you could go about doing this is a technique called masked projection, which has previously been used for 3D Face Recognition purposes where occlusions covering the facial surface were found to be an issue. You might find this paper of use: https://www.researchgate.net/publication/236848193_3-D_Face_Recognition_Under_Occlusion_Using_Masked_Projection


text:
How to make chatbot using NLP like Dialogflow?
I want to apply the concept that exists in the Dialogflow API in my e-commerce website.
I get some references in this regard :

Tokenization
Part Of Speech
Named Entity Recognition
Rule based

I just saw that I just didn't understand how to implement it on the website.
so I still don't know how the truth is.
please give me a method or explanation that can help me create a chatbot for ecommerce that can give action when a user asks for a product and wants to place an order or something else.
Please give me some explanation or method or references :(

You are right that you will have to implement the above steps before building any chat bot, this is because your computer doesn't understand text like you and I do. Therefore, text has to be pre-processed(converted) into a format that the computer can work with Tokenization, Parts of Speech Tagging and Named Entity Recognition are such pre-processing techniques.
Rule based means when you create rules for deciding how should your chat bot behave in a given situation.
You can have a look at this blog which teaches how to deploy a chat bot using DialogFlow on Slack - https://www.analyticsvidhya.com/blog/2018/03/how-to-build-an-intelligent-chatbot-for-slack-using-dialogflow-api/


text:
New face generator from Nvidia
I have just found the paper and documentation about GAN 2.0, the new face creator from Nvidia.
On the website https://thispersondoesnotexist.com/ they have used this approach to create realistic faces. Unfortunately, the website does not exist anymore.
Is there another webpage demonstrating the new face creator from Nvidia?

You can find a decent sized collection on archive.org. Just browse through the snapshots and they'll contain a few images. It probably doesn't contain every single one, but it has quite a decent set to start from. :)
Here's one I found:

There is a Youtube video: Nvidia - AI for Generating High-Resolution Images - Tero Karras
I don't know about webpages.


text:
Modelling gut-feeling/subconscious knowledge of stock market traders
Some (stock market) traders have the ability to produce a high percentage of winning trades (80%+, positive return) over years. I had the chance to look into real money trades of two such traders and I also got trading instructions from them for research.
Now the interesting part is that if you strictly follow their rules then you usually end up with more losers than winners on the long run. But after a while you get some kind of subconscious "feeling" for winners which also shows in the results. I assume that this "feeling" is a hidden function which can be modeled.
My question is: Is there work about how to model such "gut feeling" and subconscious knowledge by means of machine learning (especially with little training data sets)? Is there relevant literature about this topic?
Regards,

You could perhaps model gut feeling or subconscious bias as a prior in a Bayesian context, and then try to learn from the data how much to modify/moderate the bias in each individual case.
I think there is another issue with the problem you outlined.  We might expect it to be normal to see more losers than winners in the long run: trading is a zero-sum game where the house always takes its cut. The trick to being a successful trading algorithm seems to be to make the losers small (cut them early) and the winners big (let them run).

Thanks for your answer. The trading rules take care of the amount to lose/win (planned winners are always a multiple of the losers). For most traders I agree that money management is the main key to being profitable. But the interesting thing here is that the two traders have exact entry and exit rules based on price levels (in combination with order flow) which have a less than 50% chance of being correct when naively applied. But some alleged hidden function tells them when to take the trade and when it's better to stay out.

More interestingly is, that I can also profitable trade with their rules after some training (with increasing success) by picking the "good" occurrences of all signals.

How do you know which are the "good" occurences?  If this is subconscious, you could try to bring it to consciousness through a psychological technique, like free-association.  Alternatively, you could backtest your 'good'/'bad' calls - work through historical data, classifying each call from the algorithm as 'good' or 'bad' based on your intuition. When you have scored every call for a couple of years or whatever, you can make an objective assessment on how good your intuition is, and build a predictive model that uses the data on price, volume or whatever to predict your intuitive calls.

It sounds like a supervised learning from not-so-many samples. Representation/metric learning could be of some help.
There are two books by Bob Volman that help quite a bit to make subconscious conscious. "Understanding Price Action" and "Forex Price Action Scalping". Despite sales-driven titles, this is the best non-scam attempt to formalize discretionary trading I ever saw. (Even though, one will become a one-trick pony at most, but on the other hand it's enough). If you google a bit, you'll find a dropbox with countless price charts commented by author.


text:
Having trouble understanding some of the details of R-CNN (first one)
Here is what I understand (what I think I understand).
We first train out model on our images using transfer learning.
So now we have a pre-trained model.
For each image in out dataset, we compute selective search on it, which makes 2000 region proposals.
    These 2000 region proposals are feed through our pre-trained NN ,
However we only collect the output (feature maps) from the last convolution layer. These outputs are saved to a hard-disk.
These feature maps are fed into a SVM for another round of training, were another label, "no object" is added.
We also have regression model that trains based on the window coordinates that we also annotated.
So we have SVN and a regression model (two models) that we train.
1)Is the above correct?
2) Are each of these 2000 region proposals hand-labeled (correct label (cat, dog etc) or no-object) before feeding it into the SVM?
3) Is the regression model tied into the SVM model? Basically out loss is a combination of both regression coords and SVM classification?

Yes, your description is mostly aligned with the original paper. However, Figure 1 shows they use features from the second fully connected layer (not the last convolutional layer).
Yes. The training is performed by exploiting hand-annotated bounding boxes. If a proposal has IoU less than 0.3, then it is considered as a negative example (cf. section 2.3).
No. Appendix C explains that box regression parameters are learned separately from the SVM model.


text:
How can my Neural Network categorize message strings?
Abstract
I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. This means a human can intervene in its categorization attempts and determine whether or not it was correct and have it adjust its weights accordingly (without having to retrain all over again).
Inputs
It is know that ALL input will follow these rules:

Always of $N$ length
All messages are transformed to eliminate unnecessary complexity

Here's a brief overview of how an example message might be processed.
Starting with a message $M$:

That's an interesting perspective. I think that you should consider adding more details to your point about the cat being too silly.

The text is then transformed so that extra details are removed:

thats an interesting perspective i think that you should consider adding more details to your point about the cat being too silly

Then it's converted into a vector (appending $0$ to reach length $N$) ready to be processed by the neural network:
[116, 104, 97, 116, 115, 32, 97, . . . , 0, 0, 0, 0]
Ouputs
In my network, I wish all the outputs to be weighted on how well they fit in each category. I need multiple outputs. I'm not really focusing on one particular category per-say, but how well the message fits in all of them.
Following the input $M$ I used as an example, I'd expect the outputs to look something like this after my vector has fed-forward:
Suggestive:  0.89042Opinionated: 0.68703
The weight values for each output indicate the strength of the category in the overall message.
From message $M$:

That's an interesting perspective.

Would weigh the opinionated category as $0.68703$.
And:

I think that you should consider adding more details to your point about the cat being too silly.

Would weigh the suggestive category as $0.89042$.
Summary and Questions
I'm interested in the architectural design choices of a network that would support my feature set. The main goal is to be able to train my network to categorize messages based on pre-trained (and live-trained) data. I'd like to know things like:

What type of Neural Network I should use for this purpose? I've researched LSTM & Recurrent networks; which have been mentioned to be good at processing sequences (ie. messages).
What considerations should I account for when creating this network?.
How can the overall network support live-training so I can tell my network when its wrong and have it 'correct' itself without having to retrain completely?

How do you vectorize your inputs?

@JahKnows I've considered converting each character into a float (ie. getting the code point of each) and also converting each word into a float (ie. having a word database with a float value assigned to each). Whichever method works best for my case will be how I vectorize inputs.

have you considered some techniques like bag-of-words, tf-idf and n-grams?

(along with JK) my suggestion is try out LSA 1st to learn the concepts of converting words to vectors, and its similar to NN techniques but actually somewhat conceptually more basic using linear analysis. https://en.wikipedia.org/wiki/Latent_semantic_analysis also there are now several high quality open source off-the-shelf NN systems for NLP (natural language processing) that can be applied without learning all the theory.

@vzn Thanks. The goal for me is to learn everything from scratch. Is there any places you could point me towards that really break down NLP NN concepts?

just googling this ref by Kisler is comprehensive. there are many videos that may be esp helpful to beginners. Introduction to Natural Language Processing (NLP)  https://blog.algorithmia.com/introduction-natural-language-processing-nlp/


text:
How to make machine learning model that reports ambiguity of the input?
Suppose I want to build a neural network regression model that takes one input and return one output.
Here's the training data:
0.1 => 0.1
0.2 => 0.2
0.1 => -0.1

You will see that there are 2 inputs 0.1 that matches to different output values 0.1 and -0.1. So what will happen with most machine learning models is that they will predict the average when 0.1 is fed to the model. E.g. the output of 0.1 will be (0.1 + (-0.1))/2 = 0.
But this 0 as an average answer is an incorrect answer. I want the model to be telling me that the input is ambiguous/insufficient to infer the output. Ideally, the model would report it as a form of confidence.
How do I report predictability confidence from the input?
The application that I find very useful in many areas is that I could then later ask the model to show me inputs that are easy to predict and inputs that are ambiguous. This would make me able to collect the data that are making sense.
One way I know is to train the model then check the error on each training data, if it's high then it probably means that the input is ambiguous. But if you know any other papers or better techniques, I would be appreciated to know that!

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add "probabilistic, bayesian effect" to you neural network.
I think this method was first proposed in Dropout as a Bayesian Approximation:
Representing Model Uncertainty in Deep Learning which is called Monte Carlo Dropout.

Predicting with confidence: the best machine learning idea you never heard of by Scott Locklin might provide you an idea.

The name of this basket of ideas is “conformal prediction.”


text:
Using convnet to classify language of text contained in images
I hope this question is not too broad or general. I have a very large set of images all of which contain text (some have more, some less). All of them have been tagged as containing, say, English text or Korean. I wonder if convolutional neural networks would be a good approach to classify these images as containing English vs. Korean. Or is there any existing literature/method that does this already. Crucially though, I am not interested in "understanding" the text, so this is not an NLP task but, I suppose, a task of classifying orthographies in the images.

This sounds like a fairly straightforward task, with low risks. I think the proper term is that you're you are trying to detect the script, which would be either Latin ( "English") or Hangul ("Korean"). The chance is that you end up learning fonts, though.

The role of the CNN in this would basically just be to grab the text you want.  YOLO is good for that.  Fast and lightweight and easy to train.  It wouldn't help much in actually reading the text though.  A way to do something like this would be to use a neural net for segmentation, training it to draw boxes around text, then you could take the content of those boxes and feed them to an OCR algorithm like Tesseract, and once you have text, then there are NLP tutorials all over the internet on how to do language detection.  Here are a couple to start.
https://www.johnsnowlabs.com/how-to-detect-languages-with-python-a-comprehensive-guide/
https://towardsdatascience.com/how-to-detect-and-translate-languages-for-nlp-project-dfd52af0c3b5
It really doesn't matter that you aren't trying to "understand" the text.  NLP libraries are still where it's at if you are trying to perform a language detection task.  Trying to treat language detection as an image classification task is like trying to infer the speed limit on a highway by reading the heat signatures on the pavement.  Sure, faster cars will transfer more heat, but there is so much entropy involved here that this is never going to be the best option for this task.  Better to extract the thing you want, the words, and use models designed to deal with words.
Basic breakdown for this sort of thing in general would be:

Use a neural network to isolate text from an image.
Use OCR to convert that image text into computer readable text
Feed that text to an NLP library to detect the language

I think that you can use convolutional neural networks (CNNs) to do what you want, but I think that you will need a lot of training images and the task will be very difficult.  You are much better off extracting text from the images using Optical Character Recognition (OCR), partitioning the extracted text into individual words, and then searching if the words are found in an English dictionary or a Korean dictionary (or both?).  This approach isn't perfect, but I bet it will perform better than directly applying CNNs directly to images.


text:
In DQN, updating target network every N steps or slowly update every step is better?
The use of target network is to reduce the chance of value divergence which could happen with off-policy samples trained with semi-gradient objectives. In Deep Q network, semi-gradient TD is used and with experience replay the training could diverge.
Target network is a slow changing network designed to slowly track the main value network. In Mnih 2013, it was designed to match the main network every $N$ steps. There is another way which slowly updates the weight in the direction to match the main network every step. To someone, the latter is called Polyak updates.
I have done some very limited experiments and seen that with the same update rate, e.g. $N=10$, Polyak update would update with the rate of 0.1, I usually see Polyak updates to give smoother progress and converge faster. My experiments are by no means conclusive.
I would thence ask if it is known which one to perform better, converge faster or has smoother progress, in a wider range of tasks and settings?

Most of papers and my experience support hard update, once per N steps. N is usually very bug, range in between 10^4 to 10^6. DQN training is slow. But that depend on the problem. If you DQN converge with soft update with weight ~.1 (N~10) your problem could be very simple.

Would you mind put some more detailed research of yours? I would like to see something like this technique has been used in 2019 by the SOTA or so. Thanks.

I was doing DQN for car driving simulation. The main points: even thogh DQN is off policy it's effectively act similar to on-policy due to overfitting on latest samples(latest measired in millions). That could be helped by increasing target net update intervaland to some degree size of replay buffer. Its very important to randomly remix samples from replay buffer then assembling training minibatch. Overfitting causing DQN drift, wich is main enemy


text:
Machine Learning papers for matching packets to request flows
I'm currently doing a research project related to Distributed Tracing. My research has led me to a point where I think ML might be suited for our problem.
I'm looking for papers that are similar to this (even if they have other applications):
I want to match packets exiting a black-box system (outputs) to packets that enter a black box (inputs). I can do that easily in a non concurrent setting which should help  me grow a training set (maybe for supervised learning), but I need an algorithm that, in a concurrent setting, can separate the different request "flows" if you will.
I hope this makes sense.
The closest thing to what I'm looking for is "Aguilera, Marcos K., et al. "Performance debugging for distributed systems of black boxes." ACM SIGOPS Operating Systems Review. Vol. 37. No. 5. ACM, 2003." but it's mostly suited for finding the dependency graph of the system, which I already know.
Thank you


text:
How to study the correlation between GAN's input vector and output image
A generative adversarial network (GAN) takes a vector of numbers as input and generates an image, based on the input. Each element of the vector causes some feature of the image to change, but the mapping between input and output is not clear, as often happens in deep learning. What is the best way to study the correlation between the vector elements and the output image features? The first approach that comes to mind is to manually change every element and check the result, however I am not sure that this is the best solution.

Let me first provide a brief introduction first
GANs as VAEs are generative models which means they learn exactly what you described: to map a typically small dimensional vector/tensor into a higher dimensional one, which in your case is (interpreted as) an image.
These 2 generative models differ in the actual learning strategy which results typically in making GAN outperform VAE in realistic image generation (but it’s better not to get into the reasons for this here)
However in principle the Generator Network in GAN works in a similar way to the Decoder Network in VAE so as their output space is interpreted as an image, you can interpret their input space as a sort of semantic space hence the values you are setting define the semantic of the output image (in fact you observe the features change)
It is possible to say the generative model learns a  Semantic to Appearance Mapping hence it works the opposite way of the classifier model which learns an Appearance to Semantic Mapping
Unfortunately as it typically happens in NN this mapping are definitely hard to understand for humans, nevertheless this is an active area of research
So after this introduction, aimed at giving you an essential understanding of the problem, I can suggest you some papers to get into more details like this one
GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
In case you found it not that clear, I’m planning to write a summary of this paper and publish it on my Medium so you could check it there (I’ll update this answer or add a comment to notify about it)


text:
What would be the steps to create an sentiment analysis chatbot?
We have been assigned a project, in which we have to create a chatbot which will ask question, take the replies, analyse them and give an approximate assessment of the current emotional state of the person. There are two aspects of the project,

Training the bot to choose the next question based on the previous response
And analysing the responses individually, to detect . the sentiment.

What technology would we have to use and what would be the steps to accomplishing the tasks?
Thanks.

There are some different approaches to learning this sequential acting. First, you can use RL (reinforcement learning) and define some rewards over the action of the user over that question, to explore and exploit the environment.
Second, you can use RNN to predict the preferred next question from a history of data. In contrast with RL, you need many instances to train the network to predict better next answer more accurately.


text:
How can/should I use AI to populate a game (in the game theory sense) from text input
I'm wanting to conduct game theoretic analyses of ongoing conflict situations (e.g. the US/North Korea negotiations; Syrian conflict; etc)  as reported in the news media. I believe that AI may help me do this by helping me to pick out from the text: the parties involved; the issues over which they are in conflict; the choices they have; their preferences.  However I'm not sure whether to approach this using 'modern' 'deep learning' approaches or to try something along the lines of the classic work by Schank, deJong etc. who used the notion of scripts (and sketchy scripts) in their work with conceptual dependency approaches.  Does anyone have comments, suggestions that may guide my work please?

I would use pattern matching and frames with slots. It is much easier to see what's going on, and it will work without needing masses of training data. Schank and Riesbeck's Inside Computer Understanding includes a chapter on POLITICS by Jaime Carbonell, which sounds like what you have in mind. Another good source is Bart Kosko's Fuzzy Thinking where he describes using Fuzzy Cognitive Maps for modelling political situations.

That sounds most encouraging Oliver.  As it happens I opened my new copy of the book you mention yesterday and I see the material you mention.  I contacted Roger Schank today and he came right back (!!) also with the suggestion to also see his stuff on stories.  So I'm quite excited.  Many thanks.


text:
How do two perceptrons produce different linear decision boundaries when learning?
I've learned that you can use two perceptrons to ultimately create a classifier for non-linearly separable data. I'm trying to understand how / if  these two perceptrons converge to two different decision boundaries though.

Source: https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html
I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron? I know the weights can be initialized differently but does this second perceptron classify something else? Shouldn't the decision boundaries be converge to be the the same ultimately after training?

Source: https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html

I think your confusion comes from the fact that you are calling those two hidden nodes "perceptrons". You shouldn't call the hidden nodes in your network perceptrons. You should call them "nodes" or "neurons", although the term "multilayer perceptron" comes exactly from that.
You should think of a perceptron as something like

A perceptron simply computes a linear combination of the inputs using the corresponding weights. A single perceptron can (only) linearly separate a set of points.
However, if you have two "stacked perceptrons" (like the hidden nodes in your network), then, provided that both "perceptrons" (or hidden nodes) only compute a linear combination of the inputs, the output of each hidden node can be interpreted as line (like the violet and green lines in your picture).

I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron?

This is because the two perceptrons will eventually have different weights. You can see from your diagram that $p_0$ has weights $w_{0, 0}$ and $w_{0, 1}$, whereas perceptron $p_1$ has weights $w_{1, 1}$ and $w_{1, 0}$. Note that weights are learned, during training, depending on your loss function (at the output node $p_2$) and the labeled dataset (that you use to train that simple neural network).

I know the weights can be initialized differently but does this second perceptron classify something else?

If you consider the two hidden nodes (in your network) separately, then (as I said above) the output of these two hidden nodes can be interpreted as a line, and these lines might be different (as illustrated in your picture), because these two hidden nodes will eventually learn something different about the input. However, you should also think of the output of two hidden nodes as contributing to the input of the output node. So, in a certain way, you're combining the decisions of the two hidden nodes (or, as you call them, "perceptrons").
Note that you will need to introduce "non-linearity" in those hidden perceptrons in order to be able to non-linearly separate a set of points.

Shouldn't the decision boundaries both converge to the same ultimately after training?

No, not necessarily. The decision boundary found by the perceptron learning algorithm depends on the initialization of the weights and the order that the inputs are presented. See chapter 4 (specifically, pages 192-196) of Pattern Recognition and Machine Learning by C. Bishop. Two perceptrons would be guaranteed to converge to the same solution if the loss function was convex (i.e. there's only one optimum).


text:
Why is a constant plane of ones added into the input features of AlphaGo?
In the paper Mastering the game of Go with deep neural networks and tree search, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as following.
Feature       #of planes Description
Stone colour  3          Player stone/opponent stone/empty
Ones          1          A constant plane ﬁlled with 1
Turns since   8          How many turns since a move was played
Liberties     8          Number of liberties (empty adjacent points)
Capture size  8          How many opponent stones would be captured
Self-atari size 8        How many of own stones would be captured
Liberties after move 8   Number of liberties after this move is played
Ladder capture 1         Whether a move at this point is a successful ladder capture
Ladder escape 1          Whether a move at this point is a successful ladder escape
Sensibleness  1          Whether a move is legal and does not ﬁll its own eyes
Zeros         1          A constant plane ﬁlled with 0
Player color  1          Whether current player is black

I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.
What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?
Thank you very much.

I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.

In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when "sliding" along it, but (conceptually) also partially go "outside" the plane (where always at least one "cell" of the filter will still be inside the plane). Intuitively, in the case of a $3\times3$ filter for example, you can imagine that we pad the input features with an extra border of size $1$, and this padding around the "real" input planes is filled with $0$s.
If it's possible for all input features to also have values of $0$, it may in some situations be difficult or impossible for the neural network to distinguish the "real" $0$ inputs from the $0$ entries in the padding around the board, i.e. it may struggle to know where the game board is and where the game board ends. Having a constant input plane that's always filled with $1$s can help in this respect, because that plane can always reliably be used to distinguish "real" cells that actually exist on the game board from positions that are just outside the game board.
As for the plane filled with $0$s... I have no idea why that would ever be useful. Maybe it was useful due to some peculiar implementation detail. In this thread, some people hypothesise that on specific hardware it might make some computations slightly more efficient because of the layout of the data in memory -- it causes the number of channels to be divisible by $8$, which will.. maybe help? I really don't know too much about this, but I do know that on a smaller scale, sometimes adding unused data in classes can indeed increase performance due to better layout in memory. I suppose it's also possible that it was accidentally added by mistake, or "just in case" and that it doesn't really have much of a purpose. The amount of hardware that the AlphaGo team had available to them is quite insane anyway, one channel more or less probably wasn't too big of a concern for them.

What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?

This is explained in the "Features for policy/value network" paragraph in the paper. Quote from the paper:

"Each integer feature value is split into multiple $19 \times 19$" planes of binary values (one-hot encoding). For example, separate binary feature planes ares used to represent whether an intersection has $1$ liberty, $2$ liberties, $\dots$, $\geq 8$ liberties.

All feature planes used were strictly binary, no feature planes were used that had integer values $> 1$. This is quite common when possible, because neural networks tend to have a much easier time learning with binary features than with integer- or real-valued features.


text:
Which pathfinding algorithms can be applied on coloured graphs?
Are there any (well validated) approaches for applying pathfinding algorithms on a graph following specific rules?
To be more specific: I want to introduce a graph with coloured edges. The idea is to apply a well known pathfinding algorithm (such as Dijkstra) on the graph given the rule: "only black and red edges".

The idea is to apply a well known pathfinding algorithm (such as Dijkstra) on the graph given the rule: "only black and red edges"

Remove all the non-black-and-red edges from the graph first, then run it through any off-the-shelf pathfinding implementation.  Or implement your own, and have it completely ignore the non-black-and-red edges.


text:
Isolate the speech of two people in an audio record with two people only
I would like to find a way to isolate the speech of each of the people in an audio record so I can create a file of that form :
[
   {
       "voice_fingerprint": "701066EDD3A0A40A2F53F61EAFD0E6AB",
       "sentences": {
           {
               "sentence": "do you like red apples",
               "position": 1.39 // Seconds. Time position in the audio record
           },
           {
               "sentence": "and how do you feel about time shifts",
               "position": 7.21
           }
       }
   },
   {
       "voice_fingerprint": "8FFEA051AF3E3FB9A80A51A98FE05896",
       "sentences": {
           {
               "sentence": "yes I do like them",
               "position": 4.54
           },
           {
               "sentence": "i feel well about traveling",
               "position": 10.18
           }
       }
   }
]

This may be an interview record.
The problem IS NOT the Speech to Text, but to isolate the two people's sentences. Preferably in Python.
Have you ever worked on this ? Do you have any hints ?

For anyone working on this kind of topic : the technology is called "diarisation". But this is quite new and even Google or IBM APIs don't give good results.

The task of isolating 2 or more speakers is called speaker diarization,
here a list of softwares and useful resources.
Once you have the 2 or more audio files containing the individual voices, you could run some speech-to-text network that also outputs time information.