Card game for Gym: Reward shaping I am working on a card game for openai gym and currently I ask myself how to shape the reward function for it. One round of the game consists of each player picking a card from its hand, whereas not every card can be played depending on the card which has been played by one of the players before. For every set of played cards, there is a total order such that the player with the highest card wins the round.
In the situation in which cards are rejected I want to give the agent some reward.

In case of an invalid card, it is hard to say if that card is any nearer to one of the valid cards than any other. Also the agent should learn that this card is not playable at this point.

For completeness, the agent gets a discrete observation of everything it can remember of the game (its own cards, cards played in current round, cards played in past rounds, game mode (defines the total order of cards)). Then it should play a discrete action which either is a game mode in the beginning or a card during the round. Then it either gets a reward because its card got rejected or it gets a reward based on whether it wins the round or not. The game accounts a certain amount of points for a won round depending on the constellation of played cards in that round.

My question is how to shape the rewards for card rejection and for winning a round. Any ideas? Positive or negative?

In case any more details are required, just ask for them.

My question is how to shape the rewards for card rejection and for winning a round. Any ideas? Positive or negative?

In reinforcement learning, you must set rewards so that they are maximised when the agent achieves the goals of the problem. You should avoid trying to "help" the agent by setting interim rewards for things that might help it achieve those goals.

For card rejection, if that is part of the game (i.e. it is valid to play a "wrong" card, and you lose your turn), then either no reward, or a negative one might suffice. Probably you should go with no reward, because the punishment would be in not winning that round anyway.

If an invalid card cannot actually be played according to the rules of the game, and there is no "pass" move or equivalent, then you should not allow the agent to select it. Simply remove the action from consideration when making action selection. It is OK for the agent/environment to enforce this in a hard-coded fashion: A common way to do that, if your agent outputs a discrete set of action probabilities or preferences, is to filter that set by the environment's set of allowed actions, and renormalise.

What if you want the agent to learn about correct card selection? Once you have decided that, then it becomes a learning objective and you can use a reward scheme. The action stops being "play a card" and becomes "propose a card to play". If the proposal is valid, then the state change and reward for the round's play are processed as normal. If the proposal is not valid, then the state will not change, and the agent should receive some negative reward. Two things to note about this approach:

Turns in the game and time steps for the agent are now separate. That's not a problem, just be aware of the difference.
This will probably not encourage the agent to play better (in fact for same number of time steps, it will probably have learned less well how to win, because it is busy learning how to filter cards based on the observed features), but it will enable it to learn to propose correct cards without that being forced on it in a hard-coded fashion.

For winning a round, then you  might want to reward the agent according to the game score it accumulates. Assuming that the winner of the overall game is the player with the highest score, this should be OK.

However, there is a caveat to that: If by making certain plays the agent opens up other players to score even higher, then simply counting how many points the agent gets is not enough to make it competitive. Instead, you want very simple sparse rewards: e.g. +1 for winning the game, 0 for drawing, -1 for losing. The main advantage of using RL approach in the first place is that the algorithms can and should be able to figure out how to use this sparse information and turn it into an optimal strategy. This is entirely how AlphaGo Zero works for instance - it has absolutely no help evaluating interim positions, it is rewarded only for winning or losing.

If you go with +1 win, -1 lose rewards, then you could maybe make players' current scores part of the state observation. That may help in decision making if there is an element of risk/gambling where a player behind in the scores might be willing to risk everything on the last turns just for a small chance to win overall.


Storing engineered features in a database I have some data in raw csv files which I would like to store in a MySQL database.  The problem is there are constant feature engineering done on this dataset so coming up with one schema to fit all the needs is not possible.  The approach I thought of was to have one main table where the original data is held, and for each new feature that is created, a new table is created.  Then, the user can join the original table with other tables which includes the features they want and use it for their own purposes.

With the above approach, I'm worried about having too many joins when a user needs numerous features.  Please advise on alternative approaches to this problem.

Thanks in advance!

Depending on the technical level of your users, the frequency of the update, the complexity of the transformation, the need to share these features among the users, etc. would a custom VIEW for each user be a feasible solution?

Alternately, would you consider some ETL tools where you can create/ modify calculated columns, customizing the data pipeline as needed?


Products classification by name I am a beginner with machine learning, and I'm trying to build a model to classify products by category according to the words present in the product name.
My goal is to predict the category of some new product, just by observing the categories for existing products.
For example, having the following products:

PRODUCT                                     CATEGORY
soap bar johnsons green leaves              bath
cookie bauducco lemon 120gr                 cookie
nesfit cookie choc and st                   cookie
strawberry soap soft                        bath
spoon hercules medium                       kitchen
soap dish plastic medium                    bath
[...]

My first thought is to group the words (tokens) present in each product, indicating the designated category and the occurrences count (to be used as a weight). So, for this sample, I have:

WORD           CATEGORY         COUNT
soap           bath             3
cookie         cookie           2
medium         bath             1
medium         kitchen          1
bar            bath             1
johnsons       bath             1

Having this, I could be able to train a model, and use it to classify a new product.

For example, having a new product hands liquid soap 120oz, it could be classified as bath, because it contains the word soap, which have a strong weight for the bath category.

In other case, the new product medium hammer could be classified as bath or kitchen , according the occurrence of the word medium in the training set.

So, my doubts are:

Am I going to the correct approach?
What is the best algorithm to be used in this case?
How can I apply this using Weka?

If you have enough data and reasonable number of classes, you can definitely train your model. The grouping of words that you have done is similar to an approach called bag-of-words model. You can use that to build a classifier using Naive Bayes or SVM etc.
On a different note, you can also look at the KNN algorithm because it looks fit for your use case. You can have a look at this paper


How to use neural network's hidden layer output for feature engineering? I was wondering how can we use trained neural network model's weights or hidden layer output for simple classification problem, and then use those for feature engineering and implement some boosting algorithm on the new engineered features.

Suppose,if we have 100 rows with 5 features (100x5) matrix.

data:
      X                 Y
x1,x2,x3,x4,x5          y1  y2
0,1,2,3,4               0   1
3,2,5,6,4               1   0

Network:
2 layers, input and softmax output, compile using cross entropy  .

Can we utilise trained weights or hidden layer output of above network and use it for feature engineering on original dataset and then apply some boosting algo on modified dataset and will it increase accuracy ?

TL;DR:
Yes. You can (iiuc)

Longer Version:
In fact, this is what many popular algorithms like Word2Vec and AutoEncoders do. (With respect to hidden layer outputs)

Word2Vec:
Given an input word ('chicken'), the model tries to predict the neighbouring word ('wings')
In the process of trying to predict the correct neighbour, the model learns a hidden layer representation of the word which helps it achieve its task.

Finally, we just remove the last layer and use the hidden layer representation of the word as its $N$ dimensional vector.

So basically, we feature engineered the word vectors.

For autoencoders,
It takes in $X$ as the input and tries to predict $X$ again, in the process learning a latent representation of the input signal X. The input hidden representation in Layer $L2$ can be used in other tasks. (Note: here X and X hat are the same)

(In fact, you can use features learned by a CNN and feed them into an SVM and get good results)

Relevant to you:
You can train your model on the given data and finally chop off the last prediction layer and use the output of the intermediate layers as features. I believe this should work because it works for so many tasks which I explained above.

Source:

Learn Word2Vec by implementing it in Word2Vec : an article by me explaining word2vec. (Shameless self-advertising here but I feel the article is good and relevant)
Andrew Ng's unsupervised feature learning website


Find frequent segments of GPS trajectories I have a dataset containing GPS coordinates (latitude and longitude), a timestamp variable and a subject ID to identify different persons. What I want to explore is two things:

Find frequent routes (or segments of route) that are shared among the subjects, for example people going from different places where their home is located and go to work taking the main highway.
Find related subjects. Related to the previous one, which are the subjects that are similar (taking into account the time they use to commute, for example)

I looked at Dynamic Time Warping as shown here but it doesn't take into account that for example, subjects can come from different points (but share a segment).

Tried to implement K-Means, but I'm not sure how my centroids are defined (if I should take into account the time or not) as discussed in here, not sure if the similarity distance is making the correct assumptions in this case.

I looked at some papers on the mater but often they work with pre-defined points A and B and try to cluster, however they hint on some good strategies to transform the data.

I know I'm missing the name of the concept of what I'm looking for, maybe you can guide me on the algorithm/papers to review.

Thanks in advance.

With the initial assumption that the trajectories can follow existing routes that a number of all possible sub-trajectories is limited and can be also discreet. I would discretise the map and on all sub-paths compter a frequency analyses:

Create a map of all possible paths such that you overlay all trajectories which gives you the full map.
Dicretise the map by creating paths and path segments defined that each path segment is a path between two closest branching on the particular path, you can also see this task as generating a graph.
Represent each trajectory as sequence of this path segments and compute a frequency characteristic for each segment in the map
You can extend this task with discretization levels such that some short by-passes could be still counted as the main path...


Learning resources for Data Science for applications in road traffic data? My Background

I am a graduate student in Civil Engineering. For the analyses of road traffic data (vehicle trajectories as time series) I work with big data sets mostly about a million data points or more.
I started using R language when MS Excel could not open the big data files. Using basic statistics knowledge and R code I developed few algorithms to identify certain patterns in the data which worked for many applications. But I still lack serious programming skills in R.
Now, I am familiar with basic inferential statistics and R packages (plyr, dplyr, ggplot2, etc). Recently I came to know that Machine Learning algorithms also help in defining patterns in the data through supervised/ unsupervised learning and their application might improve the accuracy of prediction of certain 'behaviors' of drivers using the traffic data.

Question

Having the basic knowledge of Statistics and R, I want to learn about the data science/ machine learning as a beginner. I know that some concepts in Stats. and ML overlap and that might bridge the gap in my learning of ML. Keeping my background in mind, what resources (books/ online courses) would you recommend me to start learning data science and apply it in my field?

The best way to learn data science is through problem solving. I suggest you to head over to Kaggle and work through the for-knowledge problems.

To get a good start on Machine Learning problems, acclimate yourself with the tree package in R. This will help you understand how decision trees work, and building upon that, how random forests, gradient boosting machines and other sophisticated tree based algorithms work.

Then, there are SVMs and deep learning models.

To get an understanding of unsupervised learning problems, learn k-means and employ it for clustering.

Other general concepts/ ideas to understand are:

cross-validation
overfitting, regularization
bias-variance trade off
dimensionality reduction/ variable selection
generalization error
ensemble learning

For books, the most common recommendation to anyone who is familiar with statistics and wants to get into Machine Learning is "The Elements of Statistical Learning" by Hastie, Tibshirani, and Friedman.


What is the proper way to use time-series data for classification? I have a time-series dataset for a classification problem. The data contains brain signals collected via EEG eletrodes along 2 seconds in frequency (Hz). The classes are divided in different files (so each file contains only one class). This is a typical BCI (Brain-computer interface) classification problem, and as far as i know it is the usual way of separating classes for classification.

I want to know the proper way to prepare data for the classifier algorithms, since i have no X,Y values such as a common classification problem should have and the data is time-dependant. Any explanation/material is welcome.

I'm considering using scikit-learn MLPClassifier as well.

Welcome to the community!

Well ... the answer needs a bit of explanation. It is not about MLPclassfier or another python function. It is about background in Signal Processing/Analysis specially for bio-signal.

For different bio-signals like EEG there has been feature extraction studies for long while. For example in your problem, BCI, you probably know P300, etc which are common features (I am pretty outdated. You may have a look at recent review papers.). So usually you extract features from these signals and use them as the input of your classifier (those X, Y values that you mentioned). I explain a bit:

In General

There are general feature extraction methods from signals. They either extracted from signal in time-domain or frequency domain or time-frequency domain.

Have a look at those links and get an overview.

EEG

But for specific bio-signals you don't need to re-invent the wheel. There is a long history of methods for EEg signal analysis. For BCI in particular, there are again reviews to help you find good feature extraction and classification methods.

Finally my point is that, MLPclassifier is just a python function. According to my experience as a ML scientist coming originally from Biomedical Data Analysis, the question is more about "what to do?" before "how to do?". I hope it helped. If you had more questions please comment so I can update my answer.

Good Luck!


Images Dataset Augmentation with fixed parameters of Crop & Rotation Angle I have a Data Set of 170 images in total. of each image i want to create 36 images like this.

An image is cropped from its top-left corner by cropping 5 percent from the top and from the left side of the image. Then, the new image is scaled back to the original size of 188x188. This cropping method could be also done from three other corners. The same thing could be done with 10 percent cropping. This results in 8 images besides the original image. Afterward, these 9 images are rotated by 0, 90, 180 and 270 degrees. Therefore, 36 synthesized images are derived from each input image.

I need python code for this dataset augmentation.

If you need to work on images using Python, the preferred library is PIL.  Here I show a function to do the modifications you have delimited.  This code makes no effort to manage multiple files, or how to name the files converted.  But it does show how to modify the images in the ways you have asked.

Code:

from PIL import Image

def modify_image(im, crop_corner=None, crop_percent=5, rotate=0):
    """
    Crop and rotate an image

    :param im: base image
    :param crop_corner: crop corner 0-3 clockwise from top left
    :param crop_percent: crop percentage of size
    :param rotate: rotate degrees (rotate after crop)
    :return: a modified image
    """
    x, y = im.size
    if crop_corner is not None:
        crop_x, crop_y = [int(i * crop_percent / 100.0) for i in (x, y)]
        size_x = x - crop_x
        size_y = y - crop_y
        if crop_corner in (1, 2):
            crop_x = 0
        if crop_corner in (2, 3):
            crop_y = 0
        im = im.crop((crop_x, crop_y, crop_x + size_x-1, crop_y + size_y-1))
        im = im.resize((x, y), Image.BICUBIC)

    im = im.rotate(rotate, expand=True)
    return im

Test Code:

test_image = Image.open('test_image.png')
new_image = modify_image(test_image, rotate=90)
new_image.show()

To save the image see the (Docs)


Adding and multiplying higher values based on different columns of a dataframe I am trying to calculate scores based on different values in the dataframe. Since these scores are based on different conditions, I am having problems where in the final calculation, which I need in a new column, where I need to choose the higher number in one of the column(Col E) if there are similar values in two columns(Col ID & VID) and unique in one column(Col QID).

ID     VID     QID     A     B     C     D     E
121    212     123     1     2     1     1     1
121    212     435     1     2     1     1     5
223    244     567     2     3     5     1     2
313    232     709     5     1     2     1     3
313    232     887     5     1     2     1     2
454    969     457     1     3     2     2     4
454    969     457     1     2     1     2     4

The last row showcases that Col ID, VID, QID and E can be same but since Col A, B, C and D are different, it will result in different score. The multiplication of Columns A, B, C, D and E (the higher value) should be in the Col Score. The result is expected like below:

ID     VID     QID     A     B     C     D     E     Score
121    212     123     1     2     1     1     1     2
121    212     435     1     2     1     1     5     10
223    244     567     2     3     5     1     2     60
313    232     709     5     1     2     1     3     30
313    232     887     5     1     2     1     2     20
454    969     457     1     3     2     2     4     48
454    969     457     1     2     1     2     4     16

Calculation goes like Columns A * B * C * D * E. For calculating the Score based on the similar ID and VID but unique QID.

The higher value in Col E can be first or last. If it is possible via groupby and then merging them to get this result above then that also solves the purpose.

I have tried .sort in order to bring Col E in a descending or ascending format and then calculating but couldn't write the logic behind the calculation. Just a beginner trying to work this problem for few days now.

Thanks in advance!

So, from what I understand from the problem, you want create a Score column, whereby, typically:
$Score = A \times B \times C \times D \times E$
If ID == VID and the value in QID for that entry is unique in the whole data frame, then $E = max(E)$.
For this, I would create additional columns which check for these conditions before making the score column. Therefore, I would recommend this:
import numpy as np
import pandas as pd

# unique QID
QID_counts = df.groupby("QID").size().reset_index()
QID_counts.columns = ["QID", "QID_count"]

df = pd.merge(left = df, right = QID_count, on = "QID")

# checking if IDs are equal to VID
df["ID_VID"] = df[df["ID"] == df["VID"]]
df["Unique_QID"] = df[df["QID_count"] == 1]

# checking if both conditions are met
df["Max E"] = df["ID_VID"] & df["Unique_QID"]
df["Max E"] = df["Max E"].astype(int)
# obtaining indices
max_E_idxs = df[df["Max E"] == 1].index

# updating E to suit conditions
df["Score_E"] = df["E"]
df.loc[max_E_idxs,"Score_E"] = np.max(df["E"])

# creating score
df["Score"] = df["A"] * df["B"] * df["C"] * df["D"] * df["Score_E"]
```


What are differentiable modules used in deep learning I am reading this paper.

  Convolutional Neural Networks define an exceptionally powerful class of models,
  but are still limited by the lack of ability to be spatially invariant to the input data
  in a computationally and parameter efficient manner. In this work we introduce a
  new learnable module, the Spatial Transformer, which explicitly allows the spa-
  tial manipulation of data within the network. This differentiable module can be
  inserted into existing convolutional architectures, giving neural networks the abil-
  ity to actively spatially transform feature maps, conditional on the feature map
  itself, without any extra training supervision or modification to the optimisation
  process. We show that the use of spatial transformers results in models which
  learn invariance to translation, scale, rotation and more generic warping, result-
  ing in state-of-the-art performance on several benchmarks, and for a number of
  classes of transformations.

Spatial Transformers are used in CNNs to have spatial invariant transformations and consequently the process of learning would be so easier and the networks would have better performance for data with different kind of distributions (noisy data). In this paper I don't realize the meaning of differentiable module. These so called differentiable modules are used in neural networks. But what is the meaning of differentiable?

"Differentiable" means that you can compute the derivative of the operations in the module, and therefore you can compute the gradients of the loss function with respect to the module parameters (i.e. use backpropagation).

This is normally a requirement for operations involved in neural network computations.

Note: you can use non-differentiable operations as part of a computational graph, but you won't be able to backpropagate gradients through them, and therefore any learnable parameters involved in operations prior to the non-differentiable one would not be able to learn.


What are some of the available methods for handling multi-label classification for longer sequences of text I am looking to solve a multi-class classification problem with long sequences of text with some rows having 1000's of tokens. Some of the state of the art methods such as BERT have a token limit and I was wondering what is currently being done to handle longer text sequences when dealing with classification?

Traditional methods don't have such a limit: Naive Bayes, SVM, decision trees...

Also see https://stackoverflow.com/questions/58636587/how-to-use-bert-for-long-text-classification


Passing variables and values from an R script to a shell script I'm working with a shell script(#!/bin/sh) and I wanted to know if there is a way to call variables with their values from an Rscript that I have called in my Shell script.
If that doesn't make sense I want to create, for example a data frame
data=data.frame(a=seq(1,5), b=seq(1,5)) in a script called test.r
and then call that variable, with it's content in my shell script, i.e to print it with an echo:

echo $data

The Bash feature you want is called command substitution, e.g:

echo $(./test.r)

That will echo the output of your Rscript. Alternatively you could capture it in an environment variable.


How regularization parameter in SVM affects hyperplane parameters While learning the SVM classification I came across the regularization parameter $\lambda$:
$F(w,b) = \left\lVert w\right\lVert_2^2 +\lambda \sum_{i=1}^n max(0,1-y_i(w^Tx_i +b)).$
So from what I understand, the main point of SVM is to find the hyperplane $\{w^Tx+b=0\}$separating 2 classes. How does $\lambda$ affects the hyperplane parameters w and b? I tried to plot the 2D case, but still can't get it. Can anyone give the interpretation in this case? Thanks.

This is really an interesting question.
In fact, many people like Svm, many use it, many understand the idea of svm " maximize the distance between the two margins "subjects to the condition of classifying all the samples correctly!! Quite easy but can get a little bit more complicated when we are dealing with none separable data and overfitting problems.

In your question, you mentioned that this parameter is helpful to perform regularization, but why we need regularization? The answer is to avoid overfitting.

But what is overfitting? Overfitting is a problem of overfitting the data. Overfitting can be more serious if you have noisy data.

In Svm, you can think about the distance between the margins as an indicator of how much you are fitting the data. If the distance is too small, then you are trying to fit " seperate between" the samples that are very close to each other. Usually, if two samples are very close to each other and they have different labels, one of them is incorrect, or it is a noisy sample.

Now, from your question, you have two parts, the first part control the distance between the margins, the second part controls the error in miss classification. Changinging " increasing" lambda will give more attention to the error rather than the distance.

The more you reduce lambda the maximum the distance between the margins you can get.

Why you do not see this clearly in your plot?
Because you plot the hyper plane between the two margins, if you can plot the margins, you will see how changing lambda is affecting the distance between the margins.


Choosing a window size for DTW I have time series data from mobile sensors for different motions such as walking, pushups, dumbellifts, rowing and so on. All these motions have different length of time series. For classifying them using Dynamic Time Warping (DTW), how do I choose an appropriate window size that will give good results?

If you have enough data, use cross validation.

If you don't have a lot of data, use cross validation on a similar dataset, and transfer the window size (the UCR archive has a bunch of similar dataset)

Don't forget, that the best warping  window size depends on the amount of training data. As you get more data, you can have a smaller warping window, see fig 6 of http://www.cs.ucr.edu/~eamonn/DTW_myths.pdf

eamonn


Should weights on earlier layers change less than weights on later layers in a neural network I'm trying to debug why my neural network isn't working. One of the things I've observed is that the weights between the input layer and the first hidden layer hardly change at all, whereas weights later in the network (eg. the weights between the last hidden layer and the output) change significantly. Is this to be expected or a symptom of an error in my code?

I'm applying backpropagation and gradient descent to alter the weights.

This is expected and well established - Vanished Gradient

https://en.wikipedia.org/wiki/Vanishing_gradient_problem
This has the effect of multiplying n of these small numbers to compute gradients of the "front" layers in an n-layer network, meaning that the gradient (error signal) decreases exponentially with n and the front layers train very slowly.

While I'm not an expert in neural network (experts, please add an answer), I'm sure professional implementation for neural network is tricky and highly optimized. If you simply implement a vanilla text-book-styled neural network, you won't be able to use it to train a large network. Keep it small and you'll be fine.


How to properly use approximate_predict() with HDBSCAN clusterer for text clustering (NLP)? I have approached text clustering using HDBSCAN based on this article which describes how to do this in R. I've re-written this in Python using this library. The approach is to first calculate TF-IDF vectors for the documents, then calculate a distance matrix for all vector pairs and fit the HDBSCAN clusterer based on the distance matrix.

I have fitter the clusterer with a subset of my documents since the algorithm is slow and my whole set is a bit big. I've limited it to 5000 samples. The clusters that HDBSCAN has found are acceptable. I will fine-tune them later.

Now, I would like to create a Python method that would take a new document, not being a part of the original training set, and return the cluster label which the new document seems to belong to.

I have approached this task by trying to use approximate_predict(). This is where I have question.

I suspect the process for calculating the cluster label for the new document look like this:

Add the new document to the set of 5000 samples that I've used for the clusterer training
Calculate the disctance matrix for the 5001 samples (the matrix will be bigger than the matrix used for clusterer fitting)
Take the last row of the resulting matrix (should correspond to my new text) and remove the last element from the resulting vector (it should contain the distance of the new document to itself, which we can ignore). Removing the last element is to make the dimension of the last vector match the dimension of the matrix used to fit the clusterer. Otherwise the clusterer would complain.
Use the approximate_predict() method and pass the vector obtained in step 3 to get the cluster label.

My questions are:

Is my approach correct? (it seems overcomplicated but I don't know what it should look like)
Will it perform well in production when I start passing lots of documents to this method? (the processing needed before I actually call approximate_predict() seems huge)
How can it be done differently?
Would it be better to not use the approximate_predict() method, but instead take the cluster labels that HDBSCAN calculated for my my 5000 samples and use it for supervised learning to train a classifier to then classify new documents?

I am looking forward to your answers and I will appreciate them a lot.
I am completely new to this and I don't have anybody around me to discuss this.

Why do you recompute the distance matrix?

Just compute the 1x5000 vector directly for all new points. You can even do this in batches, then feed one row at a time to the predictor.


Input for LSTM for financial time series directional prediction I'm working on using an LSTM to predict the direction of the market for the next day.

My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$ represents a vector of features for day $i$, i.e $x_i \in \mathbb{R}^D$. The target variable $y_i$ is the direction of the market for the next day $$ y_i = I_{P_{i+2} - P_{i+1} > 0} $$ where $P_i$ is the opening price of the asset for day $i$.

I am wondering now how to configure the input to fit in an LSTM framework. The LSTM requires a sequence of length $T$ and uses this together with target $y_T$. One approach is a rolling window. Take input as input the sequences $I_1 = x_1, \ldots, x_T$, $I_2 = x_2, \ldots, x_{T+1} \ldots $ and use $y_T, y_{T+1} \ldots$ as the target variables. The problem with this is that $I_1, I_2$ are very similar, they differ only on two points yet the target variable may be $1$ for the first series and $0$ for the other, making it impossible for the LSTM to learn in my opinion.

I am wondering if anybody has any idea of how to approach this problem. The rolling window approach above is considering the inputs $I_k$ as independent, similar to if we would have input $I_k$ to be a sentence which should be classified as french or english. I want the LSTM to take into consideration that the sequences its fed are all parts of the same long sequence if that makes sense.

A lot of papers use recurrent neural networks for this problem, but never really specify on how they structured the input.

I think there's a partial answer to your question, about the part: "the sequences it's fed are all parts of the same long sequence"

You can train your LSTM model in stateful mode. There's a clear explanation here: http://philipperemy.github.io/keras-stateful-lstm/

In short, in stateful mode, the model will remember what it is previously fed and will also use that information for training. That way, it can sense that the current input is a part of a long sequence.


Epoch 1/5 won't stop When i run my code with 5 epochs, code gets stuck at first epoch and run continuesly. I tried applying various parameters but couldn't make it.

here is my code...

import tensorflow as tf
import numpy as np
import os
from google.colab import drive
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow import keras
from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D

training_path = 'drive/My Drive/tesla'
validation_path = 'drive/My Drive/validation'

training_generator = ImageDataGenerator(rescale=1.0/255)
validation_generator = ImageDataGenerator(rescale=1.0/255)

training_set = training_generator.flow_from_directory(batch_size=2,
                                                      directory=training_path,
                                                      target_size=(150,150),
                                                      class_mode='binary',
                                                      shuffle=True)

validation_set = validation_generator.flow_from_directory(batch_size=2,
                                                          directory=validation_path,
                                                          target_size=(150,150),

                                                          class_mode='binary')

model = keras.models.Sequential([
                                 Conv2D(8, 3, padding='same', activation='relu', input_shape=(150,150,3)),
                                 MaxPooling2D(),
                                 Conv2D(16, 3, padding='same', activation='relu'),
                                 MaxPooling2D(),
                                 Conv2D(16, 3, padding='same', activation='relu'),
                                 MaxPooling2D(),
                                 Conv2D(4, 3, activation='relu'),
                                 Flatten(),
                                 Dense(4, activation='relu'),
                                 Dense(1)

])

model.compile(optimizer='adam',
              loss=tf.losses.BinaryCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.fit(training_set,
          validation_data=validation_set,
          epochs=5,
          verbose=1,
          validation_steps=20/2)

but when i run my code, it shows this and run non stop at first epoch.

Epoch 1/5
  14781/Unknown - 630s 43ms/step - loss: 4.0421e-06 - accuracy: 1.0000

Although my training sample size is only 40 (20 cats and 20 dogs) and validation sample size is 20 (10 cats and 10 dogs). I am coding in google colab.

Thanks in advance.

You are missing one argument in your code. Following will fix your problem.
model.fit(training_set,
          training_steps= training_set.samples//training_set.batch_size,
          validation_data=validation_set,
          epochs=5,
          verbose=1,
          validation_steps=validation_set.samples//validation_set.batch_size)


Running a query in R after establishing dbconnect I do not seem to figure out what is wrong it the following statment. The connection to the DWH is establishe but the query statement in R seems not to work, with the following error :

LR=dbGetQuery(con, "select id as ID,
       date_c."Professional_Status" as Prof_Status,
       case when talk_sec >= 5 then 1 else 0 end as Established_Connection
from id_collect as id_c
left join date_conncet as date_c on id_c.date=date_c.date
where date::date = '2018-01-19' and country = 'IT' and type = 'shop'
              and 'district' = 'South' and interaction is true")

This piece of query runs in the SQL editor (using Postgresql) but in R I get the error meassage: Error: unexpected symbol in "  or unexpected numeric constant. I do use the following:

install.packages("RPostgreSQL")
install.packages("DBI")
library(RPostgreSQL)

I believe it is not about the syntax but the quote I use, whats the proper quote ethiquette in R when using numeric vs. categorical values? The whole select statment must begin and end with "select ...." correct?

Also to add how to establish a connection to your DTW (add your DWH infos):

con <- dbConnect(RPostgreSQL::PostgreSQL(),
                 dbname="DBNAME",
                 host= "HOSTNAME",
                 port=0000,
                 user="youruser",
                 password="yourpassword")

I understand your question as: I try to send this query to the database but it doesn't work.
As far as I can see the trouble is here:

"select id as ID,
       date_c."Professional_Status" as

You first start with double quotes and R thinks that the string ends at the next double quotes, so before Professional_Status. Use either single quotes like you do in date = '2018-01-19'
Or use single quotes at the ends and double in between. Don't mix and match, it will not work.
Since Profesional status is not defined variable nor is it a valid R statement this will fail.

Some small tips:
If you want to substitute professional_status with a variable you have defined in your R session, use something like paste("text", variable " text") or use the excellent glue package that allows you to write glue("text {variable} text")
A final tip, if you use an IDE, like for instance RStudio, you can actually see that the text ends at the first second quote, because the highlighting is different.


Why can't my neural network learn how to predict the squares of natural numbers? I want my neural network to learn to predict the square $n+1$ number having $n$ number.
I am considering a regression problem. That's what I'm doing:

from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense, Embedding, Dropout
import numpy as np

x = np.array([[int(i)] for i in range(1001)])
y = np.array([x*x for x in range(1001)])

model = Sequential()
model.add(Dense(100, activation = 'relu', input_dim = 1))
model.add(Dense(50, activation = 'relu'))
model.add(Dense(10, activation = 'relu'))
model.add(Dense(1))

model.compile(loss='mse',optimizer='adam', metrics=['mae'])
model.fit(x,y,epochs= 2500)
pred = model.predict([1001])
print(pred)

However, as a result, I get [[ 1000166.8125]]  instead 1002001.

Update:

x = np.array([[int(i)] for i in range(80001)])
y = np.array([x*x for x in range(80001)])
print(x)
print(y)
model = Sequential()
model.add(Dense(20, activation = 'relu', input_dim = 1))
model.add(Dense(20, activation = 'relu'))
model.add(Dense(1))

adam = optimizers.Adam(lr=0.0002,beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False,)
model.compile(loss='mse',optimizer=adam, metrics=['mae'])
model.fit(x,y,epochs= 3000)
pred = model.predict([80001])
print(pred)
model.save_weights("test.h5")
model_json = model.to_json()
json_file = open("test.json", "w")
json_file.write(model_json)
json_file.close()

result: [[  4.81360333e+09]]

Decrease the number of hidden layers; you can omit the dense layer with $50$ neurons. Furthermore, train your network more. You should also provide more data. It is not much at the moment.

Your current architecture is very deep for such a relatively easy task. Consequently, it needs more train time. You can just decrease the size of the current model by diminishing the number of hidden layers and neurons. For instance, use the following setting to see how you can train very fast and have a good accuracy.

model = Sequential()
model.add(Dense(20, activation = 'relu', input_dim = 1))
model.add(Dense(20, activation = 'relu'))
model.add(Dense(1))


What would be the best way to impute data? Other than just filling in with the mean of a feature, what other methods are there which can work well? I am trying to decide whether or not to use a denoising-autoencoder or just impute with the mean or any other method which can perform well enough which is quickly implementable.

Also is it necessary to make any assumptions about the underlying distribution? The denosing-autoencoder seems attractive here because you do not need to make any explicit assumptions.

You can treat the missing feature as the target variable of a sub-problem and create a classifier (e.g., a linear model, SVM, etc) for it.


Determine the most important documents for supervised learning I have somewhat of a general/high level question.

Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class classification). My supervised model performs fairly well on testing data but what I'm trying to do now is find a way to sort out which documents from the production data (not human labeled) if added to the training set would improve accuracy the most on that particular production data set.

The idea is once the initial predictions are made on production data, another algorithm would be run to determine the n most important documents (say 500) that if manually labeled by a human and added to the training set would improve accuracy the most, assuming the model does a 2nd round of prediction after the n documents have been added to the training set.

So the process would look like this:

Initial predictions on production data are made
Algorithm runs to determine the most important records to increase accuracy of the model (based on features most likely).
Human reviews these most important records and adds them to the training set
A 2nd round of prediction takes place, hopefully with a better accuracy metric

I'm thinking of using something like K-Means after doing some dimensionality reduction since it's NLP problem with lots of features.

Does anyone have any experience or suggestions in regard to this topic? Am I on the right track?

This is commonly called active learning.
The most common active learning approaches for text classification are random, data-based, model-based, and predication-based.


Training data from different sources I am working on a binary classification problem. My data contains 100K samples from two different sources. When I perform the training and testing on data from the first source I can achieve classification accuracy up to 98% and when perform training and testing on the data from the second source, I can achieve up to 99%. The problem is when mix both of them, the classification accuracy goes down to 89%. Any idea how to perform the training to achieve high accuracy. Knowing that one of my features is related to the source

It seems that you have a domain adaptation problem. The samples from the two sources behaves differently.

I suggest reading Frustratingly Easy Domain Adaptation. As the name hints, the solution is easy , popular (800 citation until now) and a good survey of other directions.

I understand that the classifier that you run on the entire dataset was train on it.
How well does the classifiers trained on the single sources perform on the other sources?
How many of the samples belong to the first source?
Will you have an indication at production of the source of the sample?
The answer to these question might open more directions.


Coefficients of Linear regression for minimizing MSE (I asked this in mathematics site, but nobody responded, it seems the whole problem is more related to data science than math.)

In a regression problem, loss function is:

$$L(a,b) = {\sum_{i=1}^n (y^i - (ax^i +b))^2})$$

In order to minimize this value, we need to set the derivative of L with respect to each of its parameters, equal to zero.

Hence, $\frac{dL}{db}$ would be $y^- + a \cdot x^-$
But what would $\frac{dL}{da}$ be?

$$\frac{\sigma L(a,b)}{\sigma a} = {2 \cdot \sum_{i=1}^n (y^i - (ax^i +b)}) \cdot \frac{\sigma \sum(y^i - (ax^i +b))}{\sigma a}$$
$$\frac{\sigma L(a,b))}{\sigma a} = {2 \cdot \sum_{i=1}^n (y^i - (ax^i +b)}) \cdot -\sum(x^i)$$
$$\frac{\sigma L(a,b))}{\sigma a} = {2\sum(x^i) \cdot \sum_{i=1}^n (y^i) - 2\sum(x^i) \cdot \sum_{i=1}^n (ax^i)  - 2\sum(x^i) \cdot \sum_{i=1}^nb}$$
$$\frac{\sigma L(a,b))}{\sigma a} = {2\sum_{i=1}^n (y^i \cdot x^i) - 2\sum_{i=1}^n (ax^i \cdot x^i)  - 2\sum_{i=1}^n x^i \cdot b}$$
$$\frac{\sigma L(a,b))}{\sigma a} = {2\sum_{i=1}^n (y^i \cdot x^i) - 2\sum_{i=1}^n ((ax^i +b) \cdot x^i)} = 0$$

How this would be equal to $\frac{cov(x, y)}{\sigma^2x}$

For a linear regression we have the loss function

$$J(a,b)=\sum_{n=1}^N(y_n-a-bx_n)^2.$$

The partial derivatives are

$$\dfrac{\partial J}{\partial a}=2\sum_{n=1}^N(y_n-a-bx_n)(-1)$$
$$\dfrac{\partial J}{\partial b}=2\sum_{n=1}^N(y_n-a-bx_n)(-x_n).$$

If we set both derivatives to zero and divide by the sample size $N$ we obtain

$$0=\overline{y}-a-b\overline{x}$$
$$0=\overline{xy}-a\overline{x}-b\overline{x^2}.$$

Now, solve the first equation for $a= \overline{y}-b\overline{x}$ and plug this into the second equation

$$0=\overline{xy}-\overline{x}\overline{y}+b\overline{x}^2-b\overline{x^2}$$

and solve for $b$ to obtain

$$b=\dfrac{\overline{xy}-\overline{x}\overline{y}}{\overline{x^2}-\overline{x}^2}.$$

The espression in the numerator is the covariance for a sample and the expression in the denominator is the variance of $x$.


Should I gray scale the image? I'm categorizing 30 types of clothes from the image using R-CNN Object Detection Library from tensorflow : https://github.com/tensorflow/models/tree/master/research/object_detection

Does color matter when we collect images for training and testing?

If I put only purple and blue shirts, I guess it won't recognize red shirts?

Should I gray scale all images to detect the types of clothes? :)

Your hypothesis about missing colours in your samples affecting results in production could be correct. However, it is trivial to convert images to greyscale as you load them from storage. So keep them in colour, and convert them as you load them if you need black and white.

That way you can try both with and without colour as input and you will have your answer.

This is very little effort to do in practice, and allows you to do the "science" part of data science by comparing two approaches and measuring the difference. This is standard practice, even if you are reasonably certain one approach or another is "the best", it is normal to explore a few variations. When you are not sure, then it is even more important to try it and see.

To test your hypothesis, you could put all the t-shirts of a particular colour in your test set. If that reduces the accuracy of your results with the colour model, it would back up your concern. One fix for that might be to remove colour information from the model, if it is not relevant to the task. An alternative is to collect more data so you have enough samples of different colour shirts. However, you might find if you are fine-tuning a neural network trained on many more images (e.g. Inception v5) that the impact of colour is less even though your samples do not cover all possible T-shirt colours.


Stock prediction through LSTM blue: training loss
black: validation loss

If modelling gives me such plot, what does it imply? Is it overfitting?
How should I make it better?

Code as below:

classifier = Sequential()
classifier.add(LSTM(
    input_dim=72,
    output_dim=50,
    return_sequences=True))
classifier.add(Dropout(0.20))

classifier.add(LSTM(
    100,
    return_sequences=False))
classifier.add(Dropout(0.20))

classifier.add(Dense(
    output_dim=1))
classifier.add(Activation('relu'))
classifier.compile(loss='mse', optimizer='rmsprop')

X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))

#classifier.fit(X_train, y_train, batch_size = 10, nb_epoch = 10)
cls_return = classifier.fit(X_train, y_train, batch_size=300, nb_epoch=500, validation_split=0.05)
history = cls_return.history
plt.plot(history['loss'], label='Training Loss', color='blue')
plt.plot(history['val_loss'], label='Validation Loss', color='black')
plt.show()

Yes, this is overfitting. Financial time series exhibit many peculiarities, including heteroscedasticity, high tail risk, all kinds of seasonality and all kinds of momentum. Machine learning algorithms tend to pick up these patterns and overfit on them. To my knowledge, there is no machine learning algorithm that can compete with classical time series analysis when it comes to understanding stock prices.

Please also keep in mind that if someone were able to make predictions like 'tomorrow the stock price is going to rise by 5%', they could earn billions within a few weeks.

Edit: You ask if there is any chance of training a network that is better than random guessing. Yes, for two reasons.

First, if you try hundreds of parameter combinations and evaluate them all on the same test set, you become the victim of what might be called 'overfitting of the second degree.' Performance on the test set becomes part of your decision process because you discard models that perform poorly on the test set and keep models that perform well. You are essentially training on the test set and will predict it with 100% accuracy eventually. But use this seemingly perfect model on new, previously unseen test data, and you will be disappointed.

Second, heteroscedasticity, seasonality and momentum all introduce a little bit of 'real' predictability into your time series. If you get very lucky, your algorithm might identify these patterns and at the same time ignore all the weird residual noise.
This is incredibly unlikely, and it is probably irrelevant. You should not compare your prediction accuracy to random guessing, but to a reasonable baseline model. For monthly prices of individual stocks, this could be the Fama-French three-factor model. (Daily or intraday stock prices will send you a bit deeper into the rabbit hole that is quantitative finance). I bet you five dollars that you will not be able to beat that model's performance on previously unseen data.

My advice to you: Don't clean the lion's cage on your first day as a zookeeper. Start with something nice and easy.


How to encode data with a feature having multidimensional features (colors)? My dataset has around 20 features, one of which is colors(in string format). There are around 50 different colors. I have converted them to RGB, but now I want to encode the data in such a way that the values are relevant, because I will cluster the data on this feature later on. To do this, one-hot encoding proved ineffective. Suggestions?

One-hot encoding will give you a sparse matrix, Try LabelEncoding, before converting them into RGB that is.

Also, you could try breaking the RGB values into three features (R, G, B) and try that approach as well.

Hope this helps.


SMOTE and oversampling with constraints I'm trying to apply SMOTE to a dataset that has time-constraints. I have information about users visiting a website. For some features, there are time constraints, e.g having the first visit and the last visit at the website, the first visit (timestamp) is always lower or equal than the last visit.
If I apply SMOTE(or SMOTENC for categorical), I end up having synthetic samples for which the last visit occurred before the first visit. This leads to a sample that cannot exist in the real-world, hence can affect negatively the performance of the model.
Is there a way to apply SMOTE and impose certain rules. Or in alternative, are there oversampling techniques that can deal with this problem?

One option would be to do something more similar to bootstrapping since that would be re-sampling existing data.
Another option would be to generate extra samples then prune based on the constraints.


Validation data shall be in broken down into batches or not? I am using fit_generator to train the model. The training dataset is being read from a generator function which gives data in a constant batch size. Now I want to know what approach shall I adopt for validation data. Shall I make a generator for the validation set or load it completely into memory and use it in fit_generator?

Note: Validation dataset is fitting into memory quite easily therefore batching is an option for me.

If validation batch size = 1 it has no difference in comparison with the case that load the data completely. But with any other value of batch size the evaluation metric will be an average of it in different batches; So, its real value will not be resulted.


Embedding layer before LSTM layer I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and treat it as a time series. They do some transformations and they have a have a 3-dimensional dataset which is 12x14xm users; each user, m, has 12 daily features for a total of 14 days. This set gets fed into an LSTM so that each LSTM timestep has receives a 12-dimensional vector. Hopefully, I worded this right so it's understandable. They were able to improve their results by adding a embedding layer between the input and the LSTM layer.

  We connect a fully connected feedforward neural network to the
  original daily activity vactors, which converts users' sparse activity
  features of each day into distributional activity embeddings...

I'm completely confused on what this looks like dimension-wise and am confused if there is a fully connected network before each LSTM timestep or is there one fully connected network before the feeding the LSTM? For example, is the 3-dimensional data set flattened and then fed into a fully connected network and then reshaped back into a 3-dimension dataset that has 14 timesteps?

I made these concepts to help clarify what I mean.

The most straightforward approach to understand their proposal is to look at their source code, which is linked from the article: https://github.com/yangji9181/ClusChurn/blob/master/model.py#L35

It reveals that the "embedding" is obtained by means of a mere linear layer.


How to train a model using a daunting huge training dataset I have an extremely huge dataset and I'm wondering me how could be the right way to set an experiment to use this data to train a model.
I understand that I can use data-reduction to, for instance, drop out some variables. In despite data-reduction can actually reduces the data amount, as I can see this technique is intended to improve the model training effectiveness, not to deal with the practical issues that comes out from the data amount.
One of my ideas is to suffle the whole data first and then split the data into 'small' chunks. Once I have, let's say' $N$ chuncks, I can train the same model using each chunck as follows:
initialize(M);
for(n in N) {
  D = load_chunck(n);
  M = train(M, D);
}

Although this approach can be effective to fit the experiment to computational resources at hand, I'm afraid that training the model this way can affect the model's quality by including bias from the latter chuncks. In addition, N is now a new hyperparameter to be set.
Another alternative I can see is by using statistical sampling:
D = retrieve_sampling(sampling_size);
if (D is good)
   M = train(D);

I'm wondering me if there are other ways to do it then the ones I've cited here.

Quite often with massive datasets the model doesn't actually need the whole data. So I think the first step is to check whether using the whole data is useful: run an ablation study where you use say 1%, then 2%, 3%, .., up to say 10% of the data (adapt the levels to your case of course). Each run consists in training on the x % subset and evaluating on a validation set (be sure to separate your real final test set before anything, this study should only use a validation set).
The goal is to estimate how much gain in performance is obtained by adding training data. Plotting the performance as a function of the amount of data should give a decent idea of the trend, even if it doesn't reach the point of maximal performance, i.e. when more data doesn't improve performance anymore. With this information you can make a better decision about how to proceed with the real training.


Feature selection is not that useful? I've been doing a few DataScience competitions now, and i'm noticing something quite odd and frustrating to me. Why is it frustrating? Because , in theory, when you read about datascience it's all about features, and the careful selection, extraction and engineering of those to extract the maximum information out of raw variables, and so far, throwing every variable as it is in the mix seems to work fine with the right encodings. Even removing a variable that has 80% nulls ( which in theory should be an overfitting contributor ) decreases the performance of the regression model slightly.

For a pratical case : I have long/lat for a pickup point and destination point. I did the logical task of calculating the distance ( all kinds of them ) from these points. And dropped the long/lat. Model performs way better when you include both ( coordinates & distance ) in the features list. Any explanations? And a general thought on my dilemma here with the real utility of feature selection/engineering/extraction

EDIT : could it be that the information we can get out of the coordinates is bigger than the distance? Is it just possible to extract features that are more beneficial to my model that plain long/lat?

My experience is the same. I think in my case at least it's largely down to the algorithms I would generally use, all of which have the capacity to ignore features or down-weight them to insignificance where they're not particularly useful to the model. For example, a Random Forest will simply not select particular features to split against. A neural network will just weight features into having no effect on the output and so on. My experience is that algorithms which take every feature into account (like a vanilla linear regression model) generally suffer far more.

Additionally, in a "production" rather than competitive environment I found that feature selection became much more important. This is generally due to covariate shift - the distribution of values for certain features changes over time and, where that change is significant between your training dataset and the live predictions you're making day-to-day, this can really trash your model's outputs completely. This kind of problem seems to be administered out of the datasets used for competitions, so I never experienced it until starting to use ML at work.


Clustering with categorical as well as numerical features I have dataset consisting of house prices for example. The dataset contains features such as: house size, monthly rent, house colour, location, year the house was built.
I wanted to group these all attributes into clusters. The problem is how to represent the categorical features, such as colour. And how does the clustering algorithm work with these categorical variables?
Another question is: what happens if let’s say monthly rent 0, how does this affect clustering?

thank you for your question, which asks about how to represent categorical variables in clustering.
The main way that we represent any categorical variable is to represent them as one-hot encodings (https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/).
After converting our categorical variables to one-hot encodings, we simply concatenate these vectors with our numerical values (which should be normalised!) to make a long / high-dimensional vector for a particular example in your dataset.


How does Max Pooling (of size 2x2) change the size of receptive field in CNN? I am often told that Max Pooling of 2x2 doubles the size of the receptive field from the previous layer. If that is true, I would like to understand how that happens.

I have already checked this article and this one. However, I am not able to understand the effect of max pooling on the receptive field.

Appreciate any help on this.

Thanks!

Before anything, let's try to understand what Max-pooling actually does. Intuitively, Max-Pooling takes the maximum of the value inside the kernel as the maximum value is something that causes a larger impact from the picture. When the size of the kernel is 2x2, half of the values denote the actual value so the receptive field increases. A visualization could be found in this video. Also you can look at this video too


Best way to classify plots which are overlapping? I have an experiment in which it was done under two conditions. For each condition, the experiment was performed 26 times. The output of the experiment is a plot with 70 time indices. I would like to train a classifier to predict, given a plot, to which condition it belongs. The image below shows the output of the conducted experiment under the two conditions recognised by different colors. The actual experiment begins at index 35, and thus it can be seen there is no difference in the outcome of the experiment before that regardless of the condition. The plots represent power spectral density of EEG from one channel (electrode).

I am trying to train an svm classifer ignoring the features below 35. The classifier is having hard time doing so considering the high variability of each condition. One thing is, averaging the red plots and blue plots yield a noticeably different behaviour, as can be seen from the second figure. I would like to improve the accuracy of my classifier, beyond 65%. Is LSTM suitable for this type of problem? Any other suggestions?

A quick answer is since you have time series data, LSTM is generally appropriate if you would like to model/exploit time dependencies between values in each series.  LSTMs are now combined with CNNs for even better performance.  You can refer to this Q&A for that: What is the best method for classification of time series data?Should I use LSTM or any other method.  The only problem is that it appears that you have an extremely small dataset, which normally leads to overfitting with neural networks given the number of parameters they need to compute.  And there is actually a lower bound on the number of training samples needed for LSTM (see: Number of parameters in an LSTM model).  Perhaps you can find pre-trained RNNs/LSTMs trained on similar data that you can use.

In general, any classification algorithm (especially those that perform well on time series data) can (and probably should) be tried.  With SVM, I hope you've tried different kernels.  There is at least this one paper that discusses the relative merits of different SVM kernels for time series analysis: http://www-ai.cs.uni-dortmund.de/EVENTS/FGML2001/FGML2001-Paper-Rueping.pdf.

If you have not already done so, I also recommend that you research feature extraction strategies for time series data (and particularly, EEG data) that have worked for others.  There is at least one Q&A on this topic on the Cross Validated site: https://stats.stackexchange.com/questions/66027/time-series-classification-very-poor-results.


Delete rows of a column using another column statistics in R I have a dataframe with the following columns:

If a year (see "year" column) has less than 10 weeks reported, I would like to delete it from the dataframe. The column "weeks_reporting" gives the order of the week in a year. How can I accomplish this task in R?

You can accomplish this using a logic index as follows:

us_contagious_diseases<-us_contagious_diseases[!(us_contagious_diseases$weeks_reporting<"10"),]


Optimizing decision threshold on model with oversampled/imbalanced data I'm working on developing a model with a highly imbalanced dataset (0.7% Minority class). To remedy the imbalance, I was going to oversample using algorithms from imbalanced-learn library. I had a workflow in mind which I wanted to share and get an opinion on if I'm heading in the right direction or maybe I missed something.

Split Train/Test/Val
Setup pipeline for GridSearch and optimize hyper-parameters (pipeline will only oversample training folds)
Scoring metric will be AUC as training set is balanced at that point
Since model was trained on balanced dataset, it will probably be very conservative and predict a lot of false positives
Taking above into consideration, model will be calibrated to have more accurate probabilities (CalibratedClassifierCV)
View precision/recall curve with calibrated probability thresholds on validation set and determine optimal point

Does this process sound reasonable? Would appreciate any feedback/suggestions

I am not sure if in the last point, you meant the validation set instead of the testing set.

Here is my advice:
1- understand the impact of having data imbalance. Let start with understanding the difference between overall accuracy and average class accuracy. If you only care about overall accuracy, then data imbalance is not a problem, else you need  to handle the data imbalance problem.

2- the data distribution of training set can be changed by using oversampling. Undersampling, synthetic sampling, data augmentation... etc. BUT you should NOT change the data distribution of the validation and the testing sets.

3- use the training set for training,  the validation set for tuning  the hyper parameters , BUT do not touch the testing set

4- use the testing set for testing only

5- you can control the behavior the model by controlling the data distribution, you do not need to have fully balanced data, you can control the oversampling process in a way to control the behavior of the model without using a threshold.


Unknown program 'spark-itemsimilarity' chosen I have cloudera CDH5 running inside a virtual box.

when I try to run :

mahout spark-itemsimilarity ....

I get the error:

Unknown program 'spark-itemsimilarity' chosen.

Do i have to install any additional package to run the spark-similarity?

Any help would be appreciated !

Spark support for Mahout came from Mahout 0.10 release while you are using 0.9 release. So this should explain why you get the unknown program error. I would suggest using a higher version of Mahout.


I intend to do classification modelling, but my target variable has only one value Currently I have a dataset and I am trying to predict whether someone will default on their bank loan.

The dataset is quite tricky. It covers those who have defaulted in the past, but is also tracking those with current loans. So we don’t know whether they will default in the future.

This means that the target variable (default, not default) has one unique value which is 1. There aren’t any observations that can describe the 0 class.

Assuming I can’t use external datasets, what’s the solution to this problem? How can I best understand those who have defaulted if I don’t have something to compare them to?

Assuming I can’t use external datasets, what’s the solution to this problem? How can I best understand those who have defaulted if I don’t have something to compare them to?

If you wanna use this only this data as classification task, you can't perform this task. You could way this around, by generated fake data with label 0(you think about which value of parameters comes to default).

But you could try clustering algorithms(like k-means) and find the groups of specific clients. Perhaps one of these groups its group of default.


How does a CBoW model convert a word to a vector? A CBOW model actually takes multiple words as inputs and a targeted central word as the output.
So, the trained model actually maps several words to a single one, I mean it takes context words and outputs the central word.
But what we expected to get is model mapping a word to its vector representation. It seems the output is coincident but not the input and the mapping.

So like in genism, how it really works to map a word to its vector representation?
Does it just save all final model's outputs as the central words' vector representation? But the final model's outputs would close to the ground truth's one-hot embedding rather than a vector with context information.

For short, my question is:
How does a CBoW model convert one word to its vector representation?

CBoW (Continuous bag-of-words) is a theoretical architecture, not exactly a saved model or a library like gensim. Gensim might be an implementation of CBoW to which you feed a one-hot vector and get your word vector output.
CBoW as a theoretical model gives 'representational meaning' to a word based on the 'representational meaning' of the word surrounding it. 'Representational meaning' may just be a fancy term for word vectors. So basically the model builds word vectors out of surrounding word vectors possibly using negative sampling and the NCE loss, but the main point is that it fine-tunes these vectors until their 'representational meaning' becomes well-honed.
These well honed vectors are then probably callable from gensim through one-hot inputs.


What is the difference in xgboost binary:logistic and reg:logistic What is the difference in R in xgboost between binary:logistic and reg:logistic? Is it only in evaluation metric?

If yes, how does RMSE on binary classification compare to error rate? Is the relationship between the metrics more or less monotonic, output from tuning on one metric should not differ significantly between those two approaches?

There is no difference in the algorithm, they are the same. When you use  reg:logistic the default evaluation metric is rmse, but you can change this parameter.

The error rate and the rmse may differ depending on the distribution of your output, as the error rate uses a limit of $0.5$ if you have the output values concentrated in $0$ or $1$ it will be much smaller than rmse, even though its correlated metric the model can be very different, the application will depend on your problem.

To evaluate performance of logistic regression models is usual to calculate auc, logloss, accuracy, pseudo-$R^2$ and some others.


Modelling for similarity between two descriptions I have a dataset of companies and research projects that they were involved in. A subset of the dataset is shown below.

I am trying to find a way to model similarity between the company and the research projects using this description.
For each pair of descriptions, I would like to output a number, similar to cosine similarity, which will indicate how similar the description in second column is, to the project title in third column.
How can I go about this?
Thanks in advance!

So the questions asks for how to compute similarity between the organisation description and project titles.
One initial thought would be to use a Doc2Vec model (concept, implementation), which will take the organisation descriptions and project titles as input and output a n-dimensional vector in semantic space for the given text.
From this, you can at least has a baseline which you can use with cosine similarity to see how similar the organisation description is to the project title.


How can you decide the window size on a pooling layer? On the convolutional neural network, there used one or more pooling layers. As far as I know many tutorials instruct you to set it either 2 or 3 for the window size. For example, in this tutorial:

Pooling Layers
After some ReLU layers, programmers may choose to apply a pooling layer. It is also referred to as a downsampling layer. In this category, there are also several layer options, with maxpooling being the most popular. This basically takes a filter (normally of size 2x2) and a stride of the same length. It then applies it to the input volume and outputs the maximum number in every subregion that the filter convolves around.

I make the relevant part bold. But why is it usually set to 2 (or in some cases 3)? How can I know what size is appropriate? Should I just tweak the parameter via a trial and error like a brute-force way, or is there any tips on deciding the window size?

When you get a bit more insight into network topologies these hyperparameters will make more sense, but in general this is just like any other hyperparameter, you will have to test some settings and see what works better. In the case of pooling layers it is actually relatively interpretable. Why do we use pooling? To downsample our feature maps. This is done to decrease the resolution of our processed image but also to increase the receptive fields of our following layers. Because it is a form of downsampling we throw away all the information that was not the maximum in case of max-pooling, or we throw away information about the spread with mean-pooling. By increase our pooling filter size, we decrease our resolution even further and lose more information, but we will also need less parameters in our network if we have a fully connected layer at the end and our next convolutional layers will have a bigger receptive field, but with less attention to detail.


Scikit Learn OneHotEncoded Features causing error in classifier I’m trying to prepare data for input to a Decision Tree and Multinomial Naïve Bayes Classifier.
This is what my data looks like (pandas dataframe):
Label  Feat1  Feat2  Feat3  Feat4

0        1     3       2      1
1        0     1       1      2
2        2     2       1      1
3        3     3       2      3

I have split the data into dataLabel and dataFeatures.
Prepared dataLabel using dataLabel.ravel()
I need to discretize features so the classifiers treat them as being categorical not numerical.
I’m trying to do this using OneHotEncoder:
enc = OneHotEncoder()

enc.fit(dataFeatures)
chk = enc.transform(dataFeatures)
from sklearn.naive_bayes import MultinomialNB

mnb = MultinomialNB()

from sklearn import metrics
from sklearn.cross_validation import cross_val_score
scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')

I get this error: bad input shape (64, 16)
This is the shape of label and input:
dataLabel.shape = 72
chk.shape = 72,16
Why won't the classifier accept the onehotencoded features?
EDIT: Adding how I got dataFeatures
dataFeatures = data[['Accpred', 'Gyrpred', 'Barpred', 'altpred']]
Y = dataLabel.ravel()

scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')

You have your Y and chk switched. That's it. :)

The signature of cross_val_score is sklearn.cross_validation.cross_val_score(estimator, X, y).

X is a matrix and y is a 1D vector with your class labels.

unlike in R, most (or all?) sklearn models do not support categorical variables. Most of the time, encoding your feature matrix X into what is called one-hot encoding is good enough.

Notice that, in some models, this hack is not the same as true native categorical support, and the performance of the model will be worse.

Invert One-Hot Encoding

Sklearn does not seem to have an easy method to invert the one-hot encoding.

It is not trivial how to do this. I found this suggestion:

def inverse(enc, out, shape):
    return np.array([enc.active_features_[col] for col in out.sorted_indices().indices]).reshape(shape) - enc.feature_indices_[:-1]

Example:

import numpy as np
from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder()
X = np.array([[0, 0, 3], [1, 1, 0], [0, 2, 1], [1, 0, 2]]))
Z = enc.fit_transform(X)
print(inverse(enc, Z, X.shape))
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]
print(X)
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]

Notice:

This only works when HotOneEncoding(sparse=True) (default) because it uses scipy sparse matrix methods (this could be changed by making the code only use numpy methods), but this is probably what you want since working with a dense matrix will kill your memory anyhow
I think this will only work if your variables are within the range [0,something] because you lose that information in the transformation (no work-around for this other than you using something like DictVectorizer which offers you more control over the transformation.


Which classification algorithm to choose for classifying driving patterns (GPS coordinates) and mapping them to drivers? I have been given 6 data sets out of which 5 are training sets. The 5 data sets correspond to 5 drivers.

Each row of a data set consists of a time-stamp followed by a poly-line. The poly-line gives the GPS coordinates (latitude,longitude) of the driver taken every 15th second. So the length of this line gives the time duration of each trip. I have to classify the poly-lines corresponding to each time-stamp to one of these 5 drivers in the last (test) data set.

I need assistance in choosing a classification algorithm. I have only implemented Naive Bayes (text classification) before but I don't think it will work here. I'm having trouble visualizing the approach to solve this problem.

I have done some previous work classifying vehicles (heavy or light) based on their driving behaviors. This required calculating speed and accelerations, which you can easily do by using numerical formulas such as the five-point stencil. You already know that points are separated by 0.25 seconds and the distances can be calculated using the haversine formula. More features can be derived such as the driving range, maximum and average speeds, number of left and right turns, hard breaks and sharp accelerations. Try plotting the polylines on Google Earth (or a similar mapping tool) to see if there is a distinctive geographical pattern (are they far apart from each other, or all in the same region?). If patterns are visible, a clustering algorithm may help.


WEKA Random Forest and numerical attributes I am working with Random Forests in Weka. I thought the ID3 algorithm is used to find the best split attribute at each level. But after reading a bit I noticed that ID3 can not handle numerical attributes (one needs for instance C4.5 for that). Random Forests in Weka do have no problems with numerical features as input...
How are they handeled?

(Small additional question: Is Weka still something you can work with today or do I need to switch the tool for current research? Weka seems a bit "old-fashioned" sometimes...)

Weka does use its own implementation of C4.5, named J48.

That said, its RandomForest calls on RandomTree, which appears to be implemented independently, not ever touching the J48 code.  (That's from what I can tell, and I don't normally work in Java so take it with a grain of salt.)  I can't tell from a glance whether there are substantial differences aside from the obvious (that RandomTree subsets features available to split on at each node).

Class hierarchy of module tree
tree folder of source
(click through to see J48, RandomForest, RandomTree, etc.)


Pixel labelled image is must for semantic segmentation using deep learning in matlab? I have large set of CT images containing lung regions. But I don't have corrosponding pixel labelled images. So how I could do semantic segmentation using DL in matlab. Like a test program I created 6 pixel labelled images using image Labeler. But it's not practical. I have thousands of such images. Could someone help me with a solution?

If you have the bounding boxes then you can use MaskRCNN and it will be able to give you those segmentations.
If you don't, look at unsupervised segmentation. Also, you need something as an output if you want to try out supervised learning.


Prepare data : many products per row I would like to find how to change my data structure to make it compatible with a ML model, with the following constraints :

A Product is defined by {id, price, continuousVariable1, ..., continuousVariable10}
The training set is train.csv. Each line contains a date, a Product, and a list of Products (competing products)
For each row of train.csv, the size of the list of Products may change.

It would be easy if I had only one competing products : I could put all columns in the same file. But I have many competing products.
The output variable is the product price.

I my opinion, the best way is if you  create a column for every(!) product. Yes, u will  get a high dimensional sparse Matrix but it should solve you problem quickly.


Change number format of confusion matrix I have the following confusion matrix:

I would like to change the format of the numbers that, when they exceed the value 99, appear in scientific format.
I would like them to appear in a standard format. That is: 3.3e + 02 would be 330.

This is the function I have implemented:

  cm= confusion_matrix(y_test,predicted_classes)
  plt.figure(figsize=fig_size)
  plt.title('Confusion matrix of the classifier')
  sns.heatmap(cm,annot=True)
  plt.xlabel('Predicted')
  plt.xticks(rotation=45)
  plt.ylabel('True')
  plt.ioff()
  plt.show()

Add fmt = ".1f" parameter in sns.heatmap().

You can rewrite your code as follows to get all numbers in scientific format.

sns.heatmap(cm,annot=True, fmt=".1f")

Refer this link for additional customization.


A fundamental question about the gradient descent equation So in a classic gradient descent we have

W -= epsilon * (dL / dW)

However, how does this make sense if we consider that L and W have some units? Wouldn't it be more sensible to consider instead

W -= epsilon * (dW / dL)    (where epsilon now represents a small nudge in loss and has corresponding units)

Note also that if we are not on a point with 0 derivative we can consider a "local inverse" of the function L(W) and we would have that dL/dW = 1/ (dW/dL). Therefore the signs of these two alternatives are equal - the only thing that differs is the size of the modification we apply to W.

In the standard way we take a larger step on a steeper region. "My" variant does the opposite. Imagine that a change of W results in a very small change of loss: then it would make sense to overcompensate for this during the update of W, in line with my proposed method. Where am I wrong?

You are looking for a local minimum, instead of a global minimum value.

The local minimum has $\frac{dL}{dW} = 0$, there is no direction of $W$ in which you can further reduce the loss.

If you are near the local minimum, you want to take small steps, not to 'overshoot'. If you are further away, you can take larger steps to speed up things, because you do have an idea about the direction of the minimum, namely the gradient.


What are temporal and local data? Based on the question, what are they and what are their differences? I've seen a lot those terms but cannot find an appropriate answer.

Temporal data is across time. Data points are correlated and you model them using a variant of RNNs (LSTMs, GRUs, Tree recursive NNs etc). example would be text script of speech. Notice how each data point would be correlated with the previous data point

local data is ambiguous and could mean a lot of things. local to a particular location? local data as opposed to server data? local to a particular task? etc etc. Please provide context


Machine learning algorithm that uses the Pearson or Spearman correlation? I've come across linear and multiple regression, SVM, random forests. Does any know of a machine learning algorithm that uses the Pearson correlation or Spearman correlation?
Best,
Dave

I do not think this exists. I suppose an algorithm could use pearson coefficients as starting coefficients, but honestly it seems like a waste of computational resources. Here are some reasons that occur to me as to why it is a bad idea:

Pearson and Spearman correlations become decreasingly meaningful as
the number of dimensions increase. I commonly work with millions of dimensions...Spearman correlations for individual features?
In sparse matrices, these coefficients mean next to nothing as there will be only a very slight correlations between a feature and the target. Usually, it is a multi dimensional relationship that we are trying to find (lots of caveats placed here :P)
Pearson and Spearman correlations assume certain parameters which are not usually true in ML applications ie homoscedasticity, linearity, normality, etc.

For the above and many other reasons imho it doesn't serve any purpose to use these anywhere in ML algorithms.


Does LSTM without delayed inputs work as a deep net? I want to predict a multivariate time series. My time series is $a_1(t),...,a_k(t)$ and I want to predict $a_k(t)$. I use the following keras LSTM:
model = Sequential()
model.add(LSTM(90,return_sequences=True,input_shape=(train_X.shape[1], train_X.shape[2])))
model.add(LSTM(90))
model.add(Dense(1))

I use $a_1(t),...,a_{k-1}(t)$ as input and $a_k(t)$ as output to train it. So I don't use delayed inputs like $a_s(t-l)$. My qestion is in this situation, Does LSTM work as a deep neural network? i.e. Is it same as the following keras net?
model.add(Dense(90, input_dim=12))
model.add(Dense(90))
model.add(Dense(1))

LSTMs are RNN with memory cell
It can be difficult to train standard RNNs to solve problems that require learning long-term temporal dependencies.
LSTM units include a 'memory cell' that can maintain information in memory for long periods of time. A set of gates is used to control when information enters the memory, when it's output, and when it's forgotten. This architecture lets them learn longer-term dependencies.
So without the delayed inputs a LSTM is simply a RNN.

As you can see here, RNN has a recurrent connection on the hidden state. This looping constraint ensures that sequential information is captured in the input data.
Check this for more details


Create a prediction formula from data input I have an algorithm which have as an input about 20-25 numbers. Then in every step it uses some of these numbers with a random function to calculate the local result which will lead to the final output of A, B or C.

Since every step has a random function, the formula is not deterministic. This means that with the same input, I could have either A, B or C.

My first thought was to take step by step the algorithm and calculating mathematically the probability of each output. However, it is really difficult due to the size of the core.

My next thought was to use machine learning with supervised algorithm. I can have as many labeled entries as I want.

So, I have the following questions:

How many labeled inputs should I need for a decent approach of the probabilities? Yes, I can have as many as I want, but it needs time to run the algorithm and I want to estimate the cost of the simulations to gather the labeled data.
Which technique do you suggest that works with so many inputs that can give the probability of the three possible outputs?
As an extra question, the algorithm run in 10 steps and there is a possibility that some of the inputs will change in one of the steps. My simple approach is to not include this option on the prediction formula, since I have to set different inputs for some of the steps. If I try the advanced methods, is there any other technique I could use?

I'm not sure if I understood your question! Probably better to plot a scheme at least. But according to what I guess from your question:

Q2- You probably need a simple MLP (Multilayer Perceptron)! it's a traditional architecture for Neural Networks where you have $n$ input neurons (here 20-25), one or more hidden layers with several neurons and 3 neurons as output layer. If you use a sigmoid activation function ranged from 0 to 1, the output for each class will be $P(Y=1|X=x)$.

Q1- So your question probably is: how many training data you need for learning a model? and to the best of my knowledge the answer is as many as possible!

and about the last question, I really could not figure out what you mean. You apparently have a very specific task so I suggest to share more insight for sake of clarification.

I hope I could help a little!


Association rules: Find the recipe for a list of ingredients Assume I have big database of recipes. For each recipe I have a list of ingredients.
Now I want to find all association rules in the form of (ingredient₁, ingriedient₂, …) → recipe.

Is the Apriori algorithm suitable for my problem? As far as I am able to understand the Apriori algorithm is intended to find rules like X → Y where X and Y are subsets of the same superset. But in my case, X and Y are subsets of completely disjoint supersets.
Is it possible to fit the Apriori algorithm for my problem? Are there other algorithms that suit my problem better? Or am I completely on the wrong track? I would be very grateful for any tips or hints.

It doesn't seem like an association rule problem. It seems that what you suggest could be simply solved using set operations.

First, create mappings between 1 ingredient and all recipes containing it. Then, if you want to find all possible recipes based on a list of ingredients, it would be identical to the union of all of their "recipe sets".


Why does storing the output of .count() result in all 1's or 0's? I have a dataframe which I run loc on to find all values within the loc parameters. Whatever the loc returns gets stored in a variable. I try to use this variable to store it in another dataframe column but it only returns 1's and 0's for the whole column.
How would I get what is being printed to be properly appended to a new dataframe?
I think the .count() has something to do with it, meaning it is changing the value of s4 and s5 when applied to the new dataframe.
for name in pros_unique:
    s4 = df.loc[(df.pros_split == name) & (df.orc_4 == 1)].orc_4.count()
    s5 = df.loc[(df.pros_split == name) & (df.orc_5 == 1)].orc_4.count()

    tots_df['orc_4_totals'] = s4
    tots_df['orc_5_totals'] = s5
    # This prints out in the correct format
    print(name, s4, s5)

Output of print statement:
name1 77 22
name2 62 32
name3 43 29
name4 1 1
name5 61 24

This is the tots_df dataframe:

pros    orc_4_totals    orc_5_totals
0   name1     1        0
1   name2     1        0
2   name3     1        0
3   name4     1        0
4   name5     1        0
```

Because on each loop you assign a single value to the entire column:
   tots_df['orc_4_totals'] = s4    # entire column orc_4_totals = s4!

(the same thing for the other column.
It equals all ones then, just because your final loop value fills it with 1.
You need to insert single values instead, for each row of your target dataframe.
However, because I don't know if your target dataframe tots_df has an existing index, I would suggest first saving the values in a list, then assigning them:
orc_4_values = []
orc_5_values = []
for name in pros_unique:
    # perform your checks, then assign using:
    s4 = df.loc[(df.pros_split == name) & (df.orc_4 == 1)].orc_4.count()
    s5 = df.loc[(df.pros_split == name) & (df.orc_5 == 1)].orc_4.count()
    orc_4_values.append(s4)
    orc_5_values.append(s5)

# Add to dataframe after the loop
tots_df['orc_4_totals'] = orc_4_values
tots_df['orc_5_totals'] = orc_5_values


Differences in scoring from PMML model on different platforms I've built a toy Random Forest model in R (using the German Credit dataset from the caret package), exported it in PMML 4.0 and deployed onto Hadoop, using the Cascading Pattern library.

I've run into an issue where Cascading Pattern scores the same data differently (in a binary classification problem) than the same model in R. Out of 200 observations, 2 are scored differently.

Why is this? Could it be due to a difference in the implementation of Random Forests?

The difference was, it appears, due to the different implementation of Random Forests in R and Cascading Pattern (as well as openscoring which I tried later) with respect to ties in the tree voting - i.e. when an even number of trees are built (say, 500) and exactly half classify an application as Good, and the other half as Bad, the handling of those situations differs. Solved it by growing and odd (501) number of trees.


Evaluating a IR system (Precision and Recall) I am studying by now IR system, in the field of valuation of IR system outputs related to a specific query but I need some help to understand it properly.

My book states that when an IR system has to be evaluated, we need a test document collection, a set of query examples, a valuation (relevant or not) for each couple of query/document, defined by experts in the field. So we need two measures to know quantitatively if a IR system is good: Precision and Recall.

My doubt is related to the following question: Do we use those two measures only if we are testing a IR system or not?

I'll explain: before we calculate Precision and Recall related to a specific query example (see above), we need to know how many elements belong to the relevant set, which is impossible if there isn't a valuation (relevant or not) for the query we are using. My book says we can increase Recall in a search engine by using the relevance feedback technique (query expansion and term reweighting): in this case do we assume the Recall value is unknown?

For example, everyday many documents are shared on the Internet and Google can find them. So it is impossibile to apply Recall and Precision to this scenario, in which information grows and there is no valuation for every new document for each specific query. It is also impossibile to predict all the possibile queries a user can do on a search engine.

My doubt is related to the following question: Do we use those two measures only if we are testing a IR system or not?

Technically the answer is no because precision and recall are used to evaluate not only IR systems but also many other tasks. However your question seems to be specific to IR so I'll assume that it's actually about the distinction between testing and evaluation:

Testing a ML system consists in predicting the target variable for a set of instances given as input (in the case of supervised learning a "model" obtained from a previous stage of training is required as input as well). At this stage we don't know whether the predictions are correct or not.
Evaluation is the process of assessing the quality of the predictions: it's done after obtaining the predictions from the testing stage, and it requires some form of "gold standard", i.e. data which says what is the correct answer for every instance.

In IR, the testing stage happens every time the system is run to find relevant documents based on a query.

Naturally one wants at first to make sure that the system works properly and returns actually relevant documents, so the system needs to be evaluated, for instance with precision and recall using a dataset containing some queries and their relevant documents (gold standard).
Once the quality is evaluated, the goal is to use the IR system (testing) without evaluating the results every time. Of course there's no evaluation so the performance measures (precision and recall) are not used.


What do positive and negative gradient values mean for Convolutional Neural Network? As we have the typicall pass of the neural network we make a forawrd pass to predict classes and then we have cost function and based on that we calculate gradients.

I'm wondering what are the meaning of the positive and negatives values of gradient. The specific question that i want to know answer about is why the positive values of gradient maps mean that object was detected there?

Or posting this question with example. Why here the highlighted gradients maps have parts of dog/bird/spider highlighted?

If I m confusing something correct me.

As you know, the purpose of gradient maps like these is to show which pixels are contributing most to the prediction.

  The specific question . . . is why the positive values of gradient maps mean that object was detected there?

It's not positive/negative values that matter, but rather the magnitude of the gradient.  If the gradients associated with a pixel are small in magnitude, then that pixel is likely to be unimportant to the prediction (since small changes in the pixel's value translate to small changes in network outputs).  By contrast, if the magnitudes of the gradients are large, then it signifies that the pixel is important to the prediction.

  Why here the highlighted gradients maps have parts of dog/bird/spider highlighted?

It looks like the authors of the paper are post-processing the gradient map with a function they call $\psi$.  Notice the following passages from section 4.2 (highlighting was added by me)

and

The authors are taking the absolute value of the gradient.

So the gradient maps from this paper only show the magnitude of the gradient, not the sign.  When you see a part of the image highlighted, it doesn't mean that the gradients were positive.  Rather it means that the magnitude of the gradients were large.  Blueish regions correspond to near-zero gradients (low importance), while the orange-red regions correspond to large gradients (high importance).

Hope that helps!