Interpretation of tuning parameters (shrinkage and nrounds) in XGBoost I'm using XGBoost for a multiclass classification problem in R. I'm trying different combinations of the shrinkage parameter and number of iterations to try and settle on optimal values for these parameters (using cross-validation). However, I'm having trouble with exactly interpreting the significance of the parameters. The XGBoost algorithm isn't completely a black box for me - I know how boosting works at a high-level.

The questions I have though are as follows:

Does a low shrinkage (e.g. 0.01) and high nrounds (e.g. 500) yield the same results as a high shrinkage (e.g. 0.3) and low nrounds (e.g. 50)?
Does nrounds correspond to the number of weak classifiers being fit?
The xgb.cv function in R returns test mean errors and standard deviations for each of the iterations. For the nth iteration, is this the mean over all iterations less than equal to n over all the folds? Apologies if I'm interpreting this incorrectly. Also, what is the significance of the standard deviation here?
Is there some blog post/link I can refer to for a comprehensive guide on tuning XGBoost parameters?

From my understanding (and I am no expert!)

Point 1

Let $ \hat f_k $ be a weak learner or one tree in the model, then we would make a prediction as the $$\hat y_i  = \sum_{i=0}^k \hat f_k(x) $$ with $ (x) $ being a particular feature The algorithm learns in an additive way so $$f_0(x) + f_1(x) + f_2(x) .... f_k(x) = \hat y_i $$ So adding a shinkage parameter $ \delta $ will penalise the model for each additional tree (or weak learner) you add to it, which looks like $ \delta f_k $. So having a low shinkage $ \delta $ would penalise the overall prediction score quite a lot.

Having a shrinkage score of 0.3 vs 0.01 with some $ (x) $ variable scores from each tree $f_k$;

Model 1: $ 0.3(2.1) + 0.3(2.9) + 0.3(3.4) + 0.3(3.5) = 3.57$

Model 2: $ 0.01(2.1) + 0.01(2.9) + 0.01(3.4) + 0.01(3.5) = 0.119 $

After four trees, so according to my understanding Model 2 is going to need many more trees to get to the same result as Model 1. However having a low number of rounds will likely miss important structural information in the model. I understand it this way.

Shrinkage extremely high and low, 1 and 0.01

Model 1x: $ 1(2.1) + 1(2.9) + 1(3.4) + 1(3.5) = 11.9 $

Model 2x: $ 0.01(2.1) + 0.01(2.9) + 0.01(3.4) + 0.01(3.5) = 0.119 $

The difference here being that Model 2x can capture 3 decimal places in its increments, whereas Model 1x goes up in increments of 0.1. So if we choose Model 1x then its possible to overshoot the optimal number of trees. If we were to increase the number of trees for Model 2x to 500 or 1000 then its possible that we would overfit the data, alternatively if we were to reduce the number of trees for Model 1x we would underfit the data. So this is where the cross validation and search for optimal parameters comes in.

So to answer point 1. Theoretically I understand yes, the numbers will converge, however this would lead to potential overfitting issues and a balance is necessary.

Point 2) nrounds to me, means the number of additional trees or weak learners added to the model.

At $ nround = 0 $ we have;

$$ \hat y_i^0 = 0$$

at $nround = 1$ we have;

$$ \hat y_i^1 = \hat y_t^0 + \delta f_1(x)$$

so $ \hat y_i $ adds the previous weak learners to its new prediction

then $ nround = 2 $ we have

$$ \hat y_i^2 = \hat y_i^1 + \delta f_2(x) $$

So at each $nround$ a new weak learner is added to the prediction.

Point 3) Here for me the mean evaluation metric is the mean score over k-fold cross validation. If you set k-folds to 10 then it would run the model 10 times and produce just the mean of these scores, the same with standard deviation scores.

Point 4) Theres many places to read about xgboost tuning, I have visited many of these websites countless times here

One really cool piece of code I am using from here. Although my code now has expanded this for most of the parameters of XGBoost and for an AUC evaluation metric not RMSE. I can post it if you are using AUC for evaluation.

I am still no expert but I am trying to answer some of the questions that you have myself, so I post my understanding of what I know, sure somebody will correct me :)

searchGridSubCol <- expand.grid(subsample = c(0.5, 0.75, 1),
                                colsample_bytree = c(0.6, 0.8, 1))
ntrees <- 100

#Build a xgb.DMatrix object
DMMatrixTrain <- xgb.DMatrix(data = yourMatrix, label = yourTarget)

rmseErrorsHyperparameters <- apply(searchGridSubCol, 1, function(parameterList){

    #Extract Parameters to test
    currentSubsampleRate <- parameterList[["subsample"]]
    currentColsampleRate <- parameterList[["colsample_bytree"]]

    xgboostModelCV <- xgb.cv(data =  DMMatrixTrain, nrounds = ntrees, nfold = 5, showsd = TRUE,
                           metrics = "rmse", verbose = TRUE, "eval_metric" = "rmse",
                           "objective" = "reg:linear", "max.depth" = 15, "eta" = 2/ntrees,
                           "subsample" = currentSubsampleRate, "colsample_bytree" = currentColsampleRate)

    xvalidationScores <- as.data.frame(xgboostModelCV)
    #Save rmse of the last iteration
    rmse <- tail(xvalidationScores$test.rmse.mean, 1)

    return(c(rmse, currentSubsampleRate, currentColsampleRate))

})


Predicting interactions First off, I don't really know much about machine learning.

In a virtual world, such as a video game like minecraft or an application like Google Street view, a user can navigate the world using the keyboard and mouse.

I'm looking to predict what interaction the user will perform. For example, the user might hold down w for some period of time (to move forward) and then press and hold a for a period of time to move left. Alternatively, the user may hold w and use the mouse to turn and look up/down.

I've looked into N-Grams, and I can predict interaction to a certain degree, but I am not considering how long a user is not performing an action. I think performing no action is an important indicator for correct interaction prediction as "no action" could itself be classified as an interaction. The duration of an interaction should also be important, but again, my not sure how to consider that in a model. My Ngram model simply considers the past 3 or 5 inputs.

I'd like to develop something which is very fast to predict interaction  (this is key) and learns rapidly from mistakes. Ideally, it would be "Online" in that it constantly "Learns".

I'd be interested in hearing which models you think would be well suited to my task (are ngrams appropriate?), any examples which closely match my task and I can therefore adopt to suit my needs, any datasets containing user interaction and what I should look at to predict the mouse interaction.

Interesting question.  I haven't worked with this sort of data much, but it seems to me that the bulk of the job is likely to be feature engineering.  Every "supervised" statistical method that I know of requires that you shoe-horn your data into "outcomes" and "covariates."  $\mathbf{Y}$ and $\mathbf{X}$.  Once your data is in this form you simply find an appropriate algorithm that can estimate

$$
\mathbf{Y} = f(\mathbf{X})
$$

But taking data on how people play a video game and turning it into a 2-dimensional matrix isn't necessarily obvious.  One of the most difficult things is that you'll need to make observations of different users comparable.

Say you're playing quake 3 and you're trying to predict whether the user is going to go for the BFG.  How do you define $\mathbf{Y}$?  Is it got BFG? Or should it be came within some distance of BFG?  (maybe people messed up the rocket jump, for example).  What if they got fragged on the way to the BFG?  The point is that defining the outcome involves choices.

Likewise what are the covariates?  Say I get spawned in some location, and I know the map, and immediately move toward the BFG.  I'll have a different set of w, a, s, and d keystrokes than someone who got spawned elsewhere on the map.  So is it n-grams that you want to look at?  Maybe rather it is change in euclidian distance between spawning location and BFG location over some time interval.  if so, what is the appropriate time interval?

In general, what you're doing is taking raw data and turning it into abstractions of that data.  This is actually what a neural net does.  It takes raw data $\mathbf{X}$ and forms a lower-dimensional representation of that data, $\mathbf{V}$, where $\mathbf{V}$ has fewer columns than $\mathbf{X}$.  When neural nets work well, those columns of $\mathbf{V}$ can be uncannily similar to things that a human would pick out of a stream of raw data.  Akin to this:  https://www.youtube.com/watch?v=3vAnuBtyEYE

Those $\mathbf{V}$ are then just related to your outcome $\mathbf{Y}$ in a linear model or a logistic regression.

But you don't turn raw keystrokes into blonde, brunette and redhead without a lot of data and long training times.  It'll go faster if you can prespecify functions of your raw data, which requires domain knowledge.


spikes in the cost vs iteration graph using ReLU activation function The cost vs iteration graph while using the ReLU activation function has number of spikes

The accuracy on MNIST dataset (both train and test) is around 95%, also on using sigmoid as an activation function I get a smooth downward sloping curve so I think the implementation is correct. Are these spikes expected for ReLU? How would you explain this property?

When I run my implementation of MNIST digit recognition I even get those spikes for a sigmoid transfer function. You have to question yourself: how bad are those spikes? I presume you use stochastic gradient descent right. For every iteration you consider a batch (a subspace of the samples) and train on that. Some of those batches contain very difficult pictures and your network will fail at them, resulting in a high cost. This in turn will lead to a temporary high adaptations of your weights and biases. You see that it drops again quite quickly because the other samples will again send it in the right direction.

I think you have more spikes because your ReLU is just better at generalization and thus after a while you will get more peaks (but lower) because the network gets better and better. And in the end the most important criteria is your accuracy and not the cost ;). This has less problems with the spikes because it takes into account all the samples instead of a batch.

Concluding: stochastic gradient descent will go to a low point, but because of it random walk it might bump into some peaks, but in the end it will find a nice low cost.

If you still have some questions, feel free to ask, I am still quite new here and still have to learn how to give good answers.


How exactly do I go about extracting features from timestamps for machine learning? My dataset has a timestamp column with the following format: 06/24/18 0:56
How exactly do I convert this information into features that can be used for classification algorithms like logistic regression?

A common approach for time-series classification problems is to divide the continuous stream of data into samples of a certain duration.

This is called sliding window segmentation.

You don't really use timestamps as features because they wouldn't be useful during the classification of unseen data. Imagine training a model with data obtained in 2018, and trying to classify data for 2019. The information is not on the dates but in the values of the other features!


What is AUC - ROC Curve? AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability.

Is Roc the same as AUC?

Related concepts, but not the same.

ROC-receiver operating curve

AUC area under the curve

Thank this post for explanation :

Abbreviations

AUC = Area Under the Curve.
AUROC = Area Under the Receiver Operating Characteristic curve.

AUC is used most of the time to mean AUROC, which is a bad practice since as Marc Claesen pointed out AUC is ambiguous (could be any curve) while AUROC is not.

Interpreting the AUROC

The AUROC has several equivalent interpretations:

The expectation that a uniformly drawn random positive is ranked before a uniformly drawn random negative.
The expected proportion of positives ranked before a uniformly drawn random negative.
The expected true positive rate if the ranking is split just before a uniformly drawn random negative.
The expected proportion of negatives ranked after a uniformly drawn random positive.
The expected false positive rate if the ranking is split just after a uniformly drawn random positive.

Going further: How to derive the probabilistic interpretation of the AUROC?

Computing the AUROC

Assume we have a probabilistic, binary classifier such as logistic regression.

Before presenting the ROC curve (= Receiver Operating Characteristic curve), the concept of confusion matrix must be understood. When we make a binary prediction, there can be 4 types of outcomes:

We predict 0 while the true class is actually 0: this is called a True Negative, i.e. we correctly predict that the class is negative (0). For example, an antivirus did not detect a harmless file as a virus .
We predict 0 while the true class is actually 1: this is called a False Negative, i.e. we incorrectly predict that the class is negative (0). For example, an antivirus failed to detect a virus.
We predict 1 while the true class is actually 0: this is called a False Positive, i.e. we incorrectly predict that the class is positive (1). For example, an antivirus considered a harmless file to be a virus.
We predict 1 while the true class is actually 1: this is called a True Positive, i.e. we correctly predict that the class is positive (1). For example, an antivirus rightfully detected a virus.

To get the confusion matrix, we go over all the predictions made by the model, and count how many times each of those 4 types of outcomes occur:

In this example of a confusion matrix, among the 50 data points that are classified, 45 are correctly classified and the 5 are misclassified.

Since to compare two different models it is often more convenient to have a single metric rather than several ones, we compute two metrics from the confusion matrix, which we will later combine into one:

True positive rate (TPR), aka. sensitivity, hit rate, and recall, which is defined as $ \frac{TP}{TP+FN}$. Intuitively this metric corresponds to the proportion of positive data points that are correctly considered as positive, with respect to all positive data points. In other words, the higher TPR, the fewer positive data points we will miss.
False positive rate (FPR), aka. fall-out, which is defined as $ \frac{FP}{FP+TN}$. Intuitively this metric corresponds to the proportion of negative data points that are mistakenly considered as positive, with respect to all negative data points.  In other words, the higher FPR, the more negative data points will be missclassified.

To combine the FPR and the TPR into one single metric, we first compute the two former metrics with many different threshold (for example $0.00; 0.01, 0.02, \dots, 1.00$) for the logistic regression, then plot them on a single graph, with the FPR values on the abscissa and the TPR values on the ordinate. The resulting curve is called ROC curve, and the metric we consider is the AUC of this curve, which we call AUROC.

The following figure shows the AUROC graphically:

In this figure, the blue area corresponds to the Area Under the curve of the Receiver Operating Characteristic (AUROC). The dashed line in the diagonal we present the ROC curve of a random predictor: it has an AUROC of 0.5. The random predictor is commonly used as a baseline to see whether the model is useful.

If you want to get some first-hand experience:

Python: http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
MATLAB: http://www.mathworks.com/help/stats/perfcurve.html


How to combine two different random forest models into one in R? I have two different dataset with same variables. I have built training models separately using random forest. Now i want to combine both these models. Could anyone tell me how can i this be achieved? Do we have something called combine() function in R?

Regards,
Arun

This question has already been answered here. It addresses the problem in hand.


dividing Mean by standard Deviation meaning I have played around with logistic regression a little using movement data intervals that are prelabeled as either resting or active.
I now found that if I divide the mean movement of the individual intervals by the intervals standard deviation, the outcome is quite a good predictor of whether the interval is a resting interval or or active, with an average auc = 0.93 in a 20 fold cross validation.
Does someone have an idea of what I have created dividing the mean by the standard deviation? Its like a flipped coefficient of variation, or semi-normalization?!.
I want to report this in an essay, so I am asking myself whether there is a name to this statistic.

You have created what can be called a normalised mean. Normalised in the sense that it corresponds to a related random variable which now has $\sigma = 1$ (since you divide by standard deviation, look it up in any statistics textbook).
The normalised mean can be a better predictor resulting from its normalisation which makes it robust to the range of fluctuations.


How to plot multiple models val and traing acc/loss curve from csv files? I trained multiple CNN models, after that, I saved models details (Like , training/Validation Acc/Loss )  by callbacks by using this codes :
tf.keras.callbacks.CSVLogger

Now I have multiple models training-val acc-loss respective values in different csv files.
I want to plots those in one figure from my csv files. How can I do this?

Here are the steps I solve my problem:

Save Statistical records (Training-Validation Accuraciers, Training-Validation Recall, Training-Validation Precisions, Training-Validation F1) from CSV files of this following code tf.keras.callbacks.CSVLogger after training each fold (I trained for the 3 Folds).

And load those csv files in the list:
    import pandas as pd
    l = ["/Records_of_Fold_2.csv" , "/Records_of_Fold_3.csv"]

After that, Plot using matplotlib :
for (k,i) in enumerate(l):
data = pd.read_csv(i)
plt.figure(str(k))
plt.xlabel("x"+str(k))
plt.ylabel("y"+str(k))
plt.plot(data["epoch"],data["f1_score"])
plt.plot(data["epoch"],data["loss"])
print(k)
plt.show()


Can a Conditional GAN be used for a regression task, where the condition can take any continuous value? I came across conditional GAN, where the classes can be provided as input to the GAN. I would like to know, if the same network can be used for classes which can take any continuous value.ie, the generator once trained should be able to produce output for any condition, even if it was not a part of training set.

I think it exists as it is mentionned in the article Intra-class Variation Isolation in Conditional GANs from Richard T. Marriott, Sami Romdhani et Liming Chen in 5.2

Sadly I found nothing about those CC-GANs, eventhough it is said "classical"!


Predict Customer Next Purchase with Sequence Suppose I buy products: [1,2,3,4]

Another customer X bought: [2,3]

Most probably customer X next purchase will be: 4

Sequence is very important in my problem

I tried association analysis using R, but it don't take under consideration the sequence

Please advise what algorithms I need to solve this?

Do I need to First do Clustering to find similar customers?

For sequence problems, I generally used Recurrent Neural Network. It has a property to learn its values, which is based upon the previous state and current input.
Since in your case sequence is important, you can use RNN. You can also try LSTM(type of RNN) cell.

  And Additional Note: For product suggestion(where sequence is not
  important) you can also use Apriori algorithm. This algorithm try to
  build association between the products of a single order.


Boostrap parameter in random forest regressor? There's one parameter in RandomForestRegressor which is bootstrap. By default
bootstrap=True

  bootstrap : boolean, optional (default=True)

  Whether bootstrap samples are used when building trees.

So from the docs if I set bootstrap=False then I guess bootstrap samples are not used but I'm really confused on what is bootstrap samples mean here?

There were explanations but it's really confusing. Can someone please explain it in a simpler term? And also does bootstrap=True help in improving the model accuracy?

Thank you.

Refer this answer on stack

  The benefit of random forests comes from its creating a large variety of trees by sampling both observations and features. Bootstrap = False is telling it to sample observations with or without replacement - it should still sample when it's False, just without replacement... You tell it what share of features you want to sample by setting max_features..


Is there a site like fast.ai or deeplearning.ai but for machine learning instead of deep learning? I'm looking for a summary of the pros and cons of different machine learning models in practise.

Something that includes:
- How the model works
- What kinds of outliers will be misrepresented by the model
- Effect of different parameters in each model on speed/performance

Can't comment due to low reputation, but as a comprehensive summary I'd recommend Elements of Statistical Learning by Hastie et. al. It reviews most current state-of-the-art ML methods, together with the classics, is free of charge (big thanks to the authors!), and even has a nice accompanying R package.

It's probably not the source you've expected, but is really a nice place to start, especially for practitioners.


What is the point of using MissingIndicator in Scikit-learn? I have recently discovered Sklearn's MissingIndicator but still wondering how could it improve the usual machine learning work. Clear that

from sklearn.impute import MissingIndicator
indicator = MissingIndicator()
X_mask = indicator.fit_transform(X)

could result in  X_mask as boolean indicator array of missing values in original X. But how could it be utilized actually beyond that it keeps track where imputed values are? I know that flagging NAs in a separate column per features could be useful for learners but here X_mask  is not combined with the imputed X so this is seemingly not the point.

Similar question arises with putting MissingIndicator in the ML pipeline:

transformer = FeatureUnion(
     transformer_list=[
         ('features', SimpleImputer(strategy='mean')),
         ('indicators', MissingIndicator())])

What is the point to add the pure mask array to the imputed array here?

Good question. Here is my take on it:

The following piece code creates the union of both the imputed features and the indication of which ones were imputed.

transformer = FeatureUnion(
     transformer_list=[
         ('features', SimpleImputer(strategy='mean')),
         ('indicators', MissingIndicator())])

I can see it becoming useful for a learner that may learn to not rely as much on features that are often missing. Or perhaps, the fact that some features are missing at all can be helpful for the learner. As, missing values can mean that something happened differently in the data collection, which can be useful to make a prediction.

Let's say I have the following problem:

I want to classify users based on their watching habits on my fancy streaming platform. Users have to rate a movie thumbs up or thumbs down. Of course, not all my users will watch all movies, so I'll have blank values. By using let's say the SimpleImputer, I'll basically make an educated guess on how they would have liked the movies they didn't watch. That can be useful as then I can compare users across all movies. However, knowing which movies the user has watched altogether is probably as informative. This is what the MissingIndicator provides you.


What is a YAML file and where is it used in a machine learning context? I am not entirely sure if this is on-topic here, so please let me know if it is not. I keep seeing the idea of YAML files pop up while reading machine learning literature. My question is, what exactly is a YAML file, and how does it relate to machine learning and data science projects?

There are numerous ways to serialise data. For instance, JSON, XML, and CSV are possible approaches for serialisation. Another way is YAML. It is human-readable which means it contains readable text which is written using usual words. This stands against binary serialisation. Whenever you open the file, you will not be able to see what is written, because it is serialised in binary format and not as a human-readable format. As you may find it here, it is used for configuration files mostly. In ML context depending on your task, you may face different file format. For instance, one of the widely used formats for supervised ML tasks is CSV though you may find others use their favorite encodings. You may also want to make your own serialisation too which is possible among in-company operations is rare.


GAN's for data augmentation I am working to augment my data using Generative Adversarial Networks, I have used Deep Convolutional GAN's for this purpose but they are not learning the right data distribution, so please suggest me which GAN's to use for this purpose ?

So there are few possible answers to your question.
First of all you are mentioning Deep Convolutional GAN's which refers to Architecture of a Network. So you can either change Architectures or change Loss functions which are used in GANs which both can have an positive or negativ effect.
Also you mention not learning the right data distribution, it also cann occur that your data is the problem, i.e. too low number of samples, not representative samples etc.

There is no definite answer to your question, you have to try which works best for your dataset.

But a few hints for losses you can try: Wassersteindistance, Improved WassersteinGAN, Boundary Equilibrium GAN. In all of their publications also the architectures are published.


Does k fold cross validation become less useful when number of observations is very large? As seen in the accepted answer for variance of k-fold cross validation ,
the simulation shows that k-fold CV has the same test error rate for different values of k when n=200.
Does this mean that k-fold validation is likely to be as good as having a holdout set validation? (assuming I have abundant data to make up for the high bias for holdout set validation approach)

Apart from high bias, the problem with holdout set validation approach as described in ISL book is that the test error rate is sensitive to the random splitting of data between train and validate. My intuition is that, With very high n (and well spread out data), the problem due to random splitting seems less likely to occur.

Yes, you are right that when number of observations are very large, k fold cross validation (CV) are less useful. Let's look at why this is so:

1) Very high number of observations imply high training time for model and validation. Already the number of observations is large for the model to be trained and validated and now we are demanding it to be done k times. This is a huge burden on resources that is why in the Deep Learning regime we don't generally follow k fold CV as the data needed for training good neural networks are very high compared to traditional ML algorithms.

2) The higher the number of observations, the higher is the number of data chosen for cross validation set. This inherently makes it less likely that the sampled data points do not represent the original distribution. As you would know, the more data we sample, the better we approximate the original distribution.

Because of these reasons, k fold CV is inefficient when the number of observations is very high so a hold out set for CV will do the job.


Filling created feature with values I'm trying to improve accuracy. I created a few new features based on old features. So I need to fill new feature's empty cell with same values in order to equaling shapes.Then, I tried it with median and also with mean but nothing changed. Is there any method that I can apply these cells in order to improve accuracy?

for example, there is a feature named age
I created new 2 feature named age_1 and age_2 ,
age1 consisting of age < 55 ,
age2 consisting of age > 55

therefore there is some empty cells in the new features and I have to fill these

If I understood your question right -
you've created a new feature age<55 - which gets True/False, hence, there is no need for the second feature since it has a perfect (negative) correlation with the first one (i.e False in every record that the other new feature got True..).

2 last notes:

Note that the second feature ain't addeing any new information to the first one.
Note that you've missed the recodes with age==55

I would suggest:

df['is_old'] = df['age'].apply(lambda x: x>55)


Why does reducing polynomial regression to linear regression work? Getting into machine learning, have a reasonable background in statistics and understand the basic principles of linear algebra (matrix multiplication etc.) - but am having a damn hard time figuring out why reducing a polynomial regression works.

For example, say we have this function:

$y =$ $\beta_0$ + $\beta_1$$x_1$ + $\beta_2$$x_2$$^2$

From what I've seen on 4 videos and 6 articles, we can use the following substitutions:

$x_2$ = 1
$x_3$ = $x$
$x_4$ = $x^2$

To create the following model:
$y =$ $\beta_0$ + $\beta_1$$x_1$ + ($\beta_2$$x_2$ + $\beta_3$$x_3$ + $\beta_4$$x_4$)

And then, fine - we can solve that as a normal multiple linear regression, and all is well.

But why, why does this work? I really cannot find an explanation for this.

Thanks!

I don't really understand your problem statement. You can simply add squared terms to any linear model. Say you have $y$ and $x$ and you want to model a polynomial function, you can write a model like:

$y=\beta_0 + \beta_1 x_1 + \beta_2 x_1^2 + u$.

In matrix form this would look like

$y=X\beta + u$.

An example with some numbers would write:

$\left( \begin{array}{rrrr}
5 \\
8 \\
7 \\
6 \\
\end{array}\right) $ =
$\left( \begin{array}{rrrr}
1 & 8& 8^2  \\
1 & 4 & 4^2  \\
1 & 3 & 3^2  \\
1 & 6 & 6^2  \\
\end{array}\right) \beta + u.$

You can solve this like $\hat{\beta} = (X'X)^{-1} X'y$, where $\hat{\beta}$ are the coefficients of the linear regression model.


How to predict tf-Idf vector with Neural Network? I designed a neural network in Torch for prediction of articles. I want to predict, which article will be read by user based on your reading history.

Input to the network is tf-idf vector of article and another information about user.
Output from the network should be tf-idf vector of next article, which user read.

Tf-idf vector is normalized by SoftMax function.

I know the best solution would be, if a output was a representation of classes but there are a lot of articles and by that there will be a lot of classes. So I represent articles with tf-idf vector on the output.

I tried train with DistKLDivCriterion ( Kullback–Leibler divergence ) but it didn't converge. How to design a network for prediction tf-idf vectors? Which criterion should I use?

tmodel = nn.Sequential()
tmodel:add(nn.Sequencer(nn.FastLSTM(537, 300, rho )))
tmodel:add(nn.Sequencer(nn.Dropout(0,5)))
tmodel:add(nn.Sequencer(nn.Linear(300,527)))
tmodel:add(nn.Sequencer(nn.LogSoftMax(527)))

criterion = nn.SequencerCriterion(nn.DistKLDivCriterion())

I hope you do understand, that trying to predict some abstract TF-IDF vector based on read article you are trying to teach a neural network to produce a new article for a given user. It would be a major step in artificial intelligence if achieved, so keep us posted on your progress.

However, if you would think of alternative approaches, I would suggest following steps to make your neural network to work:

You need a classification/clustering model for your articles (which can be based on your TF*IDF, or on some tags assigned to articles). The more classes you could squeeze from your model, the better would your resulting recommender be. On that step, you should be at least able to provide user with articles similar by topic.
You will need at least two-hop data set of user reading current article and hopping to the next one which would be used as a training set for actual predictor. Here, you will train a neural network (or any other model) which will try to predict a next-article class (from step one) based on current article - that would be a recommender system.

Of course, you may, given you have data from (2), train a neural network that would answer the question of "If user reads article A, will the next article contain word B", but that would require you to have the size of output variables of your TF*IDF matrix (for any reasonable dataset that would be at least 20-30k words unless you work with books for toddlers) which makes your model overcomplicated (that would be, actually, a TF*IDF matrix size number of models for every word).


Computation of a column-stochastic matrix with target row sums I want to generate an $N\times N$ matrix $A$ so as to target an $N$ vector of row sums and simultaneously all column sums should sum to 1. In addition to this, I have a prefixed number of elements which are set to zero.
For example, beginning with:
$$
  \left[\begin{array}{rrr}
    0 & 1 & 1 \\
    1 & 0 & 0 \\
    1 & 1 & 0
  \end{array}\right]
$$
and the row sum vector $[1.5, 0.25, 1]^{T}$, I want to end up with
$$
  \left[\begin{array}{rrr}
    0 & a_{12} & a_{13} \\
    a_{21} & 0 & 0 \\
    a_{31} & a_{32} & 0
  \end{array}\right]
$$
under the following conditions:

$a_{12} + a_{13}  = 1.5$

$ a_{21} = 0.25$

$a_{31}+a_{32} = 1$

$a_{21}+a_{31} = 1$

$a_{12}+a_{32} = 1$

$a_{13} = 1$

While this is simplistic, in general, I have $2N$ equations in $N^{2}-Z$ unknowns, where $Z$ is the number of elements fixed to zero. So, this system of equations could be overdetermined or underdetermined, but I would like to be able to generate matrices like this such that all nonzero elements lie in $(0,1]$.

This problem, being more of a combinatorial rather than statistical nature, can be though of as a max-flow problem in a bipartite network. Each column of your matrix corresponds to a "source node" in the left part of the bipartite graph $BP(L, R)$; each row of the matrix corresponds to a "destination node" in the right part $R$. Capacity of each source node is the corresponding column's sum; similarly, capacities are defined for the destinations via row sums. Value $a_{ij}$ in the matrix describes flow of mass from $j$'th source to $i$'th destination along an existing edge $(j \to i)$ of $BP$. Your initially given 0-1 matrix describes which edges are present (e.g., the matrix in your example prescribes the edge $(3, 2)$ to be absent from the network, along with all self-loops). If you denote $F$ to be the sum of all column sums (or, alternatively, the total capacity of all sources), then your problem narrows down to finding a max-flow from $L$ to $R$ in $BP$ and checking whether the volume of the discovered flow equals $F$.

Note 1 (Feasibility): Before plunging into finding best flows, you may want to check your row sums: a necessary condition for your problem to have a solution is that the sum of all column sums (which is $N$ in your particular case of a column-stochastic matrix) should be equal the sum of row sums, as both numbers describe the same -- the sum of all elements of the target matrix. (For example, for $N = 3$ and the vector of row sums from your example, the problem is unfeasible.)

Note 2 (Upper bound): Since $a_{ij}$ describes the flow from $j$'th source, and the capacities of all sources are 1 in your setting, then each source cannot send more than 1 unit of mass to a single destination. Thus, each $a_{ij}$ will not exceed 1.

Note 3 (Lower bound): To make $a_{ij}$ for each existing edge $(j \to i)$ strictly positive, you can augment the max-flow problem with lower bounds on edge capacities. In a general setting, edge capacity bounds are $0$ and $+\infty$. You may want to replace the lower bound with some positive number, small enough not to interfere with the discovery of the best flow and large enough to satisfy your edge saturation needs.


What can functional programming be used for in data science? In my next academic year at university, I have the option to take a course in Advanced Functional Programming. A basic description of the course is this:

"You’ll focus on a number of more advanced functional programming topics such as: programming with effects; reasoning about programs; control flow; advanced libraries; improving efficiency; type systems; and functional pearls."

Therefore I'd like to know if functional programming is useful in Data Science. If so, why it is useful, and by extension, whether this course will ultimately be useful in the pursuit of becoming a Data Scientist.

One reason why functional programming could be useful for data science is that it lends itself more easily to parallel and distributed programming, e.g. the popular frameworks Apache Spark for cluster computing and Apache Kafka for stream-processing are both written in Scala (and Java).

Other than that "functional programming" as a skill is not directly related to data science. It's a tool that may facilitate some practicalities of data science and therefore more relevant for the "data engineering" aspect of data science.

It's useful but probably not necessary. It depends on your interests.


ML Algorithm for anomaly detection in paired time-series I have many (around 40) separate time-series from different sensors, each measuring magnetic field intensity. I am looking to get an ML algorithm to identify a particular anomaly. This anomaly happens infrequently (a handful of times in 25 years of 60-second data, depending on the sensor) and is only apparent when you look at two time-series side-by-side:

The yellow circle is the anomaly, and can be described roughly like this: when the time series are particularly disturbed, the red time-series will briefly anti-correlate with the green time-series over a period of a few hours (data are sampled every 60s). This can easily be done by hand for a pair of time-series.

The Plan

I'd like to train a machine learning algorithm by manually identifying these anomalies for a sensor pair, and then set the trained algorithm on the rest of the pairs of time-series

The Problem

I have no idea what the best algorithm would be for this problem, as it involves two time-series. I messed around with gathering information in 6 hour blocks and treating it like a non-time series problem. This involved treating each 6-hour block separately (i.e., each with std, average, max, min of each time-series as well as a correlation coeff and a 1 or 0 output determined manually), but I feel like there is a neural network that will be able to better figure out the important variables.

The Question

What would be the best approach to this problem in order to train a ML algo? Is my approach above suitable?

Before think which algorithm, think what features. You may add/ignore features that imply your problem fluently.

This will be around quantifying anomalies (new variable?). Try to add a residual variable? Or simply feature different between red and green variables?


Variance in cross validation score / model selection Between cross-validation runs of a xgboost classification model, I gather different validation scores. This is normal, the Train/validation split and model state are different each time.

flds = self.gsk.Splits(X, cv_folds=cv_folds)
cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=xgb_param['n_estimators'], nfold=cv_folds, flds, metrics='auc', early_stopping_rounds=50, verbose_eval=True)

self.model.set_params(n_estimators=cvresult.shape[0])

To make the parameters selection, I run multiple times this CV and average the results in order to attenuate those differences.

Once my model parameters have been "found", what is the correct way to train the model, which seems to have some inner random states ?

Do I :

train on the full train set and hope for the best?
keep the model with the best validation score in my CV loop (I am concerned this will overfit)?
bag all of them?
bag only the good ones?

Since you want your model to be a general solution, you want to include all your data when building the final model.  You are correct in saying that keeping the model with the best validation score in the CV is overfitting.  Including these inner random states help generalize your model, and since you have already tuned your model parameters using CV, you can apply these parameters to the final model.

As for feature selection, you want to separate the data used to perform feature selection and the data used in cross-validation, so feature selection is performed on independent data in the cross-validation fold.  This prevents biasing the model.  If you were to select your features on the same data that you then use to cross-validate, you will likely overestimate your accuracy.

Here are some other great posts that help:
https://stats.stackexchange.com/questions/11602/training-with-the-full-dataset-after-cross-validation

https://stats.stackexchange.com/questions/27750/feature-selection-and-cross-validation

Check out Dikran Marsupial's answers to both, they are really good.


Multiclassification Error: NotFittedError: This MultiLabelBinarizer instance is not fitted yet After picking the model, when I try to use it, I am getting error -

  "NotFittedError: This MultiLabelBinarizer instance is not fitted yet.
  Call 'fit' with appropriate arguments before using this estimator."

X = <training_data>
y = <training_labels>

# Perform multi-label classification on class labels.
mlb = MultiLabelBinarizer()
multilabel_y = mlb.fit_transform(y)

p = Pipeline([
('vect', CountVectorizer(min_df=min_df, ngram_range=ngram_range)),
('tfidf', TfidfTransformer()),
('clf', OneVsRestClassifier(clf))
])

# Use multilabel classes to fit the pipeline.
p.fit(X, multilabel_y)

This code will work. Just let sklearn.linear_model.LogisticRegression handle the multiclassification for you.

from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.linear_model import LogisticRegression

X = ["How to join amazon company ","How to join google ",'Stay home']
y = ["Career Advice", "Fresher",'Other' ]

# Perform multi-label classification on class labels.

clf = LogisticRegression()

p = Pipeline([
('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', OneVsRestClassifier(clf))
])

# Use multilabel classes to fit the pipeline.
p.fit(X, y);
p.predict(X)
```


How to solve Ax = b for A Given two know vector x, and b (dimension 3*1 for example), what are the ways to approximate the matrix A (dimension 3*3) so that the equality Ax=b is as close as possible (something like least square), knowing that very likely the system does not have an answer.

Here you actually do not have a system of linear equations that needs to be seen at a whole and solved together. Here you have 3 independent equations, each of them with infinite valid answers. So:

$\begin{bmatrix}
a_{1} & a_{2} & a_{3}\\
a_{4} & a_{5} & a_{6}\\
a_{7} & a_{8} & a_{9}
\end{bmatrix}\times \begin{bmatrix} x_{1}\\ x_{2} \\ x_{3} \end{bmatrix} = \begin{bmatrix} b_{1}\\ b_{2} \\ b_{3} \end{bmatrix}$

equals solving

$a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$

and

$a_{4}x_{1} + a_{5}x_{2} + a_{6}x_{3} = b_{2}$

and

$a_{7}x_{1} + a_{8}x_{2} + a_{9}x_{3} = b_{3}$

independently.

Each of them in for example in $\mathbb{R}$ has infinite right answers. For instance in $a_{1}x_{1} + a_{2}x_{2} + a_{3}x_{3} = b_{1}$ you just need to randomly choose two of $a_{i}$s and simply get the last one. For example for known $x$ vector $\begin{bmatrix} 1\\ 2 \\ 3 \end{bmatrix}$, $b$ vector $\begin{bmatrix} 4\\ 5 \\ 6 \end{bmatrix}$ and $a_{1}=1$ and $a_{2}=2$, we get $a_{3}$:

$5 + 3\times a_{3} = 4$

so

$a_{3}=-\frac{1}{3}$!

Same procedure applies to other two equations as well. Randomly choose two $a_{i}$s for each equation and get the last one.

PS: I feel either something is missing in your question or I did not understand well. In either cases, please drop a comment here and I will update the answer.


How to generate image using deep learning There are lots of examples of classifier using deep learning techniques with CIFAR-10 datasets. The way it works is that, train thousands of images of cat, dog, plane etc and then classify an image as dog, plane or cat. But I want to do the reverse thing. I want to train dog, cat, planes and it should output images. Here is my idea

Group similar/clear images of cat as value 1, a less similar/blur images as 1.1 and so on.
Similarly, group similar images of dog as value 2 and less similar images as 2.1 and so on ...
Do the same for all types of images.

Generated dataset should look like this

input    output
 1      pixels(24*24) of a cat images(clear)
 1      pixels(24*24) of a cat images(clear)
 .
 .
 1.1      pixels(24*24) of another cat images(blur)
 1.1      pixels(24*24) of another cat images(blur)
  .
  .
  and so on

Now train values of input and label of images as output. Input will be just 1 dimensional, may be I will think of some other data. Output layer will have 24*24 i.e 576 units or neurons.
At the end of training, I want something like this, if I give a input for example 1.15 it should output a new image since we trained with values 1.1 and 1.2 but we didn't train with 1.11 or 1.115.

Please give me some idea how can I do this? Any link to example or papers would be nice.

DCGAN(Deep convolutioanl GAN) or WGAN can be used to generate images by training them on already existing images. GAN are very tricky to train. DCGAN and WGAN are the techniques that make it somewhat easier.

A GAN consists of a Generator and a Discriminator. Generator's job is to generate images and Discriminator's job is to predict whether the image generated by the generator is fake or real. If the generator succeeds in fooling the discriminator, we can say that generator has succeeded.

I have generated MNIST images using DCGAN, you can easily port the code to generate dogs and cats images.


Recommended papers for deep learning based classification? I'm working on a project where I have to build a classifier using Deep Neural Networks and I will be dealing with big data. I am familiar with neural networks, yet not an expert. I can obviously find a huge amount of literature online regarding deep learning but mostly will be convnets related.
Are there any good research papers for DNN-based classifiers or should I just stack more layers? I want to get some insight about the architecture and training process.
P.S: Using Deep Neural Networks is mandatory even if the accuracy is low, but it would be great if it worked out.

Maybe reading papers is a bit hard, because they usually don't explain everything. I suggest you watching deep learning specialization by professor Andrew Ng, which is brief and easy to figure out. If you insist on reading papers, this three sequential articles may help you. Choosing neural networks highly depend on your task. For most computer vision tasks convolutional neural nets are used. You can also take a look at here which contains different resources and discussions and many other useful stuff.


Regions /patterns in images that creates classification I am  having  some images on which  I   trained some neural   network  models in order to  do classification( binary  task).
I am interested  of a  method    to identify  some patterns  or regions  in those images  that are  influencial  the classification   part in  other words  what  regions or  what patterns are formed  for a particular  class.

I think visualizing convolutional neural networks is what you want.

CS231n is a good place to start, here is a step-by-step guide about how to visualize the activations, first-layer weights, and shows how to retrieve images which maximally activate a neuron.

Also there's a demo which visualized multiple convnets (including ResNet50, Inception v3, etc). You may learn a lot from this demo (at least I do).

If you're interesting in details, this well-known paper may help you.


Is there any public implementation / publication with Hintons capsules idea? In Hintons talk "What's wrong about convolutional nets" (Late 2016 or early 2015, I guess) he talks about capsules to make a modular CNN.

Is there any publicly available implementation or papers about this idea?

Following the publication of Dynamic Routing of Capsules (https://arxiv.org/abs/1710.09829), there have been open source implementations of this idea released. See https://github.com/XifengGuo/CapsNet-Keras and https://github.com/soskek/dynamic_routing_between_capsules for two such examples.


How to find and use the top features for XGBoost? Suppose I have data with X_train, X_test, y_train, y_test given. As it is a classification problem I want to use XGBoost.

The issue is that there are more than 300 features.

I have found online that there are ways to find features which are important. But as I have lot of features it's causing an issue.

My current code is below. How can I modify it to say select top n ( n = 20) features and use them for training the model. I tried sorting the features based on importance but it doesn't work.

   import xgboost as xgb
    gbm = xgb.XGBClassifier(max_depth=3, n_estimators=300, learning_rate=0.05).fit(X_train,y_train)
    predictions = gbm.predict(X_test)

SKLearn is friendly on this. Simply with:

from sklearn.feature_selection import SelectFromModel
selection = SelectFromModel(gbm, threshold=0.03, prefit=True)
selected_dataset = selection.transform(X_test)

you will get a dataset with only the features of which the importance pass the threshold, as Numpy array. Point that the threshold is relative to the total importance, so it goes from 0 to 1.

If you want to visualize the importance, maybe to manually select the features you want, you can do like this:

xgb.plot_importance(booster=gbm ); plt.show()


Ways of filtering erronous email addresses using NLP? Background:

I have a database of user information, in which they registered through a website.

Objective:

I would like to filter out erroneous emails, not by if it is malformed (i.e. it's missing an @-sign), but rather by "weird strings" in the "local-part" of the address. So examples of erroneous email address are things like:

zzzzzzzzzzzzzzzzz@gmail.com
asdfasdfasdf@gmail.com
yourenotgettingmyrealemail@gmail.com
123@yahoo.com
test@test.com

I know most of these require some "human" interpretation to figure that they're probably not real, but I was wondering if there are any algorithms that can help me out.

More important than the algorithm is having an existing corpus of labeled data.  Most ML algorithms need to train on a huge amount of text before they start producing useful results (for example a NLP algorithm was trained to generate fiction by reading the entire Harry Potter series).

Do you have a list of known good and bad e-mail addresses?

You can still try to group email addresses even if your data isn’t labeled, but it’s harder.


How to use RBM for classification? At the moment I'm playing with Restricted Boltzmann Machines and since I'm at it I would like try to classify handwritten digits with it.

The model I created is now a quite fancy generative model but I don't know how to go further with it.

In this article the author say, that after creating a good generative model, one "then trains a discriminative classifier (i.e., linear classifier, Support Vector Machine) on top of the RBM using the labelled samples" and furtherly states "since you propagate the data vectors to the hidden units of the RBM model to get hidden unit vectors, or a higher-level representation of the data". The problem is that I'm not sure if I get that right.

Does that mean all I have to do is propagate the input to the hidden units and there I have my RBM feature for classification?

Can somebody explain this process to me?

Review of Restricted Boltzmann Machines

A restricted Boltzmann machine (RBM) is a generative model, which learns a probability distribution over the input. That means, after being trained, the RBM can generate new samples from the learned probability distribution. The probability distribution over the visible units $\mathbf{v}$ is given by
$$p(\mathbf{v} \mid \mathbf{h}) = \prod_{i=0}^V p(v_i \mid \mathbf{h}),$$
where
$$p(v_i \mid \mathbf{h}) = \sigma\left( a_i + \sum_{j=0}^H w_{ji} h_j \right)$$
and $\sigma$ is the sigmoid function, $a_i$ is the bias for the visible node $i$, and $w_{ji}$ is the weight from $h_j$ to $v_i$.
From these two equations, it follows that $p(\mathbf{v} \mid \mathbf{h})$ only depends on the hidden states $\mathbf{h}$.
That means that the information on how a visible sample $\mathbf{v}$ is generated, has to be stored in the hidden units, the weights and the biases.

Using RBMs for classification

When using RBMs for classification tasks, you use the following idea: as the information on how your training or test data was generated is saved in the hidden units $\mathbf{h}$, you can extract these underlying factors by feeding a training sample into the visible units of the RBM, propagate it forward to the hidden units, and use this vector of hidden units as a feature vector. You don't do any backwards pass to the visible units anymore.

This hidden vector is just a transformed version of the input data - this can not classify anything by itself. To do a classification, you would train any classifier (linear classifier, SVM, a feedforward neural network, or anything else) with the hidden vector instead of the "raw" training data as inputs.

If you are building a deep belief network (DBN) - which was used to pre-train deep feed-forward neural networks in an unsupervised fashion - you would take this hidden vector and use it as the input to a new RBM, which you stack on top of it. That way, you can train the network layer-by-layer until reaching the desired size, without needing any labeled data. Finally, you'd add e.g. a softmax layer to the top, and train the whole network with backpropagation on your classification task.


How to check if audio samples have only noise or are silent? I have a wav file I want to split into frames in order to feed it into a machine learning model. The problem is that the audio has silence with some noise at some points.

My problem is that I do not want to include frames with no sound (or only noise) in my dataset.

One solution I believe is to use a model for speech recognition or something similar to do classification and see if a frame includes only silence or noise.

However, I am searching for a solution that will not rely on machine learning but mostly on signal processing techniques or some other pre-processing method.

So, how could I exclude these frames with only silence or noise?

First you should apply speech enhancement algorithms to remove noise from speech. Then, you should use VAD (Voice Activity Detector) to remove silence from speech.


What is the use of the np.ndarray() function and how is it used? I just started Data Science and I want to know what the np.ndarray() function's use is, and in which scenario(s) it is used in. Thanks for the help.

numpy.ndarray is not a function but a class, as you can see in the documentation.

An ndarray is meant to store data, it is just a multidimensional matrix. You store your matrix data in an ndarray and then you operate on it, adding to other ndarrays, or any other matrix operation you want to compute. From the documentation of ndarray:

  An ndarray is a (usually fixed-size) multidimensional container of items of the same type and size. The number of dimensions and items in an array is defined by its shape, which is a tuple of N non-negative integers that specify the sizes of each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each ndarray.

While you can create yourself instances of numpy.ndarray by invoking its constructor, most of the times, arrays are created with the convenience function numpy.array, which ends up creating and returning an instance of numpy.ndarray. For instance, let's create a 3x3 identity matrix manually:

import numpy as np

a = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])

And now we can compute operations on it, e.g.:

b = a + a
print(b)

The result is:

[[2 0 0]
 [0 2 0]
 [0 0 2]]


Good chromosome representation in a VRPTW genetic algorithm I have a genetic algorithm for a vehicle routing problem with time windows and I need to implement certain modifications. I am not sure what would be the best chromosome representations.
I have tasks which can be divided into 3 sub-tasks with certain ordered time windows, they have to processed in order and all 3 (they represent collecting certain goods in a storage, delivering them and returning packaging to another storage). In the algorithm crossover part these tasks are combined together and evaluated. They have to processed in order according to their task number, i.e. combination "A1, B1, B2, A2, B3, A3" is correct, but "A1, B2, A2, B3, A3, B1" or "A1, A2, B1" is not.
The problem is, I don't know how to assure the order of events will be kept. How can I represent this demand in chromosomes? Or, where in the algorithm should I keep this demand?
In the previous version of the algorithm we used only the whole task A or B without distinguishing the subtasks, which is now not sufficient. I am relatively new to genetic algorithms, so pardon me if it's something obvious.

Given you want to maximize fitness function $\phi:\Bbb{R}^n\rightarrow\Bbb{R}$, make $\phi(\vec{x})=-\infty\ \forall \vec{x}\ \notin X$ where the feasible region is $X\subset \Bbb{R}^n$. Then you can initialize the population $P=\{p\ |\ p\in X\}$ and if the GA tries any $p\notin X$, that solution would be killed off. This forces the GA to "learn" the feasible region.


Azure Cloud SQL - Querying large number of rows with Python I have a Python Flask application that connects to an Azure Cloud SQL Database, and uses the Pandas read_sql method with SQLAlchemy to perform a select operation on a table and load it into a dataframe.

recordsdf = pd.read_sql(recordstable.select(), connection)

The recordstable has around 5000 records, and the function is taking around 10 seconds to execute (I have to pull all records every time). However, the exact same operation with the same data takes around 0.5 seconds when I'm selecting from a local SQL Server database.

What can I do to reduce the time it takes to load data from Azure to a dataframe? Would moving the entire Python application to Azure serverless help? Thanks

Additional Information

Azure database is on Standard tier with 20 DTUs
Database region has been configured to be close to my location
Ideally looking for the operation to take under 2 seconds

You have several phases of the data retrieval process: connection time, download time, and database procesing time.

If you store your data in a csv file in a blob storage then the processing time will be faster (essentially zero). So every day you could save the data from the database to a csv file and then access the file when you need it.

Azure serverless will reduce the connection time and download time (if your internet connection is slow), but will not reduce the processing time of the database.


HDBSCAN Outlier Detection and labeling I am using the HDBSCAN algorithm so as to perform unsupervised clustering and detect outliers. Based on the documentation there are two outputs from the clustering process that can give insight on which points are outliers.

The GLOSH outlier detection algorithm that gives a degree of certainty of whether a point is an outlier or not.
The HDBSCAN labels that if an element in not part of a cluster is considered as noise and has the corresponding label.

I have been working on some data, and I have noticed that these two approaches do not give the same results. That means that there are points in my dataset that are labeled as not part of a cluster but are not detected by the GLOSH outlier algorithm. Should I consider the union of these two approaches as my outliers or I am missing something in their interpretation?

GLOSH works with regard to local outliers, thus if a region is very dense and then has a few sparse points near it GLOSH will rate those sparse points as more outlying than other noise points in sparser regions. This is because it is considered particularly odd for a point to be near, but not in a very dense region -- local to the region the point is a significant outlier.

With regard to your question: if you care about local outliers (things that are odd relative to the density of the nearby region) then GLOSH is a good choice. This would be useful, for example, in detecting network intrusions since an attacker will be trying to behave like normal users, but will be slightly outside the dense normal user cluster. If, on the other hand, you are more interested in generally outlying points then merely selecting the noise points from the clustering will pick out points in sparser regions of the space.

Finally note that selecting noise points gives you only a single threshold; in contract GLOSH provides a range of scores and you can tune the threshold to find the number of outliers you desire.


Root Mean Squared Error (RMSE) - significance of square root What is the significance of the square root in root-mean-square-error? Essentially, my question is: what is the difference between (rms error) and (rms error)$^2$?

It depends on what you are using the RMSE for. If you are merely trying to compare two models/estimators, then there is no significance to the square root. However, if you are trying to plot the error in terms of the same units as you made the measurements/estimates, then you need to take the square root to transform the squared units to the original units (much like variance vs standard deviation)


Applying mean encoding before or after splitting into train and test set I have a dataset of 50000 observations with columns of high cardinality. The best way to encode them is with mean encoding, then to use regularization. I will use CV rather than smoothing. But when I see people use it, they use it on train and test set.

Should I first split my dataset into train and test set and then encode or can I encode directly from the beginning on my full dataset?

If I should split the data into train and test set first, can someone tell me why?

The purpose of having a test set or a validation set is to be able to check the performance of your model on data it has not seen before. If you perform feature engineering with the test data present you will get a data leakage. That happens when give your model information about your test data during training.

It is especially bad when doing target encoding with the label mean since it will give your model information about the distribution of the labels in the test set. The effect is that you will get an overly optimistic test score that will not reflect the performance on new truly unseen data.


Given a t-SNE plot, how can I infer the "most correct" labels? How does one understand its structure? Let's say I begin with an exceptionally large dataframe (e.g. imported/munged from tsv files). Several of these columns are categorical labels.

(As a more concrete example, let's imagine a group of students in a school district, pre-school to high-school).

Now, I begin using sklearn and instantiate a t-SNE model, similar to the example here:

http://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html

 import numpy as np
 from sklearn.manifold import TSNE
 X  # my data
 model = TSNE(n_components=2, random_state=0)
 np.set_printoptions(suppress=True)
 model.fit_transform(X)

and then we plot this. The plot might look something like this: http://imgur.com/a/3amkJ

Here's my problem: with real datasets, after using t-sne to learn/cluster, you will have a number of "clusters". Then, using the categorical labels, I try to go through each of these, and try to figure out what structure the t-SNE plot is giving me.

For our school example, I'd get the t-SNE output, then I would label the datapoints. (Let's assume that the clusters are actually representative of age/classroom, e.g. the first-graders group together, the second-graders are a group, etc.)

If I try to color this plot with "grades", I'll see that the grades does not really explain the structure of this plot. (Why? Because every class-level has students with As, Bs, Cs, etc.) Then I might try height...that does pretty good (because there's a correlation between short students--> pre-school, tall students --> high school seniors).

How does one use a t-SNE plot to infer the "most correct" labels of the data? How does one use t-SNE plots to explain (and further explore) the plot structure?

With t-SNE none of the input parameters are weighted more than any other parameter so the differences you want to see like students forming islands by grade level will not happen because there is so much other data present to pull those students/data points in different directions.

I highly encourage you to have a specific question in mind and tailor your input categories so that your question can be answered by the t-sne map.

You might try asking a specific question and changing which input categories you ask t-SNE to look at. For example, do taller students get better grades? Feed in height and grades  categories while leaving out grade level and age. This is a silly example but I hope it gives an idea of how you can use t-sne to help you learn about your data.

You might also find that there are categories that mask meaningful findings. Height might not be very useful for pulling out meaningful information and since the height range is going to be much larger than the A-F grade range, it will likely influence the t-sne map more.

Looking at the data as you described on a color scale for each category is a good place to start with each new t-sne run.

It is okay to run multiple t-snes with different parameters. To be sure you are getting a meaningful answer, I recommend running t-sne multiple times with the same parameters as well.

If you have enough students, it would be great to included half in your training set where you explore and figure out what questions to ask and then when you think you've found something meaningful, apply those conditions to the other students in the testing set to see if it holds true.


Different optimizers for generator and discriminator in GAN I've seen an advice about GAN implementation, that there should be different optimizers for generator (G) and discriminator (D). As I understand, it depends on how fast each model (G and D) convergences. I'd like to clarify this question. Why is it possible to say, that, for example, GSD has to be chosen for D and Adam for G only and not vice verse? (Intuitive or mathematical explanations are welcomed both)

Maybe you can share the link to this advice, so we get the context or arguments, because it doesn’t make a lot of sense to me.

You specifically ask why it’s possible to choose different optimizers. The only requirement that the relation between the discriminator and the generator must meet, is that the discriminator must be able to handle the generator output. Everything else can be different, including the optimizer, prediction algorithm, any regularization, and so on. You could have a neural net for the generator and Mechanical Turk could be your discriminator.

There is something to be said for having different algorithms or architectures between the generator and the discriminator, essentially so the discriminator is able to pick up on the weaknesses of the generator. (Because they won’t share the same weaknesses, hopefully.) But to have everything the same and change only the optimizer? I don’t see the point in that.


Reference of SVM Using Spark Can somebody please give me some reference on implementing SVM using PySpark.

The documentation of Apache Spark's MLib library has a neat and clear reference to the implementation of linear SVM's in Python.

It supports two models: SVMWithSGD and SVMModel


How to give name to topics created using LDA? I have categorized 800,000 documents into 500 categories using the Mahout topic modelling.

Instead of representing the topic using the top 5/10 words for each topics, I want to infer a generic name for the group using any existing algorithm.
For the time being, I have used the following algorithm to arrive at the name for the topic:

For each topic

Take all the documents belonging to the topic (using the document-topic distribution output)
Run python nltk to get the noun phrases
Create the TF file from the output
name for the topic is the phrase (limited towards max 5 words)

Please suggest a approach to arrive at more relevant name for the topics.

I can suggest several papers on this topic:

Automatic Labelling of Topic Models
Automatic Labeling Hierarchical Topics
Representing Topics Labels for Exploring Digital Libraries

You can find more by looking at their citations.


what does "blocks" mean in the context of HDFS ecosystem? is it the same concept in the context of a single hard disk drive? According to Hortonworks

  HDFS has demonstrated production scalability of up to 200 PB of
  storage and a single cluster of 4500 servers, supporting close to a
  billion files and blocks.

what does "blocks" mean in this post?

Essentially yes but the size is very different. A hard drives default block size is 512 bytes but Hadoops default block size is 128 Megabytes.

Hadoop also replicates blocks to maintain integrity of the data.


Is there any NLP framework/ algorithm which is able to derive relationships between different entities? Is there any NLP framework/ algorithm which is able to derive relationships between different entities?

Dogs bark
For example, if the system can identify and understand a phrase, 'Dogs were barking loudly' and can relate to another phrase, 'Usually dogs bark loud, when they sense danger'. And from this piece of information, the system can imply/ derive: 'Dogs had sensed danger' - derived sentence.

Can BERT help in identifying and implying?

What you describe touches on several tasks in NLP:

semantic role labeling
relationship extraction
paraphrase detection and textual entaiment

I might not be aware of the recent developments on these tasks, but as far as I know this is a very complex problem for which so far there is no satisfactory solution.


if for same input values we have different class labels, Should we have to remove those training examples before building any model on it? Ex:

 A1  A2  A3  Class attribute
---------------------------
 X   Y   Z    A

 X   Y   Z    B

 P   Q   R    C

Should we remove first and second training examples.

If yes, why? If we remove those we are loosing information that if inputs are (X,Y,Z) then class may be A or B but not C

If no, why not?

No, you should not remove them.

The problem in your case are not these individual samples. The real problem is that the attributes (variables) that you chose, i.e. A1-A2-A3, are not informative enough to accurately discriminate the samples. This is visible from the fact that the same input space (i.e. X-Y-Z) leads to different classes (A-B).

If you visualize this in a 3D plot with different colors for each class, you will see that the same datapoint will have two colors (!). This is because two datapoints fall exactly on top of each other and it is impossible to find a linear/nonlinear separation for them, no matter how hard you try.

Therefore, if you wish to increase the classification accuracy, you should pick other attributes/variables and not drop these individual samples; you have to change the way you look at the problem :)


sklearn "balanced_accuracy_score" sample_weights not working I wanted a metric where I could weigh each class as I wish while measuring "total accuracy". sklearn seems to have this with balanced_accuracy_score. Irrespective of the sample_weight, I am getting the same "balanced accuracy". Why? what was the point of sample_weights?
from sklearn.metrics import balanced_accuracy_score

sample_weight = np.array([1 if i == 0 else 1000 for i in y])
balanced_accuracy_score(y,m.predict(xs),sample_weight=sample_weight)

Here are the docs.

The point of sample_weights is to give weights to specific sample (e.g. by their importance or certainty); not to specific classes.
Apparently, the "balanced accuracy" is (from the user guide):

the macro-average of recall scores per class

So, since the score is averaged across classes - only the weights within class matters, not between classes... and your weights are the same within class, and change only across classes.
Explicitly (from the user guide again):

$$\hat{w}_i = \frac{w_i}{\sum_j{1(y_j = y_i) w_j}}$$

i.e. the i-th sample is re-weighted by dividing its weight by the total weights of samples with the same label.
Now, if you want, you can just use the simple accuracy score, and plug in weights as you see fit.
In the following example:
from sklearn.metrics import balanced_accuracy_score, accuracy_score

y_true = [0, 1, 0, 0, 1, 0, 1, 1, 1, 1]
y_pred = [0, 1, 0, 0, 0, 1, 1, 1, 1, 1]

some_sample_weights =[10, 1, 1, 1, 10, 1, 0.5, 0.5, 0.5, 0.5]
weights_by_class =[1 if y==1 else 1000 for y in y_true]

print('with some weights: {:.2f}'.format(balanced_accuracy_score(y_true, y_pred, sample_weight=some_sample_weights)))
print('without weights: {:.2f}'.format(balanced_accuracy_score(y_true, y_pred)))
print('with class weights in balanced accuracy score: {:.2f}'.format(balanced_accuracy_score(y_true, y_pred, sample_weight=weights_by_class)))
print('with class weights in accuracy score: {:.5f}'.format(accuracy_score(y_true, y_pred, sample_weight=weights_by_class)))

class_sizes = [sum((1 for y in y_true if y==x))/len(y_true) for x in (0,1)]
weights_by_class_manually_balanced = [w/class_sizes[y] for w, y in zip(weights_by_class, y_true)]

print('with class weights in accuracy score (manually balanced): {:.5f}'.format(accuracy_score(y_true, y_pred, sample_weight=weights_by_class_manually_balanced)))

you get:
with some weights: 0.58
without weights: 0.79
with class weights in balanced accuracy score: 0.79
with class weights in accuracy score: 0.75012
with class weights in accuracy score (manually balanced): 0.75008

As you can see:

using class weights in the balanced accuracy score didn't matter; they just got adjusted back into class sizes.
using class weights in the accuracy score is very close to 75% (3 of out of 4 the 0 labels are correctly classified), and re-adjusting the weights according to class sizes doesn't matter much (the accuracy is a bit less, because the 0 class is larger)


Smaller test data set than training data set in machine learning I would like to train different machine learning algorithms (SVM, Random Forest, CNN etc.) for the same data set (e.g. MNIST) und then compare their accuracies.
The goal would be to find out from which training data size which method is preferable to the others.
To do this I continuously reduce the original training data set (of 60000 samples) and train the models on these reduced trainings data sets.

If I then determine the accuracy using the original MNIST-test dataset (10000 samples), of course I will get overfitting, e.g. with a training data set of 1000 samples I get a training accuracy of 95% and test accuracy of 75%.

The smaller the training data set, the lower the test accuracy, while the training accuracy remains at about the same level.

Would it make sense also to reduce the test data set to restore the original 1:6 ratio of the test set : training set?
Personally, I think that does not make sense.
Or have I thought incorrectly about that?

It's normal (and expected even) to have a Test Set that is smaller than your Training Set.

In general, the more training data you have, the better your performance should be. That is, there'll be less variation in your model parameters if trained on more examples.

This is especially true for Deep Learning approaches (your Convolutional Neural Network, CNN, you mentioned).

Below is a graph that illustrates the above. Where 'Classical Learning' refers to Machine Learning models (SVM, Random Forest, etc.)

From this explanation and graph, it should now make sense why you get a lower test accuracy when you have a smaller training set. There simply isn't enough variation in the training samples, leading to overfitting.

There are numerous ways to avoid overfitting, through data augmentation, adding Dropout layers (in the case of CNN), or regularizing your model. Going into the details is beyond this answer.

However, one of the best and easiest ways to help avoid overfitting is through increasing the the training set.

By increasing the training set, you follow the graph above and decrease variation in parameters but more importantly expose your model to more samples and variation in the dataset.

This helps explain, in part, why you see a higher test accuracy when using a larger test set.

Should you reduce your test set to maintain a ratio? Probably not, in a similar way to your parameters experiencing variation with a small training set the same is true with your test set. You'll want a big enough test set to cover enough variation in samples so the accuracy returned can be trusted.

This StackOverflow answer provides good detail and rationale on choosing the training/test split.

Let me know if you have any questions! I can't comment yet, hence a relatively general answer.


Can anyone recommend me a very good pre-trained model for face or head detection? I really need to know the best pre-trained models to detect faces and/or peoples' head. Not a face recognition model, but only to classify whether an object is a person's head/face or not.
I'm implementing one on top of Resnet50 from tensorflow.keras.applications.resnet50, but I'm not sure if it's a good approach.
Other models like centernet and efficientnet in tensorflow are pre-trained to detect several objects, but I'm not sure  if I can also use it for this purpose.
I need something better than those provided by OpenCV (e.g. cv2.HOGDescriptor, cv2.CascadeClassifier, etc.).

Using facenet pretrained model to detect faces in image, but i thing ti will not detect the head, so i suggest that you train object detection model to detect faces and heads.


If a neural network is a universal function approximator, is the irreducible error 0? How to marry the fact that (most) neural networks with a single hidden layer are universal function approximators with the fact that in the bias-variance decomposition we consider there to be an irreducible error?

Can the irreducible error be 0? (Then it would be inappropriate named.)

The answer to the title question is "no"; the contexts are a little different.

The bias-variance tradeoff holds in the context that $y=f(X)+\varepsilon$, with $\varepsilon$ having mean 0 and variance $\sigma^2$; and $\sigma^2$ turns out to be the irreducible error, so named because it is a lower bound on the expected squared-error.  So to answer your last question, the irreducible error can be zero, if there is no noise in the data.

That neural networks are universal function approximators refers to the actual output function itself (many linear functions with, say, ReLU activators), and not to the fitting procedure.  Notice too that the universality has restrictions: the approximation is only guaranteed on a compact domain, and the error in approximation can be pushed down by using enough neurons in the hidden layer.

So what the universality really implies, w.r.t. the bias-variance tradeoff, is that it is possible to push the bias to zero (or as close as desired, by using enough neurons).  Of course, this will generally result in a badly overfit model.

wikipedia - Bias-variance decomposition


What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book "Deep learning" of Bengio It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below.

It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436.

Because the ${i,y^{(t)}}$ doesn't seem to be a condition while the author has introduced a condition function as $\mathbf{1}_\text{condition} = 1$ if condition is true or $=0$ if the condition is false.

It is computing the gradient of the log-likelihood with backward propagation in recurrent neural network.

Instead of $\hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}$, I found my solution might be
$$
(\hat{y}^{(t)}_i-y_i^{(t)})\dfrac{\exp(o_i^{(t)})\sum_j\exp(z_j)-(\exp(z_i))^2}{(\sum_j\exp(z_j))^2}
$$
It is based on the prior assumption of Gaussian distribution with $\sigma=1$

Below is the equation I am asking about:

  The gradient $\nabla_{o^{(t)}}L$ on the outputs at time step $t$, for all $i$, $t$, is as
  follows:
  $$
(\nabla_{o(t)}L)_i
= \dfrac{\partial L}{\partial o_i^{(t)}}
= \dfrac{\partial L}{\partial L^{(t)}}\dfrac{\partial L^{(t)}}{\partial o_i^{(t)}}
= \hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}.   \qquad\qquad (10.18)
$$

While, the corresponding notes are:

  $$
\begin{align*}
\boldsymbol{a}^{(t)} &= \boldsymbol{b} +\boldsymbol{Wh}^{(t−1)} + \boldsymbol{Ux}^{(t)} &(10.8)\\
\boldsymbol{h}^{(t)} &= \tanh(\boldsymbol{a}^{(t)}) &(10.9)\\
\boldsymbol{o}^{(t)} &= \boldsymbol{c} + \boldsymbol{V h}^{(t)} &(10.10)\\
\hat{\boldsymbol{y}}^{(t)} &= \text{softmax}(\boldsymbol{o}^{(t)}) &(10.11)
\end{align*}
$$

And I wonder which version of softmax function is used here?
$$
\text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i)}{\sum_j\exp(z_j)}
$$
or
$$
\text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i-\max_kz_k)}{\sum_j\exp(z_j-\max_kz_k)}
$$
?

Thank you very much!

The authors or the publisher may have corrected the inconsistent notation. The current version online looks like below:

Here the $\mathbf{1}_{\text{condition}}$ is the indicator function, as you mentioned.

Regarding the two version of softmax, use the first one for calculating gradient. The second version is for improving numerical stability, and mathmatically it should yield the exact same value as the first version.


product out of stock prediction Is it possible to predict the products which are going to be out of stock in next coming days/weeks.?

If Yes, what could be the possible data points that I should collect?

How is this problem different from sales prediction?

You may don’t need any machine learning for that, just simple math.

Calculate the average use per day, define your desired minimum amount which should be on stock and collect the time for a supplier to send you new products.

After that, you can predict when to order your product from the supplier to always have your desired amount on stock.

Search for order point system :)

With the system you could estimate when a product will be out of stock and when to order to prevent that.


Batch Normalization with CUDNN I want to introduce Batch Normalization in my C++/CUDNN implementation of CNN. The implementation is currently performing well (without BN) on the MNIST dataset.

I am using the CUDNN implementation of Batch Norm, but after having read the Batch Norm paper and the CUDNN documentation carefully, still there are some points that are not clear to me.

From what I understood, batch norm layers are placed between the output of convolutional/dense layers and the non-linearity, like:

u --->[conv/dense (weights W)]--->[BN]--->[ReLU]---> y = ReLU(BN(W*u))

In  some literature I read that placing the batch norm layer after the non-linearity is a better option. Is this true?

The CUDNN documentation says to use the BATCHNORM_MODE_SPATIAL for convolutional layers, and BATCHNORM_MODE_PER_ACTIVATION for dense layers. However, in another implementation (YOLO / Darknet), I only see BATCHNORM_MODE_SPATIAL being used. So I ask myself whether SPATIAL is also OK for dense layers?
As mentioned, from what I understood, BATCHNORM_MODE_SPATIAL should be used for conv layers: but on their input (i.e. before the convolution), or on their output (i.e. after the convolution) ?

From what I understood, it is to be used on the output, like:

u ---->[ conv (weights W) ] ---->[BN with mode SPATIAL]------>[ReLU]---> y

is this correct?

I see that in Darknet / YOLO they do not update the parameters explicitly with the deltas obtained from the cudnnBatchNormalizationBackward(). Does this method also perform the parameters update directly? I don't find any info on this, and I am actually applying a Gradient Descent update after having obtained the Gamma and Beta gradients from the cudnnBatchNormalizationBackward method.
I see that in the CUDNN implementation, a Forward method for Inference is available. Is it OK to completely skip the BN units during the inference phase? In this scenario, I would train with the BN units, and remove them from the network during inference.

I really thank you in advance for your patience and your kind help.
I tried to get rid of these questions alone for quite long, but I am now blocked... your help is really appreciated.

In general, you perform batch normalization before the activation.  The entire point of the scaling/bias parameters ($\beta$ and $\gamma$) in the original paper is to scale the normalized value ($\hat{x}$ in the paper) so that it fully captures the dynamic range of the activation operator.

For example (and this is the example used in the paper), suppose the normalized output of the layer in question (convolution or dense) had unit variance (pointwise within the matrix). Then if you applied the sigmoid function pointwise as activation, you would be constrained to be within the linear region of sigmoid function, hence negating any non-linear properties that are supposedly crucial to making the network work.

However, I'm sure you can find an architecture where the order is switched. In the end, people will do what gets the best results empirically for their use-case.

As far as the updating the parameters go (your fourth question), he does update the parameters. I took a look at the source. Parameter updates occur in the "update" functions associated with the convolutional layer. I.e. he only packages batch norm with other layers, which is a reasonable optimization to make.

For questions 2&3: You are correct except for the following caveat: if you are only using dense layers in the head of a CNN, you don't need to suddenly switch strategies. You would use the MODE_PER_ACTIVATION when you expect each dimension of the feature map to have a different distribution. Think non-vision use cases where the input map might be different types of features. If you are using dense layers in a CNN, you would still still assume the normalization is the same across the "spatial" scale. After you flatten in the final layer, you essentially flatten into $C$ channels with 1x1 spatial dimension. So your "spatial" Batch Norm will have $C$ parameters anyways.

For question 5: No. Think of the BatchNorm as a learned re-scaling and a bias operation together. You wouldn't expect the output of your neural network to be consistent if you trained it with bias then removed the biases at inference time.


Intent Classification in Question Answering I am trying to write a question answer intent classification program.

My task is given a set of unlabelled question and answers, I have
to write a program where I may group all the similar questions
and identify their answers.

Once the answers for a group of similar questions are done, I have
to identify the intent or focus of answers.

For example, if I have a set of questions like:

a) Q: where is Texas? A : It is in USA.
b) Q: where is California? A: It is in USA.
c) Q: where is NASA? A: It is in USA.
d) Q: who is Queen Elizabeth II? A: Queen of England.
e) Q: who is Donald Trump? A : President of USA.

Thus,
I am trying to group questions a, b & c as Location oriented question,
and d & e as Official/Person oriented question.

To solve this problem,

I am trying to use a standard classifier, and as it identifies 'It is in USA' as the class,
I am trying to tag it tag it as "It/NA is/NA in/NA USA/LOC" to identify intent/focus
of answer as Location.

I am using a standard classifier like Naive Bayes and a standard Hidden Markov Model
based tagger.

The result is more or less fine.
I am using two training set one for classifier and the other for tagging.

If any one of the esteemed scientists may kindly suggest how I am trying
to solve the problem?

Here, Q means question, A means answer.

Apology for cross-posting

I don't know about the granularity of the intents. If they are just Person/Loc/Official, why not use question words(where,who,what etc.) to identify the intent? For example, "where" corresponds to "Loc", "Who" corresponds to "Person" etc. You can also use this as a feature in you classifier.
Hope it helps!!


Stability of value function approximation in policy gradients In DQNs, function approximation of the Q-values is unstable for correlated updates. In policy gradients with a baseline, will the value function of the policy not be plagued by the same correlated updates?

For example, in the REINFORCE with baseline algorithm the updates are applied to each time step in a temporally sequential order. I understand that in policy gradients the goal is to estimate the value of the policy and not necessarily the entire state space; however, in a stochastic environment and/or under a stochastic policy, not all states will be sampled with the same probability leading to overfitting to a specific trajectory meaning the value function will not be useful as a baseline for the other trajectories of the policy. Are there algorithms that shuffle the trajectory before fitting the data and/or collect batches of trajectories and then randomly sample from the batch as in DQNs?

The short answer is yes. Take a look at Asynchronous Methods for RL. In a similar way that sampling from experience replay helps breaking the correlations, asynchronous methods are based on having multiple agents interacting with multiple instances of the same environment.

The Process:
Each agent (called worker) collects its own experience up to a specified timestep $t_{max}$, which is stored in a batch. Then a master network performs a training update by using this batch. After the update each worker resets to an identical network to the master and they start the task all over again. The update can be synchronous (as I describe here using a batch) or asynchronous by training multiple agents with their own parameters and then update asynchronously the master network.

Why this is helpful? The experience of each agent is independent of the experience of the others and thus more diverse which helps keeping correlation down.

A couple of additional comments on your question:

The goal in Policy Gradients (PG) is to optimize the expected reward w.r.t some parameters which is analogous to the estimation of the gradient of the policy function (parametrized by these parameters).
In RL, and especially in Policy Gradients, you are dealing mainly with 3 things: state sample correlation, bias and variance of your gradient estimator.
The introduction of the baseline is to reduce variance in the gradient estimator without introducing any bias. The baseline is called control variate and is used for reducing variance in Monte Carlo estimators (our gradient estimator here) and has zero mean.


What does anneal mean in the context of machine learning? An article released by Open AI gives an overview of how Open AI Five works. There is a paragraph in the article stating:

  Our agent is trained to maximize the exponentially decayed sum of future rewards, weighted by an exponential decay factor called γ. During the latest training run of OpenAI Five, we annealed γ from 0.998 (valuing future rewards with a half-life of 46 seconds) to 0.9997 (valuing future rewards with a half-life of five minutes).

Does annealing in this context mean the network found through training that γ was better as 0.9997? How would this be determined?

My limited understanding of the topic led me to the following assumption on how γ was annealed: Different versions of the network were trained for a given amount of time using different versions of γ. Then those different versions of the network played against each other or their true skill scored were compared to determine the ideal value of γ.

Annealing is short for simulated annealing. Simulated annealing is the process of slowly decreasing the probability of accepting worse solutions as the solution space is explored. Over the course of the experiment, γ value was slowly lowered to balance exploration and exploitation. γ is a machine learning hyperparameter so any hyperparameter search method would work (e.g., manually selection or cross-validation).


How to handle One Hot Encoded columns with changing categories in supervised ML Problem? Scenario:
I have the following game data about participants, game and the winner in the following format:
id  game        player  winner
-----------------------------------
1   Monopoly    Paul    Paul
1   Monopoly    Frank   Paul
1   Monopoly    Sandra  Paul
2   Monopoly    Peter   Katie
2   Monopoly    Paul    Katie
2   Monopoly    Erwin   Katie
2   Monopoly    Katie   Katie

So the table above shows information about two sessions of monopoly that have been played, the first one with Frank, Paul (Winner) and Sandra; The second one with Peter, Paul, Erwin and Katie (Winner).
My question is: If I want to predict the Winner of a game, how would I transform the data so that each row corresponds to one session?
Idea 1
My initial idea is of course to one hot encode the "player" feature:
id  game        Paul    Frank   Sandra  Peter   Erwin   Katie   winner
-------------------------------------------------------------------------------
1   Monopoly    1       1       1       0       0       0       Paul
2   Monopoly    1       0       0       1       1       1       Katie

That way, it would be easy to predict the target with a simple model.
However, if I want to predict a winner of a new monopoly session with players Sandra, Erwin, Katie and Tim (New Player), the transformed (i.e. one hot encoded) data
would have one more column. This column is unknown to the model and thus cannot be processed.
Idea 2
Take Idea 1, add a "new player" binary dummy column (the new player name does not contain any useful information anyway), store the winner, retrain the model with the new (additional) information. However, if I play with multiple new players, this would not work (unless I add multiple dummy columns, which would be quite inflexible).
Idea 3
Monopoly can be played with 2 to 8 players, so I could change the table structure to the following:
id  game        Pl_0    Pl_1    Pl_2    Pl_3    Pl_4    Pl_5    Pl_6    Pl_7    winner
---------------------------------------------------------------------------------------
1   Monopoly    Paul    Frank   Sandra  None    None    None    None    None    Paul
2   Monopoly    Peter   Paul    Erwin   Katie   None    None    None    None    Katie

However, the problem with this approach is of course the structure assumes that the is a semantic meaning behind which player takes which "seat":
In game 1, Paul was Player 0 (i.e. the value of Pl_0 is "Paul") but in game 2, Paul was Player 1 (i.e. the value of Pl_1 is "Paul"). In this structure, the
model cannot know that it makes no (semantic) difference, whether Paul is Player 1 or 2. Furthermore, a game that can be played with 10 players cannot be mapped like that. So rejected that idea pretty quickly. Or is there any way to overcome this problem?
Final Question
How would I be able to predict the winner of a game, such that

the model can handle new (unseen) players
the model can handle multiple games with different numbers of players

The main idea of Machine Learning is that your training data is similar to test data. If that hypothesis does not hold, then big chances of failure are guaranteed.
Given that, you still can handle unseen categories even thought the solution is not good and more with One Hot Encoding
In the category encoders library you have the following hyperparameter with the following options

handle_unknown: str
options are ‘error’, ‘return_nan’, ‘value’, and ‘indicator’. The default is ‘value’. Warning: if indicator is used, an extra column will be added in if the transform matrix has unknown categories. This can cause unexpected changes in dimension in some cases.

But, given your problem and your data I am not sure if you will ever be much more successful than doing some descriptive statistics.