How to approach All vs All classification problem Let's say you are building a Star trek style medical tricorder which can diagnose any medical condition. It needs to be able to detect comorbidities where a patient has multiple conditions (e.g. perhaps the patient has COVID, diabetes and lung cancer all at the same time).
How would you build a classification system to detect the most likely set of conditions?
I see two approaches:

Build one model per disease, run predictions using every model and report that the patient is suffering from any disease for which the prediction probability exceeded a threshold (e.g. 0.95)

likely_diseases = []
THRESHOLD = 0.95
for disease in disease_columns:
  model = xgb.XGBClassifier()
  model.fit(X_train, y_train[disease])
  pred_proba = model.predict_proba(patient_data)[:, 1]
  if (pred_proba > THRESHOLD):
    likely_diseases.append(disease)

Build one model per combination of diseases and choose the combo with the highest probability

df['has_covid_lung_cancer_and_stroke'] = df.apply(lambda patient: patient['has_lung_cancer'] and patient['has_covid'] and patient['has_stroke'])

# create all other possible permutations of diseases

highest_probability = 0.0
most_likely_disease_combination = None
for disease_combination in disease_combinations:
  model = xgb.XGBClassifier()
  model.fit(X_train, y_train[disease])
  pred_proba = model.predict_proba(patient_data)[:, 1]
  if (pred_proba > highest_probability):
    most_likely_disease_combination = disease_combination
    highest_probability = pred_proba

It strikes me that approach two would probably be more accurate but might be so computationally expensive that it is intractable. Perhaps some pruning would take place where combinations that have exceedingly low occurrences in the training data set are discarded.

I think you can carry out a usual multiclass classification instead of manually carrying out the one-VS-all strategy (i.e. in the for loop), provided that you can generate the multilabels target(i.e. all the possible combinations of diseases you said). So, I would do something like:

generate the correct labels --> if you have n possible deseases combinations, you should have in an end n target labels (so using the apply-lambda expression generates n binary columns for you, which you can also convert to a single numeric column with n possible integer values).
Below you can find a way to get your final column of target values (an integer value for each possible combination); using the wines dataset as an example where I also consider all possible combinations (here a flavour type is similar to a possible disease in your case). I guess you have a dataframe like this:

so you can create a dict mapping the possible combinations to an integer target value (for the multiclass classification):
target_labels_dict = {tuple([1, 0, 0]): 0, tuple([0, 1, 0]): 1, tuple([0, 0, 1]): 2, tuple([1, 1, 0]): 3, tuple([1, 0, 1]): 4, tuple([0, 1, 1]): 5, tuple([1, 1, 1]): 6}

and you have this:

where for instance the tuple (1, 1, 0) means 'acid' and 'sweet'...
and finally, create your target column:
target_flavours_array = np.zeros(len(y_train)).astype('int')

for ind in y_train_df.index:
   combination_values = tuple(y_train_df.iloc[ind])
   target_flavours_array[ind] = target_labels_dict[combination_values]

getting something like:

use the softmax (or its analogous sofprob function check it out here) with the xgboost instance --> below a quick example of this (training on the original 3 classes dataset for the shake of the example):

from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
import numpy as np
import xgboost as xgb

dataset = load_wine()
X_train, X_test, y_train, y_test = train_test_split(dataset.data,
dataset.target, test_size=0.1, random_state=42)

dtrain = xgb.DMatrix(X_train, label=y_train)
dtest = xgb.DMatrix(X_test, label=y_test)

params = {
    'max_depth': 4,  # maximum depth of each tree
    'eta': 0.2,  # learning rate
    'objective': 'multi:softprob',  # for multiclass problem, providing probabilities
    'num_class': len(np.unique(dataset.target))} # number of classes

iters_number = 30
xgb_model = xgb.train(params, dtrain, iters_number)
xgb_model.predict(dtest)

and, the result is the prediction probability for each possible class (diseases-combination in your use case):

here you can select the highest probability value for each row.


Pre-processing (center, scale, impute) among training sets (different forms) and the test set - what is a good approach? I am currently working on a multi-class classification problem with a large training set. However, it has some specific characteristics, which induced me to experiment with it, resulting in few versions of the training set (as a result of re-sampling, removing observations, etc).

I want to perform pre-processing of the data, that is to scale, center and impute (not much imputation though) values. This is the point where I've started to get confused.

I've been taught that you should always pre-process the test set in the same way you've pre-processed the training set, that is (for scaling and centering) to measure the mean and standard deviation on the training set and apply those values to the test set. This seems reasonably to me.

But what to do in case when you have shrinked/resampled training set? Should one focus on characteristics of the data that is actually feeding the model (that is what would 'train' function in R's caret package suggest, as you can put the pre-processing object in there directly) and apply these to the test set, or maybe one should capture the real characteristics of the data (from the whole untouched training set) and apply these? If the second option is better, maybe it would be worth it to capture the characteristics of the data by merging the training and test data together just for pre-processing step to get as accurate estimates as possible (I've actually never heard of anyone doing that though)?

I know I can simply test some of the approaches specified here, and I surely will, but are there any suggestions based on theory or your intuition/experience on how to tackle this problem?

I also have one additional and optional question. Does it make sense to center but NOT scale the data (or the other way around) in any case? Can anyone present any example where that approach would be reasonable?

Thank you very much in advance.

I thought about it this way: the training and test sets are both a sample of the unknown population. We assume that the training set is representative of the population we're studying. That is, whatever transformations we make to the training set are what we would make to the overall population. In addition, whatever subset of the training data we use, we assume that this subset represents the training set, which represents the population.

So in response to your first question, it's fine to use that shrinked/resmpled training as long as you feel it's still representative of that population. That's assuming your untouched training set captures the "real characteristics" in the first place :)

As for your second question, don't merge the training and testing set. The testing set is there to act as future unknown observations. If you build these into the model then you won't know if the model wrong or not, because you used up the data you were going to test it with.


Is there a scientific method for grouping continuous numbers for this problem I use python, and I have a list of numbers that contains 19 elements and I would like to divide this list into 6 groups or fewer.

The list could have numbers between 0 and 1. and it mustn't be ordered, I need to keep it on its form and getting cut-off

list:

Numbers :  [[ 0.867   - 0.808   - 0.740    - 0.746    - 0.674   - 0.669   -
              0.648   - 0.722   - 0.781    - 0.612    - 0.575   - 0.566   -
              0.500   - 0.555   - 0.818    - 0.800    - 0.500   - 0.500   - 0.666 ]]

I would like to get clusters like :

A: [[ 0.867   - 0.808   - 0.740    - 0.746 ]]

B: [[ 0.674   - 0.669   - 0.648 ]]

C: [[ 0.722   - 0.781 ]]

D: [[ 0.612   - 0.575   - 0.566    - 0.500    - 0.555  ]]

E: [[ 0.818   - 0.800 ]]

F: [[ 0.500   - 0.500    - 0.666 ]]

I make splitting by eyes and for that, I ask for a scientific method for getting my objective.

Concerning how I defined these clusters :

I have a value that is getting from an extra function, it is equal to 0.80. I need to compare each value on the list with 0.80 for knowing the difference.

After making a comparison I get the following table

Numbers      Difference_0.80
0.867            +0.06
0.808             0.0
0.740            -0.06
0.746            -0.06

0.674            -0.13
0.669            -0.14
0.648            -0.16

0.722            -0.08
0.781            -0.02

0.612            -0.19
0.575            -0.23
0.566            -0.24
0.500            -0.3
0.555            -0.25

0.818            +0.01
0.800             0.0

0.500            -0.3
0.500            -0.3
0.666            -0.14

when I tried clustering method (with n_clusters=2) I ve got :

0   category_Kk-mean
0.867   0
0.808   0
0.740   0
0.746   0
0.674   1
0.669   1
0.648   1
0.722   0
0.781   0
0.612   1
0.575   1
0.566   1
0.500   1
0.555   1
0.818   0
0.800   0
0.500   1
0.500   1
0.666   1

But I want to know also that  this category (D) have a big decrease than a category (B):

category D
0.612   1
0.575   1
0.566   1
0.500   1
0.555   1

Category B

0.674   1
0.669   1
0.648   1

I tried with n_clusters=3 but I got a totaly bad result

Is there any method in the statistic or mathematic that can help me for getting that

Because you care about the ordering you can represent your points in two dimensions (where the x coordinate is just rescaled sequence number) and apply K-means.

import numpy as np

numbers = np.array([0.867, 0.808, 0.740, 0.746, 0.674,
                    0.669, 0.648, 0.722, 0.781, 0.612,
                    0.575, 0.566, 0.500, 0.555, 0.818,
                    0.800, 0.500, 0.500, 0.666])

#you can experiment with other step sizes
step_size = (numbers.max() - numbers.min())/len(numbers)
numbers2d = np.array([(step_size * i,x) for (i,x) in enumerate(numbers - 0.8)])

from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=6)
kmeans.fit(numbers2d) #random init - possibly different result each time

print(kmeans.labels_)

import matplotlib.pyplot as plt

for cluster in set(kmeans.labels_):
    x = np.where(kmeans.labels_==cluster)[0]
    y = numbers[x]
    x = numbers2d[:,0][x]
    plt.scatter(x,y)

plt.show()

Output:

[3 3 3 3 1 1 1 1 1 4 4 4 4 4 5 5 2 2 0]


How to define a MLP with multiple outputs I need to build a neural network to predict predict outputs based on a set a input features, for a multi-channel system. Once I figure out this problem, I will implement it in Keras. Let me describe it in further detail:

I have a set of input variables ('temperature', 'current', 'voltage' ... )

the outputs are ('gainA', 'gainB', 'gainC', 'gainD')

However, each letter (A,B,C and D) represents a different channel, and the training data contains lots of different configurations of when the channel is ON/OFF. Therefore a typical training dataset would look like:

'ID' |  'Temperature'  | 'current' | 'voltage' | 'gainA' | 'gainB' | 'gainC' | 'gainD'
  1  |       23.1      |     2.1   |    5.1    |    0    |   1.5   |   3.1   |    1.2
  2  |       23.2      |     2.3   |    5.2    |    0    |   1.5   |   2.1   |    1.1
  3  |       23.4      |     2.0   |    5.0    |   1.3   |    0    |   1.7   |    1.0
  4  |       22.8      |     2.3   |    5.4    |   1.5   |   1.3   |   3.2   |     0
  5  |       22.9      |     2.1   |    5.1    |    0    |    0    |    0    |    1.2
  6  |       23.2      |     2.2   |    5.3    |   1.2   |   1.5   |   3.4   |    1.3
...

As you can see from the dataset above, there is '0' for some gains. This is when the channel is OFF. There is only a non-zero value of gain when the channel is ON.

My model needs to be able to predict the gain of a channel if I were to switch it on

For example if we consider row 5 as the test data, I want to predict the numerical value of 'gainC' if I was to activate that channel

'ID' |  'Temperature'  | 'current' | 'voltage' | 'gainA' | 'gainB' | 'gainC' | 'gainD'
  5  |       22.9      |     2.1   |    5.1    |    0    |    0    |[predict this]|1.2

Initially, I considered training 4 models (one for each channel) however that would not work because the combination of gains are correlated and thus a multi-dimensional MLP is required.

If I used the 3 other gains as part of the input, it would just predict the gain to be 0 because that's what the model sees it to be.

Please could someone help me figure out a way of predicting the non-zero value of a gain. Perhaps part of the answer could include the binary representation of the channels. E.g

for ID number 5, the active channels are 0001 and we want to predict 0011

Maybe this could be part of the training data? Or an additional input to the predictor?

I'd appreciate some advice on how to approach this problem. (I'm fairly new to neural networks)

Have four neurons in your output layer, depicting gain $A, B, C, D$ and apply relu activation function throughout all your layers since it sets all negative values to $0$ and passes the rest.

Also you can use a simple mean squared error as your loss function. This setup will allow you to predict non-negative values, greater than $0$ for your gains, which is, what I believe, you require.


How can RL agents be monitored? My question is about how to monitor RL agents in production. To make the question easier to discuss, here is a use case. Please don't focus on difficulties in implementing such an agent, but rather on how to monitor if it is still doing well:

  Suppose Amazon would use Reinforcement Learning for optimizing the order of a search:

  An episode starts when a user starts a search
  An episode ends after a threshold or when the user buys things.
  The observation the agent gets is the search terms as well as 20 products for which he has to give an ordering.
  He gets a reward if one of those 20 things is bought.

Of course, one can (should?) let the agent learn all the time as products change and probably search terms / language changes. But when do I know that the agent learned something weird / that I should stop it?

I can imagine the following:

Case-based single examples
Having a ground-truth for some trivial searches + products and checking if the model does those right (in non-exploratory mode)
Letting the agent learn in batch-mode (e.g. update the model only once a week) and doing A/B tests for the current model / new model.
Measuring mean reward and setting a threshold. If the mean reward of the agent drops below the threshold, reset the agent to a past "save" state.

Is there literature about it? Blog posts?

I know at least one example where RL went wrong / monitoring didn't quite work:

Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day (The Guardian)

This is an open issue in Reinforcement Learning (and all of machine learning). Google has published a paper titled "Machine Learning: The High Interest Credit Card of Technical Debt" which addresses the many ways these systems can degrade over time.

One way is to follow fundamental production engineering techniques (e.g., test coverage and graceful exit).

Another way is to monitor the data, then retrain the agent if new data is statistically out-of-sample.


XGBoost Objective is Changed I am trying to use XGBoost in python for logistic regression. I am calling it as follows

import numpy as np
from xgboost import XGBClassifier

x_train = np.array([[1], [2], [3], [4]])
y_train = np.array([0, .25, .75, 1])

params = {
    "objective": "reg:logistic"
}

model = XGBClassifier(**params)
model.fit(x_train, y_train)
print(model.objective)

This outputs an objective of "multi:softprob" instead of "reg:logistic." Therefore, it isn't doing a logistic regression. How can I make sure that XGBoost doesn't switch the objective?

The solution to the above problem was to use XGBRegressor instead of XGBClassifier. Just swapping it in seems to have worked.


Output prediction from a sequence of data Given the machine learning libraries available for many different languages, it's possible to utilise algorithms where you don't need to have a detailed understanding of their application or workings or datascience in general.

I have tried to find a list of algorithms that are suitable for sequence prediction from a window of previous input values, which I could then apply in such a library e.g. Accord for C# but have failed.

Each observation I have is an array of 2-dimensonal data in the following form: {MODE1/2/3, int 1-98}. That is, multiple inputs produce multiple outputs. I'd like to examine preceding sets of values from either t-1 or t-x where x could be a variable amount of preceding value sets.

So, I'd like to ask what algorithms excel at prediction from a window of sequence data, and what are their strengths / weaknesses.

Have a look on conditional random field. Conditional random fields  are a probabilistic framework for labeling and segmenting structured data, such as sequences, trees and lattices. The underlying idea is that of defining a conditional probability distribution over label sequences given a particular observation sequence, rather than a joint distribution over both label and observation sequences. The primary advantage of CRFs is their conditional nature, resulting in the relaxation of the independence assumptions. Additionally, CRFs avoid the label bias problem. It is often used for labeling or parsing of sequential data.


finding the mean for each of the channels (RGB) across an array of images How can I find the mean for each of the channels (RGB) across an array of images?
For example train_dataset[0]['image'].shape is (600, 800, 3) and len(train_dataset) is 720 meaning it includes 720 images of dimension 600x800 and 3 channels. train_dataset[0]['image'] is an ndarray.
I am looking to end up with 3 numbers each representing the mean for each of the channels across all these 720 images.
I have this very dumb solution but I wonder if there's a better solution?

I also did it this other way and got almost the same answer:

Could probably be more elegant, but here's an idea:
mean[:, :, 0:2] = np.mean(imgs[:, :, :, 0:2], axis=0)


Inference from text data without label or Target I have a use case where I have text data entered by an approver while approving of some loan.
I have to make some inferences as to what could be the reasons for approval using NLP. How should I go about it?
It's a Non english language. Can Clustering of text help?? Is it possible to cluster TEXT OF non English language using python libraries.

Is it possible to cluster TEXT OF non English language using python libraries?
Sure! classic approaches based on Bag-of-Words are language independent. For modern approaches based on DNNs, mostly pre-trained models, you just need to find a model in your language or train one model from scratch (for this you need lots of text in that language). For example in case of using AWS infrastructure, check Object2Vec algorithm.
Can Clustering of text help?
Can help. For instance for an initial labeling you can cluster data into similar texts and labels each according to overal concept. More sophisticated solution (easily implemented in python) is topic modeling e.g. LDA algorithm.
More sophisticated solution is, again, pre-trained models like S-BERT.
In this direction, I also recommend having an analysis on keywords for algorithms like RAKE or YAKE.
Hope it helps!


Neural network type question This web link is to a site that talks about forecasting building electricity, like a time series regression concept.
In the article they talk about the NN architecture as:

the architecture of this neural network can be written as 120:7:24

Is an MLP type NN? What I also dont understand is if they account for times series methods to forecast/predict. For example I thought for time series applications a sliding window concept needs to be used Vs a typical regression problem that does not have any element of time. Any tips greatly appreciated!

MLP typically refers to a type of neural network called a 'Multi-Layer Perceptron'. As you can read on the Wikipedia page, these neural networks consist of neurons organized in layers. Each neuron has a (typically fixed) function that transforms its weighted input to produce the output for that particular neuron known as its activation function.
A 'network architecture' refers to the way the neurons are laid out in layers in the network, sometimes including the activation function. In the given example, there are 120 neurons in the first input layer. These are connected to 7 neurons in the next layer, which in turn are connected to 24 neurons in the last output layer.
The 120 input neurons in the linked article appear to include both history, i.e. a sliding_window as well as some known future information (temperature, from weather forecasts it seems).


Having difficult interpreting the eigenvectors for a simple 3x2 matrix I calculated the eigenvectors and eigenvalues from a covariance matrix given a data matrix of 3 columns and 2 rows.

I am trying to interpret results but I can't understand on how to interpret them.

Create a 2x3 matrix:

# Create a 2x3 matrix

data = np.around(np.random.uniform(size=(2,3)) * 100)

The data looks as follows:

    [
      [ 4., 65., 77.],
      [68., 12., 89.]
    ]

# Here each row represents one data point
# and columns represent the features in the data set
# So there are 3 features and 2 data points

Calculate the mean for each feature in the data set.

mean = np.mean(data, axis = 0)

Center the data around origin, by subtracting mean from the data set.

difference = np.subtract(data, mean)

Now, calculate the covariance matrix:

cov = np.dot(difference.T, difference)

The cov matrix looks as follows:

[
    [ 2048. , -1696. ,   384. ],
    [-1696. ,  1404.5,  -318. ],
    [  384. ,  -318. ,    72. ]
]

As I understand about the covariance matrix, it explains the variance between all feature-pairs. Since there are 3 features, it gives out a 3x3 matrix explaining the variance between all possible pairs.

Finally, calculate the eigenvectors and eigenvalues:

val, vec = np.linalg.eigh(cov)

The vec matrix looks as follows:

[
    [ 0.60999981,  0.21639063,  0.76228297],
    [ 0.77451164,  0.040441  , -0.63126559],
    [ 0.16742745, -0.97546892,  0.14292806]
]

How do I interpret the the vector matrix? I understand what are eigenvectors physically. They do not change in position when an object undergoes a transformation but only a scalar change by their eigenvalues.

What are some possible ways, I could use this vec matrix?

The eigenvectors can be used to transform your data into a coordinate system in which no covariance is there. Assume we have a $p$ dimensional multivariate normal distribution with

$$\mathcal{N}(\boldsymbol{x}|\boldsymbol{\mu}=\boldsymbol{0},\boldsymbol{\Sigma})=\dfrac{1}{\sqrt{(2\pi)^p\det\boldsymbol{\Sigma}}}\exp\left[-\dfrac{1}{2}\boldsymbol{x}^T\boldsymbol{\Sigma}\boldsymbol{x} \right].\quad (*)$$

As the covariance matrix is real and symmetric we know it is diagonalizable and that we can scale the eigenvectors to represent an orthonormal basis by the set of all eigenvectors. The eigenvalue equation for the covariance matrix is given by

$$\boldsymbol{\Sigma}\boldsymbol{v}_i=\lambda_i\boldsymbol{v}_i \quad \forall i=1,...,p.$$

We can combine all equations into a single matrix equation

$$\boldsymbol{\Sigma}[\boldsymbol{v}_1,\ldots,\boldsymbol{v}_p]=[\boldsymbol{v}_1,\ldots,\boldsymbol{v}_p]\text{diag}\left[\lambda_1,\ldots,\lambda_p \right].$$

If we call $\boldsymbol{V}=[\boldsymbol{v}_1,\ldots,\boldsymbol{v}_p]$ and $\boldsymbol{\Lambda}=\left[\lambda_1,\ldots,\lambda_p \right].$ With these definitions in hand we can write the eigenvalue equation as

$$\boldsymbol{\Sigma}\boldsymbol{V}=\boldsymbol{V}\boldsymbol{\Lambda}$$
$$\implies \boldsymbol{\Sigma}=\boldsymbol{V}\boldsymbol{\Lambda}\boldsymbol{V}^{-1}.$$

as $\boldsymbol{V}$ is orthogonal (consists of orthonormal vectors) we can write $\boldsymbol{V}^{-1}=\boldsymbol{V}^T.$ This implies

$$\boldsymbol{\Sigma}=\boldsymbol{V}\boldsymbol{\Lambda}\boldsymbol{V}^{T}.$$

We plug this into $(*)$ and introduce the new variable $\boldsymbol{z}=\boldsymbol{V}^T\boldsymbol{x}$.

$$\mathcal{N}(\boldsymbol{x}|\boldsymbol{\mu}=\boldsymbol{0},\boldsymbol{\Sigma})=\dfrac{1}{\sqrt{(2\pi)^p\det\boldsymbol{V}\boldsymbol{\Lambda}\boldsymbol{V}^T}}\exp\left[-\dfrac{1}{2}\boldsymbol{x}^T\boldsymbol{V}\boldsymbol{\Lambda}\boldsymbol{V}^{T}\boldsymbol{x} \right]$$
$$=\dfrac{1}{\sqrt{(2\pi)^p\det\boldsymbol{V}\boldsymbol{\Lambda}\boldsymbol{V}^T}}\exp\left[-\dfrac{1}{2}\boldsymbol{z}\boldsymbol{\Lambda}\boldsymbol{z} \right]$$
$$\implies \mathcal{N}(\boldsymbol{z}|\boldsymbol{\mu}=\boldsymbol{0},\boldsymbol{\Lambda})=\dfrac{1}{\sqrt{(2\pi)^p\det\boldsymbol{\Lambda}}}\exp\left[-\dfrac{1}{2}\boldsymbol{z}\boldsymbol{\Lambda}\boldsymbol{z} \right].$$

In the last step I used $\det \boldsymbol{ABC}=\det\boldsymbol{A}\det\boldsymbol{B}\det\boldsymbol{C}$, $\det\boldsymbol{A}=\det\boldsymbol{A}^T$ for orthogonal matrices and $|\det\boldsymbol{A}|=1$ for orthogonal matrices. Hence, we proved that we can use the eigenvectors to linearly transform our variables to obtain a new coordinate system which has a diagonal covariance matrix $\boldsymbol{\Lambda}$ (which is diagonal). This implies that we do not have any covariance anymore.

Remark 1: This diagonalization is what the principal components analysis is also doing under the hood.

Remark 2: There is one suboptimal part of your code. You should explicitly determine the sample_size = data.shape[0] and then calculate cov = 1 / (sample_size - 1) * np.dot(difference.T, difference).


What is a distribution-wise asymmetric measure? I was trying to understand KL-Divergence, $$D_{KL} \langle P(X) \Vert P(Y) \rangle,$$ and was going through its Wikipedia article. It says the following

In contrast to variation of information, it is a distribution-wise asymmetric measure and thus does not qualify as a statistical metric of spread - it also does not satisfy the triangle inequality.

What is the meaning of distribution-wise asymmetric measure? Is there a symmetric measure? What are the rules that a quantity should follow to be qualified as a statistical metric of spread?

What is the meaning of distribution-wise asymmetric measure?

The (forward) KL-divergence is distribution-wise asymmetric because if you calculate it as $$D_{KL} \langle P(X) \Vert P(Y) \rangle$$ where $P(X)$ and $P(Y)$ are two different probability distributions with the latter being the reference distribution, then $$D_{KL} \langle P(Y)\Vert P(X)\rangle \neq D_{KL}\langle P(X)\Vert P(Y)\rangle.$$ In other words, the reverse KL-divergence is not equal to the forward KLD. If forward KLD were symmetric, then the above would be an equality, not an inequality.

Is there a symmetric measure?

A distribution-wise symmetric measure would, for example, be mutual information:
$$I(X;Y) = H(X)+H(Y)-H(X,Y) = D_{KL} \langle P(X,Y) || P(X) \cdot P(Y) \rangle,$$
where $H(X)$ is the entropy of a variable $X$'s probability distribution, since $I(Y;X) = I(X;Y)$. Mutual information is a special case of the KLD in which the joint distribution is measured against the product of marginal distributions.

What are the rules that a quantity should follow to be qualified as a statistical metric of spread?

The three axioms that a distance metric should meet are:

Identity of indiscernibles
Symmetry
Sub-additivity or triangle inequality

Since mutual information does not obey the triangle of inequality, it does not fit the full criteria for being a distance metric. Instead, variation of information does meet all of the above requirements and is a true metric:
$$VI(X;Y) = H(X,Y) - I(X;Y)$$
where $H(X,Y)$ is the joint entropy.


Is there any use to running Pandas on Spark? I just started working with PySpark this week, and the instance I have access two has Pandas installed. But what use is having Pandas on Spark?

Isn't the whole purpose of running scripts on PySpark to overcome the limitations of packages like Pandas?

Does Pandas performance improve if it run on Spark? Is it compatible with Spark's RDD ?

There is no need for pandas module to be installed because your data is generally stored in spark RDD or spark dataframes objects.

The only interest I have found using Spark with pandas is when you want to load a local CSV / Excel dataset and then transform it into a spark dataframe. The "createDataFrame" method handles this approach.

>>> sqlContext.createDataFrame(pandas.DataFrame([[1, 2]])).collect()
[Row(0=1, 1=2)]


How to fill missing values by looking at another row with same value in one column(or more)? Let's say we have a 6*4 data frame in which third and fourth column contain missing value

1 2   3   L1
4 5   6   L2
7 8   9   L3
4 8   NaN NaN
2 3   4   5
7 9   NaN NaN

I'd like to fill the missing value by looking at another row that has the same value for the first column. So, in the end, I should have:

1 2   3   L1
4 5   6   L2
7 8   9   L3
4 8   6   L2    <- Taken from 4 5 6 L2 row
2 3   4   L4
7 9   9   L3    <- Taken from 7 8 9 L3 row

How can we do it with Pandas in the fastest way possible?

Sorted and did a forward-fill NaN

import pandas as pd, numpy as np
data = np.array([[1,2,3,'L1'],[4,5,6,'L2'],[7,8,9,'L3'],[4,8,np.nan,np.nan],[2,3,4,5],[7,9,np.nan,np.nan]],dtype='object')
df = pd.DataFrame(data,columns=['A','B','C','D'])

df.sort_values(by='A',inplace=True)
df.fillna(method='ffill')


Identifying most informative (sub)words/vectors that help classify a sample I am classifying text using fastText which is a word2vec library that can also create vectors for character level n-grams and I have successfully trained a binary classifier.

Now I’d like to see what words or subword n-grams are the most predictive of a class for the two classes (e.g. if classifier sees a word forest or a subword res then that might be a strong indication that the document has label Nature, but if it sees word “and” then that is probably not very informative for this classification task).

Therefore, I guess the question could be phrased as:

Given vectors representing words and subwords and a trained fastText classifier, what would be the best way to get a list of e.g. top 10 most informative words and subwords for deciding which class a sample belongs to?

Even though I’d be glad if you could make specific suggestions that consider my current setup with fastText, I’m also open to the more general solution suggestions.

Thanks

Note 1: I believe the technical term for what you are looking for is '(sub)word salience'.

First thought (which is not the best method, but could be worth a quick try if your examples are smallish) - Run the text with all words through fastText, and get the baseline most likely labels with probabilities (predict-prob function). Then remove each word that you are interested in testing, (maybe removing common words like 'a' 'the' etc), and compare the predicted probabilities of each obtained set of predicted_probs. The ones that give the greatest difference between baseline and missing wordX can be interpreted as the most informative.

A permutation of this idea is to get the embedding for the interesting words, and then get a distance measure (I assume cosine distance) to the labels you have. Then you have a problem, as they may be distributed evenly across labels in terms of distance, but if there is a group of words that are close to a particular label, that can be interpreted as informative.

I am sure there is a better technique to do this, so I am curious as well :). You may want to check these out (but they use additional modeling to get salience)

Learning Neural Word Salience Scores.

A NEW METHOD OF REGION EMBEDDING FOR TEXT CLASSIFICATION - this deals with the exact problems of how to identify the useful bits of an input text


Business exception reporting I work in an analytical role at a a large financial services firm. We do a ton of daily reporting over metrics that rarely change in a meaningful way from day to day. From this daily reporting, our management is required to extract what was important yesterday and what important trends have developed / are developing over time.

I want to change this to a model of daily exception reporting and weekly trend reporting.

Features might include:

User report consolidation (so there's only one daily email)
report ordering based upon level of variance from past performance (see the most important stuff first)
HTML email support (with my audience, pretty counts)
Web interface to allow preference changes, including LDAP support (make administration easier)
Unsubscribe feature at the report level

Here's what I'd like to know:

What are the practical problems I might run into?
What is the best way to display the new reports?
How should  I define an "exception"? How can I know if my definition is a good one?
I assume I'd be using a mix of Python, SQL, and powershell. Anything else I should consider, e.g. R? What are some good resources?

Try exploring the rich field of "Anomaly Detection in Time Series". Control charts and CUSUMs (or cumulative sum control charts) might help you.

Simple Bullet Graphs might be all you need. Based on historical data and domain knowledge, define normal variance. Then make it clear to stakeholders when the current value is outside of predefined ranges.

Stephen Few is an expert in business dashboards. Any of his books will help you.

If you are open to R, try Shiny to create simple interactive web applications (It is very straightforward). There is also an open source package for anomaly detection, both local and global.

Create quick prototypes and get feedback!


Surface Pro 6 vs Macbook Pro for Professional Data Science Practice [I strongly agree this is totally very opinionated question, thus narrators feel free to vote to close it if you feel it is right, but I find endless pros and cons on the Internet, I've decided to ask the community here.]

Surface Pro 6 or Macbook Pro for Data Scientist Job?

About 8 years ago I was a Windows user. The most annoying part was that it was a quite unstable. It is noted that I was not a developer or a data scientist back then. Anyways, I moved to Mac OSx both at work during my PhD studies since it was a better choice for doing Computational Physics. I totally felt great with such a smooth, stable OS that allows access to Linux kernel as well, and eventually I bought a Macbook Pro for my personal use.

At present I am about to start my new role in industry as Data Scientist. The whole team is based on Microsoft, even very few developers in the team using the latest Surface Pro 6. They give me a choice of either Surface Pro 6 or Macbook Pro, although I felt it was implied they would rather me going with Surface Pro 6 (still I can choose!). The configues will be exactly the same as I wish, so this wouldn't matter.

Obviously I am leaning towards Macbook Pro as I have been with for some time now and am quite efficient and productive. Nevertheless, majority of other employees are Windows-base, and I feel if I go with Mac I would be an outlier.

At the same I am not feeling completely stranger around Windows and I am not totally blind! In fact recently I hear in the news a lot about Microsoft drastic changes naming: incorporating Linux kernel starting from end of this month (would be so sure if it would work as the Mac one though) in Windows 10 Pro, Visual Studio Code is a product of Microsoft that has been gaining lots of interest and I am using and loving it and perhaps would run even more smoother in Windows machine than Mac, and not that it is very relevant but Widows acquired GitHub. it is worth  to mention that I will have GPU if I go with Surface Pro 6, and has a nice touchscreen feature too!

I guess I still looking for an actual use case, where someone did in fact had such a transition recently, and would appreciate sharing your thoughts and feedbacks.

Although once asked I knew this was a very subjective question, and I was not looking for specific yes or no, but rather hearing the community's experience, here now I would like the share the following short conclusion for those that might have to make a similar decision.

Golden Rule is Majority Voting What I do mean by that is that it is best to be aligned with other team members and the companies's IT support. If majority or all are Windows based, then one should go for that, and vice versa for Linux, otherwise you are on your own and will suck up much of your time to get certain things up and running (printer driver etc.).
Minor Differences I have witnessed minor differences actually. I have to admit that I do miss Linux terminal. Windows has been trying so hard to integrate a proper terminal, but so far all solutions like Windows PowerShell or cmd or Ubuntu terminal for Windows or even Anaconda Prompt doesn't give that freedom and experience one have with Linux kernel. The rest is pretty much the same, and there has been no really big issue for me migrating from Mac OSx to Surface Pro 6.
Hardware/Software Stability: For long I have been biased that Mac OSx is the most stable OS one could experience, perhaps due to terrible past experience working with Windows (Win 98, XP, 2000 etc.). Although still typical issues remain, which is true to all OSs, I have to say that Windows there days is a super reliable system comparable to Mac OS if not better, you know for daily tasks (I am not bench-marking anything here) and not causing dramas. For the Hardware, honestly if we account for Hardware for Price comparison, Windows wins. It is far more powerful better CPU, RAM, Graphics, Storage for the price you pay. I would get a much lower performing Mac OSx, and nothing is worst that having worried about 250 GM storage you have, while you can get a double or tripple space easily in a Windows equivalent machines. Certainly if price is not a concern, one should not be concerned about this as well. Last but not least, since we are comparing Hardware, I have to mention I am loving my touchable Surface Pro 6 display, and one can detach the screen to turn it to a Tablet which is awesome, and the screen itself is so durable, in contrast to a super sensitive Mac screens. Please note I am not advocating for Windows, just stating facts and sharing personal experience using both.

In case over long period of time, these facts change, I will get back to this post and update again my personal high-level experience.


Recommendation for boolean dataset with apache mahout I was trying to implement Item based Recommender System with the boolean dataset, Dataset example:

      User-id | movie-id | Action | Comedy | Drama
        1              200    0         1        1
        2              210    1         1        0

And I tried implementing it with item-based similarity algorithm as follows:

package prediction.contentrecommender;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.apache.mahout.cf.taste.common.TasteException;
import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender;
import org.apache.mahout.cf.taste.impl.similarity.TanimotoCoefficientSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;
import org.apache.mahout.cf.taste.recommender.RecommendedItem;
import org.apache.mahout.cf.taste.similarity.ItemSimilarity;
import org.apache.mahout.cf.taste.impl.recommender.GenericBooleanPrefItemBasedRecommender;

public class ContentRecommender {

/**
 * @param args
 */
public static void main(String[] args) {
    int userid=10;
       int noOfRecommendations=1;

    try {

        DataModel dm = new FileDataModel(new File("data/dataset.csv"));
        ItemSimilarity sim = new TanimotoCoefficientSimilarity(dm);
        GenericItemBasedRecommender recommender =
            new GenericBooleanPrefItemBasedRecommender(dm,sim);

     List<RecommendedItem> recommendations =
         recommender.recommend(userid, noOfRecommendations);
            for(RecommendedItem recommendation : recommendations)
             {
                System.out.println(recommendation.getItemID());
            }

    }
    catch (IOException e)
    {
        System.out.println("there is an error");
        e.printStackTrace();
    }
   catch (TasteException e)
    {
        System.out.println("error");
        e.printStackTrace();
    }

}

}

When I am running this code, I am not getting any output, although there is no error.I don't know what can be the problem. Can anyone figure out how to implement item-based similarity algorithm with Boolean data-set.

The code and data combination is too small for predicting a solution. But note that you will not get any answer if there are no recommendations made. In this case there will be no exception either. Looks like your code

List<RecommendedItem> recommendations =
     recommender.recommend(userid, noOfRecommendations);

returned no recommendations. You should try with a variety of data.


Predicting time of event from multiple timeseries I apologize for the newbie question, but I'm hoping someone here can point me in the right direction.
I want to predict the time at which an event (x) occurs given several timeseries which I suspect are correlated with the timing of X. My model would take in the full timeseries as input, and output a single value for the time at which event X is predicted to have occurred. Note that I'm not trying to predict the timing of a future event, but the timing of a past event. When using the model, I will have access to the entire timeseries. Here's a quick sketch of what the data looks like:

My dataset consists of many examples like the above, that is, several timeseries within a set window along with the time of the event.
Can something like an LSTM model be trained for this task? After doing a bit of digging it looks like LSTM and RNN models are good for predicting the next point in a timeseries, not so much outputting the time of an event like I've described.
I am able to fairly consistently fit curves to some of the timeseries, so I tried training some tree-based ML algorithms on the coefficients of these fitted curves with little success. I'm hoping to try a more sophisticated timeseries-based approach next.
I would appreciate being pointed in the right direction here - ie. what type of model or approach should I be looking into.

This kind of problem can be modelled in two different ways, which I call time-of-event and time-to-event. The first is what you have tried out so far, and illustrated in your picture.
Time-of-event
Feed in the entire time-series, and use as target the time of the event. Target can be encoded as the second from start of series, a percentage of the duration of entire series etc.
Advantages:

Simple. Standard regression setup.

Disadvantages:

Time-series might be very long, difficult for model to learn.
Few instances in training set. 1 per time series.
Single prediction for one whole series, no distribution of values

Time-to-event
Divide the time-series into multiple overlapping analysis windows. For each window, compute the distance (in time) to the event of interest, and use this as the target. The target can be expressed in seconds, or number of time-steps. Note that it may be positive (in future, relative to window) or negative (in the past, relative to the window).
At training time, the simplest setup is to treat each such analysis-window,time-target pair is treated as independent instances.
At prediction time each individual analysis window then passed through the model. Prediction for each individual window then has to be aggregated to give a single prediction. In the simplest case this can be just the mean or median. More advanced cases would for example weight predictions based on their time (maybe
Advantages:

Multiplies the number of training instances in dataset
Model input has lower data complexity. Since only seeing section of time
Can get predictions in real-time/streaming fashion
Get multiple predictions, forming a kind of distribution

Disadvantages

Adds another function block, aggregating of windowed predictions into a single prediction
Adds additional hyper-parameters. Window length, overlap, aggregation method
Overlap means that model runs many times on each data point. Might increase compute time for predictions

The window length can be quite short relative to entire time series. Ex a 15 minute series with 60 second windows using 5 second forwarding between each window (91.66% overlap). That would give 15min*60s/5s = 180 instances per series.
Or the window length can be quite close to the entire time series. Ex a 15 minute series with 10 minute windows and 1 minute forwarding between each window (90% overlap). Giving 15min/1min = 15 instances per series.
What works best here is highly problem and data dependent.
The use of multiple analysis windows like this is effectively a data augmentation technique (time-shift). Since it is applied also at prediction time, it is also an example of Test-Time Augmentation (TTA).
Models
I would always recommend starting with a non-time series model as a baseline. Summarize the features inside each window (mean,std,(max-min) of each variable), and use Linear Regression, Random Forest to see how it performs. This gives an indication of the complexity of the problem. And is super-fast to do some initial hyper-parameters scans on (for example on window length). And you should plot these features/predictions as part of Exploratory Data Analysis / Error Analysis!
After that I would try a simple LSTM / GRU model, or maybe a 1D CNN.


Determine when entry/series of entries are outliers A common question I face is this:

I have a stream of incoming data. Let's call it a vector of entries where each entry represents a value. As I'm getting this stream of entries that is getting added to the vector, I want to figure out if one of them is an outlier. So my question is two essentially:

How do you establish a good baseline as you consistently keep getting data?
How do you determine if a value that is appendede to the vector is significantly different than the baseline?

Look into six-sigma techniques: e.g.: http://www.whatissixsigma.net/imr/

In short: create a moving range (moving average +/- 3 times moving std.deviation).

Points outside that range can be considered outliers and need inspection (not just discarding).


precision@k and recall@k Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems.
I checked many sources, I understood everything I could not understand a point.
One more thing,
Every source is different between each other in terms of calculation ( 1 , 2, 3, 4 )
let's get this example

I'll give you an example so you can explain to me
Let's say we have 5 users. We are trying to give location recommendations for the next visit to each user.  We are analyzing users' historical check-in data and we are giving recommendation for the next visit.
User 1 is visiting: Museum1, Park1, Night Club, "?" (What is next)
We are trying to find the next visiting locations. Let's say our ground truth "Restaurant"
How can I calculate precision@5 and recall@5?
Extra:
This youtube video is explaining very good (go to 51:45 on video)
What is 5-6 relevant item means? If we are giving recommendation it should be just 1 item that is gonna be relevant for the user. They are trying to make a movie recommendation but they are saying we have 5 relevant movies. What is that mean?

A quick answer.
Note: Checking the references I could access fully, there are no discrepancies between the definitions as long as the terms are translated correctly.
Some definitions:
Relevant items: Are items the user(s) have themselves established as relevant with their actions.
Recommended items: Are items which the recomendation system predicts will be relevant to the user(s).
Concerning your example:

Let’s understand the definitions of recall@k and precision@k, assume
we are providing $5$ recommendations in this order — 1 0 1 0 1, where 1
represents relevant and 0 irrelevant. So the precision@k at different
values of k will be precision@3 is $2/3$, precision@4 is $2/4$, and
precision@5 is $3/5$. The recall@k would be, recall@3 is $2/3$, recall@4
is $2/3$, and recall@5 is $3/3$.


What are some (non image) datasets where deep networks are required for good performance? It is well known that deep networks tend to outperform shallow networks and other classical machine learning techniques such as boosting on learning tasks involving images. I believe this is because extracting useful high-level features from low-level pixel intensities requires deep models.

I'm curious to know if there are any non-image data sets (e.g., tabular datasets that are typically found in UCI repository) where deep networks are required for good performance and where shallow networks, boosting tend to perform poorly.

Welcome to the community!

In text-related tasks, DL have shown amazing breakthrough e.g. in NLU for reading comprehension or question answering. If you have heard about Google's BERT, that was one of big steps which, to some extent, pushed the boundaries of SOTA in NLP/NLU last year and currently, a heavy wave is going on doing research on different NLP/NLU tasks mostly based on either the idea coming from BERT or directly from its pretrained models.

The quora QA dataset, SQuAD dataset, etc. are great starting points to play with bert and its successors and see how great the idea was.

Hope it helps!


Can generic data sets be suitable for specific sentiment analysis I have used the stanford movie review dataset for creating a experimentation of sentiment analysis.

Managed to create a basic application on top of Spark using the Naive bayes classification algorithm.

Steps that I did for pre-processing from the spark ML pipeline

Tokenization
Bigrams

The provided dataset above also has a testing dataset with itself which is separate of the training set. After training it I got around 97% accuracy which I believe is pretty good for Naive bayes.

Now can I use this ML model to predict for other texts such as email/chat etc., My guess is that this dataset has a large enough collection of words to perform good predictions and certain english words regardless of the business context like "I dont like this","This does not look good" is the same across different domains such as Movies/Emails/Chats etc.

I have not done the experiment since the data that I need to get hold of belongs to the customer and due to privacy restrictions we cannot access the data.

Any help/guidance would be much appreciated.

It depends.

You're basically asking if your sample (training data) is representative of the population (all written words).

Are you doing sentiment analysis on movie reviews? It'll work great.
Are you doing sentiment analysis on TV reviews? It'll probably work great.
Are you doing sentiment analysis on book reviews? I would give better than 50-50 odds it'll work.
Are you doing sentiment analysis on Twitter posts? Now we're getting shaky. People tend to write much less, use less formal language, and use more emojis which your movie review model wouldn't have seen.

That being said, there are definitely "generic" sentiment analysis services like here. Try out your model against Algorithmia on what you would consider a generic set of data (e.g. a bunch of tweets) and see how it does.


Standardization of variables in log-scale I have a doubt regarding standardization, I have to use a multivariate regression and one of the variables is in log-scale. Is using standardization enough to re-scale the log variable, or is it necessary to apply a transformation to a linear scale and then apply standardization?

There are many ways to do feature scaling, and it's hard to know which one is better (if you find any white paper on this please comment below). I would say that if you have outliers, nonlinear transformers such as applying the log are good options, but there are many others (see this great sklearn documentation page on this).

Regarding your question, it depends. You must do some exploratory analysis to check the variable's distribution on each configuration. If you see that your variable's histogram is "good enough" (in terms of a well-defined distribution -- sorry to use too general language), then go ahead and use it without further standardizations.


How to select the learned model using $k$-fold cross validation? Let us consider a case where $1000$ data is given, i.e., the data set $U=\{x_1, \ldots, x_{1000}\}.$

When we want to use $k$-fold validation scheme, we first divide the data set into $k$ groups.

With out loss of generality, the parameter $k$ is assumed to be $10$.

Hence, we have $S_1=\{x_1, \ldots, x_{100}\}$, $S_2=\{x_{101}, \ldots, x_{200}\}$, $\ldots$, $S_{10}=\{x_{901}, \ldots, x_{1000}\}$.

I can obtain models, $f_k$, by learning a data set $U \setminus S_k$ for $k=1, 2, \ldots, 10$.

I can obtain error rates, $r_k$, by testing a data set $S_k$ with $f_k$.

Hence, I can obtain the error rates, $r$, by averaging $r_k$'s, i.e., $\sum_{k=1}^{10}r_k/10$.

I understand the $k$-fold cross validation so far.

Most materials I've seen just say the error rate averaged by $k$ scenarios in $k$-fold cross validation. However, they do no t say about $f_k$'s.

However, in this case, which model do I have to use among all $f_k$'s?

$k$-fold is just performed to obtain a measure of your accuracy, as using the training accuracy is generally a too optimistic measure of the accuracy. If you want to deploy a final model, what is recommended is to train a last model with all the data. In fact, when you compare two models, $f$ and $g$, what you do is obtain two error rates $r_f$ and $r_g$ by cross-validation, and you keep the model with the lowest error rate. After that, if the model with the lowest error rate is $f$, you retrain $f$ with all your data.

To sum up, $k$-fold cross-validation is a method to give measures of the performance of the model, if you want to obtain the best model just train it with all your data.


How does DQN solve Open AI Cartpole - v0? Context

I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with the basic principle of the env.

Unlike the tutorials, I converted the state space representation into just the 1x4 returned state as opposed to an image. Also, I converted the action output to be a binned output. So the action 1x2 becomes 3x2 when binning is set to 3. So instead of getting the max action row-wise, I get a max action column-wise. I am using fixed targeting (training a primary and target DQN).

Question / Concern

My main issue with the env in general is that keeping the pole vertical is no different to the DQN as holding the pole to near-failure. How does the DQN get better if it is getting +1 reward regardless? My hypothesis is that keeping the pole tilted creates more samples in the memory. Then after, when optimizing the model, the tilted pole states get higher reward than the vertical pole states since there is a higher distribution of tilted pole states. How are we expected to expect a DQN to do well with this kind of reward set up? Wouldn't it be better to have Cart pole produce +1 reward only for if the pole is near vertical?

Extra Information

The goal here is to use the cart-pole for debugging a RL model, then shift it to multi-joint robot control.

The state is normalized to the expected min/maxes:
Current State [[0.45564668 0.51196048 0.53126856 0.52450375]] The action input is Raw Action tensor([[0.9477, 0.9471]])
Bin Action [1.0, 0.0]. I am just using simplest action rep for testing below. I have also tested double dqn's, dueling dqns, and the use of PER. I have also been testing dropping the state space down to 1x1 via just inputting the angle of the pole.

X axis is the number of steps during an episode. Y axis is the number of episodes.

You are mixing up two concepts from reinforcement learning, reward and return (aka utility)

Rewards are used to identify or specify goals of the agent. Whilst you can change them to help an agent focus on useful heuristics of the problem, it is more usual, especially in test/toy problems to have them very simple. In the case of CartPole, there is a positive reward for "not falling over" which importantly ends when the episode ends.
Returns (or utility) are what the agent learns to maximise. A return is typically the sum of all rewards, and might be discounted to prevent infinite results. In the case of CartPole, this means that the longer the agent can balance the pole into the future, the larger the return is.

With Q-learning, the action values predict the expected future return. So it doesn't matter that the rewards are dense. It matters how long the agent can keep balance going into the long term, the longer the better, because the return will be higher. A combination of state and actions that the agent associates with longer-lasting not failing will predict a larger return and be chosen in preference to shorter-term success. This is how the Q-learning agent handles a situation with dense positive rewards that may end on a mistake.

In practice, the OpenAI Gym CartPoleV0 environment does take a small liberty. Episodes will end at a fixed step in future. This is not available in the state information, and technically makes the problem non-Markov. However, it is possible to get away with this provided the timespan for maintaining stability is shorter than the maximum possible episode.


BERT is running out of memory in forward pass for my dictionary Running code from this answer, my BERT is running out for my 4k words dictionary.
I don't need to do anything with these words yet, just make embeddings for my data. So, using this exactly:
from transformers import BertModel, BertTokenizer

model = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

encoded_inputs = tokenizer(labels, padding = True, truncation = True, return_tensors = 'pt')

ids = encoded_inputs['input_ids']
mask = encoded_inputs['attention_mask']

output = model(ids, mask)
lab_embeddings = output.last_hidden_state.tolist()

gives me memory leakage. How can I manage this with batching since I don't have labels for classification or something like that?

It is likely to be independent of dictionary. Loading BERT model and running a forward pass has its own memory requirements. How did you figure that it is memory leak?
Try visualizing memory footprints on each step in your code by using some break points. It will give you clear idea about the hardware requirements and memory leak if any.


Dealing with diverse groups in regression What happens if a certain dataset contains different "groups" that follow different linear models?

For example, let's imagine that examining the scatterplot of a certain feature $x_i$ against $y$ we can see that some points follow a linear relationship with a coefficient $\beta_A<0$ while other points clearly have $\beta_B>0$. We can infer that these points belong to two different populations, population $A$ responds negatively to high values of feature $x_i$ while population $B$ responds positively. We then create a categorical feature (or one hot encoding) to show which population each row belongs to.

Is splitting the dataset required or are commonly used algorithms able to recognize the different relations between features from different categorical variables?

You can't really do that, there may be some factor which binds certain "groups" of data together, but there are many reasons for this. Your relationship may be nonlinear, or the "groups" of data may represent subjects / objects, where a stronger correlation exists. Unless you know for a fact that these points belong to different populations you shouldn't do that, use the data that you have to model these groupings.


How to get Euclidean Distance between feature maps I'm trying to find "keyframes" within a video using this paper however I'm a little new to machine learning and I'm stuck on the distance vector step. The goal is to calculate the distance vector between consecutive frames to detect a big context change indicating a keyframe. here's what I've done:
Using Googlenet pre-trained model I extracted a (1024,7,7) feature map, so a vector of 7x7 matrices for every other frame in the video.
Now I want to calculate the euclidean distance between two consecutive frames but I'm not sure how that would be.
My intuition is that the distance vector would also be (1024,7,7).
The last step would be to apply a "convolution of 4-window of last distance values with vector [0.1, 0.1, 0.1, 0.99]" on the distance vector?? I don't understand this step either.
Any help or guidance would be appreciated!

The Euclidean between two images $p$ and $q$ can be calculated as follows:
$d(p, q) = \sqrt{(q_1 - p_1)^2 + (q_2 - p_2)^2 + ... + (q_{49} - p_{49})^2}$
which is the distance between the 49 (7x7) features of the two images.
This should then give you a vector of shape (1024, 1) where each value is the Euclidean distance of the feature maps of the previous image, with the first one being all NA since it's the first image. Then a convolution is applied with a window of 4 with the vector/kernel [0.1, 0.1, 0.1, 0.99], which basically multiplies 4 values of the (1024, 1) vector by the kernel values and adds them up. I.e. given the first 4 values of the (1024, 1) vector of [0.3, 0.5, 0.2, 0.4] the result of this multiplication would be:
$0.3 * 0.1 + 0.5 * 0.1 + 0.2 * 0.1 + 0.99 * 0.4 = 0.03 + 0.05 + 0.02 + 3.96 = 4.06$


Sampling for multi categorical variable My hypothesis h depends on multiple categorical variables (a,b,c) each with their corresponding set of possible values (A,B,C). Now each of my data point exist in this space where I have no control over the values (observational data).

  For e.g. Hypothesis to predict user shopping probability say depends
  on (Age, Country, Gender, Devicetype etc.)

How could I sample the above data set so that it would give me a good representation. The techniques I have learned from the books very well apply to one dimension but that is a rare case in practice. If I sample across one dimension my other dimensions will be heavily skewed towards some values. Is there any standard algorithm to give good sampling?

Let me give you some pointers (assuming that I'm right on this, which might not necessarily be true, so proceed with caution :-). First, I'd figure out the applicable terminology. It seems to me that your case can be categorized as multivariate sampling from a categorical distribution (see this section on categorical distribution sampling). Perhaps, the simplest approach to it is to use R ecosystem's rich functionality. In particular, standard stats package contains rmultinom function (link).

If you need more complex types of sampling, there are other packages that might be worth exploring, for example sampling (link), miscF (link), offering rMultinom function (link). If your complex sampling is focused on survey data, consider reading this interesting paper "Complex Sampling and R" by Thomas Lumley.

If you use languages other than R, check multinomial function from Python's numpy package and, for Stata, this blog post. Finally, if you are interested in Bayesian statistics, the following two documents seems to be relevant: this blog post and this survey paper. Hope this helps.


Autoencoder feature extraction plateau I am working with a large dataset (approximately 55K observations x 11K features) and trying to perform dimensionality reduction to about 150 features. So far, I tried PCA, LDA, and autoencoder. The autoencoder that I tried was 12000-8000-5000-100-500-250-150-, all layers were Dense with sigmoid activation, except the final layer, which had a linear activation in order to reproduce the continuous data from the input. The autoencoder loss effectively plateaus after 10-15 epochs, regardless of the learning rate (here, I used the ReduceLROnPlateau feature in Keras). For the record, I am normalizing each feature by z-score prior to the training. I'm not sure how to get this loss to stop reaching a plateau.
Should my next attempt be to use a convolutional neural network on this dataset to see if I can reduce the dimensionality more successfully? Are there any pre-trained convolutional autoencoders that I could use? Training a convolutional autoencorder from scratch seems to require quite a bit of memory and time, but if I could work off of a pre-trained CNN autoencoder this might save me memory and time.

A convolutional autoencoder will only make sense if you work with images (2D signals) or time series (1D signals). Convolutions identify local patterns in data, if this is not the case in your data it will most likely not solve your problem.
Using pre-trained AE will only help, if it was trained on similar data. Similar data in this case does not refer to the data type, but rather to what the data represents. If you have an AE which was trained to compress images of cats it will not work well on images of chairs, because cats and chairs do not share the same features. Although if you like to compress images of dogs you could use the weights of the AE for cats as a starting point (Transfer Learning).
What kind of loss are you using? MSE or Cross-entropy? Speaking from my experience, using cross-entropy yields better results (although this is problem dependent). Another issue could be vanishing gradients which can happen in very deep networks and with activation functions like sigmoid. What you can do is reduce the depth of your network, replace the sigmoid with ReLU and maybe try a different optimizer.
In any case PCA is a safe bet. It's linear, deterministic, well studied and quicker to use than to train a NN. Whatever method you use, you can use PCA as benchmark to see if your method beats it. Although with the size of your data you may run into memory issues.


Can I use sentiment analysis techniques and text processing for detection social bot in online social networks? I want to study in field of " Detecting fake account on online social network by content/sentiment analysis " for Master's Thesis.is this possible?

Yes, because it has been done before:

http://dl.acm.org/citation.cfm?id=2808779


Keras next(); what does (2, 256, 128, 128, 3) mean I have used the next() method on a Keras generator. I then turned this into a numpy array to get the shape:
data = generator.next()
data = np.array(data)
print(data.shape)

>>> (2, 256, 128, 128, 3)

256 is my batch size, and (128, 128, 3) is my image size. But what does 2 represent?

Keras generator returns a Tuple for data and label. So is that 2.
First you should unpack it with a tuple then use image and labels.

yielding tuples of (x, y) where x is a numpy array containing a batch of images with shape (batch_size, *target_size, channels) and y is a numpy array of corresponding labels

img, labels = traindata.next()
img.shape, labels.shape

Output - ((16, 224, 224, 3), (16, 2))
We are also supposed to return a Tuple of batches of images/labels in case we build a custom generator for Keras.


Query on unstable loss curves for RNN I’m currently building sequence models for forecasting, and have tried using RNNs, LSTMs, and GRUs.

Something unusual I noticed was the highly unstable loss curves, where the loss sometimes goes back to the loss level in the first few epochs. Interesting, the severity of this decreases from RNNs to LSTMs to GRUs.

Would anyone have an idea why this occurs?

For reference, here are the loss curves for the following models, across 500 epochs.

There can be some other factors that affect this, such as using simulated annealing (in a NN context) or other learning rate schedules. Are you using a specific LR schedule?

A schedule might be that the LR decreases by 50%, every time the validation loss of 5 epochs in a row does not decrease. This will help get closer and closer to a minimum of the loss. However, we know it is possible to get stuck within a local minimum, which may be far from optimal, so we can shake things up by increasing the LR once again, which will essentially throw the algorithm our of tyhe local minima and on its way to a new minima (at least that is the hope). This kind of schedule often produces loss curves like the ones you see.

Another alternative is simple that your batch sizes are quite small, and every once ina  while, you get a batch that consists of example which your model really struggles with, so the loss for that batch (and so the epoch) would spike in comparison to other epochs.

A final idea - thinking more about your data - if it is time-series e.g. of stock prices or weather - there could be a regime change/shift. Meaning that the underlying function or system suddenly switches to a new pattern. Something like this could throw your model off the scent for a while, and so produce bumps in the loss curve.

A small point on terminology: LSTM and GRU architectures are themselves RNNs. A recurrent network is one in which connections do not only move forward in a network, but can also go sidewards across a layer or indeed backwards. So it is a more general term, whereas LSTM/GRU layers are specific examples of RNNs.

If you can say a little more about your three model architectures, perhaps it might be clearer which names to use - and maybe even better understand these loss curves  :-)


computer science student - asking for some machine learning guiding (voice cloning) I have chosen my synopsis topic for my second last semester.
I want to make a text-to-speech program, that speaks with the voice of a game character.
I have worked with machine learning in my class, but nothing that covers audio or text-to-speech.
I know how big of a project I have taken on my shoulders, but that is just what I need.
I don't know where to start at all, since there is way too much information out there. In other programming areas, I'm usually pretty good at finding information, but I just don't know enough about ML yet to do that here (plus the keywords used online kind of contradict each other).
I'm not sure which tools I need to look at. Where should I begin?
I have 68.871 audio files, which is about 76 hours of audio in total.
The files are of different lengths, they start at around under a second and end around 49 mins.
I have not heard all the audio myself, but I'm pretty sure all of it is completely clean.
(I know that I probably can't feed any model with 68.871 files, that could all be of different lengths, so I think I have to pick out a smaller sample size, with more equal lengths)
I would like to use these files as audio samples, so a model can learn how to speak like this character.
Any push in the right direction would be deeply appreciated.

Try to look into existing text-to-speech/speech synthesis. This link list multiple different models/approaches in this domain. When trying to look at state-of-the-art (SOTA) methods in this domain I find this GitHub page on which a lot of papers can be found on audio and speech. A very good (SOTA) model/architecture seems to be Tacotron 2/WaveNet. Hope this helps you with your project.


tuning a convolution neural net, sample size I keep reading that convolution neural net (CNN) performs best with lots and lots (100k+) of data.  Is there any rule of thumb, or lower limit for data size during the grid search phase?

For example, if I run a CNN with 100 data points, vary just one parameter (say add an extra layer, or increase a filter size), and get better results, can I reasonably expect better results with those parameters during the actual training phase?

If you use pre-trained weights, you need significantly lesser data as the initial layers have already learned from a ton of data and you just need to fine tune the later ones.

What you said is not true, you can train on CIFAR10 and get 90%+ and that is not 100k+. It depends on the complexity of the data and how similar the features are. If they are easily Seperable -less data. If the disctintions are harder then the model needs a lot of examples to figure out which of features are seperate.

I would say you could IF you sample was representive of the population.


Photorealistic synthetic data for object segmentation Let's say that we have a very small labelled dataset for instance segmentation and there is a photorealistic physics engine available that can produce synthetic data for us. Looking on the web I haven't found any paper about a similar issue.
Has anyone tried to use solely a photorealistic synthetic dataset or mixed with real data?
If so, I would appreciate your conclusions or any relevant work that you can cite here.

I used such an approach at the Computer Vision Lab at TU Delft for my Bachelor thesis.
The goal was to analyze different aspects of the same problem: identifying LEGO bricks in a photograph of an unsorted pile of LEGO bricks. We built a small real dataset as well as a large (relatively) photorealistic synthetic dataset using Blender.
Personally, I looked at it from the explainable artificial intelligence (XAI) point of view, but some of my team mates did have objectives closer to yours (image segmentation, object detection, classification).
You can find a description which is a bit more detailed here, in section 4.1.
I can edit this answer with more details if you're interested.


Categorical data - how to handle Few questions on categorical data. Need suggestions / pointers:

How can we check for correlation between categorical features and target or between the features themselves?
How about correlation between cat and num features?
How can we find cases where the target has a dependency on a combination of categorical features, but may not show corr with individual features?
Can we handle scenarios where, test data has a categorical feature value that is not present in training data?

If the categorical variable is binary(like e.g sex) you try
Point biserial correlation coefficient.
Or recode levels of var(woman->1, man->0) and use pearson correlation. Recoding it's a risky way because of you indicate order(woman>man). You should be aware of that.

Also $\chi^{2}$ test is used to determine whether an association (or relationship) between 2 categorical variables.

To find association between categorical and numerical variable
try perform One-way ANOVA test.


Please suggest some simple functions that I can use to test my neural network implementation I am a beginner and I have developed code in Octave to train a neural network (as part of Andrew Ng's coursera course).

Now I would like to generate some datasets in Excel for simple mathematical functions and try and train my neural network implementation.

I tried y = x^2 and y = sin(x), but my training accuracy is only around 67%.

I have varied the number of hidden layer units, varied the lambda, but still don't get good results. Where could I be wrong?

Without code you could have made to many mistakes to answer your question directly.

However, I suggest two things:

Gradient checking (e.g. this explanation, but there are many more - pretty sure also in the course)
XOR problem with a 2:2:1 network with only sigmoid activations. Plot the error surface. Compare the decision surface with that of this interactive demo.

It should look similar to this:

However, please keep in mind that the problem might not be your implementation, but rather the network architecture / hyperparameters such as the number of epochs you're training or the training data.

Also, very important: I doubt that you will get good results for $y = x^2$, except if $x$ is restricted to $[-1,1]$ or something similar simple. Please keep the domain of your output layer in mind.


AttributeError: module 'tensorflow.python.keras.utils' has no attribute 'to_categorical' I'm trying to run the code below in my Jupyter Notebook.
I get:

AttributeError: module 'tensorflow.python.keras.utils' has no
attribute 'to_categorical'

This is code from Kaggle tutorial. I have installed Keras and Tensorflow.
 import numpy as np
    import pandas as pd
    from sklearn.model_selection import train_test_split
    from tensorflow.python import keras
    from tensorflow.python.keras.models import Sequential
    from tensorflow.python.keras.layers import Dense, Flatten, Conv2D, Dropout

      img_rows, img_cols = 28, 28
    num_classes = 10

    def data_prep(raw):
        out_y = keras.utils.to_categorical(raw.label, num_classes)

        num_images = raw.shape[0]
        x_as_array = raw.values[:,1:]
        x_shaped_array = x_as_array.reshape(num_images, img_rows, img_cols, 1)
        out_x = x_shaped_array / 255
        return out_x, out_y

    raw_data = pd.read_csv('trainMNIST.csv')

    x, y = data_prep(raw_data)

    model = Sequential()
    model.add(Conv2D(20, kernel_size=(3, 3),
                     activation='relu',
                     input_shape=(img_rows, img_cols, 1)))
    model.add(Conv2D(20, kernel_size=(3, 3), activation='relu'))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))

    model.compile(loss=keras.losses.categorical_crossentropy,
                  optimizer='adam',
                  metrics=['accuracy'])
    model.fit(x, y,
              batch_size=128,
              epochs=2,
              validation_split = 0.2)

Use keras>=2.2 and tensorflow >=1.14 to resolve the issue.


Logic Check: Building a SKLearn Pipeline I am new to the concept of building a pipeline in SKLearn and would appreciate some sense-checking to ensure that I am not leaking info from my training sets into my test set.

Background:

I have a sparse, high-dimensional data-set (370x1000) with a continuous variable as the target. At present, I have been running a random forest regression on all the features with a 90/10 split, followed by parameter tuning via grid search on the training set, followed by 5-fold cross validation on the optimized model (with the entire dataset).

Problems with this approach:

As I understand the situation, there are a number of things I am doing that might be harming the model and introducing undesired bias. Specifically, my concerns are:

As I am tuning parameters only on the initial train/test split, I am not accounting for other split combinations that arise during the K-Fold CV. Might optimal settings for fold 1 be different for fold 2? Intuitively I would assume so.
I am not doing any feature selection that might remove otherwise redundant features and shorten my feature-space (I know RF is generally quite good with high-dimensional spaces but I would still like to try). Some suggestions I have read have included removing features with very low variance. But I find myself in the same conundrum as above: if I remove low variance features from the original training set I am not accounting for other combinations during K-fold. Comparatively, if I remove all low-var features prior to splitting the data I am surely leaking info between the train/test states.
An alternative approach I have seen is recursive feature selection (with CV as per SKLearn) - this looks promising, as I think it means that it will partition the data-set in folds and conduct RFE on each fold, presumably giving me an averaged score of the best number of features to keep.

My possible solution:

I have been doing some reading around Pipelines in SKLearn and think that might be the way to go. My understanding is that an advantage of a pipeline is that i can stack transforms together, preserving the individual folds and allowing me to address the problems I have detailed above. What I am considering, and what I would appreciate sense-checking from anyone with more experience, is the following:

As the data-set is small, I would not split the data-set in the conventional train/test split manner, but would use K-Fold across the whole data-set.
Run a RF (using default params) with K-Fold to get a baseline level of performance.
Create a pipeline whereby I (3.1) create folds, (3.2) and then within each fold find the optimal number of features to keep, (3.3) tune hyper-parameters for that fold, and finally (3.4) predict the values in the test fold.

As you might be able to tell I am struggling to get to grips with what order things should go in, and whether step 3 is actually what a Pipeline does. If someone can provide pointers/recommendations/corrections it would be appreciated.

You are on the right path. It appears you might have analysis paralysis. You should start building, then see what works and what does not work.
Here is code to get you started:
from sklearn.ensemble             import RandomForestRegressor
from sklearn.feature_selection    import VarianceThreshold
from sklearn.model_selection      import GridSearchCV
from sklearn.pipeline             import Pipeline

regressor = Pipeline([('vt', VarianceThreshold(threshold=0.0)),
                      ('rf', RandomForestRegressor())])

gsvc = GridSearchCV(regressor, {'vt__threshold':    [0, .1, .5, .7, 1],
                                'rf__n_estimators': [10, 100, 1_000],
                                'rf__max_depth':    [2, 10, 100]})


Word2Vec: Identifying many-to-one relationships between words Standard introductory examples in Word2Vec, like king - queen = man - woman and tokyo - japan = london - uk, involve one-to-one relationships between words: Tokyo is the exclusive capital of Japan.
More generally, we might want to test for many-to-one relationships: e.g. we might want to ask if Kyoto is a city in Japan. I presume we are still interested in vectors of the form kyoto - japan, houston - us, etc., but these vectors are no longer equal.
Do these "relationship vectors" form a particularly interesting vector space? Do they sample some known distribution? How can I check a many-to-one relationship from the word embeddings?

Those vector relations are not exact. Rest assured that king - queen ≠ man - woman. What we do is finding the closest vectors to the result of king - man + woman. One of the closest vectors is queen.
Nevertheless, when we try the "parallelogram approach" to verify word relations, in most cases, the closest vector is the original one. The fact that we disregard the original word as a sensible result is due to the original implementation of word2vec, as shown in the study Fair is Better than Sensational:
Man is to Doctor as Woman is to Doctor. This implementation quirk is actually one of the sources of gender bias attributed to word embeddings.
There are other articles studying the limitations of the analogies based on word embeddings, like this and this.
That being said, there are also studies of the relationships that can be found in word representation spaces, like this one, which studies different types of relationships, e.g.  class-inclussion, part-whole, attributes, etc. You may find the main figure of the article interesting:


Learning comparisons Suppose I want (the machine) to learn the ">" function, and my training set is a collection of pairs of numbers $(n_1, n_2)$ with the output True if $n_1 > n_2$ and False otherwise. Which method is the most likely to work (actually, my secret goal is a form of the Netflix challenge - I have a set of $n$-tuples of real and boolean values, and I want to rank them based on some training rank data).

This has surely been studied a lot...

The classical solution is to try a one-class SVM applied only to the elements ranked above the others - you can read some papers about it looking by SVM rank or SVM ranking that will show you why that works.


where does the database created in spark go? I am following this tutorial:https://docs.microsoft.com/en-us/learn/modules/intro-to-azure-databricks/4-using-notebooks

In this tutorial we create a database like this:

%sql

CREATE DATABASE IF NOT EXISTS Databricks;
USE Databricks;

CREATE TABLE IF NOT EXISTS AirlineFlight
USING CSV
OPTIONS (
  header="true",
  delimiter=",",
  inferSchema="true",
  path="dbfs:/mnt/training/asa/flights/small.csv"
);

CACHE TABLE AirlineFlight;

SELECT * FROM AirlineFlight;

Where is this database created? moreover there is a question asked

Question: Which of the following are good applications for Apache Spark? (Select all that apply.)

Querying, exploring, and analyzing very large files and data sets
Joining data lakes
Machine learning and predictive analytics
Processing streaming data
Graph analytics
Overnight batch processing of very large files
Updating individual records in a database
Answer: All but #7. Apache Spark uses SQL to read and performs analysis on large files, but it is not a Database.

If we can create a database using spark then why can't we change its records too

Where is this database created?

A powerful paradigm in modern data storage and processing is the separation of compute and storage. Building systems with decoupled compute and storage has benefits associated with scalability, availability, and cost.
Apache Spark loads and performs computation on the data - it is a distributed data processing engine. It does not handle permanent storage. In Databricks (you are using databricks documentation) data is often stored in Delta Lake, which is specifically designed to work with Spark. However, Spark can work with data stored in many other ways, such as other cloud storage (eg Amazon S3, Azure Blob), traditional SQL databases, NoSQL databases, HDFS and many more.


How to migrate R decision tree to Java I have trained a conditional inference decision tree in R using library party with function ctree and saved the model in an .Rda file.

I need to migrate this model from R to Java so that I can utilise the tree to make predictions in a Java environment. Can someone please point me in the right direction on how I can do this?

You could convert your decision tree model to the PMML format. In Java you could use JPMML to parse/read the model and predict.


Does modeling with Random Forests require cross-validation? As far as I've seen, opinions tend to differ about this. Best practice would certainly dictate using cross-validation (especially if comparing RFs with other algorithms on the same dataset). On the other hand, the original source states that the fact OOB error is calculated during model training is enough of an indicator of test set performance. Even Trevor Hastie, in a relatively recent talks says that "Random Forests provide free cross-validation". Intuitively, this makes sense to me, if training and trying to improve a RF-based model on one dataset.

Can someone please lay out the arguments for and against the need for cross-validation with random forests?

By default random forest picks up 2/3rd data for training and rest for testing for regression and almost 70% data for training and rest for testing during classification.By principle since it randomizes the variable selection during each tree split it's not prone to overfit unlike other models.However if you want to use CV using nfolds in sklearn you can still use the concept of hold out set such as oob_score(out of bag)=True which shows model performance with or without using CV.
So in a nutshell using oob_score=True with or without nfolds can itself tell whether using CV is good for your data.Generally if your target is following a certain distribution and you don't have much observation data with you then using CV will not give much improvement.


Estimating Predictive Uncertainty for unlabeled data I am trying to estimate the predictive uncertainty for a deep neural network. While I do have a labeled training set, I´m trying to measure uncertainty for some unlabeled production data.

This paper proposes the use of Deep Ensembles and Adversarial Training to compute a measurement of uncertainty. However, it uses Brier Score as a metric which requires me to know the real label of my production data.
Is there a similar way or metric which does not require labeled data?

Another approach was described by Yarin Gal utilizing Monte Carlo Dropout. However I can´t get any useful results using that technique on my model.

I want to use my model in an online learning task. As the given data may very over time, I need to detect examples where my model is highly uncertain, so I can manually classify those examples.

See Authors comment in the thread here

So by default the dropout is only used in training time. Then in test time the dropout probability is set to 0. But there is a easy way to enable it so that it stays 'on' even during the test time.
    import keras

inputs = keras.Input(shape=(10,))
x = keras.layers.Dense(3)(inputs)
outputs = keras.layers.Dropout(0.5)(x, training=True)

model = keras.Model(inputs, outputs)

By setting training=True it remains active in during test time. This can be called Monte Carlo dropout or more popularly dropout at inference time. I hope it helps.


Q-learning when minimising a total cost instead of maximising a total reward I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a reward.

Which one is better:

Initializing Q-values for all actions with zeros, then getting the agent to learn the actions that maximize the Q-values, and later filter out the actions with minimum Q-values. The Q-values update would then be:

q_dict['state1']['act1'] +=
    r + (max([q_dict['state2'][u] for u in q_dict['state2']]))

Initializing Q-values with a big number then getting the agent to learn actions that minimize the Q-values, and later filtering out the actions with minimum Q-values. The Q-values update would then be:

q_dict['state1']['act1'] -=
    r + (max([q_dict['state2'][u] for u in q_dict['state2']]))

I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a reward.

The simplest way to do this, without changing anything else about your learning algorithm, is to notice that

$$Reward = -Cost$$

So literally just optimise against the expected return based on sums over your negative costs, using standard Q learning. Everything will work as normal. Your best agent may still end up with a negative expected return (and negative Q values), but maximising this should still restult in an optimal policy.

If you really must use minimising cost as your objective for some reason, then there are a few small changes you need to make for that to work with Q learning.

The definition of your Q function becomes expected discounted sum of future costs:

$$Q(s,a) = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty} \gamma^k C_{t+k+1} | S_t=s, A_t=a]$$

(This literally just replaces $R_t$ with $C_t$)

Then your best guess at the optimal policy is the one that minimises expected future costs:

$$\pi(s) = \text{argmin}_a Q(s,a)$$

And your Q-learning update rule is also based on assumed optimisation of minimising the next step

$$Q(s, a) \leftarrow Q(s, a) + \alpha(c + \gamma \text{min}_{a'}[Q(s',a')] - Q(s, a))$$

This doesn't match either of your suggestions. If I was to correct your code, then using rewards (with r = -c), it would look like this:

q_dict[state1][act1] += alpha * (r + max(q_dict[state2].values()) - q_dict[state1][act1])

where alpha is the learning rate, and I assume there is no discounting (so it must be an episodic problem, not continuous).

If you wanted to use cost c directly, and find policies that minimise total cost, then it looks like this:

q_dict[state1][act1] += alpha * (c + min(q_dict[state2].values()) - q_dict[state1][act1])

i.e. you substitute c for r and min for max.

You ideas about having different starting values might make some difference to convergence rates. However, this is not related to whether you use a cost or a reward.

I advise against using the cost directly like this. Although it is simple and will work, whenever you read any RL articles you will have to keep adjusting whether to switch max for min. The convention of maximising the sum (or average) reward is much more common in RL tutorials, and whilst you are learning it, you will save yourself just a little bit of effort to follow this convention.


Classification using xgboost - predictions I was trying to build a 0-1 classifier using xgboost R package. My question is how predictions are made? For example in random forests, trees "vote" against each option and the final prediction is based on majority. As regard xgboost, the regression case is simple since prediction on whole model is equal to sum of predcitions for weak learners (boosted trees), but what about classification?

Does xgboost classifier works the same as in the random forest (I don't think so, since it can return predictive probabilities, not class membership).

The gradient boost algorithm create a set of decision tree.

The prediction process used here use these steps:

for each tree, create a temporary "predicted variable", applying the tree to the new data set.
use a formula to aggregate all these tree. Depending on the model:

bernoulli: 1/(1 + exp(-(intercept + SUM(temporary pred))))
poisson, gamma: exp(intercept + SUM(temporary pred))
adaboost: 1 /(1 + exp(-2*(intercept + SUM(temporary pred))))

The temporary "predicted variable" is a probability, having no sense by its own.

The more tree you have, the more smooth is your prediction.( as for each tree, only a finite set of value is spread across your observations)

The R process is probably optimised, but it is enough to understand the concept.

In the h2o implementation of the gradient boost, the output is a flag 0/1.
I think the F1 score is used by default to convert probability into flag. I'll do some search/test to confirm that.

In that same implementation, one of the default output for a binary outcome is a confusion matrix, which is a great way to assess your model ( and open a whole new bunch of interrogations).

The intercept is "the initial predicted value to which trees make adjustments". Basically,just an initial adjustment.

In addition: h2o.gbm documentation


Cosine Similarity for Ratings Recommendations? Why use it? Lets say I have a database of users who rate different products on a scale of 1-5. Our recommendation engine recommends products to users based on the preferences of other users who are highly similar. My first approach to finding similar users was to use Cosine Similarity, and just treat user ratings as vector components. The main problem with this approach is that it just measures vector angles and doesn't take rating scale or magnitude into consideration.

My question is this:

Are there any drawbacks to just using the percentage difference between the vector components of two vectors as a measure of similarity? What disadvantages, if any, would I encounter if I used that method, instead of Cosine Similarity or Euclidean Distance?

For Example, why not just do this:

n = 5 stars
a = (1,4,4)
b = (2,3,4)

similarity(a,b) = 1 - ( (|1-2|/5) + (|4-3|/5) + (|4-4|/5) ) / 3 = .86667

Instead of Cosine Similarity :

a = (1,4,4)
b = (2,3,4)

CosSimilarity(a,b) =
(1*2)+(4*3)+(4*4) / sqrt( (1^2)+(4^2)+(4^2) ) * sqrt( (2^2)+(3^2)+(4^2) ) = .9697

Rating bias and scale can easily be accounted for by standardization. The point of using  Euclidean similarity metrics in vector space co-embeddings is that it reduces the recommendation problem to one of finding the nearest neighbors, which can be done efficiently both exactly and approximately. What you don't want to do in real-life settings is to have to compare every item/user pair and sort them according to some expensive metric. That just doesn't scale.

One trick is to use an approximation to cull the herd to a managable size of tentative recommendations, then to run your expensive ranking on top of that.

edit: Microsoft Research is presenting a paper that covers this very topic at RecSys right now: Speeding Up the Xbox Recommender System Using a
Euclidean Transformation for Inner-Product Spaces


I am confused about these models In the first problem, it's been told to accept the maximum number of good customers, if at least 98% of the customers that are do not repay their debt correctly identified.
I am confused about what is meant by this? Do I need to train the models and set the threshold value accordingly so that I may obtain 98% accuracy? And for the second one, I need to decrease the threshold value so that accuracy comes down to 85%?
I am stuck in this for a while. Please help me show the correct path. Thanks.
data: German Credit Risk data.
Variables: Several independent variables with the dependent variable "Credit_Risk" which responses in "Good" and "Bad".

The goal is to identify at least 98% of the customers, that do not repay their debt. So the bank can "accept a maximum number of 'good' customers, that can be granted loans" Here the goal is focused on the bad customers.

There should be at least 85% good customers accepted while the side focus is to reject as many bad customers as possible.

I think the difference between 1) and 2) is identifying bad customers in 1) and good customers in 2)


Multimodal distribution and GANs What is intuition behind multimodal distribution? and
How does GANs generate samples from it?

You can think of a multimodal distribution as a union of multiple unimodal distributions. In the case of GANs for image processing, each mode could be a category of images. To significantly simplify what's going on to motivate the idea, if you have a dataset of cat pictures and dog pictures, you can think of that as bimodal. Just cat pictures, that's unimodal. Training a GAN on a multimodal dataset of images means that it should be able to generate members of any image category in your dataset, and it should generate different categories with the same frequency with which they appeared in the training data.

GANs are trained by taking a random vector as input and attempt to construct a feasible member of the data distribution as output. In effect, the GAN learns a (surjective) mapping from the random space onto the multimodal distribution, such that random inputs will generate samples from the multimodal data distribution as outputs.