Game theory in Reinforcement Learning In one of the recent blog post by Deepmind, they have used game theory in Alpha Star algorithm.
Deep Mind Alpha-Star:

Mastering this problem requires breakthroughs in several AI research challenges including:

Game theory: StarCraft is a game where, just like rock-paper-scissors, there is no single best strategy. As such, an AI training process needs to continually explore and expand the frontiers of strategic knowledge.

Where the game theory is applied when it comes to reinforcement learning?

Where the game theory is applied when it comes to reinforcement learning?

It is not used directly in this case, and AlphaStar makes no breakthroughs in game theory. The blog's wording here is not super precise.

The point of the quote was to explain the extra challenge, which occurs in many games where opponents can react to each other's choices and there is often a counter-strategy to any given policy. Rock-paper-scissors is the simplest game that has this challenge, but it is common in many strategy games, as the game designers typically don't want a single best strategy to dominate the game, often going to some lengths to balance options in the game so that more of their game content is used and to keep a level of uncertainty and excitement in the game-playing community.

The actual breakthroughs in regards to the quote in your question, are in finding ways to perform the kinds of long-term exploration that allow for different high-level strategies. Many RL algorithms perform relatively local exploration which would be too weak to keep track of entirely different strategies and decide when to use them.

The way that Deep Mind team approached this is explained in their blog:

  To encourage diversity in the league, each agent has its own learning objective: for example, which competitors should this agent aim to beat, and any additional internal motivations that bias how the agent plays. One agent may have an objective to beat one specific competitor, while another agent may have to beat a whole distribution of competitors [ . . . ]

So Deep Mind have not resolved any of that at a theoretical level, and have not used game theory in any direct sense. However, they have identified the kind of game theory scenario that applies, and have used that in the design, making steps in an engineering sense towards practical solutions.

Other solutions in RL might also apply, such as hierarchical RL for capturing high-level actions as strategies to inform lower-level decisions, or using slow changing noise functions to drive exploration (as opposed to something which changes faster such as epsilon-greedy).

In general, game theory is related to reinforcement learning, in that both construct a formal view of optimising utility:

Game theory is useful for analysing multi-agent scenarios, but generally analyses optimal policies for relatively simple single-step or repeated games.
Reinforcement learning is well-described for single agents, and deals well with sequential decision making, but does not have much quite as much material for dealing with competitive and co-operative multi-agent environments - typically treating other agents as "part of the environment".

There is enough cross-over between the two theories that they can be used to inform each other in an intuitive way, as Deep Mind have done here.

In more tractable game environments, game theory is able to determine stable and effective policies - for instance in rock-paper-scissors, the Nash equilibrium policy (one which players will be punished for moving away from) is randomly selecting each action with 1/3 probability. Note this is not necessarily the optimal policy - that depends on the opponent's behaviour - but it is an expected stable outcome for two rational and capable opponents to arrive at.

If you develop a rock-player-scissor learning bot using RL, and it learns this strategy through self play, then you can be relatively happy that your learning algorithm worked. That would be one way of using RL and game theory together.

Deep Mind don't know the Nash equilibrium of Star Craft strategies, and in fact the strategies are only loosely defined in terms of low-level actions, so it is not clear whether it is possible. The analysis of strategies supplied in the blog (e.g. a "rushing" strategy) are based on observations of the game and adding a human narrative to help understand what is going on. In practice, it is the sampling of opponents each preferring a different strategy or set a particular goal in the game, that trains a single neural-network based bot that has experience of countering multiple strategies and can express actions that optimally beat any strategy that matches patterns it has learned in self-play and observes an opponent using.


Understanding regularisation and a preference for small weights This has been a spot of confusion for me lately upon reading more into regularisation in neural networks.

I have always viewed weights as being a measure of the importance of a feature in a model. E.g. A model of ice cream sales starts off with random weights and eventually learns to increase the weights connected to the temperature feature.

With regularisation (L2, I have been looking at most recently), small weights are favoured more by the model. This makes good sense to me when we are trying to prevent overfitting in our models. The model generalises well between different pieces of input data because the smaller weights give their associated feature less of an impact in the model. This will draw out the true patterns of the model as opposed to a few cases that may skew the weights.

Now my confusion is what if a feature like temperature is a consistent predictor of ice cream sales in our model, what does its associated weights look like after regularisation? Is their impact on the cost function so great that the model doesn't prefer smaller weights instead? Also how do folks usually set a value for lambda?

Ridge or L2 regularization is used to prevent over-fitting when having multi-col-linearity in your features. Here is an example : suppose you want to train a linear regression model on following dataset:

  x1      x2      y
 1.0    -1.0      5
-1.0     1.0      5

Training a linear regression model, we are searching for an hypothesis function with following format :

As you can observe, there are multiple solutions to this problem. For example :

theta0 = 5, theta1 = 1 and theta2 = 1

But, there are also multiple other solutions to this problem. Including :

theta0 = 5, theta1 = 1000000 and theta2 = 1000000

Now imagine you have a new observation such as:

      x1      x2      y
 1.00001    -1.0      5

A solution with small theta coefficients is most likely to yield better results than a solution with large theta coefficients. To understand L2 regularization, we have to understand that multicollinearity can yield to less robust solutions because is can appear ridges in cost function surface and so result in a less good solution. By adding a penalization term, we will smooth the cost function surface to make learning focus on small theta coefficients :

So, going back to your temperature example, ridge regression will reduce temperature theta coefficient as much as possible in order to avoid a potential multicollinearity with other features. By adding L2 regularization, it is now possible to address overfitting issues in model if there is multicollinearity in your features. However, if your regularization strength is too large, you risk to force your learning to focus on small values of theta only and so it could yield a bias emergence in your model and so an underfitting situation.


How to wisely choose machine algorithm for the given data set? I have a data sample of RSSI value from three different devices and based on the RSSI sample, it should tell from which location the data sample is arrived. Below are the sample data set,

 device_1  device_2  device_3  location
 -45       -56       -78       drawing_room
 -48       -51       -82       drawing_room
 -41       -59       -73       drawing_room
 -71       -59       -59       conference
 -69       -60       -65       conference
 -73       -60       -52       conference
 -33       -68       -64       kitchen
 -32       -66       -63       kitchen
 -37       -67       -61       kitchen
 -63       -48       -48       lab
 -62       -48       -46       lab
 -59       -48       -54       lab

I will have "n" No.of data samples for "m" No.of location. The actual data set can be find here.

I want to predict "location" from the d1 d2 d3. Based on what parameters (i.e., correlation matrix or visualization), the machine learning algorithm can be chosen for the given data set?

The visualization chart for the considered data set is

From the visualization chart, Does it mean only the column "device_1" and "device_2" has a good separation among the location compared to other columns?

Note: If required, the data samples can be considered as positive.

I used a simple Neural Network with 3 Inputs - 1 hidden layer with 8 neurons - 3 Outputs (Labels) with k-fold crossvalidation in keras for your data set.

import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import LabelEncoder, scale
from sklearn.pipeline import Pipeline

path = r"User\train.csv"
data = pd.read_csv(path)

data.location.value_counts()
dataset = data.values

x = dataset[:,0:3]
x_scaled = scale(x)
y = dataset[:,3]

encoder = LabelEncoder()
encoder.fit(y)
encoded_Y = encoder.transform(y)

dummy_y = np_utils.to_categorical(encoded_Y)

def model():
    model = Sequential()
    model.add(Dense(8, input_dim=3, activation='relu'))
    model.add(Dense(3, activation='softmax'))

    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

    return model

estimator = KerasClassifier(build_fn=model, epochs=200, batch_size=5, verbose=0)
kfold = KFold(n_splits=10, shuffle=True)

results = cross_val_score(estimator, x_scaled, dummy_y, cv=kfold)
print("Model Prediction: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))

The used Model can categorize around 94% of the data correctly. Maybe this helps for the start.

Out:
Prediction: 93.95% (3.13%)

In data science and statistics in general you have to try and train several models to see which suits your dataset best (model benchmarking). In the end it is often a trade-off between computational costs and accuracy.


In PCA, every principal component a eigen vector? In pca, we convert predictors into principal components for dimensionality reduction. My assumption is every principal component is a eigen vector with eigen value as sum of squared distance of orthogonal projection of data points or vectors on it..
Is my assumption correct. If not pls clarify

Yes, a principal component is defined as the eigenvector of the covariance matrix of the data (zero-centered).

The eigenvector with the largest eigenvalue points in the direction of the greatest variance in the data. The smaller the eigenvalue, the less dispersed data points are in the direction of that eigenvector.

I'd recommend reading on the intuition behind PCA on Wikipedia.


Maximum Entropy modelling - likelihood equation I am trying to understand maximum entropy modelling and I came across log likelihood equation of the empirical distribution, which I did not quite understand, which also eventually turns out to be equal to the dual function we get when trying to maximise the entropy with constraints using Lagrange's multipliers

$$ L_\widetilde{p}(p) \equiv log \prod\limits_{x,y}p(y|x)^{\widetilde{p}(x,y)} = \sum\limits_{x,y}\widetilde{p}(x,y)\log{p(y|x)}$$

Where,

$y$ is the outcome produced by a random process
$x$ is the feature influencing the outcome $y$
$L_\widetilde{p}(p)$ is the log likelihood
$\widetilde{p}$ is the empirical distribution of training data
$p(y|x)$ is the model
$\widetilde{p}(x,y) \equiv \frac{1}{N} \times \text{number of times that } (x,y) \text{ occurs in the sample} $

Can someone please explain how $p(y|x)$ is raised to the power of $\widetilde{p}(x,y)$ in the log likelihood equation mentioned above? Instead, shouldn't $p(x|y)$ be raised to the power of number of times that $(x,y)$ occurs in the sample.

I went through this reference tutorial on max entropy.

I'd merely add a comment if I could ...

What is your interpretation of p(y|x)? The usual expression is "the probability of y given x", i.e. the dependent variable corresponding to the independent variable in the probability distribution. This is the function you want to maximize, not the inverse function.

As a side note, there are two ways that the entropy form of the equation on the right can be used. This can be added to a cost function as a regularization kernel, with a modulating coefficient; or it can be the primary term in the cost function -- the entropy to be maximized -- with constraints from the physical system introduced with Lagrange multipliers.


Dimension Mismatch Error during dot product in Python I have two matrices user_vecs and item_vecs

I am trying to take the dot product of the two to build a recommendation engine:

The shape of the two vectors are as follows:

user_vecs.shape

(20051, 20)

item_vecs.shape

(20,1808)

When i take the dot product of the transpose as follows:

a = user_vecs.dot(item_vecs.transpose())

I get the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-41-f4cd01978711> in <module>
----> 1 a = user_vecs.dot(item_vecs.transpose())

C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\base.py in dot(self, other)
    362
    363         """
--> 364         return self * other
    365
    366     def power(self, n, dtype=None):

C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\base.py in __mul__(self, other)
    479         if issparse(other):
    480             if self.shape[1] != other.shape[0]:
--> 481                 raise ValueError('dimension mismatch')
    482             return self._mul_sparse_matrix(other)
    483

ValueError: dimension mismatch

i understand that the dimensions of the two matrices are not matching, but the transpose should have fixed that. Why am I still getting this error?

Firstly user_vecs, item_vecs are matrices, not vectors, since both the shapes are of two dimensions. If seems like you are trying to multiply both the matrices, so you can simply use user_vecs.dot(item_vecs). No need to transpose item_vecs. You can also use np.matmul(user_vecs, item_vecs).  Refer to this thread for more details.


Python: Handling imbalance Classes in python Machine Learning I have a dataset for which I am trying to predict target variables.

Col1    Col2    Col3    Col4    Col5
  1      2       23      11     1
  2      22      12      14     1
  22     11      43      38     3
  14     22      25      19     3
  12     42      11      14     1
  22     11      43      38     2
  1      2       23      11     4
  2      22      12      14     2
  22     11      43      38     3

I have provided a sample data, but mine has thousands of records distributed in a similar way. Here, Col1, Col2, Col3, Col4 are my features and Col5 is target variable. Hence prediction should be 1,2,3 or 4 as these are my values for target variable. I have tried using algorithms such as random forest, decision tree etc. for predictions.

Here if you see, values 1,2 and 3 are occurring more times as compared to 4. Hence while predicting, my model is more biased towards 1 2 and 3 whereas I am getting only less number of predictions for 4 (Got only 1 predicted for policy4 out of thousands of records when I saw the confusion matrix).

In order to make my model generalize, I removed equal percentage of data that belongs to 1,2 and 3 value randomly. I grouped by each value in Col5 and then removed certain percentage, so that I brought down the number of records. Now I could see certain increase in percentage of accuracy and also reasonable increase in predictions for value 4 in confusion matrix.

Is this the right approach to deal with (removing the data randomly from those groups on which the model is biased)?

I tried for in-built python algorithms like Adaboost, GradientBoost techniques using sklearn. I read these algorithms are for handling imbalance class. But I couldnt succeed in improving my accuracy, rather by randomly removing the data, where I could see some improvements.

Is this reduction is undersampling technique and is this the right approach for under-sampling?

Is there are any pre-defined packages in sklearn or any logic which I can implement in python to get this done, if my random removal is wrong?

Also, I learnt about SMOTE technique, which deals with oversampling. Should I try this for value 4? And can we do this using any in-built packages in python? It would be great if someone helps me in this situation.

This paper suggests using ranking (I wrote it). Instead of using, for instance, SVM directly, you would use RankSVM. Since rankers compare observation against observation, training is necessarily balanced. There are two "buts" however: training is much slower, and, in the end, what these models do is rank your observations from how likely they are to belong to one class to how likely they are to belong to another so you need to apply a threshold afterwards.

If you are going to use pre-processing to fix your imbalance I would suggest you look into MetaCost. This algorithm involves building a bagging of models and then changing the class priors to make them balanced based on the hard to predict cases. It is very elegant. The cool thing about methods like SMOTE is that by fabricating new observations, you might making small datasets more robust.

Anyhow, even though I wrote some things on class imbalance, I am still skeptic that it is an important problem in the real world. I would think it is very uncommon that you have imbalance priors in your training set, but balanced priors in your real world data. Do you? What usually happens is that type I errors are different than type II errors and I would bet most people would be better off using a cost matrix, which most training methods accept or you can apply it by pre-processing using MetaCost or SMOTE. I think many times "fixing imbalance" is short to "I do not want to bother thinking about the relative trade-off between type I and II errors."

Addendum:

  I tried for in-built python algorithms like Adaboost, GradientBoost
  techniques using sklearn. I read these algorithms are for handling
  imbalance class.

AdaBoost gives better results for class imbalance when you initialize the weight distribution with imbalance in mind. I can dig the thesis where I read this if you want.

Anyhow, of course, those methods won't give good accuracies. Do you have class imbalance in both your training and your validation dataset? You should use metrics such as F1 score, or pass a cost matrix to the accuracy function. "Fixing" class imbalance is when your priors are different in your training and your validation cases.


If this is “big-endian”, then what is this? If an ISO 8601 date is “big-endian”, i.e. the most significant bit is first and the least significant is last, then what do we call data structures where the most significant part is first/at the top, and the least significant is last/at the bottom?

ISO 8601 date example: 2019-02-01T2157
data structure example: atom -> neutron -> quark
data structure example: body -> hand -> finger

I would like to be able to refer to this terminology when talking about any natural hierarchy such that any one branch makes ‘common sense’ from left to right (or top to bottom).

That's an interesting question. I would argue that the answer to your question lies in implicit assumptions that you (are willing to) make and expecting others to make.

In the ISO 8601 example, the significance is well defined, as 1 year = 365 days, we could say that a year is more significant than a day. In the last data structure example, what do you mean with significant? There is no clear definition there. Without framing what context you are referring to (such as level of abstraction or chronological order), interpretations are bound to differ to the people you are communicating with (and that defies the purpose of using the communication strategy, doesn't it?).

Instead of compressing the idea to a single concept, you could consider to be as explicit and unambiguous in the relation between the data structures as possible.


How do I access data on my EBS Volume from R-Studio Server on Ubuntu EC2 Instance I have setup R-Studio Server on an Ubuntu EC2 instance for the first time and successfully started r-studio server in my browser. I also have putty ssh client. How do I set path in r-studio server to my mounted EBS volume and why do I not see the contents of my EBS volume in the r-studio files area (bottom right side? ) . Also, I had a file in an s3 bucket. I passed this command to bring it from s3 to my ebs volume: s3cmd get s3://data-analysis/input-data/filename.csv . I assume this command downloads the file from s3 into the ebs volume. But I can't find it in RStudio Server! I have scoured the internet looking for help on this but not able to solve my problem.

I see this question got up-voted so someone else is looking for the answer. Here it is:

I was logged in as the default user on Ubuntu instance when i did the

$s3cmd get s3://data-analysis/input-data/filename.csv

The data got saved from the s3 bucket to the home directory of the user named ubuntu. this can be verified with :

$ df -h /home/ubuntu/

Since I had followed randy zwitch's tutorial on installing R on aws - i had created a user named rstudio and that was the user name I was using to log-in to rstudio server. Hence I had to move the file from the default 'ubuntu' user's home directory to the user 'rstudio'. this can be done with:

$ sudo mv /home/ubuntu/filename.csv /home/rstudio

Happy to answer follow up questions.


logistic regression coefs to excel formulation I have done a Logistic regression in r with 3 variables involving 1 interation:

lm(predicthead~vol+weekday*head, mydata)

weekday is a factor - 7 days of the week.

The coef values are :

    Estimate
(Intercept) -14.81119721
Vol 0.118819705
weekday2    -24.30795615
weekday3    -4.634180612
weekday4    -21.64229775
weekday5    0.607492058
weekday6    42.80131468
weekday7    19.16098601
heads   0.059934818
weekday2:heads  0.026481088
weekday3:heads  0.006665015
weekday4:heads  0.022825105
weekday5:heads  0.005045576
weekday6:heads  -0.041724212
weekday7:heads  -0.019254475

From my Understanding below is what I do in excel:

intercept +
(coef of vol * vol)+
(coef of corresponding week days) + #there no is value for weekday 1
(coef * head)+
(coef of corresponding weekday:head * head) #no value for weekday1 so it will be zero.

The sum of the above should be equal to predict() value - but I am not able to match it.
Could you please explain where I have done wrong in calculating predicthead value using excel?

By running a logistic regression model, the objective is to get the chance of a binary outcome based on the different predictor variables. Since the result is a chance, what is modeled is actually:

$$
log \left(\frac{P_{+}}{1 - P_{+}}\right) = \beta_{0} + \beta_{1}x_{1} + ... + \beta_{n}x_{n}
$$

So the coefficients you get from R are the $\beta$ values.
The function you have so far in Excel computes correctly the right hand side (RHS) of the above equation, but to get the probability $P_{+}$, you need to solve the equation for it. Let's call the RHS, $\mathbf{Bx}$

$$
log \left(\frac{P_{+}}{1 - P_{+}}\right) = \mathbf{Bx} \\
\frac{P_{+}}{1 - P_{+}} = e^{\mathbf{Bx}} \\
\frac{1}{e^{\mathbf{Bx}}} = \frac{1}{P_{+}} - 1 \\
\frac{e^{\mathbf{Bx}} + 1}{e^{\mathbf{Bx}}} = \frac{1}{P_{+}} \\
P_{+} = \frac{1}{1 + e^{-\mathbf{Bx}}}
$$

To your results, apply this transformation and you should get the same results as the predict function in R.


can't install gRain package I am new to R and trying to install the gRain package. However, I am getting the error:

 dependency ‘gRbase’ is not available

Then, when trying to install gRbase I get:

package ‘gRbase’ is not available (for R version 3.5.2)

Looks like it requires R 3.6.0 or higher. At least that is what the docs here say.


Computer Vision: Handling dataset(3D data or scan) with different timesteps I'm planning on training a CNN on CT scans for classification. The problem is CT scans are taken slice by slice, and in a typical scan, there could be more than 200 slices. The number of slices in a scan isn't uniform and depend on the scanning machine and age of the person(for whom the scan is taken).

1)How should I make the number of slices uniform for feeding to a deep learning network?

This sorta problem is handled in NLP by padding a chosen vector( or something similar) to sentences which have lengths less than the predefined length and truncating sentences which have lengths greater than the preset length.

2)Can a similar approach be used to make slices(timesteps) uniform or is there a better way?

I would use an LSTM-RNN to encode sequences of arbitrary length (length meaning the number of slices) into fixed-size output. It should be very easy to implement it in Keras :)

Then feed the sequences in your CNN.

Check sequence-to-sequence encoder-decoder scheme, like this one.

The same concept is applied with state-of-the-art algorithms that require fixed size inputs, like this one.


How to convert images (.jpg) to vectors for image classification I'm currently working on a project that involves classifying an image as either that of a dog or that of a cat. The twist is that I want to do this without using Convolutional Neural Networks, mainly because I do not quite understand them yet and I don't just want to copy someone's code off of Github.

I know that the classification algorithms on Sci-Kit Learn require the x input data to be in vectorized form for the classifier to fit the data, but I'm unsure of how to do this. It's easy to do with text data (feature_selection.text.CountVectorizer/TfidfVectorizer), but I have no idea how this works with images. How do I convert .jpg image files to vectors/matrices so that the models on SK-Learn understand it?

Thanks in advance, and I'm sorry if this is a stupid question.

You can use imread function from imageio library with np.array from numpy:

from imageio import imread
import numpy as np
filename = '/path/to/dog.jpg'
vectorized_picture = np.array(imread(filename))

When entering this data into a ML engine, you'll probably need to reshape it to a column matrix (at least in Prof. Andrew Ng's Coursera Machine Learning courses that I've been doing). If so, you can do that by:

reshaped_vectorized_picture = vectorized_picture.reshape(1, -1).T

You need to install both libraries through apt, brew or pip (preferred one)

pip install numpy imageio


How to calculate the different metrics for multi class classification My confusion matrix has the following structure:
                (Predicted)

C=   ( actual)    [TN FP
                 FN TP]

How can I calculate the Mathews Correlation Coefficient (MCC) value for multi-class expressed as
MCC = (TP .* TN - FP .* FN) ./ ... sqrt( (TP + FP) .* (TP + FN) .* (TN + FP) .* (TN + FN) );
Also, I have some doubts regarding the calculation of the following measures for multi-class. Please correct me where wrong.
for i=1:nClasses
   TN(i)=C(i,i);
    FP(i)=sum(C(i,:))-C(i,i);
    FN(i)=sum(C(:,i))-C(i,i);
    TP(i)=sum(C(:))-TP(i)-FP(i)-FN(i);
end

As you can see this MCC formula is for binary classification, so you can only calculate its results by considering the problem as binary.
[edited to clarify OP's confusion] What is a confusion matrix? It shows for every true class $X$ as a row and every predicted class $Y$ as a column how many instances have true class $X$ and are predicted as $Y$. If there are only two classes (binary classification), the only possibilities are

$X$ positive and $Y$ positive -> TP
$X$ positive and $Y$ negative -> FP
$X$ negative and $Y$ positive -> FN
$X$ negative and $Y$ negative -> TN

However when there are more than two classes (multiclass classification) it's impossible to use this distinction positive/negative directly, so there are no general TP,FP,FN,TN cases.
With multiple classes one can calculate binary classification metrics for every class. This is done by considering the target class as positive and all the other classes as negative (as if they are merged into one big negative class).
Example: suppose we have classes A, B, C. If we focus on class A, the confusion matrix is like this:
    A   B   C
A   TP  FN  FN
B   FP  TN  TN
C   FP  TN  TN

to present it another way:
         A    B or C
A        TP    FN
B or C   FP    TN

Now if we focus on class B the confusion matrix becomes:
    A   B   C
A   TN  FP  TN
B   FN  TP  FN
C   TN  FP  TN

In your code the TP and TN categories are swapped:
TP(i)=C(i,i);
...
TN(i)=sum(C(:))-TP(i)-FP(i)-FN(i);


Having averaged trials which are less than the number of features Suppose I have an experiment where I have 70 features and 48 samples. The target variable is binary (0,1) and the 48 samples are divided such that 24 of them correspond to outcome 1 and the other 24 correspond to outcome 0.

Note that the data I have is an averaged data. In other words, each of the 48 samples is a result of "n" averaged trials.

I am interested in using SVM and obtaining the weight vector (70x1).
In a way, I have less number of samples vs features.
However, the data which I have is an outcome of averaged trials.

In case I don't have access to the raw data, what is the best way to deal with the data? And does having the mean of trials as my data give a privilege over having 48 raw-trails instead?

About dealing with the mean of trials, I should say you can not recover the original trials and therefore you have to trust the results of classifucation model trained on averaged trials.

About dealing with a dataset with higher number of features than number of samples, I would suggest the following steps:

1- reduce the dimension of your data from 70 to 2 using PCA or t-SNE and

2- visualize it using a scatter plot to see if your labels are linearly separable in 2D space

3- If the result of PCA linearly separates labels, it means that your labels are kind of linearly separable in the original 70D space. So, you would apply the results of PCA (or even your original features) to linear-SVM.

4- If the result of t-SNE linearly separates labels, it means that your labels are not linearly separable in the original 70D space because unlike PCA, t-SNE is not a linear transformer. In this case, you would apply kernel SVM to your data.

5- If because of lack of data none of aforementioned methods worked, you would try low-variance classifiers such as Random Forest.

6- If Random Forest didn’t work, you would try using Bayesian methods such as Bayesian logistic regression because these methods assume a prior for parameters’ distributions and update it to posterior distributions using training data which compensates the lack of enough training data.


What technique to use in order to identify what position an audio sample is at in a longer audio sample? I am interested in what techniques and algorithms could be used in order to tackle the following problem:

I have a database of audio samples, specifically live performances of various songs. I have about a dozen songs and for each of them I have again about a dozen samples of that song's performance.

I am hoping that by having more than one sample for each song, it will be possible to better "lock down" the song's general features and filter out noise and differences between performances. These being live performances, each sample is a bit different, some are captured at better quality than others (directly from the sound guy versus a phone recording in the crowd), some songs have interludes, false starts (guitarist forgot to turn on the amp), start too late, end too early…

Now the next thing that I aside from this database is a live feed of a currently playing song and I am interested in using ML to find out which song is the live feed most likely to be. The way I see this, it could either be that the live feed continues to be captured so the changes of matching it up with features of the existing sample database grow or it might be more practical to periodically chop of fixed sized chunks if it is not possible to use a live feed like this.

I am interested in finding out what is the most common / most reliable approach to find what song the live audio most likely is and on top of that what position in the song is the live feed currently at.

In the first step, I think you should start with discretizing your song wave and then take a Fourier Transform for each chunk (in numpy for example you can use numpy.fft()). This Link might be helpful. After that, you can try sequence pattern recognition models.


Whether add bias or not in a perceptron In some places, perceptron is described as having added bias, while in some places, bias is not added.
Which one is right for you?

Suppose bias as a threshold. Using threshold, your activation function moves across the $x$ axis which may get complicated. Consequently, people usually use the bias term and always centre the activation function which is the step function at zero. There is nothing wrong in both cases.


model.predict in Keras, Python error I trained a model in Keras with input dimension 15 and output dimension 1. Then I tried to predict the output for a single input np.array, which I chose to be a toy example np.arange(15). However, the input is not accepted. Can someone tell me where the problem is?
Here is the code for a simplified problem:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
X = np.arange(15)
Y = 0
model = Sequential()
model.add(Dense(32, input_dim=15, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, Y, epochs=10, verbose=1, batch_size=batch_size)
model.predict(X)

The following error occurs:
ValueError: Error when checking input: expected dense_4_input to have shape (15,) but got array with shape (1,).
But then again, the input clearly has the correct shape. What is going on here?
Thanks for your help!

But then again, the input clearly has the correct shape.

>>> import numpy as np
>>> X = np.arange(15)
>>> X.shape
(15,)

In Keras, input_dim represents the number of input parameters, in your case that would be the number of columns of X or its second dimension (sometimes also referred to as number of features). It is clearly not 15. It is the first dimension that is 15. That means: X consists of 15 rows, also called samples (of one and the same feature).

So in that case, input_dim=1.

However, you will then run into the problem of having specified Y = 0. First, Keras will throw an error because it is an integer. You could do Y = [0], but then you will get

ValueError: Input arrays should have the same number of samples as target arrays. Found 15 input samples and 1 target samples.

So you have to turn this into an array-like object containing 15 samples, e.g. a list of length 15.

However, in case you meant to feed one single sample X to your model, that maps to one single output Y = [0], then you need to reshape X accordingly, for example via

X = np.arange(15).reshape(n_samples, n_features)

Can you figure out now what n_samples, n_features needs to be?


What is advantage of using Dryad instead of Spark? I found that Apache-Spark very powerful in Big-Data processing. but I want to know about Dryad (Microsoft) benefits. Is there any advantage for this framework than Spark?

Why we must use Dryad instead of Spark?

Dryad is an academic project, whereas Spark is widely deployed in production, and now has a company behind it for support. Just focus on Spark.


Should I have "normal" sampled data in my dataset? I am busy working on a project to find the reasons why kids in normal households are doing badly in school.

I have a dataset of which consists of kids that live in environments where the family is middle class, has access to necessary facilities and the kid is not suffering from any disorders but is failing grades in school.

It is understandable for kids that has a poor living condition to have problems at school but for kids that has all the necessities in life to do at least average in school, needs a bit more research.

Now that I have this dataset, does it make sense to add kids with the same living environments that are doing ok at school to the dataset?
I am planning to use SOM for data mining if that helps.

In short, yes.

If your goal is to understand the drivers behind poor school performance (or even model + predict school performance), you will need both "positive" and "negative" outcomes in your dataset.

Of course, you need to be careful that the positive outcomes are "similar" enough to the negative outcomes to avoid introducing distortions into your data - i.e. ensure that the data you are adding are from middle-class families with access to facilities etc.

You should also consider the ratio of positive:negative outcomes in your final data set, as you will ideally like to avoid imbalanced data. However, there are techniques and approaches to handle imbalanced data if necessary.


Reward dependent on (state, action) versus (state, action, successor state) I am studying reinforcement learning and I am working methodically through Sutton and Barto's book plus David Silver's lectures.

I have noticed a minor difference in how the Markov Decision Processes (MDPs) are defined in those two sources, that affects the formulation of the Bellman equations, and I wonder about the reasoning behind the differences and when I might choose one or the other.

In Sutton and Barto, the expected reward function is written $R^a_{ss'}$, whilst in David Silver's lectures it is written $R^a_{s}$. In turn this leads to slightly different formulations of all the Bellman equations. For instance, in Sutton and Barto, the equation for policy evaluation is given by:

\begin{align}
v_{\pi}(s) = \sum_a \pi(a|s) \sum_{s'} P_{ss'}^a(R_{ss'}^a + \gamma v_{\pi}(s'))
\end{align}

Whilst David Silver's lectures show:

\begin{align}
v_{\pi}(s) = \sum_a \pi(a|s) \left(R_{s}^a + \gamma \sum_{s'} P_{ss'}^a v_{\pi}(s') \right)
\end{align}

In both cases:

$\pi(a|s)$ is policy function - probability of choosing action $a$ given state $s$.
$\gamma$ is discount factor.
$P_{ss'}^a$ is transition function, probability of state changing to $s'$ given $s, a$

I understand that $R_{ss'}^a$ and $R_{s}^a$ are related (via $P_{ss'}^a$), so that these two sources are explaining the exact same thing. Note that the first equation can also be written as

\begin{align}
v_{\pi}(s)
&= \sum_a \pi(a|s) \sum_{s'} (P_{ss'}^aR_{ss'}^a + \gamma P_{ss'}^av_{\pi}(s'))\\
&= \sum_a \pi(a|s) \left( \sum_{s'} P_{ss'}^aR_{ss'}^a + \sum_{s'} \gamma P_{ss'}^av_{\pi}(s') \right) \\
&= \sum_a \pi(a|s) \left( \sum_{s'} P_{ss'}^aR_{ss'}^a + \gamma \sum_{s'}  P_{ss'}^av_{\pi}(s') \right)
\end{align}

Hence, it must be true that $R_{s}^a = \sum_{s'} P_{ss'}^a R_{ss'}^a$.

My question is whether there is any reason I should prefer to use one or the other notation?

I started with Sutton and Barto, and find that notation more intuitive - the reward may depend on the eventual state, and this is explicit in the equations. However, it looks like in practice that the notation used in the video lectures describes more efficient calculations (essentially $R_{s}^a = \sum_{s'} P_{ss'}^a R_{ss'}^a$ is cached, if the formula is translated directly to code). Is that all there is to it?

Your intuition is correct. In the most general case (Sutton's definitions), the model of the environment consists of the state transition distribution and the reward distribution. The latter one is rarely considered, as many times the reward is being assigned by the modeler, dependent only by the selected action from the current state and being deterministic. As you mentioned it simplifies a lot the coding implementation.


How to compute the Jaccard Similarity in this example? (Jaccard vs. Cosine) I am trying to understand the difference between Jaccard and Cosine. However, there seem to be a disagreement in the answers provided in Applications and differences for Jaccard similarity and Cosine Similarity.

I am seeking if anyone could step me through the calculations of the Jaccard Similarity in this Cosine Similarity example from https://bioinformatics.oxfordjournals.org/content/suppl/2009/10/24/btp613.DC1/bioinf-2008-1835-File004.pdf

Given:

Question: How do we compute the Jaccard Similarity index between t1 and t2?

Thank you.

Cosine similarity is for comparing two real-valued vectors, but Jaccard similarity is for comparing two binary vectors (sets). So you cannot compute the standard Jaccard similarity index between your two vectors, but there is a generalized version of the Jaccard index for real valued vectors which you can use in this case:

$J_g(\Bbb{a}, \Bbb{b}) =\frac{\sum_i min(\Bbb{a}_i, \Bbb{b}_i)}{\sum_i
max(\Bbb{a}_i, \Bbb{b}_i)}$

So for your examples of $t_1 = (1, 1, 0, 1), t_2 = (2, 0, 1, 1)$, the generalized Jaccard similarity index can be computed as follows:

$J(t_1, t_2) = \frac{1+0+0+1}{2+1+1+1} = 0.4$

Alternatively you can treat your bag-of-words vector as a binary vector, where a value $1$ indicates a words presence and $0$ indicates a words absence i.e. $t_1 = (1, 1, 0, 1), t_2 = (1, 0, 1, 1)$. From there, you can compute the original Jaccard similarity index:

$J(t_1, t_2) = \frac{2}{2+1+1} = 0.5$


Binary classification: best ways to pre-procees the data About the dataset
I have a training dataset of

129 columns(last column being the classes, i.e., y values)
6068 rows

I have to train some algo to do binary classification. The data set has

701 examples of type A
5367 examples of type B

The test set consists of 1398 examples. Here is the accuracy I got for various algorithms.

voting ensemble -> 0.73963
stochastic gradient boosting -> 0.77682
Adaboost -> 0.75107
bagging classifier(Decision tree) -> 0.76538
Random Forests -> 0.75250
Extra trees -> 0.75393

All the above results are from kaggle so they probably are just from half the test set.
The above methods were implemented using scikit-learn library in python

Could someone please suggest ways to improve accuracy, may be by things like dimensionality reduction or better algorithms. Also, please provide sample code in possible.

A better clarity about the kind of problem that you are trying to solve will be really useful.

I any case the first thing I will suggest, is please go through your features and try to understand the relationship with the response. All tree based methods are highly prone to overfitting and hence they don't generalise well.
If you come to the conclusion that some variables show a linear relationship and some show a non-linear then you might be better off using Linear methods such as LogisticRegression or LinearDiscriminantClassifier(if your features show an indication that they are sampled from a normal distribution). Obviously you can use PCA or regularization methods if you are certain that some features are highly correlated and offer no prediction value.
Secondly, if you have done all the above then you are probably done with the feature selection step. The next approach is feature creation. See if certain variables are non-linear in nature. Then try adding higher degree polynomial terms. Try diagnosing interaction effects between features, try to include features for those interactions. If there are any features you think are redundant to the problem. Think again. See if you can extract any information out of them.

Kindly tag the link of the problem here.


how to implement Excel vlookup on Python I have a dataset A and dataset B, both data has a column in common which is the ID, i want to check if dataset A ID values can be found in dataset B ID values. Also get the records of dataset B that have the same value with dataset A. I can do this on excel using vlookup, but I want to automate the process using python.

NOTE:
1. dataset B has 10020 rows while dataset A has 2039 rows.
2. The ID column is numeric.

Thanks.

You will want to load your data in two pandas data frames (if you haven't already)
Next you will need to use dataframe.join():

Note: You might want to think about the mode (inner, outer, left or right)


Library to count number of objects in image? I'm reading a paper on counting cells, humans, etc. Is there an off-the-shelf library for Python/Theano or TensorFlow I could use?

I think the OP was asking for a specific library for Theano or TensorFlow that provides off-the-shelf functionality for object identification.
I would look first into scikit image, an image processing library for Python in the Scipy/Numpy stack. It has all sorts of edge detectors, blob detectors, shape identifications, color separations... Look at their gallery of examples.
Btw, counting humans in a crowd may not be so trivial to automate...


Does small batch size improve the model? I'm training an LSTM with Keras.

I've noticed that the smaller the batch size, the more the loss decreases during periods: so this makes me think that the network can process fewer items better at a time.

Is it a normal behavior in general?

In general smaller or larger batch size doesn't guarantee better convergence. Batch size is more or less treated as a hyperparameter to tune keeping in the memory constraints you have.

There is a tradeoff for bigger and smaller batch size which have their own disadvantage,  making it a hyperparameter to tune in some sense.

Theory says that, bigger the batch size, lesser is the noise in the gradients and so better is the gradient estimate. This allows the model to take a better step towards a minima. However, the challenge is that bigger batch size needs more memory and each step is time consuming.

Even if somehow we can avoid the time and space constraints, bigger batch size still wouldn't give better solution in practice as compared to smaller batch size. This is because the surface of the neural networks objective is generally non-convex, which means that there might be local optimums. Just having an accurate gradient estimate doesn't guarantee us reaching the global optimum (which we seek). It could lead us to a local optimum accurately! Keeping the batch size small makes the gradient estimate noisy which might allow us to bypass a local optimum during convergence. But having very small batch size would be too noisy for the model to convergence anywhere.

So, the optimum batch size depends on the network you are training, data you are training on and the objective function you are trying to optimize.


Is it faster and better to train a GAN on just one digit as opposed to the whole mnist dataset? When going through an introductory GAN tutorial to generate mnist like handwritten digits I wondered whether the systematic variance in the training data due to the different digits makes the model harder to train.

Would't it be easier to train a model if all real samples would be 1s instead of 0-9s?

My question

Would an approach where I train a separate model to generate each digit (0-9) separately using only training data of one type of digit be feasible at all?
Would training those models for just one digit be faster (i.e. need less epochs to reach a certain accuracy/quality) than a model using all of mnist.

Edit

I found an acceptable answer for me (see below) but obviously I am happy for anyone else to chime in.

After some further research and experimentation, I think I am able to answer my own question.

1. Yes it is feasible!

The quickest way to answer my first question was to try it out, which I did. Training a GAN on only a single digit (3 in my example) worked and produced the expected results:

Besides the obvious practical example, I was able to find some further articles discussing GAN which mentioned that reducing the classes in the training sample to reduce variance is advised (e.g. in CIFAR-10 concentrate on frogs OR trucks not both). So while I suspected as much it is still nice to confirm that this approach is valid and feasible.

2. I'm not sure it is faster

The second part of my question was whether such an approach is better (i.e. more accurate) and faster.

I was unable to confirm this because mnist is a very simple example and accuracy is very good, very fast in either case. I learned that speed of model fitting depends on the model complexity and number of epochs, which do not change by focusing on just one digit! So naturally covering all digits individual takes a lot longer than just training one whole model.

It might be the case that for more complex classes/image types more complex models and epochs are needed and that cutting down on variance (i.e. just frogs, no trucks) can help cutting down those two main drivers of computation duration but this was not the case for mnist like images.

The only direct benefit I was able to find is the greater utility of individual models. Due to the clear separation I am able to generate a specific number on demand (by calling the relevant model) instead of generating only a random number from 0-9.


Alternatives to imputation of missing values? So I'm quickly learning that dealing with missing values for feature(s) in some of your observations is a part of every day life in data.  I get the gist of imputation, when/how it's appropriate and when it's not, and I'll read up on it in the near future.  But, what about this:

Suppose you have predictors $X_1, \dots, X_p$, and you want to model, say, a binary response $Y$ via, say, logistic regression.  Instead of just imputing values to missing predictor values, couldn't you have a separate model built for every possible subset of the predictors, and apply that model for prediction when those predictors happen to be present?  Each of those models would naturally be trained on just the data for which those predictors (and also others) are present.  This seems to me a more reasonable approach than just making up values, but I have no theoretical justification for this.

I do realize that this involves building $2^p$ different models, with $2^p$ different model matrices, etc., but for a moderate $p$ and $n$ it could be feasible, and especially if only a few of your features tend to be missing more often than others.

Is this ever done?  And if so, is there a standard way to implement this in R?  In the case of logistic regression, you can specify to R's glm function how you want it to handle NA values, but your only options seem to be either tossing out observations altogether, or some kind of imputation scheme.

Thoughts?

Supposedly you had these models, divided into subspaces, then what if in your test dataset a point is close proximity to where the missing point is in your training data. What would be the best solution? Have the average output of the two closest models? Is the output binary? And then would you have to weight each model, based on that proximity? Then I am thinking that many submodels could overfit on the subset of the data. Just some thoughts, which may be challenging to overcome.

Moreover, you could use other types of Classifiers which deal with missing data, like some decision trees implementations.

Finally, data imputation is a delicate art, and doing it right will leverage your model. For instance, you wouldn't want to impute all the missing data with the average as then you may change your data distribution. Knn techniques or ensemble regressions would be preferred.


How to Replace an object in Pandas array using replace with dictionary from excel file? Suppose I have a data like this:

and I want to change the value of name and gender into an integer, and I have a dictionary like this:

I understand that I can use replace function in pandas, but I don't know how to use an excel dictionary to replace the value
thank you

Assuming you have a pair of csv files: "replace.csv" representing the first table and "table.csv" representing the second table i.e. "dict". Use the following code:

import pandas as pd

frame = pd.read_csv('replace.csv')
table = pd.read_csv('table.csv', header=None)

mapping = dict([(k, v) for k, v in table.values])

frame.replace(to_replace={"name": mapping, "gender": mapping}, inplace=True)
print frame

The line mapping = dict([(k, v) for k, v in table.values]) is the one that transforms your dataframe into a dict. For more on transforming a dataframe into a dictionary see the documentation, also this question provides different ways of transforming a dataframe into a dictionary. For reading excel files, instead of csv files, see this.


Doing predictive modeling on predicted value It's a project that I'm working on. Here are the steps I took:

I want to make a recommendation service based on the customer data. I first used a collaborative filtering method to get the recommended products. Then I am trying to use that result as a target variable of my random forest model.

The specifics:

My data: Columns A, B, C, D, E, F. Each row indicates each customer.
Do unsupervised learning(collaborative filtering) to make a recommendation. Only used column A (which has information about the products they have bought)
Choose one recommendation product for each customer, and make that into a categorical column
Merge that column with the original data, and take out column A
My data: Columns B, C, D, E, F, NEW_COLUMN
Use supervised learning to get important features that explains the NEW_COLUMN
I am trying to use supervised learning on the NEW_COLUMN because I want to get some useful predictions that could be used to explain the reason for the NEW_COLUMN(the recommended products)

This is the path that I'm taking. However, is it okay to do predictive modeling with the data that I made using predictive modeling?

Also, if there are any references, it would be a great help!

Your approach seems valid. It is not even biaised because you don't use other features than A to build F.

Performing unsupervised learning prior to supervised learning in a very common approach, and there are numerous types of workflows where supervised and unsupervised techniques are combined to solve the problem.

However, be careful with random forest feature importance: there are several ways to compute importances, and not all will give the same results. There are also other techniques to make complex models interpretable, such as LIME for local explanation or SHAP values for both local and global explanation.


How is the GridsearchCV Score calculated? How is the score of GridsearchCV calculated? Is the score a percentage? Does this mean higher is a better?

The score is based on the scorer defined in the scoring argument. Meaning, the scorer can be any of the default metrics, such as precision, accuracy or F1-score
(e.g., this); or a custom scorer.
For a scorer (by convention), higher value is better.
The value is not necessarily a percentage, but is often normalized between 0 and 1.


LSTM with linear activation function I'm trying to do multi-step regression and I use an output layer:

LSTM(1, activation='linear', return_sequences=True)

Is this the wrong way of achieving this?
Should I use a TimeDistributed(Dense(1)) as output? If yes, why?

I don't see any particular advantage in using linear (i.e.: none) activation. The power of Neural Network lies in their ability to "learn" non-linear patterns in your data. Moreover, the Tanh and sigmoid gates are thought to control for the stream of information that unrolls through time, they have been designed for that, and personally I'd be cautious in changing that.

If you want to do some multistep regression you have many options, but the most advanced architecture is the sequence-to-sequence, or seq2seq:

from tensorflow.keras import Sequential

from tensorflow.keras.layers import Bidirectional, LSTM
from tensorflow.keras.layers import RepeatVector, TimeDistributed
from tensorflow.keras.layers import Dense

from tensorflow.keras.activations import elu, relu

seq2seq = Sequential([

    Bidirectional(LSTM(len_input), input_shape = (len_input, no_vars)),

    RepeatVector(len_input),

    Bidirectional(LSTM(len_input, return_sequences = True)),

    TimeDistributed(Dense(hidden_size, activation = elu)),

    TimeDistributed(Dense(1, activation = relu))

])

where len_input is the length of the input sequence. The output is the same sequence shifted forward a number of steps equivalent to the lenght of your prediction. hidden_size is the size of Dense() layers, that is completely optional.

I used the Bidirectional() wrapper, but this is optional too. Not all tasks require bi-LSTM, feel free to remove it if you need.

The (combined) role of RepeatVector() and TimeDistributed() layers is to replicate the latent representation and the following Neural Network architecture for the number of steps necessary to reconstruct the output sequence. RepeatVector() generates this "multiplication", while TimeDistributed() repeats the application of on each of these repeated signals, generating the final sequence in practice.

Both input and output must be 3-dimensional numpy arrays of shape:

  ( number of observations , length of input sequence , number of variables )

Seq2seq models are harder to train (higher number of parameters, longer training times), but their performance is superior to other RNN architectures. At the present time, any state-of-the-art RNN is some kind of seq2seq.


mathematical accurate definition of the binary independence model I have a hard time understanding the exact mathematical meaning behind the binary independence model. On wikipedia we can see the following definition  or similarly in the book from Manning and Schütze,

it claims that

  The probability P(R|d,q) that a document is relevant derives from the probability of relevance of the terms vector of that document P(R|x,q). By using the Bayes rule we get:

$P(R|x,q) = \dfrac{P(x|R,q)P(R|q)}{P(x|q)}$

Now, Bayes rule is as follows:
$P(A|B) = \dfrac{P(B|A)P(A)}{P(B)}$

If you set $A := R$ and $B := x,q$ you get:
$P(A|B) = P(R|x,q) = \dfrac{P(x,q|R)P(R)}{P(x,q)}$

If you compare the terms, you'll notice why I am confused:

$x=x,q$
$R,q=q$
$R|q=R$
$x|q=x,q$

This result has nothing to do with the initial claim.
I think that I miss the definition of the 'comma' in this context. I am not aware of multi-dimensional probabilities. As I understand, a probability $P$ is always defined over a $\sigma$ algebra of an event space $\Omega$

In order to understand what's the idea behind the formula above, here a few things that could help:

What does the comma  precisely mean in the formula above (in mathematical notation)?
What is the underlying $\Omega$ ? If there is a probability $P$, then there must be an Omega $\Omega$ which serves as the space on which we define probabilities. It's not clear at all what this space is. If a document is a vector $x \in \{{0,1}\}^n$ and the query is a vector $q \in \{{0,1}\}^n$, then defining a space like $\Omega := \{{0,1}\}^n \times \{{0,1}\}^n$ could make sense. In this case it's not clear what $R$ is. Maybe the intent is to use $\Omega := \{{0,1}\}^n \times \{{0,1}\}^n \times \{{relevant, nonrelevant}\}$
Do $R$, $x$ or $q$ have anything to do with random variables? If yes, then it would help to see their domain, e.g: $R : \Omega \mapsto {0,1}  $
Because the conditional probability is defined between 2 sets, $R$ and $x,q$, as well as $x$ and $R,q$ or $R|q$ should represent sets. If $R$ is a random variable, then maybe in the formula above the term $R$ represents the set $\{\omega \in \Omega\ | R(\omega)=relevant\}$. What is then the set for $x$ or for $q$ ?

It's better to talk about x, q and R as (random) events - set of outcomes of the random experiment. x and q will be one element sets but R is an event denoting x is relevant to q and thus it is a subset of cartesian product X x Q (pair of a document and a query).

Comma is then set conjunction and P(A,B) = P(A ^ B) which equals to P(A) * P(B) when A is independent of B.

The wiki statement 'by using Bayesian rule' is a bit of a shortcut since you need to apply it several times. To derive the above formula for P(R|x,q), I would start with conditional probability definition (which is the root of Bayesian rule):

P(A|B) = P(A ^ B) / P(B)

Then:

P(R|x,q) = P(R ^ x ^ q) / P(x,q) = P(x|R,q) * P(R,q) / [ P(x|q) * P(q) ] =

= P(x|R,q) * P(R|q) / P(q) / [ P(x|q) * P(q) ]

When you divide nominator and denominator by P(q), you obtain

P(R|x,q) = P(x|R,q) * P(R|q) / P(x|q)


Cast regression problem into classification problem In the TA session, my TA claimed, that regression problems should often be cast into classification problems by dividing the output range into bins and then using a multi-loss, since we have better classification than regression algorithms.
In my understanding, this is inherently wrong as it discards the property, that "close to correct is better than far correct". All wrong classes are equally wrong.
I asked my professor, but he just said, there are applications where it makes sense and did not want to discuss it more.
Am I wrong? When should I cast a regression problem into a classification problem?
Edit: I do not know if my TA referred to it, but here is a tweet from A. Karpathy: https://twitter.com/karpathy/status/708480082831024128

not-widely-enough-known-protip: Do not use L2 loss (regression) in
neural nets unless you absolutely have to. Softmax likely to work
better.

The statement in principle seems very vague, how can one state that there are better classification than regression algorithms?
With that said I would rephrase the statement to:
Sometime it is feasible to turn a regression into a classification problem due to for the problem itself, it makes sense to predict a range/bin instead of a continuous value.
When doing this we have to be careful since we are not facing a "common" classification problem but we have an ordinal classification problem in which we have a natural order from the new target (bins)
For reference check:
https://stats.stackexchange.com/questions/493254/why-ordinal-target-in-classification-problems-needs-special-attention


Which model for a content suggestion in which only some of the items are currently available I am trying to understand the type of model that would be used in a content suggestion scenario where not all of the choices are available at a given time.  For example, when an online movie subscription service suggests a show it does not need to worry about if the content is available because it's unlimited.  A storefront renting DVD's however, does.  My problem is like the storefront where only a certain few of the content items is available at a given time.
For the environment, let there be 1000 pieces of content in total, 100 of which are available at a point in time.
I am considering the pros and cons of approaching this in two way:

use a model where I can ask for a probability for all 1000 pieces of content in one call for the current environment, then check which ones are available.

check which items are available, ask the model for the probability of each.

Both of these seem inefficient. Performance is certainly a consideration as this model will live in an environment where it needs to be very fast.
Are there any models which can account for some of the choices not being currently available?

It is pretty common in recommendation systems to have business constraints to filter against before a final recommendation is made (e.g. checking whether each recommendation is in stock).
One way to manage this is to produce a superset of recommendations (e.g. 100 if you need to produce 10 recommendations), apply your filters (which could drop the number of recommendations down considerably), and then return the top-10 recommendations from the filtered set. So, you may not need to produce the full 1000 item recommendation set each time, but you most likely will need to generate a list which is initially large and filter down.
A potentially helpful link:
https://buildingrecommenders.wordpress.com/2015/11/10/the-components-of-a-recommender-system/


Datasets with only single feature variable I mainly have 2 queries:

1) I want to know if there are any standard datasets available who have single feature variable?

2) In these types of datasets, can there be some data points having same values for feature variable but a slight variation in target variable?

For exmaple, following table (X = feature variable and Y = target variable):

X | Y

1 | 2.345

1 | 2.3

2 | 3.76

2 | 3.7

3 | 4

4 | 5.56

4 | 5.57

Kindly let me know a few of them.

Finding a dataset with just two variables might be difficult because these would be a very naive dataset.

Therefore, what I would suggest, is for your to take a more complex dataset and pick two columns that you would like to analyse.

For example, you could look at this dataset: https://www.kaggle.com/sohier/calcofi#bottle.csv

Then, pick the column Depthm as your feature variable and the column T_degC are your target variable. These two columns seem to match the type of data that you expect.

Here is how these two variables look like (for Depthm > 2000)

or you could look at  another two variables instead, for example Depthm vs Salnty:


K fold cross validation reduces accuracy I am working on a machine learning classifier and when I arrive at the moment of dividing my data into training set and test set Iwant to confron two different approches. In one approch I just split the dataset into training set and test set, while with the other approch I use k fold cross validation.

The strange thing is that with the cross validation the accuracy decreases, so if I have 0.87 with the first approch, with cross validation I have 0.86.

Shouldn't cross validation increase my accuracy? Thank's in advance.

Chance plays a big role when the data is split. For example maybe the training set contains a particular combination of features, maybe it doesn't; maybe the test set contains a large proportion of regular "easy" instances, maybe it doesn't. As a consequence the performance varies depending on the split.

Let's imagine that the performance of your classifier would vary between 0.80 and 0.90:

  In one approch I just split the dataset into training set and test set

With this approach you throw the dice only once: maybe you're lucky and the performance will be close to 0.9, or you're not and it will be close to 0.8.

  while with the other approch I use k fold cross validation.

With this approach you throw the dice k times, and the performance is the average across these $k$ runs. It's more accurate than the previous one, because by averaging over several runs the performance is more likely to be close to the mean, i.e. the most common case.

Conclusion: k-fold cross-validation isn't meant to increase performance, it's meant to provide a more accurate measure of the performance.


Binary predication from binary variables Logistic Regression generates a binary outcome for a non-binary variable. I need a binary outcome from a binary variable. This is the requirement.

  How to predict binary A using previous values of A?

or

  How to predict binary A,B,C,D values from previous A,B,C,D values?

I know I can gather data and calculate basic probability. I that the most suitable approach? Thank you in advance.

This is really a job for Logistic Regression. Input variables can be categorical/boolean and the prediction can be categorical/boolean as well.

However, if your target variable contains multiple categories, you should use multinomial logistic regression or Ordinal Logistic Regression (if there is an order in your categories). It also changes the way you interpret the output.

The another approach would be to take any nonparametric method, Decision Tree for example. Where input and target can be boolean/categorical/continuous. Note that in case of a categorical variable, it is often recommended to transform it into dummy variables.


What's the difference between Haar-like Features and Haar Cascades? Are they the same in terms of their algorithm? Or do they differ in their respective detection methods?

Haar-like features are, as the name says, features. They are basically some filters, just like the CNN filters. The main difference is that CNN learns the filters by training, while Haar filters are hand designed. These filters are called Haar-like features due to their similarity with Haar wavelets.

Haar cascades are a bunch of Haar-like features arranged into a classifier.


How to install boruta in conda? I want to install boruta in my anaconda environment, but if I execute

conda install boruta

It displays,

PackagesNotFoundError: The following packages are not available from current channels:

  - boruta

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

I guess I could install it using the pip that belongs to the anaconda environment, but I am not sure if that is the best way.

Any suggestions?

According to the instructions at both Github and Anaconda cloud, you should try

 conda install -c conda-forge boruta_py


neural network to match image I am pretty new to neural networks and I would appreciate some guidance... maybe books or articles on the following topic:

I am an airfoil designer. At fixed flow conditions, the pressure on the airfoil wall will be different depending on its geometry, e.g. thickness.  For different geometries, I have a large data set of pressure distributions along the airfoil (they can be treated as images). My idea is to use those "pictures" as training data of a neural network (input), in which the output will be the geometry of airfoil. Then, I would like to target a pressure on the wall (picture) and get a nice airfoil design...

Please let me know if you are aware of similar studies and if there is any book to start with!

If I understood correctly you have images similar to a heat-map like Fig 6:

Well, non-natural images related applications can benefit from DNN as much as natural images, but you are unlikely to be able to use any transfer learning technique.

If you have this heat-map-like images (please consider posting some samples) you can try a simple CNN structure. Since you are a starter you could benefit from using this modeler called Ennui and outputting the source-code in Python or Julia, just remember to take a quick class on CNN to understand your options. For a complete course see Stanford's YouTube playlist on CNNs.

Some papers related to it:

Inverse Design of Airfoil Using a Deep Convolutional Neural Network
Application of Convolutional Neural Network to Predict Airfoil Lift Coefficient
Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows

You can inspire your models architecture by theirs

If you want a book to cover NN for deep learning try Goodfellow's.


Random Forest Modelling? I use random forest to train on my data (My data had imbalance in the target class, i.e. rare 1 and abundant 0). I face 3 issues about the stability of estimator and its prediction power. I think these problems could be common on many machine learning algorithms.

I found the ROC_AUC_Score highly changeable when I resample the training set (the rest was test set). It can vary from $0.85$ to $0.45$ while changing the training set.
The parameter tuning can also cause the move of the estimator and ROC_AUC_Score but the effect was weaker than the first case above.
When running some iteration of model fitting, the results were also different from each other but the effect was the weakest.

For 2, 3, I think we can get the best parameter setting and fitting results by recording each iteration and parameter tuning. (Maybe can do it more efficiently, please advice)

Please also advice how to deal with the first problem to make the fitting convinced and reliable? Cross Validated?

Many thanks

You should definitely use nested cross-validation for model selection and performance estimation. I have also found that the AUC of the precision-recall (PR) curve (compared to that of the ROC curve) to be a better, that is, more stable, estimator of the performance of my Random Forest classifiers when I have a highly unbalanced dataset; there is research on this topic, showing that the AUC of the PR curve is more informative than that of the ROC curve. You can use average_precision_score() in scikit-learn to use the PR AUC score. Along the lines of resampling the data, you could try approaches like EasyEnsenble and BalanceCascade; search for the papers titled "Exploratory Undersampling for
Class-Imbalance Learning" and "Learning from Imbalanced Data" for more information.


Training a floor detection model: use full room images or only the cropped floor? I'm trying to build a floor type image classification model.There's an open dataset called OpenSurfaces containing images segmented by the material type of every item appearing on a room.

Something like this:

I thought that using this dataset to train a floor detection model would be a good thing, so I wrote a script to extract the materials I'd like to detect (wood, tile, carpet, marble, stone, ...). These are some examples of the images I've got as a result of the script:

Wood material:

Tile:

Carpet:

Then I trained a CNN but I've only got something like 70% of accuracy and I don't really now if I'm going along the right path. Is it better to train a model with the pictures I extracted or it'd be better to train it with the full room image, not the segmented part?

I'm quite lost, so any guidance will be greatly appreciated.

It isn't clear what you are trying to do (floor detection or just classification).

If your task is to classify different types of floor, you could use just the segmented parts. But if you want to detect the floor and classify it, you have to give your model some negative examples as well (it needs to learn how not-floor looks like as well).

However, in general it would be better practice to train the model on full images. Even if it is just a floor classification task, the context of the image can also help (kitchen increases the probability of tiles, bedroom increases the probability of a wooden floor, etc.).


Bayesian optimization with Keras tuner for time series Goal: trying to use walk-forward validation strategy with keras tuner for time series when training a neural network (mainly LSTM and/or CNN).

Did anyone find a direct way of doing this?

One possible way I can think of is:

implementing a custom 'objective' function, e.g. 'mean squared error' using walk-forward validations trategy
this custom function could be passed via the 'objective' input parameter in the tuner (be it RandomSearch, BayesianOptimization, Hyperband...)
when calling tuner.search, we pass the train & validation data, to be used in the walk-forward custom function

Thanks in advance

I finally came out with a way to implement bayesian hyperparameter optimization for a time series neural network model using walk-forward validation. You can find a gist with the code here

I used some helper functions for the walk-forward validation from this Jason Brownlee book

It is basically as follows:

original time series data:

define a function to convert your time series dataset into a supervised dataset format, another one for the train-test split (ordered in time as done in sklearn.model_selection.TimeSeriesSplit)

define your evaluation metric, root mean squared error in this case
we also add a differencing function, in case we want to make our time series stationary (by differencing, we can reduce trend and/or stationality)
define the walk-forward validation functions (walk_forward_validation and repeat_evaluate)
define the keras tuner bayesian optimizer, based on a build_model function wich contains the LSTM network in this case with the hidden layers units and the learning rate as optimizable hyperparameters
define the model_fit function which will be used in the walk-forward training and evaluation step

lastly, find the evaluation metric value and std

To keep in mind: this is a proof-of-concept example about how to use keras tuner for time series using walk-forward validation. Further analysis could be done to improve forecasts.


Does this mean underfitting? I am training model to classify fruit images belonging to 60 classes. I have this result:

Validation accuracy is greater than training accuracy. Does this mean underfitting? If yes, can I fix this by adding more layers to the neural network or by increasing the number of neurons?

It seems weird.

Your validation has a higher score than your training. This literally means that your model performs better in unseen data than what it sees.

Typical underfitting is that you achieve the same in train than in test.

In my opinion, since you are not providing much information, you are not splitting right the data. It might be for a lot of reasons:

The test is too small or too easy to predict
There is a temporal dependency and you are not using it (data leakage).
There are groups in your dataset and you are splitting by groups...

And a thousand more. From the visualizations that  you are adding my guess your train test split is not performed correctly


Exploration in Q learning: Epsilon greedy vs Exploration function I am trying to understand how to make sure that our agent explores the state space enough before exploiting what it knows. I am aware that we use epsilon-greedy approach with a decaying epsilon to achieve this. However I came across another concept, that of using exploration functions to make sure that our agent explores the state space.
Q Learning with Epsilon Greedy
$sample = R(s,a,s') + \gamma \max_{a'}Q(s',a')$
$Q(s,a) = (1 - \alpha)*Q(s,a) + \alpha*sample$

 Exploration function
$
f(u,n) = u + k/n $
$Q(s,a) = R(s,a,s') + \gamma*max_{a'}f(Q(s',a'), N(s',a'))$
Where N(s',a') counts the number of times you have seen this (s',a') pair before.
Now my question is, are these 2 above strategies just 2 different ways of Q learning? Because I can't seem to find much details on exploration function but can find plenty on epsilon greedy Q learning. Or can the exploration function be used along with the epsilon greedy Q learning algorithm as a form of some optimization? I am confused as to where exactly would we make use of this exploration function Q learning strategy. Any help/suggestions are much appreciated!

Any exploration function that ensures the behaviour policy covers all possible actions will work in theory with Q learning. By covers I mean that there is a non-zero probability of selecting each action in each state. This is required so that all estimates will converge on true action values given enough time.
As a result, there are many ways to construct behaviour policies. It is possible in Q learning to use equiprobable random action choice - ignoring current Q values - as a behaviour policy, or even with some caveats learn from observations of an agent that uses an unknown policy.
There are practical concerns:

If the behaviour policy is radically different from optimal policy, then learning may be slow as most information collected is not relevant or very high variance once adjusted to learning the value of the target policy.

When using function approximation - e.g. in DQN with a neural network - the distribution of samples of state, action pairs seen has an impact on the approximation. It is desirable to have similar population of input data to that which the target policy would generate*.

In some situations, a consistent policy over multiple time steps gives better exploration characteristics. Examples of this occur when controlling agents navigating physical spaces that may have to deviate quite far from current best guess at optimal in order to discover new high value rewards.

These concerns drive designs of different exploration techniques.
The epsilon-greedy approach is very popular. It is simple, has a single parameter which can be tuned for better learning characteristics for any environment, and in practice often does well.
The exploration function you give attempts to address the last bullet point. It adds complexity, but may be useful in a non-stationary environment since it encourages occasional exploration paths far away from the current optimal one.

Now my question is, are these 2 above strategies just 2 different ways of Q learning?

Yes.

Or can the exploration function be used along with the epsilon greedy Q learning algorithm as a form of some optimization?

Yes it should be possible to combine the two approaches. It would add complexity, but it might offer some benefit in terms of learning speed, stability or ability to cope with non-stationary environments.
Provided the behaviour policy covers all possible actions over the long term, then your choice of how exploration for off-policy reinforcement learning works is one of the hyperparameters for a learning agent. Which means that if you have an idea for a different approach (such as combining two that you have read about), then you need to try it to see if it helps for your specific problem.

* But not necessarily identical, because the approximate Q function would then lose learning inputs that differentiate between optimal and non-optimal behaviour. This can be a tricky balance to get right, and is an issue with catastrophic forgetting in reinforcement learning agents.


Visualizing Support Vector Machines (SVM) with Multiple Explanatory Variables I was wondering if anyone was aware of any methods for visualizing an SVM model where there are more than three continuous explanatory variables. In my particular situation, my response variable is binomial, with 6 continuous explanatory variables (predictors), one categorical explanatory variable (predictor). I have already reduced the number of predictors and I am primarily using R for my analysis.

(I am unaware if such a task is possible/ worth pursuing.)

Thanks for your time.

Does it matter that the model is created in the form of SVM?

If no, I have seen a clever 6-D visualization. Its varieties are becoming popular in medical presentations.
3 dimensions are shown as usual, in orthographic projection.
Dimension 4 is color (0..255)
Dimension 5 is thickness of the symbol
Dimension 6 requires animation. It is a frequency of vibration of a dot on the screen.
In static, printed versions, one can replace frequency of vibration by blur around the point, for a comparable visual perception.

If yes, and you specifically need to draw separating hyperplanes, and make them look like lines\planes, the previous trick will not produce good results. Multiple 3-D images are better.


How to check for dead relu neurons Background:
While fitting neural networks with relu activation, I found that sometimes the prediction becomes near constant. I believe that this is due to the relu neurons dieing during training as stated here. (What is the "dying ReLU" problem in neural networks?)

Question:
What im hoping to do is to implement a check in the code itself to check if the neurons are dead. After that, the code could refit the network if needed.

As such, what is a good citeria to check for dead neurons? Currently im thinking of checking for low variance in the prediction as a citeria.

If it helps, im using keras.

A dead ReLU pretty much just means that its argument value is negative such that the gradient stays at 0; no matter how you train it from that point on. You can simply have a look at the gradient during training to see whether a ReLU is dead or not.

In practice you may simply want to use leaky ReLUs, i.e. instead of f(x) = max(0,x) you set f(x) = x if x > 0 and f(x) = 0.01x if x <= 0. This way you always allow a small non-zero gradient and the unit should not get fully stuck in training anymore.


How to train model to predict 1 value from multiple input samples Very new in ML...

Actual problem is more complex, but I'll give it shorter.

Train data is a collection of samples, describing features of some items. There are many samples for each item. So ML model is implied to predict a few target variables, but one time for each item:

It appears, that I can't find in sklearn or a similar kit a way to fit model with this type of train-input/train-output. Is there any way to do this without hand-implementing whole pipeline?

I've considered this answer: How to predict based on multiple samples?

But I'm not sure it's applicable, since there are many-many samples for each item

Edit 1:
Final model should get a few new samples for completely new item and predict same target variables

Is there really any benefit than just creating three different models, and combining them at the end? If the end use case is just to predict the best final models, then I would just train on them individually.

You could have also do a NN with 3 output nodes, for each target. If you're just looking to do regression or something, I don't see why you wouldn't just predict them independently. There is some literature on multi-output regression (good summary here), but these involve scales of predicting high dimensional vectors(or even matrices), rather than just 3 numbers, so I don't know how useful this literature would be for your case, but that's where I would look if you're insistent on having a single model predict 3 outputs.


Shape extracting on 2D geometric data Given a set of lines, is there a way to train to extract geometric shapes. For example, the picture on left has some blue lines (with red endpoints). How can I train to extract shapes like on the right side? The output must be different shapes with each having a set of points.

I searched for related research papers. I could not find any paper with this exact problem, though some of them came close to this with RNN.

I haven't attempted this, but I'm pretty sure it would work reasonably well.

Identify all points that are connected to more than two lines. We know these must be vertex points.
For for all the points that are attached to only one line, identify their nearest neighbor and draw a line between them (this would connect that hole on the right wall of your house). All points should now be either labeled as a vertex or touching only two lines.
For each point touching two lines, calculate the angle between the two lines.
Find the min and max of these angles.
Use these as prototypes to classify angles into two proximity clusters using agglomerative clustering. Alternatively, pick a fixed angle value as your threshold (maybe around 120 degrees) and use that as the decision boundary for this classification.
The angles that clustered with the min correspond to the missing vertices.
Infer the vertex-vertex connectivity graph through the non-vertex points.(i.e. find all paths between pairs of vertices that do not pass through a third vertex).
Draw the graph.
Find the cycle's in the graph and color each one separately.

Everything doesn't have to be a big fancy neural network :)


Grouping and summing with r I have a data.frame that contains two columns that I want to group and sum the direct labor for employees.

Employees   Actual work
Joe Smo      8
Joe Smo      7
Joe Smo      5
Sam Adams    7
Sam Adams    5
Sam Adams    3

The desired outcome is

Results:
Joe Smo      20
Sam Adams    15

Here is the code I used that doesn't work.

aggregate(directLaborNov18[c('Employee')], directLaborNov18['Actual_work'], sum)

You were very close. Just switch the first two input arguments.

> aggregate(directLaborNov18['Actual_work'], directLaborNov18[c('Employee')],  sum)
   Employee Actual_work
1   Joe Smo          20
2 Sam Adams          15


Python XGBoost killing kernel My Jupyter notebook's python kernel keeps dying when attempting to train an XGBoost logistic classifier. Previously, I have run all of the following code successfully. Presently, there are issues. First, I will show you the code chunk that I am able to run successfully:

import xgboost as xgb
xgtrain = xgb.DMatrix(data = X_train_sub.values, label = Y_train.values)       # create dense matrix of training values
xgtest  = xgb.DMatrix(data = X_test_sub.values,  label = Y_test.values)        # create dense matrix of test values
param   = {'max_depth':2, 'eta':1, 'silent':1, 'objective':'binary:logistic'}  # specify parameters via map

where my data is small:

X_train_imp_sub.shape
(1365, 18)

however, my notebook's kernel keeps dying on this chunk:

xgmodel = xgb.train(param,  xgtrain, num_boost_round = 2)                      # train the model
predictions = xgmodel.predict(xgtest)                                          # make prediction
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_true = Y_test,
                          y_pred = predictions.round(),
                          normalize = True) # If False, return # of correctly classified samples. Else, return fraction of correctly classified samples
print("Accuracy: %.2f%%" % (accuracy * 100.0))

As stated before, I have been able to run both chunks successfully before. I have shut down all other notebooks and restarted my computer without luck. I am launching jupyter through Anaconda Navigator on a Macbook Pro.

UPDATE:

To further debug, I used Terminal CLI and ran:

conda install runipy
runipy MyNotebook.ipynb

which was very helpful, because it produced the following error message that I could then Google:

OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

I found this thread:

https://github.com/dmlc/xgboost/issues/1715

Which shows others having the same problem on MacOS. I also suspected the problem being linked to matplotlib somehow because running any cell with  matplotlib related code somehow causes the xgb.train to kill the kernel.

When I selected a cell beneath my xgboost training cell, then selected: Cells --> Run All Above, the kernel would always die on the xgboost training cell. This happened probably 40-50 times in a row.

When I ran the same cells one-by-one, it completed fine on the first time I tried this and every time after.


Help improving time series prediction with LSTM on PyTorch So, I am trying to use a LSTM model to forecast temperature data on PyTorch. I am relatively new to both PyTorch and the use of recurrent networks so I took a model I found on the internet to start. Unfortunately, I am working with missing data and I am assigning the value 0 to it. The whole project is on github if you need more details.
From the templates, I split it into two approaches.
One train2.py takes a tuple $x=(x_0,...,x_{n-1})$ as input and uses $y=(x_1,...,x_n)$ as target. From there, I recursively call the model $N$ times to forecast $N$ times into the future. Here's the part of the code that matters:
class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1):
        super().__init__()
        self.input_size = input_size

        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)

        self.lstm2 = nn.LSTM(hidden_layer_size, input_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())

        self.hidden_cell_2 = (torch.zeros(1,1,self.input_size).double(),
                            torch.zeros(1,1,self.input_size).double())

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        lstm_out2, self.hidden_cell_2 = self.lstm2(lstm_out,self.hidden_cell_2)
        predictions = lstm_out2.view(len(input_seq), -1)
        return predictions.view(-1)

Here's how the output looks like (predict2.html)

It kind of gets the oscillatory behavior but the amplitude is way off.
The second one, train.py takes a tuple $x=(x_0,...,x_{\frac{n}{2}-1})$ as input and $y=(x_{\frac{n}{2}},...,x_n)$ as output. For predictions in this one, I make a single call to the model and I can only look at $N<\frac{n}{2}$ points into the future. Here's the code:
class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1, window=10):
        super().__init__()
        self.window = window

        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)

        self.linear = nn.Linear(hidden_layer_size, output_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())
        self.sigmoid = nn.Sigmoid()

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        predictions = self.linear(lstm_out.view(len(input_seq), -1))
        return self.sigmoid(predictions[-self.window:].view(-1))

The output looks like this:

It fails to keep continuity most of the time, probably because it doesn't have a reference starting point in the output?
With all of that said my question is, how to improve the prediction? Should I add more data to it? Am I using too many hidden layers? Is the use of sigmoid flattening the fluctuations in the data too much? Is the model just completely wrong? I need guidance here.
Thank you for your time.

I think I was able to get it working now. What I did was to change the assigned nan value to 25 instead of 0 (which would be about the average) and normalize the values within the interval (-1,1) with a scaled sigmoid shifted on 25.
This is how it looks now:


How to include both origin and destination in your features? I'm trying to predict the price of transportation for trucking freight. Two important features that I think would be of great impact are Origin and Destination. What's the best way to include that in your features?

They are categorical variables and if I encode it, the dataset will be extremely sparse. I thought about converting both features into latlong coordinates too and treat them as numerical variables.

Anyone dealt with this situation before?

Since your goal is to predict the price, I think it would be more useful to include features such as:

distance between origin and destination
whether origin and destination are in the same state/country/continent/area...

However the actual origin and destination might still be useful, at least the frequent ones, so it's worth experimenting. Those which appear only once or twice in your training data are unlikely to help the model, on the contrary they might cause overfitting. So you could filter out the origin/destination cities which appear less then $N$ times and keep the other ones. It's very likely that this will reduce the number of possible values a lot.


Correct theoretical regularized objective function for XGB/LGBM (regression task) I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective function for XGBoosting. Below you can find the equation given by the developers of the XGB algorithm for the regularized objective function (equation 2). The paper is called "XGBoost: A Scalable Tree Boosting System" by Chen & Guestrin (2016).
In the Python API from the xgb library there is a way to end up with a reg_lambda parameter (L2 regularization parameter; Ridge regression equivalent) and a reg_alpha parameter (L1 regularization parameter; Lasso regression equivalent). And I am a bit confused about the way the authors set up the regularized objective function. According to my understanding the ridge penalty is given by $\lambda \sum_{j=1}^p \beta_j^2$ and the lasso penalty is given by $\lambda \sum_{j=1}^p |\beta_j|$. However, the authors of the paper seem to use a combination of the two things that is not equivalent to either one since they apply the absolute value and square the beta term.
My questions are therefore the following ones:
(a) How does the regularized objective function look like if you want to allow for the inclusion of a l1 and a l2 parameter in the same model?
(b) How does the objective function of the LGBM(light GBM) algorithm differ from XGB or is it identical since the developers of the LGBM algorithm do not provide any theoretical details about the objective function or the iterations of boosting in their paper "LightGBM: A Highly Efficient Gradient Boosting" (Ke et al., 2017). According to my understanding, the only difference is the computation which is faster since the LGBM algorithm computes the information gains on the regression tree splits more efficiently and the features are bundled or am I misinterpreting/overlooking any aspects here?
(c) does it make sense to include both a l1 and a l2 regularization term in the same boosting model from a theoretical and practical perspective?

For question (a), it should be as simple as adding the usual L1 penalty:
$$\begin{align*}
\Omega(f) &= \gamma T + \alpha \|w\|_1 + \frac12\lambda\|w\|_2^2 \\
&= \gamma T + \alpha \sum |w_i| + \frac12\lambda \sum w_i^2
\end{align*}$$
For (b), see LightGBM vs XGBoost
For (c), see L1 & L2 Regularization in Light GBM