Looking for an RL solution for sequence generation I am looking for a proper reinforcement learning solution for the following problem:

Suppose I have a pool of candidate functions f \in Pool(it's like f1, f2, ... fn), and I am trying to synthesize several sequences of functions, each sequence contains arbitrary amount of f.

Given a sequence s, when deciding whether to extend s by appending a new function f and forming a new sequence s' = sf, I have the following reward scheme:

when F(s') = true, reward += 1
when T(s') = true, reward += 3
every time the sequence grows by one, I try to penalize by reward -= 0.5

In other words, I don't want each sequence s to be too long, but also will consider cases when special property F and T are satisfied.

I view this as a learning problem, and envision it can be solved by reinforcement learning techniques, such as DQN or so. But I am a newbie to this field and find it difficult for me to formalize this problem. Could anyone can shed some lights on this and what kind of reinforcement learning algorithms/scheme I can have a try? Thanks a lot. Any suggestion or advice will be appreciated.

To formulate your problem as an RL problem, we first need to index the functions from $1$ to $n$. This way, there would be $n+1$ actions in total, i.e.
$$\begin{align*}
a^i=\left\{\begin{matrix}
\text{stop} && i=0\\
\text{add } f_i && 1\le i\le n
\end{matrix}\right.
\end{align*}$$
Accordingly, state $s_t$ would be represented by a sequence of $t$ actions, where a final state ends with action $a^0$. Also, the reward at step $t$ (after doing action $a_t$) would be as follows
$$r_t = \left\{\begin{matrix}
-0.5 + 1_{F(s_ta_t)=True} + 3_{T(s_ta_t)=True} & a_t \neq a^0\\
0 & a_t = a^0
\end{matrix}\right.$$

To prepare such a representation for a neural network, we need to represent the actions with one-hot encoding. Here is an example,

s = [
      [0, 0, 0, 1, 0]  #a^1
      [0, 0, 1, 0, 0]  #a^2
      [0, 0, 0, 0, 1]  #a^0
    ]

which represents a final state with three actions (and four functions).

The tricky part of your problem is the ever-changing representation of the state. As an opposite example, for those video games that are studied in DQN paper, each state is a constant number of fixed-sized 2D images.

To solve this design challenge, we can opt for one of these two approaches (among possibly many others):

Limit the final length of states to $T$ and add a "not taken" action for those steps that are not taken (yet). For example,

 s = [
    [0, 0, 0, 0, 1, 0]  #a^1
    [0, 0, 0, 1, 0, 0]  #a^2
    [0, 0, 0, 1, 0, 0]  #a^2
    [1, 0, 0, 0, 0, 0]  #not taken
    [1, 0, 0, 0, 0, 0]  #not taken
  ]

for four functions and $T=5$. This way, we end up with a total of $n+2$ symbols for representing each state.
Using a recurrent neural network (RNN, LSTM, etc.) to encode the variable-length states to a fixed length representation; this is similar to encoding variable-length sentences to a fixed length representation in NLP tasks.

The problem is now completely formulated as an RL problem along with taking some implementation challenges into consideration.

From this point onward, we can do RL  by, for example, designing a 1D Dense network on top of RNN representations, or a 2D Dense or Convolutional network on top of fixed-length 2D representations (with $m \times (n+2)$-sized kernels that only move along actions, not along $n+2$ dimensions of one-hot encoding) that receives the representation of state $s$, denoted as $\phi(s)$, and outputs $Q(s, a)$ for each of $n+1$ actions, the same as DQN paper (see Algorithm 1: deep Q-learning with experience replay).

Extra notes

You may take a look at this post for feeding variable-length multi-dimensional sequences to LSTM using Keras.


How do I interpret my result of clustering? I am working on a clustering problem. I have 11 features. My complete data frame has 70-80% zeros. The data had outliers that I capped at 0.5 and 0.95 percentile. However, I tried k-means (python) on data and received a very unusual cluster that looks like a cuboid. I am not sure if this result is really a cluster or has something gone wrong?

The main reason for my worry, why is it looking like a cuboid and why are the axes orthogonal?

one thing to notice is that: I first reduced the dimensionality using PCA to two dimensions and performed clustering on the same and the plot here is on the 2-dim PCA data

Edit : I chose k using silhouette index in python.

K-means don't modify the underlying structure of your data. K-means will just provide the 'color' part of your graph.

To answer the question about why do you get a cuboid, it's because your underlying data are a cuboid. Not necessarily by construction, but that's what happen when you cap your data. As an exemple, look at the following code :

X1 = c(rnorm(1000))
X2 = c(rnorm(1000))
q95_1 = quantile(X1,0.95)
q95_2 = quantile(X2,0.95)
q5_1 = quantile(X1,0.05)
q5_2 = quantile(X2,0.05)
X1[X1>q95_1]=q95_1
X2[X2>q95_2]=q95_2
X1[X1<q5_1]=q5_1
X2[X2<q5_2]=q5_2
plot(X1,X2)

The code simulate two random gaussian and cap them at 5% and 95%.

this is what you get :

Notice the squaroid pattern ? This is why you get a cuboid in 3D.

Ps: I can't help but say that's what you get when you do k-means without properly looking at your variables (see: What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering? for an infinite loop).


Algortihm for making predictions from minimal data I am working with classification problem.
 I have a dataset with a lot of features. A lot of them can easily determine class. On production, I want to ask the user to provide me with only part of the information.

Which algorithm is able to make pretty accurate predictions from incomplete data?

I was thinking about decisions tree, but I don't want to determine the entry point to my algorithm.

Suppose that my features are binary. I want to give choice to my user which features are most common for him and based on that information make pretty accurate predictions.

If your feature set is really huge and most of the variables are strong features and can determine a class on their own, i would try to go with Logistic Regression using elastic net (probably less on the lasso and more on the ridge side, so you do not throw away features which happen to always co-occur in your dataset but not in reality).

Just be sure to only consider features that appear often enough in order to avoid overfitting. This can easily happen, when the number of examples is smaller than the number of features.

Random Forests could also be worth a try, but to my experience if the number of strong features is really great but they all occur relatively rarely, RFs underperform compared to Logistic Regression.

Btw. the fact that the user only reveals partial information might not be a problem at all as long as the information given can determine the class easily - as you said. I would just give it a try.


Difference between interpolate() and fillna() in pandas Since interpolate and fillna method does the same work of filling na values. What is the basic difference between the two. What is the significance of having these two different methods?? Can anyone explain me in layman terms. I already visited through the official documentation and wanted to know the difference

fillna fills the NaN values with a given number with which you want to substitute. It gives you an option to fill according to the index of rows of a pd.DataFrame or on the name of the columns in the form of a python dict.

But interpolate is a god in filling. It gives you the flexibility to fill the missing values with many kinds of interpolations between the values like linear (which fillna does not provide) in the example provided below and many more interpolations possible. For example

>> import pandas as pd, numpy as np
>> df = pd.Series([1, np.nan, np.nan, 3])
>> df.interpolate()
0    1.000000
1    1.666667
2    2.333333
3    3.000000
dtype: float64

Pandas documentation on fillna and interpolate is very clear on this.


How to plot two distribution curves in a faceted way in R/ggplot2? I have two probability distribution curves, a Gamma and a standarized Normal, that I need to compare:

library(ggplot2)

pgammaX <- function(x) pgamma(x, shape = 64.57849, scale = 0.08854802)

f <- ggplot(data.frame(x=c(-4, 9)), aes(x)) + stat_function(fun=pgammaX)
f + stat_function(fun = pnorm)

The output is like this

However I need to have the two curves separated by means of the faceting mechanism provided by ggplot2, sharing the Y axis, in a way like shown below:

I know how to do the faceting if the depicted graphics come from data (i.e., from a data.frame), but I don't understand how to do it in a case like this, when the graphics are generated on line by functions.
Do you have any idea on this?

I would separate the creation of the data from the plotting operation, if I were you. ggplot2::facet_wrap allows you to plot each of the cumulative distributions in each own panel. Here is how I would do it.

pgammaX <- function(x) pgamma(x, shape = 64.57849, scale = 0.08854802)

Gamma <- pgammaX(seq(from=4, to = 8, by = 0.1))
Normal <- pnorm(seq(from=-4, to = 3.9, by = 0.1))

data <- data_frame(
  x = seq(-4,8,by=0.1),
  y = c(Normal, Gamma),
  # use factor to make the plot for Normal to appear first
  distribution = factor(c(rep(c("Normal", "Gamma"),
                          times = c(length(Normal), length(Gamma)))),
                          levels = c("Normal", "Gamma"))
)

library(dplyr)
library(ggplot2)

data %>%  ggplot(aes(x, y)) +
  geom_line() +
  # use facet_wrap to plot each cummulative distribution in each own panel
  facet_wrap(~distribution, scales = "free_x")


How to create a stacked bar chart with gaps between values of each variables in Python I have a problem with creating a stacked bar chart in Python. I have data with 3 variables as below:

A=[3,5,7]
B=[4,5,7]
C=[2,3,4,5,6,7]

I would like to create a bar chart with gaps of values of each variable as below bar chart with gaps between values of each variable :

Could anyone know how to create it in Python?
Thank you very much.

You may apply Wolfram Language to your project. There is a free Wolfram Engine for developers and with the Wolfram Client Library for Python you can use these functions in Python.

from wolframclient.evaluation import WolframLanguageSession
from wolframclient.language import wl, wlexpr

Start the session.

wolfSession = WolframLanguageSession();
A=[3,5,7];
B=[4,5,7];
C=[2,3,4,5,6,7];

Either the ArrayPlot or MatrixPlot functions can be applied. These functions take 2D matrices so first need to convert the lists.

With SparseArray a matrix can be specified without explicitly setting values for all positions. Empty positions are set to '' below.

pos=wolfSession.evaluate(
    wl.Flatten(
        wl.MapIndexed(
            wl.Function(wl.Rule([wl.First(wl.Slot(2)), wl.Slot(1)], wl.First(wl.Slot(2)))),
            [A,B,C],
            [2]
        ),
        1
    )
);
sparse = wolfSession.evaluate(wl.SparseArray(pos, wl.Automatic, ''));

print(wolfSession.evaluate(wl.Normal(sparse)))

[['', '', 1, '', 1, '', 1],
 ['', '', '', 2, 2, '', 2],
 ['', 3, 3, 3, 3, 3, 3]]

Using custom ColorRules and FrameTicks to get a basic representation of your plot.  You can use these and many other options to get you precise look you

stacked=wolfSession.evaluate(
    wl.ArrayPlot(
        sparse,
        AspectRatio = 1/4,
        ColorRules = wl.Flatten(
            [wl.Thread(wl.Rule(wl.Range(3),[wl.Green,wl.Orange,wl.Blue])),
            wl.Rule(wl.Blank(),wl.White)]
        ),
        FrameTicks = [
            [wl.Thread([range(1,4), ['A','B','C']]), wlexpr('None')],
            [
                wl.Thread([
                    wl.Range(7),
                    list(map(lambda r: str(r)+'k', range(1,8)))
                ]),
                wlexpr('None')
            ]
        ]
    )
);

Then Export in one of the supported Raster Image Formats or Vector Graphics Formats.

wolfSession.evaluate(
    wl.Export(
        '<path with image filename>',
        stacked
    )
)

Terminate session

wolfSession.terminate()

Hope this helps.


What exactly means CNN is position equivariant There is quite a good explanation which fully comply with my vision. But seems it lacks one final step. As Jean states, moving an object significantly in the input image will cause the change in which neuron is activated in the yellow layer (the one previous to the first fully connected layer). So that we see that the part of the network before FCL is  position equivariant. Then  author says that because the network detects an object at any location, the FCLs should have taken care of it.

Does the equivariance property hold for the whole network, including trailing classifier? (I've read Difference between "equivariant to translation" and "invariant to translation" but not sure I apply it to this case correctly)
Should a network detect an object in right-top corner if only trained with images with that object at left-bottom corner?

(Tried  this demo but it doesn't seem to address the second question)

In your link, the author Jean states:

  Additionally, I believe that if a CNN is trained showing faces only at one corner, during the learning process, the fully-connected layer may become insensitive to faces in other corners.

I also believe this to be correct. The FCN does not in any way add or improve translation invariance. Instead it will treat all outputs - each individual pixel of the "feature maps" - of the last convolutional layer as entirely different features. It must be trained with enough examples in order to generalise well.

However, the feature maps are not themselves simple, clean detectors of objects as the simplified explanation of CNNs might imply. In a deep network they can be very complex and respond to a wide range of stimuli. They will also respond somewhat fuzzily, so that e.g. an eye or the side of a head showing an ear can trigger multiple feature map pixels (to the feature map's kernels, the same object slightly translated will look like distorted version of the same feature, and will still match enough sub-components of the object to trigger a positive response). The last layer will not necessarily detect full objects, but significant chunks of objects, areas of important texture etc. So position can still be quite fluid, and the "head detector one pixel off in last convolutional layer" scenario is not particularly realistic - although it may affect relative strength/confidence of predictions.

This can still be a problem if you need your network to generalise. If you suspect that your training data might not be covering enough variations in position, orientation etc of images, then a common approach is data augmentation. As a reflected, rotated or cropped image of an object should usually be classified as the same object, then you can pre-process your training data using those transforms to make many random variations of the input images. Some deep learning frameworks will allow you to do this continuously, generating fresh images for each and every batch.


Is applying pre-trained model on a different type of corpus called transfer learning? I trained my classification model on corpus A and evaluated it on corpus B.
I do it, because for corpus A I have a lot more labeled sentences than for B. Nature of sentences used in A is different than sentences using in B. A has name of products from e-shop, B has names of products as they appear in shopping lists, with all slang, abbreviations, spelling errors and private notes.
Am I doing transfer learning?

The basic concept of transfer learning is: Storing knowledge gained from solving one problem and applying it to different but related problem
I guess to be precise this is called Transductive Transfer Learning. In this we learn from the already observed training dataset and then predict the labels of the testing dataset. Even though we do not know the labels of the testing datasets, we can make use of the patterns and additional information present in this data during the learning process.
Refer: Ruder


How to distinguish between normal fluctuation and outliers in ARIMA model? I have a dataset about sales per day of certain products at the ITEM/DAY/STORE level , I've plotted the series and visually examined it for any outliers, volatility, or irregularities.

And this is what i got :

So there is huge spikes and drops in sales , so i used the  tsclean() function - tsclean() identifies and replaces outliers using series smoothing and decomposition-

And this is what i got after plotting :

But now , i'm wondering if what i did was right or not , in the sense maybe those fluctuations were more natural fluctuations than outliers , even though it seems quit weird to me to have a spike in sale of 4000 units in a day and than drops to 2000 the next day.

Any insight would be much appreciated , thank you .

What you have is the fundamental question of statistics, what is noise and what is signal. It’s true that from a modeling perspective it’s easier to model something that’s been nicely smoothed and trimmed, but is that what you want? That’s a hard question to answer in general. I will say that with time data it is quite normal to have seasonal components which I do see in the original but do seem absent in the cleaned data so I err on the side of too clean. I would look for some a more sensitive method than tsclean.


numerical or categorical data I have a feature for machine learning (using methods like SVM, naive bayes, neural network and random forest) called member duration as follows:
Should I make it as numerical or categorical data?

You definitely have interval data, that is, data which takes on discrete values, as opposed to continuous data, which takes on values along a continuum.

It may be of value to additionally determine if the data is ordinal, meaning that the order of the values is important, for example if [0, 1, 2] signifies [small, medium, large] or some analogous system.

In the case of ordinal data, it may be best to keep the data as exposed to the SVM training process in integer form, as the integer representation encodes some information about the relationship between the categories.

This approach would also be more reasonable if the values that the variable could take on in a production setting could expand beyond the values you've already observed in the training set- a categorical approach would be less able to handle new values in that context.

If there are no ordinal relationships and you suspect all of the possible values are enumerated in the training set,treating the variable as categorical would be approriate.


Deep Learning vs gradient boosting: When to use what? I have a big data problem with a large dataset (take for example 50 million rows and 200 columns). The dataset consists of about 100 numerical columns and 100 categorical columns and a response column that represents a binary class problem. The cardinality of each of the categorical columns is less than 50.

I want to know a priori whether I should go for deep learning methods or ensemble tree based methods (for example gradient boosting, adaboost, or random forests). Are there some exploratory data analysis or some other techniques that can help me decide for one method over the other?

Why restrict yourself to those two approaches? Because they're cool? I would always start with a simple linear classifier \ regressor. So in this case a Linear SVM or Logistic Regression, preferably with an algorithm implementation that can take advantage of sparsity due to the size of the data. It will take a long time to run a DL algorithm on that dataset, and I would only normally try deep learning on specialist problems where there's some hierarchical structure in the data, such as images or text. It's overkill for a lot of simpler learning problems, and takes a lot of time and expertise to learn and also DL algorithms are very slow to train. Additionally, just because you have 50M rows, doesn't mean you need to use the entire dataset to get good results. Depending on the data, you may get good results with a sample of a few 100,000 rows or a few million. I would start simple, with a small sample and a linear classifier, and get more complicated from there if the results are not satisfactory. At least that way you'll get a baseline. We've often found simple linear models to out perform more sophisticated models on most tasks, so you want to always start there.


Does it make sense to concatenate datasets to improve accuracy of model? I'm planning on training an NER model, I already do have a large corpus but I did find one more large corpus and I'm quite confident that I can source even more corpora and format its data to my needs.

Assuming all datasets eventually are in the same format and don't have any inconsistencies, does it make sense to concatenate all datasets?

Or does the model converge anyways at some point? Does that make sense at all? What other problems could I encounter?

More data is always better. It makes 100% sense to concatenate all datasets in to a larger one, assuming that they are in the same format. This will most likely improve the performance of your model.


Theano in deep learning research How widely is Theano used in deep learning research?

Is Theano a good start to learn the implementation of machine learning algorithms?

Will learning the implementation of something like a feed forward network really help? Do graduate students implement neural networks or other algorithms at least once during their college days?

Background:

I have a reasonable idea about feed forward and recurrent networks, backpropagation, the general pipeline for a machine learning problem and the necessary mathematics.

Theano used to be very popular in the last few years. However, from seeing what the top research labs in the world currently use instead of theano (Facebook AI and Google Deepmind), I would either go for caffe or Torch7, with a stronger preference on the latter.
The reason is that you can implement from Recurrent Neural Nets to Convolutional Neural Networks very easily and there is a wide range of examples on github.
Graduate students in NYU, Stanford, Oxford definitely implement advanced models in their studies; your background will help you a lot, a great place to start and familiarise with such models, and implement them in Torch7, is the lessons of Prof. Nando de Freitas.


Product Recommendation based on purchase history I am dealing with problem where i have to increase the sales by product recommendation.I only have customer data and product that they have purchased.No ratings,reviews or feedback is present.What approach fit best for my problem.

Even if you don't have ratings or reviews, you can use the customer purchases to help creating your model and selecting the most appropriate one. If the customer has bought the product, you can suppose he has liked it. If he hasn't, we can suppose he has disliked it. This is an approach which is mainly used by e-shops.

You can find more on recommendation system selection and validation below. It is discussed how to choose the best recommender with both offline and online approaches.

https://medium.com/recombee-blog/evaluating-recommender-systems-choosing-the-best-one-for-your-business-c688ab781a35


Advantages of pandas dataframe to regular relational database In Data Science, many seem to be using pandas dataframes as the datastore. What are the features of pandas that make it a superior datastore compared to regular relational databases like MySQL, which are used to store data in many other fields of programming?

While pandas does provide some useful functions for data exploration, you can't use SQL and you lose features like query optimization or access restriction.

I think the premise of your question has a problem. Pandas is not a "datastore" in the way an RDBMS is. Pandas is a Python library for manipulating data that will fit in memory. Disadvantages:

Pandas does not persist data. It even has a (slow) function called TO_SQL that will persist your pandas data frame to an RDBMS table.
Pandas will only handle results that fit in memory, which is easy to fill. You can either use dask to work around that, or you can work on the data in the RDBMS (which uses all sorts of tricks like temp space) to operate on data that exceeds RAM.


Linear Regression Error I tried creating a simple linear regression model on just 30 rows of data. I got this error while trying to fit the model:

dataset = pd.read_csv('Salary_Data.csv')
x=dataset.iloc[:, :-1]
y=dataset.iloc[:, 1]

from sklearn.cross_validation import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 1/3, random_state = 0)
regressor = LinearRegression()
regressor.fit(x_train, y_test)

Here is the error message I got:

ValueError                                Traceback (most recent call last)
<ipython-input-53-8dc82dc6fe8b> in <module>()
----> 1 regressor.fit(x_train, y_test)

~\Anaconda3\lib\site-packages\sklearn\linear_model\base.py in fit(self, X, y, sample_weight)
    480         n_jobs_ = self.n_jobs
    481         X, y = check_X_y(X, y, accept_sparse=['csr', 'csc', 'coo'],
--> 482                          y_numeric=True, multi_output=True)
    483
    484         if sample_weight is not None and np.atleast_1d(sample_weight).ndim > 1:

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_X_y(X, y, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, warn_on_dtype, estimator)
    581         y = y.astype(np.float64)
    582
--> 583     check_consistent_length(X, y)
    584
    585     return X, y

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_consistent_length(*arrays)
    202     if len(uniques) > 1:
    203         raise ValueError("Found input variables with inconsistent numbers of"
--> 204                          " samples: %r" % [int(l) for l in lengths])
    205
    206

ValueError: Found input variables with inconsistent numbers of samples: [20, 10]

Your code has two typos:

1. Error when selecting data for the target variables.

In the second line, you are using [:, :-1] which means that you select all rows, and all columns except the last one.

In the third line, you are using [:, 1] which means that you select all rows, and only the first column instead of the last.
Instead, you want to select the columns of your target variable, i.e. the last one:

y=dataset.iloc[:, -1]

2. Error when fitting the model

You are fitting your model using the X training set and the Y validation set. They have different length. Use this instead:

regressor.fit(x_train, y_train)


Are support vector machines and logistic regression equivalent if data is linearly separable? I understand that SVMs separate data drawing an hyperplane with the biggest margin, but doesn't logistic regression do the same thing if data is linearly separable?

Very closely, but not exactly. This post by Georgios Drakos goes through the question mathematically and visually. Here is an image from the post:

which compares soft-margin SVM (SVM) and Logistic Regression (LR).


Creating labels for Text classification using keras I have a text file with information that needs to classified based on keywords. The text file contains many number of paragraphs. And the paragraph contains keywords that we want (lets say salary amount, interest rate and so on..)

I want to write a model which will extract the paragraph (or 3 to 4 lines of text) containing the keyword i want. How do i create a label in this case? All i have is a raw text.

I am new to NLP. Any suggestions how i can approach this?

You can build the text classification application with CNN algorithm by Keras library.
Please take a look at this git repository. Here

As you can see, you need to create training and testing data by loading polarity data from files, splitting the data into words, generating labels and returning split sentences and labels.
And you can create the convolutional neural network with Dense, Embedding, Conv2D, MaxPool2D of keras.

Here is the final model training snippet.

from keras.layers import Input, Dense, Embedding, Conv2D, MaxPool2D
from keras.layers import Reshape, Flatten, Dropout, Concatenate
from keras.callbacks import ModelCheckpoint
from keras.optimizers import Adam
from keras.models import Model
from sklearn.model_selection import train_test_split
from data_helpers import load_data

print('Loading data')
x, y, vocabulary, vocabulary_inv = load_data()

# x.shape -> (10662, 56)
# y.shape -> (10662, 2)
# len(vocabulary) -> 18765
# len(vocabulary_inv) -> 18765

X_train, X_test, y_train, y_test = train_test_split( x, y,     test_size=0.2, random_state=42)

# X_train.shape -> (8529, 56)
# y_train.shape -> (8529, 2)
# X_test.shape -> (2133, 56)
# y_test.shape -> (2133, 2)

sequence_length = x.shape[1] # 56
vocabulary_size = len(vocabulary_inv) # 18765
embedding_dim = 256
filter_sizes = [3,4,5]
num_filters = 512
drop = 0.5

epochs = 100
batch_size = 30

# this returns a tensor
print("Creating Model...")
inputs = Input(shape=(sequence_length,), dtype='int32')
embedding = Embedding(input_dim=vocabulary_size,     output_dim=embedding_dim, input_length=sequence_length)(inputs)
reshape = Reshape((sequence_length,embedding_dim,1))(embedding)

conv_0 = Conv2D(num_filters, kernel_size=(filter_sizes[0],     embedding_dim), padding='valid', kernel_initializer='normal',     activation='relu')(reshape)
conv_1 = Conv2D(num_filters, kernel_size=(filter_sizes[1],     embedding_dim), padding='valid', kernel_initializer='normal',     activation='relu')(reshape)
conv_2 = Conv2D(num_filters, kernel_size=(filter_sizes[2],     embedding_dim), padding='valid', kernel_initializer='normal', activation='relu')(reshape)

maxpool_0 = MaxPool2D(pool_size=(sequence_length - filter_sizes[0] + 1, 1), strides=(1,1), padding='valid')(conv_0)
maxpool_1 = MaxPool2D(pool_size=(sequence_length - filter_sizes[1] + 1, 1), strides=(1,1), padding='valid')(conv_1)
maxpool_2 = MaxPool2D(pool_size=(sequence_length - filter_sizes[2] + 1, 1), strides=(1,1), padding='valid')(conv_2)

concatenated_tensor = Concatenate(axis=1)([maxpool_0, maxpool_1, maxpool_2])
flatten = Flatten()(concatenated_tensor)
dropout = Dropout(drop)(flatten)
output = Dense(units=2, activation='softmax')(dropout)

# this creates a model that includes
model = Model(inputs=inputs, outputs=output)

checkpoint = ModelCheckpoint('weights.{epoch:03d}-{val_acc:.4f}.hdf5',     monitor='val_acc', verbose=1, save_best_only=True, mode='auto')
adam = Adam(lr=1e-4, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)

model.compile(optimizer=adam, loss='binary_crossentropy', metrics=['accuracy'])
print("Traning Model...")
model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, callbacks=[checkpoint], validation_data=(X_test, y_test))  #     starts training

By running this code, you will get the trained model with the format of hd5. Finally, you can use your model for prediction.


Optimizing parameters for a closed (black-box) system I am working on a problem that involves finding optimal parameter values for a black-box system. This system consists of 6 inputs and produces a single output. I can compare the returned values to observed values to determine whether the system is well calibrated based on the parameter values I specified.

The main problem that I am having is that I do not know how these parameters are used in the black-box system to produce the output. The parameters may be correlated so changing the value of one parameter may affect the way the others behave in the system.

Since I do not know the function I am not sure how to optimize this problem efficiently.

Question: what are some methods for optimization when the function is unknown?

Edits:

The variable types are each a vector of real numbers but we can make some assumptions if it is is helpful.

The cost to run the black-box system is only time. Let's say that it takes 10 seconds to run the system but I happen to have five black boxes -- so if I can run my algorithm in parallel then I can cut down the run time. If the algorithm runs sequentially then I don't gain anything by having the extra boxes (expect perhaps choosing different starting positions).

For this problem I am interested in learning about how to solve this type of problem -- not trying to brute force my way to a solution. That is to say I am less interested in obtaining an answer and more finding out how I would obtain the the answer. I think it is easy to say that given sufficient resources (time, processing power) that simply generating many many random numbers and choosing the combination that gives the best result is the simplest (and probably worst) approach.

There are lots of function optimising routines that could be applied, based on the description so far. Random search, grid search, hill-climbing, gradient descent, genetic algorithms, simulated annealing, particle swarm optimisation are all possible contenders that I have heard of, and I am probably missing a few.

The trouble is, starting with next to zero knowledge of the black box, it is almost impossible to guess a good candidate from these search options. All of them have strengths and weaknesses. To start with, you seem to have no indication of scale - should you be trying input parameters in any particular ranges? So you might want to try very crude searches through a range of magnitudes (positive and negative values) to find the area worth searching. Such a grid search is expensive - if you have $k$ dimensions and want to search $n$ different magnitudes, then you need to call your black box $n^k$ times.

This can be done in parallel though, and given you are confident that the function is roughly unimodal, you can start with a relatively low number of n (maybe check -10, -1, 0, +1, +10 for 15625 calls to your function taking roughly 8 hours 40 mins using 5 boxes). You may need to repeat with other params once you know whether you have found a bounding box for the mode or need to try yet more values, so this process could take a while longer - potentially days if the optimal value for param 6 is more like 20,000. You could also refine more closely, once you have a potential mode you might want to define another grid of values to search based around. This basic grid search might be my first point of attack on a black box system where I had no clue about parameter meaning, but some confidence that the black box output had a rough unimodal form.

Given the speed of response you should be storing all input and output values in a database for faster lookup and better model building later. No point repeating a call taking 10 seconds when a cache could look it up in 1 millisecond.

Once you have some range of values you think that a mode might be in, then it is time to pick a suitable optimiser.

Given the information so far, I would be tempted to run either more grid search (with a separate linear scaling between values of each param) and/or a random search, constrained roughly to the boxes defined by the set of $2^6$ corner points around the best result found in initial order-of-magnitude search.

At that point you could also consider graphing the data, to see if there is any intuition about which other algorithms could perform well.

With the possibility of parallel calls, then gradient descent might be a reasonable guess, because you can get approximate gradients by adding a small offset to each param and dividing difference that causes in the output by it. In addition, gradient descent (or simple hill climbing) has some chance of optimising with less calls to evaluate the function than approaches that rely on many iterations (simulated annealing) or lots of work in parallel (particle swarm or genetic algorithms). Gradient descent optimisers as used in neural networks, with additions like Nesterov momentum or RMSProp, can cope with changes in function output "feature scale" such as different sizes and heights of peaks, ridges, saddle points.

However, gradient descent and hill climbing algorithms are not robust against all function shapes. A graph or several of what your explorations are seeing may help you to decide on a different approach. So keep all the data and graph it in case you can get clues.

Finally, don't rule out random brute-force search, and being able to just accept "best so far" under time constraints. With low knowledge of the internals of the black box, it is a reasonable strategy.


How to find anomalies in (almost) constant stream of data? I have a process that (simply put),  starts every 5 minutes, collects data, and put that data into the database.
More detailed explanation would be that process starts, collects data (which takes some time) and put it on kafka topic (which takes some time). Finally, data from kafka topic are consumed by database (which also take some time).
Every record in the database has its insertedOn time rounded up to the second.
When I count records (for 4 hours) by insertedOn time the graph look like this:

If I count records by 5 minutes intervals, the graph look like this:

On this graph you can see that all dots are around the same level (just a little bit above than 7000), but the dot marked with red arrow and its neighbor from the left, are bellow 7000.
Min, mean and max for these counts per 5 minutes are:
min     6262
mean    7154
max     7186

As the dot marked with red arrow is around 12% bellow mean or max, we could (probably) consider it as anomaly.
I am aware of several ML outlier/anomaly detection algorithms, but I am not sure how to use them when new data are constantly coming to the database. Whatever I do, I would like to avoid using fixed thresholds (like if count falls 10% bellow an average raise an alarm).
For example, the dot market with red arrow on the graph above happened at 2021-01-06 14:30:00, so few minutes later I should have raised an alarm because of that.
At the moment, this is the procedure that I have come up with regarding this 5 minutes data collection process. The following will be executed every 5 minutes:

take from database last few hours (time window) of counts by 5 minutes (n) periods
drop last period (n) because it is maybe not completed (records still not in the database)
use some ML algorithm (still not sure which one) on time window (up to n-2) in order to see whether count for last period (n-1) is an anomaly
if anomaly, raise an alarm
exclude an anomaly data point from future data collections in step 1.

I am not sure that this is a good approach.
If someone has done something similar before, with stream of data, please share your best practice about it.
In case that someone needs to see a dataset you can find it on https://pastebin.com/UaXeEjq9 as a csv.

Try looking into the Robust Random Cut Forest Algorithm (RRCF). There's a python library implementation that supports streaming data where you create rolling windows (called shingles): https://github.com/kLabUM/rrcf
RRCF builds a binary decision tree by randomly picking a number between the min and max of your variable and splitting the data there. If a point is alone after the split it becomes a leaf in the tree. The more separate from the rest of the data a point is (ie the more of an outlier/anomaly it is), the more likely it is to become a leaf higher in the tree. This works well for higher-dimensional data since it will just split all variables randomly.
Here's a video that explains RRCF better than I could in a written post: https://youtu.be/yx1vf3uapX8?t=355


Checking if ML model is possible How can I check if a machine learning model is feasible on a given dataset? What techniques like EDA, correlation etc. can be used to judge if a model is possible i.e. data and predictor variables will give reasonably accurate forecasts or in other words there is good enough signal in predictors?

When in doubt simply build a model and test how good it is (accuracy, MSE, whatever).

If those KPIs are not up to par, think about improving the parameters, feature engineering, etc.

I find this approach to be more valuable than to spend a lot of time analyzing correlations and other classical statistical analysis.

If simple models (Regression, GLM, RandomForest, XGB) do not produce results with any sensible accuracy and you see no path to successful feature engineering then you have your answer as well.

The only downside to this approach might be a perfectly good model on the first try ;)


What is the difference between C and lambda in the context of an SVM? I don't understand the difference between the parameter $C$ and $\lambda$ in terms of the SVM. It seems to me that they are both involved in regulating over-fitting of the data.

What difference between $C$ and $\lambda$?

You are right to be confused. What's going on is that the hyperparameters refer to different formulations. On the one hand there is a generic empirical risk minimizer

$$\lambda\lVert \mathbf w \rVert^2 + \frac 1 n \sum_{i=1}^n \max\left(0, 1 - y_i(\mathbf x_i^T \mathbf w + w_0)\right)$$

On the other there is the soft-margin objective

$$\frac{1}{2} \lVert \mathbf w \rVert^2 + C \sum_{i=1}^n \xi_i$$

such that $$\xi_i \geq 0, y_i\left(\mathbf x_i^T \mathbf w + w_0\right) \geq 1-\xi_i, \forall i$$

The first one places SVM in the framework of empirical risk minimization, in which the objective is an expected loss plus the $L_p$-norm of the main parameter. The purpose of introducing slack variables and constraints in the second formulation is to allow a fraction of points (adjustable by $C$) to lie on the wrong side of the margin, and eliminate the non-differentiable $\max$ function. Moreover, it draws attention to the fact that SVMs encode sparsity in the loss function, rather than the prior. This is called the primal form and can be solved using quadratic programming. Wikipedia currently explains this in the Computing the classifier section. You should be able to see that $C=1/{2n\lambda}$. Some texts omit the $1/2$ or the $n$.


Why does my model sometimes not learn well from same data? I have a dataset of 2 classes, both containing 2K images. I have split that into 1500 images for training and 500 images for validation.

This is a simple structure for testing purposes, and each image is classified depending on the colour of a particular pixel. Either Green or Red.

I have run this model many times and I find that sometimes the models gets   low loss/ high accuracy within a few epochs, but other times it gets stuck at accuracy 50%.

The datasets are exactly the same each time with only difference coming from model.fit "shuffle" option.

I tested the LR Range first:

and I "cycle" the learning rate through an appropriate range.

model = keras.Sequential([
        keras.layers.Dense(112, activation=tf.nn.relu, input_shape=(224, 224, 3)),
        keras.layers.Dropout(0.4),
        keras.layers.Flatten(),
        keras.layers.Dense(3, activation=tf.nn.softmax)
    ])

LRS = CyclicLR(base_lr=0.000005, max_lr=0.0003, step_size=200.)

model.fit(train_images, train_labels, shuffle=True, epochs=10,
              callbacks=[checkpoint,
                         LRS],
              validation_data = (test_images, test_labels)
              )

Why does the model sometimes NOT get a good fit?

  EDIT 1

Re Serali's suggestion:

myOpt = keras.optimizers.Adam(lr=0.001,decay=0.01)
model.compile(optimizer=myOpt, loss='categorical_crossentropy',  metrics=['accuracy'])

reduce_lr = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.2,verbose=1,patience=5, min_lr=0.00001)

Shuffle = True should give better results particularly when you are running for more epochs. But I am not clear why there is so huge difference in the accuracy when shuffle changes . One thing you can try is , to increase the number of epochs and see if the accuracy improves. For shuffle = False

Shuffle set to false, allows you to use the previously trained data. Setting this to true means that you either want to retrain or set the epoch to some value greater than 10. To learn, but this increases the chances of memorization (over fitting)


World's Wine's Dataset I'm conducting a machine learning project and therefore looking for a world's wine's dataset to create a classification model based on user's ratings. Can someone recommend me any datasets? Haven't found such public datasets that contain enough data and features.

Kaggle has a "Wine Reviews" dataset with over 130k reviews. There are even several tutorials based on this dataset I believe. I would check those notebooks before starting any new project.

https://www.kaggle.com/zynicide/wine-reviews


What is the current best state of the art algorithm for graph embedding of directed weighted graphs for binary classification? Sorry if this is a newbee question,  I'm not an expert in data science, so this is the problem:

We have a directed and weighted graph, which higher or lower weight values does not imply the importance of the edge (so preferably the embedding algorithm shouldn't consider higher weights as more important), they are just used to imply the timing of the events which connect the nodes, so the higher  weighted edges are events that have happened after the lower ones.

There can be multiple edges between nodes, and I want to do a binary classification using deep learning, meaning that my model gets a embedded graph vector as input and decides if its malicious(1) or not(0).

So what is the best state of the art graph embedding algorithm for this task that can capture as much information as possible from the graph? i read some graph embedding papers but couldn't find any good comparison of them since there are so many new ones.

IMPORTANT NOTE: One problem i have seen with some of the graph embedding algorithms, is that they try to have a small vector dimension, since i guess they are used in fields which there are a LOT of nodes so they need to do this, but in this task its not really important, the nodes are the functions in the program, and they very rarely go above 2000 functions, so even if the algorithm creates a 20k dimensions its no problem, I'm saying this because some of the algorithms that I'm reading will produce a vector that even has lower dimensions compared to number of the nodes in the graph! and that causes loss of information in my opinion. so to sum up, the performance and large vector size is not a problem in my task. so preferably the algorithm should gather as much  information  as possible from the graph.

I'm no expert myself, but recently (i.e. this is true to 2019), I've heard (at a meetup from an expert) that Node2Vec is the SOTA.

Here's a link to a post on Medium explaining it - basically, Node2Vec generates random walks on the graph (with hyper-parameters relating to walk length, etc), and embeds nodes in walks the same way that Word2Vec embeds words in a sentence.

Note that since random walks are generated on the graph, intuitively you don't have to use edge weights, and can use multiple edges.

P.S. There's an older (2014) algorithm for this, called "DeepWalk", which I don't know much about but is supposed to be similar, simpler, and not as performative. I'm just adding this in case you want to search the term.


Found input variables with inconsistent numbers of samples: [11232, 5616] I don't know what is the reason for the error please guide me and help me out. I am at a learning stage.

It seems like the train and train_w2v dataframes have s variable number of rows. You will  have to make their shapes consistent before doing a train-test-split


Scikit-learn SelectKBest is picking up obviously unwanted Features Dataset
Dataset Summary: Bank Loan (classification) problem
Problem Summary:

I am exploring ways to simplify EDA Process (Exploratory Data Analysis) of finding the best fit variables
I came across SelectKBest from Scikit Package
The implementation went fine except some variables it returned me are obviously not going be a good factor (like primary keys in the dataset)
Is there a problem in the implementation? or is the package supposed to behave in that manner?

import numpy
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
from sklearn.preprocessing import LabelEncoder

# My internal code to read the data file
from src.api.data import LoadDataStore

# Preping Data
raw = LoadDataStore.get_raw()
x_raw = raw.drop(["default_ind", "issue_d"], axis=1)
y_raw = raw[["default_ind"]].values.ravel()

# NA and Encoding
for num_var in x_raw.select_dtypes(include=[numpy.float64]).columns.values:
    x_raw[num_var] = x_raw[num_var].fillna(-1)
encoder = LabelEncoder()
for cat_var in x_raw.select_dtypes(include=[numpy.object]).columns.values:
    x_raw[cat_var] = x_raw[cat_var].fillna("NA")
    x_raw[cat_var] = encoder.fit_transform(x_raw[cat_var])

# Main Part of this problem
test = SelectKBest(score_func=f_classif, k=15)
fit = test.fit(x_raw, y_raw)
ok_var = []
not_var = []
for flag, var in zip(fit.get_support(), x_raw.columns.values):
    if flag:
        ok_var.append(var)
    else:
        not_var.append(var)

ok_var
['id', 'member_id', 'int_rate', 'grade', 'sub_grade', 'desc', 'title', 'initial_list_status', 'out_prncp', 'out_prncp_inv', 'total_rec_late_fee', 'recoveries', 'collection_recovery_fee', 'last_pymnt_d', 'next_pymnt_d']

not_var
['loan_amnt', 'funded_amnt', 'funded_amnt_inv', 'term', 'installment', 'emp_title', 'emp_length', 'home_ownership', 'annual_inc', 'verification_status', 'pymnt_plan', 'purpose', 'zip_code', 'addr_state', 'dti', 'delinq_2yrs', 'earliest_cr_line', 'inq_last_6mths', 'mths_since_last_delinq', 'mths_since_last_record', 'open_acc', 'pub_rec', 'revol_bal', 'revol_util', 'total_acc', 'total_pymnt', 'total_pymnt_inv', 'total_rec_prncp', 'total_rec_int', 'last_pymnt_amnt', 'last_credit_pull_d', 'collections_12_mths_ex_med', 'mths_since_last_major_derog', 'policy_code', 'application_type', 'annual_inc_joint', 'dti_joint', 'verification_status_joint', 'acc_now_delinq', 'tot_coll_amt', 'tot_cur_bal', 'open_acc_6m', 'open_il_6m', 'open_il_12m', 'open_il_24m', 'mths_since_rcnt_il', 'total_bal_il', 'il_util', 'open_rv_12m', 'open_rv_24m', 'max_bal_bc', 'all_util', 'total_rev_hi_lim', 'inq_fi', 'total_cu_tl', 'inq_last_12m']

Its clear id, member_id should NOT belong to the best features list! any idea what I am doing wrong?

Edit: Did more digging and the reply by @Icrmorin is right. (its a kaggle dataset, so will not know why) but here is the box plot for id

There seems to be two possible approaches to your problem :

If they are just identification features that you know aren't informative, you should remove them yourself. SelectKBest - like almost any other EDA tools - works on all the features you provide it, there is no way it knows what features are supposedly uninformative identification features and which are not.

It is possible that somehow the identification feature is informative. I can think of at least two reasons : correlation with time as the instance are entered in order and what you want to observe change with time. Or, if your identification feature is not unique (instances observed trough multiple different times), correlation between your observation. Depending on how your identification feature is built and what you want to achieve, you might want to keep this information or not.


neural network training algorithms When I first read about neural networks, I learned that Backpropagation is the algorithm used to train the neural network. I am interested if there are other alternatives (or better?) to BP.

What are the other training algorithms used in NN? And is BP is the best one, and that's why almost everyone uses it for training the NN model?

Yes, some alternatives are feedback-alignment (FA), Direct Feedback Alignment (DFA) and Indirect Feedback Alignment (IFA).


Unable to print in Jupyter Notebook using Pandas I am doing basic data analysis on an csv file in jupyter notebook

def answer_two():
    return (df['Gold']-df['Gold.1']).argmax()
answer_two

The above code snippet is to subtract two columns of the dataframe.I am expecting an answer in the form of a country name but I am getting the following output.

<function __main__.answer_two>

I am unable to figure out why this is happening .Occasionally ,the required output in the form of country name is coming but not running it everytime.

Instead of calling answer_two, call answer_two(). you are referring to the function object now. You have to call the function.


Pandas datastructure I'm trying to analyze database performance over a period of time and detect anomalies. The database server consists of many threads that perform different actions. I run a query to determine the number of active threads and the action they are performing.

A sample dataset is below:

My Objective:
I need to analyze over a period of time and determine what is normal at a given timestamp and detect any abnormalities. For example, Monday at 10 am, there are 10 active threads; out of which, there are 2 threads with the action 'Preparestatement' and 10 threads with the action 'Readtable'. Any other thread actions is potentially an anomaly.

As you can see from the image above, the actions (executestatement, Fetchcursor and so on) could be different at each timestamps. I want to understand if the structure of the pandas dataframe is the right choice to meet my objective.

Not sure if I understood your question correctly, but to dodata analysis, you would need to have a proper DF.

You have something like:

time A B C
1    3 4 6
time B C D
2    9 4 6

You need one table header, so remove any other header text in the rows > 1.

Also (of course) the content of each row must match the column it belongs to.

So the DF above would change to:

time A B C D
1    3 4 6 0
2    9 4 0 6

Note the "zero" entries in time=1 -> D=0 and time=2 -> C=0. Here the "zero" is chosen to fill the gaps (zero activity in this case).

This type of data representation is what you usually use for any kind of modelling.


Next best predictions in decision tree I am using decision tree classifier to predict some block selected based on the below data.

I am able to predict the "Block selected" column based on the data. How to get the second best, third best prediction and so on(I need a ordered list)? Can I get this using decision tree? Or should I be using a different model?
Any thoughts on how to do this using python scikit-learn?

What you are looking for is a probabilistic classification for 2+ classes (multi class) i.e. evaluating a probability of being associated to a list of k classes given the observation. k = 4 blocks/classes for your example. Its a one vs many situation.

Binary classifiers (like logistic regression) have been extended to multi class problems.

One of the strategy used is to train/test your model k times, each time keeping one class in predicted block intact and turning other three predicted class to a single dummy class. Then use a binary probabilistic classifier(eg. logistic regression) to predict class probabilities. The order of class preference will be in decreasing order of class probabilities.
For logistic regression and for other classifiers, this is inherently built in by scikit and applied by default:

lr = linear_model.LogisticRegression(multi_class = 'ovr')
# ovr is "one vs rest"
lr.predict_proba(test_features)

refer to Multi-class capability built
 in scikit

Yes, you can even use a pruned decision tree to get the class probabilities. But most probably you will not be able to get 2nd, 3rd... best predictions for most of your observations from a single tree due to the underlying splitting mechanism of algorithm.

 dt = tree.DecisionTreeClassifier(min_samples_split=25)
 dt.predict_proba(test_features)

Therefore, rather then relying on a single decision three, better option would be to use a randomforest classifier to obtain proportion of votes going to each class while making a prediction for an observation.

In short, any multi-class classifier (or ensemble) which spits out a likelihood over multiple classes will do. Another eg. XGBoost as pointed out in below comment by bradS.


What's the right way to setup an image classifier by multiple params? I'm very new to the data science and machine learning, so apologies for my ignorance.
What I'm trying to understand is how to setup an image classifier system (maybe based on CNN) which will classify my image by multiple params. Most of the examples I found are about classifying by single class, i.e. "cat", "dog", "horse", etc, but what I'd like to have is, for example, {"red", "dog", "tongue"}. Is there a simple way to do it? The best option would be to have a ready setup, so I can just change their test dataset with mine and see the right formatting. Thanks!

UPDATE:

Also please help me understand if it's a complicated task for an experienced machine learning engineer? What'd be the timing and cost given a dataset?

You normally one hot encode your labels so that every possible attribute gets it's own binary representation. So if you have 10 attributes they would be represented as [attr1, attr2, attr3, ..., attr10], with values either 1 if the attribute is present, or 0 if it is not.

Then you train a network with the same number of output neurons as possible categories and use a sigmoid activation function.


Role and need of Mathematics in Data Science I am Bachelors of Engineering in Software Engineering field and having 3 years professional work experience in Oracle ERP/Database related technologies. Now I'm planning to pursue MSc in Data Science. What I need to know that what sort of Mathematics skills should a Data Scientist have? As we had learned numerous subjects like Numerical Analysis, Statistics, Linear Algebra etc.

Firstly, I think this question is both broad and opinion-based, however most people don't think so. I'll explain why:

Data Science is a domain which various very much with the industry it is applied in, so very much application-based.

For example: Someone like me in the growth and customer support segment, I use markov models and reinforcement learning, as customers moves can most likely be modeled as a (stochastic) graph. So, when I look out for hiring someone into my team, I look for graph theory skills, linear algebra (obviously) and statistics.

However, for someone working in the segment where NLP is used, the math needed is different, and so does with every other application of data science.

However, some really very important courses which help someone in the path of a data scientist according to me are:

Linear Algebra
Statistics
Probability theory
Discrete Math
Analytical methodology (Numerical Analysis would be a close to precise name for that)

Some would argue that Information Theory is also very important, however I haven't really used it heavily in my day job, so I don't think it's absolutely necessary.

Helpful Post


How to count occurrences of values within specific range by row I have a data frame of 3000 rows x 101 columns like as follow:

Time   id0  id1  id2     ………… id99

1      1.71 6.99 4.01    ………… 4.98

2      1.72 6.78 3.15    ………… 4.97

.

.

3000   0.36 0.23 0.14    ………… 0.28

Using Python, how could we add a column that counts for each row the number of values (in column id0, to id99) that are within a specific range?

You can apply a function to each row of the DataFrame with apply method. In the applied function, you can first transform the row into a boolean array using between method or with standard relational operators, and then count the True values of the boolean array with sum method.

import pandas as pd

df = pd.DataFrame({
    'id0': [1.71, 1.72, 1.72, 1.23, 1.71],
    'id1': [6.99, 6.78, 6.01, 8.78, 6.43],
    'id2': [3.11, 3.11, 4.99, 0.11, 2.88]})

def count_values_in_range(series, range_min, range_max):

    # "between" returns a boolean Series equivalent to left <= series <= right.
    # NA values will be treated as False.
    return series.between(left=range_min, right=range_max).sum()

    # Alternative approach:
    # return ((range_min <= series) & (series <= range_max)).sum()

range_min, range_max = 1.72, 6.43

df["n_values_in_range"] = df.apply(
    func=lambda row: count_values_in_range(row, range_min, range_max), axis=1)

print(df)

Resulting DataFrame:

    id0   id1   id2  n_values_in_range
0  1.71  6.99  3.11                  1
1  1.72  6.78  3.11                  2
2  1.72  6.01  4.99                  3
3  1.23  8.78  0.11                  0
4  1.71  6.43  2.88                  2


what machine/deep learning/ nlp techniques are used to classify a given words as name, mobile number, address, email, state, county, city etc I am trying to generate an intelligent model which can scan a set of words or strings and classify them as names, mobile numbers, addresses, cities, states, countries and other entities using machine learning or deep learning.

I had searched for approaches, but unfortunately I didn't find any approach to take.  I had tried with bag of words model and glove word embedding to predict whether a string is name or city etc..

But, I didn't succeed with bag of words model and with GloVe there are a lot of names which are not covered in the embedding example :- lauren is present in Glove and laurena isn't

I did find this post here, which had a reasonable answer but I couldn't the approached used to solve that problem apart from the fact that NLP and SVM were used to solve it.

Any suggestions are appreciated

Thanks and Regards,
Sai Charan Adurthi.

You could apply character grams - Intuitively, there might be a huge difference in character set between a phone number and an email address. and then pass the character gram vector to SVM to make a prediction. You could implement this using in sklearn using the below feature extractors.

TfIdfVectorizer(analyzer='character')
CountVectorizer(analyzer='character')

Cross validate on the ngram range and slack variables of SVM to fine tune your model.


Python Time series: extracting features on a rolling window basis I have a long univariate time series, and before performing some machine learning models with it, I want to extract as many features as I can from the time series on a rolling-window basis.

As a quick example, for a window of size 10, I would like to calculate statistics like mean and std deviation for the first t=0:9 points in my dataset, and have those two results occupy one row in a some new feature table, and the next row in the table will have mean and std deviation calculated on points t=1:10, and so on and so forth, until the end of the data.

Is there an efficient way to do this in Python?

Yes, there are easy ways to do this in Python. My favourite would be to put the data into a Pandas DataFrame, which has a convenient method called rolling that will cycle over your data in a given frame-size and compute whatever you like on that block.

Let me show you an example - say we start with the following column of data:

In [1]: import pandas as pd
In [2]: import numpy as np
In [3]: df = pd.DataFrame({"A": np.random.randint(0, 100, (20,)),
                           "B": np.random.randn(20)})

Look at the first 10 rows:

In [4]: df.head(10)
Out[4]:
     A         B
0   63 -0.003947
1   55  0.442597
2    6  0.684125
3   17  0.968987
4   33 -0.018640
5   50 -0.579558
6   71  0.563125
7   31  1.417384
8    8  0.607813
9   36  0.186146

We can compute the rolling average over each column and save it back to the dataframe like this:

In [6]: df[["rolling_a", "rolling_b"]] = df.rolling(5).mean()
In [7]: df.head(10)
In [9]: df
Out[9]:
     A         B  rolling_a  rolling_b
0   63 -0.003947        NaN        NaN
1   55  0.442597        NaN        NaN
2    6  0.684125        NaN        NaN
3   17  0.968987        NaN        NaN
4   33 -0.018640       34.8   0.414624
5   50 -0.579558       32.2   0.299502
6   71  0.563125       35.4   0.323608
7   31  1.417384       40.4   0.470260
8    8  0.607813       38.6   0.398025
9   36  0.186146       39.2   0.438982

You might notice that the first 4 rows contain NaN values (Not a Number). This is because the rolling() method will let the mean() method work an a window-size smaller than 5 (in our example). There are a lot of options in the rolling() method that you can experiment with.

You can do the same above for single column of a large dataframe like this:

>>> df["rolling_some_column_name"] = df.some_column_name.rolling(5).mean()

You can also apply just about any function to the rolling frame - not just mean().


Feature construction widget on Orange 3.13 I got stucked with this widget... I am an absolute beginner and I am now working with Orange software for my thesis with logs and core data.
I need to combine different features to compare them.

What kind of information should I put in "Values" field with a categorical feature?

If you have some example with will be really appreciated. the ones from orange doesn't help me

As per Feature Constructor widget documentation, the expression field should return values from $[0, n]$, and the Values field should contain $n+1$ comma-separated labels.


Batching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step? I have scoured the internet and books, but everything seems to use num_steps and batch_size or similar terms interchangeably and I can not get a grasp on their specific use, or where batches fit in when talking about RNNs ( I understand their use with Gradient Descent, and maybe I am misunderstanding RNNs and I need to be set straight at the fundamental level- if that is the case, please do so.

So where does batching apply in Recurrent Neural Networks? At the level of instances within time steps or at the level of timesteps themselves.

By my current understanding the data fed into the model is structured like this:

data = [
    "time_step_1": [
        "instance_1": [
            "feature_1": "some_value1",
            "feature_2": "some_value2",
            "feature_3": "some_value3",
            "feature_4": "some_value4"
        ],
    "time_step_2": [
        "instance_2_but_first_in_time_step_2": [
            "feature_1": "some_value1",
            "feature_2": "some_value2",
            "feature_3": "some_value3",
            "feature_4": "some_value4"
        ],
    "time_step_3": [
        "instance_3_but_first_in_time_step_3": [
            "feature_1": "some_value1",
            "feature_2": "some_value2",
            "feature_3": "some_value3",
            "feature_4": "some_value4"
        ],
    "time_step_4": [
        "instance_4_but_first_in_time_step_4": [
            "feature_1": "some_value1",
            "feature_2": "some_value2",
            "feature_3": "some_value3",
            "feature_4": "some_value4"
        ],
    "time_step_5": [
        "instance_5_but_first_in_time_step_5": [
            "feature_1": "some_value1",
            "feature_2": "some_value2",
            "feature_3": "some_value3",
            "feature_4": "some_value4"
        ],

    ]
]

Except: That many people (and examples) have more instances per time step.

A typical training loop looks like:

for epoch in range(do_num_epochs):
    for time_step_count in range(len(data)):
        run(model, data[time_step_count])

And that is if you had (for example) 10,000 instances at each time step, then batching would allow the weights to be updated more often than having to wait for the entire 10,000 instances to be processed. Then the loop would look like:

batch_size = len(data) // do_num_batches

for epoch in range(do_num_epochs):
    for time_step_count in range(len(data)):
        for batch_count in range(do_num_batches):
            batch_data = data[time_step_count][batch_count * batch_size : (batch_count + 1) * batch_size]
            run(model, batch_data)

But there is no point in processing the data in batches if there is only a single instance of data per time step is there?

I made a chart to try to explain ( to myself as well ) what the flow and data structure would look like:

I am just trying to understand the role of the batch in all of this, how it fits in?

(Edit/Addition)

After some more research on the subject, I am fairly certain that batches are a way to feed in sequences in chunks, and control the frequency of weight updates. By utilizing batches, the weights are updated at batch level, thus letting you group time-series together in groups. I illustrated what I believe to be right below, in the below example there are:

4 instances,
each with 5 features
across 30 time steps

In the first example (mini-batch), there are 3 batches, of batch_size= 10 in that example, the weights would be updated 3 times, once after the conclusion of each batch.

In the second example, is online learning with an effective batch_size=1 and in that example, the weights would be updated 30 times, once after each time_series

In the final example (Batch learning), the entire data set is processed at once, the weights are updated only once (1) at the conclusion of the 'batch'.

In each of these example, the entire data set would be run through many multiple of times, each one of these cycles is an epoch.

Your edit is not right. from the keras documentation you can actually understand the difference between timesteps and batches. I take your examples:

For the first example.
You have 4 instances, or samples, or sequences. The length of each sequence is 30, now
you actually take 3 batches, each of these batch is composed by 4 instances, or samples, or subsequences, of length 10(timesteps). your batch size is not 10 but 4, the timesteps don't affect the batch size. the subsequences of each batch can have the length that you want, this doesn't change the fact that the batch size is 4.

For the second example is the same.
Online learning doesn't mean that the number of timesteps of each subsequence is 1, you can take the number of timesteps you want, but when you update the parameters of the network you should consider only 1 subsequence at the time.

You can actually confirm this from here:

http://philipperemy.github.io/keras-stateful-lstm/

In this part:

"Said differently, whenever you train or test your LSTM, you first have to build your input matrix X of shape nb_samples, timesteps, input_dim where your batch size divides nb_samples. For instance, if nb_samples=1024 and batch_size=64, it means that your model will receive blocks of 64 samples, compute each output (whatever the number of timesteps is for every sample), average the gradients and propagate it to update the parameters vector."

So the timesteps don't affect the batch size.


Linear Regression on data with bimodal outcome I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am building linear regression models that forecast the time, but none of the models are able to make predictions; the $R^2$ values of all of the models are $0$. I plotted the residuals of the models and verified that they are normally distributed

I have used LassoCV/Lasso, ElasticNetCV/ElasticNet, XGBoost, and Kernel Ridge Regression in Scikit-Learn to make the models. I used the weights of the Lasso coefficients to filter my data frame so that I use the features that have the most magnitude in my predictions, but that did not work. I used the feature_importance_ method for XGBoost to do something similar, but that also did not help.

Kernel Ridge Regression ran with the parameters

KernelRidge(alpha=1, coef0=1, degree=3, gamma=0.001, kernel='polynomial',
      kernel_params=None)

but overfit the training data completely with $R^2 = 1$.

I have been looking into non-parametric methods like Kernel Density Estimate and Gaussian Mixture Models, but am uncertain in how to use them for my regression task. I have also been looking into Modal Regression and fitting of multi-functions, but am not familiar in how to implement them with Python.

I have also considered finding out what features are contributing to the modes and splitting the data set as two distributions and running the models on the two separate data sets, but am not sure on how to paint the picture for the entire data set.

In general, how should I approach building a regression model with a multimodal dependent variable?

One option would be use sklearn.compose.TransformedTargetRegressor  to make the dependent variable more normal distributed.


Exploratory Data Analysis on dataset divided by winners and losers I have a dataset where I have features from winning tennis players and the other half are from a losing tennis players:
winner_age, winner_rank / loser_age, loser_rank

In order to conduct a proper EDA, do I have to concatenate both losers and winner to further analysis or split the data frame in two?

So, the question asks that during Exploratory Data Analysis whether it is better to analyse the data with both winning and losing tennis players in one data frame or split them into two. In this question, I am going to assume that you will be using this data in a classification task (do correct me if I am wrong!): to discriminate between winning and losing tennis players.
I believe the best approach would be to analyse with both winning and losing players together. The main reasons for it are:

When you are running this data through a model, you won't split the data by class (winning vs losing players)
More importantly, in the context of feature engineering, the usual rule of thumb normally is the you choose features which can greatly discriminate between the classes. So you can only do that by reviewing the feature distributions by classes to see if adding those features to the model will improve generalisation performance of the model.


Should I put time on my Vanilla ANN for classifying MNIST Dataset I am building a Vanilla Neural Network in Python for my Final Year project, just using Numpy and Matplotlib, to classify the MNIST dataset. Here's the specifications of the model:

One Input Layer + One Hidden Layers + One Softmax Layer
Number Nodes in each layer :[784, 800, 10]
Activation function used: ReLU and Softmax.
Also Normalized the Train and Test set, by dividing it by 255
Have used Mini Batch Gradient Descent(Mini Batch Size=4096).
The Model shows very low accuracy on the Test set. Around 8% -11% accuracy.
And it shows an accuracy of around 66% - 72% for the Train set.

I haven't used Regularization until now. And I don't know if that will help.

Now fed up with this, I am thinking of just implementing the model in Tensorflow and see if that works. Or is it necessary to implement it from scratch? Because I think I know how every concept works(although not being able to find the loophole in my model). What do you have to say about it?

If you want to have a look at my code here's the link. It would be great if I can get any suggestions.

P.S.: Does it create any problem if a DL model is not written using OOP?

Some quick notes:

A mini-batch size of 4096 is rather large, try training with sizes like 16, 32, or even 1 to see what happens.

Your test accuracy seems to be around random since there are 9/10 classes to predict. Does your model predict randomly (i.e. every class gets a similar score) or is it just wrong often?

is it necessary to implement it from scratch?

Necessary for who? Unless it's required for your coursework, nobody is forcing you to implement this from scratch.

P.S.: Does it create any problem if a DL model is not written using OOP?

If an implementation is correct, it's correct. Whether it's spaghetti code, whether it's a single file, whether it's beautiful, tested, and reviewed OOP software.

OOP is there to help human write more maintainable, robust, and readable software, but under the hood, the computer really couldn't care less.


How to predict custom value after using linear regression? I'm new to machine learning, and I'm currently practicing by playing around with datasets that I find on Kaggle. Currently I'm trying to predict the price of an Audi, based on the model, mileage and manufacturing year, using a slighly modified version of this set (only columns I use are model, mileage, price and year).
I have the following code written down which makes use of linear regression.
import pandas as pd
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt

df = pd.read_csv('audi.csv', sep=";")
cars=pd.get_dummies(df['model'],prefix='car')

new_df=pd.concat([df,cars],axis='columns')
new_df=new_df.drop(['model'],axis='columns')

x_data=new_df.drop(['price'],axis='columns')
y_data=new_df['price']

model=LinearRegression()
model.fit(x_data,y_data)

This gives me a meagre model.score of 0.8290666609212749.
To test for a custom car that I find on one of the many used car websites, I do the following:
#Audi A3, 42100 mileage, 2016 built
model.predict([[42100,2016,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]])

This works, but as you can see it's a huge hassle to work with because of all the dummy variables. Is there a way of making this simpler?

As an alternative to pd.dummies() you could use sklearn.preprocessing.OneHotEncoder().The advantage here would be that after creating an encoder object you can reuse it to transform data points you have to the one hot form. You could use OneHotEncoder(categories) to manually assign categories for the encoder object or you could use OneHotEncoder.fit() to extract categories from a particular feature. You can then use OneHotEncoder.transform(feature) to transform it to one hot form. pd.dummies() does not provide the re usability part, therefore, sklearn.OneHotEncoder() makes things simpler.


How to handle image classification network where output classes can be subsets of one another? For instance, if I wanted a train network that can output

van
truck
sedan
vehicle
pedestrian

does it make sense to only train it on van, truck, sedan, and pedestrian and then make "vehicle" a synonym of van, truck, sedan? Or does it make sense to train vehicle as its own class made up of the combined training data of van, truck, sedan?

EDIT ---

Here's a more clear example I think, what if there are labels that are not exclusive? For instance, a child can also be a pedestrian. How do I train a network to output both labels if it, for example, sees a child walking down the street? It seems that using a softmax later as the final layer of the network would not allow this to work as all probabilities need to sum to one, in this case it could be 90% child and 80% pedestrian and that's entirely valid.

It depends what you're trying to achieve.

In general, there are things that are vehicles but aren't vans, trucks, or sedans, so your two proposed schemes are not equivalent even if you were somehow able to obtain a 100% accurate classifier.

In general, the way to answer questions like this is to try it both ways and see which works better.


Creating a Object Detection model from scratch using Keras I have a dataset containing 330 images which contain guns. Along with the images, I have a text file associated with each image file which contains,

The number of objects ( guns ) in the image.
Coordinates for bounding boxes around the gun in the image.

I need to train a model which takes an image as an input and outputs 4 integer values which are the coordinates for the bounding box ( vertices of the bounding box ).

For training an object detection model, should the image be kept as an input and the coordinates as the output of the model? Should there be Convolution layers for feature extraction and then FC layers for learning the features for producing 4 outputs ( coordinates of the bounding box )?
Is this notion of the model architecture correct? Any other tips/suggestions?

I am creating this model entirely in TensorFlow Keras without using any of the pretrained stuff.

Before I answer your question, let me tell you this, You can go on and train a model from scratch, but you will definitely end up using one of the object detection architectures, be it Mask R-CNN, Faster R-CNN, Yolo or SSD. Your problem is a simplified version of what these architectures are trying to solve. These are generic object detectors that some of which supports more 1k classes. You have a single class detection problem.

Now back to your question.

  For training an object detection model, should the image be kept as an input and the coordinates as the output of the model? Should there be Convolution layers for feature extraction and then FC layers for learning the features for producing 4 outputs ( coordinates of the bounding box )?
  No, it is not that simple. Training a FCN to output 4 values as Bounding boxes wont work.

All object detectors mentioned earlier are based on assumptions, Faster R-CNN for example generates proposals (regions) its assuption is that these regions are very likely to contain an object, then does an additional step by classifying which class each region contains (you only have one class) and finally refining the output. The most important thing in Faster R-CNN is region proposal network, which iterates through the final convolution layer in a sliding window fashion, generating proposals in the different aspect ratios, for example 1:1, 1:2 and 2:1.

Why am I saying all this? Because I want you to understand, that the problem is not as easy as you think it is.

Further Reading:

https://medium.com/@tanaykarmarkar/region-proposal-network-rpn-backbone-of-faster-r-cnn-4a744a38d7f9
https://medium.com/diaryofawannapreneur/yolo-you-only-look-once-for-object-detection-explained-6f80ea7aaa1e


Is there enough datapoints to make a reasonable predcitction? I am planning to use this dataset (for a school project) in order to determine the most important features in predicting whether a student will receive a placement.
Further I would like to create a model which would predict if a student will receive a placement based on their own features.
Link: https://www.kaggle.com/benroshan/factors-affecting-campus-placement
However, I have noticed that this dataset only has 215 rows. So my question is:

Is this enough data points for the tasks I plan to conduct?
If so, what classification algo would be most appropriate for a dataset which is this small?

Thank you for any help in advance!

Having enough data points is an empirical question. Split the data into train and test. See if the model trained on the training data can predict the test data.
For small data sets, pick algorithms that are less likely to overfit. One example is a decision stump.


High error machine learning regressor algorithm in Python - XGBOOST Regressor I have a dataframe with real state data from florida, it includes single apartments and buildings data:

'TRUE_SITE_CITY': The city where the building is. variable: Miami, Aventura...;
'CONDO_FLAG': If it is a condominium or not, variable: yes/no;
'BEDROOM_COUNT': Number of total bethrooms, variable: integuer,
'BUILDING_actual_AREA': The area of the entire building, or apartment in the case that there are only one apartment or house. variable: integuer;
'FLOOR_COUNT': Number of the floors that the building has;
'DOR_CODE_CUR': the type of the building. Variable: categorical;
'UNIT_COUNT': Number of apartments or houses in the building. Variable: integuer;
'YEAR_BUILT': Year that the building or house or apartment was build: Variable: categorical;
'public_transport_min_distance': I have calculated the nearest stations of the public transport;
'Price': The variable that I want to predict.Variable: integer.

I have done an exploratory data analysis and I have dropped some data that has null values and some data that was incorrect. Also I have dropped the values with outliers.

The basic statistics of the price column (targeted column):

I have checked the categorical features and they have enough variables in each one to keep they in the model.

I have done a pipeline to make a one hot encoder for the categorical values and a standard standardisation for the numerical values. In it I have include a XGBOOST regression:

from xgboost import XGBRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import cross_validate
from sklearn import metrics
from sklearn import preprocessing, feature_extraction
from sklearn.pipeline import Pipeline
from sklearn import preprocessing, feature_extraction
from sklearn.pipeline import make_pipeline, make_union
from mlxtend.feature_selection import ColumnSelector
from sklearn.preprocessing import StandardScaler
from category_encoders import OneHotEncoder

x_numeric = df_x[['BEDROOM_COUNT','BATHROOM_COUNT',
       'HALF_BATHROOM_COUNT', 'FLOOR_COUNT','UNIT_COUNT','public_transport_min_distance','BUILDING_actual_AREA']]

x_categorical = df_x[['TRUE_SITE_CITY','CONDO_FLAG','YEAR_BUILT']]

categorical_col = x_categorical.columns

numeric_col = x_numeric.columns

estimator_pipeline = Pipeline([
    ('procesador', procesing_pipeline),
    ('estimador', estimator)
])

score2 = cross_validate(estimator_pipeline, X= df_x, y= df_y, scoring=scoring,return_train_score=False, cv=5,n_jobs=2)

But I am obtaining a high error. The mean value of the price is almost 200.000 and the error that I obtain is:

I have done feature selection using RFE but I obtain a high error as well.

Also I have run it doing RandomizedSearchCV

from sklearn.model_selection import RandomizedSearchCV

params = {"estimator__learning_rate"    : [0.05, 0.10, 0.15, 0.20, 0.25, 0.30 ] ,
 "estimator__max_depth"        : [ 3, 4, 5, 6, 8, 10, 12, 15],
 "estimator__min_child_weight" : [ 1, 3, 5, 7 ],
 "estimator__gamma"            : [ 0.0, 0.1, 0.2 , 0.3, 0.4 ],
 "estimator__colsample_bytree" : [ 0.3, 0.4, 0.5 , 0.7 ] }

random_search = RandomizedSearchCV(
    estimator=estimator_pipeline,
    param_distributions=params, cv=5, refit=True,
    scoring="neg_mean_squared_error", n_jobs= 3,
    return_train_score=True,
    n_iter=50)

But I obtain a similar error value.

What could I do?

It is not necessary to one-hot-encode your categorical features when using tree-based methods. Basically the idea is that the tree has to make many splits to figure out the category. Instead, you can use ordinal encoder (even if categories are not ordered).
I also would not have set 'YEAR_BUILT' as a categorical variable, even though it is discrete. By one-hot encoding this variable, you end up with a very splitted dataset where the tree has to consider every YEAR_BUILT as a subset and it probably has too few observations to do so. By considering it as continuous it enables the algorithm to determine 2 or 3 pivot years that are relevant (changed currency, economic crisis, ...).

I think the second point can really improve performance.


How to apply .replace() inside a column in a pandas data frame to clean data I have a pandas data frame where I want to replace all the values that are not numeric in one column for ""

The error I'm getting in my code is the following:

ValueError: could not convert string to float: '690,276.00'

Since I'm trying to convert all values into float so I can do internal operations with them.

Part of my cleaning data frame code looks like this:

    # Cleaning:
df_clean = df_read[~(df_read['Ratio of Similarity (Gray)'] <= .2)]
print(df_clean, 'clean 1: Eliminate Ratio of similarity less than 0.2')
df_clean_2 = df_clean.dropna(subset=['buybox_price'])
print(df_clean_2, 'clean 2: Eliminate Nan Buybox Prices')
df_clean_2 = df_clean_2.replace(",", "").replace('', '').astype({'product_ranking':'float64'})
df_clean_3 = df_clean_2[~(df_clean_2['product_ranking'] >= 5000000)]
print(df_clean_3, 'clean 3: Eliminate Product Ranking + than 5.000.000')
df_clean_4 = df_clean_3[~(df_clean_3['buybox_price'] <= 6)]
print(df_clean_4, 'clean 4: Eliminate Buybox Price less than 6$')
# Save Cleaned File
path_file = os.path.join(BASE_DIR, 'csv/amazon_product_comparator.csv')
df_hc = df_clean_4.to_csv(path_file)

The error can be found in the line:

df_clean_3 = df_clean_2[~(df_clean_2['product_ranking'] >= 5000000)]
    print(df_clean_3, 'clean 3: Eliminate Product Ranking + than 5.000.000')

EDITED to include a working example.

I have made an example with string type numeric data. I have also include a poorly formatted string number 1,002,*8320.

The code below will convert the numbers in the format of digits and digits with commas to floats, and convert all other strings to NaN. I have used a regular expression to replace commas with blanks.

The NaN must be used over "" since the comparison operator will not work on strings

import pandas as pd
import numpy as np
import re

# example data frame
df2 = pd.DataFrame([['1', '2000000000'],
                    ['2', '1,002,*8320'],
                    ['3', '1,000,000']],
                   columns = ['idx','product_ranking'])
# Remove Commas
df2['product_ranking'] = df2['product_ranking'].map(lambda x: re.sub('[,]*' , '', x))

# Remove Convert strings that are numeric into floats.
df2['product_ranking'] = df2['product_ranking'].map(lambda x: float(x) if x.isnumeric() else np.nan)

#Comparison is working
df2['product_ranking'] > 100000000


What is the difference between bootstrapping and sampling in reinforcement learning? I have come across a David Silver's slide which contains both the terms "bootstrapping" and "sampling". Is there any realistic example which helps me to understand the concepts better.

I will try to answer this question conceptually and not technically so you get a grasp of the mechanisms in RL.

Bootstrapping: When you estimate something based on another estimation. In the case of Q-learning for example this is what is happening when you modify your current reward estimation $r_t$ by adding the correction term $\max_a' Q(s',a')$ which is the maximum of the action value over all actions of the next state. Essentially you are estimating your current action value Q by using an estimation of the future Q. Neil has answered that in detail here.
Sampling: Imagine samples as realizations (different values) of a function. Many times it is really difficult to estimate, or come up with an analytical expression, of the underlying process that generated your observations. However sampling values can help you determine lots of characteristics of the underlying generative mechanism and even make assumptions of its properties. Sampling can come in many forms in RL. For example, Q learning is a sampled-based point estimation of the optimal action value function (Bellman equation). In a world that your agent knows nothing you cannot use Dynamic Programming to determine the expected reward form every state. Thus, you need to sample "experience" from your world and estimate the expected reward from any state. In Policy Gradient methods in order to determine the gradient of your expected reward you need to sample trajectories over states and actions from a probability distribution as you cannot determine it analytically.

Hope this helps!


How to Interpret the output of PCA? I have dataset of 50000 values (rows) and 1000 variables (columns). Since this is high dimensional, I am unable to work with just DBSCAN.
So I am trying to use PCA (principle component analysis). Since PCA reduces dimension to few 100's.
I request you to help you to help me with
EITHER
 1. Other high dimensional clustering algorithm
 2. How to interpret results of PCA

In short PCA, returns an orthogonal set of basis features that best represent the variance in the data.

Intuitively, imagine you want to identify whether we are talking about a dog or a cat. Your features are: size, weight, color, fur type, etc... but you also have features like weather, owner name, etc...

It should be evident that the first set of features obviously explains the variance between a dog and a cat much better than the second set. Thus, you should only consider those and completely disregard the second set.

This is what PCA does however, it adds the extra restriction that all features must be orthogonal. It then assigns a metric to each component based on the amount of variance that feature explains. You can then sort the components and extract the top $n$ components (feature reduction).


Ordinal Classification As Cost Function - In Keras or Tensorflow I am having a hard time implementing/using an ordinal classification cost function for my Convolutional Neural Network in Tensorflow.

Any implementations/documentation/repository on that which you know of?

Whilst this question is now quite old, I was just looking for the same and came across this implementation for in Keras:

https://github.com/JHart96/keras_ordinal_categorical_crossentropy


Synthetic Gradients good number of Layers & neurons I would like to train my LSTM with a "synthetic gradients" Decoupled Neural Interface (DNI).

  How to decide on the number of layers and neurons for my DNI?
  Searching for them by trial end error or what's worse - by Genetic
  algorithm which would seem to outweigh [sic] defeat the purpose of Synthetic Gradients.

  And, if my DNI is an LSTM itself - it seems it would take even longer
  to determine its optimal structure

SG speed up the training speed, by allowing multiple forward passes (with immediate weight adjustments), since DNI will already predict the future gradient.

However, we will lose time "experiencing" a few hundreds of training sessions only to find a optimal structure of DNI with which it will predict gradient the best way.

By that time we could have already finished our training with an oldschool Backprop through Time.

  Also, how should we avoid our DNI overfitting, how to monitor and ensure it's not happening?

Several months later, I have a couple of insights on it:

  Also, how should we avoid our DNI overfitting, how to monitor and ensure it's not happening?

I don't think it matters, because DNI overfitting on the gradient is actually what we want. We want it to figure-out the pattern which reduces the error in the fastest way for our data.  However, as always, we should pay attention to the Validation of our entire network while doing so.

That probably means the more DNI neurons the better, as long as the designated training data stays unmodified o_O
If new or extra data is used for training, we should throw away our DNI, and just train once more, to overfit them on this new "adjusted" training-data.

Also, more on Synthetic Grads here

wish for a better answer, so won't select this one - plz post if you have a better one


Reinforcement algorithm for Trading I am trying to implement a Reinforcement Learning Algorithm in a trading scenario.
It seems natural to me to use final profit as the reward. However, in this scenario, for each trading episode, there is only one reward, obtained at the end. Is there any RL algorithm that could learn only based on final reward ?
If it was a mario game, how could I train a network only based on a reward earned at the end of the level, but that depends of all previous actions ?

What I'm trying to do is very similar to https://dzone.com/articles/trading-strategies-using-deep-reinforcement-learni

If you would have posted this in quantF SE site the most people there would have argued that it is not possible with machine learning techniques to do trading. - One can verify this because of the thread you linked in your comment. - The underlaing argument they are making have something to do with Efficient Market Hypothesis (Fama 1970) and the CAPM. You can read more about that in the original article or on wikipedia.
The point is that the EMH is a scientific hypothesis, that holds under certain circumstances. Nothing more and less. The research in the past decades proved the hypothesis in several markets and configuration. But the problem - which has been fixed by Fama later - is the size premium and the value premium. Because of this, a lot of people (e.g. Buffett and Graham) relayed on value investing (or fundamental) strategies.
The EMH additionally builds the ground for the ETF hype. - I'm not saying that this isn't working. It is just the only reliable product we have to invest in stock markets with sustainable return. - If you think about how ETFs work, you will quickly understand that one ETF build a large selection of stocks. In statistical terms: You widen your sample to get less bias, but you also get lower variance.
What AI practitioners try to do, and deeply believe is: It could be that a human is not able to beat the market by picking stocks, but a computer, which is a excellent machine for pattern recognition - would be able to do it; be able to pick a sample what represents the winners in every market situation.
For deeper diving in to this topic you can read the Article by Hsu et al. (2016). They did research about if and how this could be possible.
The beautifully that Deep Reinforcement Learning brings us is, that it learns itself to recognize the patterns hidden in the data, extract the features and learn how to take the actions. It is forecasting and action taking in one step. For a deeper understanding of these algorithms and underlaying theory I would recommend you to read the book by Sutton and Barto (2018). It is like the bible in (Deep) RL.
Additionally you should watch the Deep RL course by David Silver and the updated one by other researchers of DeepMind.
A good starting point to understand DRL for trading with code is the ICAIF 2020 paper and Github Repo by Yang et al. (2020), which I relay on for my masters thesis.
References
Fama, E. (1970). Efficient Capital Markets: A Review of Theory and Empirical Work. The Journal of Finance, 25(2), 383-417. doi:10.2307/2325486
Hsu, Ming-Wei; Lessmann, Stefan; Sung, Ming-Chien; Ma, Tiejun; Johnson, Johnnie E. V. (2016): Bridging the divide in financial market forecasting: machine learners vs. financial economists. In: Expert Systems with Applications 61, pp. 215-234.
Sutton, Richard S.; Barto, Andrew (2018): Reinforcement learning. An introduction. Second edition. Cambridge, MA, London: The MIT Press (Adaptive computation and machine learning).
Yang, Hongyang and Liu, Xiao-Yang and Zhong, Shan and Walid, Anwar, Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy (September 11, 2020). Available at SSRN: https://ssrn.com/abstract=3690996 or http://dx.doi.org/10.2139/ssrn.3690996