What are the disadvantages of Azure's ML vs a pure code approach (R/SKlearn) Good Day,
Microsoft offers their Azure Machine Learning Platform: https://azure.microsoft.com/en-ca/services/machine-learning/

Azure Machine Learning is designed for applied machine learning. Use best-in-class algorithms and a simple drag-and-drop interface—and go from idea to deployment in a matter of clicks.
...
Use Azure Machine Learning to deploy your model into production as a web service in minutes—a web service that can be called from any device, anywhere, and that can use any data source.

By their demo and their photos online it looks like a simple GUI application you can drag and drop your various pre-processing elements, estimators, and testing schemes and appears to make it easy to get started on Machine learning projects.
What no website selling a product will do is advertise their downsides.  With a GUI like Weka you lack the ability to fine tune and tweek the different parameters.
Azure ML seems to allow hooks into Python or R to give you more control and it sure would be appealing to have to do less work to get the same results.

What are the known disadvantages of using Azure ML for ML projects vs writing the same process in code (e.g. Sci-kit Learn or in R)?

I learned the Azure ML platform after scikit-learn (Python) and caret (R).

Advantages:

Low or no coding involved. It is more like Orange in that you set up the ML exercise as a large process flow. It is easier for non-technical people to get started with.
Easy to convert to production-ready. You click a button and it deploys a web-service that you could send to a programmer to use. In Python, you would need to add another library and in R it is even dicier.

Disadvantages:

Fewer algorithms (e.g. no XGBoost) and other transformations (e.g. NLP) built-in. You can of course reference Python and R to do this, but that is much tougher than using the GUI. If you can figure out how to call Python/R from within Azure ML, you likely can do it on your own.
Price.


Should weight distribution change more when fine-tuning transformers-based classifier? I'm using pre-trained DistilBERT model from Huggingface with custom classification head, which is almost the same as in the reference implementation:

class PretrainedTransformer(nn.Module):
    def __init__(
        self, target_classes):
        super().__init__()
        base_model_output_shape=768
        self.base_model = DistilBertModel.from_pretrained("distilbert-base-uncased")
        self.classifier = nn.Sequential(
            nn.Linear(base_model_output_shape, out_features=base_model_output_shape),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(base_model_output_shape, out_features=target_classes),
        )

        for layer in self.classifier:
            if isinstance(layer, nn.Linear):
                layer.weight.data.normal_(mean=0.0, std=0.02)
                if layer.bias is not None:
                    layer.bias.data.zero_()

    def forward(self, input_, y=None):
        X, length, attention_mask = input_
        base_output = self.base_model(X, attention_mask=attention_mask)[0]
        base_model_last_layer = base_output[:, 0]
        cls = self.classifier(base_model_last_layer)
        return cls

During training, I use linear LR warmup schedule with max LR=5-e5 and cross entropy loss.
In general, the model is able to learn on my dataset and reach high precision/recall metrics.

My question is:

Should weights distributions and biases in classification layers change more during training? It seems like the weights almost do not change at all, even when I do not initialize them as in the code (to mean=0.0 and std=0.02). Is this an indication that something is wrong with my model or it's just because the layers I've added are redundant and model does not learn nothing new?

Take look at the image of weight from the tensorboard:

Its difficult to judge by just looking at the weight distribution. You will be in a better position to identify the situation if you check the following things also:

1) Check if the loss is decreasing during training. It is very unlikely that a randomly initialised last layer will not need training to succeed in a downstream task. This is because even if the initialise layers are producing meaningful values, the randomly initialised final layer will output garbage values in the beginning. If the loss is decreasing then the model training is fine. Alternatively, check if the final if the final metric is improving e.g., accuracy.

2) Check the gradient received by your custom layers. One of the reason that the weights distribution is not changing could be due to zeroed or negligible gradient. I don't think gradients are zero because your plots show the weights to be changing, but still if If the gradient is zero then check your implementation. Something must be off like not using the actual output of the model while computing the loss for backpropagation. If the grads are very very small, try increasing the learning rate of the optimzer.

Conducting the above 2 analysis will strengthen your understanding of the current situation for the model training.


non-binary nominal variable in linear regression For example, I have a variable Color with 3 different values. How can I transform it, so that I can use this variable in linear model?
I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables? Shouldn't independent variables be not dependent on each other?

I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables?

Yes, you can use dummy variables to represent a multilevel qualitative variable like Color.

ISLR Example

An example that may be helpful to you can be found in ISLR, chapter 3 on pages 85 and 86:

The results in the book can be recreated using the Credit data set.

In the example given in the book, balance is regressed onto ethnicity. That is, balance is the quantitative dependent variable (if it were qualitative this would be a classification problem rather than a regression problem), and ethnicity is a 3-level qualitative variable.

> Credit <- read.csv("http://www-bcf.usc.edu/~gareth/ISL/Credit.csv", header = TRUE)
> attach(Credit)
> summary(Balance)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
   0.00   68.75  459.50  520.01  863.00 1999.00

> summary(Ethnicity)
African American            Asian        Caucasian
              99              102              199

The task is to predict Balance based on Ethnicity. Therefore Ethnicity needs to be dummified.

As stated in the text pictured above,

  There will always be one fewer dummy variable than the number of levels. The level with no dummy variable—African American in this example—is known as the baseline.

is.Caucasian <- ifelse(Ethnicity=="Caucasian", 1, 0)
is.Asian <- ifelse(Ethnicity=="Asian", 1, 0)

The above code accomplishes creating dummy variables for Caucasian and Asian, with African American as the baseline. Now to create the linear model:

balance.lm <- lm(Balance ~ is.Caucasian + is.Asian, Credit)
summary(balance.lm)

Call:
lm(formula = Balance ~ is.Caucasian + is.Asian, data = Credit)

Residuals:
    Min      1Q  Median      3Q     Max
-531.00 -457.08  -63.25  339.25 1480.50

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept)    531.00      46.32  11.464   <2e-16 ***
is.Caucasian   -12.50      56.68  -0.221    0.826
is.Asian       -18.69      65.02  -0.287    0.774
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 460.9 on 397 degrees of freedom
Multiple R-squared:  0.0002188, Adjusted R-squared:  -0.004818
F-statistic: 0.04344 on 2 and 397 DF,  p-value: 0.9575

The results are an exact match to the information displayed in Table 3.8 on page 86.

Example using 3 colors

Let's say we wanted to predict the mass of an ball using its color. In this case quantitative variable mass is the response (dependent variable) and color is the independent variable.

> color <- c("red", "black", "green", "red", "green", "red", "red")
> mass <- c(   12,      40,       1,    15,       2,    14,    14)
> ball.data <- data.frame(mass, color)
> head(ball.data)

  mass color
1   12   red
2   40 black
3    1 green
4   15   red
5    2 green
6   14   red

We can dummify color into two variables, one for red and one for black, with green as the baseline.

> ball.data$is.red <- ifelse(ball.data$color=="red", 1, 0)
> ball.data$is.black <- ifelse(ball.data$color=="black", 1, 0)
> head(ball.data)

  mass color is.red is.black
1   12   red      1        0
2   40 black      0        1
3    1 green      0        0
4   15   red      1        0
5    2 green      0        0
6   14   red      1        0

Then we fit a linear model like so:

> ball.lm <- lm(mass ~ is.red + is.black, ball.data)
> summary(ball.lm)

Call:
lm(formula = mass ~ is.red + is.black, data = ball.data)

Residuals:
         1          2          3          4          5          6          7
-1.750e+00  5.551e-17 -5.000e-01  1.250e+00  5.000e-01  2.500e-01  2.500e-01

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)   1.5000     0.8101   1.852 0.137727
is.red       12.2500     0.9922  12.347 0.000247 ***
is.black     38.5000     1.4031  27.439 1.05e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.146 on 4 degrees of freedom
Multiple R-squared:  0.9947,    Adjusted R-squared:  0.9921
F-statistic: 376.7 on 2 and 4 DF,  p-value: 2.79e-05

When we compare the coefficient estimates in the linear model summary to the values in the data set, we see they are pretty close, as expected. That is about it.

More info

PSU STAT 501: Regression Methods, Lesson 8: Categorical Predictors

CODING FOR CATEGORICAL VARIABLES IN REGRESSION MODELS | R LEARNING MODULES


Do we have any method which handles the imbalanced classification with sample weighting instead of class weighting? I am looking for methods that use sample weighting instead of class weighting for the imbalanced classification. I think sample weighting is more precise than weighting all the samples from one class with the same weight. Am I thinking right? Is there any paper that discusses this issue?

If you assign to each instance the weight of the corresponding class the effect will be similar and sometimes exact.
I say similar because there are methods which uses sample weighting intrinsically, for example AdaBoost or almost all the types of decision trees, but they do perform also some arithmetics with those weights which can lead to different results. For example in CART trees instances have a weight and if it happens that at a test node to have a missing value, the same instance goes into both child nodes but with weights splitted (their sum is equal with the original weight). So instace weights could have a more granular effect, and perhaps a better resolution and in the end maybe a better performance.
A second way often encountered in modelling is to introduce learning bias into the problem. Take for example weighted linear regression. The learning bias is your knowledge about how much you trust the value of an observation. It can be done automatically for example judging how much variability has the observation relatively to it's neighbour observations (high variance means less trustful value). The same idea of encoding trust into observations could be pushed into any classifier if it allows you to weight instances.
A last note is that a common interpretation of class weighting is in terms of cost of error (how much it will hurt you if a class A instances is predicted as class B instance) and instance weighting is usually encoded like how much trust do you have in an observation or, alternatively, how much do you want that that observation and similar with that one to be correctly classified. And, as I said in the beggining, they can have a similar effect.
Class imbalance is just a particular problem to be solved, instance weighting allows you to solve this problem and also other issues related with your data.


Architecture Question I have a situation where I need to propose a solution along with stack of technologies that need to be used for the below business case

Business Case:
I am receiving a car manufacturing data set from various car manufacturing companies, the data set for each car model is different and have different number of attributes but I do receive them in csv files every 2 minutes. I need to choose a particular storage format so that it is easy to query by each car model and based on the model, i select, i should be able to display the different features of that car model. Is there a recommended architecture for this type of data analysis?

Given that the data model you receive is different, it makes best sense for you to use a NoSql store like mongodb.

Here is the tech stack I would use ( python ) :

To receive requests with csv file, I would use flask microweb framework.
I would run flask with gunicorn and gevent.
For every csv file received, I would parse the rows based on varied columns and store them in a mongodb.
I would write queries on the collection in mongodb to extract information.

I would be more than happy to elaborate more if required .


What is the difference between SVM and GMM classifier What is the difference between support vector machine and Gaussian mixture model classifiers?

A Gaussian mixture model is a special case of a mixture distribution, which is a simple way of combining probability distributions.

However, the SVM does not make any assumptions. It is just a function, which depends on the distance of a data point from another point/plane. Som there is absolutely no probabilistic assumptions.


How to implement sequence to sequence models? I have a dataset with patient demographics, diagnosis history, hospital visit dates, drugs consumed etc.
All these events have time stamp information (except static info like demographics such gender, age, education etc).
My objective is to predict the next 3 visits to hospitals and risk factors for his next 3 visits based on his previous hospital records such as visit information, drug information, demographics, diagnosis information etc.
Can experts here let me know the list of steps that I need to follow so I can learn and try to achieve my project objective?

you need to define how many time steps you want to have in each time series block.
then for each unique patient, you need to create these blocks so the training set going to be a 3D matrix, and the dimensions are:
number of blocks * number of time steps * number of features
in addition to time series data, you can also add another head to feed NN with the constant data like patient id, day of the week, and ...
for the time series part, you can use biLSTM or biGRU or and other RNN layers and then a Dense layer and then concatanate the results with the results coming from constant features.
for the output, you can also use an RNN layer to predict the sequence of future visits or simply use Dense layers.
==============Updated
consider below flow chart
if your other features (like drugs, lab tests) don't have the same time structure, consider it as groups of time series and for each one add other heads along with RNNs with different input shapes (for the number of time steps). the key is to bring all your features to the latent space and from there, do the decoding and predict the sequence of the future.


K-means clustering for one class classification I want to know if I can use the k-means clustering algorithm for a one class classification (as in the case of one class SVM), which means I have data for 2 classes, and I labelled only the one class that I used for training?

The K-means algorithm has the capacity of retrieving which are the "boundaries" your data has for knowing the only-class, is possible that you don't find the only-class boundaries to be the same boundaries your k-means algorithm found. This is the risk of comparing k-means with the one class classification: Clustering can be looking different things from one class classification.
This answer may guide you to another solution.

How to perform model selection for One-Class Classification?


Label distribution over training, validation and test I am wondering over whether the number of classes distributed over my training, validation, and test label affects the model.

yes of course, the model will learn to recognize the label 0.0 very well and maybe have an idea about the others, unbalanced dataset is common problem in ML, look at Oversampling and Undersampling


Cluster Hyperspectral Images in Orange Orange has a hyperspectral dataset called "liver cirrhosis" and you can visualize the hyperspectral image using the hyperspectral image widget. However, I would like to perform various clustering methods e.g. k-means on the spectra at each pixel and then display those clusters on the hyperspectral image widget like in this photo

does anyone know how to do this kind of analysis?

As a first step, You can take the table of measurements as-is, run k-means on all the columns representing intensities.
In the image, I see pixels in 5 colors (red, orange, green, azure, magenta), so the author of the top image has decided to run k-means with k=5 clusters. (k in in k-means is often chosen to be an odd number)
To get a clearer signal, people often run preprocessing steps such as Principal Components analysis (PCA) on their datasets before running k-means, and then choose the first, say, 10 components (should be larger than 5). Then people run k-means on those 10 columns.
Choosing the number of components to calculate k from, and then choosing a value for k, is basically guesswork, at least initially.
It is also guesswork to assign colors to cluster numbers found by the k-means algorithm.
So you probably have to set a lookup table manually, in another preprocessing step, such as
{
1 => "red".
2 => "azure",
3 => "limegreen"
4 => "yellow",
5 => "orange" }. And then make an x-y plot coloring the pixels accordingly. The sequence is arbitrary.
Sorry, I don't know how to do this in Orange.


How to build a webscraper for indeed.com How can I scrape a URL such as: https://www.indeed.com/jobs?q=data+scientist+%2420%2C000&l=New+York&start=10
where I want to get the following parameters: job title, company name, job description, city, location, and salary.
What are the basic steps I need to take to build a scraper?

Scraping a website is very particular to the case. To try and make this answer useful to other people I will go through some basic concepts as to how this can be done successfully. Below I will use your URL as an example to put everything together.

First we need a means to access the HTML content of the website. For static websites we can use BeautifulSoup. This is a very powerful package which allows you to navigate an HTML. To get the HTML we will use the requests packages.

from bs4 import BeautifulSoup
import requests

url = 'https://www.indeed.com/jobs?q=data+scientist+$20,000&l=New+York&start=10'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')

BeautifulSoup requires a parser, I have had a lot of luck using lxml, however html.parser is also very popular. The parser is what is used to access the HTML tags and identify its inner elements.

Most websites are quite hard to scrape because they will reuse the same name for multiple element tages. This makes it harder to get the elements and extract their values. This is either due to the developers laziness or their expertise to avoid people abusing their resources by running scrapers on their data. To circumvent this you can use this simple BeautifulSoup trick. You can chain your .find calls to hone into the part of the HTML you want to scrape.

Using the above example URL we can identify where in the HTML we can find the results on the page for each job posting.

len(soup.find_all('div', {'class': 'row'}))

  16

Now we will want to collect data from each of these HTML elements. The data we will want to collect is: job title, company name, job description, city, location, and salary.

Each record of data will be collected into a dictionary with all the desired data as listed above. These dictionaries will be collected into a list which will be used to create the pandas DataFrame.

To find the individual data points you can use your browser and inspect elements to find out the name of the element parameters in order to use BeautifulSoup to get the information.

data = []
for i in soup.find_all('div', {'class': 'row'}):
    job_title = i.find('a', {'data-tn-element': 'jobTitle'})['title']
    company_name = i.find('span', {'class': 'company'}).text.strip()
    job_summary = ''.join([j.text.strip() for j in i.find_all('span',
                                                              {'class': 'summary'})])
    location = i.find('span', {'class': 'location'})
    if location is not None:
        location = location.text.strip()

    salary_range = i.find('span', {'class': 'no-wrap'})
    if salary_range is not None:
        salary_range = salary_range.text.strip()

    datum = {'job_title': job_title,
             'company_name': company_name,
             'job_summary': job_summary,
             'location': location,
             'salary_range': salary_range}

    data.append(datum)

df = pd.DataFrame(data)
df.head()


Time Series Data Multi-Class Classification This is a very general question, as I'm still very much in the learning phase with machine learning.  I have some utility data around problematic meters.  Even tho the data is "time series", I believe I can perform a multi-class classification (looking at 3 labels) towards the data, but would like some opinions before I pursue down that road.

I have been doing some feature engineering to derive other data points to help with the classification process (examples below are columns "Error1" and "Error2").

The meters come in 2 classes, those that are estimated issues ="1", and does that are non-estimated issues ="0".

My dataset roughly looks like below (I have several other Error features):

 Estimated     Meter ID          Date             DaysInDuration    Error1  Error2
     0            BBA         11/19/2019               31              0       0
     0            BBA         12/19/2019               62              1       0
     0            BBA         12/19/2019               92              1       0
     1            JJL         11/2/2019               120              1       0
     1            JJL         12/2/20019              150              1       1
     1            JJL         1/20/2020               180              2       2

What I would like to attempt is to use a classification model that can handle multi-class classification (possibly a decision tree), and produce a output such as below:

 Estimated     Meter ID          Date             DaysInDuration    Error1  Error2   Classification Label
     0            BBA         11/19/2019               31              0       0            1
     0            BBA         12/19/2019               62              1       0            1
     0            BBA         12/19/2019               92              1       0            2
     1            BBA         11/2/2019               120              1       0            3
     1            JJL         12/2/2020                30              1       1            1
     1            JJL         1/20/2020                60              2       2            1

Labels Meaning = "1" = low risk issue/ "2" = medium risk issue/ "3" = high risk issue

The model would classify the either "1","2", or "3" depending on the length of days the meter has been in the "DaysInDuration" column, and the number of counted errors in the "Error1" and "Error2" columns.

In my thoughts it feels like classification would still work, including with train test splits, as the classification is moreso from other data points versus the actual order dependency in a typical time series problem.

Your time intervals seem irregular. For most examples of time series, we have observations at fixed intervals, such as GDP every quarter. You can instead just create a regular machine learning model. If you think there is some sort of time component where you would not want January data to make predictions for November data, then you can manually create your cross validation partitions instead of creating them randomly.

However, you are predicting 1, 2, or 3 based on low, medium, or high DaysInDuration. A classifier would not know that low and high are farther apart than low and medium. You can instead create a regression model that predicts DaysInDuration as your target feature. Once you have your prediction (e.g. 31), you can act accordingly whether the predictions is low, medium, or high.

Finally, you wouldn't be able to use DaysInDuration as a feature to predict the Classification_Label because it would give you the perfect answer every time. That would be target leakage.


PCA and orange software I am analysing if 15 books can be grouped according to 6 variables (of the 15 books, 2 are written by an author, 6 by an other one, and 7 by an other one).
I counted the number of occurrences of the variables and I calculated the percentage. Then I used Orange software to use PCA. I uploaded the file. selected the columns and row. And when it comes to PCA the program asks me if I want to normalize the data or not, but I am not sure about that because I have already calculated the percentage - is normalize different from calculating the percentage? Moreover, below the normalize button it asks me to show only:... and I have to choose a number between 0 and 100 but I don’t really know what it is.

Could you help me understand what I should do?
Thank you in advance

You should normalize data but you dont have to. If you dont normalize your data bigger range data will have higher effect on the model.

Just draw a box plot to compare the ranges of your data. If they have different ranges (like 100->200 and 1->10) you should definitely normalize your data. (Like mean normalization)


Sampling Big Data for Predictive Analytics in Python In practice, how does one go about sampling a from big data set (eg. +/- 50 million distinct observations) to perform ML using Python?
Most non-parametric models (e.g., SVM, ensemble models) start to push computer resources with much smaller sets (e.g., 200 features, 200K observations).

How is this done in practice in industry?

Other questions here get at this but don't explicitly ask. So this is not a duplicate.
Thanks in advance.

This is what I do in projects :

Pre-process data in DB / Data Lake. aim is to :
A. Form batches (might require a new table with shuffled indices)
B. Create a copy with Normalization and other feature related tasks

After this, try algorithms that support batch learning. Neural networks support batch learning and few other algos (https://sklearn.org/modules/scaling_strategies.html#incremental-learning).

With batch learning, you can plot loss for each batch and see if algo is working or not.


Difference between Data Engineer and Data Scientist I'm very confuse with the term Data Engineer and Data Scientist. There are lot of jobs available for both roles in current market with almost same technical skills requirement. Are they same or different ? Which role is recommended for someone from coding background (C++, Java, Python) along with RDBMS knowledge ?

Data engineering is infrastructure work; maintaining "big data" pipelines from ingestion to output. Today you might be expected to know things like SQL, Hadoop, Spark, Docker, and AWS.

Data science is an umbrella term, so it can mean a lot of things, including data engineering. But it can also mean pure data analysis without any production work. It really depends on who's using the term; read the job description and ask the company for details.


sklearn.model_selection: GridSearchCV vs. KFold Here is the explain of cv parameter in the  sklearn.model_selection.GridSearchCV:

cv : int, cross-validation generator or an iterable, optional
Determines the cross-validation splitting strategy. Possible inputs for cv are:

integer, to specify the number of folds in a (Stratified)KFold

For example, can I replace
CV = 5

to
CV = KFold(n_splits=5, random_state=None, shuffle=False)

or replace in the opposite way. CV is called in the function
GridSearchCV(estimator=XXX, ... , cv = CV)

I am not sure whether any difference?

Yes, you can replace the cv=5 with cv=KFold(n_splits=5, random_state=None, shuffle=False). Leaving it set to an integer, like 5, is the equivalent of setting it to either KFold(n_splits=5) or StratifiedKFold(n_splits=5), depending on the model you pass to the estimator parameter of GridSearchCV()


Gradient passthough in PyTorch I need to quantize the inputs, but the method (bucketize) I need to do so is indifferentiable.  I can of course detach the tensor, but then I lose the flow of gradients to earlier weights.  I guess the question is quite simple, how do you continue the flow of gradients when necessary.  For example, using the following code ...
x = self.linear1(x)
min, max = int(x.min()), int(x.max())
bins = torch.linspace(min, max+1, 16)
x = torch.bucketize(x.detach(), bins) # forced to detach here
x = self.linear2(x)

I know it's possible, as it was done with VQVAE and the gradients still flow fine for their quantization purposes.  But I checked several versions of VQVAE code and it doesn't seem logical how the gradients pass through the argmin method.  It just seems to work regardless, which confuses me.  I'm quite perplexed by this.  I would be grateful for any help on this one.  Thanks in advance.

Thank you @thanatoz for answering that in the comments.  In fact, this did solve my problem.  I also realized from other people's VQVAE code, there is also another possible solution in which the retrain_graph parameter is not needed.  My example code could be written ...
x = self.linear1(x)
min, max = int(x.min()), int(x.max())
bins = torch.linspace(min, max+1, 16)
x_buckets = torch.bucketize(x.detach(), bins) # forced to detach here
x = x + (x - x_buckets).detach() # Reintroduce gradients here. <------------
x = self.linear2(x)

The answer is in the fifth line, by subtracting and then adding back again.  Gradients can then be reintroduced.


Tactics to avoid feeling overwhelmed by machine learning Short version: despite lots of reading, machine learning still feels like being a monkey in the dark. Any advice?
For background, I'm a researcher in computer science, in a field non-related to machine learning.
I have been trying to get more proficient in machine learning*, yet no matter how much I read and fiddle with code/toy datasets, when I try to go to a harder problem, I always feel overwhelmed by the choices I need to make:

I have to choose the algorithm: This is the part I typically  find the most straightforward;
For said algorithm, I have to choose the objective function :  usually, many are applicable, and I find it difficult to gain a good intuition of what makes an objective function adapted in some cases rather than others, apart from the very classical ones for linear or logistic regression
And then, I should devise the features: this still feels completely arcane to me, apart from using content-based features readily available in the data.

I am under the impression that I have to "create" the tailored algorithm and the data.
Concerning the algorithm, I have spent some time into studying gradient boosting and the math behind it, to the point that I have a reasonably solid comprehension of how it works, and an intuition of parameter tuning for simple datasets. However, that knowledge does not generalize.
How are these issues typically approached? Are there any resources that can help?
* By taking the Machine Learning  Coursera course and its more in-depth version, reading more XgBoost-specific material (on its internals and parameter tuning and intuition), as well as playing with the Titanic dataset, and a housing market dataset.

We have to climb up a steep learning curve when we learn about machine learning. Your question is quite general: One of the tactics I use when learning is divide and conquer. Get some coarse overview about the whole area, then pick some particular area and dig deeper only there.
Perhaps the question is too general, the best tactic may vary and depend on the area you address.
But I am not sure if learning the math is always helpful (although it may always be interesting for those who care).

The algorithms can often be applied in a black box approach, and it may be sometimes not necessary to understand an algorithm in math terms (white box), but sufficient to know it's function, strengths and weaknesses (black box).

You may be the first one that tests that algorithm for the domain, so pure experimentation is useful in the end.


Logic behind pre-trained weights and transfer learning I am not sure about the logic behind, how pre-trained weights actually make sense and translate into a new problem.
To be more specific; for example in a object detection network, how would a model's weights that was trained, let's say, on the COCO dataset, with 80 categories, would translate into my new problem that only has 2 categories (classes).
How does this make sense? What kind of meaningful features could even be transfered from the previously pre-trained model to my new problem, since the number of categories (classes) have been changed, and also I'm trying to detect completely different type of objects than the previous model? Why do we do transfer learning then?

If we classify new objects using transfer learning:
1.We delete the top Dense layer of the pre trained neural network.
2.Now suppose you have to classify 5 classes, so your final dense layer will contain 5 nodes.
3.Also you will add some dense layers prior to your new 5 node Dense layer, so that you can train the model with new data.
4.All the layers prior to the few dense layers you added will be frozen.
5.Now you will train your new model with your dataset.

The Logic behind these steps are:
1.As CNN uses hierarchical based learning, meaning the initial layers will try to learn basics features in a dataset ie. lines, corners, shapes etc. So the frozen layers of your model have already learnt the basic features which are common to your dataset as well as the dataset it has been previously trained on.
2.The few Dense layers you added, will try to learn high level features specific to your dataset when you train your model with your dataset.


High model accuracy vs very low validation accuarcy I'm building a sentiment analysis program in python using Keras Sequential model for deep learning

my data is 20,000 tweets:

positive tweets: 9152 tweets
negative tweets: 10849 tweets

I wrote a sequential model script to make the binary classification as follows:

model=Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_words))
model.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(250, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Fit the model

print(model.summary())
history=model.fit(X_train[train], y1[train], validation_split=0.30,epochs=2, batch_size=128,verbose=2)

however I get very strange results! The model accuracy is almost perfect (>90)
whereas the validation accuracy is very low (<1) (shown bellow)

Train on 9417 samples, validate on 4036 samples
 Epoch 1/2
- 13s - loss: 0.5478 - acc: 0.7133 - val_loss: 3.6157 - val_acc: 0.0243
 Epoch 2/2
- 11s - loss: 0.2287 - acc: 0.8995 - val_loss: 5.4746 - val_acc: 0.0339

I tried to increase the number of epoch, and it only increases the model accuracy and lowers the validation accuracy

Any advice on how to overcome this issue?

Update:

this is how I handle my data

#read training data
pos_file=open('pos2.txt', 'r', encoding="Latin-1")
 neg_file=open('neg3.txt', 'r', encoding="Latin-1")
# Load data from files
pos = list(pos_file.readlines())
neg = list(neg_file.readlines())
x = pos + neg
docs = numpy.array(x)
#read Testing Data
pos_test=open('posTest2.txt', 'r',encoding="Latin-1")
posT = list(pos_test.readlines())
neg_test=open('negTest2.txt', 'r',encoding="Latin-1")
negT = list(neg_test.readlines())
xTest = posT + negT
total2 = numpy.array(xTest)

CombinedDocs=numpy.append(total2,docs)

# Generate labels
positive_labels = [1 for _ in pos]
negative_labels = [0 for _ in neg]
labels = numpy.concatenate([positive_labels, negative_labels], 0)

# prepare tokenizer
t = Tokenizer()
t.fit_on_texts(CombinedDocs)
vocab_size = len(t.word_index) + 1
# integer encode the documents
encoded_docs = t.texts_to_sequences(docs)
#print(encoded_docs)

# pad documents to a max length of 140 words
max_length = 140
padded_docs = pad_sequences(encoded_docs, maxlen=max_length, padding='post')

Here I used Google public word2vec

# load the whole embedding into memory
embeddings_index = dict()
f = open('Google28.bin',encoding="latin-1")
for line in f:
values = line.split()
word = values[0]
coefs = asarray(values[1:], dtype='str')
embeddings_index[word] = coefs
f.close()

# create a weight matrix for words in training docs
embedding_matrix = zeros((vocab_size, 100))

for word, i in t.word_index.items():
embedding_vector = embeddings_index.get(word)
if embedding_vector is not None:
    embedding_matrix[i] = embedding_vector

#Convert to numpy
NewTraining=numpy.array(padded_docs)
NewLabels=numpy.array(labels)
encoded_docs2 = t.texts_to_sequences(total2)

# pad documents to a max length of 140 words

padded_docs2 = pad_sequences(encoded_docs2, maxlen=max_length, padding='post')

# Generate labels
positive_labels2 = [1 for _ in posT]
negative_labels2 = [0 for _ in negT]
yTest = numpy.concatenate([positive_labels2, negative_labels2], 0)
NewTesting=numpy.array(padded_docs2)
NewLabelsTsting=numpy.array(yTest)

When a machine learning model has high training accuracy and very low validation then this case is probably known as over-fitting. The reasons for this can be as follows:

The hypothesis function you are using is too complex that your model perfectly fits the training data but fails to do on test/validation data.
The number of learning parameters in your model is way too big that instead of generalizing the examples , your model learns those examples and hence the model performs badly on test/validation data.

To solve the above problems a number of solutions can be tried depending on your dataset:

Use a simple cost and loss function.
Use regulation which helps in reducing over-fitting i.e Dropout.
Reduce the number of learning parameters in your model.

These are the 3 solutions that are most likely to improve the validation accuracy of your model and still if these don't work check your inputs whether they have the right shapes and sizes.


What does "batch" and "batch_size" mean in word2vec skip-gram model? I am reading tensorflow documentation now and I can not understand what does "batch" and "batch_size" mean in explanation skip-gram model. Please, can someone explain me?
Here this paragraph:

Recall that skip-gram inverts contexts and targets, and tries to predict each context word from its target word, so the task becomes to predict 'the' and 'brown' from 'quick', 'quick' and 'fox' from 'brown', etc. Therefore our dataset becomes
(quick, the), (quick, brown), (brown, quick), (brown, fox), ...
of (input, output) pairs. The objective function is defined over the entire dataset, but we typically optimize this with stochastic gradient descent (SGD) using one example at a time (or a 'minibatch' of batch_size examples, where typically 16 <= batch_size <= 512).

A minibatch, is a group of (input,output) pairs that you present to your neural net in one pass (or epoch), without computing the stochastic gradient descent between two pairs (you only do it at the end of the minibatch, summing the errors over the pairs). This improve the speed and prevent over learning on single elements, thus improving learning. It also doesn't need to much memory since your batches are "mini".

Some people call stochastic training when your mini-batch has a size of 1, and batch training when it has the size of your training dataset.

You can get some nice explanations here.


How do I define my feature variables I want to predict the grade of a (german) health insurance by some features it offers and I am not sure about the best way to define these features.

How should I treat a variable which can be "80%", "90%" or "100%". If I use it categorical I loose the fact that 100% is better than 90%.
If numerical is better, should I use 80,90,100 or 0.8, 0.9, 1 ?
Categorical Variables like "multi bed", "double bed", "singe bed" also have a "is-better-than" relation but how should I transform them into numerical values? Ascending integers, or from -1 to 1 or from 0 to 1. Or does all this make no sense at all.
Should I treat grade as a numerical target or as categorical (1.0, 1.1 ...) as only these values are allowed outcomes.
What method of prediction do U suggest? I have around 80 features, most of the categorical (mostly yes/no).

I think that before anything else it's important to explore the data to identify features that aren't bringing anything worthwhile (v. low variance, duplicates, collinearity) and to see if you've got any issues with missing values/outliers in your numeric data.

With that said, I think you'd want to keep your numeric variables as such (initially at least), but how many do you have? You may wish to normalise them so that the scale of one variable doesn't dominate (eg. is income included?). Or if you have time, try both? Look at the distribution and see what bins might be appropriate to discriminate by:
https://en.wikipedia.org/wiki/Normalization_(statistics)

Ordinal data is tricky; in your case you can't have 1.5 beds, but that's what you might end up with if you treat them as numeric. I would initially run with one hot encoding, and when you get some results you can look at the impact of your features to decide if you want to keep them that way:
https://www.quora.com/What-is-one-hot-encoding-and-when-is-it-used-in-data-science

Without knowing the full range of your grade values or how granular/continuous your prediction needs to be, you could easily do either - I'd say think about what the ideal outcome would be and build towards that. If you've got time try both, try different bins for a multi-class classifier; but if you have a massive range of target outputs then regression is probably preferable.

Finally - start with something simple and interpretable, and build from there. If you run with classification, can a decision tree deal with your dataset, and how does it do/where does it fall down? You might get lucky, but if not you can try an SVM or random forest. Given the number of variables you might want to consider some dimensionality reduction first; you might get some good results from clustering.

Hope this helps!


Fitting a distribution to data I have a large data set of property values at two points in time. I have calculated the change in value between those two points. I want to assess which distribution most closely matches the variation in change in property values. What are some possible ways of selecting the closest type of distribution?

I plan to use python, scipy and pandas to do this, although my question is more about what are the correct statistical tests to do to determine whether one distribution is a better fit to the data than another.

My understanding of your problem: you have realizations of $X$ (property value at first point in time) and $Y$ (value at second point in time). Want to fit a distribution for $Y-X$--more generally, a "model for the data."

One way of doing this would be plot a histogram of these values and then come up with plausible distributions that might fit the data, and then use maximum likelihood estimation to find the optimal parameters. If the models are nested, then you can use, e.g., the likelihood ratio test to determine which is a better fit to the data.

If not, then things are trickier. One good option is to hold out some of the data for testing, fit the models on the training data, predict the test data using each model, and see which one performs better.


vanishing gradient and gradient zero There is a well known problem vanishing gradient in BackPropagation training of Feedforward Neural Network (FNN)(here we don't consider the vanishing gradient of Recurrent Neural Network).
I don't understand why vanishing gradient does not mean the zero gradient namely the optimal solution we want? I saw some answer said vanishing gradient is not exactly the zero gradient, just means the update of gradient is very slow. However, the stopping rule of gradient decent is just the unchange of parameter within $\epsilon.$
So can anyone give me a clear answer?

The setting:
We have a neural network $\phi_{\mathbf{w}}:\mathbb{R}^{n} \rightarrow \mathbb{R}^{m}$ with weights $\mathbf{w} \in \mathbb{R}^{q}$. A loss function $\hat{L}: \mathbb{R}^{m} \times \mathbb{R}^{m} \rightarrow \mathbb{R}$ evaluates the quality of a prediction.  If $x \in \mathbb{R}^{n}$ shall be mapped to $y \in \mathbb{R}^{m}$ by the neural network, the loss is given as $\hat{L}(\phi(x),y)$.
For a fixed dataset $D \subset \mathbb{R}^{n} \times \mathbb{R}^{m}$, we obtain the empirical error
$F(\mathbf{w}):= \sum_{(x,y) \in D} \hat{L}(\phi_{\mathbf{w}}(x),y)$.
Then $F: \mathbb{R}^{q} \rightarrow \mathbb{R}$.
Now $F$ is minimized using backpropagation.
Let us try to define the vanishing gradient term. I am not sure if there is a proper definition, but I would say we have a vanishing gradient at $p$ if $0 <||\nabla F(p)|| \leq c$ for some small $c$.
Raised questions:

If the gradient is almost zero due to the vanishing gradient, does that mean the current solution is very close to the optimum ? So we can stop iterating..

Why is it bad to have "vanishing gradients" ?

Adressing Question 1
Recall from school that if a functional $F$ has a local optimum at $p$, then $\nabla F(p) = \mathbf{0}$ and $D^2 F(p)$ definite.
If $D^2 F(p)$ is positive definite ($x^T D^2 F(p) x > 0$, for all $x$, where $D^2 F(p)$ is the Hesse matrix), then $p$ is local minimum.
If $\nabla F(p) = \mathbf{0}$ and $D^2 F(p)$ in definite, then $p$ is a saddle point.
In particular this shows that having a zero gradient does not always imply that the position is a local optimum.
(In case of $q = 1$ and $F$ being two times differentiable, $F$ has a local optimum at $p$, if $F'(p) = 0$ and $F''(p) \neq 0 $. )
We can also construct a function that can have arbitrary small gradient while being far away from the minimum:
Consider the  function $f_{c}(x) = \max\{0,cx\}$ with $c>0$. Then $\min_{x \in \mathbb{R}} f_c(x) = 0$. For any $p>0$, we have $f'_{c}(p) = c$.
As an example let $p = 10^{9999}$ and $c = 10^{-90}$. Then, the value $f_{c}(p)$ is far away from the minimum, still for the gradient $f'_{c}(p) = 10^{-99}$ holds, which shows that a small gradient does not imply that the current point is close to the optimum.
Adressing Question 2
Note that performing backpropagation is  performing the gradient descent algorithm.
Now to address the section questions, there are two directions (an analytical answer and a numerical answer).
The analytical answer would be that a vanishing gradient is nothing special that needs to be considered.
If the step size is choosen appropriately, it can be shown that the sequence of iterates $(p_k)$ is either finite with $\nabla F(p) = 0$, or it is an infinite sequence, and $\lim_{k \rightarrow \infty} \nabla F(p_{k}) = 0$, so that each limit point is a stationary point. This will work independent of any "vanishing gradients".
However, if we consider the question from the numerical aspect, there are certain issues.
1.) There is a machine epsilon $\epsilon$ so that updates with values smaller than $\epsilon$ cannot be performed numerically in a computer. This effectively means that the algorithm converges to some point if $||\nabla F(p)|| \leq \epsilon$.
2.) Even if the values are bigger than $\epsilon$, a "small" gradient vector results in very slow weight updates.
3.) The vanishing gradient problem may arise for example if the sigmoid function is used as activation in a deep neural network. This can be understood from the chain rule.
Let us compute a simple example where we have neural network consisting of $L$ layers, and each layer consists of a single neuron, without any bias. As activation function, we use the sigmoid function $\sigma(t) = \frac{1}{1+e^{-t}}$. Then $\sigma'(t) = \sigma(t)*(1-\sigma(t))$.
The output of the i-th layer is given as
$f_{i}(w) := \sigma(w o_{i-1})$, where $o_{i-1}$ is the output of the $i-1$-th layer.
Let us ignore the loss function and assume that $F$ is exactly the neural network.
The output at layer $i$ is denoted as $o_{i}$ so that we have $o_{i} := \sigma(w_{i} o_{i-1})$ and $o_{1} = w_{1}$.
Then $F(w_{1},\ldots,w_{L}) = o_{L} = \sigma(w_{L} o_{L-1})$.
According to the chain rule, we have $\frac{\mathrm{d}F}{\mathrm{d}w_{1}}(w) = \frac{\mathrm{d} \sigma}{\mathrm{d}w_{1}}(w_{L} o_{L-1}) = \sigma'(w_{L} o_{L-1})  \frac{\mathrm{d} w_{L} o_{L-1}}{\mathrm{d}w_{1}} = \sigma'(w_{L} o_{L-1}) w_{L}  \frac{\mathrm{d}  o_{L-1}}{\mathrm{d}w_{1}}$.
Repeatedly applying the chain rule results in:
$\frac{\mathrm{d}F}{\mathrm{d}w_{1}}(w) = \prod_{i = 2}^{L} w_{i} \prod_{i = 2}^{L} \sigma'(w_{i}o_{i-1})$.
Since $\sigma(t) \in [0,1]$, we have $\sigma'(t) \in [0,1]$. In particular, we often want $\sigma$ to output either $0$ or $1$. However, if $\sigma(w_{i}o_{i-1})$ is close to $0$ or $1$, then $\sigma'(w_{i}o_{i-1})$ will be close to $0$.
Now if one (or multiply) numbers are close to zero in $\frac{\mathrm{d}F}{\mathrm{d}w_{1}}(w)$, we obtain a very small number, which results in numerical issues (due to reaching machine epsilon), causing a very slow update during the algorithm.


Image regression problem I've tried a number of experiments with machine learning. From trying to use GANs to upscale images to playing with auto-encoders.
There is one problem that haunts me and always ends up ruining my experiments.
My networks always seem to "learn" to ignore the input and always produce the same image output. This output is often an interesting blend of the input images.
It seems that it's learning to cheat the test and produce the same image regardless of the input.
Here are some examples:

This has happened in both my GAN and Auto-encoder projects.
How do I avoid this sort of thing happening?

Couple of Things straight up:

Inject some noise in the process. When the gan or autoencoder learns that there is some noise it will start to generalise better

Use weaker architectures. (Analogy to weak learners, you cant build random forest wit hstrong trees). Basically allowing for your Architecture to be weak enough to be able to generalise and not learn everything by heart, i.e. map Input to Output.


Over-fitting issue in a classification problem (unbalanced data) I am working on a rare event (unbalanced target variable) classification problem using decision trees. My dataset comprises of 95% non-event and 5% minority (events) class.

I used decision tree over logistic regression because I had many categorical variables comparing to continuous variables. I get a  good performance for training data with the decision tree C5.0. However I get poor results for the new data. I use the confusion matrix as a measure of performance. Training model is over-fitting.

I did pruning to reduce the over-fitting caused by the decision tree.  I used the following code to build the model

Classifi_C5.0 <- C5.0(TARGET ~., , data = training_data_SMOTED, trails = 500,
                      control = C5.0Control(minCases = mincases_count,
                                            noGlobalPruning = FALSE))

I balanced the minority and majority class using the following code:

training_data_SMOTED <- SMOTE(TARGET ~ ., training_data,
                              perc.over = 100, k = 5, perc.under = 200)

Any sort of advice will be helpful.

I'm working on a similar classification problem using Quinlans C5.0 and in my experience there are couple of tweaks inside that might help you.

First, I believe using such big number (trials=500) for boosting is
affecting the overfitting, since the model will repeatedly try and
improve on the misclassified instances in every iteration. Try using
a smaller boosting size, or even turning it off - it might help with
the overfitting problem.
There are a couple of other parameters you might find useful. The minimal leaf size (minCases in C5.0Control() is something to you want to keep as big as possible when avoiding overfitting issues.
Another one is the confidence factor (CF) which lets you control
the severity of pruning of the tree, meaning lower factor levels will likely prune away the leaves which overspecify the classification.
Finally, you might try using the fuzzyThreshold control option which lets you 'soften' sharp thresholds with an inbetween grey area.

These are all very well documented in the C5.0 vignette as well as on the informal tutorial page.

Hope this helps you a bit.


Could you generate search queries to poison data analysis by a search engine? A simple problem with search engines is that you have to trust that they will not build a profile of search queries you submit. (Without Tor or e.g. homomorphic encryption, that is.)
Suppose we put together a search engine server with a use policy that permits constant queries being sent by paid customers.
The search engine's client transmits, at some frequency, generated search queries (e.g. markov, ML-generated, random dictionary words, sourced from news, whatever; up to you) in order to intentionally obscure the real search queries performed by customers. In other words it pretends to be a thousand contradictory personalities, nationalities, genders, races, hobbies, etc.
How difficult would it be to generate enough queries to hide yourself in the data?

I am not sure how many queries you'd need to perform to drown out your actual search queries, but there is already is an actual browser addon which does this. This addon is called TrackMeNot and is available to install for both Google Chrome and Firefox. More in-depth information on how this addon works can be found on their website and the whitepaper (section 3), but in short it create a dynamic list of queries based on popular search terms.


how can I recognize multiple faces from one image in python I wants to create an app which can recognize multiple faces from one image, I planed to built an android app through KIVI, I am not getting from where to start, it is my first minor project, any help will be appreciated.

In general, you perform object detection to find potential faces, and then you perform face recognition on the face(s) detected.

This article walks through the various steps quite well:
https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78

The author also has a Python library that does recognition of multiple faces in a scene with video examples:
https://github.com/ageitgey/face_recognition


How do I set/get heap size for Spark (via Python notebook) I'm using Spark (1.5.1) from an IPython notebook on a macbook pro. After installing Spark and Anaconda, I start IPython from a terminal by executing: IPYTHON_OPTS="notebook" pyspark. This opens a webpage listing all my IPython notebooks. I can select one of them, opening it in a second webpage. SparkContext (sc) is available already, and my first command in the notebook is help(sc), which runs fine. The problem I'm having is that I am getting a Java heap space error that I don't know how to address. How do I view my current Java heap setting, and how do I increase it within the context of my setup. The error message I'm getting follows:

Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 19 in stage 247.0 failed 1 times, most recent failure: Lost task 19.0 in stage 247.0 (TID 953, localhost): java.lang.OutOfMemoryError: Java heap space

You can manage Spark memory limits programmatically (by the API).

As SparkContext is already available in your Notebook:

sc._conf.get('spark.driver.memory')

You can set as well, but you have to shutdown the existing SparkContext first:

conf = SparkConf().setAppName("App")
conf = (conf.setMaster('local[*]')
        .set('spark.executor.memory', '4G')
        .set('spark.driver.memory', '45G')
        .set('spark.driver.maxResultSize', '10G'))
sc = SparkContext(conf=conf)

If your workload is the same for all analysis, then editing spark-defaults.conf as cited above is the way to go.


Splitting data in scikit-learn I know how to split the dataset into train and test sets using train_test_split but is there any way that I can split the dataset into three different sets, i.e., "Train set", "Test set" and "Validation Set". An example should be enough.

train_test_split is just a utility function around ShuffleSplit, which on its turn just randomly assigns each sample to either train or test, taking the desired probability into account.
You can do that however you'd like, and there's no real reason to use that specific function.

Its not too hard to come up with some code that does that for three values or N values, if you rather avoid calling train_test_split twice.


Neo4j vs OrientDB vs Titan I am working on a data-science project related on social relationship mining and need to store data in some graph databases. Initially I chose Neo4j as the database. But it seams Neo4j doesn't scale well. The alternative I found out are Titan  and oriebtDB. I have gone through this comparison on these three Databases, But I would like to get more details on  these databases. So Could some one help me in choosing the best one. Mainly I would like to compare performance, scaling, on line documentation/tutorials available, Python library support, query language complexity and graph algorithm support of these databases. Also is there any other good database options ?

I think you might have to keep overall Data pipelines and Machine learning pipelines in mind. For which you need a robust framework to move data between table like and graph like storage apart from powerful distributed processing. From my understanding Spark GraphX is promising to build these pipelines. Joseph Gonzalez's (one of the creator of GraphLab from CMU) talk on GraphX on youtube is worth watching.


feature selection techniques Is it always a good idea to remove features that have high mutual information with each other and to remove features that have very low mutual information with the target variable? Why or why not?

Doing that is a very good idea. The problem is that doing that is very hard.
Feature selection is a NP-complete problem.
The practical meaning as that we don't know any fast algorithm that can select only the needed feature.

In the other direction, omitting features that don't have mutual information (MI) with the concept might cause you to throw the features you need most.
There are cases in which a single feature is useless but given more features it becomes important.

Consider a concept which is the XOR of some features. Given all the features, the concept is totally predictable. Given one of them, you have 0 MI.

A more real life example is of age at death. Birth date and death date give you the age. One of them will have very low correlation (due to increase in life expectancy).

In practice, omitting features with low MI is OK. Many learning algorithms are using MI so they won't be able to use the omitted variables anyway.
As for the selection itself, there are many algorithms, usually heuristics or approximation algorithm that are quite handy.


How to deal with class imbalance in a neural network? Suppose we have a game and its action space contains two possible actions: A and B.

We have a labelled dataset of state-action pairs but 95% of actions are A and only 5% are B

If we train a neural network, it will always output A as it will choose the most probable class to decrease its loss.

Are there ways to solve this problem?

This is for classification, and I am not sure if it is possible to extend them to reinforcement learning.

As you figured out, accuracy should not be used as a metric for a dataset as imbalanced as the one you have. Instead, you should look at a metric such as Area Under Curve(AUC). If you would have infinite data, then you could just rebalance and remove some of the data from the class that has the most samples. However, in many cases data is sparse and you want to use as much of it as possible. Removing data can have a disastrous effect on many applications.

So what are good and convenient ways of handling this?

Add weights to the loss function. One weight for class A and one for B. By increasing the magnitude of the loss for the B class the model should not get stuck in a suboptimal solution that just predicts one class.
Use another objective(loss) function. F1-score can, for example, be implemented and used as an objective(loss) function.

What is great with these approaches is that it will allow you to use all the data.


Which NN architecture solves my problem? I want to convert images frame into text, for instance, if I have a image with a dog that plays with a ball I will produce a simple text "Dog plays with ball" (Ok, I know I must have x samples training, but I mean ipotetically).

Which type of NN allows me to perform that action?

Thank you

Welcome to SE:DataScience.

What you are looking for is called image captioning. A common approach for this is called an encoder-decoder model, where the encoder is a CNN-based NN to encode the content of an image into a vector representation, and a RNN-based decoder generates its caption conditioned on that vector.

This is a relatively well-studied area, with a lot of well-formatted dataset and open-source implementations, for example in TensorFlow. The link above can also serve as a good tutorial for this topic.

More recent models can use the attention mechanism to focus on specific part (object) in an image, do dense captioning, or answer questions about images, etc.


How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN? For several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all.

How does IQN differ from Quantile Regression Network, in plain language?

In Reinforcement Learning, a DQN would simply output a Q-value for each action. This allows for Temporal Difference learning: linearly interpolating the current estimate of Q-value (of the currently chosen action) towards Q' - the value of the best action from the next state.

Quantile-Regression Network takes it a step further, outputting a range of values for each action. So if with DQN we had a vector of Q values, QR-DQN gives us a vector of sub-vectors of Q values. These Q-values are equally likely to occur when compared amongst each other (in that sub-vector).

For example, if we have 3 possible actions, QR-Network might output the following:

Q estimates for Action0:  -15,  -5,  120,  121   130
Q estimates for Action1:   20,  25,  200,  210   300
Q estimates for Action2:   18,  40,   41,   54   120

Once again, when working with QR-Network, we treat -15,  -5,  120,  121,  130  as having the same probability of occurring: in my case  0.2  or 20%

In other words, each action now has values sitting at the middles of 5 separate quantiles:  at $\tau$ = 0.1, $\tau$=0.3, $\tau$=0.5, $\tau$=0.7, $\tau$=0.9

We can therefore say, that for the Action0:

with probability of 20% values within [$-\infty$, -10] will occur
with probability of 40% values within [$-\infty$, 57.5] will occur
with probability of 60% values within [$-\infty$, 120.5] will occur
with probability of 80% values within [$-\infty$, 121.5] will occur
with probability of 100% values within [$-\infty$, $+\infty$] will occur

To conclude, with QR-Net we demand it to output correct sub-vectors, knowing that each Q-score must have the same probability as others in that sub-vector.

Now, how does IQN work?

From what I understood after checking the paper, we have to feed the network the $\tau$ itself (in my example either 0.1, 0.3, 0.5, 0.7 or 0.9),

Or should we instead feed it several random $\tau$, pulled from [0,1] range?
Say, we pulled-out 0.35,  0.05,  0.1,  0.2,  0.11, should we continue getting them, or should we stop once we taken out 5 of those $\tau$ values? Would that mean we have to do a forward-pass for every $\tau$ or can we use a vector of them all, at once?

As I understood, we have to feed this $\tau$ as an input, into the network, along with the state vector (concatenating).

Does IQN output only 1 value per action, or a vector of vectors, like QR-Network does?

The IQN paper gives a picture, but I am unable to grasp the intuition behind the 4th image:

Why does the above image show the $f$ function being used in DQN and in IQN but not in C51 or QR-DQN?  Does that mean that the output is no longer a vector of vectors, but is just a vector, like in a usual DQN?

The gist of IQN is in the following paragraph, but I just can't grasp it fully. What's the intuition behind the cosine? I know it's a periodical function, why does it get computed 64 times with i=0 to i=64, with the same $\tau$ (for example $\tau=0.4$ etc)

Would like to thank Will Dabney and Georg Ostrovski (two of the paper's several authors),  and Massimiliano Tomassoli (Simple-Machine-Learning blog owner) for helping me understand IQN.

To understand this algorithm, you will need to know:

TD Learning
Value Functions of Reinforcement Learning
What a DQN is.
What C51 is
What a Cummulative Distribution Function is (CDF)
What an Inverse CDF is.

Just like C51 or QR-DQN, IQN allows us to estimate an entire distribution of possible rewards if we take some action $a$.  However, IQN doesn't output a distribution like those did.  Instead, it outputs a single sample every time you ask it. It outputs that red dot you see on the right, of the bottom image.

The final output of IQN is 1 scalar value per action. In other words, just a single sample per action.  We re-run IQN several times to get more samples for the "CurrentState".

IQN takes input at 2 different stages: 1)  and 2)

First, the IQN takes a current state S (which is a vector), transforms it into another vector V (for example, of dimension 10).
now, we take any random scalar value called $\tau$ (drawn from a uniform [0,1] range), and feed that scalar value into function $\phi(\tau)$.  It gives us a vector H,  which has the same dimension as V
then, vectors V and H get combined via concatenation. In fact, we perform multiplication instead of concatenation if there won't be many forward layers further on.
At the end of this forward-pass, our IQN outputs an |A| dimensional vector,  containing samples of the action-distributions we are trying to estimate. |A| is the other way to say the "number of actions". Notice: because IQN outputs a vector, and not a matrix (like QR-DQN does), we only get 1 scalar sample per action when performing a forward pass from the current State.
To get more samples, we have to return back to step 2)  and re-run the forward pass with some another scalar $\tau$  once again sampled from [0,1] range. Or, of course, we can re-use our IQN, to run the needed number of such forward-passes in parallel, if hardware allows. No need to start all the way from step 1) because we only care about the end-layers where $\tau$ actually enters the system. "Any preceding layers" would be re-computed to the same values anyway.

When training, we evaluate "CurrentState+1" as well. We bootstrap our results from "CurrentState" towards "CurrentState+1" via usual TD-learning, just like a usual DQN would do.  Don't forget, we will have to re-run forward pass several times (as step 5. has mentioned) both for "CurrentState" and for "CurrentState+1" to get several samples of the current and target distributions. This gives us a fairly good idea about what those two distributions look like.

According to the paper, in most cases re-running the forward pass 8 times (thus getting 8 samples) suffices for approximating both the $Z_{\tau}$ and our target distribution $Z$ (the distribution of the best action for "CurrState+1").  So we perform 16 forward passes in total:  8 for  "CurrentState"  and 8 for "CurrentState+1" to get an idea about those two distributions.

Why does feeding a random $\tau$ into the network work?

You have to understand that a well-trained IQN network represents an Inverse CDF itself.  That is, given a request of some scalar 'amount' value (taken from a uniform range of 0 to 1), the Inverse CDF will output a value from the actual distribution, that sits at that amount.

Although distributions won't necessarily look like Bell-curves, have a look at this example:

Why do authors use cosine inside $\phi(\tau)$ ?  I still have no idea, but the Appendix section of the paper states it seems to work the best. Perhaps it works good with ReLU.

I also got confused by $w_{ij}$ or $b_j$. To me it seemed there is a matrix of weights and a vector of bias values. ...Do we input an entire vector of different $\tau$ values at once?

The answer is no. Notice, authors used $j$ only to remind that forward pass has to run several times.
In total, there is only i number of weights in $\phi(\tau)$ and 1 bias.

Therefore, we get some scalar $\tau$, and use it with all those weights.  Basically $\phi(\tau)$ resembles a simple fully-connected layer, which only accept 1 scalar input value at a time.  We will re-use this same layer for different $\tau$ scalar values.


Gradient descent in a noisy environment How to know the right direction in a noisy environment?

In the typical example of neural network learning, we can see several local minima. The gradient descent is choosing one local minimum and moves in that direction and somehow it works.

I imagine that if there are lots of neurons, that there is a large space of possibilities.

I am electrical engineer, so I am used to encounter noise. I am also new to the topic of neural networks, so pardon me if this is a beginner's question.

I fear that the space of possibilities looks very noisy if I take too close a look. Gradient descent knows only the close look: You get the gradient at THIS micropoint.

Add noise, your derived vector probably points anywhere it wants.

How can I achieve a less fine grain approach, to find a global minimum in a noisy plane, then go down to a better resolution in order to find the local minimum inside the global minimum?

Yes, one probable solution would be to train the network on some points in the near neighbourhood and to estimate the noise from there plus make an average... but that takes a lot of trainings, and those are expensive.

Am I thinking too complicated here?

If I understand correctly, your question is whether if you cannot compute the gradient of the function exactly, but only within some errors, then gradient descent method still works in that it gives you convergence to a minimum point?

The answer to this is Yes, if you use Backtracking gradient descent. (And not the standard version of gradient descent, that version works only if you have some strong assumptions on the cost function, such as being convex or having Lipschitz continuous gradients, together with some other assumptions such as the function has compact sub level sets. The point is that if you consider a regularisation technique, as mentioned in the answer by Victor, then even your original function has Lipschitz continuous gradients and has compact sub level sets, after you do regularisation, the new function won't satisfy these assumptions any more.) You can find more details in the paper I mention in my answer in the following link:

Does gradient descent always converge to an optimum?


Cluster evolution over time I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but since customers have very different behaviours (roughly 50% of the time a customer will change cluster the week after), I was wondering what would be a statistically sound approach. Is it a good idea to train a clustering algorithm every week and look backwards at the weekly evolution of each segment?

You can try

Dynamic mode decomposition.
Dynamic Time Warping. Found a nice resource on Towards data science blog.

These two have proven better approaches than PCA for time series clustering.
Happy coding


Learning resources for data science to win political campaigns? Does anyone know, where I can learn about applying data science to win a political campaign? I know the Obama campaign had 12 data scientists in 2008 and 165 data scientists in 2012. In 2012, they ran over 65,000 simulations every night, for 14 months. They correctly predicted every state within 0.5% and Florida within 0.05%. How did they do this? And where can I find the data they used?

This is an interesting and relevant question. I think that from data science perspective, it should not be, in principle, any different from any other similar data science tasks, such as prediction, forecasting or other analyses. Similarly to any data science work, the quality of applying data science to politics very much depends on understanding not only data science approaches, methods and tools, but, first and foremost, the domain being analyzed, that is politics domain.

Rapidly rising popularity of data science and machine learning (ML), in general, certainly has a significant impact on particular verticals and politics is not an exception. This impact can be seen not only in increased research interest in applying data science and ML to political science (for example, see this presentation, this paper, this overview paper and this whole virtual/open issue in a prominent Oxford journal), but in practical applications. Moreover, a new term - political informatics or poliInformatics or poli-informatics - has been coined to name an interdisciplinary field, which stated goal is to study and use data science, big data and ML in the government and politics domains. As I've said earlier, the interest in applying data science to politics goes beyond research and often results in politics-focused startups, such as PoliticIt or Para Bellum Labs. Following the unfortunate, but established trend in startup ecosystem, many of those ventures fail. For example, read the story of one of such startups.

I am pretty sure that you will be able to find neither proprietary algorithms that political startups or election data science teams used and use, nor the their data sets. However, I am rather positive that you can get some understanding about typical data sets as well as data collection and analysis methods via the resources that I have referenced above. Hope this helps.


MSE to predict Values 0 and 1 I am building a deep neural network to predict values 0 and 1 .My training data contains class labels 0 and 1 .Currently i am getting a cross validation loss around .25 . How good is the model performing?

Predicting $0.5$ for all items in your case would also give you MSE of $0.25$. That is because independently of whether the true label is $0$ or $1$, the squared error for each example will be $0.5^2 = 0.25$

Your model is performing badly under an MSE measure, when such a simple model that does not take input data into account can get the same score.

However, it is debatable whether MSE gives you a useful metric here.


Creating validation data for model comparison I am working on building a scoring algorithm for student data,

say the attributes are :

name, location, age, class, school_name, skill1, skill2, skill3

based on these data I need to create a student score.

I need to assign weight-ages for age, class, school_name skills and come up with a score for student.

say I have 2 scoring models like :

score_1 = x1*location_weight + x2*age_weight + x2*class_weight + x3*school_name_weight + x4*skill1_weight + x5*skill2_weight + x6*skill3_weight

score_2 = y1*location_weight + y2*age_weight + y2*class_weight + y3*school_name_weight + y4*skill1_weight + y5*skill2_weight + y6*skill3_weight

now how can I compare these models and evaluate them?

The problem is I don't have a test or validation set to prove or compare how accurate each of these model is, so in this case what is the best approach to compare and validate different models? also what is the best ways to build a validation set from scratch?

Predicting and scoring are two different tasks.

And according to your answers and comments you are not solving prediction problem. You just want to set to each student a number in range [1,100] according to some rule. This is ranking (or scoring, whatever).

Therefore, the terms #prediction_model, #accuracy, #validation, #training_set are out of this scope. You don't need to validate anything. You are not making predictions.

What you want is to map ranks to students.

But a problem is that you have mostely categorical data (school name, location etc) that cannot be 'ranked'. Some of them are useless at all: how does the student name refer to his school progress? :)

If you change it somehow to numerical (e.g. 'Skill_1_level', 'Skill_2_level', 'remoteness_of_location', 'school rank' etc) than you can do some ranking:

Normalize data: each of your factors

Multiple by 100, as you want [0,100] range instead of [0,1]
Set up weights based on your experience according to factor's importance. So that the sum of weights is 1.
And finally build a rank (score):

Rank = 0.1 * skill_1_level + 0.2 * skill_2_level + 0.05 * remoteness_of_location + 0.5 * school_rank + ...


First layer weights for transfer learning with new input tensor in keras.applications models? In the pre-implemented models in keras (VGG16 ect) it is specified that we can change shape of the inputs of the models and still load the pre-trained imagenet weights.

What I am confused about is then what happens to the first layer weights? If the input tensor  has a different shape, then the number of weights will be different than for the pre-trained models isn't it?

Here is the implementation of the Keras VGG16 model for reference.

The first layers are convolution and pooling ones:

For the convolutional layers, the only weights are the kernels and the biases, and they have fixed size (e.g. 3x3x3, 5x5x3) and do not depend on the input tensor shape.
The pooling layers do not have weights at all.

That's why you can reuse the weights independently from the input tensor shape.

With dense layers (i.e. the final layers), you need shapes to match, so you cannot reuse them if they do not.


XGBoost outputs tend towards the extremes I am currently using XGBoost for risk prediction, it seems to be doing a good job in the binary classification department but the probability outputs are way off, i.e., changing the value of a feature in an observation by a very small amount can make the probability output jump from 0.5 to 0.99.

I barely see outputs in the 0.6-0.8 range. In all cases, the probability is less than 0.99 or 1.

I am aware of post training calibration methods such as Platt Scaling and Logistic Correction, but I was wondering if there is anything I can tweak in the XGBoost training process.

I call XGBoost from different languages using FFI, so it would be nice if I can fix this issue without introducing other calibration libraries, e.g., changing eval metric from AUC to log loss.

The first I'd ask would be "What is positive/negative ratios?". Just because I had a similar issue multiple times when classes were very imbalanced. If it's your case you can try to balance dataset or try to play with scale_pos_weight parameter of XGboost. In the case of 50/50 classes ratios, probablities might be more or less normalised.

Also, it's very likely you have overfitting at the same time. Try to tune learning rate, sampling parameters and regularization parameters (reg_alpha, reg_lambda).


Feature Extraction Technique - Summarizing a Sequence of Data I often am building a model (classification or regression) where I have some predictor variables that are sequences and I have been trying to find technique recommendations for summarizing them in the best way possible for inclusion as predictors in the model.

As a concrete example, say a model is being built to predict if a customer will leave the company in the next 90 days (anytime between t and t+90; thus a binary outcome). One of the predictors available is the level of the customers financial balance for periods t_0 to t-1. Maybe this represents monthly observations for the prior 12 months (i.e. 12 measurements).

I am looking for ways to construct features from this series. I use descriptives of each customers series such as the mean, high, low, std dev., fit a OLS regression to get the trend. Are their other methods of calculating features? Other measures of change or volatility?

ADD:

As mentioned in a response below, I also considered (but forgot to add here) using Dynamic Time Warping (DTW) and then hierarchical clustering on the resulting distance matrix - creating some number of clusters and then using the cluster membership as a feature. Scoring test data would likely have to follow a process where the DTW was done on new cases and the cluster centroids - matching the new data series to their closest centroids...

would LOVE to see a box written which collected case studies on feature engineering / extraction

Please advise if this helps

Discretization of Time Series Data
http://arxiv.org/ftp/q-bio/papers/0505/0505028.pdf
Optimizing Time Series Discretization for Knowledge Discovery
https://www.uni-marburg.de/fb12/datenbionik/pdf/pubs/2005/moerchen05optimizing
Experiencing SAX: a Novel Symbolic Representation of Time Series
http://cs.gmu.edu/~jessica/SAX_DAMI_preprint.pdf
Indexing for Interactive Exploration of Big Data Series
http://acs.ict.ac.cn/storage/slides/Indexing_for_Interactive_Exploration_of_Big_Data_Series.pdf
Generalized Feature Extraction for Structural Pattern Recognition in
Time-series Data
http://www.semanticscholar.org/paper/Generalized-Feature-Extraction-for-Structural-Olszewski-Maxion/7838bcd87bb6616e9fd3ffd92d4676a7082da34c
Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package
https://cran.r-project.org/web/packages/dtw/vignettes/dtw.pdf


Generating synonyms or similar words from multiples word embeddings I am looking for a way to generate synonyms, using word embeddings. From one word, and from multiple words. Such as the two example below:

"word" -> Word embedding -> generate synonym of "word"

"word", "synonym of word .. "->  -> Word embeddings -> generate a synonym of both word

I am very new at this. What do you think I should use ?

I also want to use a tools that, in further work, would take into account context for word embedding generation, such as:

"sentence in including a word" -> Word embedding of word -> generate synonym of "word" in that context

I think I will start to do it with BERT... How should I start ? or which alternative should I use ?

Thanks for your help !

An automatic system for finding synonyms using word embeddings is not possible. Word embeddings find co-occurrence. For example, "good" and "bad" co-occur together in a corpus, thus are near each other in the embedding space. However, "good" and "bad" are antonyms.

A copilot system could work. Word embeddings can find a set of candidate synonyms by finding the nearest words as measured by a distance metric. Then other methods (e.g., a person) can select amongst the candidates.


Similarity between two scatter plots I would like to know if there is a metric used to compute the similarity between two scatter plots?

The simplest method is to calculate the euclidean distance between the baricenters of the two distributions; This do not take in account of the variance between the distributions, however.

If you want something more accurate, you can add to that distance the spread of the distributions; between the others, you can calculate the spread as the standard deviation of the distances between the baricenter (already calculated) and each of the points.

You can also think of using a join metric of the first two (baricenter distance and spread distance).

An example in Python. Take the three distributions in the image:

As you can see, distribution A and B are centered around the same point, but they have different spreads. Distribution A and C have different centers, but their spreads are more similar.

Here the code to calculate the distances I described. The more the distances are small, the more the distributions are similar.

#create three dummy distributions
dist_a=[]
dist_b=[]
dist_c=[]
for i in range (100):
    dist_a.append(np.random.randn(2)+10)
    dist_c.append(np.random.randn(2)+25)
    dist_b.append(np.random.randn(2)*5.5+10)

plt.scatter([a for a, _ in dist_a], [b for _, b in dist_a], label='distribution a')
plt.scatter([a for a, _ in dist_b], [b for _, b in dist_b], label='distribution b')
plt.scatter([a for a, _ in dist_c], [b for _, b in dist_c], label='distribution c')

plt.legend()

#calculate baricenters
bc_a=np.mean(dist_a, axis=0)
bc_b=np.mean(dist_b, axis=0)
bc_c=np.mean(dist_c, axis=0)

#calculate the distance between baricenters
dist_a_b=np.linalg.norm(bc_a-bc_b)
dist_a_c=np.linalg.norm(bc_a-bc_c)
dist_b_c=np.linalg.norm(bc_b-bc_c)

print("baricenter distante between distribution A and distribution B=", dist_a_b)
print("baricenter distante between distribution A and distribution C=", dist_a_c )
print("baricenter distante between distribution B and distribution C=", dist_b_c )
print ("\n")

#calculate the spread of the distributions, e.g. their standard deviation
spread_a=np.std(dist_a)
spread_b=np.std(dist_b)
spread_c=np.std(dist_c)

dist_spread_a_b=np.abs(spread_a-spread_b)
dist_spread_a_c=np.abs(spread_a-spread_c)
dist_spread_b_c=np.abs(spread_b-spread_c)

print("spread distance between distribution A and distribution B=", dist_spread_a_b)
print("spread distance between distribution A and distribution C=", dist_spread_a_c)
print("spread distance between of distribution B and distribution C=", dist_spread_b_c)
print ("\n")

#put in a single metric. NB, the paramenter of this join is subjective, and depend on the usecase
#alpha=0 : don't care about the euclidean distance between the baricenters
#alpha=1 : don't care about the spread distance between the baricenters

alpha=0.3
joint_metric_a_b=alpha*dist_a_b + (1-alpha)*dist_spread_a_b
joint_metric_a_c=alpha*dist_a_c + (1-alpha)*dist_spread_a_c
joint_metric_b_c=alpha*dist_b_c + (1-alpha)*dist_spread_b_c

print("joined metric distance between distribution A and distribution B=", joint_metric_a_b)
print("joined metric distance between distribution A and distribution C=", joint_metric_a_c)
print("joined metric distance between distribution B and distribution C=", joint_metric_b_c)

Outputs:

baricenter distante between distribution A and distribution B= 0.22454217332627005
baricenter distante between distribution A and distribution C= 21.028862497007008
baricenter distante between distribution B and distribution C= 20.98580645790957

spread distance between distribution A and distribution B= 4.153324630270008
spread distance between distribution A and distribution C= 0.004700454831506384
spread distance between of distribution B and distribution C= 4.158025085101515

joined metric distance between distribution A and distribution B= 2.974689893186887
joined metric distance between distribution A and distribution C= 6.311949067484157
joined metric distance between distribution B and distribution C= 9.206359496943932

Following the first metric (euclidean distance between baricenters), distribution A and B are more similar.

Following the second metric (spread distance), distribution A and C are more similar.

The third metric is tunable, by the parameter alpha, which accept values between 0 and 1. Depending on your usecase, you can be more interested that the distributions lie around the same point, and so you care more on the distance between their baricenters, or that the distributions have the same spread, even if their baricenters are slightly displaced. So, you have to adapt the paramenter alpha to your case.


Dueling DQN - why should we decompose and then combine them back into? could anyone who can help me if we decompose them and combine back them into a single Q, what the network can learn?
from my perspective，the V means the total reward when the agent follow the current policy;the Q means if we give a specific action then follow the current policy what the total reward;
and if we get the optimal policy，the V will equal to Q;so we should learn to make the A reach zero;just like the answer ：Dueling DQN - can't understand its mechanism

but in that paper，i cannot understand what is the matter if we cannot identify the sense that given Q we cannot recover V and A uniqueluy.

and it will get this blow finally：

and as well as this：

refer to this blog:https://medium.com/@awjuliani/simple-reinforcement-learning-with-tensorflow-part-4-deep-q-networks-and-beyond-8438a3e2b8df

The Dueling DQN is a good strategy to avoid your network be overoptimistic. Otherwise it will cause actual DQN and to get as optimistic as it explodes. So the Q-values become larger and larger over time on some games and sometimes they never get back.
https://arxiv.org/pdf/1509.06461.pdf
A couple words about Double Q-Learning
$$Q_1^{\hat{}}(s_t,a_t) = r_t + \gamma*Q_2(s_{t+1}, argmaxQ_1(s_{t+1}, a^*))$$
$$Q_2^{\hat{}}(s_t,a_t) = r_t + \gamma*Q_1(s_{t+1}, argmaxQ_1(s_{t+1}, a^*))$$
If one Q function is not enough to make an accurate approximation, Double Q-learning suggest to learn two networks and train one another. So you implement Q1 and Q2 - two independent estimates of the actual value function. Q1 and Q2 in a DQN case are networks with a different set of weights.
If take an action which is optimal in the first Q function, there won't be any connection between this overoptimism for selected action in Q1, and the same overoptimism in Q2. In fact, the same action in Q2, is more or less independent. It can be overoptimistic or overpessimistic and it can be exactly the true value.
The idea here is that, the noise in Q2 is independent of the noise in Q1. And if you update them that way, then you take the maximization which will take account of the sampling error.
Since they two networks are more or less decorrelated, they have different kinds of noises, then the overoptimism disappears.
The drawback of this method - training two networks separately will double the convergence time.
But a DQN algorithm that can serve as an alternative source of q-values, aside from the main q-network - we take the older snapshot of your network as the source of independent randomness as the other Q network
$$Q^{\hat{}}(s_t,a_t) = r_t + \gamma*Q_{old}(s_{t+1}, argmaxQ(s_{t+1}, a^*))$$
Another huge problem is that another huge problem with DQN is that it actually tries to approximate a set of values that are very interrelated. And double Q-learning can't solve this problem easily.
Dueling DQN
The Dueling DQN  can solve the overoptimistic problem as well
$$ Q(s,a) = V(s) - A(s,a)$$
The intuition behind the advantage is how much you're action value differs from the state value.
For example, if you have a state in which you have two actions, the first brings you returns of + 100 and the second is + 1. In this case, both action values are positive because you get positive reward. The advantages on the contrary are different. If you take this suboptimal action, then you get the action value of (+ 1) minus the state value of (+ 100), which is -99. Basically, it tells you that you have just lost 99 potential units of rewards in this case.
To predict advantages, we have to constrain them. In case of $V^*$, the maximum possible advantage value is zero because you can never get action value which is larger than maximum over all possible action values from the state.
So you you train two halves together and then just add them up to get your action values.


Expand or compact features? I have a classification task for people with 3 categories. I want to apply machine learning for that. I have 10 sources of data, which have the same fields (say 4: age, job title, a number of organizations, a number of followers). Data is incomplete, some fields can be missing in some profiles. The training set is limited (say, 300 examples).

I have two strategies for feature engineering, and I don't know which one to use.

Expand features: take 40 features (Profile 1 age, Profile 1 job title, ..., Profile 10 age, Profile 10 job title).
Compact features: take 4 features, and apply some heuristics to merge the values from different profiles. Say, take age and job title which occur most frequently, take a maximum number of organizations, take a sum of numbers of followers.

What strategy is generally used to give best results and why?

The way I see it is that your 10 sources of data, they all refer to the same set of people. Depending on the attributes, some can be expanded, some can be merged ...

Attributes such as age should be unique, so it doesn't make sense to expand it to Profile 1 age, profile 2 age ... One simple way is merge them is by using the average or use max. Expanding age only add redundant data to your feature matrix, and increase its dimensionality, in most cases, this doesn't help generalization performance of your model.

On the other hand, number of followers can be expanded. Depending on the data source, a guy has 10 followers on Twitter but 1000 followers on Google+ might simply mean that he barely uses Twitter.

That being said, the way you pick your features or engineer new features should increase your model performance, so if expanding number of followers actually decrease Cross Validation or Test performance, compared to the one using sum of followers then you can simply use sum of followers.


Can embeddings generated by word2vec be similar for words which never share any words in the same sentence? Is it possible for word2vec to produce similar embedding vectors for two words which never share any common words in the sentences that the words are found in?

Specifically, imagine I have the words A and B. Next imagine that I have words(A) and words(B), which represent the set of all words which show up in the same sentences as A and B, respectively. If the intersection of words(A) and words(B) is the null set (meaning the two words never have common words), then would it be possible for word2vec to put the embedding vectors for A and B in similar regions of the vectorspace?

As you send a bag of words into the CBOW as an input, and it works based on the n-grams, as in your case words A and B do not share any co-occurrences with each other, it means their not presented in any shared n-grams, and their vectors should not be near each other at all.


Deep learning test loss curve won't go down I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to classify the customers as being either honest or dishonest.
The current model that I have now only uses CNNs, but I'm planning on expanding it with LSTMs or other models in the future. Here is the code for my model.
model = Sequential([
    Input(batch_input_shape = (None, 1036, 1)),

    Conv1D(
        filters=32,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    Conv1D(
        filters=16,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    MaxPooling1D(),

    Conv1D(
        filters=8,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    Conv1D(
        filters=8,
        kernel_size=3,
        padding='same',
        activation='relu',
        activity_regularizer=l2(5e-4),
    ),
    MaxPooling1D(),
    Flatten(),

    Dense(10, activation='relu'),
    Dropout(0.25),
    Dense(1, activation='sigmoid'),
])

model.compile(
    # optimizer=tf.keras.optimizers.Adam(learning_rate=1e-4),
    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=[f1_m,precision_m, recall_m, matthews_correlation, 'accuracy', fpr_m]
)

After training the model for 100 EPOCHs the loss curve is this

I've tried a number of things which are:

Reducing the network size. This still results in the same issue but at a different point in the network
Reducing the learning rate of the optimizer. Same with the point above. This seems to work but it is in fact changing when this happens.

I am open to any input or recommendations I could get on how to make the testing curve follow the training curve more. I should note that my dataset is imbalanced and I'm ONLY balancing the x_train and y_train using SMOTE and not making and balancing to the testing and validation datasets to keep the data as clean as possible.

Your learning curves suggest overfitting.
In this respect you should try

increasing dropout
adjust l1/l2 regularisation to higher levels
inspect class ratios in training and validation tests ("learning on easy predicting on difficult" imbalances)

Additionally, you may want to

add batchnorm layer
try different activations
check to what extend weights/bias of each layer get updated during training iterations, in case of vahishing/exploding gradients