how is correct usage of the validation split in neural networks? I have a dataset separated in train, test and validation splits.

After each epoch, I evaluate the loss and accuracy in the validation split.

When the loss in validation split is not better, I stop train and choose that as final model.

But, I should merge train and validation as final model? How can I choose the best model?

I really like this post with the answer on your question.

Once we have the estimated skill, we are finished with the resampling method.
You finalize a model by applying the chosen machine learning procedure on all of your data.

It means that after achieving enough performance of the model, we don't need anymore splitting ("resampling methods"). Then we can merge all available data and repeat learning procedure. We can rely on the final performance (have no qualms about, who model will behave using new data during training), because it has already learned how to generalized input for the particular task (differ cats from dogs for example).

The careful design of your test harness is so absolutely critical in applied machine learning. A more robust test harness will allow you to lean on the estimated performance all the more.


Hashing trick with random forest in scala I am trying to perform a hashing trick and then a random forest with scala. I have the following code:

val documents: RDD[Seq[String]] = sc.textFile("hdfs:///tmp/new_cromosoma12v2.csv").map(_.split(",").toSeq)

val hashingTF = new HashingTF()
val tf: RDD[Vector] = hashingTF.transform(documents)

val splits = tf.randomSplit(Array(0.7, 0.3))
val (trainingData, testData) = (splits(0), splits(1))

val numClasses = 3
val categoricalFeaturesInfo = Map[Int, Int]()
val numTrees = 10
val featureSubsetStrategy = "auto"
val impurity = "gini"
val maxDepth = 8
val maxBins = 32

**val trainingData2=LabeledPoint(1.0,trainingData.collect())**

val model = RandomForest.trainClassifier(trainingData2, numClasses, categoricalFeaturesInfo, numTrees, featureSubsetStrategy, impurity, maxDepth, maxBins)

But I have the error

found   : Array[org.apache.spark.mllib.linalg.Vector]
 required: org.apache.spark.mllib.linalg.Vector

in the bold line.

Do you know how I can solve it?

Thank you,

Laia

Let's look at the error message:

found : Array[org.apache.spark.mllib.linalg.Vector] required: org.apache.spark.mllib.linalg.Vector

"found" is the type of object that it came across, "required" is the type of the object that the function accepts. The types look mostly the same ( org.apache.spark.mllib.linalg.Vector ) but the first is Array[X] and the second is X.

Why? Because trainingData.collect() gives you the RDD as an array of objects in the array.

What you probably want to do instead is map, to turn each vector into a LabeledPoint, instead of trying to turn the whole RDD into a single LabeledPoint.


use hidden layer of word2vec instead of 'one-hot', to reduce number of weights in other nets? I've been reading about word2vec and it's ability to encode words into vector representations. Coordinates (probabilities) of these words are clustered together with their usual context-neighbor words.

For example, if we have 10k unique words in our dataset, we will feed 10 000 features into the network. The network can have for example 300 hidden neurons, each with linear activation function.

The output of the net are 10 000 neurons, each with a softmax classifier. Every such a "soft-maxed output" represents probability to select the appropriate word from our 10k dictionary.

Question 1: Well, I was able to get benefit of "semantically grouped" words via that final softmaxed-output layer. However, it seems way too large - it still has dimension of 10 000 probabilities.

Can I actually use that 300 neurons result in some other networks of mine? Say, in my LSTM, etc?

Then weights of my LSTM wouldn't take so much space on disk. However, I would need to decode the 300-dimensional state back into my 10k-dimensional state, so I can look it up

Question 2: ...are the "Encoded" vectors actually already sitting in the Hidden layer or in the Output Layer?

References:
first, second, third

Are you asking if the 300 dimension 'middle layer' has learned to encode words into an embedding like word2vec?

If this is the case, I'd say no. There's nothing in the one-hot vector that specifies distance between two words. For instance, the one-hot vectors for cat and table will be the same distance away for cat and lion. With embeddings, clearly you want to have the second example closer together.


How many images can be trained in Google Colab? I am using the ResNet50 pretrained model to train my images using TensorFlow. I have 70k images and upgraded to Google Colab Pro, but still I am facing a memory error. So how many images I can train in Google Colab? And how much RAM is needed for 70k images?
This is how I labeled and loaded images from the drive.
labels = []

imagePaths_generater = paths.list_images(Config.DATASET_PATH)
imagePaths = []
for item in imagePaths_generater:
  imagePaths.append(item)

for imagePath in imagePaths:
  label = imagePath.split(os.path.sep)[-2].split("_")
  #print(label)
  image = load_img(imagePath, target_size=(224, 224))
  image = img_to_array(image)
  image = preprocess_input(image)
  data.append(image)
  labels.append(label)

data = np.array(data, dtype="float")
labels = np.array(labels)```

The real problem is that you should not try to fit all your images in memory. Instead, you should small groups of images, normally called "minibatches", which can fit in the GPU/CPU memory.
For that, tensorflow offers the function tf.keras.preprocessing.image_dataset_from_directory that loads images from a directory. I suggest you take a look at Tensorflow's tutorial for image loading, which guides you to the image loading process, as well as the training of a model with those images.


Orange Geo Map does not show background map We are running Orange with our students and most of them are having problems with running the Geo Map widget. It does not show any background map. In the bottom an error message is displayed: "Cannot fetch map from the Internet. Displaying only cached parts."
I have found this Github page, but it does not explain how to update or add any JSON files that might be missing:
https://github.com/biolab/orange3-geo/blob/master/CONTRIBUTING.md
What do we have to do to get the background maps to appear again in the Geo Map widget?

The issue with the Geo Map widget has been solved in a new release of Orange3. The Geo Map widget now works correctly again in Orange3 version 3.23.1 and Geo Add-on version 0.26. Great!


Getting negative r2_score with new set of dimensions I am trying to predict flight take off delay using my current dataset. At this point of time, I only have four dimensions.

scheduled_time_departure (A), flight_id, day_of_week actual_time_departure(B), take_off_delay(B-A)

when I try to predict actual_time_departure(B) using scikit linear regression model using (scheduled_time_departure (A), flight_id, day_of_week) on x-axis, I am getting good r2_score
However when I am trying to predict take_off_delay (which is actually difference between actual_time_departure and scheduled_time_departure), in that case I am getting negative r2_score.

Note:

to convert string to integer I am using LabelEncoder.
scheduled_time_departure and actual_time_departure are in seconds not a timestamp, that is second of day, 86400 is max value it can have.
I even tried doing normalization of scheduled_time_departure
I have ensured that take_off_delay is always positive.
For the case when I was predicting actual_time_departure, I tried using one hot encoder but that agravated r2_score

P.S: I am new to machine learning and data science, let me know if I am making dumb mistake :)
P.P.S: I understand model can be worst if r2 score is negative, however I want to understand the reason.

A negative $R^2$ on your training set typically means you didn't fit an intercept. A negative $R^2$ on your test set means the model is simply very bad. Indeed flight ID is not a meaningful scalar value and can't be in a linear regression.


Testing if a sample fits into an existing cluster I have a sample of data I'd like to create a model from, which would create N clusters.
After the fitting to clusters, I'd like to test various samples against the existing clusters, seeing if the samples fit any of the clusters, and if not see it as an "anomaly".
I started working on it using DBScan (scikit), my data splits into 3 clusters but I can't find any function/method to test the specific sample against current results, without "refitting".
Is there any other model that would fit my needs? or am I missing something when using DBScan?

Thanks in advance

The way to do anomaly detection with clustering is to compute the distance to each the fitted clusters. If a new sample has a distance above a certain threshold for all clusters, then it is considered an anomaly.

For KMeans the distance would be the distance to the center of the cluster. And the threshold to use is a new hyperparameter.

For DBSCAN one would typically compute the distance to each Core point in a cluster, and find the smallest distance. DBSCAN already has a distance threshold eps used during fitting. But to allow tuning sensitivity of the anomaly detection without redoing the clustering, you may want to use a threshold of eps*ratio, where ratio is a new hyperparamter.


In which epoch should i stop the training to avoid overfitting I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and the validation sets is shown in the graph below:

As you can see the validation accuracy keeps rising with smaller steps than the training accuracy. Should i stop training at the epoch 280 in which the training and the validation accuracy have the same value or should i proceed the training process as long as the validation accuracy is rising, even thought the training accuracy value is also getting at overfitted values (eg. 93%).

As long as your validation accuracy increases, you should keep training. I would stop when the test accuracy starts decreasing (this is known as early stopping). The general advise is always to keep the model that performs the best in your validation set.
Although it is right that your model overfits a little since epoch 280, it is not necessarily a bad thing provided that your validation accuracy is high. In general, most machine learning models will have higher training accuracy compared to validation accuracy, but this doesn't have to be bad.
In a general case, you expect your accuracy to behave in the following way.

In your case, you're before the early stopping epoch, so even if your training set accuracy is higher than your test set accuracy, it is not necessarily an issue.


How can I go about building a model for large number of outputs? I have previously worked on small-scale feedforward neural network problems.

But I have started working on a new project where the goal is to predict air quality in 25 locations throughout the country a day ahead. Now, I am quite well-versed with the air quality side of things.

My question:

In a problem like this, would I develop 25 independent models (which share the same input structure) or one model with 25 outputs.

I guess what I want to do - is there something like parallel neural networks? Or is this 25 different problems? I have mostly worked on physical models where the physics would be shared by all 25 locations. And the inputs would be different.

Would this be a data parallelism or a model parallelism problem?

Neural networks can have 25 outputs.  You would probably get a slightly more accurate result with 25 independent models, but the computation and training time would be 25x of one independent model.  One model with 25 outputs would only take slightly longer than one independent model.


Loss function range normalization This is from a referee report in a conference to which I submitted my paper - I don't quite get it and I'm not sure what I need to do about it.

I use Euclidean loss and Softmax cross-entropy (pixelwise) loss in my model. The referee pointed out that the losses are not in the same range and they are not weighted.

I've never seen that done in other models (e.g. Faster R-CNN, Mask R-CNN). Any suggestions how to address this issue?

I agree with the referee. When your loss is composed from different terms, you should add a mechanism that enable you to adjust the contribution off those terms.
Just create a new hyperparameter (usually called lambda) and use it like that:
$L = L_1 + \lambda L_2$
You have to find the right lambda with cross-validation.
This is used in most of the papers.
For example : https://arxiv.org/pdf/1806.08462v1.pdf page 3


What is the intuition behind using 2 consecutive convolutional filters in a Convolutional Neural Network? I understand the purpose of Convolutional filters (or kernels). I visualize them as learnable feature extractors. E.g. Extract vertical edges or horizontal edges, etc.

Could somebody kindly explain to me the intuition behind stacking 2 or more consecutive convolution filters? Why couldn't the 2 filters be merged into 1?
Picture from Andrew Ng's video

Video URL
Here is the link to Andrew Ng's lecture on Machine learning as a reference

Could somebody kindly explain to me the intuition behind stacking 2 or more consecutive convolution filters. Why couldn't the 2 filters be merged into 1?

No, when having two consecutive convolution layers can't be combined into one. The subsequent filter's inputs are the features extracted from the previous one. This results in the second layer's features are of higher-level than the previous.

This is the basis of the whole CNN. Having multiple convolutional layers stacked along the depth of the network, allows the network to extract high-level features (not just edges and corners) from the input images.

Edit.

The first convolutional layer of a CNN is essentially a standard image filter (+ a ReLU). Its goal is to take a raw image and extract basic features from it (e.g. edges, corners). These are referred to as low-level features.

The second convolutional layer, instead of the raw image, accepts the features extracted by the first as its input. This allows it to combine these basic shapes into more complex features.

The features extracted become more and more complex as we go further down the network. Layers near the middle of the network extract the so called mid-level features, while the final layers extract high-level features.

CNNs are powerful tools because it is trained to extract the best features for each task. This results in the network extracting different features for different tasks.

For example, take the image below:

While the low-level features are the same for each task, mid and high-level features differ greatly. For instance, for the first CNN that recognizes faces:

The low-level features are basic geometric shapes (edges, corners, circles, etc.).
The mid-level features combine the previously extracted shapes to extract more complex features (eyes, noses, lips), specific to the task of facial recognition.
Finally, the high-level features combine the previous to extract even more complex features (in this case different faces).


Cocktail party problem ICA I'm trying to solve the cocktail party problem (Independent Component Analysis) in a real application. So let's say there are two speakers and two microphones. What I'd to know is if it's mandatory that the two microphones start to record exactly at the same time.
Thanks for your time.

It's not exactly mandatory to start recording at the same time, but it would be helpful.

You definitely need to be able to align the timeseries recorded by the microphones.  If they start recording at exactly the same time, then the time-series will already be aligned.  If they don't start at the same time, then you need to know exactly how much time passed between microphone 1's start time and microphone 2's start time, so you can align the timeseries after-the-fact.  Also, you won't be able to run ICA for the time period when microphone 2 was off.


Data augmentation: ImageDataGenerator vs openCV I would like to increase the data in my dataset to create a CNN deep learning classification model.

Which is better for the model, using data augmentation by ImageDataGenerator or using openCV to increase the data?

By the way, I am using Keras and floydhub.

Keras' ImageDataGenerator doesn't offer much support by itself for data augmentation. However it has a parameter called preprocessing_function which allows you to use custom augmentors with it.

I personally use imgaug which offers virtually any augmentation you can think of and works well with ImageDataGenerator like I said.


Encoding Categorical Data Without Increasing the Dimension I've been exploring methods for encoding categorical data. I was hoping to find a good method that does not increase the dimension of the dataset, similar to the one used on this dataset about drug use: Drug consumption (quantified) Data Set

Each piece of categorical data in this dataset was converted to some real number, but yet the dimension of the dataset was not increased. Instead of just randomly replacing values with numbers, there appears to be some thought out method behind this. Can anyone shed some light on this matter?

I would strongly advise against doing anything like this: a features which is semantically discrete should be typed as such. There's nothing to be gained about casting categorical values into real numbers:

It obfuscates the meaning to a human analyzer
For categorical variables which are not ordinal (i.e. have no natural order), it introduces a serious bias for learning algorithms (note: most attributes in the dataset mentioned in the question are ordinal).
Real number binary representation can lead to approximations, thus possible errors when used as labels and compared for strict equality (something that a categorical variable should support)
It becomes impossible to fix errors in the data

Any conversion in the data should always rely on reasons which are specific to the problem/method to be used, not on some agnostic technical transformation. There's no universal recipe for encoding features (categorical or other), one has to understand what they represent and how an algorithm could use them in order to determine the best representation.


MemoryError for np.array I was trying the Keras CNN Stater Code on Ubuntu 16.04, from the below link:
https://www.hackerearth.com/challenge/competitive/deep-learning-3/machine-learning/predict-the-energy-used-612632a9/#c144537

I get “MemoryError:” for

X_train = np.array(train_img, np.float32) / 255.

Any idea, what should I be doing?

MemoryError is exactly what it means, you have run out of memory in your RAM for your code to execute.

When this error occurs it is likely because you have loaded the entire data into memory. For large datasets you will want to use batch processing. Instead of loading your entire dataset into memory you should keep your data in your hard drive and access it in batches. If you are using Keras there is a helper class with a very efficient implementation of batch processing. Take a look at this blog post. This is a good starting point for avoiding MemoryError.

As a short term fix you can train your model using a subset of the data available to you and discard the rest. Doing this really is a shame however.


Prediction for not completely well classified data I have a DataFrame of users, some of them are "bots" and they are identified with a bit equal to 1 in the "is_bot" column, if the bit is 0, the user is considered as "human".

The problem is that some users may be misclassified as "humans" instead of "bots" since the "bots" have been identified on the basis on an incomplete list in the gathering data phase.

I will train and test my model on this partially correct data, but when I test it, I will find that my model correctly predicts some users as "bots" even if in the original dataset they are "humans".

Correctly predicts means that, in reality, the users are bots because I checked some of them manually, but I can't do this for my entire dataset of 1 mln users.

This would result in a model with low accuracy, even if the predictions may be right.

How do I handle this problem?

You are dealing with noisy labels. I would not switch the labelling according to a trained model that learned on that particular data set, since probably you don't know which patterns lead to your models decision.

Otherwise if you know the reason for the wrong labelling, you could try to build methods yourself that run a sanity check on your data.

Nonetheless, models that don't overfit often can deal with a bit of noise in the labels. But you can also try methods that incorporate the noise aspect. Maybe check out https://stats.stackexchange.com/questions/218656/classification-with-noisy-labels


Outlier Detection using K-Means using one column I have done and read a csv file and then plotted the values of a single column using K-means

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import style
style.use("ggplot")
from sklearn.cluster import KMeans

data=pd.read_csv(r'Plot_file.csv', encoding='unicode_escape', sep=';')
data.head()

feature_names = ['Plot_Column]
X = np.asarray(data[feature_names])

from sklearn.cluster import KMeans

labels = KMeans(5, random_state=10).fit_predict(X)
plt.scatter(X[:, 0], X[:, 0], c=labels,
    s=50, cmap='rainbow');

The output looks like this, it is linear because when clustering one column it can only look at the relative distance between the values in that column and will always be linear on any chart as it is only clustering one dimension

How would I go with detecting anomalies in this case?

The column that I am clustering the values from has around 12 thousand rows and varying numbers.

If you have a one dim data, why do you need to use K-means?

In such a case, to detect the outlier I would recommend creating a simple histogram and then based on its shape you can visually find the outliers. To get a proper outlier threshold you can use np.quantile() function.


Thematic clustering of text Please advise on starting points, research (papers,frameworks) related to thematic clustering of text. In  particular on a system with two levels of clustering where second level has a temporal nature.
Thanks!

Update:

Sorry for ambiguity in my initial question. I need to clarify, that I have experience with clustering in general and document clustering in particular, for text using TFIDF, word embeddings (word2vec, Glove and BERT Sentence embeddings) in a vector space.

My question originates from some text mentioning "two levels of thematic clustering of text where second level has a temporal nature". Just wanted to know about this technique and in particular "temporal nature" used in clustering.

Your question is way to vague. What did you try, what do you want to achieve in detail. However, find below some R examples how to approach topic modeling. This book might also be helpful: Text Mining with R.

library(topicmodels)
data("AssociatedPress")
AssociatedPress

# LDA (Latent Dirichlet Allocation)
ap_lda <- LDA(AssociatedPress, k = 2, control = list(seed = 1234))
ap_lda

# Get "topics"
library(tidytext)
ap_topics <- tidy(ap_lda, matrix = "beta")
ap_topics

library(ggplot2)
library(dplyr)

# Plot topics
ap_top_terms <- ap_topics %>%
  group_by(topic) %>%
  top_n(10, beta) %>%
  ungroup() %>%
  arrange(topic, -beta)

ap_top_terms %>%
  mutate(term = reorder(term, beta)) %>%
  ggplot(aes(term, beta, fill = factor(topic))) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ topic, scales = "free") +
  coord_flip()

# (words with) Greatest difference between categories
library(tidyr)

beta_spread <- ap_topics %>%
  mutate(topic = paste0("topic", topic)) %>%
  spread(topic, beta) %>%
  filter(topic1 > .001 | topic2 > .001) %>%
  mutate(log_ratio = log2(topic2 / topic1))

beta_spread

list(beta_spread$term)

# Select 10 largest and smallest values of "log_ratio")
df1 <- subset(beta_spread, beta_spread$log_ratio<=max(tail(sort(beta_spread$log_ratio, decreasing = T),10)))
df2 <- subset(beta_spread, beta_spread$log_ratio>=min(tail(sort(beta_spread$log_ratio, decreasing = F),10)))
df = rbind(df1,df2)
df

# Plot
barplot(sort(df$log_ratio), names.arg=df$term[order(df$log_ratio)], las=2)

# Gamma contains the probability of document X to belong to topic Y
ap_documents <- tidy(ap_lda, matrix = "gamma")
ap_documents
# the model estimates that only about 24.8% of the words in document 1 were generated from topic 1


Benefits of stochastic gradient descent besides speed/overhead and their optimization Say I am training a neural network and can fit all my data into memory. Are there any benefits to using mini batches with SGD in this case? Or is batch training with the full gradient always superior when possible?

Also, it seems like many of the more modern optimization algorithms (RMSProp, Adam, etc.) were designed with SGD in mind. Are these methods still superior to standard gradient descent (with momentum) with the full gradient available?

On large datasets, SGD can converge faster than batch training because it performs updates more frequently. We can get away with this because the data often contains redundant information, so the gradient can be reasonably approximated without using the full dataset. Minibatch training can be faster than training on single data points because it can take advantage of vectorized operations to process the entire minibatch at once. The stochastic nature of online/minibatch training can also make it possible to hop out of local minima that might otherwise trap batch training.

One reason to use batch training is cases where the gradient can't be approximated using individual points/minibatches (e.g. where the loss function can't be decomposed as a sum of errors for each data point). This isn't an issue for standard classification/regression problems.

I don't recall seeing RMSprop/Adam/etc. compared to batch gradient descent. But, given their potential advantages over vanilla SGD, and the potential advantages of vanilla SGD over batch gradient descent, I imagine they'd compare favorably.

Of course, we have to keep the no free lunch theorem in mind; there must exist objective functions for which each of these optimization algorithms performs better than the others. But, there's no guarantee whether or not these functions pertain to the set of practically useful, real-world learning problems.


Different results every time I train a reinforcement learning agent I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it.

The objective of an agent is to maintain a temperature of 24 deg in a zone and it takes actions every 15 mins.The length of episode is 9 hrs. I have trained the model for 1  million steps and the rewards have converged. I assume that the agent is trained enough. I have done some experiments and have few questions regarding the training

I test an agent by letting it take actions from a fixed initial state, and monitor the actions taken by actions and states for an episode. When I test the agent multiple times, actions taken and states resulted are different every time. Why is this happening when the agent is trained enough?
I train an agent for 1 million steps. I train another agent for 1 million steps on the same environment with same step of hyperparameters and every thing else same. Both these agents converge. Now when I test these agents actions taken by these agents are not identical/similar. Why is this so?

Can someone help me with these.?

Thank you

A part of the agent consists of taking random actions. So there is a % chance that the agent will take a random action instead of an action based on the training. This is called "exploration". This page describes this as "The amount of randomness in action selection depends on both initial conditions and the training procedure. Over the course of training, the policy typically becomes progressively less random, as the update rule encourages it to exploit rewards that it has already found. "
This is normal. The agent's network is initialized with random weights, and part of the actions it takes during the training are also random (see above). So different training runs will produce different results. If you want to circumvent this issue, you could use a fixed seed for the random-number generator.


What is the purpose of standardization in machine learning? I'm just getting started with learning about K-nearest neighbor and am having a hard time understanding why standardization is required. Reading through, I came across a section saying

  When independent variables in training data are measured in different
  units, it is important to standardize variables before calculating
  distance. For example, if one variable is based on height in cms, and
  the other is based on weight in kgs then height will influence more on
  the distance calculation.

Since K nearest neighbor is just a comparison of distances apart, why does it matter if one of the variables has values of a larger range since it is what it is.

Considering 3 points A,B & C with x,y co-ordinates (x in cm, y in grams) A(2,2000), B(8,9000) and C(10,20000), the ranking of the points as distance from origin for example (or any other point), will be the same whether the y values are in grams,pounds, tonnes or any other combinations of units for both x and y so where's the need to standardise. Every example or QA i see brushes through with the same statement of 'one variable influencing the other' without a real example of how this might occur. Again, how does one know when this influence is too much as to call for standardization.

Also,what exactly does standardization do to the values? One of the formulas does it by Xs = (X-mean)/(max-min) Where does such a formula come from and what is it really doing? Hopefully someone can offer me a simplified explanation or give me a link to a site or book that explains this in simple terms for beginners.

Considering 3 points A,B & C with x,y co-ordinates (x in cm, y in
  grams) A(2,2000), B(8,9000) and C(10,20000), the ranking of the points
  as distance from origin for example (or any other point), will be the
  same whether the y values are in grams,pounds, tonnes or any other
  combinations of units for both x and y so where's the need to
  standardise.

This is true for the example you provided, but not for euclidean distance between points in general. Look at this example:

def euclidian_distance(a, b):
    return ((a[0] - b[0])**2 + (a[1] - b[1])**2)**0.5

a1 = 10   #10 grams
a2 = 10  #10 cm
b1 = 10   #10 gram
b2 = 100  #100 cm
c1 = 100  #100 gram
c2 = 10   #10 cm

# using (grams, cm)
A_g_cm = [a1, a2]
B_g_cm = [b1, b2]
C_g_cm = [c1, c2]

print('[g, cm] A-B:', euclidian_distance(A_g_cm, B_g_cm))
print('[g, cm] A-C:', euclidian_distance(A_g_cm, C_g_cm))

# using (kg, cm)
A_kg_cm = [a1/1000, a2]
B_kg_cm = [b1/1000, b2]
C_kg_cm = [c1/1000, c2]

print('[kg, cm] A-B:', euclidian_distance(A_kg_cm, B_kg_cm))
print('[kg, cm] A-C:', euclidian_distance(A_kg_cm, C_kg_cm))

# using (grams, m)
A_g_m = [a1, a2/100]
B_g_m = [b1, b2/100]
C_g_m = [c1, c2/100]

print('[g, m] A-B:', euclidian_distance(A_g_m, B_g_m))
print('[g, m] A-C:', euclidian_distance(A_g_m, C_g_m))

# using (kilo, m)
A_kg_m = [a1/1000, a2/100]
B_kg_m = [b1/1000, b2/100]
C_kg_m = [c1/1000, c2/100]

print('[kg, m] A-B:', euclidian_distance(A_kg_m, B_kg_m))
print('[kg, m] A-C:', euclidian_distance(A_kg_m, C_kg_m))

Output:

[g, cm] A-B: 90.0
[g, cm] A-C: 90.0
[kg, cm] A-B: 90.0
[kg, cm] A-C: 0.09000000000000001
[g, m] A-B: 0.9
[g, m] A-C: 90.0
[kg, m] A-B: 0.9
[kg, m] A-C: 0.09000000000000001

Here you can clearly see that the choice of units influence the ranking, thus the need for standardization.


How is the E(X) of a Poisson distribution lambda? I was recently learning about the Poisson distribution, and was very perplexed about the E(X) equaling lambda. Like how did lambda even come into the picture here, isn't it a symbol of wavelength?
Also isn't wavelength just a smaller unit of distance? In that case isn't saying E(X)=5 cars per hour(assume) wrong? Because Distance=Speed * Time, and saying per hour signifies division.

Thanks in advance for solving my doubts. If possible please explain everything in layman's terms.

The probability mass function for a Poisson-distributed random variable $X$ is $f_X(x)=\dfrac{\lambda^xe^{-\lambda}}{x!}$. The expected value is a sum over all possible values of $x$, the natural numbers (non-negative integers).

$$\mathbb{E}[X]=\sum_{x=0}^{\infty} x f_X(x)$$

When you evaluate that sum, the result is $\lambda$.

Yes, $\lambda$ often means a wavelength in physics. That’s just notation, however, and easily could be $\xi$ or $\beta$ or $\gamma$. Your physics knowledge will also tell you that $\theta$ is an angle, yet statisticians often use $\theta$ as a parameter. For the Poisson distribution, $\lambda$ denotes a rate, and this is how you get five cars per hour as the average rate of cars passing through a remote intersection (or whatever they’re doing).

EDIT

I like JBStatistics. He derives the expected value of a Poisson distribution, if you’re interested in seeing how that series calculation works: https://youtube.com/watch?v=65n_v92JZeE.


Instances vs. cores when using EC2 Working on what could often be called "medium data" projects, I've been able to parallelize my code (mostly for modeling and prediction in Python) on a single system across anywhere from 4 to 32 cores. Now I'm looking at scaling up to clusters on EC2 (probably with StarCluster/IPython, but open to other suggestions as well), and have been puzzled by how to reconcile distributing work across cores on an instance vs. instances on a cluster.

Is it even practical to parallelize across instances as well as across cores on each instance? If so, can anyone give a quick rundown of the pros + cons of running many instances with few cores each vs. a few instances with many cores? Is there a rule of thumb for choosing the right ratio of instances to cores per instance?

Bandwidth and RAM are non-trivial concerns in my projects, but it's easy to spot when those are the bottlenecks and readjust. It's much harder, I'd imagine, to benchmark the right mix of cores to instances without repeated testing, and my projects vary too much for any single test to apply to all circumstances. Thanks in advance, and if I've just failed to google this one properly, feel free to point me to the right answer somewhere else!

A general rule of thumb is to not distribute until you have to. It's usually more efficient to have N servers of a certain capacity than 2N servers of half that capacity. More of the data access will be local, and therefore fast in memory versus slow across the network.

At a certain point, scaling up one machine becomes uneconomical because the cost of additional resource scales more than linearly. However this point is still amazingly high.

On Amazon in particular though, the economics of each instance type can vary a lot if you are using spot market instances. The default pricing more or less means that the same amount of resource costs about the same regardless of the instance type, that can vary a lot; large instances can be cheaper than small ones, or N small instances can be much cheaper than one large machine with equivalent resources.

One massive consideration here is that the computation paradigm can change quite a lot when you move from one machine to multiple machines. The tradeoffs that the communication overhead induce may force you to, for example, adopt a data-parallel paradigm to scale. That means a different choice of tools and algorithm. For example, SGD looks quite different in-memory and in Python than on MapReduce. So you would have to consider this before parallelizing.

You may choose to distribute work across a cluster, even if a single node and non-distributed paradigms work for you, for reliability. If a single node fails, you lose all of the computation; a distributed computation can potentially recover and complete just the part of the computation that was lost.


how can I replicate working of Multi Label Binarizer from sklearn package in R? I want to achieve same working of MultiLabelBinarizer from sklearn.preprocessing package in R. I have list of labels for each example (for Predicated and Actual) like below.

[[1]][1] "a"      "b"      "d"    "f"
[[2]][1] "g"      "b"      "f"    "d"
....

what I want is to get this in -

[[1 1 0 1 0 1 ...]
[0 1 0 1 0 1 1 ..]]...

So that I can get it in the form for multilabel_confusion_matrix.

One way to do this is to convert the source code from Python to R - see line 289 here.

You could also check for functionality in existing R packages - for instance, mldr, mlr or caret packages.


Understanding ELBO Learning Dynamics for VAE? As I understand it, I'm basically minimizing the KL Divergence between the Prior and Encoder latent distribution, and the log probability of the decoder distribution. I have a model that does generate some useful latent distribution for classification using this loss function, however, it has weird learning dynamics. My decoder distribution is a 300 dimension Gaussian and my latent is a 40 dimension Gaussian, so in all instances, both terms of my ELBO should vary wildly in order of magnitude. It seems from looking around on the internet, most use cases of VAEs look similar - high dimension decoders and low dimensional encoders, so I would imagine some people see a similar a problem, but I haven't mentioned any note of this in any literature or tutorial. Is there any way to correct for this?

There is a nice paper that discusses this problem of named as 'Modelling Bias' in VAE's. ELBO objective is:

$$L_{ELBO}(x) = E_{q_φ(z|x)}[\log p_θ(x|z)] − D_{KL}(q_{φ}(z|x) || p(z))$$

The first term forces the VAE to put all the mass of $q_φ(z|x)$ the mode of $p_θ(x|z)$ while the second term forces it to be as diffuse as possible. But generally  for images, the dimensionality of X is often orders of magnitude larger than the dimensionality of Z.

Because the same per dimensional modeling error incurs a much larger loss in X space than Z space, when the two objectives are conflicting
(e.g., because of limited modeling capacity), the model will tend to sacrifice divergences on Z and focus on minimizing divergences on X. Hence it may lead to over-fitting rather than generalization.


Machine Learning in Practice I worked on a machine learning project where we dealt with relatively small data sets. I noticed that the way that we tried to increase performance was basically to try out a bunch of different models with different hyperparameters, try out a bunch of different sets of features, etc. Basically, it seemed like we approached the problem fairly randomly and that we had no real theoretical basis for anything we tried. This disillusioned me a fair amount and made me think if this is what machine learning engineers do in practice.
Have people found that this is fairly common? How can you work on a machine learning problem in a non-random-try-everything kind of way?

Here are a few comments from my perspective:

First, methodologically this kind of approach must be done carefully, but people who work this way are rarely careful. training/evaluating many different types of models and/or parameters is equivalent to doing parameter tuning, i.e. it's essentially training a model. As a consequence the performance should be evaluated on a validation set, and after that the best model is selected and only this model should be evaluated on the (unseen) test set. A common mistake is to evaluate all the models on the same test set and assume that this gives the real performance. This is wrong, the best performance could have been obtained by chance and therefore the true performance is likely lower.
While it is normal to try to maximize performance and to some extent this can involve a fair amount of experimental evaluation, trying many different models/parameters blindly is in my opinion very poor practice and it rarely leads to the best results (but it can lead to an illusion of good result, see point above). Most of the time there is much more to gain by doing a fine-grained analysis of the data, including studying expert knowledge about the context, than blindly relying on black box algorithms. A simple example is feature engineering: sometimes some simple redesigning of the features can drastically improve performance, but there's no ML technique which can replace this (DL methods may come close to it, but usually at a much higher computational cost).
Final point: the quest for maximum performance is often misguided. Sometimes performance is used as a lazy excuse for not doing the job properly: what really matters in production? Is it really 0.3% more F1-score or an interpretable result? Was the evaluation measure chosen to really reflect the quality of the prediction or just "because it's standard"? And the training/test sets, do we know if the production data will look exactly the same? If not, is the ML method robust enough to handle these variations? When optimizing performance one often relies on the model catching the most subtle patterns, which means a higher risk of overfitting and mediocre performance in production. More generally, there are more questions (at least in research) about explainability, the role of human actors in a ML process, and of course the computational complexity and the environmental cost of the ML method. All of these points are not reflected by a performance measure, but they matter.

Overall I would consider this kind of approach as mediocre: it's inelegant, it's prone to mistakes (which often won't be detected) and it's short-sighted because it ignores other dimensions of the quality of a ML system.
By the way this approach can be automated. Personally I would not be very proud (and probably even worried) if my job could be done entirely by a program.

Have people found that this is fairly common?

In research my guess is that this would be common enough in low-level publications, but it's unlikely to pass at any reputable journal/conference (because expert reviewers can usually detect it).

How can you work on a machine learning problem in a non-random-try-everything kind of way?

That's the thing, there's no recipe... But that's the beauty of it, isn't it? :)
In my opinion it's a bit like medicine: a good doctor has a good intuition because they have a solid theoretical background and a lot of practice. They don't just follow some rigid manual, they see the patient as a unique individual and they do their best to understand every aspect of the problem.


Finding aggregated information of data I am new to data science. I have a dataset of around 200,000 records, having 5 columns. There is a field called, class. For each class, there are one or many divisions. I have to do this:
1. Filter the dataset, such that only those classes with at least 5 divisions turn up.

For each division, I have to calculate attendance from another column.
There is a minimum attendance value for each class. I have to find the percentage of divisions in each class with the minimum attendance.

I started with importing the data in python using Pandas and started writing loops for processing this. But I am sure this is not the right way to do. Can you please give some idea.Can I do this in Excel pivot table?

It is a bit hard to solve your problem without data but I tried to give it a go
with a how I think the data would be encoded. I used R with data.tables. You can read data.tables with fread().

Step 1

 require(data.table)

 # Assume sample_data has the following format:
 #   class: the class
 #   division: the division
 #   attendance: the attandance for a match
 #
 # I assume the table is in long format e.g. multiple rows exist per class with
 # per class one or different divisions.

 # Make the list of classes with at least 5 divisions.
 classes_of_interest <-
   sample_data[,
               .(num_divisions = length(unique(divisions))),
               by = class][num_divisions > 4, class]

Step 2

 # Only consider the classes that were in at least 5 divisions.
 attandance_by_division <-
   sample_data[class %in% classes_of_interest,
               .(attendance = sum(num_people)),
               by = list(division, class)]
 setkey(attandance_by_division, "class")

Step 3

 # Merge the data set with a datas set that contains the required number
 # of attendants per class.
 # The  format is as follows:
 #   class: the class
 #   mininum_attendance: the minimum attendance
 attendance_data <-
   merge(attendance_requirements,
         attandance_by_division, by = "class")

 # Here I exploit the fact that the true/false condition will be converted
 # to a 1 and 0. So I can sum and divide by the length of index subset created
 # by aggregating on 'class'.
 pct_of_division <-
   sample_data[,
               .(pct_with_min_attendance = (sum(attendance > minimum_attendance)
                                            / length(.I))),
                 by = class]

Hope this helps


Are there studies which examine dropout vs other regularizations? Are there any papers published which show differences of the regularization methods for neural networks, preferably on different domains (or at least different datasets)?

I am asking because I currently have the feeling that most people seem to use only dropout for regularization in computer vision. I would like to check if there would be a reason (not) to use different ways of regularization.

Two points:

Dropout is also usually compared with neural networks ensembles. It seems it has some of the performance benefits of training and averaging several neural networks.
Dropout is easier to calibrate than regularization. There is only one hyperparameter which is the dropout rate and people widely use 0.5 while training (and then 1.0 on evaluation of course :)), see e.g. this TensorFlow example.

Anyhow, I am a little skeptical of neural networks empirical studies. There are just too many hyperparameters to fine tune, from the topology of the network to the gradient descent optimization procedure to activation functions and whatever it is you are testing like regularization. Then, the entire thing is stochastic and usually performance gains are so small that you can hardly statistical test for differences. Many authors do not even bother doing statistical testing. They just average cross-validation and declare whatever model had the highest decimal point gain to be the winner.

You may find a study promoting dropout only to be contradicted by another promoting regularization.

I think it all boils down to aesthetics preferences. Dropout IMHO sounds more biological plausible than regularization. It also seems easier to calibrate. So, I personally prefer it when using a framework like TensorFlow. If we have to use our own neural network, which we often do, we will use regularization because it was easier to implement.


Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits? I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits.

But for some reason, the cost using this function is never equal to zero even if the prediction is equal to the correct valuel.

I tried plotting the output using the formula on TensorFlow's website:
https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits

This formula:

max(x, 0) - x * z + log(1 + exp(-abs(x)))

And by making this plot, I realized that it really isn't zero when the outputs are equal. For example, if z = 0 and x = 0, the result of this function is ~0.693.

This isn't really making sense to me. Can someone shed some light on why it isn't zero when the prediction is correct?

For a binary classification (1 or 0), you should use softmax cross-entropy with logits since the output classes are mutually exclusive / discrete.

The function you used is for cases with multiple outputs (multilabel) all of which can be ones (their documentation for example talks of the case where image can contain both elephant and a dog). The calculation is different than mutually exclusive case.


Connection between Regularization and Gradient Descent I would like to understand regularization/shrinkage in the light of MLE/Gradient Descent. I know both concepts but I do not know/understand whether both are used to determine coefficients of a linear model. If so, what are the steps followed?

To further elaborate, regularization is used to reduce variance which is accomplished through penalizing coefficients of a linear model. The tuning parameter, lambda, is determined through cross-validation. Once, lambda is determined the coefficients are automatically determined, right? Hence, why do we need to minimize (RSS + regularization term) to find coefficients? Are the steps the following:

Find lambda through cross-validation
Minimize (RSS + regularization) through MLE or GD
Find coefficients
Penalize coefficients to decrease variance
We are left with a small subset of coefficients

The fitting procedure is the one that actually finds the coefficients of the model. The regularization term is used to indirectly find the coefficients by penalizing big coefficients during the fitting procedure. A simple (albeit somewhat biased/naive) example might help illustrate this difference between regularization and gradient descent:

X, y <- read input data
for different values of lambda L
    for each fold of cross-validation using X,y,L
        theta <- minimize (RSS + regularization using L) via MLE/GD
        score <- calculate performance of model using theta on the validation set
    if average score across folds for L is better than the current best average score
        L_best <- L

As you can see, the fitting procedure (MLE or GD in our case) finds the best coefficients given the specific value of lambda.

As a side note, I would look at this answer here about tuning the regularization parameter, because it tends a little bit murky in terms of bias.


Visualize the predicted and actual class after training and testing The data set X has 10 features with 50 instances labelled as 0 and 1. Considering only 6 instances as an example here, let YPred labels are [1,0,1,0,1,1] and the actual ground truth labels are YTest = [0,0,1,1,1,0]. I cannot draw a decision boundary after classification since the data set is multi-dimensional. In the following code, pred = predict(svmModel, X(testIdx,:)); runs k times.

I cannot understand which predicted class labels should be taken after the cross-validation ends to say that this is the final "good" prediction.

The whole essence of cross-validation is to check the model with various sets of data and to know how well it predicts with 'unseen' data. So, with the 10 folds, you create 10 different training and test/validation set. But, the model remains the same.

What you are trying to ask is which model to choose. There is only one model. We are only checking the model that's already built using cross-validation.

After checking the model you have built, use the complete dataset to train and predict the labels on test set using this final model.

Usually, the dataset is split into training, validation and test set. Test set is not touched till the final model has been built. The test set you create during cross-validation is actually the validation set. The ability of the model to predict well is validated on this set.

Hope it helps!

Update:
fitcsvm() is what trains the data. This is the modelling bit. What you are doing is right. What you need to understand is that cross-validation is used for checking the model. There would be cases where one model wouldn't predict as well as another one. Testing with only one test would not give good insights on how well the model is working. This is why, we use cross-validation - to test the model with different training and test sets.

10 folds for only 50 records might be an overkill. 2-3 folds would suffice.


3 images as one input in CNN (U-Net) I have been advised by my supervisor that if my U-Net segmentation network has RGB images at the input then I could use the channels for different images - median filter for R, normalization for G, canny-edge detection for B (example). I have no idea how to do that. Tried to find something similiar here but unfortunately without a success. Would be greateful if somebody could explain me this in detail, I'm new in DL. Thanks

An input to a U-Net are normally images with 3 (or 4) channels, as images have a red, green, and blue channel. It is however technically possibly to use other images (or arrays for that matter) as the input which a larger number of channels. If you wanted to use the three different methods that you mention, you would first have to apply all three methods to original image, which would then return an array with 1 or more channels. You then have to stack them in the channels direction, which would result in an array of shape HxWxC, where C is the total number of channels. The total number of channels in the input then depends on the image preprocessing methods you use and the number of channels of the original image.


Data visualization with extreme far away points I want to show points across two groups. However, for both groups, there are some points which are far away from most of the other points within each group, shown below. Any suggestions for this situation? Thank you.

If you want to see the distribution of the data that is hidden in the bottom portion, you can add a histogram or probability plot, or even a violin plot. Each will show the distribution of the data more clearly than this boxplot does, and you can still see the true value directly. You can also add some jitter to the boxplot to see more of the overlapping points displayed.
Jitter:

Violin:

Probability Plot with Boxplot:


convert dataframe without header to dictionary with a row of number I am trying to put a dataframe into dictionary  ,with the first column as the key ,the numbers in a row would be the value . how should I do it ? I tried to_dict but did not work , my df has no header or rownumber.

Works for me with numbers as headers:

import pandas as pd
df = pd.DataFrame({'1': [1, 2],
                 '2': [0.5, 0.75]},
                 index=['row1', 'row2'])
df.to_dict()

Output:

{'1': {'row1': 1, 'row2': 2}, '2': {'row1': 0.5, 'row2': 0.75}}

You may also specify column names when you read the data:

Cov = pd.read_csv("path/to/file.txt",
                  sep='\t',
                  names=["Sequence", "Start", "End", "Coverage"])

See: https://stackoverflow.com/questions/34091877/how-to-add-header-row-to-a-pandas-dataframe


How to use scikit-learn normalize data to [-1, 1]? I am using scikit-learn MinMaxScaler() to normalize to $[0, 1]$, but I want to normalize to $[-1, 1].$ What function do I have to use to normalize this way?

Scaling between 0 and 1 is simply written for an array of values arr = $[x_{1}, x_{2}, ...., x_{n}]$ as scaled_array = (arr-arr.min())/(arr.max()-arr.min())

But scaling between two values can be inherited from normalisation in Images when values are scaled between 0 to 255 (normally).

You can write scaling as scaled_values = ((val - min)*(new_max - new_min)/(max - min)) + new_min

where new_min, new_max are the maximum and minimum values you are scaling in between and max and min are the maximum and minimum values in your array.

For example if you want to scale values between -1 and 1 for an array [2, 5, 0], it reduces to ((arr - 0)*(1-(-1))/(5-0)) - 1 = [-0.2, 1, -1]

Check this wiki page.

Edit:

As pointed out rightly in comments by @AnkitSeth, MinMaxScalar do have feature_range argument you can pass on. But I am sure the internal job done by feature_range is the same thing as I had mentioned above.


How does KNN work if there are duplicates? I am currently debating with my friend about how KNN handles duplicates. Suppose K = 2, and we have a 1-dimensional set of data points to illustrate my dilemma

I = {1, 2, 2, 2, 2, 2, 6}

Thus is it correct to say that the K=2 nearest neighbours of data point 1 is simply {2, 2}? Also, same reasoning if we did the 2 nearest neighbours of data point 2 it would be {2, 2} as well not including itself?

Your reasoning is correct - you should consider duplicate points as separate. You can see that this must be the case in several ways:

Introduction of small random noise to the data should not affect the classifier on average. This would not be the case if you removed duplicates.
Suppose that your input space only has two possible values - 1 and 2, and all points "1" belong to the positive class while points "2" - to the negative. If you remove duplicates in the KNN(2) algorithm, you would always end up with both possible input values as the nearest neighbors of any point, and would have to predict a 50% probability for either class, which is certainly not a consistent classification strategy.

The extra question to think about is how to deal with the situation when you have different Y labels assigned to several points with the same X coordinate.

You could mix all classes together and say that the label of each point in the set of duplicates is represented by the distribution of labels in the whole set of points with that coordinate. Alternatively, you could simply sample K random points from the set.

Both strategies should result in a consistent classifier, however in the second case your predictions may not be deterministic. Most practical implementations (including, for example, sklearn.neighbors.KNeighborsClassifier), however, use this simpler, nondeterministic strategy, as it is perhaps slightly more straightforward.


Scala mllib 2.2.0 k means. Number of runs As per the documentation here, the runs parameter does nothing.
I have looked at the actual Scala code for k means and it appears that the train method only does one start. There is no way to change this.

QUESTION:
Am I correct that if I want 5 starts I need to write my own code that calls train 5 times (or until some criteria has been met)?

That's correct. It used to control making multiple starts at once but for various reasons this didn't work well, and was deprecated / made a no-op. I removed it for Spark 3.0.


How to predict number of orders for coming weeks using previous week's data? I'm getting confused as how to proceed for this problem:

I need to predict a city's food supply, for number of orders, for the upcoming weeks based on previous weeks' data. What should I do? Do I need to use a time related algorithm or a simple algorithm?

If you treat this problem as a time-series one, you will gain more information for predictions, because it includes several time related components, and you can customize your orders predictions based on week day(week-end, Black Fridays)

If you treat this problem as linear regression, you will only be able to predict for the one upcoming week and not several weeks in advance, as they will only depend on the latest record you gave as the predictor.

analytics vidhya has great tutorials on time-series problems.


Very basic question: what is an accepted term for "linear order distance" In data science we have "Manhattan Distance" as a slang term for Level 1 Distance and "Euclidean Distance" as a slang term for Level 2 Distance. Is there an accepted term for linear distance in memory of cells in different rows in a matrix? That is, given an 8x8 matrix, the "linear distance" from [3,2] to [4,5] is:
(4-3) * 8 + (5-2)

This is the distance in memory addressing. "Level 0 Distance" doesn't really work as a technical term, because computer memory layout is not a native concept in mathematics. Is there a standard term for this? It reminds me of the old flyback transformers in cathode ray tubes, so "flyback distance" makes sense as a slang term.

Actually this is the Manhattan / $L_1$ / Norm 1 distance, if you just multiply the row by $8$ (or $N$).

The $L_1$ distance between $[24, 2]$ and $[32, 5]$ is what you are looking for.


clarification on train, test and val and how to use/implement it So far I think I understood the differences between the training, test and validation set. Basically it is like in this image:

Training set: The data where the model is trained on

Validation set: Data the model has not been trained on and used to tune hyperparameters

Test set: In principle the same like the validation set.. just used at the final end after the model has been tailored.

The training set is usually set up via some cross validation. When fitting via model.fit(X_train, y_train,..) will Keras shuffle the data autonomously ?

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2
What is the difference?

And after that, the test/prediction set will be taken into account just as model.pred(X_pred, y_pred,..). Right?

When fitting via model.fit(X_train, y_train,..), will Keras shuffle the
data autonomously?
Yes. shuffle = True is default. So, it basically shuffles every time.

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2 What is the difference?
The difference is that you can manually provide a validation data set. It's not X_test, Y_test. Rather X_test and Y_test are used for model evaluation model.evaluate() or model.predict(). model.fit is used for the training dataset. When you say validation_set = 0.2, it takes 20% data from training dataset and provides you the validation accuracy and loss.

And after that, the test/prediction set will be taken into account just as model.fit(X_pred, y_pred,..). Right?
You need to use model.predict(X_pred, Y_pred)


Clustering for mixed numeric and nominal discrete data My data includes survey responses that are binary (numeric) and nominal / categorical. All responses are discrete and at individual level.
Data is of shape (n=7219, p=105).
Couple things:

I am trying to identify a clustering technique with a similarity measure that would work for categorical and numeric binary data. There are techniques in R kmodes clustering and kprototype that are designed for this type of problem, but I am using Python and need a technique from sklearn clustering that works well with this type of problems.

I want to build profiles of segments of individuals. meaning this group of individuals care more about these set of features.

Taking a stab:

  I am trying to identify a clustering technique with a similarity measure that would work for categorical and numeric binary data.

Gower Distance is a useful distance metric when the data contains both continuous and categorical variables.

  There are techniques in R kmodes clustering and kprototype that are designed for this type of problem, but I am using Python and need a technique from sklearn clustering that works well with this type of problems.

I wasn't able to find an implementation of Gower Distance in Python when I searched for it about 4-5 months back. So I came up with my own implementation.

import pandas as pd
import numpy as np
from sklearn.neighbors import DistanceMetric

def gower_distance(X):
    """
    This function expects a pandas dataframe as input
    The data frame is to contain the features along the columns. Based on these features a
    distance matrix will be returned which will contain the pairwise gower distance between the rows
    All variables of object type will be treated as nominal variables and the others will be treated as
    numeric variables.
    Distance metrics used for:
    Nominal variables: Dice distance (https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient)
    Numeric variables: Manhattan distance normalized by the range of the variable (https://en.wikipedia.org/wiki/Taxicab_geometry)
    """
    individual_variable_distances = []

    for i in range(X.shape[1]):
        feature = X.iloc[:,[i]]
        if feature.dtypes[0] == np.object:
            feature_dist = DistanceMetric.get_metric('dice').pairwise(pd.get_dummies(feature))
        else:
            feature_dist = DistanceMetric.get_metric('manhattan').pairwise(feature) / np.ptp(feature.values)

        individual_variable_distances.append(feature_dist)

    return np.array(individual_variable_distances).mean(0)

The link to the same piece of code: https://github.com/matchado/Misc/blob/master/gower_dist.py

With regards to the clustering technique, I haven't used the ones you've mentioned. But I've used hierarchical clustering in R along with gower distance with success in the past.

Looking into the clustering techniques available in scikit learn, Agglomerative Clustering seems to fit the bill. http://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering

  I want to build profiles of segments of individuals. meaning this group of individuals care more about these set of features.

Once you've assigned cluster labels to each row of your data, for each cluster look into the distribution of the features (summary stats for continuous variables & frequency distributions for categorical variables). This is easier to analyze visually if your number of features are manageable (<20 maybe?).

But since you have 100+ features, I suggest a more organized approach. Create a matrix with cluster labels in the columns and the summary stat of the features in the rows (I suggest using median for continuous variable and percentage occurrence of most frequent value in cluster for categorical variable)

It might look something like this.

╔═══════════════════════╦═══════════╦═══════════╦════╦═══════════╗
║        Feature        ║ Cluster 1 ║ Cluster 2 ║ …  ║ Cluster N ║
╠═══════════════════════╬═══════════╬═══════════╬════╬═══════════╣
║ Numeric feature 1     ║ 15        ║ 37        ║ .. ║ 1         ║
║ Numeric feature 2     ║ 34        ║ 56        ║ …  ║ 56        ║
║ Categorical feature 1 ║ 47%       ║ 87%       ║ …  ║ 25%       ║
║ …                     ║ …         ║ …         ║ …  ║ …         ║
║ Categorical feature N ║ 25%       ║ 91%       ║ …  ║ 11%       ║
║ Numeric feature N     ║ 0.2       ║ 0.7       ║ …  ║ 0.5       ║
╚═══════════════════════╩═══════════╩═══════════╩════╩═══════════╝


Classified Enron Email Dataset I'm building a system able to classify emails into different categories (positive, negative, out of office, etc...) and I'm looking for a dataset of already classified emails to avoid hand classification on a 70k database.

I know it exists the Enron email dataset but do you know if it exists a version of this dataset with classified emails ? Or any other already classified email dataset ?

You can download the corpus from this site. To the best of my knowledge this is the most complete email corpus available. A project to label a subset of this email corpus can be found on this UC Berkley site. I am not sure though whether these emails have the right training labels for you.


strings as features in decision tree/random forest I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-learn takes only numbers as parameters, but I want to inject the strings as well as they carry a significant amount of knowledge.

How do I handle such a scenario?

I can convert a string to numbers by some mechanism such as hashing in Python. But I would like to know the best practice on how strings are handled in decision tree problems.

In most of the well-established machine learning systems, categorical variables are handled naturally. For example in R you would use factors, in WEKA you would use nominal variables. This is not the case in scikit-learn. The decision trees implemented in scikit-learn uses only numerical features and these features are interpreted always as continuous numeric variables.

Thus, simply replacing the strings with a hash code should be avoided, because being considered as a continuous numerical feature any coding you will use will induce an order which simply does not exist in your data.

One example is to code ['red','green','blue'] with [1,2,3], would produce weird things like 'red' is lower than 'blue', and if you average a 'red' and a 'blue' you will get a 'green'. Another more subtle example might happen when you code ['low', 'medium', 'high'] with [1,2,3]. In the latter case it might happen to have an ordering which makes sense, however, some subtle inconsistencies might happen when 'medium' in not in the middle of 'low' and 'high'.

Finally, the answer to your question lies in coding the categorical feature into multiple binary features. For example, you might code ['red','green','blue'] with 3 columns, one for each category, having 1 when the category match and 0 otherwise. This is called one-hot-encoding, binary encoding, one-of-k-encoding or whatever. You can check documentation here for encoding categorical features and feature extraction - hashing and dicts. Obviously one-hot-encoding will expand your space requirements and sometimes it hurts the performance as well.


My CNN model Accuracy doesn't increase (high loss and low acc) Well, I need to do a CNN to classify if a Image is from one or another class. But my model return high losses (6.~8.) and low accuracies (0.50 on max). I tried to include more layers, change my activation functions, and nothing works. My database is 142 .jpg imgs (71 for each class)

This is my code:
OLD CODE

def ReadImages(Path):
    LabelList = list()
    ImageCV = list()
    classes = ["nonPdr", "pdr"]

    # Get all subdirectories
    FolderList = [f for f in os.listdir(Path) if not f.startswith('.')]
    print(FolderList)

    # Loop over each directory
    for File in FolderList:
        for index, Image in enumerate(os.listdir(os.path.join(Path, File))):
            # Convert the path into a file
            ImageCV.append(cv2.resize(cv2.imread(os.path.join(Path, File) + os.path.sep + Image), (600,700)))
            LabelList.append(classes.index(os.path.splitext(File)[0]))

    return ImageCV, LabelList

model = Sequential()
model.add(Conv2D(64, kernel_size=(3,3), padding="same",activation="relu", input_shape=(700,600,3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128,  kernel_size=(4,4), padding="same",activation="relu"))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='RMSprop', loss='binary_crossentropy', metrics=['accuracy'])

data, labels = ReadImages(TRAIN_DIR)

model.fit(np.array(data), np.array(labels), epochs=10, batch_size=20)

model.save('model.h5')

What can I do to improve my model? I appreciate your help!

UPDATE
I tried to do what Shubham Panchal said but isn't resolve the problem:

THINGS THAT I TRIED
- Reduce Img size
- lr=0.0001
- optimizers: adam, sgd, rmsprop
- put more layers
- put dropout layer
- normalize the data with np.array(data) / 255.0
- Increase the data (1400 total, 700 each class)

My code:

model = Sequential()
model.add(Conv2D(64, kernel_size=(3,3), padding="same",activation="relu", input_shape=(150,150,3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, kernel_size=(3,3), padding="same",activation="relu"))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128,  kernel_size=(3,3), padding="same",activation="relu"))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128,  kernel_size=(3,3), padding="same",activation="relu"))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(1, activation='softmax'))
opt = SGD(lr=0.0001, momentum=0.9)
model.compile(optimizer = opt, loss="binary_crossentropy", metrics=['accuracy'])
#model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

data, labels = ReadImages(TRAIN_DIR)

model.fit(np.array(data) / 255.0, np.array(labels), epochs=10, batch_size=16)

My console:

Epoch 1/10
1400/1400 [==============================] - 58s 42ms/step - loss: 7.9712 - acc: 0.5000
Epoch 2/10
1400/1400 [==============================] - 59s 42ms/step - loss: 7.9712 - acc: 0.5000
Epoch 3/10
1400/1400 [==============================] - 59s 42ms/step - loss: 7.9712 - acc: 0.5000
...

Anyone have any ideia what can I do??

Here are some hacks which you can use to improve then model.

The dataset seems to be inadequate. Try image augmentation.

Image augmentation basically applies to different transformations to your images. Like the rotation, scale, color, whitening etc. are changed. It helps the model to generalise better on the image. See here.

Use softmax classification function.

Instead of sigmoid, try softmax activation function since you are working on a classification task. Sigmoid is mostly reserved for binary classification tasks.

model.add(Dense(1, activation='softmax'))

Add more convolution layers.

Consider adding more Conv2D layers to the model because the image size is quite large. More the number of layers, the better feature extraction will take place. Due to less layers, your model is not able to extract smaller features which may be required for proper classification.

Tips:

Try adam optimizer instead of rmsprop.
Restrict the kernel size to ( 3 , 3 ).
Use a smaller batch size. Your batch size is 20 for 142 images. That makes only ~7 batches. Lower it to a number like 6 or 10.
Use Dropout layers in between the Dense layers.
The smaller learning rate always helps like 0.001 or 0.0001.


Would it make a difference to clean the data before using R findCorrelation()? I'm new in data analysis area and didn't have very strong sattistical background...

Now, I'm trying to filter out those numeric columns which have high correlation. At this moment, I don't plan to use dimensional reduction algorithms such as PCA because I'm still collecting data, and there are too many columns, I hope to remove some columns first before linking different sources of data together.

Playing with PCA and machine learning models will be after linking those sources of data.

So, at this "before" stage, I'm trying to use R library caret, findCorrelation() method, but I could not find how does this method work.

Now my question is, I should use this method after data cleaning (such as, deal with missing data, outliers, data imbalance) or it doesn't matter to do it before or after?

In most cases, you will want to run findCorrelation after cleaning. The reason being the cleaning process is either removing unwanted data points (that you deem irrelevant anyway) or computing some transformation that you'll want findCorrelation to be aware of.

Now, that said, I've often run it twice, once before cleaning and once after if my transformation could alter the correlation structure of the data.

Fortunately, since you're using caret, it is a simple matter to estimate your models both ways, then compare the ultimate differences in accuracy.  Unfortunately, the correct answer to many questions in ML is 'do it both ways then compare'


Inception Score (IS) and Fréchet Inception Distance (FID), which one is better for GAN evaluation? IS uses two criteria in measuring the performance of GAN: The quality of the generated images, and their diversity based on the entropy of the distribution of synthetic data.

On the other hand, FID uses the Inception network to extract features from an intermediate layer.

But how can I know that what to be used in a given situation. Any comparisons between them or recommendation for usage?

Here is the original paper proposing FID.
Here is an excerpt from
Jason Brownlee's https://machinelearningmastery.com/how-to-implement-the-frechet-inception-distance-fid-from-scratch/ along with a couple of quotes from the paper:
The inception score estimates the quality of a collection of synthetic images based on how well the top-performing image classification model Inception v3 classifies them as one of 1,000 known objects. The scores combine both the confidence of the conditional class predictions for each synthetic image (quality) and the integral of the marginal probability of the predicted classes (diversity).
The inception score does not capture how synthetic images compare to real images. The goal in developing the FID score was to evaluate synthetic images based on the statistics of a collection of synthetic images compared to the statistics of a collection of real images from the target domain.

For the evaluation of the performance of GANs at image generation, we introduce the “Frechet Inception Distance” (FID) which captures the
similarity of generated images to real ones better than the Inception
Score.

Like the inception score, the FID score uses the inception v3 model. Specifically, the coding layer of the model (the last pooling layer prior to the output classification of images) is used to capture computer-vision-specific features of an input image. These activations are calculated for a collection of real and generated images.
The activations are summarized as a multivariate Gaussian by calculating the mean and covariance of the images. These statistics are then calculated for the activations across the collection of real and generated images.
The distance between these two distributions is then calculated using the Frechet distance, also called the Wasserstein-2 distance.

The difference of two Gaussians (synthetic and real-world images) is measured by the Frechet distance also known as Wasserstein-2
distance.


Neural networks - Find most similar images I am working with Python, scikit-learn and keras. I have 3000 thousands images of front-faced watches like the following ones:
Watch_1, Watch_2, Watch_3.

I want to write a program which receives as an input a photo of a real watch which maybe taken under less ideal conditions than the photos above (different background colour, darker lightning etc) and find the most similar watches among the 3000 ones to it. By similarity I mean that if I give as an input a photo of a round, brown watch with thin lace then I expect as an output watches of round shape, of dark colour and with thin lace.

What is the most efficient machine learning algorithm to do this?

For example, by following this link I have two different solutions in my mind:

1) Using a CNN as a feature extractor and compare the distances between the these features for every pair of images with reference to the input image.

2) Using two CNNs in a Siamese Neural Network to compare the images.

Are these two options the best ones for this task or would you suggest something else?

Do you know any pre-trained neural network (with pre-determined hyperparameters) for this task?

I have found some interesting posts on StackOverflow about this but they are pretty old: Post_1, Post_2, Post_3.

I don't think that a high level architecture as such is the best fit but it rather depends on many factors and details. From what I am aware of the first approach is promising especially when extended by additional steps as done in the TiefVision:

An additional bounding box network is used to discriminate the relevant part of the image from the rest
The feature vectors are not simply compared directly but rather used to train a comparison network using triplets (learning similarity based on feature vectors and examples for more and less similar instances).

This work is a lot more recent (2016/17) than what you posted and comes with a nice tool set and a more detailed paper.

Why using triplets aka Deep Ranking?

As stated in the comments: Why should one use triplets for image similarity instead of learning feature vectors and calculate their distance? Triplets are a way of formulating the similarity question as a learning problem instead of learning feature vectors that basically do not care for similarity. This approach makes especially sense in cases where human-perceived similarity is important, which might differ from machine perception.

Triplets work like this: You provide 3 images. One to compare to, one similar (close) and one not so similar (distant) image. This is your training/test/validation data. Training your network on those samples and predicting the correct order (classify similar from non-similar images) overall lets the network learn how to order images based on their similarity.

All in all this approach is comparatively complex. It might be overengineered but you also asked for the best way to do this and Deep Ranking achieves very high precision values.


How to use historic data (granularity at day level) for ML modeling? There is a scenario where I have to use historic data which is at the day level for the past 5 years.Actually it is water flow data, what quantity of water was flown on that particular day. I have to use this feature along with a few other features like material, coating, etc,. for EDA and prediction. I tried averaging it out but not useful.
Data is like this flow1,flow2, and flow3 (including other features not shown here) for each day on that particular route id. This continues for 5 years for many routes.

 I am not able to figure out how to consolidate this data so that I can feed it to the model.
I am trying to predict the corrosion in the pipeline.
 Any guidance will be helpful.
Thanks

Welcome to DSSE, Abdul.
So, you trying to predict pipe corrosion based on water flow. I assume that the corrosion measurement is taken in different timesteps than the water flow measurements.
If you have a fixed timestep for your corrosion measure both sum and average would work just fine, as this would be two features scaled by $\dfrac{1}{n}$, where $n$ is just the number of timesteps taken by your water flow measurement system in one timestep of your corrosion data measurements.
If your corrosion measure has varying timesteps then you should use the sum of the water flow since averaging would cause inconsistency. For example, the average of flow in 100 days could be the same as the average in 10 days, and the corrosion in 100 days would be way bigger. Having two similar inputs that map to two completely different outputs would cause performance issues for a regression algorithm.


Why it is taking more training time on one machine comparatively to similar another? I have a HP15-R203TX notebook with CPU i5-5200U 2.20GHzx4 / RAM 4Gb /2 Gb NVIDIA GeForce 820M. My friend has the same notebook. Another one has a dell notebook but same graphic card.

When i ran same jupyter notebook on same dataset to train the same number of layers, it takes 20 minutes on mine but 6-8 minutes on both of my friend's.

I tried resetting the whole environment, but didn't helped.

In machine learning, while your choice of inputs and hyper-parameters do matter, most of the time convergence ultimately comes down to how lucky you are. If you're lucky you might reach optimum in a very short period of time, whereas if you're unlucky you might get stuck in a local optimum with a huge error and get stuck in a very long loop.

That's one reason; another is possibly that if you're using TensorFlow and you installed the version for non-GPU computers, and your friends installed the version for computers with GPUs.


What is clustering used for? I know that clustering can be used for unsupervised learning and some people told me for many more techniques, but I was left with no answer, when I asked for what else clustering is used.

Labeling data is not always an easy task. There are occasion that the data in hand does not have label and you need to make a model using them. You have to find the similarities and differences in your input data. Clustering approaches try to find these similarities and differences to find similar data. Also they are used as a pre-processing before doing supervised classification. In cases that the input data does not have any label, employing clustering approaches can be a way to label the data and use them for training supervised models.


Issue with IPython/Jupyter on Spark (Unrecognized alias) I am working on setting up a set of VMs to experiment with Spark before I spend go out and spend money on building up a cluster with some hardware. Quick note: I am an academic with a background in applied machine learning and work quit a bit in data science. I use the tools for computing, rarely would I need to set them up.

I've created 3 VMs (1 master, 2 slaves) and installed Spark successfully. Everything appears to be working as it should. My problem lies in creating a Jupyter server that can be connected to from a browser not running on a machine on the cluster.

I've installed Jupyter notebook successfully... and it runs. I've added a new IPython profile connecting to a remote server with Spark.

now the problem

The command

$ ipython --profile=pyspark

runs fine and it connects to the spark cluster. However,

$ ipython notebook --profile=pyspark
[<stuff is here>] Unrecognized alias: "profile=pyspark", it will probably have no effect.

defaults to the default profile not the pyspark profile.

My notebook config for pyspark has:

c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8880
c.NotebookApp.server_extensions.append('ipyparallel.nbextension')
c.NotebookApp.password = u'some password is here'

IPython has now moved to version 4.0, which means that if you are using it, it will be reading its configuration from ~/.jupyter, not ~/.ipython. You have to create a new configuration file with

jupyter notebook --generate-config

and then edit the resulting ~/.jupyter/jupyter_notebook_config.py file according to your needs.

More installation instructions here.


How do I recommend items to out of training users based on its recent views? I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings.

Since we have a lot of users in system, I had to sample some users to train model to avoid overfitting.

Now how do I construct user embeddings for out of training users. I tried constructing user embeddings by averaging item embeddings for user's items. But when I compared performance of average vector vs original user embeddings, it is not that great.

So how would I generate user embeddings using item matrix and rating matrix?

Making predictions for new users is often called the cold start problem.
This problem is difficult to over come in a purely item-to-item recommendation system. The most common solution is to include more than just item interaction information. Examples include user profile information and item content information.


Getting a balanced sample across many variables Let’s say each element in my population has several attributes.
Let’s call then A, B, C, D, E, F.
Let’s say, for simplicity, each attribute has 10 values (but could be any number between 2 and 30).
Now I want to get a sample such that the distribution is the same across all features.
So for example if the whole population has about 15% of people in feature A with value 1, my sample should be the same.
What should be the way for me to select a size for the sample and choose a sample that has the desired properties?

If I understand correctly, you want a sample with along each feature the value below which a given percentage of observations in your population falls. If so, you might want to try something like this using np.percentile:
import numpy as np
import pandas as pd

data = [ np.random.randint(1,10,6) for i in range(20) ] # fake data
df = pd.DataFrame(data=data, columns=['A', 'B', 'C', 'D', 'E', 'F']) # fake dataframe

# 15 being your desired percentage
balanced_sample = [ np.percentile(df[col], 15) for col in df.columns ]


Clusterize item set with items as vectors of features I have to clusterize this dataset in which I have houses and water consumption in this form:
$$
House1 = (x_{1},x_{2}... x_{n});\\
House2 = (y_{1},y_{2}... y_{n});\\
House3 = (z_{1},z_{2}... z_{n});\\
$$
where $x_{i}$ is the daily consumption in liters while  $n$ is a fixed parameter (length of dataset).
I need to cluster these houses in k clusters based on their water consumption.
My question is: how can I handle data expressed in this form to feed in the clustering algorithm?
Maybe I will have to agglomerate each vector in some real value?

1.you just have to represent those  features as numeric in a vector
eg:[2,4,8,10]
2.Its a good  practise to normalize vector i just took sum of elements and divide by each element by that sum of elements  =[0.06666666666666667,0.13333333333333333,0.2,0.26666666666666666,0.3333333333333333]
normalize the values in that vector to be between 0 -1
3.feed the vectors into clustering algorithm (you can try with kmeans)


Are there other large margin classifiers than SVMs? When reading about SVMs (e.g. on the German Wikipedia) there is a sentence like "an svm is a large-margin classifier).

Are there other large margin classifiers than SVMs?

Yes, one famous example are boosting techniques like Adaboost. It uses small classifiers to create a big one.

Here you can find more info about margin classifiers.


Why does Q-learning use an actor model and critic model? I'm currently reading Hands on Machine Learning with Scikit-Learn & Tensorflow, and I'm wondering why does Q-learning require an actor model and a critic model to learn?

On page 465, it states:

  As we will see, the training algorithm we will use requires two DQNs
  with the same archicture (but different parameters): one will be used
  to drive Ms. Pac-Man during training (the actor), and the other will
  watch the actor and learn from its trials and errors (the critic).

Is this a typical Q-learning implementation? If not, what is?

The book you are reading is being somewhat lax with terms. It uses the terms "actor" and "critic", but there is another algorithm called actor-critic which is very popular recently and is quite different from Q learning. Actor-critic does have two function estimators with the roles suggested in the quote. Q-learning has one such estimator*.

I have looked at the chapter in more detail, and where it says:

  one will be used to drive Ms. Pac-Man during training (the actor), and the other will watch the actor and learn from its trials and errors (the critic).

I would substitute:

  one will be used learn from current actions, and the other will remember results from some time steps ago in order to estimate the values for next actions.

This is not something that's inherently part of Q-learning, but it is part of DQN's adjustments when combining Q-learning with neural networks. Both experience replay and having two copies of the learning network (one a temporarily "frozen" version of the other) are important for stabilising the learning algorithm. Without them it can become numerically unstable.

  Is this a typical Q-learning implementation?

It's a typical implementation of basic DQN, which is how many people nowadays would implement Q-learning with neural networks.

You can ignore the references to "actor" and "critic". Instead it is easier to consider that there is just one "action value" network, and you keep an old copy of it around to help with stability.

* Generally in RL, the term "model" is reserved for a model of the environment - which neither Q-learning nor actor-critic provide. So you will also read that Q-learning is a "model free" algorithm. For the rest of the book, you will have seen "model" to refer to any statistical learning algorithm (or the architecture and learned parameters) . . . what you will see in RL texts is the careful use of "function estimator" or other terms for networks which learn something else other than how the environment behaves.


Cooperative Reinforcement Learning I already have a functioning $Q(\lambda)$ implementation for a single agent working on a dynamic pricing problem with the goal of maximizing revenue. The problem that I'm working with, however, involves several different products that are replacements for each other, so dynamically pricing them all with independent learners seems incorrect, because the price of one influences the reward of the other. The goal would be to dynamically price them all so as to maximize the sum of each individual revenue.

I've been doing some research to try to find something that applies reinforcement learning in this way, but many multi-agent implementations I have found focus more on competitive games than cooperative, or they assume incomplete knowledge of other agents (I would have complete knowledge of each agent in this scenario). Are there any well-researched/documented applications of cooperative learning in this way?

You can look these paper. First one is quite related to your task.
http://icamt2016.org/papers/SS-LTMLBDA-06-05.pdf
http://researcher.watson.ibm.com/researcher/files/us-kephart/icml00_qrt.pdf


Why Adaboost SAMME needs f to be estimable? I am trying to understand the mathematics behind SAMME AdaBoost:
At some stage, the paper adds a constraint for f to be estimable:

I do not understand why this is required. Can someone explain a bit better why this restriction is needed?
As well, would be possible to use a different constraint than the one added in the paper that would make f estimable?

Think for a while, if $f$ is not estimable, it can have any constant added to it with no difference on the result of the process.
This means that if no other constraints are imposed, $f$ is not well/uniquely defined and in fact represents a whole class of functionals. This obviously needs to be fixed and some natural constraint needs to be added (to uniquely fix $f$).
On the other hand, the additional constraint can also be seen as a free parameter of the process which nevertheless needs to be fixed somehow in a concrete instance of the process in order to take place.
In an analogy with physics, the algorithm is "gauge-invariant", but for any concrete physical problem to be solved some "gauge" needs to be chosen and fixed.
They choose to impose the symmetric constraint which reduces to the usual AdaBoost in the 2-class case.
One can impose another constraint on $f$ (than the symmetric one), choosing to satisfy other criteria if so desirable (as long as it can uniquely fix $f$).
For example, the general non-symmetric constraint is also valid:
$$f_1 + \dots + f_K = c$$
for arbitrary constant $c$. This also fixes $f$ uniquely and it can introduce bias favoring certain classes over others (eg for an imbalanced problem). Additionaly, unless $c=0$, does not reduce to the (symmetric) AdaBoost in 2-classes case (which may or may not be desirable).


What issue is there, when training this network with gradient descent? Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent?

what issue arises, when trying to train this network with gradient descent?

The activation function is sign function or signum function (A little modified).
So, its Derivative will be 0 at all the points
 Hence, the Gradient descent won’t be able to make progress in updating the weights and backpropagation will fail.