<jupyter_start><jupyter_text>Reproducing Philander's 2014 study using [*gamba*](https://github.com/gamba-dev/gamba)This notebook shows how gamba can be used to reproduce findings from Philander's 2014 study on data mining methods for detecting high-risk gamblers.- [Data Download (thetransparencyproject.org)](http://www.thetransparencyproject.org/download_index.php)- [Data Description]()- [Original Paper](https://www.tandfonline.com/doi/abs/10.1080/14459795.2013.841721)It uses data available through the transaparency project above, and performs eight distinct supervised machine learning techniques.**Note:** given the high dimensionality of this data (17), the sample size (530) doesn't meet the [rule of thumb](https://youtu.be/Dc0sr0kdBVI?t=3414) that 10x17 (or 1700) observations are required for learning to be generalisable. This means that the ouputs of the methods below may change drastically upon repeated executions.To begin, import gamba as usual;<jupyter_code>import gamba as gb
philander_data = gb.prepare_philander_data('AnalyticDataSet_HighRisk.txt', loud=True)
train_measures, test_measures = gb.split_measures_table(philander_data, frac=.696, loud=True)
display(train_measures.head(3))<jupyter_output>530 players loaded
train:test
 369 : 161 ready<jupyter_text>Neural NetworksThe following cell uses the [Keras](https://keras.io) library to create and train some neural networks as described in the study. The original study uses the R [nnet](https://cran.r-project.org/web/packages/nnet/nnet.pdfRfn.optim) and [caret](https://cran.r-project.org/web/packages/caret/vignettes/caret.html) packages, [this stackoverflow post](https://stackoverflow.com/questions/42417948/how-to-use-size-and-decay-in-nnet) was helpful in understanding the original parameters. Framing the self_exclude label (0 or 1) as a regression problem means creating a neural network with a single output node and clipping the prediction. The classification version of the neural network used in the original analysis uses an identical network topology but passes two strings as values instead of a 1 or 0. This should in theory have no substantial difference on the performance of the network (given the sample size and identical architectures). See final cell for proof.<jupyter_code>from keras.layers import Dense, Activation
from keras.models import Sequential
import numpy as np
# classification = discrete output
# regression = continuous output

def simple_classification_neural_network(train_measures, test_measures, label):
    
    train_data = train_measures.drop(['player_id', label], axis=1)
    train_labels = train_measures[label]
    test_data = test_measures.drop(['player_id',label], axis=1)
    test_labels = test_measures[label]

    model = Sequential()
    model.add(Dense(17, activation = 'relu', input_dim = 17))
    model.add(Dense(50, activation = 'relu'))
    model.add(Dense(units = 1, activation='sigmoid'))

    model.compile(optimizer = 'adam', 
                  loss = 'mean_squared_error',
                  metrics=['accuracy'])

    history = model.fit(train_data, train_labels, 
                        batch_size = 20, epochs = 100,
                        validation_data=(test_data, test_labels),
                        verbose=False)

    # now make a prediction and clip the values to 0 or 1 as in the original code
    raw_prediction = model.predict(test_data)
    prediction = [value[0] for value in np.where(raw_prediction >= 0.5, 1, 0)]
    
    return prediction

predicted_values = simple_classification_neural_network(train_measures, test_measures, 'self_exclude')

test_labels = test_measures['self_exclude']
from sklearn import metrics
import pandas as pd
performance('nn_r', test_labels, predicted_values)<jupyter_output><empty_output><jupyter_text>Support Vector Machines (SVMs)The following cell uses [scikit-learn's SVM](https://scikit-learn.org/stable/modules/classes.htmlmodule-sklearn.svm) methods to create and trains some SVM's. The original paper uses Dimitriadou et al's [implementations in R described here](https://www.researchgate.net/profile/Friedrich_Leisch/publication/221678005_E1071_Misc_Functions_of_the_Department_of_Statistics_E1071_TU_Wien/links/547305880cf24bc8ea19ad1d/E1071-Misc-Functions-of-the-Department-of-Statistics-E1071-TU-Wien.pdf).  Random ForestThis section implements [scikit-learn's ensemble methods](https://scikit-learn.org/stable/modules/ensemble.htmlforest) to create random forests for classification and regression  All Methods Together<jupyter_code>def performance(method, actual, predicted):
    # resources:
    # describes odds ratio and precision equations
    # https://cran.r-project.org/web/packages/ROCR/ROCR.pdf
    # describes sklearn's confusion matrix
    # https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html#sklearn-metrics-confusion-matrix

    result = metrics.classification_report(actual, y_pred=predicted, output_dict=True)
    sensitivity = result['1']['recall']
    specificity = result['0']['recall']
    accuracy = result['accuracy']
    fpr, tpr, thresholds = metrics.roc_curve(actual, predicted)
    auc = metrics.auc(fpr, tpr)
    confusion_matrix = metrics.confusion_matrix(actual, predicted)
    tn, fp, fn, tp = confusion_matrix.ravel()
    
    # odds ratio is (tp x tn)/(fp x fn)
    odds_ratio = 0
    if fp != 0 and fn != 0:
        odds_ratio = (tp * tn)/(fp * fn)
        
    # precision is tp / (tp + fp)
    precision = tp / (tp + fp)

    metrics_df = pd.DataFrame()
    metrics_df['sensitivity'] = [round(sensitivity, 3)]
    metrics_df['specificity'] = [round(specificity, 3)]
    metrics_df['accuracy'] = [round(accuracy, 3)]
    metrics_df['precision'] = [round(precision, 3)]
    metrics_df['auc'] = [round(auc, 3)]
    metrics_df['odds_ratio'] = [round(odds_ratio, 3)]
    metrics_df.index = [method]
    return metrics_df

log_r = logistic_regression(train_measures, test_measures, 'self_exclude')
lasso_l = lasso_logistic_regression(train_measures, test_measures, 'self_exclude')
nn_r = shallow_neural_network(train_measures, test_measures, 'self_exclude')
nn_c = shallow_neural_network(train_measures, test_measures, 'self_exclude')
svm_e = svm_eps_regression(train_measures, test_measures, 'self_exclude')
svm_c = svm_c_classification(train_measures, test_measures, 'self_exclude')
svm_o = svm_one_classification(train_measures, test_measures, 'self_exclude')
rf_r = rf_regression(train_measures, test_measures, 'self_exclude')
rf_c = rf_classification(train_measures, test_measures, 'self_exclude')

test_labels = test_measures['self_exclude']
all_results = [
    performance('Logistic Regression', test_labels, log_r),
    performance('Lasso Logistic Regression', test_labels, lasso_l),
    performance('NN Regression', test_labels, nn_r),
    performance('Same NN Again', test_labels, nn_c),
    performance('SVM eps-Regression', test_labels, svm_e),
    performance('SVM c-Classification', test_labels, svm_c),
    performance('SVM one-Classification', test_labels, svm_o),
    performance('RF Regression', test_labels, rf_r),
    performance('RF Classification', test_labels, rf_c)
]

all_results_df = pd.concat(all_results)
display(all_results_df)<jupyter_output><empty_output><jupyter_text>Visualisations<jupyter_code>import matplotlib.pyplot as plt

def plot_individual(measures_table, player_id):
    
    measure_cols = measures_table.columns[1:]
    player = measures_table[measures_table['player_id'] == player_id]
    
    for measure in measure_cols:
        plt.figure(figsize=[4,2])
        hist = plt.hist(measures_table[measure].values, bins=25, alpha=0.5)
        plt.plot([player[measure].values[0], player[measure].values[0]], [0, hist[0].max()], label=measure, color='black')
        plt.legend()
        plt.show()
    
plot_individual(philander_data, 1324368)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup