<jupyter_start><jupyter_text>We begin by simply loading in the dataset, it is unprocessed destr. structural data, along with the Cognition Fluid Composite Uncorrected Standard Score.<jupyter_code>data = pd.read_csv('Data/Raw_Cognition_data.csv')<jupyter_output><empty_output><jupyter_text>We want to ensure that the target variable does not contain any horrendeous outliers.<jupyter_code>print(np.min(data.score), np.max(data.score))
sns.distplot(data.score, bins=30)
plt.show()
X = np.array(data.drop('score', axis=1))
y = np.array(data.score)
print(np.shape(X), np.shape(y))
X = X.astype(np.float32)
y = y.astype(np.float32)
scaler = StandardScaler()
y = scaler.fit_transform(y.reshape(-1, 1))
y = y.squeeze()
X = winsorize(X, (.02), axis=0).data<jupyter_output><empty_output><jupyter_text>Preform the train, test split<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
X_train.shape, X_test.shape, y_train.shape, y_test.shape<jupyter_output><empty_output><jupyter_text>Would next want to consider any windsorizing, or log/box cox transform for dealing with skewed features Next the different options for scaling<jupyter_code>#For now just
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup