<jupyter_start><jupyter_text>P1_Data_Preprocessing<jupyter_code># Import the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# import the dataset
dataset = pd.read_csv('datasets/Data.csv')
# Overview of datset
dataset.head()<jupyter_output><empty_output><jupyter_text>So there are 3 independent variables and 1 dependent variable.<jupyter_code># Now create matrices of independent and dependent varaibles.
X = dataset.iloc[:, :-1].values
# The first ':' left of ',' means we are taking all the rows
# :-1 means we take all the columns except the last column
X
Y = dataset.iloc[:, 3].values
Y<jupyter_output><empty_output><jupyter_text>Handle Missing data<jupyter_code># very simple missing data handling methode is mean of columns 
from sklearn.preprocessing import Imputer
# From preprocessing library import imputer class
imputer = Imputer(missing_values = 'NaN', strategy = "mean", axis=0)
# axis = 0 means take mean across columns. for rows axis=1
imputer.fit(X[:, 1:3])
# We are not imputing the whole dataset as there are missin
# values only in 2nd and 3rd column.
# So acc. to index starting from zero, the two columns are 
# 1 and 2. 
imputer = imputer.fit(X[:, 1:3])
X[:, 1:3] = imputer.transform(X[:, 1:3])
# Now replace the missing data with the mean
# Now check X
X<jupyter_output><empty_output><jupyter_text>Handle Categorical Data As seen earlier Country and Purchased are categorical variables Encode the categorical variables into numbers<jupyter_code>from sklearn.preprocessing import LabelEncoder
labelencoder_X = LabelEncoder()
X[:, 0] = labelencoder_X.fit_transform(X[:, 0])
X<jupyter_output><empty_output><jupyter_text>But there is problem here, the mathmatical model may see 2>1 instead of taking them as labels. So we are going to useone hot encoder.<jupyter_code>from sklearn.preprocessing import OneHotEncoder
onehotencoder = OneHotEncoder(categorical_features = [0])
# this means which column you are taking as having 
# categories. Here 1st column i.e. Column 0
# Now our object is ready, lets replace
X = onehotencoder.fit_transform(X).toarray()
X<jupyter_output><empty_output><jupyter_text>Now we will encode the dependent variable. We dont need to hot encode this<jupyter_code>labelencoder_Y = LabelEncoder()
Y = labelencoder_Y.fit_transform(Y)
Y<jupyter_output><empty_output><jupyter_text>Split the dataset into train and test<jupyter_code>from sklearn.cross_validation import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2, random_state = 0)
# You can use random_state=0 if you want the exactly same result as mine

X_train
X_test
Y_train
Y_test<jupyter_output><empty_output><jupyter_text>Feature Scaling Columns Age and Salary are not in same scale which may create problems in ML models. Types- Standardisation and Normalisation<jupyter_code>from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
# for training set fit and transform, for test 
# set only transform
X_test = sc_X.transform(X_test)

X_train
X_test<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup