<jupyter_start><jupyter_text>Quick concept of Machine LearningMost machine learning algorithms share the same pipeline structure as the one shown below:        Scikit-learn BasicsThis session will cover the basics of Scikit-Learn, a popular package containing a collection of tools for machine learning written in Python. See more at http://scikit-learn.org.  Loading an example datasetLet's start by loading some [pre-existing datasets](http://scikit-learn.org/stable/modules/classes.htmlmodule-sklearn.datasets) in the scikit-learn, which comes with a few standard datasets. For example, the [iris](https://en.wikipedia.org/wiki/Iris_flower_data_set) and [digits](http://archive.ics.uci.edu/ml/datasets/Pen-Based+Recognition+of+Handwritten+Digits) datasets for classification and the [boston house prices](http://archive.ics.uci.edu/ml/datasets/Housing) dataset for regression. Using these existing datasets, we can easily test the algorithms that we are interested in. A dataset is a dictionary-like object that holds all the data and some metadata about the data. This data is stored in the .data member, which is a n_samples, n_features array. In the case of supervised problem, one or more response variables are stored in the .target member. More details on the different datasets can be found in the dedicated section.  Load irisThe iris dataset consists of 50 samples from each of three species of Iris (Iris setosa, Iris virginica and Iris versicolor). Four features were measured from each sample: the length and the width of the sepals and petals, in centimetres.| [![Iris Setosa](https://upload.wikimedia.org/wikipedia/commons/5/56/Kosaciec_szczecinkowaty_Iris_setosa.jpg)](https://en.wikipedia.org/wiki/Iris_setosa)  | [![Iris Virginica](https://upload.wikimedia.org/wikipedia/commons/thumb/9/9f/Iris_virginica.jpg/1920px-Iris_virginica.jpg)](https://en.wikipedia.org/wiki/Iris_virginica) | [![Iris Versicolor](https://upload.wikimedia.org/wikipedia/commons/2/27/Blue_Flag%2C_Ottawa.jpg)](https://en.wikipedia.org/wiki/Iris_versicolor) ||:---:|:---:|:---:|| Iris Setosa| Iris Virginica| Iris Versicolor|<jupyter_code>import warnings
warnings.filterwarnings("ignore")

from sklearn import datasets
import matplotlib.pyplot as plt
import numpy as np

plt.style.use('seaborn-poster')
%matplotlib inline
iris = datasets.load_iris()
print(iris.feature_names)
# only print the first 10 samples
print(iris.data[:10])
print('We have %d data samples with %d features'%(iris.data.shape[0], iris.data.shape[1]))<jupyter_output>['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
[[ 5.1  3.5  1.4  0.2]
 [ 4.9  3.   1.4  0.2]
 [ 4.7  3.2  1.3  0.2]
 [ 4.6  3.1  1.5  0.2]
 [ 5.   3.6  1.4  0.2]
 [ 5.4  3.9  1.7  0.4]
 [ 4.6  3.4  1.4  0.3]
 [ 5.   3.4  1.5  0.2]
 [ 4.4  2.9  1.4  0.2]
 [ 4.9  3.1  1.5  0.1]]
We have 150 data samples with 4 features<jupyter_text>The data is always a 2D array, shape (n_samples, n_features), although the original data may have had a different shape. The following prints out the target names and the representatoin of the target using 0, 1, 2. Each of them represent a class.<jupyter_code>print(iris.target_names)
print(set(iris.target))<jupyter_output>['setosa' 'versicolor' 'virginica']
{0, 1, 2}<jupyter_text>Load DigitsThis dataset is made up of 1797 8x8 images. Each image, like the ones shown below, is of a hand-written digit. In order to utilize an 8x8 figure like this, we’d have to first transform it into a feature vector with length 64.<jupyter_code>digits = datasets.load_digits()
print('We have %d samples'%len(digits.target))
print(digits.data)
print('The targets are:')
print(digits.target_names)<jupyter_output>[[  0.   0.   5. ...,   0.   0.   0.]
 [  0.   0.   0. ...,  10.   0.   0.]
 [  0.   0.   0. ...,  16.   9.   0.]
 ..., 
 [  0.   0.   1. ...,   6.   0.   0.]
 [  0.   0.   2. ...,  12.   0.   0.]
 [  0.   0.  10. ...,  12.   1.   0.]]
The targets are:
[0 1 2 3 4 5 6 7 8 9]<jupyter_text>In the digits, each original sample is an image of shape (8, 8) and it is flattened into a 64 dimension vector here.<jupyter_code>print(digits.data.shape)
## plot the first 64 samples, and get a sense of the data
fig = plt.figure(figsize = (8,8))
fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)
for i in range(64):
    ax = fig.add_subplot(8, 8, i+1, xticks=[], yticks=[])
    ax.imshow(digits.images[i],cmap=plt.cm.binary,interpolation='nearest')
    ax.text(0, 7, str(digits.target[i]))<jupyter_output><empty_output><jupyter_text>Load Boston Housing DataThe Boston housing dataset reports the median value of owner-occupied homes in various places in the Boston area, together with several variables which might help to explain the variation in median value, such as Crime (CRIM), areas of non-retail business in the town (INDUS), the age of people who own the house (AGE), and there are many other attributes that you can find the details [here](https://archive.ics.uci.edu/ml/datasets/housing).<jupyter_code>boston = datasets.load_boston()
print(boston.DESCR)
boston.feature_names
# let's just plot the average number of rooms per dwelling with the price
plt.figure(figsize = (10,8))
plt.plot(boston.data[:,5], boston.target, 'o')
plt.xlabel('Number of rooms')
plt.ylabel('Price (thousands)')<jupyter_output><empty_output><jupyter_text>The Scikit-learn Estimator ObjectEvery algorithm is exposed in scikit-learn via an ''Estimator'' object. For instance a linear regression is implemented as so:<jupyter_code>from sklearn.linear_model import LinearRegression<jupyter_output><empty_output><jupyter_text>**Estimator parameters**: All the parameters of an estimator can be set when it is instantiated, and have suitable default values:<jupyter_code># you can check the parameters as
LinearRegression?
# let's change one parameter
model = LinearRegression(normalize=True)
print(model.normalize)
print(model)<jupyter_output>LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=True)<jupyter_text>**Estimated Model parameters**: When data is *fit* with an estimator, parameters are estimated from the data at hand. All the estimated parameters are attributes of the estimator object ending by an underscore:  Simple regression problemLet's fit a simple linear regression model to see what is the sklearn API looks like. We use a very simple datasets with 10 samples with added in noise.<jupyter_code>x = np.arange(10)
y = 2 * x + 1
plt.figure(figsize = (10,8))
plt.plot(x,y,'o')
# generate noise between -1 to 1
# this seed is just to make sure your results are the same as mine
np.random.seed(42)
noise = 2 * np.random.rand(10) - 1

# add noise to the data
y_noise = y + noise
plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')
# The input data for sklearn is 2D: (samples == 10 x features == 1)
X = x[:, np.newaxis]
print(X)
print(y_noise)
# model fitting is via the fit function
model.fit(X, y_noise)
# underscore at the end indicates a fit parameter
print(model.coef_)
print(model.intercept_)
# then we can use the fitted model to predict new data
predicted = model.predict(X)
plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')
plt.plot(x,predicted, label = 'Prediction')
plt.legend()<jupyter_output><empty_output><jupyter_text>ExerciseIn the next section, we will use support vector machine (SVM) to do a classification problem, have a look of the [sklearn API](http://scikit-learn.org/stable/modules/classes.html), and find out which class we will use for a classification problem using SVM. (hint: we will use the one with C support).<jupyter_code>%load ../solutions/solution_01.py<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup