<jupyter_start><jupyter_text>Copyright 2020 <NAME>, <NAME>, <NAME> and made available under [CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0) for text and [Apache-2.0](http://www.apache.org/licenses/LICENSE-2.0) for code.  $K$-Nearest Neighbor Regression Unlike clustering, where the goal is to uncover latent patterns, and classification, where the goal is to predict a class label, $K$-nearest neighbor (KNN) regression is a technique that is used to predict a numeric value.KNN regression is a _non-parametric_ regression method and is simple to understand and implement.Non-parametric methods are statistical models that do not assume the data come from a particular probability distribution.Like the KNN classifier, KNN regression is based on the idea that observation vectors that are close to each other in space tend to be similar. What you will learnIn the sections that follow you will learn about KNN regression and how it can be used to model your data for prediction. We will study the following:- KNN regression- Choosing K When to use KNN regressionKNN regression is a non-parametric alternative to simple or multiple linear regression.It is useful in situations that involve response variables that are not normally distributed and when the relationship between the predictors and response is not linear. How it works KNN regression predicts the numeric value of a new datapoint as the *average distance* between the datapoint and its $K$ nearest neighbors.We illustrate this using the data in the table below on home loans, age, and House Price Index (HPI).The two numerical variables, `Age` and `Loan`, are used to predict the `HPI` and so they are called **predictors**. The `HPI` being predicted is called the **response** or **dependent variable**. We observe the data in the table, and using these predictors, we would like to predict the `HPI` for an individual 47 years old with home loan of $150,000. In the table below, the `Distance` column is calculated as the Euclidean distance between each predictor datapoint and the values of age (47) and loan (150,000) that we want to predict the HPI for.The formula looks like: $$Distance = \sqrt{(\text{Age}-47)^2 + (\text{Loan}-150000)^2}$$ AgeLoanHPIDistance47150,745244 74534198,33326948,33342187,33815437,33844139,50517310,49541142,1132567,88744102,83927447,16132120,02019129,98049191,77211041,77239142,664 1387,33645165,81723815,81734163,72814013,72834143,132 1986,868 -->    Training DataAge=47 and Loan=150,000          | Age | Loan    | HPI ||-----|---------|-----|| **47**  | **150,745** |  **244** || 34  | 198,333 | 269 || 42  | 187,338 | 154 || 44  | 139,505 | 173 || 41  | 142,113 | 256 || 44  | 102,839 | 274 || 32  | 120,020 | 191 || 49  | 191,772 | 110 || **39**  | **142,664** |  **138** || 45  | 165,817 | 238 || 34  | 163,728 | 140 || **34**  | **143,132** |  **198** |    | Distance ||----------|| **745**      || 48,333   || 37,338   || 10,495   || 7,887    || 47,161   || 29,980   || 41,772   || **7,336**    || 15,817   || 13,728   || **6,868**    |     If we use $K=3$ then to predict the HPI for age = 47 and loan = 150,000 we average the HPI values for the 3 nearest neighbors.These are the neighbors with the smallest distance as shown in red in the table.The predicted HPI is found as $$HPI = \frac{244+198+138}{3} = 193.33$$  Residuals Typically the KNN regression is trained on a training set and the performance of the model is tested by comparing predicted values with the actual responses in the the testing set.The table below shows the performance of the KNN regression model on 3 members of the test set. AgeLoanHPI ObservedHPI PredictedResidual44107,889133222-8932127,0522071881935163,46324220735 -->Test DataKNN Regression Results       | Age | Loan   | HPI ||-----|--------|-----|| 44  | 107889 | 133 || 32  | 127052 | 207 || 35  | 163463 | 242 || HPI Predicted | Residual ||---------------|----------|| 222           | -89      || 188           | 19       || 207           | 35       |        The last column in the table titled `Residual` is computed as the Observed HPI minus the Predicted HPI and can be used to diagnose the model fit.Checking the residuals is an important part of model diagnostics.The residuals can be summarized in a single measure called the _root mean square error_ (RMSE).It is the square root of the sum of the squared residuals divided by the number of sample points.The smaller the RMSE the better the fit. The formula for the RMSE is $$RMSE = \sqrt{\sum_{i=1}^n \frac{res_i^2}{n}}$$where $res_i$ is the residual for the $i$th observation.For the testing set in this example, the RMSE is $56.29$.Alone this does not tell us anything about the fit of the model, but we can use it to compare various models.For example, if you wanted to try different values of $K$ you could use the RMSE to compare the fits.  Standardization This example highlights one issue that you may run into when you are running a KNN regression model (or KNN classifier). The magnitude (size) of the Loan variable is much bigger than the magnitude of the Age variable. This difference in magnitude can lead the distance measure to favor differences in the Loan variable and the Age variable can have much less of an impact. One solution to this problem is to standardize each variable by subtracting the sample mean, $\bar{X}$, and dividing by the sample standard deviation, $S=\sqrt{S^2}$, as $$Z_i = \frac{(X_i - \bar{X})}{S}$$and performing the KNN regression on the standardized variables.  Choosing $K$ Similar to the KNN classifier, the choice of $K$ has a big impact on the regression.When $K$ is low, the regression fit will be very choppy and gets smoother as $K$ increases.The figures below show a progression of values of $K$ from $K=1$ to $K=n=506$.For this data, there is only one predictor on the x axis.The points in blue are the predictor (x-axis) and the associated response (y-axis).The orange line is the KNN regression prediction.As you can see, in Figure 1 where $K=1$, the regression prediction is very choppy and smooths out as $K$ increases.When $K=n=506$ as in Figure 6, the prediction line is completely smooth and horizontal.The choice of $K$ is a compromise between accurate but choppy predictions (low $K$) and less accurate but less variable predictions (high $K$). &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;  **k = 1****Figure 1** **Figure 2** **Figure 3** **Figure 4** **Figure 5**  How does the KNN algorithm work? We have now seen that the KNN algorithm can be used for both _classification_ and _regression_ problems.The KNN algorithm uses __feature similarity__ to predict values of new data.This means the new data is assigned a value based on how closely it resembles the closest $K$ data in the training set.  The step-by-step instructions for KNN regression is as follows: 0. Pre-process the data. The data needs to be *standardized* and split into testing and training sets.When we say standardized, we mean put onto the same scale.Otherwise variables with large magnitude (like `Loan` above) will contribute more to distance than variables with small magnitude (like `Age` above). 1. Calculate the distance between the new point and each training point.There are various methods for calculating the distance between the new data and each training data.The most common methods are _Euclidean_, _Manhattan_, and _Hamming_.We will use the Euclidean Distance calculated as the square root of the sum of the squared differences between a new point and an existing point.  2. The closest $K$ data points are selected based on the calculated distance. In general, the KNN model overfits on the training data for a very low value of $K$ and performs poorly on both training and validation sets for a very high value of $K$.The _elbow curve_ and _grid search techniques_ are methods to help you find the __best__ values for $K$. 3. Average the response of the closest $K$ data points to get the final prediction for the new data.   Example: House prices To start to perform KNN regression, we need data. The `boston` dataset was collected in the 1970s and has been used to examine the relationship between clean air and house prices.It contains 13 variables:| Variable | Type | Description ||:----|:-----|:----------||crim | Ratio | per capita crime rate by town | |zn | Ratio | proportion of residential land zoned for lots over 25,000 sq. ft. | |indus | Ratio | proportion of non-retain business acres per town | |chas | Nominal (binary) | Charles River dummy variable (=1 if tract bounds river, =0 otherwise) | |nox | Ratio | nitrogen oxides concentration (parts per 10 million) | |rm | Ratio | average number of rooms per dwelling | |age | Ratio | proportion of owner-occupied units built prior to 1940 | |dis | Ratio | weighted mean of distances to five Boston employment centers | |rad | Ordinal | index of accessibility to radial highways | |tax | Ratio | full-value property tax rate per \$10,000 | |ptratio | Ratio | pupil-teacher ratio by town | |lstat | Ratio | percent lower status of population (defined as non-high school graduate, manual labor) | |medv | Ratio | median value of owner-occupied homes in $1000s |     Source: This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.     Load the dataFirst import `pandas` so we can work with dataframes:- `import pandas as pd`<jupyter_code>import pandas as pd

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="A]5Vf/x,=8dk=KKk3OrJ">pd</variable></variables><block type="importAs" id="/k@:}OLWtAh)hpx[.2^]" x="150" y="308"><field name="libraryName">pandas</field><field name="libraryAlias" id="A]5Vf/x,=8dk=KKk3OrJ">pd</field></block></xml><jupyter_output><empty_output><jupyter_text>Load the dataframe:- Create variable `dataframe`- Set it to `with pd do read_csv using "datasets/boston.csv"`- `dataframe` (to display)<jupyter_code>dataframe = pd.read_csv('datasets/boston.csv')

dataframe

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="$%drMkQ:qhLh-L8dv:ow">dataframe</variable><variable id="A]5Vf/x,=8dk=KKk3OrJ">pd</variable></variables><block type="variables_set" id="FJGldw(qFp?V=E]uaN)4" x="31" y="222"><field name="VAR" id="$%drMkQ:qhLh-L8dv:ow">dataframe</field><value name="VALUE"><block type="varDoMethod" id="4s18v%0iqE%_l@{fLYCo"><field name="VAR" id="A]5Vf/x,=8dk=KKk3OrJ">pd</field><field name="MEMBER">read_csv</field><data>pd:read_csv</data><value name="INPUT"><block type="text" id="tN4Jmaffy/={ZFo;uu;W"><field name="TEXT">datasets/boston.csv</field></block></value></block></value></block><block type="variables_get" id="yN6MT1B[)Pk)cEECN_YZ" x="31" y="286"><field name="VAR" id="$%drMkQ:qhLh-L8dv:ow">dataframe</field></block></xml><jupyter_output><empty_output><jupyter_text>Visualize the dataLet's look at the relationship between percent of lower status (`lstat`) and the value of a home (`medv`). First, import `plotly` for plotting:- `import plotly.express as px`<jupyter_code>import plotly.express as px

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="x2E1re-+t1/@.:kt|~XV">px</variable></variables><block type="importAs" id="+cd]v=]w[Y{ypm^J_~c}" x="198" y="290"><field name="libraryName">plotly.express</field><field name="libraryAlias" id="x2E1re-+t1/@.:kt|~XV">px</field></block></xml><jupyter_output><empty_output><jupyter_text>Make the scatterplot:- `with px do scatter using` a list containing    - `dataframe`    - freestyle `x="lstat"`    - freestyle `y="medv"`<jupyter_code>px.scatter(dataframe, x="lstat", y="medv")

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="x2E1re-+t1/@.:kt|~XV">px</variable><variable id="$%drMkQ:qhLh-L8dv:ow">dataframe</variable></variables><block type="varDoMethod" id="#{`uqQJ7o((KWM@lD9qM" x="189" y="257"><field name="VAR" id="x2E1re-+t1/@.:kt|~XV">px</field><field name="MEMBER">scatter</field><data>px:scatter</data><value name="INPUT"><block type="lists_create_with" id="06@|Mq6tTAw|c;Zqa,w0"><mutation items="3"></mutation><value name="ADD0"><block type="variables_get" id="%+W1]bM6TMW-dcT(D)!q"><field name="VAR" id="$%drMkQ:qhLh-L8dv:ow">dataframe</field></block></value><value name="ADD1"><block type="dummyOutputCodeBlock" id="_db~Ig?DLuc}U+3QVLY+"><field name="CODE">x="lstat"</field></block></value><value name="ADD2"><block type="dummyOutputCodeBlock" id="}PsvFr/L@QtObEW4I4*|"><field name="CODE">y="medv"</field></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>There appears to be a non-linear relationship between `lstat` and `medv`.  Prepare train/test setsNow, we need to separate the dataset into predictor features and corresponding response (`medv`). In other words, split the dataframe into training data and testing data.Let's start with everything but the response.We'll use a new function called `drop` to remove `medv`.This is faster than listing all the columns we want to keep:- Create variable `X`- Set it to `with dataframe do drop using` a list containing    - freestyle `columns=["medv"]`- `X` (to display)<jupyter_code>X = dataframe.drop(columns=["medv"])

X

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id=".X4`4]:_GBz1DeII8[o2">X</variable><variable id="$%drMkQ:qhLh-L8dv:ow">dataframe</variable></variables><block type="variables_set" id="j12HEn?u}2$P-S)Ax6d(" x="-16" y="265"><field name="VAR" id=".X4`4]:_GBz1DeII8[o2">X</field><value name="VALUE"><block type="varDoMethod" id="PMH^/O2y?rD(Y}r-N6eY"><field name="VAR" id="$%drMkQ:qhLh-L8dv:ow">dataframe</field><field name="MEMBER">drop</field><data>dataframe:drop</data><value name="INPUT"><block type="lists_create_with" id="bGn#S3b1DV-0-egwl87D"><mutation items="1"></mutation><value name="ADD0"><block type="dummyOutputCodeBlock" id="8N-YF(7ms%)oox.~Z;b/"><field name="CODE">columns=["medv"]</field></block></value></block></value></block></value></block><block type="variables_get" id="*Q(nsPMh9xg`HceqC3eN" x="-11" y="350"><field name="VAR" id=".X4`4]:_GBz1DeII8[o2">X</field></block></xml><jupyter_output><empty_output><jupyter_text>Save a dataframe with just `medv` in `Y`:- Create variable `Y`- Set it to `dataframe [ ] ` containing the following in a list    - `"medv"`- `Y` (to display)<jupyter_code>Y = dataframe[['medv']]

Y

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="fnSh@+0BWBw|hcm3M|WE">Y</variable><variable id="$%drMkQ:qhLh-L8dv:ow">dataframe</variable></variables><block type="variables_set" id="_:J1([xryT4i+F7k{#[%" x="17" y="175"><field name="VAR" id="fnSh@+0BWBw|hcm3M|WE">Y</field><value name="VALUE"><block type="indexer" id="22hOF=T:mHvJrMybHkPs"><field name="VAR" id="$%drMkQ:qhLh-L8dv:ow">dataframe</field><value name="INDEX"><block type="lists_create_with" id="qT2G_{7q~*L;[5j:vtKq"><mutation items="1"></mutation><value name="ADD0"><block type="text" id="=UVLwuCW$1d9ZJ]6HF_,"><field name="TEXT">medv</field></block></value></block></value></block></value></block><block type="variables_get" id="pog__n6Jf{VK_sNEB)B$" x="17" y="239"><field name="VAR" id="fnSh@+0BWBw|hcm3M|WE">Y</field></block></xml><jupyter_output><empty_output><jupyter_text>Next we are going to split our `X` and `Y` into train and test sets. That means we'll end up with four data frames: `Xtrain`, `Xtest`, `Ytrain`, `Ytest`.First, the `sklearn` import for splitting:- `import sklearn.model_selection as model_selection`<jupyter_code>import sklearn.model_selection as model_selection

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="uASGz64Zb$AOvQyV4pRj">model_selection</variable></variables><block type="importAs" id="sN1YO5FEzpHyxb31@j,Z" x="16" y="10"><field name="libraryName">sklearn.model_selection</field><field name="libraryAlias" id="uASGz64Zb$AOvQyV4pRj">model_selection</field></block></xml><jupyter_output><empty_output><jupyter_text>Now do the splits:- Create variable `splits`- Set it to `with model_selection do train_test_split using` a list containing    - `X` (the features in an array)    - `Y` (the labels in an array)    - freestyle `test_size=0.2` (the proportion of the dataset to include in the test split)<jupyter_code>splits = model_selection.train_test_split(X, Y, test_size=0.2)

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable><variable id="uASGz64Zb$AOvQyV4pRj">model_selection</variable><variable id=".X4`4]:_GBz1DeII8[o2">X</variable><variable id="fnSh@+0BWBw|hcm3M|WE">Y</variable></variables><block type="variables_set" id="oTGRJ#{R!U^we@Bl@pkT" x="31" y="224"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field><value name="VALUE"><block type="varDoMethod" id="f?j@ker(a#hJv;Nh)IGX"><field name="VAR" id="uASGz64Zb$AOvQyV4pRj">model_selection</field><field name="MEMBER">train_test_split</field><data>model_selection:train_test_split</data><value name="INPUT"><block type="lists_create_with" id="er6r2]}|nA;1;}VsM5I7"><mutation items="3"></mutation><value name="ADD0"><block type="variables_get" id=".mm}`*H4)i%Eq5z={e-$"><field name="VAR" id=".X4`4]:_GBz1DeII8[o2">X</field></block></value><value name="ADD1"><block type="variables_get" id="I3dOV;CPBf^~E%BvgthZ"><field name="VAR" id="fnSh@+0BWBw|hcm3M|WE">Y</field></block></value><value name="ADD2"><block type="dummyOutputCodeBlock" id="@Hg?ib/!8fH$;f3pWJy2"><field name="CODE">test_size=0.2</field></block></value></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>You may have noticed we haven't scaled (standardized) the predictor features yet.We'll do that next.  KNN regression modelWe need to import libraries for:- KNN regression- Mean squared error- RavelAs well as libraries we need to standardize:- Scale- PipelineSo do the following imports:- `import sklearn.neighbors as neighbors`- `import sklearn.metrics as metrics`- `import numpy as np`- `import sklearn.preprocessing as pp`- `import sklearn.pipeline as pipe`<jupyter_code>import sklearn.neighbors as neighbors
import sklearn.metrics as metrics
import numpy as np
import sklearn.preprocessing as pp
import sklearn.pipeline as pipe

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="MvUG^P$o3=I*6/67?Rk:">neighbors</variable><variable id=")}+w@N9iB/j=:+PLkMv8">metrics</variable><variable id="Zhzp)s*VL?V@ES3(j:*b">np</variable><variable id=".T`xqZHAu7r0$MQ4s=Yg">pp</variable><variable id="=z9KfSBWpoDS,oj-?yx,">pipe</variable></variables><block type="importAs" id="Jkis0rqxTm[rs0.yA(O7" x="183" y="249"><field name="libraryName">sklearn.neighbors</field><field name="libraryAlias" id="MvUG^P$o3=I*6/67?Rk:">neighbors</field><next><block type="importAs" id="*G_SVgZ;}hIr,Hi1~$Z6"><field name="libraryName">sklearn.metrics</field><field name="libraryAlias" id=")}+w@N9iB/j=:+PLkMv8">metrics</field><next><block type="importAs" id="1ODp#_M`@7Fj/@US}H%#"><field name="libraryName">numpy</field><field name="libraryAlias" id="Zhzp)s*VL?V@ES3(j:*b">np</field><next><block type="importAs" id="/YAkt4vqDQp/.5;pPcY."><field name="libraryName">sklearn.preprocessing</field><field name="libraryAlias" id=".T`xqZHAu7r0$MQ4s=Yg">pp</field><next><block type="importAs" id="O15G]dI#1oM;`7^r@Pr,"><field name="libraryName">sklearn.pipeline</field><field name="libraryAlias" id="=z9KfSBWpoDS,oj-?yx,">pipe</field></block></next></block></next></block></next></block></next></block></xml><jupyter_output><empty_output><jupyter_text>Before we can train the KNN regression model, we need to scale the data.To do this easily, we introduce a new concept, **pipelines**.A pipeline is a series of steps, or stages, that process your data.You can think of it like an assembly line, with each stage adding something (or taking something away).The reason we need a pipeline is because when we scale our data, we need to scale not only the training set but also the testing set, and we need to scale them all the same way (so they match).Pipelines make this easy because we can wrap the scaler and our KNN model in the same pipeline and then forget about it.Let's make a pipeline with both a scaler and KNN regression:- Create variable `std_clf`- Set it to `with pipe do make_pipeline using` a list containing    - `with pp create StandardScaler using`    - `with neighbors create KNeighborsRegressor using` a list containing        - freestyle `n_neighbors=10`<jupyter_code>std_clf = pipe.make_pipeline(pp.StandardScaler(), neighbors.KNeighborsRegressor(n_neighbors = 10))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="`P)`+jl|;h{.[r}{78q6">std_clf</variable><variable id="=z9KfSBWpoDS,oj-?yx,">pipe</variable><variable id="MvUG^P$o3=I*6/67?Rk:">neighbors</variable><variable id=".T`xqZHAu7r0$MQ4s=Yg">pp</variable></variables><block type="variables_set" id="}81D/tZY#o}$E:M}:u4x" x="77" y="398"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><value name="VALUE"><block type="varDoMethod" id="(Noh/e~gvTM3-4|;g]s["><field name="VAR" id="=z9KfSBWpoDS,oj-?yx,">pipe</field><field name="MEMBER">make_pipeline</field><data>pipe:make_pipeline</data><value name="INPUT"><block type="lists_create_with" id="(Q-}ch^M{g#1!2LN+l,@"><mutation items="2"></mutation><value name="ADD0"><block type="varCreateObject" id="e9G_EPu`q{*0`$!~:jh~"><field name="VAR" id=".T`xqZHAu7r0$MQ4s=Yg">pp</field><field name="MEMBER">StandardScaler</field><data>pp:StandardScaler</data></block></value><value name="ADD1"><block type="varCreateObject" id="ar7keIh-Yv)+b+#Edsp_"><field name="VAR" id="MvUG^P$o3=I*6/67?Rk:">neighbors</field><field name="MEMBER">KNeighborsRegressor</field><data>neighbors:KNeighborsRegressor</data><value name="INPUT"><block type="lists_create_with" id="Fdc{)AkUpH{!v_dA{/89"><mutation items="1"></mutation><value name="ADD0"><block type="dummyOutputCodeBlock" id="Q!`Pnb+=Gq!aL4Xa}S{z"><field name="CODE">n_neighbors = 10</field></block></value></block></value></block></value></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>Because our KNN regression model is inside the pipeline, we can go ahead and train it.In fact, we can treat the whole pipeline like it's a classifier:-  `with std_clf do fit using` a list containing    - `in list splits get  1` (in menu LISTS)    - `with np do ravel using` a list containing        - `in list splits get  3` (in menu LISTS)        Remember the first and third elements in `splits` are `Xtrain` and `Ytrain`, respectively.<jupyter_code>std_clf.fit(splits[0], np.ravel(splits[2]))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="`P)`+jl|;h{.[r}{78q6">std_clf</variable><variable id="Zhzp)s*VL?V@ES3(j:*b">np</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable></variables><block type="varDoMethod" id="Z)$q-zn.KTC{+^l-wH6u" x="259" y="169"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><field name="MEMBER">fit</field><data>std_clf:fit</data><value name="INPUT"><block type="lists_create_with" id="e_B;36VOJ^lH70V=aWY}"><mutation items="2"></mutation><value name="ADD0"><block type="lists_getIndex" id="C#,#1*rEm+]qEx?L1x[L"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="b_Sz{9#d7d=ystO|k?l_"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id=":s{r1~S,,@.CSh#9`$R;"><field name="NUM">1</field></block></value></block></value><value name="ADD1"><block type="varDoMethod" id="zYBlZ,!^P^%Ib5R@Ch.H"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="lists_create_with" id="9s({WSn={~Ink.5O+6Cc"><mutation items="1"></mutation><value name="ADD0"><block type="lists_getIndex" id="^)UBt0jM;BnGmWeG7pw*"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="4Vo!*g]qQ=D}XtD2i39/"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="RysCD3.C27sBxztz(T}2"><field name="NUM">3</field></block></value></block></value></block></value></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>The final step is to predict house prices (median value of owner-occupied homes, in $1000s) using the test data:- Create variable `predictions`- Set it to `with std_clf do predict using` a list containing    - `in list splits get  2` - `predictions` (to display)<jupyter_code>predictions = std_clf.predict(splits[1])

predictions

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="5lkT$zi8W,`TXDny?Cv7">predictions</variable><variable id="`P)`+jl|;h{.[r}{78q6">std_clf</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable></variables><block type="variables_set" id="NHw$/HH988vNLbZgn)CM" x="88" y="212"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field><value name="VALUE"><block type="varDoMethod" id="N}3ds6:i%0MtTA:(2im4"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><field name="MEMBER">predict</field><data>std_clf:predict</data><value name="INPUT"><block type="lists_create_with" id="3Ru6U*^.a`oD7$bu/I%y"><mutation items="1"></mutation><value name="ADD0"><block type="lists_getIndex" id="@lpyN+:CEcPQ#Q:Svm|9"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="ng))4fZyb@U1|eswo1}:"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="Lo)w=2LL|Tf-L/gkeTdT"><field name="NUM">2</field></block></value></block></value></block></value></block></value></block><block type="variables_get" id="I#4y[,+*#I5s;b;h3o/M" x="75" y="307"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field></block></xml><jupyter_output><empty_output><jupyter_text>Assessing the model Use the `sqrt` of `mean_squared_error` to calculate the root mean squared error (RMSE) between `Ytest` and our predictions:- `square root` of `with metrics do mean_squared_error using` a list containing:    - `in list splits get  4`     - `predictions`    Remember `square root` lives in the MATH menu.<jupyter_code>import math

math.sqrt(metrics.mean_squared_error(splits[3], predictions))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id=")}+w@N9iB/j=:+PLkMv8">metrics</variable><variable id="5lkT$zi8W,`TXDny?Cv7">predictions</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable></variables><block type="math_single" id="K+~AfTut!=kX_{{*?zjq" x="76" y="293"><field name="OP">ROOT</field><value name="NUM"><shadow type="math_number" id="TwxVz{fz8[Th0JEpyxPq"><field name="NUM">9</field></shadow><block type="varDoMethod" id="dtb+F,1WedaH`AXMNxY["><field name="VAR" id=")}+w@N9iB/j=:+PLkMv8">metrics</field><field name="MEMBER">mean_squared_error</field><data>metrics:mean_squared_error</data><value name="INPUT"><block type="lists_create_with" id="-=NmFJemtj{-P/.D|`Zg"><mutation items="2"></mutation><value name="ADD0"><block type="lists_getIndex" id="%Q5nelLnQN_U[o);kAqv"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="eCZagv7%IlQ;!+X*/wn7"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="+UoB2X],lB*~xVCJOMRM"><field name="NUM">4</field></block></value></block></value><value name="ADD1"><block type="variables_get" id="Y`5[z@^QpYeXO!b/zuki"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field></block></value></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>If you have the time, try taking the scaler stage out of the pipeline, train the model, and assess it.You should find that your RMSE about doubled without the scaling.  Finding the optimal K In figures 1-5 above, we saw how different values of K affect the shape of the line in KNN regression.Both small values of K and large values of K don't seem very good.Instead we want a K that is "just right," somewhere in the middle.One way to find the optimal K is to train models for a range of K values, say 1 to 100.It would take you a long time to do this, even copy/pasting the code above!To make this kind of task doable, we're going to introduce a basic programming concept called a loop (see the LOOPS menu).But don't be fooled, even though loops are basic, they are one of the hardest things to understand in programming.The basic idea of a loop is that you create a template of the thing you want to do many times, and you put that template inside the loop.Then you have a few values that change depending on the *iteration* of the loop.Let's look a simple example, a loop that prints the numbers 1 to 3:- Get the **green** `for each item i in list` block- To its right, attach a list containing 1, 2, and 3- Inside the loop block, put `print i`<jupyter_code>for i in [1, 2, 3]:
  print(i)

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="eV4EZ~~D[TKTsoJ=Y9i/">i</variable></variables><block type="controls_forEach" id="FQ8X,Yn$Q`D}l2yQmxgn" x="99" y="276"><field name="VAR" id="eV4EZ~~D[TKTsoJ=Y9i/">i</field><value name="LIST"><block type="lists_create_with" id="gJi:5JuudMItfG.a90l7"><mutation items="3"></mutation><value name="ADD0"><block type="math_number" id="x%u{x?:sZy@{o?3py;AZ"><field name="NUM">1</field></block></value><value name="ADD1"><block type="math_number" id="8uQpfZhd]$_:!+h{X+gX"><field name="NUM">2</field></block></value><value name="ADD2"><block type="math_number" id="Y@I~L`X)g,BYB$-Yk;i_"><field name="NUM">3</field></block></value></block></value><statement name="DO"><block type="text_print" id="GOAMMS}Lq-d0X-vGEKQn"><value name="TEXT"><shadow type="text" id="$wPTCb2SBOUHe@SW]e-A"><field name="TEXT">abc</field></shadow><block type="variables_get" id="$;paAJN}M27hOi7BpPVE"><field name="VAR" id="eV4EZ~~D[TKTsoJ=Y9i/">i</field></block></value></block></statement></block></xml><jupyter_output>1
2
3<jupyter_text>In this example:- `print i` is our template- `i` changes at each iteration of the loop to the next thing in the listNext, we're going to follow the same logic, but instead of just printing inside the loop, we will build a KNN regression and print out its RMSE.<!-- First, the imports:- `import plotly.graph_objects as go` --> Here are the steps:- Get the **green** `for each item j in list` block and attach a list containing 1, 10, 25, 50, 100, and 250- Copy the entire `std_clf` block (that makes the pipeline) and put it inside the loop- Change the freestyle `n_neighbors = 10` to `n_neighbors = j`- Copy the entire `std_clf.fit` block (that trains the pipeline) attach a freestyle to the front of it (see below) and put it in the loop (this adapter block lets you but `std_clf.fit` in the loop block)![image.png](attachment:image.png)- Copy the entire `predictions` block (that does the prediction) and put it inside the loop- Create a variable `rmse` and set it to the entire `square root` block and put it inside the loop- Inside the loop do `print create text with`    - `j`    - `":"`    - `rmse`    **In each of the above steps, when you put something in the loop, it goes below the block from the previous step**<jupyter_code>import math

for j in [1, 10, 25, 50, 100, 250]:
  std_clf = pipe.make_pipeline(pp.StandardScaler(), neighbors.KNeighborsRegressor(n_neighbors = j))
  (std_clf.fit(splits[0], np.ravel(splits[2])))
  predictions = std_clf.predict(splits[1])
  rmse = math.sqrt(metrics.mean_squared_error(splits[3], predictions))
  print(''.join([str(x) for x in [j, ':', rmse]]))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="nr%W_%EA(*n$_f,PuH#c">j</variable><variable id="`P)`+jl|;h{.[r}{78q6">std_clf</variable><variable id="=z9KfSBWpoDS,oj-?yx,">pipe</variable><variable id="5lkT$zi8W,`TXDny?Cv7">predictions</variable><variable id=".T`xqZHAu7r0$MQ4s=Yg">pp</variable><variable id="MvUG^P$o3=I*6/67?Rk:">neighbors</variable><variable id="RN+myW;vx){}Q0LuFA9y">rmse</variable><variable id="Zhzp)s*VL?V@ES3(j:*b">np</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable><variable id=")}+w@N9iB/j=:+PLkMv8">metrics</variable></variables><block type="controls_forEach" id="MbWYZ/u3Q#+3*u`TM}ce" x="-3" y="182"><field name="VAR" id="nr%W_%EA(*n$_f,PuH#c">j</field><value name="LIST"><block type="lists_create_with" id="Q6_EXa41T:y(.uy[pm.V"><mutation items="6"></mutation><value name="ADD0"><block type="math_number" id="+BJ@yT0IgJORgO^/))c~"><field name="NUM">1</field></block></value><value name="ADD1"><block type="math_number" id=".~r[qy7$p-Jy_g:ACHYr"><field name="NUM">10</field></block></value><value name="ADD2"><block type="math_number" id="w!m`o}C;PB7OhYP5ZTq~"><field name="NUM">25</field></block></value><value name="ADD3"><block type="math_number" id="?!EK)M.j{u~@|;ytL|nd"><field name="NUM">50</field></block></value><value name="ADD4"><block type="math_number" id="cLtaU]#w-;S(sUq8pw/l"><field name="NUM">100</field></block></value><value name="ADD5"><block type="math_number" id="910qY[[4~4nziBPT;K+G"><field name="NUM">250</field></block></value></block></value><statement name="DO"><block type="variables_set" id="8APEX{xGs3Qz/$NzTFrY"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><value name="VALUE"><block type="varDoMethod" id="W=ep`rJjDXvsAV^R+NIG"><field name="VAR" id="=z9KfSBWpoDS,oj-?yx,">pipe</field><field name="MEMBER">make_pipeline</field><data>pipe:make_pipeline</data><value name="INPUT"><block type="lists_create_with" id="ed7Q+#-kPzf4R,8)[q_Z"><mutation items="2"></mutation><value name="ADD0"><block type="varCreateObject" id="ME+Z@t~BnN}LL22r%Hxt"><field name="VAR" id=".T`xqZHAu7r0$MQ4s=Yg">pp</field><field name="MEMBER">StandardScaler</field><data>pp:StandardScaler</data></block></value><value name="ADD1"><block type="varCreateObject" id="p=kmgsx6gSLD(RSPi]We"><field name="VAR" id="MvUG^P$o3=I*6/67?Rk:">neighbors</field><field name="MEMBER">KNeighborsRegressor</field><data>neighbors:KNeighborsRegressor</data><value name="INPUT"><block type="lists_create_with" id="%{hTJRYu`A$B-B}N|W^|"><mutation items="1"></mutation><value name="ADD0"><block type="dummyOutputCodeBlock" id="4*MO9`pDvGbjv^%|]#Jm"><field name="CODE">n_neighbors = j</field></block></value></block></value></block></value></block></value></block></value><next><block type="valueNoOutputCodeBlock" id="nCe{WO~.SVUJ*REfHN;r"><field name="CODE"></field><value name="INPUT"><block type="varDoMethod" id="Fm%sV]3?)?kc0h:9S9IQ"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><field name="MEMBER">fit</field><data>std_clf:fit</data><value name="INPUT"><block type="lists_create_with" id="bkqo.bU~bUHGJR3.PCNj"><mutation items="2"></mutation><value name="ADD0"><block type="lists_getIndex" id="c-OmE5.Qf*x_-O+oEcL("><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="xDX74mH}[mNJlpBTYt{)"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="qrLzq]pv]HBj!dNAu-9i"><field name="NUM">1</field></block></value></block></value><value name="ADD1"><block type="varDoMethod" id="?U(hh`0ZFjUnfsrL}xQK"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="lists_create_with" id="(UU{x68[{=A[p}uG##Eq"><mutation items="1"></mutation><value name="ADD0"><block type="lists_getIndex" id="kcy}-[{;=}P{onxAUnZc"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="di`_Ko4qTbeTQ^v!pa2A"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id=",i2p$Oad~vpnzpz8YDss"><field name="NUM">3</field></block></value></block></value></block></value></block></value></block></value></block></value><next><block type="variables_set" id="t4FH|^O1fvcXbQ/%eX{o"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field><value name="VALUE"><block type="varDoMethod" id="@~[XlP)bAA[7?k^=6%J2"><field name="VAR" id="`P)`+jl|;h{.[r}{78q6">std_clf</field><field name="MEMBER">predict</field><data>std_clf:predict</data><value name="INPUT"><block type="lists_create_with" id="~`MOj}QguYf}BCZOc7C!"><mutation items="1"></mutation><value name="ADD0"><block type="lists_getIndex" id="h$p4(I@O~CtLDZv)+/nX"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="Kh1qgk~(:R(}S`sfh}?{"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="jyi_*c[[~,{K#t[0Gghj"><field name="NUM">2</field></block></value></block></value></block></value></block></value><next><block type="variables_set" id="6eYPMqa.T*II`(7!/U;_"><field name="VAR" id="RN+myW;vx){}Q0LuFA9y">rmse</field><value name="VALUE"><block type="math_single" id="SI.B#46RorSW7}ZGS{D-"><field name="OP">ROOT</field><value name="NUM"><shadow type="math_number"><field name="NUM">9</field></shadow><block type="varDoMethod" id="7PlC*+UVpHU@Lyq2@_?h"><field name="VAR" id=")}+w@N9iB/j=:+PLkMv8">metrics</field><field name="MEMBER">mean_squared_error</field><data>metrics:mean_squared_error</data><value name="INPUT"><block type="lists_create_with" id="ExgoS4A$UNPym77J;1L2"><mutation items="2"></mutation><value name="ADD0"><block type="lists_getIndex" id="JoSeD*7Q$:S1sXw#)5=+"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="OR=]poUbkR)eqh!eV:XW"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="gJ*8#!h/B4O65oy_[TKL"><field name="NUM">4</field></block></value></block></value><value name="ADD1"><block type="variables_get" id="::s[{k@xAHJRWvDND!GY"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field></block></value></block></value></block></value></block></value><next><block type="text_print" id="qD^5qCGb|~Z/e)-2,/Xe"><value name="TEXT"><shadow type="text" id="jjAd~$4y5!nZc!%qRsJ}"><field name="TEXT">abc</field></shadow><block type="text_join" id="G#,=(,gYP!hQgbLLojMo"><mutation items="3"></mutation><value name="ADD0"><block type="variables_get" id="N0QQN{Ql.g=*FApk}*2b"><field name="VAR" id="nr%W_%EA(*n$_f,PuH#c">j</field></block></value><value name="ADD1"><block type="text" id="`f+]K88}p[|+.2@^Mpb9"><field name="TEXT">:</field></block></value><value name="ADD2"><block type="variables_get" id="5es9)`8wQ`Wh@fitgT3Y"><field name="VAR" id="RN+myW;vx){}Q0LuFA9y">rmse</field></block></value></block></value></block></next></block></next></block></next></block></next></block></statement></block></xml><jupyter_output>1:5.433104770845156
10:5.843666590051684
25:6.231880124196304
50:6.552589911555552
100:7.345896891943068
250:8.396098233131397<jupyter_text>The optimal K (in terms of rmse) is 10 for me, but may be different for you depending on your data split.This is an example of what is sometimes called an *elbow*: errors start and end high, but have a distinct minimum somewhere in between.  Plotting the data and the model predictionsPreviously we used a scatterplot to examine the relationship between percent of lower status (`lstat`) and the value of a home (`medv`). Although our model uses many other predictors, we can only easily plot one of them (e.g. `lstat`) against `medv` at a time, because scatterplots are 2D.However, we can still plot our predictions, which are based on all predictor features, on the same 2D plot.This allows us to look at the model's fit, at least with respect to the one dimension of `lstat`.To do this, we need to import a more sophisticated part of `plotly` called `graph_objects`:- `import plotly.graph_objects as go`<jupyter_code>import plotly.graph_objects as go

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="`iml3^SQ8#}M{aSRdsjz">go</variable></variables><block type="importAs" id="I=D{X(f@]J?iDq8EKrNP" x="65" y="163"><field name="libraryName">plotly.graph_objects</field><field name="libraryAlias" id="`iml3^SQ8#}M{aSRdsjz">go</field></block></xml><jupyter_output><empty_output><jupyter_text>The way `go` works is that we first create an *empty* figure, then we add data to it in layers.This is in contrast to `px`, where we created the figure and added data all at once.To create the empty figure:- Create variable `fig`- Set it to `with go create Figure using`<jupyter_code>fig = go.Figure()

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="@U5l^JIp,}.H=ECCW?@g">fig</variable><variable id="`iml3^SQ8#}M{aSRdsjz">go</variable></variables><block type="variables_set" id="jhv+dBJpS#?]Xz;mB/y}" x="63" y="341"><field name="VAR" id="@U5l^JIp,}.H=ECCW?@g">fig</field><value name="VALUE"><block type="varCreateObject" id="U[~m;7}PfW266d{R4B_w"><field name="VAR" id="`iml3^SQ8#}M{aSRdsjz">go</field><field name="MEMBER">Figure</field><data>go:Figure</data></block></value></block></xml><jupyter_output><empty_output><jupyter_text>Now we need to add two kinds of data to `fig`:- The test data (both `Xtest` and `Ytest` from `split`- The predictions (both `Xtest` and `predictions`Each of these can be done with the function `add_trace`.However, notice the both times we need just a column from `Xtest`, not the whole thing.So we need to get the `lstat` column on its own first.Here are the steps:**Getting `lstat` out of `splits`**- Create variable `Xtest`- Set it to `in list splits get  2`- Create variable `lstat`- Set it to `Xtest [` a list containing `"lstat"` `]`<jupyter_code>Xtest = splits[1]
lstat = Xtest[['lstat']]

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="fe.98]:,bU@x*=}zaJ1/">Xtest</variable><variable id="rVUZld`HfW,n/PC{X!9-">lstat</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable></variables><block type="variables_set" id="Q=9pQHE7F@.Gw^K.L-dt" x="11" y="385"><field name="VAR" id="fe.98]:,bU@x*=}zaJ1/">Xtest</field><value name="VALUE"><block type="lists_getIndex" id="57k*lJ0;vmKpSg*!H`]8"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="F4hyTfPiqx9)S?z+xUIq"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="|irh~@jIqN~E)ZBhA5j/"><field name="NUM">2</field></block></value></block></value><next><block type="variables_set" id="%B!g^jAyg[JC}a?Gd;]2"><field name="VAR" id="rVUZld`HfW,n/PC{X!9-">lstat</field><value name="VALUE"><block type="indexer" id="L]ClG49hl?`v1EmAZY)h"><field name="VAR" id="fe.98]:,bU@x*=}zaJ1/">Xtest</field><value name="INDEX"><block type="lists_create_with" id=",YhZdkj[DdlL=cwFv4FA"><mutation items="1"></mutation><value name="ADD0"><block type="text" id="opa@$sxEyy6xpto~juJ:"><field name="TEXT">lstat</field></block></value></block></value></block></value></block></next></block></xml><jupyter_output><empty_output><jupyter_text>**Adding scatterplot of observed data**- `with fig do add_scatter using` a list containing    - freestyle `x=` then `with np do ravel using lstat`  (see image below for this freestyle)    - freestyle `y=` then `with np do ravel using` a list containing        - `in list splits get  4`     - freestyle `mode="markers"`    - freestyle `name="observed"`    - freestyle `line=dict(color='royalblue')`        ![image.png](attachment:image.png)<jupyter_code>fig.add_scatter(x= (np.ravel(lstat)), y= (np.ravel(splits[3])), mode="markers", name="observed", line=dict(color='royalblue'))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="@U5l^JIp,}.H=ECCW?@g">fig</variable><variable id="Zhzp)s*VL?V@ES3(j:*b">np</variable><variable id="rVUZld`HfW,n/PC{X!9-">lstat</variable><variable id="_ut$e0PL4OMi4o1MXTpw">splits</variable></variables><block type="varDoMethod" id="qZ|Z9.ijg3CRAUbY+PrR" x="8" y="159"><field name="VAR" id="@U5l^JIp,}.H=ECCW?@g">fig</field><field name="MEMBER">add_scatter</field><data>fig:add_scatter</data><value name="INPUT"><block type="lists_create_with" id="9V^E?ADZgip/oSvw)bOB"><mutation items="5"></mutation><value name="ADD0"><block type="valueOutputCodeBlock" id="RTv-p{TaUq@4_58RZB%1"><field name="CODE">x=</field><value name="INPUT"><block type="varDoMethod" id="j8@mjp~f?I7StnE+y$s,"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="variables_get" id="AMp#2-apaCBpWlKv]`Ce"><field name="VAR" id="rVUZld`HfW,n/PC{X!9-">lstat</field></block></value></block></value></block></value><value name="ADD1"><block type="valueOutputCodeBlock" id="agyiU0@xgh8VMC=hx.gY"><field name="CODE">y=</field><value name="INPUT"><block type="varDoMethod" id="iKrKee;JuM8Vwq`^}m3A"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="lists_create_with" id="rY,yP@FZAYWucAR(exwa"><mutation items="1"></mutation><value name="ADD0"><block type="lists_getIndex" id=")EgFJH/WE}~Z=UZXlm{M"><mutation statement="false" at="true"></mutation><field name="MODE">GET</field><field name="WHERE">FROM_START</field><value name="VALUE"><block type="variables_get" id="?BL*os/nYs9.ODN!G/Ri"><field name="VAR" id="_ut$e0PL4OMi4o1MXTpw">splits</field></block></value><value name="AT"><block type="math_number" id="-H.`~{I{y`PmcIPv%E}v"><field name="NUM">4</field></block></value></block></value></block></value></block></value></block></value><value name="ADD2"><block type="dummyOutputCodeBlock" id="d2`of@jGbPV@.JFQeFdr"><field name="CODE">mode="markers"</field></block></value><value name="ADD3"><block type="dummyOutputCodeBlock" id=":jlV[+9%@+Wr2HMhzd^z"><field name="CODE">name="observed"</field></block></value><value name="ADD4"><block type="dummyOutputCodeBlock" id="wi|So(Dc8k#fv}G~sp0m"><field name="CODE">line=dict(color='royalblue')</field></block></value></block></value></block></xml><jupyter_output><empty_output><jupyter_text>**Adding scatterplot of predictions**- `with fig do add_scatter using` a list containing    - freestyle `x=` then `with np do ravel using lstat`    - freestyle `y=` then `with np do ravel using predictions`    - freestyle `mode=markers`    - freestyle `name=predicted`    - line=dict(color='orange')<jupyter_code>fig.add_scatter(x= (np.ravel(lstat)), y= (np.ravel(predictions)), mode="markers", name="predicted", line=dict(color='orange'))

#<xml xmlns="https://developers.google.com/blockly/xml"><variables><variable id="@U5l^JIp,}.H=ECCW?@g">fig</variable><variable id="Zhzp)s*VL?V@ES3(j:*b">np</variable><variable id="rVUZld`HfW,n/PC{X!9-">lstat</variable><variable id="5lkT$zi8W,`TXDny?Cv7">predictions</variable></variables><block type="varDoMethod" id="qZ|Z9.ijg3CRAUbY+PrR" x="8" y="159"><field name="VAR" id="@U5l^JIp,}.H=ECCW?@g">fig</field><field name="MEMBER">add_scatter</field><data>fig:add_scatter</data><value name="INPUT"><block type="lists_create_with" id="9V^E?ADZgip/oSvw)bOB"><mutation items="5"></mutation><value name="ADD0"><block type="valueOutputCodeBlock" id="RTv-p{TaUq@4_58RZB%1"><field name="CODE">x=</field><value name="INPUT"><block type="varDoMethod" id="j8@mjp~f?I7StnE+y$s,"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="variables_get" id="AMp#2-apaCBpWlKv]`Ce"><field name="VAR" id="rVUZld`HfW,n/PC{X!9-">lstat</field></block></value></block></value></block></value><value name="ADD1"><block type="valueOutputCodeBlock" id="agyiU0@xgh8VMC=hx.gY"><field name="CODE">y=</field><value name="INPUT"><block type="varDoMethod" id="iKrKee;JuM8Vwq`^}m3A"><field name="VAR" id="Zhzp)s*VL?V@ES3(j:*b">np</field><field name="MEMBER">ravel</field><data>np:ravel</data><value name="INPUT"><block type="variables_get" id="EB_-zyfAx?M()$2`t9[H"><field name="VAR" id="5lkT$zi8W,`TXDny?Cv7">predictions</field></block></value></block></value></block></value><value name="ADD2"><block type="dummyOutputCodeBlock" id="d2`of@jGbPV@.JFQeFdr"><field name="CODE">mode="markers"</field></block></value><value name="ADD3"><block type="dummyOutputCodeBlock" id=":jlV[+9%@+Wr2HMhzd^z"><field name="CODE">name="predicted"</field></block></value><value name="ADD4"><block type="dummyOutputCodeBlock" id="wi|So(Dc8k#fv}G~sp0m"><field name="CODE">line=dict(color='orange')</field></block></value></block></value></block></xml><jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup