<jupyter_start><jupyter_text>California Prices Imports<jupyter_code>import keras
import tensorflow as tf
import numpy as np
import seaborn as sns
import pandas as pd
import matplotlib.pyplot as plt

from keras.losses import mean_squared_error
from keras.models import Sequential
from keras.layers import Flatten, Dense, SimpleRNN, Input, Concatenate

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
random_seed = 532
np.random.seed(random_seed)
tf.random.set_seed(random_seed)<jupyter_output><empty_output><jupyter_text>DataLoad the california prices dataset, then split it into a training, validation and test set. Finally check the dimensions of the training set and notice that we have 8 features.<jupyter_code># load data and split into training, validation and test set
housing = fetch_california_housing()
x_train, x_test, y_train, y_test = train_test_split(
                                housing.data, housing.target, random_state = random_seed)
x_train, x_val, y_train, y_val = train_test_split(
                                x_train, y_train, random_state = random_seed)

# check dimensions
x_train.shape<jupyter_output><empty_output><jupyter_text>The next step is to scale features. In this case we'll use the standard scaler rather than the min-max scaler.<jupyter_code>scl = StandardScaler()
x_train = scl.fit_transform(x_train)
x_val = scl.transform(x_val)
x_test = scl.transform(x_test)<jupyter_output><empty_output><jupyter_text>BaselineAs a baseline against which to compare results obtained from more complex models we use a simple linear regression.<jupyter_code>baseline = Sequential([
        Dense(1, input_shape = x_train.shape[1:])
        ])
baseline.compile(loss = 'mean_squared_error', optimizer = 'sgd')
history = baseline.fit(x_train, y_train, epochs=20, validation_data=(x_val, y_val))<jupyter_output>Train on 11610 samples, validate on 3870 samples
Epoch 1/20
11610/11610 [==============================] - 1s 47us/step - loss: 1.3208 - val_loss: 0.5709
Epoch 2/20
11610/11610 [==============================] - 0s 33us/step - loss: 0.6048 - val_loss: 0.7096
Epoch 3/20
11610/11610 [==============================] - 0s 36us/step - loss: 0.6495 - val_loss: 0.5447
Epoch 4/20
11610/11610 [==============================] - 0s 38us/step - loss: 0.5431 - val_loss: 0.5409
Epoch 5/20
11610/11610 [==============================] - 0s 39us/step - loss: 0.5457 - val_loss: 0.6778
Epoch 6/20
11610/11610 [==============================] - 0s 42us/step - loss: 0.5799 - val_loss: 0.5414
Epoch 7/20
11610/11610 [==============================] - 0s 36us/step - loss: 0.5510 - val_loss: 0.5438
Epoch 8/20
11610/11610 [==============================] - 0s 33us/step - loss: 0.5571 - val_loss: 0.5456
Epoch 9/20
11610/11610 [==============================] - 0s 35us/step - loss: 0.5313 - val_loss: 0.6635
Epoch [...]<jupyter_text>Model - Sequential APIUse a fully connected network with two hiddens layers.<jupyter_code>model = Sequential([
        Dense(50, activation='relu', input_shape = x_train.shape[1:]),
        Dense(50, activation='relu'),
        Dense(1)
        ])<jupyter_output><empty_output><jupyter_text>Now we can compile the model, i.e. choose a loss function and an optimizer. Next, we can call the fit() method to train the network.<jupyter_code>model.compile(loss = 'mean_squared_error', optimizer = 'sgd')
history = model.fit(x_train, y_train, epochs=50, validation_data=(x_val, y_val))<jupyter_output>Train on 11610 samples, validate on 3870 samples
Epoch 1/50
11610/11610 [==============================] - 1s 43us/step - loss: 0.9217 - val_loss: 0.4944
Epoch 2/50
11610/11610 [==============================] - 1s 48us/step - loss: 0.4454 - val_loss: 0.4474
Epoch 3/50
11610/11610 [==============================] - 0s 40us/step - loss: 0.4188 - val_loss: 0.4216
Epoch 4/50
11610/11610 [==============================] - 0s 37us/step - loss: 0.4026 - val_loss: 0.4217
Epoch 5/50
11610/11610 [==============================] - 0s 38us/step - loss: 0.3840 - val_loss: 0.3977
Epoch 6/50
11610/11610 [==============================] - 0s 37us/step - loss: 0.3798 - val_loss: 0.3904
Epoch 7/50
11610/11610 [==============================] - 0s 38us/step - loss: 0.3675 - val_loss: 0.3822
Epoch 8/50
11610/11610 [==============================] - 0s 39us/step - loss: 0.3642 - val_loss: 0.3782
Epoch 9/50
11610/11610 [==============================] - 0s 39us/step - loss: 0.3528 - val_loss: 0.3810
Epoch [...]<jupyter_text>We can retrieve and plot learning curves from the history object.<jupyter_code># learning curves
pd.DataFrame(history.history).plot(figsize = (6,4))
plt.grid(True)
plt.gca().set_ylim(0, 1)
plt.show()<jupyter_output><empty_output><jupyter_text>We can evaluated the model's performances on the test set and/or use it to make predictions.<jupyter_code>mse_test = model.evaluate(x_test, y_test)
print(mse_test)

y_preds = model.predict(x_test[:5])
print(y_preds)<jupyter_output>5160/5160 [==============================] - 0s 18us/step
0.2962511199389317
[[2.1330814]
 [1.1361966]
 [4.6874075]
 [1.4011399]
 [3.889085 ]]<jupyter_text>Model - Functional APIWe can try to use the Keras Functional API to fit a Wide&Deep neural network, i.e. a model which also features a direct connection between the input and the output layer to ensure the network capability to learn simple patterns.<jupyter_code>input_ = Input(shape = x_train.shape[1:])
hidden1 = Dense(50, activation='relu')(input_)
hidden2 = Dense(50, activation='relu')(hidden1)
concat = Concatenate()([input_, hidden2])
output = Dense(1)(concat)
model2 = keras.Model(inputs = [input_], outputs=[output])<jupyter_output><empty_output><jupyter_text>As usual we can call the summary() method to get a glimpse about the network structure and its parameters.<jupyter_code>model2.summary()<jupyter_output>Model: "model_2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            (None, 8)            0                                            
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 50)           450         input_2[0][0]                    
__________________________________________________________________________________________________
dense_13 (Dense)                (None, 50)           2550        dense_12[0][0]                   
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 58)           0           input_2[0][0]              [...]<jupyter_text>We may also the **plot_model()** function to obtain a schema of the network.<jupyter_code>keras.utils.plot_model(model2, "wide&deep.png", show_shapes=True)<jupyter_output><empty_output><jupyter_text>As before, compile and train the network:<jupyter_code>model2.compile(loss = 'mean_squared_error', optimizer = 'sgd')
history2 = model2.fit(x_train, y_train, epochs=50, validation_data=(x_val, y_val))<jupyter_output>Train on 11610 samples, validate on 3870 samples
Epoch 1/50
11610/11610 [==============================] - 1s 51us/step - loss: 1.6858 - val_loss: 0.5517
Epoch 2/50
11610/11610 [==============================] - 0s 37us/step - loss: 0.7304 - val_loss: 0.5149
Epoch 3/50
11610/11610 [==============================] - 1s 46us/step - loss: 0.5210 - val_loss: 0.4676
Epoch 4/50
11610/11610 [==============================] - 1s 44us/step - loss: 0.4515 - val_loss: 0.4511
Epoch 5/50
11610/11610 [==============================] - 1s 46us/step - loss: 0.4458 - val_loss: 0.4343
Epoch 6/50
11610/11610 [==============================] - 1s 44us/step - loss: 0.4204 - val_loss: 0.4202
Epoch 7/50
11610/11610 [==============================] - 1s 48us/step - loss: 0.4138 - val_loss: 0.4122
Epoch 8/50
11610/11610 [==============================] - 1s 45us/step - loss: 0.4003 - val_loss: 0.4079
Epoch 9/50
11610/11610 [==============================] - 0s 39us/step - loss: 0.5693 - val_loss: 0.4029
Epoch [...]<jupyter_text>Plot learning curves.<jupyter_code>pd.DataFrame(history2.history).plot(figsize = (6,4))
plt.grid(True)
plt.gca().set_ylim(0, 1)
plt.show()<jupyter_output><empty_output><jupyter_text>Evaluate performances on test set and/or make predictions.<jupyter_code>mse_test2 = model2.evaluate(x_test, y_test)
print(mse_test2)

y_preds = model2.predict(x_test[:5])
print(y_preds)<jupyter_output>5160/5160 [==============================] - 0s 19us/step
0.3060086814008018
[[2.0562084]
 [1.1079824]
 [4.755809 ]
 [1.3724029]
 [3.7811775]]
StarcoderdataJupyterStructuredCleanDedup