<jupyter_start><jupyter_text>Project IntroductionThe clothing Fashion_MNIST is a dataset of images from Zalando(Europe's biggest online fashion retailer)It's consists of 60000 training samples and 10000 testing samplesEach sample is a 28*28 grayscale image assciated with a label from 10 classes0	T-shirt/top1	Trouser2	Pullover3	Dress4	Coat5	Sandal6	Shirt7	Sneaker8	Bag9	Ankle boot  Importing the neccessary library and data<jupyter_code># TensorFlow and tf.keras
import tensorflow as tf

# Helper libraries
import numpy as np
import matplotlib.pyplot as plt

fashion_mnist = tf.keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
print(train_labels)<jupyter_output>[9 0 0 ... 3 0 5]<jupyter_text>Each image is mapped to a single label. Since the class names are not included with the dataset, store them here to use later when plotting the images:<jupyter_code>class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat','Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']<jupyter_output><empty_output><jupyter_text>Explore the dataLet's explore the format of the dataset before training the model. The following shows there are 60,000 images in the training set, with each image represented as 28 x 28 pixels:<jupyter_code>train_images.shape
# There are 10,000 images in the test set. Again, each image is represented as 28 x 28 pixels:
test_images.shape
# Likewise, there are 60,000 labels in the training set:
len(train_labels)
# Each label is an integer between 0 and 9:
train_labels<jupyter_output><empty_output><jupyter_text>Preprocess the dataThe data must be preprocessed before training the network. If inspect the first image in the training set, we will see that the pixel values fall in the range of 0 to 255:<jupyter_code>plt.figure()
plt.imshow(train_images[0])
plt.colorbar()
plt.grid(False)
plt.show()<jupyter_output><empty_output><jupyter_text>Scale these values to a range of 0 to 1 before feeding them to the neural network model. To do so, divide the values by 255. It's important that the training set and the testing set be preprocessed in the same way:<jupyter_code>train_images = train_images / 255.0

test_images = test_images / 255.0<jupyter_output><empty_output><jupyter_text>To verify that the data is in the correct format and that we're ready to build and train the network, let's display the first 25 images from the training set and display the class name below each image.<jupyter_code>plt.figure(figsize=(15,15))
for i in range(16):
    plt.subplot(8,8,i+1)
    plt.xticks([])
    plt.yticks([])
    plt.grid(False)
    plt.imshow(train_images[i], cmap=plt.cm.binary)
    plt.xlabel(class_names[train_labels[i]])
plt.show()<jupyter_output><empty_output><jupyter_text>Architecting CNN ModelThe CNN has two main components: The features extractor of a set of convolutional and pooling layers and the classifier backend similar to a regular neural network<jupyter_code># Reshape the data into 4-dimentions
x_train = train_images.reshape((train_images.shape[0], 28, 28, 1))
x_test = test_images.reshape((test_images.shape[0], 28, 28, 1))
print(x_train.shape)
print(x_test.shape)<jupyter_output>(60000, 28, 28, 1)
(10000, 28, 28, 1)<jupyter_text>The first dimension is the number of samples---The fourth dimension is the appended one representing the grayscale images Build the modelBuilding the neural network requires configuring the layers of the model, then compiling the model.Set up the layers<jupyter_code># Random seed in tensorflow for reproducility
tf.random.set_seed(42)
from tensorflow.keras import datasets, layers, models, losses 
model = models.Sequential()
# add the first convolutional layers with 32 small-sized 3*3 filters
model.add(layers.Conv2D(32,(3,3), activation='relu', input_shape=(28, 28, 1)))
# convolutional layers is followed by a max-pooling layer with a  2*2 filter
model.add(layers.MaxPool2D(2,2))

# Second convolutional layers with 64 size 3*3 filters
model.add(layers.Conv2D(64,(3,3), activation='relu'))
model.add(layers.MaxPool2D(2,2))

# Third convolutional layers with 128 size 3*3 filters
model.add(layers.Conv2D(128,(3,3), activation='relu'))
# The resulting filter maps are flatten to provide features to the downstream classifier backend
model.add(layers.Flatten())

# Classifier backed
model.add(layers.Dense(64, activation='relu'))

# Output with 10 nodes demonstrating 10 classes
model.add(layers.Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss=losses.sparse_categorical_crossentropy, metrics=['accuracy'])

# Summarices the model
model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 26, 26, 32)        320       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 13, 13, 32)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 11, 11, 64)        18496     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 5, 5, 64)         0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 3, 3, 128)         73856     
                                                        [...]<jupyter_text>**The benefits of applying pooling are:**1.   Avoiding overfitting2.   Reducing training cost  Train the model**Training the neural network model requires the following steps:**Feed the training data to the model. The training data is in the train_images and train_labels arrays.The model learns to associate images and labels. To make predictions about a test set—in this example, the test_images array.Verify that the predictions match the labels from the test_labels array.  Feed the modelTo start training, call the model.fit method—so called because it "fits" the model to the training data:<jupyter_code># Batch size is 32 by default
 model.fit(x_train, train_labels, validation_data=(x_test, test_labels), epochs=10)<jupyter_output>Epoch 1/10
1875/1875 [==============================] - 21s 6ms/step - loss: 0.4732 - accuracy: 0.8262 - val_loss: 0.3361 - val_accuracy: 0.8804
Epoch 2/10
1875/1875 [==============================] - 11s 6ms/step - loss: 0.3041 - accuracy: 0.8891 - val_loss: 0.3639 - val_accuracy: 0.8685
Epoch 3/10
1875/1875 [==============================] - 10s 6ms/step - loss: 0.2598 - accuracy: 0.9040 - val_loss: 0.2805 - val_accuracy: 0.8960
Epoch 4/10
1875/1875 [==============================] - 11s 6ms/step - loss: 0.2278 - accuracy: 0.9155 - val_loss: 0.2636 - val_accuracy: 0.9062
Epoch 5/10
1875/1875 [==============================] - 11s 6ms/step - loss: 0.2053 - accuracy: 0.9244 - val_loss: 0.2656 - val_accuracy: 0.9049
Epoch 6/10
1875/1875 [==============================] - 11s 6ms/step - loss: 0.1821 - accuracy: 0.9317 - val_loss: 0.2552 - val_accuracy: 0.9075
Epoch 7/10
1875/1875 [==============================] - 11s 6ms/step - loss: 0.1636 - accuracy: 0.9381 - val_loss: 0.2580 - val_ac[...]<jupyter_text>Evaluate accuracyNext, compare how the model performs on the test dataset:<jupyter_code># 96% on training set and 91% on testing set
# Double check the performence
test_loss , test_acc = model.evaluate(x_test, test_labels, verbose=2)
print("Accuracy on the test set: ", test_acc)<jupyter_output>313/313 - 1s - loss: 0.3049 - accuracy: 0.9095 - 638ms/epoch - 2ms/step
Accuracy on the test set:  0.909500002861023<jupyter_text>Make predictionsWith the model trained, to make predictions about some images. The model's linear outputs.<jupyter_code>predictions = model.predict(x_test)
print(predictions[0])<jupyter_output>[2.6538676e-12 5.1054824e-12 3.1595245e-13 7.5452260e-16 6.1111787e-14
 1.0551089e-07 1.4841165e-17 3.2814410e-07 2.0824588e-10 9.9999952e-01]<jupyter_text>A prediction is an array of 10 numbers. They represent the model's "confidence" that the image corresponds to each of the 10 different articles of clothing. You can see which label has the highest confidence value:<jupyter_code>print("Predicted label for the test label sample: ", np.argmax(predictions[0]))
print("True label for the test sample: ", test_labels[0])<jupyter_output>True label for the test sample:  9<jupyter_text>**Graph this to look at the full set of 10 class predictions.**  Verify predictionsWith the model trained,  use it to make predictions about some images.Let's look at the 0th image, predictions, and prediction array. Correct prediction labels are blue and incorrect prediction labels are red. The number gives the percentage (out of 100) for the predicted label.<jupyter_code>def plot_image_prediction(i,images, predictions, labels, class_names):
  plt.subplot(1,2,1)
  plt.imshow(images[i], cmap=plt.cm.binary)
  prediction = np.argmax(predictions[i])
  color = 'blue' if prediction == labels[i] else 'red'
  plt.title(f"{class_names[labels[i]]}(predicted{class_names[prediction]})", color=color)
  plt.subplot(1,2,2)
  plt.grid(False)
  plt.xticks(range(10))
  plot = plt.bar(range(10), predictions[i], color='green')
  plt.ylim([0,1])
  plot[prediction].set_color('red')
  plot[prediction].set_color('blue')
  plt.show()
plot_image_prediction(0, test_images, predictions, test_labels, class_names)
plot_image_prediction(1, test_images, predictions, test_labels, class_names)
filters, _ = model.layers[2].get_weights()
f_min, f_max = filters.min(), filters.max()
filters = (filters - f_min) / (f_max - f_min)
n_filters = 16
for i in range(n_filters):
  filter = filters[:, :, :, i]
  plt.subplot(4, 4, i+1)
  plt.xticks([])
  plt.yticks([])
  plt.imshow(filter[:,:,0], cmap='gray')
plt.show()<jupyter_output><empty_output><jupyter_text>Improving the classifier with data augmentation<jupyter_code># Data augmentation
from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img
n_small = 500
x_train = x_train[:n_small]
train_labels = train_labels[:n_small]
print(x_train.shape)
model = models.Sequential()
# add the first convolutional layers with 32 small-sized 3*3 filters
model.add(layers.Conv2D(32,(3,3), activation='relu', input_shape=(28, 28, 1)))
# convolutional layers is followed by a max-pooling layer with a  2*2 filter
model.add(layers.MaxPool2D(2,2))

# Classifier backed
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))

# Compile the model
model.compile(optimizer='adam', loss=losses.sparse_categorical_crossentropy, metrics=['accuracy'])
model.fit(x_train, train_labels, validation_data=(x_test, test_labels), epochs=20,batch_size=40)
model.summary()
# Double check the performence
test_loss, test_acc = model.evaluate(x_test, test_labels, verbose=2)
print("Accuracy on the test set: ", test_acc)<jupyter_output>313/313 - 2s - loss: 0.6824 - accuracy: 0.8036 - 2s/epoch - 5ms/step
Accuracy on the test set:  0.803600013256073<jupyter_text>The model without augmentation has a classification accuracy of 80.36%<jupyter_code>datagen = ImageDataGenerator(height_shift_range=3, horizontal_flip=True)
model_aug = tf.keras.models.clone_model(model)
model_aug.compile(optimizer='adam', loss=losses.sparse_categorical_crossentropy, metrics=['accuracy'])
train_generator = datagen.flow(x_train, train_labels, seed=2,batch_size=40)
model_aug.fit(train_generator, validation_data=(x_test, test_labels), epochs=50)
model_aug.summary()
# Double check the performence
test_loss, test_acc = model_aug.evaluate(x_test, test_labels, verbose=2)
print("Accuracy on the test set: ", test_acc)<jupyter_output>313/313 - 1s - loss: 0.7799 - accuracy: 0.8136 - 1s/epoch - 4ms/step
Accuracy on the test set:  0.8136000037193298
StarcoderdataJupyterStructuredCleanDedup