<jupyter_start><jupyter_text>Deep Feedforward NetworkA Deep Feedforward Network, also called Feedforward Neural Network, or Multi-layer perceptrons (MLPs).Among the layers, you can distinguish an input layer, hidden layers and an output layer. In general, there can be multiple hidden layers. Each node in the layer is a Neuron. An Artifical Neuron is the basic unit of a neural network.Multi-layer perceptrons are often fully connected. This means that there’s a connection from each perceptron in a certain layer to each perceptron in the next layer.  NOTE:Basicaly this notebook prepared to use within **Google Colab**: https://colab.research.google.com/. The Google Colabatory has **free Tesla K80 GPU** and already prepared to develop deep learning applications.First time opens this notebook, do not forget to enable **Python 3** runtime and **GPU** accelerator in Google Colab **Notebook Settings**.   Setup ProjectCreate workspace and change directory.<jupyter_code>PROJECT_HOME = '/content/keras-movie-reviews-classification'

import os.path
if not os.path.exists(PROJECT_HOME):
  os.makedirs(PROJECT_HOME)
os.chdir(PROJECT_HOME)

!pwd<jupyter_output>/content/keras-movie-reviews-classification<jupyter_text>Import ProjectImport GitHub project to workspace.<jupyter_code># Import project and override existing data.
!git init .
!git remote add -t \* -f origin https://github.com/alex-agency/keras-movie-reviews-classification.git
!git reset --hard origin/master
!git checkout

!ls -la input<jupyter_output>Initialized empty Git repository in /content/keras-movie-reviews-classification/.git/
Updating origin
remote: Counting objects: 22, done.[K
remote: Compressing objects: 100% (17/17), done.[K
remote: Total 22 (delta 4), reused 17 (delta 3), pack-reused 0[K
Unpacking objects: 100% (22/22), done.
From https://github.com/alex-agency/keras-movie-reviews-classification
 * [new branch]      master     -> origin/master
HEAD is now at 10b2912 modified dataset and updated explanations
total 35692
drwxr-xr-x 2 root root     4096 Apr 13 21:12 .
drwxr-xr-x 4 root root     4096 Apr 13 21:12 ..
-rw-r--r-- 1 root root 17163254 Apr 13 21:12 dataset.npz
-rw-r--r-- 1 root root 19372046 Apr 13 21:12 reviews.tsv.bz2<jupyter_text>Load Dataset<jupyter_code>import numpy as np
# load data from numpy array
loaded = np.load('input/dataset.npz')

(x_train, y_train), (x_test, y_test) = loaded['dataset']
dictionary = loaded['dictionary'].item()
label_encoder = loaded['label_encoder'].item()

print('Vocabulary: {} words.'.format(len(dictionary)))
print('Categorical classes: {}\n'.format(list(label_encoder.classes_)))

print('x_train shape:', x_train.shape)
print('y_train shape:', y_train.shape)

print('x_test shape:', x_test.shape)
print('y_test shape:', y_test.shape)<jupyter_output>Vocabulary: 5000 words.
Categorical classes: ['negative', 'positive']

x_train shape: (40000, 400)
y_train shape: (40000, 2)
x_test shape: (10000, 400)
y_test shape: (10000, 2)<jupyter_text>KerasKeras is a high-level API, written in Python and capable of running on top of TensorFlow, Theano, or CNTK deep learning frameworks.Keras provides a simple and modular API to create and train Neural Networks, hiding most of the complicated details under the hood.By default, Keras is configured to use Tensorflow as the backend since it is the most popular choice.Keras is becoming super popular recently because of its simplicity. Keras workflow<jupyter_code># Load Keras libraries
from keras.models import Sequential
from keras.layers.embeddings import Embedding
from keras.layers import Dense, Flatten, Dropout<jupyter_output>Using TensorFlow backend.<jupyter_text>Create Deep Feedforward Network model<jupyter_code># Model parameters
MAX_NB_WORDS = len(dictionary) # vocabulary size
MAX_SEQUENCE_LENGTH = x_train.shape[1] # review length
MAX_NB_LABELS = y_train.shape[1] # feedback legth
EMBEDDING_SIZE = 1 # embedding layer dimension
HIDDEN_SIZE = (MAX_SEQUENCE_LENGTH*EMBEDDING_SIZE//2) # hidden layer dimension
DROPOUT_RATE = 0.2 # 20% dropout rate, use a small dropout value of 20%-50% of neurons.

# Initialise neural network model.
# The Sequential model is a linear stack of layers.
# It allows to create models layer-by-layer.
# Keras provides two ways to define a model: Sequential and Functional API.
# Functional API model used for designing complex model architectures.
model = Sequential()

# This is the first layer. It is used to provide the input data or features to the network.
# Embedding layer turns positive integers (indexes) into dense vectors of fixed size.
model.add(Embedding(input_dim=MAX_NB_WORDS, # Number of features
                    output_dim=EMBEDDING_SIZE, # Dimension of the dense embedding
                    input_length=MAX_SEQUENCE_LENGTH)) # Length of input sequences

# Flatten shape from 3 to 2 dimensions to fit with output layer.
model.add(Flatten())

# The Feedforward network in which all the neurons from one layer are connected to the neurons in the previous layer.

# Hidden layer of the network.
# Dense layer is a fully connected layer, since, each node in the input is connected to every node in the output.  
model.add(Dense(HIDDEN_SIZE, # number of neurons in the hidden layer
                activation='relu')) # the ReLU activation function which introduce non-linearity.

# Dropout layer prevents network from overfitting.
model.add(Dropout(DROPOUT_RATE))

# Output layer, which gives out the predictions.
# The activation function to be used in this layer is different for different problems.
# For binary classification, we can use sigmoid activation.
model.add(Dense(MAX_NB_LABELS, # Number of labels which is used in prediction
                activation='sigmoid')) # Sigmoid activation function

print("Model Summary")
print(model.summary())<jupyter_output>Model Summary
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 400, 1)            5000      
_________________________________________________________________
flatten_1 (Flatten)          (None, 400)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 200)               80200     
_________________________________________________________________
dropout_1 (Dropout)          (None, 200)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 2)                 402       
=================================================================
Total params: 85,602
Trainable params: 85,602
Non-trainable params: 0
__________________________________________________________[...]<jupyter_text>Activation FunctionsThe activation function is used as a decision making body at the output of a neuron. The neuron learns Linear or Non-linear decision boundaries based on the activation function. It also has a normalizing effect on the neuron output which prevents the output of neurons after several layers to become very large, due to the cascading effect. There are three most widely used activation functions: **Sigmoid** - It maps the input (x axis) to values between 0 and 1.**Tanh** - It is similar to the sigmoid function butmaps the input to values between -1 and 1.**Rectified Linear Unit (ReLU)** - It allows only positive values to pass through it. The negative values are mapped to zero. DropoutDuring training, when dropout is applied to a layer, some percentage of its neurons (a hyperparameter, with common values being between 20 and 50%) are randomly deactivated or “dropped out,” along with their connections. Which neurons are dropped out are constantly shuffled randomly during training.  This forces the network to learn a more balanced representation, and helps with overfitting.  Export model to fileSaving whole Keras model into a single HDF5 file which will contain:* the architecture of the model, allowing to re-create the model* the weights of the model* the training configuration (loss, optimizer)* the state of the optimizer, allowing to resume training exactly where you left off.<jupyter_code># Model filename
model_filename = 'mlps-model-definition.h5'

model_file = os.path.join('input', model_filename)

# Export model into HDF5 file.
model.save(model_file)

!ls -la input<jupyter_output>total 36044
drwxr-xr-x 2 root root     4096 Apr 13 21:13 .
drwxr-xr-x 4 root root     4096 Apr 13 21:12 ..
-rw-r--r-- 1 root root 17163254 Apr 13 21:12 dataset.npz
-rw-r--r-- 1 root root   360416 Apr 13 21:13 mlps-model-definition.h5
-rw-r--r-- 1 root root 19372046 Apr 13 21:12 reviews.tsv.bz2<jupyter_text>Downloading file to your local file systemIt will invoke a browser download of the file to your local computer.<jupyter_code>from google.colab import files
# Download file
files.download(model_file)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup