<jupyter_start><jupyter_text>Deep LearningPractical 9a - Keras Functional APIAY2020/21 Semester      1. Introduction to the functional APIIn the functional API, you directly manipulate tensors, and you use layers as functionsthat take tensors and return tensors (hence, the name functional API):<jupyter_code>from tensorflow.keras import Input, layers
input_tensor = Input(shape=(32,))
dense = layers.Dense(32, activation='relu')
output_tensor = dense(input_tensor)<jupyter_output>WARNING:tensorflow:From C:\Users\lji6\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.<jupyter_text>Let’s start with a minimal example that shows side by side a simple Sequential modeland its equivalent in the functional API:<jupyter_code>from tensorflow.keras.models import Sequential, Model
from tensorflow.keras import layers
from tensorflow.keras import Input<jupyter_output><empty_output><jupyter_text>* ***A Sequential Model***<jupyter_code>seq_model = Sequential()
seq_model.add(layers.Dense(32, activation='relu', input_shape=(64,)))
seq_model.add(layers.Dense(32, activation='relu'))
seq_model.add(layers.Dense(10, activation='softmax'))
seq_model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 32)                2080      
_________________________________________________________________
dense_2 (Dense)              (None, 32)                1056      
_________________________________________________________________
dense_3 (Dense)              (None, 10)                330       
=================================================================
Total params: 3,466
Trainable params: 3,466
Non-trainable params: 0
_________________________________________________________________<jupyter_text>* ***A Functional API Model***<jupyter_code>input_tensor = Input(shape=(64,))
x = layers.Dense(32, activation='relu')(input_tensor)
x = layers.Dense(32, activation='relu')(x)
output_tensor = layers.Dense(10, activation='softmax')(x)
model = Model(input_tensor, output_tensor)
model.summary()<jupyter_output>Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         [(None, 64)]              0         
_________________________________________________________________
dense_4 (Dense)              (None, 32)                2080      
_________________________________________________________________
dense_5 (Dense)              (None, 32)                1056      
_________________________________________________________________
dense_6 (Dense)              (None, 10)                330       
=================================================================
Total params: 3,466
Trainable params: 3,466
Non-trainable params: 0
_________________________________________________________________<jupyter_text>The only part that may seem a bit magical at this point is instantiating a `Model` objectusing only an input tensor and an output tensor. Behind the scenes, Keras retrievesevery layer involved in going from `input_tensor` to `output_tensor`, bringing themtogether into a graph-like data structure—a `Model`. Of course, the reason it works isthat `output_tensor` was obtained by repeatedly transforming `input_tensor`.When it comes to compiling, training, or evaluating such an instance of `Model`, theAPI is the same as that of Sequential:<jupyter_code>model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics =['acc'])
import numpy as np
x_train = np.random.random((1000, 64))
y_train = np.random.random((1000, 10))
model.fit(x_train, y_train, epochs=10, batch_size=128)
score = model.evaluate(x_train, y_train, verbose=2)<jupyter_output>1000/1000 - 0s - loss: 61.7702 - acc: 0.1120<jupyter_text>2. Build a multiple inputs model using functional APIWe will build a question-answering model using functional API. The model has two inputs: a natural-language questionand a text snippet (such as a news article) providing information to be used for answering the question. The model merge two input branches using a `concatenate` layer, after that model produce an answer via a softmax over some predefined vocabulary.First, we build up model using funcational API.<jupyter_code>from tensorflow.keras.models import Model
from tensorflow.keras import layers
from tensorflow.keras import Input
text_vocabulary_size = 10000
question_vocabulary_size = 10000
answer_vocabulary_size = 500

text_input = Input(shape=(100,), dtype='int32', name='text')
embedded_text = layers.Embedding(text_vocabulary_size, 64)(text_input)
encoded_text = layers.LSTM(32)(embedded_text)

question_input = Input(shape=(100,),dtype='int32',name='question')
embedded_question = layers.Embedding(question_vocabulary_size,32)(question_input)
encoded_question = layers.LSTM(16)(embedded_question)

concatenated = layers.concatenate([encoded_text, encoded_question],axis=-1)
answer = layers.Dense(answer_vocabulary_size,activation='softmax')(concatenated)

model = Model([text_input, question_input], answer)
model.compile(optimizer='rmsprop',loss='categorical_crossentropy',metrics=['acc'])
model.summary()<jupyter_output>WARNING:tensorflow:From C:\Users\lji6\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\keras\initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text (InputLayer)               [(None, 100)]        0                                            
__________________________________________________________________________________________________
question (InputLayer)           [(None, 100)]        0                                         [...]<jupyter_text>Then, we generate some dummy inputs (text, questions, answers) using `numpy` random number generation fuction.<jupyter_code>import numpy as np
num_samples = 1000
max_length = 100
text = np.random.randint(1, text_vocabulary_size, size=(num_samples, max_length))
question = np.random.randint(1, question_vocabulary_size, size=(num_samples, max_length))
answers = np.random.randint(0, answer_vocabulary_size, size=(num_samples, answer_vocabulary_size))
print(answers.shape)<jupyter_output>(1000, 500)<jupyter_text>Finally, we train the model using the above dummy inputs.<jupyter_code>model.fit({'text': text, 'question': question}, answers, epochs=10, batch_size=128)
from tensorflow.keras.utils import plot_model
plot_model(model, show_shapes = True, to_file='model.png')<jupyter_output>Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.
StarcoderdataJupyterStructuredCleanDedup