Update recurrent.py
Update recurrent.py

Motivation and Context

Build a dynamic RNN model in static mode for deploying. The dynamic RNN means outputs the last without-padding output.

Description

Although, the official document gives an example of building dynamic RNN model in eager mode, we want the model to be static for serving.

'''

data = tf.convert_to_tensor(data, dtype=tf.float32)
class DynamicRNNExample(tl.models.Model):
def init(self):
super(DynamicRNNExample, self).init()
self.rnnlayer = tl.layers.RNN(
cell=tf.keras.layers.SimpleRNNCell(units=6, dropout=0.1), in_channels=1, return_last_output=True,
return_last_state=True
)
def forward(self, x):
z, s = self.rnnlayer(x, sequence_length=tl.layers.retrieve_seq_length_op3(x))
return z, s
model = DynamicRNNExample()
model.eval()
output, state = model(data)
'''
The current RNN layer cannot be built as a dynamic RNN layer for a static model, which is hard to save as trackable model for serving. I test the following code

'''
ni = tl.layers.Input(inputs_shape, name='input_layer')
seq = tl.layers.retrieve_seq_length_op3(ni)
cell = tf.keras.layers.LSTMCell(units=n_hidden, recurrent_dropout=0)
out = RNN(cell=cell,
return_last_output=True,
return_last_state=False,
return_seq_2d=True)(ni,sequence_length=seq)
nn = tl.layers.Dense(n_units=2, act=tf.nn.softmax, name="dense")(out)
model = tl.models.Model(inputs=ni, outputs=nn, name='rnn')
'''

which actually is built as a static RNN.

we force the RNN always to be dynamic to promote the accuracy.
zsdonghao/tensorlayer,zsdonghao/tensorlayer