<jupyter_start><jupyter_text>Load pre-trained IMDB model and data<jupyter_code># 아래 코드를 수행하기 전에
# mkdir word2vec
# cp imdb_word2vec.txt word2vec/word2vec_50_imdb.txt
# python imdb_model.py
imdb_model = DocumentModel.load_model(config.MODEL_DIR+ '/imdb/model_04.json')
imdb_model.load_model_weights(config.MODEL_DIR+ '/imdb/model_04.hdf5')

model = imdb_model.get_classification_model()
model.compile(loss="binary_crossentropy", optimizer='rmsprop', metrics=["accuracy"])
model.targets
import config
from dataloader.loader import Loader

train_df = Loader.load_imdb_data(directory = 'train')
print(train_df.shape)

corpus = train_df['review'].tolist()
target = train_df['sentiment'].tolist()
corpus, target = remove_empty_docs(corpus, target)
print(len(corpus))<jupyter_output>(25000, 2)
25000<jupyter_text>Pre process input and compute document embeddings<jupyter_code>Preprocess.NUM_SENTENCES = 20
preprocessor = Preprocess(corpus=corpus)
corpus_to_seq = preprocessor.fit()

corpus = train_df['review'].tolist()
target = train_df['sentiment'].tolist()
corpus_to_seq = preprocessor.transform(corpus)

x_train = np.array(corpus_to_seq)
y_train = np.array(target)

print(x_train.shape, y_train.shape)
print('Evaluating Model ...')
print(model.evaluate(x_train, y_train))

preds = model.predict(x_train)

#invert predicted label
pseudo_label = np.subtract(1,preds)<jupyter_output>Evaluating Model ...
25000/25000 [==============================] - 29s 1ms/sample - loss: 0.9458 - acc: 0.5070
[0.9458074434661865, 0.50696]<jupyter_text>Gradient Calculation of inverted output w.r.t sentence embeddings<jupyter_code>#Get the learned sentence embeddings
sentence_ebd = imdb_model.get_sentence_model().predict(x_train)

input_tensors = [model.inputs[0], # input data
                 model.sample_weights[0], # how much to weight each sample by
                 model.targets[0], # labels                 
]
#variable tensor at the sentence embeding layer
weights = imdb_model.get_sentence_model().outputs

#calculate gradient of the total model loss w.r.t 
#the variables at sentence embd layer                                     
gradients = model.optimizer.get_gradients(model.total_loss, weights) 
get_gradients = K.function(inputs=input_tensors, outputs=gradients)
document_number = 10
K.set_learning_phase(0)
inputs = [np.asarray([x_train[document_number]]), # X
          np.asarray([1]), # sample weights
          np.asarray([[pseudo_label[document_number][0]]]), # y
]
grad = get_gradients(inputs)

doc = corpus[document_number]
label = y_train[document_number]
prediction = preds[document_number]
print(doc, label , prediction)

sentences = sent_tokenize(doc)

sent_score = []
for i in range(len(sentences)):
    #sent_score.append((i, -np.abs(np.dot(grad[0][0][i],sentence_ebd[document_number][i])))) #DECREASING
    sent_score.append((i, -np.linalg.norm(grad[0][0][i])))

sent_score.sort(key=lambda tup: tup[1])
summary_sentences = [ i for i, s in sent_score[:4]]

for i in summary_sentences:
    print(i, sentences[i])<jupyter_output>Being half-portuguese doesn't render me half-blind (nor half-prejudiced) when discussing portuguese films. Not that I get to do that very often anyway. But this film was such a rush of adrenaline! Yes, that's right - it was mostly accurate as far as history went/goes - but it pulled no punches on venturing beyond usual portuguese-film territory: things like using real locations in the middle of traffic-congested Lisbon and recruiting a real crowd to stand in for the real crowd of almost 30 years ago. And by God did they get it right! OK, to sum it up: very emotional if you've lived through it, but you'll spot minor improvements that could have been made as well as plot necessities that were. If you're just watching it randomly, you're in for a good historical romp, only of the very recent History kind and a bit more thought-proving than usual. Even by European standards, yes. 1 [0.32008418]
7 Even by European standards, yes.
1 Not that I get to do that very often anyway.
4 And by God d[...]
StarcoderdataJupyterStructuredCleanDedup