<jupyter_start><jupyter_text>FastText on Google Colab<jupyter_code>from tensorflow.python.client import device_lib
device_lib.list_local_devices()<jupyter_output><empty_output><jupyter_text>Downloading fastText model:<jupyter_code>!wget https://github.com/facebookresearch/fastText/archive/v0.1.0.zip
!unzip v0.1.0.zip
cd fastText-0.1.0
!make
!ls
!./fasttext<jupyter_output>usage: fasttext <command> <args>

The commands supported by fasttext are:

  supervised              train a supervised classifier
  quantize                quantize a model to reduce the memory usage
  test                    evaluate a supervised classifier
  predict                 predict most likely labels
  predict-prob            predict most likely labels with probabilities
  skipgram                train a skipgram model
  cbow                    train a cbow model
  print-word-vectors      print word vectors given a trained model
  print-sentence-vectors  print sentence vectors given a trained model
  nn                      query for nearest neighbors
  analogies               query for analogies<jupyter_text>Exploring the model<jupyter_code>#!wget https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.en.zip
!wget https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.es.zip  
!unzip wiki.es.zip
!ls
!echo "notebook" | ./fasttext print-word-vectors wiki.es.bin
!./fasttext nn wiki.es.bin 5
!./fasttext analogies wiki.es.bin<jupyter_output>tcmalloc: large alloc 3582803968 bytes == 0x55d49a356000 @  0x7fdc6936a887 0x55d483ee943c 0x55d483ef69cc 0x55d483ef73e1 0x55d483f0198c 0x55d483ed3dcd 0x7fdc68407b97 0x55d483ed3eaa
Pre-computing word vectors... done.
Query triplet (A - B + C)? amor odio ternura
ternura» 0.659729
ternuras 0.630301
amor», 0.571151
islaternura 0.546598
amor» 0.538784
sensualidad 0.537367
dulzura 0.536799
ensoñadora 0.527915
enamoradísima 0.525622
dulzuras 0.517982
Query triplet (A - B + C)? ^C<jupyter_text>TEXT Classification Data from: https://www.kaggle.com/bittlingmayer/amazonreviews/homeAmazon reviews:We have only two classes 1 and 2, where __label__1 signifies that the reviewer gave either 1 or 2 stars for the product, while __label__2 indicates a 4 or 5 star rating.  Installing bzip2<jupyter_code>!apt install bzip2<jupyter_output>Reading package lists... Done
Building dependency tree       
Reading state information... Done
bzip2 is already the newest version (1.0.6-8.1).
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.<jupyter_text>Loading the data:<jupyter_code>!wget https://storage.googleapis.com/amazonreviews/train.ft.txt.bz2
!wget https://storage.googleapis.com/amazonreviews/test.ft.txt.bz2
!bzip2 -d train.ft.txt.bz2
!bzip2 -d test.ft.txt.bz2
!ls
!wc train.ft.txt
#number of lines, number of words, number of characters
!head -n 20 train.ft.txt<jupyter_output>__label__2 Stuning even for the non-gamer: This sound track was beautiful! It paints the senery in your mind so well I would recomend it even to people who hate vid. game music! I have played the game Chrono Cross but out of all of the games I have ever played it has the best music! It backs away from crude keyboarding and takes a fresher step with grate guitars and soulful orchestras. It would impress anyone who cares to listen! ^_^
__label__2 The best soundtrack ever to anything.: I'm reading a lot of reviews saying that this is the best 'game soundtrack' and I figured that I'd write a review to disagree a bit. This in my opinino is Yasunori Mitsuda's ultimate masterpiece. The music is timeless and I'm been listening to it for years now and its beauty simply refuses to fade.The price tag on this is pretty staggering I must say, but if you are going to buy any cd for this much money, this is the only one that I feel would be worth every penny.
__label__2 Amazing!: This soundtrack is m[...]<jupyter_text>Cleaning the text:<jupyter_code>!cat train.ft.txt | sed -e "s/\([.\!?,’/()]\)/ \1 /g" | tr "[:upper:]" "[:lower:]" > train_clean.ft.txt
!head -n 20 train_clean.ft.txt
!wc train_clean.ft.txt
!head -n 1000000 train_clean.ft.txt > model_reduced.ft.txt
!ls<jupyter_output>args.o			   matrix.o		    test.ft.txt
classification-example.sh  model.o		    train_clean.ft.txt
classification-results.sh  model_reduced.ft.txt     train.ft.txt
CONTRIBUTING.md		   PATENTS		    tutorials
dictionary.o		   pretrained-vectors.md    utils.o
eval.py			   productquantizer.o	    vector.o
fasttext		   qmatrix.o		    wiki.es.bin
fasttext.o		   quantization-example.sh  wiki.es.vec
get-wikimedia.sh	   quantization-results.sh  wiki.es.zip
LICENSE			   README.md		    wikifil.pl
Makefile		   src			    word-vector-example.sh<jupyter_text>Training the model:<jupyter_code>%%time
!./fasttext supervised -input model_reduced.ft.txt -output sentiment_amazon -label __label__ -epoch 5 -lr 0.5 -wordNgrams 3
!ls<jupyter_output>args.o			   model.o		    test.ft.txt
classification-example.sh  model_reduced.ft.txt     train_clean.ft.txt
classification-results.sh  PATENTS		    train.ft.txt
CONTRIBUTING.md		   pretrained-vectors.md    tutorials
dictionary.o		   productquantizer.o	    utils.o
eval.py			   qmatrix.o		    vector.o
fasttext		   quantization-example.sh  wiki.es.bin
fasttext.o		   quantization-results.sh  wiki.es.vec
get-wikimedia.sh	   README.md		    wiki.es.zip
LICENSE			   sentiment_amazon.bin     wikifil.pl
Makefile		   sentiment_amazon.vec     word-vector-example.sh
matrix.o		   src<jupyter_text>Testing the model:<jupyter_code>!./fasttext test sentiment_amazon.bin test.ft.txt
#Output: precision / recall<jupyter_output>tcmalloc: large alloc 1122500608 bytes == 0x564187c4a000 @  0x7f00944a9887 0x56417665743c 0x5641766649cc 0x5641766653e1 0x56417666f483 0x564176641e49 0x7f0093546b97 0x564176641eaa
N	400000
P@1	0.881
R@1	0.881
Number of examples: 400000<jupyter_text>Making Predictions<jupyter_code>!./fasttext predict sentiment_amazon.bin test.ft.txt 1
#k: last number, represents the top k labels for each review<jupyter_output><empty_output><jupyter_text>Embedding Vector<jupyter_code>!echo "laptop" | ./fasttext print-word-vectors sentiment_amazon.bin<jupyter_output>tcmalloc: large alloc 1122500608 bytes == 0x559ebe43c000 @  0x7ff9fc86f887 0x559eabe5243c 0x559eabe5f9cc 0x559eabe603e1 0x559eabe6a1dd 0x559eabe3ce0e 0x7ff9fb90cb97 0x559eabe3ceaa
laptop 0.011225 0.00090258 -0.013487 0.0053669 0.0046355 0.0013649 -0.010597 0.0056601 0.0059065 0.0053603 -0.00046055 -0.0018156 0.00099282 0.0080198 -0.0039922 -0.0036041 -0.00013655 -0.013985 -0.0069846 -0.0029751 -0.0033721 0.0034321 -0.0024614 0.0057717 -0.0063592 0.010745 -0.0024222 -0.0082812 -0.017762 0.010009 4.4463e-05 -0.0061159 -0.0052048 -0.0040597 -0.0073048 -0.0019388 -9.1917e-05 -0.0065816 -0.005934 0.015955 -0.0008324 0.010783 -0.0098569 0.010083 0.0075091 0.0060111 -0.010531 -0.0082694 -0.014075 0.011761 -0.0040339 -0.0092697 -0.008327 -0.0014735 -0.011839 -0.012271 -0.0020662 0.0090999 0.0010633 -0.0085731 0.0034415 0.018382 0.0128 0.0041434 -0.005198 0.0017859 -0.001986 -0.0084128 -0.012297 0.0048784 0.0020976 -0.0010526 -0.0032358 0.0035031 -0.0019306 0.0053505 0.0057838 0.0054278 -0.0094[...]<jupyter_text>Finding similar words:<jupyter_code>!./fasttext nn sentiment_amazon.bin<jupyter_output>tcmalloc: large alloc 1122500608 bytes == 0x55638cb4e000 @  0x7fb627c7e887 0x556379f5d43c 0x556379f6a9cc 0x556379f6b3e1 0x556379f7587c 0x556379f47de0 0x7fb626d1bb97 0x556379f47eaa
Pre-computing word vectors... done.
Query word? alexa
boudt 0.887984
yto 0.884258
tales: 0.882203
asimov's 0.880595
scenerios 0.878894
jasc 0.87852
nichols 0.877574
inculcated 0.877555
farting 0.877552
9+ 0.877336
Query word? oled
secondsreads 0.530047
loach 0.494326
omnis 0.484163
mega-blast 0.474459
agrivated 0.4734
singing-samples 0.469985
carved-foam 0.469775
deal--the 0.468878
showme 0.468573
youbecome 0.466695
Query word? laptop
"stare 0.699268
dizzyingly 0.69152
vraca 0.690675
:-d 0.685596
cayce-edgar 0.685436
spivok 0.679247
ccountry 0.678104
1866 0.677955
hatin': 0.677723
obrigatory 0.677345
Query word? ^C
StarcoderdataJupyterStructuredCleanDedup