<jupyter_start><jupyter_text>Data Augmentation using NLPaugThis notebook demostrate the usage of a character augmenter, word augmenter. There are other types such as augmentation for sentences, audio, spectrogram inputs etc. All of the types many before mentioned types and many more can be found at the [github repo](https://github.com/makcedward/nlpaug) and [docs](https://nlpaug.readthedocs.io/en/latest/) of nlpaug.<jupyter_code>#Installing the nlpaug package
!pip install nlpaug==0.0.14
#this will be the base text which we will be using throughout this notebook
text="The quick brown fox jumps over the lazy dog ."
import nlpaug.augmenter.char as nac
import nlpaug.augmenter.word as naw
import nlpaug.augmenter.sentence as nas
import nlpaug.flow as nafc

from nlpaug.util import Action
import os
!git clone https://github.com/makcedward/nlpaug.git
os.environ["MODEL_DIR"] = 'nlpaug/model/'<jupyter_output>Cloning into 'nlpaug'...<jupyter_text>Augmentation at the Character Level1.   OCR Augmenter: To read textual data from on image, we need an OCR(optical character recognition) model. Once the text is extracted from the image, there may be errors like; '0' instead of an 'o', '2' instead of 'z' and other such similar errors.  2.   Keyboard Augmenter: While typing/texting typos are fairly common this augmenter simulates the errors by substituting characters in words with ones at a similar distance on a keyboard.<jupyter_code>#OCR augmenter
#import nlpaug.augmenter.char as nac

aug = nac.OcrAug()  
augmented_texts = aug.augment(text, n=3) #specifying n=3 gives us only 3 augmented versions of the sentence.

print("Original:")
print(text)

print("Augmented Texts:")
print(augmented_texts)
#Keyboard Augmenter
#import nlpaug.augmenter.word as naw


aug = nac.KeyboardAug()
augmented_text = aug.augment(text, n=3) #specifying n=3 gives us only 3 augmented versions of the sentence.

print("Original:")
print(text)

print("Augmented Text:")
print(augmented_text)<jupyter_output>Original:
The quick brown fox jumps over the lazy dog .
Augmented Text:
['The quick brown fox ,umps over the lazy dog .', 'The quivk browm fox jumps over the Kazy dog .', 'The quKck brown fox jumps over the lazy dog .']<jupyter_text>There are other types of character augmenters too. Their details are avaiable in the links mentioned at the beginning of this notebook.  Augmentation at the Word LevelAugmentation is important at the word level as well , here we use word2vec to insert or substitute a similar word. **Spelling** **augmentor**<jupyter_code>#Downloading the required txt file
import wget

if not os.path.exists("spelling_en.txt"):
    wget.download("https://raw.githubusercontent.com/makcedward/nlpaug/5238e0be734841b69651d2043df535d78a8cc594/nlpaug/res/word/spelling/spelling_en.txt")
else:
    print("File already exists")
#Substitute word by spelling mistake words dictionary
aug = naw.SpellingAug('spelling_en.txt')
augmented_texts = aug.augment(text)
print("Original:")
print(text)
print("Augmented Texts:")
print(augmented_texts)<jupyter_output>Original:
The quick brown fox jumps over the lazy dog .
Augmented Texts:
Tha qchick brown fox jumps over athe lazy dog .<jupyter_text>**Word embeddings augmentor**<jupyter_code>import gzip
import shutil

gn_vec_path = "GoogleNews-vectors-negative300.bin"
if not os.path.exists("GoogleNews-vectors-negative300.bin"):
    if not os.path.exists("../Ch3/GoogleNews-vectors-negative300.bin"):
        #Downloading the reqired model
        if not os.path.exists("../Ch3/GoogleNews-vectors-negative300.bin.gz"):
            if not os.path.exists("GoogleNews-vectors-negative300.bin.gz"):
                wget.download("https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz")
            gn_vec_zip_path = "GoogleNews-vectors-negative300.bin.gz"
        else:
            gn_vec_zip_path = "../Ch3/GoogleNews-vectors-negative300.bin.gz"
        #Extracting the required model
        with gzip.open(gn_vec_zip_path, 'rb') as f_in:
            with open(gn_vec_path, 'wb') as f_out:
                shutil.copyfileobj(f_in, f_out)
    else:
        gn_vec_path = "../Ch3/" + gn_vec_path

print(f"Model at {gn_vec_path}")<jupyter_output>Model at GoogleNews-vectors-negative300.bin<jupyter_text>Insert word randomly by word embeddings similarity<jupyter_code># model_type: word2vec, glove or fasttext
aug = naw.WordEmbsAug(
    model_type='word2vec', model_path=gn_vec_path,
    action="insert")
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented Text:")
print(augmented_text)<jupyter_output>Original:
The quick brown fox jumps over the lazy dog .
Augmented Text:
The quick GlobeOp brown fox jumps over Exchange the Khulna lazy dog .<jupyter_text>Substitute word by word2vec similarity<jupyter_code>aug = naw.WordEmbsAug(
    model_type='word2vec', model_path=gn_vec_path,
    action="substitute")
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented Text:")
print(augmented_text)<jupyter_output>Original:
The quick brown fox jumps over the lazy dog .
Augmented Text:
Though quick brown fox Leaping over it lazy dog .
StarcoderdataJupyterStructuredCleanDedup