<jupyter_start><jupyter_text>Lane Boundary Segmentation  Setting up Colab You can delete this "Setting up Colab" section if you work locally and do not want to use Google Colab<jupyter_code>colab_nb = 'google.colab' in str(get_ipython())
if colab_nb:
  from google.colab import drive
  drive.mount('/content/drive')
if colab_nb:
  %cd drive/My\ Drive/aad/code/solutions/lane_detection
if colab_nb:
    !pip installfastai --upgrade
    !pip install fastseg<jupyter_output><empty_output><jupyter_text>1. Loading data<jupyter_code>import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

import numpy as np
import cv2
import matplotlib.pyplot as plt
import re
import sys
sys.path.append("../../util")<jupyter_output><empty_output><jupyter_text>If you have collected data yourself in a folder "data" using `collect_data.py` and you want to use it for training, set the boolean in the next cell to `True`<jupyter_code>own_data = False
if own_data:
    from seg_data_util import sort_collected_data
    # copy and sort content of 'data' into 'data_lane_segmentation' folder:
    sort_collected_data()
    # Since data was copied, you can remove files in 'data' directory afterwards
else:
    # if you stopped the download before completion, please delete the 'data_lane_segmentation' folder and run this cell again
    from seg_data_util import download_segmentation_data
    download_segmentation_data()<jupyter_output><empty_output><jupyter_text>Independent of what you chose, you will have a directory 'data_lane_segmentation' now<jupyter_code>from fastai.vision.all import *
DATA_DIR = "data_lane_segmentation"


x_train_dir = os.path.join(DATA_DIR, 'train')
y_train_dir = os.path.join(DATA_DIR, 'train_label')

x_valid_dir = os.path.join(DATA_DIR, 'val')
y_valid_dir = os.path.join(DATA_DIR, 'val_label')<jupyter_output><empty_output><jupyter_text>2. Import fastai<jupyter_code>from fastai.vision.all import *
# some other usefuls libs
import os
import matplotlib.pyplot as plt
import cv2
def get_image_array_from_fn(fn):
    image = cv2.imread(fn)
    return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)<jupyter_output><empty_output><jupyter_text>3. Prepare data for usage with fastai library We will use a modified version of the fastai code for image segmentation that is given in the fastai documentation: https://docs.fast.ai/tutorial.vision.htmlSegmentation---With-the-data-block-API  3.1 label_func<jupyter_code>from sys import platform
folder_token = "\\" if platform == "win32" else "/"
# function that takes filename of a training image 'fn' and returns the filename of the corresponding label image
def label_func(fn): 
    return str(fn).replace(".png", "_label.png").replace("train", "train_label").replace("val"+folder_token, "val_label"+folder_token)
# pick the first image from the training directory and show it
sample_fn = os.path.join(x_valid_dir, os.listdir(x_valid_dir)[0])
print(sample_fn)
plt.imshow(get_image_array_from_fn(sample_fn));
# get corresponding label image using our 'label_func' function
label_fn = label_func(sample_fn)
print(label_fn)
# we multiply the image intensity by 100 to make lane lines visible for the human eye:
plt.imshow(100*get_image_array_from_fn(label_fn));<jupyter_output><empty_output><jupyter_text>3.2 get_image_files For the datablock API of the fastai library we need a function that takes a file path and returns a list of all the training images. We cannot **directly** use the built-in function 'get_image_files', since it would fetch all images, even the label images. Hence we define a function 'my_get_image_files' that does the same thing as 'get_image_files', just that it only looks into the folders "train" and "val". It will not look into "train_label" and "val_label". We can do this by inspecting the documentation of get_image_files on [docs.fast.ai](https://docs.fast.ai/data.transforms.htmlget_image_files) and using ['partial'](https://www.geeksforgeeks.org/partial-functions-python/)<jupyter_code>my_get_image_files = partial(get_image_files, folders=["train", "val"])<jupyter_output><empty_output><jupyter_text>3.3 DataBlock, DataLoaders and data augmentation<jupyter_code>codes = np.array(['back', 'left','right'],dtype=str)
carla = DataBlock(blocks=(ImageBlock, MaskBlock(codes)),
                   get_items = my_get_image_files,
                   get_y = label_func,
                   splitter = FuncSplitter(lambda x: str(x).find('validation_set')!=-1),
                   batch_tfms=aug_transforms(do_flip=False, p_affine=0, p_lighting=0.75))
dls = carla.dataloaders(Path(DATA_DIR), path=Path("."), bs=2)
dls.show_batch(max_n=6)<jupyter_output><empty_output><jupyter_text>4. Model and training<jupyter_code>from fastseg import MobileV3Small

model = MobileV3Small(num_classes=3, use_aspp=True, num_filters=64)
learn = Learner(dls, model, metrics=[DiceMulti(), foreground_acc])
learn.fine_tune(5)
learn.show_results(max_n=6, figsize=(7,8))
torch.save(learn.model, './fastai_model.pth')<jupyter_output><empty_output><jupyter_text>Experiments with inference<jupyter_code>import cv2
img = cv2.imread(str(get_image_files(x_valid_dir)[3]))
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(np.array(learn.predict(img)[0]))
# %timeit learn.predict(img); # => more than 100ms!
def get_pred_for_mobilenet(model, img_array):
    with torch.no_grad():
        image_tensor = img_array.transpose(2,0,1).astype('float32')/255
        x_tensor = torch.from_numpy(image_tensor).to("cuda").unsqueeze(0)
        model_output = F.softmax( model.forward(x_tensor), dim=1 ).cpu().numpy()
    return model_output
learn.model.eval();
plt.imshow(get_pred_for_mobilenet(learn.model,img)[0][2])
%timeit get_pred_for_mobilenet(learn.model,img)
# this is much faster!!!<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup