<jupyter_start><jupyter_text>Pre-request setup TensorflowCurrent working version for tensorflow is v2.3.Install the tensorflow and other [requirements](https://www.tensorflow.org/install/gpu) for getting GPU support.To use the CUDA, make sure the GPU you used is in the [CUDA capatable list](https://developer.nvidia.com/cuda-gpus) CUDA ToolkitCheck the pre-requirment based on the [CUDA toolkit installation guide](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.htmlverify-you-have-cuda-enabled-system).Download the [CUDA Toolkit 10.1 and CUPTI](https://developer.nvidia.com/cuda-10.1-download-archive-base), installing commands:```shsudo dpkg -i cuda-repo-ubuntu1810-10-1-local-10.1.105-418.39_1.0-1_amd64.debsudo apt-key add /var/cuda-repo-/7fa2af80.pubsudo apt-get updatesudo apt-get install cuda``` cuDNNDownload the `version 7.6.5` of cuDNN from the [archive of the cuDNN](https://developer.nvidia.com/rdp/cudnn-archive) , unzip it and then install it by:```shsudo cp cuda/include/cudnn*.h /usr/local/cuda/includesudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*Add the environment variable:```shexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64```Don't forget to reboot the machine to let the system detect the GPU.  dOD Sample preparationFor the sample preparation, a `dataWrapper` base object is made targeting to standardizethe sample API. Three major functions:1. `pop` (vritual) : generate a single sample2. `generate`: generate a batch of samples.3. `show` : generate few samples with visualizationFor new sample generator, inherit the `dataWrapper`module in `sample` folder and implement the `pop` for different sample object.<jupyter_code>import sys
sys.path.insert(1, '/home/tabris/flShrine/dOD/sample')

import sample_v0 as sp

sg = sp.sample_v0('geo0', 200, 200, 12)
sg.show()<jupyter_output><empty_output><jupyter_text>Model structureA good website for ML: [Dive into deep learning](http://d2l.ai/index.html) Convolutional layerRefering to this [introduction](https://cs231n.github.io/convolutional-networks/).Given an input tensor $x$ with the shape $W_0\times H_0\times D_0$, a 2D convolutional layer with a $W\times H\times D$ kernel $K$ has $(W\times H\times D_0+1)\times D$ the independent variables (including biases) based on the sharing parameter strategy. They are $D$ filters and each of them has $D_0$ layers ($L^j_i$, the i-th layer of the j-th filter) with shape $W\times H$. One bias is assigned to each filter. Denote the convolution operation between $A$ and $B$ by $A\circ B$, then the j-th depth convolution layer output can be expressed as$$O^j=b_j+\sum_{i=1}^{D_0}L^j_i\circ x_i$$ SoftdropThe function of this layer is to prevent the over-fitting. It will randomly disable some of the elements from the input, and the probablity for disabling the elements are called drop-rate. Max pool layerThe function of the max pool layer is preventing the over-fitting by reducing the outputs from selecting the maximum value among the inputs within a loca region as the outputs. For instance, for a size $W\times H$ max pooling layer, it will slice the input into a many small region with shape $W\times H$, (padding or stride may apply). And only the maximum value of each slices will be the output from that region. Transpose convolutional layerMore conceptial detials could be found [here](http://d2l.ai/chapter_computer-vision/transposed-conv.html)  Loss function and accuracy cross-entropy standardSuppose the prediction output is a tensor $p_{i}$ and the truth is $x_{i}$, then the cross entroy py is defined as$$H(x,p) = \frac{1}{N}\sum_{i=1}^N-x_i\log p_i,$$where the truth $x_i$ can be either 0 or 1 and $p_i<1$. The summary of the feagures:* Only the preidiction on the truth bit will affect the loss, no penlty/reward for the prediction on other bits.* Exponentially sensetive for the predictoin on the truth bit. Binary cross-entropySuppose the prediction $p_i$ is a binary variable (either 0 or 1), the cross-entropy could be modified to be$$H'(x,p)= \frac{1}{N}\sum_{i=1}^N-x_i\log p_i-(1-x_i)\log (1-p_i),$$which makes the wrong prediction takes penalty as well. categorical cross-entropy  U-net attempation<jupyter_code>import sys
sys.path.insert(1, '/home/tabris/flShrine/dOD/tf_model')

%load_ext tensorboard
import tensorflow as tf
from model import *

m = mynet(input_shape=(572,572,10),drop_rate=0.2)
m.buildNet(True)

%tensorboard --logdir logs<jupyter_output>The tensorboard extension is already loaded. To reload it, use:
  %reload_ext tensorboard
------------------downstream layer 0  shape:  (None, 284, 284, 64)
------------------downstream layer 1  shape:  (None, 140, 140, 128)
------------------downstream layer 2  shape:  (None, 68, 68, 256)
------------------downstream layer 3  shape:  (None, 32, 32, 512)
------------------downstream layer 4  shape:  (None, 28, 28, 1024)
------------------upstream layer 3  shape  (None, 52, 52, 512)
------------------upstream layer 2  shape  (None, 100, 100, 256)
------------------upstream layer 1  shape  (None, 196, 196, 128)
------------------upstream layer 0  shape  (None, 388, 388, 64)
------------------output shape  (None, 388, 388, 2)
StarcoderdataJupyterStructuredCleanDedup