<jupyter_start><jupyter_text>**Computer Vision Basics**This notebook covers multiple topics relevant for computer vision and machine learning preparation.  **Setup**<jupyter_code># import the modules 
import numpy as np
import matplotlib.pyplot as plt
import cv2
from google.colab.patches import cv2_imshow

# get an example
from PIL import Image
import requests
im = Image.open(requests.get('https://images.unsplash.com/photo-1525697472245-fb8fd8594791?ixlib=rb-1.2.1&auto=format&fit=crop&w=2100&q=80', stream=True).raw)
im<jupyter_output><empty_output><jupyter_text>**What's an array?**Arrays in NumPy are multi-dimensional and can represent vectors, matrices, and images. An array is much like a list (or list of lists) but is restricted to having all elements of the same type. Unless specified on creation, the type will automatically be set depending on the data. After reading images to NumPy arrays, we can perform any mathematical operation we like on the image.<jupyter_code>im = np.asarray(im)<jupyter_output><empty_output><jupyter_text>**What is an image from a machine's perspective?**They are defined as matrices of numbers representing the discrete color or intensity values present in every image pixel. Each image is considered as input data displayable in numerous ways, whether as arrays of pixel values or either multidimensional plots representing the distribution of pixel intensities.<jupyter_code># can we still see the picture?
plt.imshow(im)
# What are the dimensions of our image?
print(im.shape)
im.shape[0]*im.shape[1]<jupyter_output><empty_output><jupyter_text>**Basic Annotations**<jupyter_code># How can we crop the car?
plt.imshow(im[900:,400:1750,:])
plt.show()
# How can we mirror the picture on a vertical line?
plt.imshow(im[:,::-1,:])
plt.show()
# How can we reverse color order?
plt.imshow(im[:,:,::-1])
plt.show()
# How can we rotate the image by 90 degrees?
plt.imshow(np.swapaxes(im,0,1))
plt.show()<jupyter_output><empty_output><jupyter_text>**Reverse Engineer Color Logic**<jupyter_code># Let's build an image, with 0s only
new_im = np.zeros([100,100,3])
new_im = new_im.astype(int)
plt.imshow(new_im)
# Let's make one color dimension = 100
new_im[:,:,2] = int(255)
plt.imshow(new_im)
# Let's filter to one color channel only for our original image 'im'
im_2 = im.copy()
im_2[:,:,1:] = np.zeros([im.shape[0],im.shape[1],2])
plt.imshow(im_2)
# Let's mix green and red for new_im
new_im[:,:,0] = int(255)
new_im[:,:,1] = int(255)
new_im[:,:,2] = int(0)
plt.imshow(new_im)<jupyter_output><empty_output><jupyter_text>**Monochromic Images**Monochromic images only have one 'color' dimension, and may be used to represent all sorts of 2D data (e.g., heatmaps, terrain, maps)<jupyter_code># For simplicity, let's only look at one of our color dimensions and create a grayscale image
plt.imshow(np.average(im,axis=2),cmap='gray')<jupyter_output><empty_output><jupyter_text>**Playtime** Take the following code and create an artsy image<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

image = np.zeros([100,100,3])
image = image.astype(int)


'''
Your code: Feel free to use and combine any basic operations and numpy functions you want. 
You may also want to use loops.

Once you are happy with your work, feel free to copy your picture and send it to:
jasper dot schwenzow dot uni dash hamburg dot de.

'''

for i in range(80):
  image[:,i,0] = np.linspace(50,255,100)
  image[i,:,2] = np.linspace(50,255,100)

for i in range(20,80):
  image[:,i,0] = np.linspace(255,50,100)
  image[i,:,2] = np.linspace(255,50,100)

plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Other Annotations Using cv2** **Image interpolation**It occurs when you resize or distort your image from one pixel grid to another. Image resizing is necessary when you need to increase or decrease the total number of pixels, whereas remapping can occur when you are correcting for lens distortion or rotating an image. Zooming refers to increase the quantity of pixels, so that when you zoom an image, you will see more detail.<jupyter_code># let's get a random image
np.random.seed(7)
grid = np.random.rand(4,4)
print(grid.shape)
plt.imshow(grid)
# how can we resize the image
resized = cv2.resize(grid, (8,8), interpolation = cv2.INTER_CUBIC)
resized.shape
plt.imshow(resized)
plt.show()
plt.imshow(grid)
plt.show()<jupyter_output><empty_output><jupyter_text>**Overview - Interpolations for plt.show()**To better see the differences in using different interpolations, the following example shows the changes due to interpolations.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np

methods = [None, 'none', 'nearest', 'bilinear', 'bicubic', 'spline16',
           'spline36', 'hanning', 'hamming', 'hermite', 'kaiser', 'quadric',
           'catrom', 'gaussian', 'bessel', 'mitchell', 'sinc', 'lanczos']

# Fixing random state for reproducibility
np.random.seed(42)

grid = np.random.rand(4, 4)

fig, axs = plt.subplots(nrows=3, ncols=6, figsize=(9, 6),
                        subplot_kw={'xticks': [], 'yticks': []})
for ax, interp_method in zip(axs.flat, methods):
    ax.imshow(grid, interpolation=interp_method, cmap='viridis')
    ax.set_title(str(interp_method))

plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>**Contrast Adjustment**<jupyter_code>alpha = 2.0   # Contrast Control [1.0-3.0]
beta = 0    # Brightness Control [0-100]

# Scaling and converting the image contrast and brightness
adjusted_image = cv2.convertScaleAbs(im, alpha=alpha, beta=beta)

# Displaying the adusted image
plt.imshow(adjusted_image)
result_denoising = cv2.fastNlMeansDenoisingColored(im,None,20,10,7,21)
plt.imshow(result_denoising)<jupyter_output><empty_output><jupyter_text>**Kernel Operations**<jupyter_code># let's take the grayscale image again
gray = np.average(im,2)
gray = cv2.resize(gray,(420,280))
plt.imshow(gray,cmap='gray')
# Can we replicate the top sobel kernel?
kernel = np.zeros((3,3))

kernel[0,:] = 1*np.array([1,2,1])
kernel[-1,:] = -1*np.array([1,2,1])
print(kernel)

im_2 = cv2.filter2D(gray, -1,kernel)
cv2_imshow(im_2)

# Let's get a smaller image and also fix RGB to BGR to use cv2_imshow
im2 = cv2.resize(im[:,:,::-1], (420, 280))
cv2_imshow(im2)<jupyter_output><empty_output><jupyter_text>**Image Blurring (Image Smoothing)**<jupyter_code># Can we get distorting blurriness??
kernel = np.zeros((30,3))

for i in range(30):
  kernel[i,:] = 1*np.array([1,2,1])

kernel = kernel/kernel.sum()

im_kernel = cv2.filter2D(im2,-1,kernel)

cv2_imshow(im_kernel)
# cv2 has some prebuild functions
averaging_blur = cv2.blur(im2,(3,30))
cv2_imshow(averaging_blur)
# Gaussian blur
gaussianblur_image = cv2.GaussianBlur(im2, (7,7), 0)
cv2_imshow(gaussianblur_image)
# Median blur
medianblur_image = cv2.medianBlur(im2,7)
cv2_imshow(medianblur_image)<jupyter_output><empty_output><jupyter_text>**Edge Detection**<jupyter_code>edge_im = cv2.Canny(im2,100,200) # minVal and maxVal are the minimum and maximum intensity gradient values
cv2_imshow(edge_im)<jupyter_output><empty_output><jupyter_text>**Histogram Equalization**  It's used as a "reference tool" to make all images with same lighting conditions. For example, it is useful in face recognition, before training the face data, the images of faces are histogram equalized to make them all with same lighting conditions. So, what's the goal? - Visualizing the set of lightness values for an image to improve the contrast of an image- A good image will have pixels from all regions of the image -> you need to strech the histogram to either ends - Real objective: To find an automated process to equally distribute light across the entire image without having to check area by area, pixel by pixel Step 1: Plotting a histogram showing intensity distribution by color channel:<jupyter_code>plt.imshow(im2[:,:,1])
plt.show()
plt.imshow(cv2.equalizeHist(im2[:,:,1]))
plt.show()
hist,bins = np.histogram(im2.flatten(),256,[0,256])
cdf = hist.cumsum()
float(hist.max()) / cdf.max()
cdf_normalized = cdf * float(hist.max()) / cdf.max()
plt.plot(cdf_normalized, color = 'b')
plt.hist(im2.flatten(),256,[0,256], color = 'r')
plt.xlim([0,256])
plt.legend(('cumulated distribution','color histogram'), loc = 'upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>The Histogram shows that we do have abrupt variations in the range values.So now, we need a transformation function which maps the input pixels in brighter region to output pixels in full region (= histogram equalization).Step 2: We find the minimum histogram value (excluding 0) and apply the histogram equalization equation using the masked array concept array from Numpy. For masked array, all operations are performed on non-masked elements:<jupyter_code>cdf_m = np.ma.masked_equal(cdf,0)
cdf_m = (cdf_m - cdf_m.min())*255/(cdf_m.max()-cdf_m.min())
cdf = np.ma.filled(cdf_m,0).astype('uint8')
img2 = cdf[im2]<jupyter_output><empty_output><jupyter_text>Step 3: Plotting the new histogram and cdf<jupyter_code>hist,bins = np.histogram(img2.flatten(),256,[0,256])
cdf = hist.cumsum()
cdf_normalized = cdf * float(hist.max()) / cdf.max()
plt.plot(cdf_normalized, color = 'b')
plt.hist(img2.flatten(),256,[0,256], color = 'r')
plt.xlim([0,256])
plt.legend(('cumulated distribution','color histogram'), loc = 'upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>Step 4: Showing the "new" image<jupyter_code>cv2_imshow(img2)
cv2_imshow(im2)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup