<jupyter_start><jupyter_text>Assignment1. Sample a data set D of size n from N (x; μ, Σ). Fix a percentage ρ.2. Use the function get_theta(mu, cov, rho) provided by the notebook toobtain an explicit threshold θ given the percentage ρ. Note that θ is partof the ground-truth and therefore considered as unknown.3. Determine the true anomalies of D. For this, use the explicit threshold θtogether with the Mahalanobis distance d∗M defined by the true μ and Σ.4. Use the data D to estimate μ and Σ. Construct the Mahalanobis distancedM defined by the estimates ˆμ and ˆΣ.5. Predict the anomalies of D using the Mahalanobis distance dM and Eu-clidean distance dE . Anomalies are the ρ percent points xi ∈ D farthestfrom ˆμ (do not use θ). Assess precision and recall of both detectors.<jupyter_code># function to evaluate the whole assignment and save the plotted figure
def evaluate(mu_T=np.array([0, 0]), covariance=.9, rho=3.5, size=2000, savefig=True):
    # fixate groundtruth mean and covariance matrix for the bivariate gaussian distribution
    # '_T' nominator stands for groundtruth variable
    # '_E' nominator stands for estimated variable
    sigma_T = np.array([[1, covariance], [covariance, 1]])  # covariance matrix
    
    # 1. generate dataset (RandomVariableS)
    D = multivariate_normal.rvs(mean=mu_T, cov=sigma_T, size=size)
    
    # 2. use get_theta to get the 'groundtruth' explicit treshold theta
    theta = get_theta(mu_T, sigma_T, rho)
    
    # 3. determine subset of true anomalies of dataset D
    # start by calculating the mahalanobis distance of each point from the mean
    Z_T = D - mu_T
    d_m_T = np.sqrt(np.sum(Z_T.dot(inv(sigma_T)) * Z_T, axis=1))
    
    # filter out values (indices) over the groundtruth threshold theta (True / False array)
    I_T = d_m_T > theta  # indices of true anomalies with mahalanobis distance
    
    # 4. Use the data D to estimate mu and sigma
    mu_E = D.mean(axis=0)
    sigma_E = np.cov(D.T) 
    
    # Construct the Mahalanobis distance d_m_E defined by the estimates mu_E and sigma_E
    Z_E = D - mu_E
    d_m_E = np.sqrt(np.sum(Z_E.dot(inv(sigma_E)) * Z_E, axis=1))
    
    # construct euclidean distance d_e_E in the same manner (with mu_E and sigma_E)
    d_e_E = np.sqrt(np.sum(Z_E ** 2, axis=1)) 
    
    # 5. predict anomalies with estimated eucilidian (d_e_E) and mahalanobis distance (d_m_E)
    # create list of indices (True / False array) (on axis 0 of dataset)
    # estimated thresholds (eta) are rho percent points with the farthest distance from mu_E
    eta_m = np.percentile(d_m_E, 100-rho)
    eta_e = np.percentile(d_e_E, 100-rho) 

    I_m_E = d_m_E > eta_m
    I_e_E = d_e_E > eta_e 
    
    
    # Comparison:
    # Assess precision and recall of both detectors. (5)
    # calculate tp, fp and fn for euclidean distance and for mahalanobis distance

    # np.logical_and(I_m_T, I_m_E) [here: land] creates a logical AND mask over the two boolean arrays etc.
    # (I_m_T * I_m_E)
    tp_m = land(I_T, I_m_E).sum()
    tp_e = land(I_T, I_e_E).sum()

    fp_m = land(lin(I_T), I_m_E).sum()
    fp_e = land(lin(I_T), I_e_E).sum()

    fn_m = land(I_T, lin(I_m_E)).sum()
    fn_e = land(I_T, lin(I_e_E)).sum()

    # precisions and recalls mahalanobis (m) and euclidean (e) distance
    precision_m = tp_m / (tp_m + fp_m) 
    recall_m = tp_m / (tp_m + fn_m)

    precision_e = tp_e / (tp_e + fp_e)
    recall_e = tp_e / (tp_e + fn_e)

    print(f'precision euclidean : {precision_e}')
    print(f'recall euclidean : {recall_e}')
    print(f'precision mahalanobis : {precision_m}')
    print(f'recall mahalanobis : {recall_m}')
    
    # save figure
    if savefig:
        save_point_plot(D, I_T, f'./research_question_imgs/{rho}_groundtruth.png')
        save_point_plot(D, I_m_E, f'./research_question_imgs/{rho}_mahalanobis_estimated.png')
        save_point_plot(D, I_e_E, f'./research_question_imgs/{rho}_euclidean_estimated.png')
    
    
evaluate(rho=40)<jupyter_output>precision euclidean : 0.675
recall euclidean : 0.7267833109017496
precision mahalanobis : 0.92875
recall mahalanobis : 1.0<jupyter_text>Research question of choice : How do precision and recall depend on percentage __p__?<jupyter_code># loop through the whole evaluation process with always 
# augmenting values of rho
# save the anomalies plots for each percentage evaluation
# for groundtruth, estimated euclidean and mahalanobis distance
for i in range(1, 99, 3):
    print(f'\nrho = {i}')
    evaluate(rho=i)<jupyter_output>rho = 1
precision euclidean : 0.45
recall euclidean : 0.5
precision mahalanobis : 0.85
recall mahalanobis : 0.9444444444444444

rho = 4
precision euclidean : 0.5
recall euclidean : 0.5128205128205128
precision mahalanobis : 0.975
recall mahalanobis : 1.0

rho = 7
precision euclidean : 0.5214285714285715
recall euclidean : 0.4899328859060403
precision mahalanobis : 0.9857142857142858
recall mahalanobis : 0.9261744966442953

rho = 10
precision euclidean : 0.5
recall euclidean : 0.5154639175257731
precision mahalanobis : 0.965
recall mahalanobis : 0.9948453608247423

rho = 13
precision euclidean : 0.5692307692307692
recall euclidean : 0.5758754863813229
precision mahalanobis : 0.9615384615384616
recall mahalanobis : 0.9727626459143969

rho = 16
precision euclidean : 0.49375
recall euclidean : 0.5249169435215947
precision mahalanobis : 0.928125
recall mahalanobis : 0.9867109634551495

rho = 19
precision euclidean : 0.618421052631579
recall euclidean : 0.6056701030927835
precision mahalano[...]
StarcoderdataJupyterStructuredCleanDedup