<jupyter_start><jupyter_text>OverviewProblem Approach:The give data on predicting corrosion rate of carbon steel contains features such as1. Temperature 2. Time of wetness  3. sulfur dioxide 4. chloride concentrations 5. Exposure Time (Years)and doesn't contain two features "Relative humidity and precipitation " Models Used* [Lasso Regression](Lasso-Regression)* [Ridge Regression](Ridge-Regression)* [PCA with Ridge Regression](PCA-with-Ridge-Regression)* [Regression Trees](Regression-Trees)* [XG Boost with Regression](XG-Boost-with-Regression)* [ANN](A-N-N)Out of these models the XG Boost comes out to be the best model (with the R2 of 98.8% and 84.2% for training and testing datasets respectively) Summary: 1. The data is pretty clean with no missing values there aren't many outliers as well 2. The distribution of target variable is skewed which is some what corrected by applying log transformation 3. The Simple models of Lasso and Ridge reression gave moderate R2 values 4. The PCA is applied and as expected the performance is dropped, this is due to the low number of indipendent    variables and they are condensed into only single PCA component, so the loss of information  accounted for the    drop in the performance 5. So Regression tree is employed, The simple Regression tree got proned to overfitting with training value of 1 and    test value of 0.52, so hyper parameter tuning is done, but the performance didn't improve much compared to    previou models 6. Later XG boost with parameter tuning is applied and the model performed significantly well then the rest 7. The ANN model though comparatively better than others is not as good as the XG boost  But overall all the models colud be improved with more number of features such as Humidity and Precipitation   Import all the necessary libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler

from sklearn.linear_model import Ridge
from sklearn.linear_model import Lasso
from sklearn.model_selection import GridSearchCV
from sklearn import linear_model

from sklearn.metrics import r2_score

from sklearn.decomposition import IncrementalPCA
from sklearn.decomposition import PCA
from sklearn.metrics import accuracy_score,confusion_matrix,classification_report,roc_auc_score,precision_score,recall_score,f1_score
from sklearn.model_selection import GridSearchCV,StratifiedKFold,RandomizedSearchCV

from sklearn.tree import DecisionTreeRegressor

import xgboost

from sklearn import metrics

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input, Dense, Activation, Dropout
from tensorflow.keras.optimizers import Adam
df = pd.read_excel('Corrosion data.xlsx')
df.head()
df.info()
df = df.drop(df.columns[0], axis=1)
df = df.rename(columns = {"Temp©":"temp","TOW(%)":"time of wetness"}, inplace = False)
df.describe()<jupyter_output><empty_output><jupyter_text>EDA for checking correlations and distibution<jupyter_code>plt.figure(figsize=(16,8))
plt.subplot(2,3,1)
plt.scatter(df.temp,df["Corrosion depth in steel (mm)"])
plt.subplot(2,3,2)
plt.scatter(df["time of wetness"],df["Corrosion depth in steel (mm)"])
plt.subplot(2,3,3)
plt.scatter(df["SO2 (microgm/m^3)"],df["Corrosion depth in steel (mm)"])
plt.subplot(2,3,4)
plt.scatter(df["Cl-(mg/m^2/day)"],df["Corrosion depth in steel (mm)"])
plt.subplot(2,3,5)
plt.scatter(df["Exposure Time (Years)"],df["Corrosion depth in steel (mm)"])
plt.subplot(2,3,6)
plt.scatter(df["Corrosion depth in steel (mm)"],df["Corrosion depth in steel (mm)"])
plt.figure(figsize=(6,6))
ax = sns.heatmap(df.corr(),annot=True)
bottom, top = ax.get_ylim()
ax.set_ylim(bottom + 0.5, top - 0.5)
plt.show()
plt.figure(figsize=(8,6))
sns.distplot(df["Corrosion depth in steel (mm)"])
plt.show()
sns.distplot(np.log(df["Corrosion depth in steel (mm)"]))<jupyter_output><empty_output><jupyter_text>Data Preparation<jupyter_code>def cap_outliers(array, k=3):
    upper_limit = array.mean() + k*array.std()
    lower_limit = array.mean() - k*array.std()
    array[array<lower_limit] = lower_limit
    array[array>upper_limit] = upper_limit
    return array
df = df.apply(cap_outliers, axis=0)
df
df["Corrosion depth in steel (mm)"] = np.log(df["Corrosion depth in steel (mm)"])
X = df.drop(["Corrosion depth in steel (mm)"], axis=1).values
y = df["Corrosion depth in steel (mm)"].values
# split into train and test

X_train, X_test, y_train, y_test = train_test_split(X, y,train_size=0.7,test_size = 0.3, random_state=100)
scaler = MinMaxScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)<jupyter_output><empty_output><jupyter_text>Model 5: XG Boost with Regression<jupyter_code>n_estimators = [100, 500, 900, 1100, 1500]
max_depth = [2, 3, 5, 10, 15]
booster = ['gbtree', 'gblinear']
base_score = [0.25, 0.5, 0.75, 1]
learning_rate = [0.05, 0.1, 0.15, 0.20]
min_child_weight = [1, 2, 3, 4]
# Define the grid of hyperparameters to search
hyperparameter_grid = {
    'n_estimators': n_estimators,
    'max_depth': max_depth,
    'learning_rate' : learning_rate,
    'min_child_weight' : min_child_weight,
    'booster' : booster,
    'base_score' : base_score
    }
regressor = xgboost.XGBRegressor()
random_cv = RandomizedSearchCV(estimator=regressor, param_distributions=hyperparameter_grid, cv=5,n_iter=50, 
                               scoring = 'neg_mean_absolute_error',n_jobs = 4,
                               verbose = 5, return_train_score = True, random_state=42)
regressor = xgboost.XGBRegressor(base_score=0.25, 
                                 booster='gbtree', 
                                 learning_rate=0.1, 
                                 max_delta_step=0,
                                 max_depth=2, 
                                 min_child_weight=1, 
                                 n_estimators=900,
                                 verbosity=1)
regressor.fit(X_train, y_train)
y_train_pred = regressor.predict(X_train)
y_test_pred = regressor.predict(X_test)

print(r2_score(y_true=y_train,y_pred=y_train_pred))
print(r2_score(y_true=y_test,y_pred=y_test_pred))
Train5 = pd.DataFrame(y_train,y_train_pred)
Train5.reset_index(level=0, inplace=True)
Train5 = Train5.rename(columns = {'index': 'y train', 0: 'y train pred'}, inplace = False)
Test5 = pd.DataFrame(y_test,y_test_pred)
Test5.reset_index(level=0, inplace=True)
Test5 = Test5.rename(columns = {'index': 'y test', 0: 'y test pred'}, inplace = False)
Train5
Test5
import openpyxl
from openpyxl import load_workbook
workbook = openpyxl.load_workbook("multiple.xlsx")
writer = pd.ExcelWriter('multiple.xlsx', engine='openpyxl')
writer.book = workbook
writer.sheets = dict((ws.title, ws) for ws in workbook.worksheets)
Train5.to_excel(writer, sheet_name='Train5')
Test5.to_excel(writer, sheet_name='Test5')
writer.save()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup