<jupyter_start><jupyter_text>Wrangle Flight Data 2008 Author <NAME>. In this project we are going to analyze:1.	On-ground performance of various airlines and airports operating in United States during 2008. 2.	Connectivity map and airline frequency.3.	Cause of delay.We need perform following data wrangling steps before doing any analysis.1.	Gather data2.	Assess data3.	Clean data  Gather DataFor this project we have manually downloaded __[2008.csv,airports.csv, carriers.csv](http://stat-computing.org/dataexpo/2009/the-data.html)__ and loaded the data into filghts_2008 data frame.<jupyter_code>import pandas as pd
import datetime
flights_2008 = pd.read_csv("2008.csv")<jupyter_output><empty_output><jupyter_text>Assess DataWe will perform visual and programmatic assessment of data in filghts_2008 data frame.  Visual AssessmentFrom below visual assessment we find:1.	Following columns are loaded as float64 instead of Time    * DepTime    * ArrTime2.	Following columns are loaded as int64 instead of Time    * CRSDepTime    * CRSArrTime3.	Year, Month and Day of Month are separate int64 attributes instead of Date.4.	Following columns are loaded as int64 instead of Boolean    * Cancelled    * Diverted5.	We will create a new column with Boolean Delayed, which will populate if flight is delayed.    __Note__: In this case we cannot restructure delay time into one column because single delay can be caused by more than one reason.6.	We will create a new column Route of type object which is a Origin + “-“ + Dest.7. Change Carrier, Origin and Dest to Categorical.<jupyter_code>flights_2008.info();<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 7009728 entries, 0 to 7009727
Data columns (total 29 columns):
Year                 int64
Month                int64
DayofMonth           int64
DayOfWeek            int64
DepTime              float64
CRSDepTime           int64
ArrTime              float64
CRSArrTime           int64
UniqueCarrier        object
FlightNum            int64
TailNum              object
ActualElapsedTime    float64
CRSElapsedTime       float64
AirTime              float64
ArrDelay             float64
DepDelay             float64
Origin               object
Dest                 object
Distance             int64
TaxiIn               float64
TaxiOut              float64
Cancelled            int64
CancellationCode     object
Diverted             int64
CarrierDelay         float64
WeatherDelay         float64
NASDelay             float64
SecurityDelay        float64
LateAircraftDelay    float64
dtypes: float64(14), int64(10), object(5)
memory usage: 1.5+ GB<jupyter_text>Programmatic AssessmentFrom below programmatic assessment we find:1.	Following columns have nan values:    * DepTime – Missing for cancelled flights    * ArrTime  – Missing for cancelled or diverted flights    * TailNum – Missing for cancelled or diverted flights. There are 5 records with no tail number. We are not going to utilize this column in our analysis.    * ActualElapsedTime – Missing for cancelled or diverted flights    * CRSElapsedTime – Missing for cancelled or diverted flights    * AirTime  – Missing for cancelled or diverted flights    * ArrDelay – Missing for cancelled or diverted flights    * DepDelay – Missing for cancelled or diverted flights    * TaxiIn – Missing for cancelled or diverted flights    * TaxiOut – Missing for cancelled or diverted flights    * CancellationCode – Missing for non-cancelled flight    * CarrierDelay – Missing for non-delayed flights    * WeatherDelay – Missing for non-delayed flights    * NASDelay – Missing for non-delayed flights    * SecurityDelay – Missing for non-delayed flights    * LateAircraftDelay – Missing for non-delayed flights<jupyter_code>flights_2008.isna().any()
len(flights_2008[(flights_2008.DepTime.isna())])
len(flights_2008[(flights_2008.DepTime.isna()) & ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[(flights_2008.ArrTime.isna())])
len(flights_2008[(flights_2008.ArrTime.isna()) & ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[(flights_2008.TailNum.isna())])
len(flights_2008[(flights_2008.TailNum.isna()) & ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
flights_2008[(flights_2008.TailNum.isna()) & ((flights_2008.Cancelled == 0) & (flights_2008.Diverted == 0))]
len(flights_2008[flights_2008.ActualElapsedTime.isna()])
len(flights_2008[(flights_2008.ActualElapsedTime.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.AirTime.isna()])
len(flights_2008[(flights_2008.AirTime.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.ArrDelay.isna()])
len(flights_2008[(flights_2008.ArrDelay.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.DepDelay.isna()])
len(flights_2008[(flights_2008.DepDelay.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.TaxiIn.isna()])
len(flights_2008[(flights_2008.TaxiIn.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.TaxiOut.isna()])
len(flights_2008[(flights_2008.TaxiOut.isna())& ((flights_2008.Cancelled == 1) | (flights_2008.Diverted == 1))])
len(flights_2008[flights_2008.CancellationCode.isna()])
len(flights_2008[(flights_2008.CancellationCode.isna())& (flights_2008.Cancelled == 0)])
len(flights_2008[flights_2008.CarrierDelay.isna()])
len(flights_2008[flights_2008.WeatherDelay.isna()])
len(flights_2008[flights_2008.NASDelay.isna()])
len(flights_2008[flights_2008.SecurityDelay.isna()])
len(flights_2008[flights_2008.LateAircraftDelay.isna()])<jupyter_output><empty_output><jupyter_text>From the above visual and programmatic assessment, we find following quality and tidiness issues. Quality issues:1. Following columns have nan values:    * DepTime – Missing for cancelled flights    * ArrTime  – Missing for cancelled or diverted flights    * ActualElapsedTime – Missing for cancelled or diverted flights    * CRSElapsedTime – Missing for cancelled or diverted flights    * AirTime  – Missing for cancelled or diverted flights    * ArrDelay – Missing for cancelled or diverted flights    * DepDelay – Missing for cancelled or diverted flights    * TaxiIn – Missing for cancelled or diverted flights    * TaxiOut – Missing for cancelled or diverted flights    * CancellationCode – Missing for non-cancelled flight    * CarrierDelay – Missing for non-delayed flights    * WeatherDelay – Missing for non-delayed flights    * NASDelay – Missing for non-delayed flights    * SecurityDelay – Missing for non-delayed flights    * LateAircraftDelay – Missing for non-delayed flights2. Following columns are loaded as float64 instead of Time    * DepTime    * ArrTime3. Following columns are loaded as int64 instead of Time    * CRSDepTime    * CRSArrTime4. Following columns are loaded as int64 instead of Boolean    * Cancelled    * Diverted5. Change Carrier, Origin and Dest to Categorical. Tidiness issues:1. Year, Month and Day of Month are separate int64 attributes instead of Date. We will delete individual fields and create consolidated date time columns for CRSDep, CRSArr, Dep and Arr. There are some time fields with value 24:00 which should change date to next date.2.	We will create a new column with Boolean Delayed, which will populate if flight is delayed.    __Note__: In this case we cannot restructure delay time into one column because single delay can be caused by more than one reason.3.	We will create a new column Route of type object which is a Origin + “-“ + Dest.4. TailNum column will be deleted as we are not going to utilize this column in our analysis.  Clean Data__Note__ as the dataframe memory foot print is large we are not going to create a new clean copy.  Quality issues:1. Following columns have nan values:    * DepTime – Missing for cancelled flights    * ArrTime  – Missing for cancelled or diverted flights    * ActualElapsedTime – Missing for cancelled or diverted flights    * CRSElapsedTime – Missing for cancelled or diverted flights    * AirTime  – Missing for cancelled or diverted flights    * ArrDelay – Missing for cancelled or diverted flights    * DepDelay – Missing for cancelled or diverted flights    * TaxiIn – Missing for cancelled or diverted flights    * TaxiOut – Missing for cancelled or diverted flights    * CancellationCode – Missing for non-cancelled flight    * CarrierDelay – Missing for non-delayed flights    * WeatherDelay – Missing for non-delayed flights    * NASDelay – Missing for non-delayed flights    * SecurityDelay – Missing for non-delayed flights    * LateAircraftDelay – Missing for non-delayed flights2. Following columns are loaded as float64 instead of Time    * DepTime    * ArrTime3. Following columns are loaded as int64 instead of Time    * CRSDepTime    * CRSArrTime4. Following columns are loaded as int64 instead of Boolean    * Cancelled    * Diverted5. Change Carrier, Origin and Dest to Categorical.  CodeDepTime – Missing for cancelled flights<jupyter_code>flights_2008.DepTime.dtype
flights_2008.DepTime.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestDepTime – Missing for cancelled flights<jupyter_code>len(flights_2008[flights_2008.DepTime.isna()])<jupyter_output><empty_output><jupyter_text>CodeArrTime  – Missing for cancelled or diverted flights<jupyter_code>flights_2008.ArrTime.dtype
flights_2008.ArrTime.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestArrTime  – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.ArrTime.isna()])<jupyter_output><empty_output><jupyter_text>CodeActualElapsedTime – Missing for cancelled or diverted flights<jupyter_code>flights_2008.ActualElapsedTime.dtype
flights_2008.ActualElapsedTime.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestActualElapsedTime – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.ActualElapsedTime.isna()])<jupyter_output><empty_output><jupyter_text>CodeCRSElapsedTime – Missing for cancelled or diverted flights<jupyter_code>flights_2008.CRSElapsedTime.dtype
flights_2008.CRSElapsedTime.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestCRSElapsedTime – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.CRSElapsedTime.isna()])<jupyter_output><empty_output><jupyter_text>CodeAirTime  – Missing for cancelled or diverted flights<jupyter_code>flights_2008.AirTime.dtype
flights_2008.AirTime.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestAirTime  – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.AirTime.isna()])<jupyter_output><empty_output><jupyter_text>CodeArrDelay – Missing for cancelled or diverted flight<jupyter_code>flights_2008.ArrDelay.dtype
flights_2008.ArrDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestArrDelay – Missing for cancelled or diverted flight<jupyter_code>len(flights_2008[flights_2008.ArrDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeDepDelay – Missing for cancelled or diverted flights<jupyter_code>flights_2008.DepDelay.dtype
flights_2008.DepDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestDepDelay – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.DepDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeTaxiIn – Missing for cancelled or diverted flights<jupyter_code>flights_2008.TaxiIn.dtype
flights_2008.TaxiIn.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestTaxiIn – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.TaxiIn.isna()])<jupyter_output><empty_output><jupyter_text>CodeTaxiOut – Missing for cancelled or diverted flights<jupyter_code>flights_2008.TaxiOut.dtype
flights_2008.TaxiOut.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestTaxiOut – Missing for cancelled or diverted flights<jupyter_code>len(flights_2008[flights_2008.TaxiOut.isna()])<jupyter_output><empty_output><jupyter_text>CodeCancellationCode – Missing for non-cancelled flights<jupyter_code>flights_2008.CancellationCode.dtype
flights_2008.CancellationCode.fillna(value='',inplace=True)<jupyter_output><empty_output><jupyter_text>TestCancellationCode – Missing for non-cancelled flights<jupyter_code>len(flights_2008[flights_2008.CancellationCode.isna()])<jupyter_output><empty_output><jupyter_text>CodeCarrierDelay – Missing for non-delayed flights<jupyter_code>flights_2008.CarrierDelay.dtype
flights_2008.CarrierDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestCarrierDelay – Missing for non-delayed flights<jupyter_code>len(flights_2008[flights_2008.CarrierDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeWeatherDelay – Missing for non-delayed flights<jupyter_code>flights_2008.WeatherDelay.dtype
flights_2008.WeatherDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestWeatherDelay – Missing for non-delayed flights<jupyter_code>len(flights_2008[flights_2008.WeatherDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeNASDelay – Missing for non-delayed flights<jupyter_code>flights_2008.NASDelay.dtype
flights_2008.NASDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestNASDelay – Missing for non-delayed flights<jupyter_code>len(flights_2008[flights_2008.NASDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeSecurityDelay – Missing for non-delayed flights<jupyter_code>flights_2008.SecurityDelay.dtype
flights_2008.SecurityDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestSecurityDelay – Missing for non-delayed flights<jupyter_code>len(flights_2008[flights_2008.SecurityDelay.isna()])<jupyter_output><empty_output><jupyter_text>CodeLateAircraftDelay – Missing for non-delayed flights<jupyter_code>flights_2008.LateAircraftDelay.dtype
flights_2008.LateAircraftDelay.fillna(value=0.0,inplace=True)<jupyter_output><empty_output><jupyter_text>TestLateAircraftDelay – Missing for non-delayed flights<jupyter_code>len(flights_2008[flights_2008.LateAircraftDelay.isna()])<jupyter_output><empty_output><jupyter_text>TestNow that we have cleaned identified columns that contain NAN value, below code should return False for all columns except "TailNum" which we are going to delete later in the cleaning process.<jupyter_code>flights_2008.isna().any()<jupyter_output><empty_output><jupyter_text>CodeBelow code also covers Quality issues:2. Following columns are loaded as float64 instead of Time    * DepTime    * ArrTime3. Following columns are loaded as int64 instead of Time    * CRSDepTime    * CRSArrTime     Tidiness issues:1. Year, Month and Day of Month are separate int64 attributes instead of Date. We will delete individual fields and create consolidated date time columns for CRSDep, CRSArr, Dep and Arr. There are some time fields with value 24:00 which should change date to next date.4. TailNum column will be deleted as we are not going to utilize this column in our analysis.<jupyter_code>flights_2008['DepTime']=flights_2008.DepTime.astype(int).astype(str)
flights_2008.DepTime.dtype
len(flights_2008[(flights_2008.DepTime.str.len() < 4)])
flights_2008['DepTime']=flights_2008.DepTime.str.rjust(4,'0')
len(flights_2008[(flights_2008.DepTime.str.len() < 4)])
flights_2008['DepTime']=flights_2008.DepTime.str[:2]+":"+flights_2008.DepTime.str[2:]
flights_2008['ArrTime']=flights_2008.ArrTime.astype(int).astype(str)
flights_2008.ArrTime.dtype
len(flights_2008[(flights_2008.ArrTime.str.len() < 4)])
flights_2008['ArrTime']=flights_2008.ArrTime.str.rjust(4,'0')
len(flights_2008[(flights_2008.ArrTime.str.len() < 4)])
flights_2008['ArrTime']=flights_2008.ArrTime.str[:2]+":"+flights_2008.ArrTime.str[2:]
flights_2008['CRSDepTime']=flights_2008.CRSDepTime.astype(str)
flights_2008.CRSDepTime.dtype
len(flights_2008[(flights_2008.CRSDepTime.str.len() < 4)])
flights_2008['CRSDepTime']=flights_2008.CRSDepTime.str.rjust(4,'0')
len(flights_2008[(flights_2008.CRSDepTime.str.len() < 4)])
flights_2008['CRSDepTime']=flights_2008.CRSDepTime.str[:2]+":"+flights_2008.CRSDepTime.str[2:]
flights_2008['CRSArrTime']=flights_2008.CRSArrTime.astype(str)
flights_2008.CRSArrTime.dtype
len(flights_2008[(flights_2008.CRSArrTime.str.len() < 4)])
flights_2008['CRSArrTime']=flights_2008.CRSArrTime.str.rjust(4,'0')
len(flights_2008[(flights_2008.CRSArrTime.str.len() < 4)])
flights_2008['CRSArrTime']=flights_2008.CRSArrTime.str[:2]+":"+flights_2008.CRSArrTime.str[2:]
len(flights_2008[flights_2008.DepTime=='24:00']),len(flights_2008[flights_2008.ArrTime=='24:00'])
len(flights_2008[flights_2008.CRSDepTime=='24:00']),len(flights_2008[flights_2008.CRSArrTime=='24:00'])
flights_2008[(flights_2008.DepTime=='24:00')&(flights_2008.ArrTime=='24:00')&(flights_2008.CRSArrTime=='24:00')]
flights_2008['DepTimeDate'] = pd.to_datetime(flights_2008.Year.astype(str) +"-"+flights_2008.Month.astype(str)+"-"+flights_2008.DayofMonth.astype(str))
flights_2008['ArrTimeDate'] = flights_2008['DepTimeDate']
flights_2008['CRSArrTimeDate'] = flights_2008['DepTimeDate']
flights_2008['CRSDepTimeDate'] = flights_2008['DepTimeDate']
flights_2008['DepTimeDate'] = flights_2008[flights_2008.DepTime=='24:00'].DepTimeDate+ datetime.timedelta(days=1)
len(flights_2008[flights_2008.DepTimeDate.isna()])
flights_2008.DepTimeDate.fillna(flights_2008.ArrTimeDate,inplace=True)
len(flights_2008[flights_2008.DepTimeDate.isna()])
flights_2008['ArrTimeDate'] =flights_2008[flights_2008.ArrTime=='24:00'].ArrTimeDate+ datetime.timedelta(days=1)
len(flights_2008[flights_2008.ArrTimeDate.isna()])
flights_2008.ArrTimeDate.fillna(flights_2008.DepTimeDate,inplace=True)
len(flights_2008[flights_2008.ArrTimeDate.isna()])
flights_2008['CRSArrTimeDate'] =flights_2008[flights_2008.CRSArrTime=='24:00'].CRSArrTimeDate+ datetime.timedelta(days=1)
len(flights_2008[flights_2008.CRSArrTimeDate.isna()])
flights_2008.CRSArrTimeDate.fillna(flights_2008.DepTimeDate,inplace=True)
len(flights_2008[flights_2008.CRSArrTimeDate.isna()])
flights_2008.DepTime.replace("24:00","00:00",inplace=True);
flights_2008.CRSDepTime.replace("24:00","00:00",inplace=True);
flights_2008.ArrTime.replace("24:00","00:00",inplace=True);
flights_2008.CRSArrTime.replace("24:00","00:00",inplace=True);
flights_2008['DepDateTime']=pd.to_datetime(flights_2008.DepTimeDate.astype(str)+" "+flights_2008.DepTime);
flights_2008['CRSDepDateTime']=pd.to_datetime(flights_2008.CRSDepTimeDate.astype(str)+" "+flights_2008.CRSDepTime);
flights_2008['ArrDateTime']=pd.to_datetime(flights_2008.ArrTimeDate.astype(str)+" "+flights_2008.ArrTime);
flights_2008['CRSArrDateTime']=pd.to_datetime(flights_2008.CRSArrTimeDate.astype(str)+" "+flights_2008.CRSArrTime);
col_del_list = ['Year','Month','DayofMonth','DepTime','CRSDepTime','ArrTime','CRSArrTime','TailNum','DepTimeDate','ArrTimeDate','CRSArrTimeDate','CRSDepTimeDate']
flights_2008.drop(col_del_list,axis=1,inplace=True)<jupyter_output><empty_output><jupyter_text>Test<jupyter_code>flights_2008.info()
flights_2008.isna().any()
flights_2008.head()<jupyter_output><empty_output><jupyter_text>Code Quality issues:4. Following columns are loaded as int64 instead of Boolean    * Cancelled    * Diverted<jupyter_code>flights_2008.Cancelled.dtype
flights_2008.Diverted.dtype
flights_2008['Cancelled']=flights_2008.Cancelled.astype(bool)
flights_2008['Diverted']=flights_2008.Diverted.astype(bool)<jupyter_output><empty_output><jupyter_text>Test<jupyter_code>flights_2008.Cancelled.dtype
flights_2008.Diverted.dtype
flights_2008[['Cancelled','Diverted']].head()<jupyter_output><empty_output><jupyter_text>CodeWe will create a new column Route of type object which is a Origin + “-“ + Dest.<jupyter_code>flights_2008['Route'] = flights_2008.Origin +" - "+flights_2008.Dest<jupyter_output><empty_output><jupyter_text>Test<jupyter_code>flights_2008.head()<jupyter_output><empty_output><jupyter_text>Code5. Change Carrier, Origin and Dest to Categorical.<jupyter_code>flights_2008.UniqueCarrier.dtype,flights_2008.Origin.dtype,flights_2008.Dest.dtype
flights_2008['UniqueCarrier']=flights_2008.UniqueCarrier.astype('category')
flights_2008['Origin']=flights_2008.Origin.astype('category')
flights_2008['Dest']=flights_2008.Dest.astype('category')<jupyter_output><empty_output><jupyter_text>Test<jupyter_code>flights_2008.UniqueCarrier.dtype,flights_2008.Origin.dtype,flights_2008.Dest.dtype<jupyter_output><empty_output><jupyter_text>CodeWe will create a new column with Boolean Delayed, which will populate if flight is delayed.<jupyter_code>flights_2008['Delayed']=(flights_2008.CarrierDelay >0) |(flights_2008.WeatherDelay >0) | (flights_2008.NASDelay >0) |(flights_2008.SecurityDelay >0) |(flights_2008.LateAircraftDelay >0)<jupyter_output><empty_output><jupyter_text>Test<jupyter_code>flights_2008[flights_2008.Delayed].head()<jupyter_output><empty_output><jupyter_text>Checkpoint and StorageWe have addressed below issues in data wrangling. Now we will store the dataframe into new CSV for Tableau.1. Following columns have nan values:    * DepTime – Missing for cancelled flights    * ArrTime  – Missing for cancelled or diverted flights    * ActualElapsedTime – Missing for cancelled or diverted flights    * CRSElapsedTime – Missing for cancelled or diverted flights    * AirTime  – Missing for cancelled or diverted flights    * ArrDelay – Missing for cancelled or diverted flights    * DepDelay – Missing for cancelled or diverted flights    * TaxiIn – Missing for cancelled or diverted flights    * TaxiOut – Missing for cancelled or diverted flights    * CancellationCode – Missing for non-cancelled flight    * CarrierDelay – Missing for non-delayed flights    * WeatherDelay – Missing for non-delayed flights    * NASDelay – Missing for non-delayed flights    * SecurityDelay – Missing for non-delayed flights    * LateAircraftDelay – Missing for non-delayed flights2. Following columns are loaded as float64 instead of Time    * DepTime    * ArrTime3. Following columns are loaded as int64 instead of Time    * CRSDepTime    * CRSArrTime4. Following columns are loaded as int64 instead of Boolean    * Cancelled    * Diverted5. Change Carrier, Origin and Dest to Categorical.6. Year, Month and Day of Month are separate int64 attributes instead of Date. We will delete individual fields and create consolidated date time columns for CRSDep, CRSArr, Dep and Arr. There are some time fields with value 24:00 which should change date to next date.7.	We will create a new column with Boolean Delayed, which will populate if flight is delayed.    __Note__: In this case we cannot restructure delay time into one column because single delay can be caused by more than one reason.8.	We will create a new column Route of type object which is a Origin + “-“ + Dest.9. TailNum column will be deleted as we are not going to utilize this column in our analysis.<jupyter_code>flights_2008.to_csv("flights_2008.csv",index=False)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup