<jupyter_start><jupyter_text>EDA In this notebook I perform EDA on listings.csv file<jupyter_code>#import required python packages
import numpy as np 
import matplotlib.pyplot as plt 
import pandas as pd 
import seaborn as sns 
%matplotlib inline
df_listings = pd.read_csv("listings.csv") #import the file
df_listings.head() #dataset overview
print("The given dataset contains {} listings and {} corresponding fields".format(df_listings.shape[0],df_listings.shape[1]))<jupyter_output>The given dataset contains 3818 listings and 92 corresponding fields<jupyter_text>The first task is to remove unwanted fields<jupyter_code>print("Columns in the dataframe:")
for column in df_listings.columns:
    print(column)
cols = df_listings.columns[df_listings.isna().sum() > 0.75*df_listings.shape[0]]
print("Columns with more than 75% data missing :{}".format(cols))
#removing unwanted columns 
df_listings = df_listings.drop(columns=['id', 'listing_url','square_feet','thumbnail_url', 'medium_url', 'picture_url','xl_picture_url', 'host_id', 'host_url', 'host_name','host_thumbnail_url', 'host_picture_url','scrape_id', 'last_scraped','license'])
# lisf of Smart locations and their counts 
df_listings.smart_location.value_counts()<jupyter_output><empty_output><jupyter_text>From the above data it is clear that almost all of smart locations are in Seattle.    In the following cells, I will explore details about hosts  Type of host verifications<jupyter_code>df_listings.host_verifications.astype(str)[1]
df_listings['host_identity_verified'].value_counts().plot(kind='pie', )
plt.title("Host indentity verification")
plt.ylabel("")
plt.show()
df_listings.host_has_profile_pic.value_counts().plot(kind='pie', title = 'Host profile picture')
plt.ylabel("")
plt.show()
df_listings['require_guest_phone_verification'].value_counts().plot(kind='pie', )
plt.title("Guest phone verification")
#plt.xlabel("require_guest_phone_verification")
plt.ylabel("")
plt.savefig("PhoneVerification.jpg")
plt.show()
df_listings.require_guest_profile_picture.value_counts().plot(kind='pie', title = 'Guest Profile picture requirement', use_index=False)
plt.ylabel("")
plt.show()<jupyter_output><empty_output><jupyter_text>In the following cells, I will explore details about the listings  Number of bedroom distribution<jupyter_code>df_listings['bedrooms'].value_counts().plot(kind='bar', )
#plt.title("# of Bedrooms")
plt.xlabel("# of Bedrooms")
plt.ylabel("Count")
plt.show()<jupyter_output><empty_output><jupyter_text>From the above data, it is clear that most of the rentals have a single bedroom<jupyter_code>df_listings['bathrooms'].value_counts().plot(kind='bar', )
plt.title("# of Bedrooms")
plt.xlabel("# of bathrooms")
plt.ylabel("Count")
plt.show()<jupyter_output><empty_output><jupyter_text>From the above data, it is clear that most of the rentals have a single bathroom<jupyter_code>df_listings['minimum_nights'].value_counts().plot(kind='bar', )
#plt.title("# Number of nights")
plt.xlabel("# of minimum_nights")
plt.ylabel("Count")
plt.show()
#maximum nights
df_listings['maximum_nights'].value_counts()[:5]<jupyter_output><empty_output><jupyter_text>From the above data, it is clear that most of the rentals require a minimum of  one or two night stay and many rentals offer rentals upto 6 years<jupyter_code>fig, ax = plt.subplots()
df_listings['room_type'].value_counts().plot(kind='pie' )
ax.set_title('Room type')
plt.ylabel("")
plt.show()<jupyter_output><empty_output><jupyter_text>Majority of the listings rent the entire home or apartment<jupyter_code>df_listings.cancellation_policy
df_listings['cancellation_policy'].value_counts().plot(kind='pie' )
#plt.title("# of Bedrooms")
plt.title("Cancellation policy")
plt.ylabel("")
plt.show()<jupyter_output><empty_output><jupyter_text>Cancellation policy is equally dirstibuted between 'flexible', 'strict' and 'moderate'<jupyter_code>df_listings['bed_type'].value_counts().plot(kind='barh', )
#plt.title("# of Bedrooms")
plt.title("bed_type")

plt.show()<jupyter_output><empty_output><jupyter_text>Cleaning fee<jupyter_code>df_cleaning_fee = df_listings.cleaning_fee.dropna().reset_index(drop=True)
df_cleaning_fee = df_cleaning_fee.str.replace(r'$', '').astype(float)
average_fee = round(df_cleaning_fee.describe()['mean'],2)
maximum_fee = round(df_cleaning_fee.describe()['max'],2)
mininmum_fee = round(df_cleaning_fee.describe()['min'],2)
print("The average cleaning fee is ${}, mininmum fee is ${} and maximum fee is ${}".format(average_fee,mininmum_fee,maximum_fee))
plt.hist(df_cleaning_fee)
plt.title("Histogram of cleaning fee")
plt.xlabel("Price")
plt.show()
df_listings.instant_bookable.value_counts().plot(kind='pie', title = 'Instant booking option')
plt.ylabel("")
plt.show()<jupyter_output><empty_output><jupyter_text>In the following cells I'll analyse columns more about hosts and super hosts<jupyter_code>df_listings.host_response_time.value_counts().plot(kind='pie', )
plt.ylabel("")
plt.title("Response time of hosts")
plt.show()
df_listings[df_listings['host_is_superhost'] == 't']['host_response_time'].value_counts().plot(kind='pie', )
plt.ylabel("")
plt.title("Response time of super hosts")
plt.show()
df_listings[df_listings['host_is_superhost'] == 't']['is_location_exact'].value_counts().plot(kind='pie', )
plt.show()
df_listings[df_listings['host_is_superhost'] == 'f']['is_location_exact'].value_counts().plot(kind='pie', )
plt.show()
df_listings[df_listings['host_is_superhost'] == 't']['review_scores_cleanliness'].value_counts()[:8].plot(kind='pie', )
plt.ylabel("")
plt.title("Cleanliness score of super hosts")
plt.show()
df_listings[df_listings['host_is_superhost'] == 'f']['review_scores_cleanliness'].value_counts()[:6].plot(kind='pie', )
plt.ylabel("")
plt.title("Cleanliness score of normal hosts")
plt.show()
#Number of super hosts
normal_hosts = df_listings.host_is_superhost.value_counts()['f']
super_hosts  = df_listings.host_is_superhost.value_counts()['t']
ratio_super_hosts = (super_hosts)/(super_hosts+normal_hosts)
percent_super_hosts = round(ratio_super_hosts*100,2)
print("Number of super hosts : {}".format(super_hosts))
print("Percentage of hosts who are super hosts in Seattle {}%".format(percent_super_hosts))
df_listings.host_is_superhost.value_counts().plot(kind='pie', )
plt.ylabel("")
plt.title("Is host a super host")
plt.show()<jupyter_output><empty_output><jupyter_text>Security deposit<jupyter_code>df_listings.security_deposit = df_listings.security_deposit.str.replace(r'$', '')
df_listings.security_deposit = df_listings.security_deposit.str.replace(r',', '').astype(float)
average_fee = round(df_listings.security_deposit.describe()['mean'],2)
maximum_fee = round(df_listings.security_deposit.describe()['max'],2)
mininmum_fee = round(df_listings.security_deposit.describe()['min'],2)
print("The average security deposit is ${}, mininmum deposit is ${} and maximum deposit is ${}".format(average_fee,mininmum_fee,maximum_fee))
df_listings.security_deposit.dropna().hist()
plt.title("Histogram of security deposit fee")
plt.xlabel("Price")
plt.show()

df_listings['accommodates'].value_counts().plot(kind='barh', )
plt.title("accommodates")

plt.show()<jupyter_output><empty_output><jupyter_text>In the following cells, I will analyse the features that have more impact on the pricing<jupyter_code>amenities_dict = {}
no_of_amenties = []
for row in df_listings.amenities:
    for i,item in enumerate(row.replace(r'{', '').replace(r'}','').split(',')):
            #temp.append(item)
            if item in amenities_dict:
                 amenities_dict[item] += 1
            else:
                amenities_dict[item] = 1
    no_of_amenties.append(i+1)
df_listings['no_of_amenties'] = no_of_amenties
amenities_list = sorted(amenities_dict, key=amenities_dict.get)
print("Top 15 amenties")
amenities_list[-15:]
t4 = {}
for amenity in amenities_list[-15:]:
    t4[amenity] = amenities_dict[amenity]
t4
plt.barh(range(len(t4)), list(t4.values()), align='center')
plt.yticks(range(len(t4)), list(t4.keys()), )
plt.title("Top 15 Amenities")
plt.show()
#Listing price
df_listings.price = df_listings.price.str.replace(r'$', '')
df_listings.price = df_listings.price.str.replace(r',', '').astype(float)

#Weekly listing price
df_listings.weekly_price = df_listings.weekly_price.str.replace(r'$', '')
df_listings.weekly_price = df_listings.weekly_price.str.replace(r',', '').astype(float)

#Monthly listing price
df_listings.monthly_price = df_listings.monthly_price.str.replace(r'$', '')
df_listings.monthly_price = df_listings.monthly_price.str.replace(r',', '').astype(float)
plt.plot( df_listings.no_of_amenties, df_listings.weekly_price,'*')
plt.ylim([0,df_listings.weekly_price.max()])
plt.title("Number of amenties vs weekly price")
plt.show()
ax = sns.boxplot(data=df_listings, x='property_type', y='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels(), rotation=90)
ax.set_xlabel("Property type")
ax.set_ylabel("Price")
ax.set_title("Property type vs Price")
plt.show()
ax = sns.boxplot(data=df_listings, x='room_type', y='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels())
ax.set_xlabel("Room type")
ax.set_ylabel("Price")
ax.set_title('Room type vs Price')
plt.show()
ax = sns.boxplot(data=df_listings[df_listings['bedrooms'] != 0], x='bedrooms', y='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels())
ax.set_title('Number of bedrooms vs Price')
ax.set_xlabel("Number of bedrooms")
ax.set_ylabel("Price")
plt.show()
ax = sns.boxplot(data=df_listings, x='review_scores_cleanliness', y='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels())
ax.set_title('Cleanliness vs Price')
ax.set_xlabel("Score")
ax.set_ylabel("Price")
plt.show()
ax = sns.boxplot(data=df_listings, x='review_scores_value', y='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels())
ax.set_title('Review rating vs Price') 
ax.set_xlabel("Score")
ax.set_ylabel("Price")
plt.show()
ax = sns.boxplot(data=df_listings, x ='neighbourhood_group_cleansed', y ='price', fliersize=0.1)
ax.set_xticklabels(ax.get_xticklabels(), rotation = 90)
plt.title("Price variation in different neighborhoods")
plt.xlabel("")
plt.show()
#more the number of bedrooms, higher the price
price_neighbourhood = df_listings.groupby(["neighbourhood_group_cleansed"])["price"].mean().sort_values(ascending=False)
price_neighbourhood.plot(kind='bar', )
plt.title("Average price in different neighborhood")
plt.xlabel("")
plt.ylabel("Average price")
plt.show()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup