<jupyter_start><jupyter_text>Reading Tabular Data into DataFramesWith Pandas we can easily read a file to a DataFrame. The method for reading a CSV file is `read_csv()` and for a JSON file is `read_json()`. ``read_csv()```california_housing_test.csv` is California housing data from the 1990 US Census.The following read_csv code should run on both local Jupyter notebooks as well as Colab. The data file is available to Colab notebook by default in the sample_data folder.<jupyter_code>import pandas as pd
pd.read_csv("sample_data/california_housing_test.csv").head()<jupyter_output><empty_output><jupyter_text>Let's read this file again but this time save it as a DataFrame by providing an appropriate name:<jupyter_code>ca_housing = pd.read_csv("sample_data/california_housing_test.csv")<jupyter_output><empty_output><jupyter_text>`head()`Now we can look at the first few rows by `head()` function:<jupyter_code>ca_housing.head()<jupyter_output><empty_output><jupyter_text>`describe()``describe()` function is a fast way to get an overview of the basic statistics of the DataFrame:<jupyter_code>ca_housing.describe()<jupyter_output><empty_output><jupyter_text>`info()`Another way to get a high level summary is `info()`. This method prints information about a DataFrame including the index dtype and column dtypes, non-null values and memory usage.<jupyter_code>ca_housing.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 3000 entries, 0 to 2999
Data columns (total 9 columns):
 #   Column              Non-Null Count  Dtype  
---  ------              --------------  -----  
 0   longitude           3000 non-null   float64
 1   latitude            3000 non-null   float64
 2   housing_median_age  3000 non-null   float64
 3   total_rooms         3000 non-null   float64
 4   total_bedrooms      3000 non-null   float64
 5   population          3000 non-null   float64
 6   households          3000 non-null   float64
 7   median_income       3000 non-null   float64
 8   median_house_value  3000 non-null   float64
dtypes: float64(9)
memory usage: 211.1 KB<jupyter_text>`.columns`Lists all the columns of the DataFrame:<jupyter_code>ca_housing.columns<jupyter_output><empty_output><jupyter_text>`.shape`This DataFrame attribute provides the number of rows and columns of a DataFrame:<jupyter_code>ca_housing.shape<jupyter_output><empty_output><jupyter_text>Anscombe's quartetAnscombe's quartet comprises four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. Each dataset consists of eleven (x,y) points. They were constructed in 1973 by the statistician Francis Anscombe to demonstrate both **the importance of graphing data before analyzing it** and **the effect of outliers on statistical properties**. He described the article as being intended to counter the impression among statisticians that "numerical calculations are exact, but graphs are rough".  `read_json()``anscombe.json` contains a copy of [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet). The file contains 11 data points for 4 series. These series can be described by a similar statistics but in fact are very different from each other when plotted:<jupyter_code>anscombe = pd.read_json("sample_data/anscombe.json")
anscombe.head(20)<jupyter_output><empty_output><jupyter_text>Visualizing with SeabornSeaborn is a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.<jupyter_code>%matplotlib inline
import seaborn as sns

# Show the results of a linear regression within each dataset
sns.lmplot(x="X", y="Y", col="Series", hue="Series", data=anscombe,
           col_wrap=2, ci=None, palette="muted", height=4)<jupyter_output><empty_output><jupyter_text>Filter rows<jupyter_code>anscombe[anscombe.Series == 'I']
# Splitting the DataFrame into 4 based on their Series values
anscombe_I = anscombe[anscombe.Series == 'I']
anscombe_II = anscombe[anscombe.Series == 'II']
anscombe_III = anscombe[anscombe.Series == 'III']
anscombe_IV = anscombe[anscombe.Series == 'IV']<jupyter_output><empty_output><jupyter_text>Using `describe()` method to get the statistics:<jupyter_code># Applying the descibe method on all the 4 DataFrames
desc_I = anscombe_I.describe()
desc_II = anscombe_II.describe()
desc_III = anscombe_III.describe()
desc_IV = anscombe_IV.describe()

# Renaming the column names
desc_I.columns = ['X_I', 'Y_I']
desc_II.columns = ['X_II', 'Y_II']
desc_III.columns = ['X_III', 'Y_III']
desc_IV.columns = ['X_IV', 'Y_IV']

# Joining the summary DataFrames
summary = desc_I.join(desc_II)
summary = summary.join(desc_III)
summary = summary.join(desc_IV)

# Reordering the columns
summary[['X_I', 'X_II', 'X_III', 'X_IV', 'Y_I', 'Y_II', 'Y_III', 'Y_IV']]<jupyter_output><empty_output><jupyter_text>Loading a sample DataFrame from a packageSome packages contain sample data that can be loaded by a method specific to that library. For instance seaborn sample dataset can be loaded to a DataFrame by `sns.load_dataset()`:<jupyter_code># Load the example dataset for Anscombe's quartet
df = sns.load_dataset("anscombe")
df.head()<jupyter_output><empty_output><jupyter_text>You can access a list of all seaborn sample datasets by `sns.get_dataset_names()`.  Reading from ExcelFor this example we are using an Excel file that is located in the Github folder sample_date: `datasets.xlsx`If you are using Colab you will need to upload this file manually to your sample_date folder.<jupyter_code>import pandas as pd
 
iris = pd.read_excel('sample_data/datasets.xlsx', sheet_name='iris')
iris.head()
mtcars = pd.read_excel('sample_data/datasets.xlsx', sheet_name='mtcars')
mtcars.head()<jupyter_output><empty_output><jupyter_text>For more information I refer you to the help page `pd.read_excel?`.  Writing to CSVLet's write the `mtcars` DataFrame that we just read form our Excel file into a separate CSV file using the `to_csv()` function.In order to do this we use this method on the DataFrame we would like to export. If you type `mtcars.to_` and the `` you can see all the other options available to you:<jupyter_code>mtcars.to_csv('sample_data/mtcars.csv')<jupyter_output><empty_output><jupyter_text>Writing to ExcelLet's create a new Excel file using `anscombe` and `ca_housing` DataFrames that we imported above.<jupyter_code># First create a writer object with a filename
writer = pd.ExcelWriter('sample_data/Pandas-Example.xlsx')

# Add DataFrames that need to be exported to the sheet defined above
anscombe.to_excel(writer,'anscombe',index=False)
ca_housing.to_excel(writer,'ca_housing',index=False)

# Write it to disk
writer.save()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup