<jupyter_start><jupyter_text>Groupby Operations and Multi-level Index<jupyter_code>import numpy as np
import pandas as pd
df = pd.read_csv('Data/mpg.csv')
df.head()
df['cylinders'].value_counts()
df['model_year'].unique()
df['model_year'].value_counts()<jupyter_output><empty_output><jupyter_text>groupby() method Groupby methods are lazy. So it always wait for aggregrate functions to perform further.Common Options:    mean(): Compute mean of groups    sum(): Compute sum of group values    size(): Compute group sizes    count(): Compute count of group    std(): Standard deviation of groups    var(): Compute variance of groups    sem(): Standard error of the mean of groups    describe(): Generates descriptive statistics    first(): Compute first of group values    last(): Compute last of group values    nth() : Take nth value, or a subset if n is a list    min(): Compute min of group values    max(): Compute max of group values<jupyter_code># Creates a groupby object waiting for an aggregate method
df.groupby('model_year')
# now 'model_year' became index
df.groupby('model_year').mean()
df.groupby('model_year').mean()['mpg']<jupyter_output><empty_output><jupyter_text>multi level groupby<jupyter_code>df.groupby(['model_year', 'cylinders']).mean().columns
df.groupby(['model_year', 'cylinders']).mean().index
df.groupby(['model_year', 'cylinders']).mean()
df.groupby('model_year').describe().transpose()<jupyter_output><empty_output><jupyter_text>---------  Groupby Multiple Columns<jupyter_code>year_cyl = df.groupby(['model_year', 'cylinders']).mean()
year_cyl.head()
year_cyl.index.names
# we can see there are 2 levels
year_cyl.index.levels
year_cyl.head()
# we want to know only 1970 data
# now we can see there is no index '70' as we know this is only for year 1970
year_cyl.loc[70]
year_cyl.loc[[70, 82]]
# interested only in 1970, 8 cylinders cars
year_cyl.loc[(70, 8)]<jupyter_output><empty_output><jupyter_text>------------  MultiIndex  The MultiIndex Object<jupyter_code>year_cyl.head()<jupyter_output><empty_output><jupyter_text>Cross Section- it allow you to go deep insides of specific level<jupyter_code># key tied to one value only and tellng to look at which level, in this case 'model_year'
year_cyl.xs(key=70, level='model_year')
year_cyl.head()
# want to get 4 cylinders for every year
# this tell to look at cylinders level and get values with 4

four_cylinders = year_cyl.xs(key=4, level='cylinders')
four_cylinders<jupyter_output><empty_output><jupyter_text>-------<jupyter_code>df.head()
# we only want for cylinders 6 and 8
# it is a lot easier to do it in dataframe level, rather than trying to everthing with cross section
df[df['cylinders'].isin([6,8])]
six_eight_cyl = df[df['cylinders'].isin([6,8])].groupby(['model_year', 'cylinders']).mean()
six_eight_cyl<jupyter_output><empty_output><jupyter_text>-------  Swap Levels<jupyter_code>year_cyl.head()
# we want to swap the index
year_cyl.swaplevel()<jupyter_output><empty_output><jupyter_text>-------  Sorting MultiIndex- it only make sense to sort by the most outer layer, otherwise the relation will break up.<jupyter_code>year_cyl.head()
# we want to sort mode_year by decedning order
year_cyl.sort_index(level='model_year', ascending=False)
# it only make sense to sort by the most outer layer, otherwise the relation will break up.
# we can see that the relation is broken.
year_cyl.sort_index(level='cylinders', ascending=False)<jupyter_output><empty_output><jupyter_text>-------------<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>Advanced: agg() methodthe funciton name must matched up with the following listCommon Options:    mean(): Compute mean of groups    sum(): Compute sum of group values    size(): Compute group sizes    count(): Compute count of group    std(): Standard deviation of groups    var(): Compute variance of groups    sem(): Standard error of the mean of groups    describe(): Generates descriptive statistics    first(): Compute first of group values    last(): Compute last of group values    nth() : Take nth value, or a subset if n is a list    min(): Compute min of group values    max(): Compute max of group values  agg() on a DataFrame<jupyter_code>df.agg(['mean', 'std', 'count'])
df.agg(['mean', 'std', 'count'])['mpg']<jupyter_output><empty_output><jupyter_text>Specify aggregate methods per column | more elegant way- for each column, which aggregate methods should be applied => passed them in dictionary<jupyter_code>df.agg({'mpg': ['mean', 'std'], 'acceleration':['count', 'sum']})<jupyter_output><empty_output><jupyter_text>agg() with groupby()<jupyter_code>df.head()
df.groupby('model_year').agg({'mpg': ['mean', 'std'], 'acceleration':['count', 'sum']})<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup