<jupyter_start><jupyter_text>ASSISTments2015 Data Analysis  Data Description Column Description| Field    | Annotation                                          || -------- | --------------------------------------------------- || user id | Id of the student || log id | Unique ID of the logged actions | | sequence id | Id of the problem set || correct | Correct on the fisrt attempt or Incorrect on the first attempt, or asked for help |<jupyter_code>import numpy as np
import pandas as pd

import plotly.express as px
from plotly.subplots import make_subplots
import plotly.graph_objs as go
path = "2015_100_skill_builders_main_problems.csv"
data = pd.read_csv(path, encoding = "ISO-8859-15",low_memory=False)<jupyter_output><empty_output><jupyter_text>Record Examples<jupyter_code>pd.set_option('display.max_columns', 500)
data.head()<jupyter_output><empty_output><jupyter_text>---- General features<jupyter_code>data.describe()
print("The number of records: "+ str(len(data['log_id'].unique())))
print('Part of missing values for every column')
print(data.isnull().sum() / len(data))
len(data.user_id.unique())
len(data.sequence_id.unique())<jupyter_output><empty_output><jupyter_text>Sort by user id<jupyter_code>ds = data['user_id'].value_counts().reset_index()

ds.columns = [
    'user_id',
    'count'
]

ds['user_id'] = ds['user_id'].astype(str) + '-'
ds = ds.sort_values(['count']).tail(40)

fig = px.bar(
    ds,
    x = 'count',
    y = 'user_id',
    orientation='h',
    title='Top 40 students by number of actions'
)

fig.show("svg")
ds = data['user_id'].value_counts().reset_index()

ds.columns = [
    'user_id',
    'count'
]

ds = ds.sort_values('user_id')

fig = px.histogram(
    ds,
    x = 'user_id',
    y = 'count',
    title = 'User action distribution'
)

fig.show("svg")<jupyter_output><empty_output><jupyter_text>Correct answers<jupyter_code>ds = data['correct'].value_counts().reset_index()

ds.columns = [
    'correct',
    'percent'
]

ds['percent'] /= len(data)
ds = ds.sort_values(['percent'])

fig = px.pie(
    ds,
    names = ['0', '1/10','1/5', '1/4','1/3', '1/2','2/3', '3/4','4/5','9/10', '1'],
    values = 'percent',
    title = 'Percent of correct answers'    
)

fig.show("svg")<jupyter_output><empty_output><jupyter_text>Minor note: we also have Essay questions that teachers can grade. If this value is say .25 that means the teacher gave it a 1 our of 4.   Sort by sequence id<jupyter_code>ds = data['sequence_id'].value_counts().reset_index() 

ds.columns = [
    'sequence_id',
    'count'
]

ds['sequence_id'] = ds['sequence_id'].astype(str) + '-'
ds = ds.sort_values(['count']).tail(40)

fig = px.bar(
    ds,
    x = 'count',
    y = 'sequence_id',
    orientation = 'h',
    title = 'Top 40 useful sequence_ids'
)

fig.show("svg")
ds = data.groupby('sequence_id')['correct'].mean() 
ds = ds.reset_index()

ds['sequence_id'] = ds['sequence_id'].astype(str) + '-'
ds1 = ds.sort_values(['correct']).tail(20)

fig1 = px.bar(
    ds1,
    x = 'correct',
    y = 'sequence_id',
    orientation = 'h',
    title = 'Average number correct answers of problem sets (top 20)'
)

fig1.show("svg")

ds2 = ds.sort_values(['correct']).head(20)

fig2 = px.bar(
    ds2,
    x = 'correct',
    y = 'sequence_id',
    orientation = 'h',
    title = 'Average number correct answers of problem sets (bottom 20)'
)
fig2.show("svg")<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup