<jupyter_start><jupyter_text>Exploring actions of the DQNhere we will see the actions of a debug DQN run on the last of 1000 episodes.Parameters:```    num_dcs = 10    num_customers = 2    num_commodities = 4    orders_per_day = 2    dcs_per_customer = 3    demand_mean = 100    demand_var = 20    num_steps = 50    num_episodes = 1000```**Commit hash:** 74ab129039874c95bbeee44585d11c75ed881e13<jupyter_code>import numpy as np
import pandas as pd 
import matplotlib.pyplot as plt
import seaborn as sns
%config InlineBackend.figure_format = "retina"
from pathlib import Path
import os
project_dir = Path(globals()['_dh'][0]+"/..").resolve()
os.chdir(project_dir)
print("Running notebook from: " + os.path.abspath(""))

from IPython.core.interactiveshell import InteractiveShell
# InteractiveShell.ast_node_interactivity = "last_expr" # Default jupyter behavior
InteractiveShell.ast_node_interactivity = "all" # All expressions are shown.<jupyter_output>Running notebook from: /Users/aleph/Documents/jota/tesis/ts_mcfrl<jupyter_text>1 - Actions on episode 1<jupyter_code>def add_is_m(df):
    df['unit_cost']=(df.customer_cost/df.customer_units).fillna(0)
    df['is_m'] = df.unit_cost==1000
    return df
details_1 = pd.read_csv("python/data/results/two_customers_dqn_debug/ep_1/movement_detail_report.csv")
details_1 = add_is_m(details_1)
to_customer_1 = details_1.query('source_kind=="DC"').query('destination_kind=="C"')<jupyter_output><empty_output><jupyter_text>On episode 1, customer movements were distributed between many DCs<jupyter_code>to_customer_1.groupby(["source_name","destination_name"]).size().reset_index().sort_values(["destination_name",'source_name'])<jupyter_output><empty_output><jupyter_text>2 - Checking actions on episode 999<jupyter_code>details_999 = pd.read_csv("python/data/results/two_customers_dqn_debug/ep_999/movement_detail_report.csv")
details_999 = add_is_m(details_999)
to_customer_999 = details_999.query('source_kind=="DC"').query('destination_kind=="C"')<jupyter_output><empty_output><jupyter_text>By the end of episode 999, the DQN was serving from a single DC 100% of the time<jupyter_code>to_customer_999.groupby(["source_name","destination_name"]).size().reset_index()<jupyter_output><empty_output><jupyter_text>3 - Compare the number of big MsThe number of Big Ms should have dropped from ep 1 to 999 Total big M numbers<jupyter_code>display("To customer M summary ep 1")
to_customer_1.groupby(['is_m']).size().reset_index().sort_values(['is_m'])
display("To customer M summary ep 999")
to_customer_999.groupby(['is_m']).size().reset_index().sort_values(['is_m'])<jupyter_output><empty_output><jupyter_text>Per Customer Big M numbers<jupyter_code>display("To customer M summary ep 1")
to_customer_1.groupby(['is_m','source_name','destination_name']).size().reset_index().sort_values(['destination_name','is_m'])
display("To customer M summary ep 999")
to_customer_999.groupby(['is_m','source_name','destination_name']).size().reset_index().sort_values(['destination_name','is_m'])<jupyter_output><empty_output><jupyter_text>Total costTotal cost actually went up in the last EP compared to the first.<jupyter_code>to_customer_1.customer_cost.sum()
to_customer_999.customer_cost.sum()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup