<jupyter_start><jupyter_text>5.2 Experiments  **Problem4. CartPole**  **Small Batch -Reward to Go**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/sb_(rtg)_dna.png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Small Batch -Normalizing Advantageous Function**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/sb_rtg_(dna).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Small Batch -Reward to Go + Normalizing Advantageous Function**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/sb_(rtg)_(dna).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Large Batch -Reward to Go**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/lb_(rtg)_dna.png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Large Batch -Normalizing Advantageous Function**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/lb_rtg_(dna).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Large Batch -Reward to Go + Normalizing Advantageous Function**<jupyter_code>image = mpimg.imread('./plot/CartPole-v0/lb_(rtg)_(dna).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>***Brief Analysis****1. Reward to Go allows agent to hit maximum rewards. Without reward-to-go, agent seems to concentrate on earlier behvaiors, rather decreases the continuity.**2. Normalizing Advantageous Function stabilizes the training with reducing the high variance in Advantageous Function. Even though the agent reaches the maximum score, raw advantageous function disturbs agent from the convergence**3. Since large batch size reduces the high variance, it helps agent's policy to stabilize. However, it doesn't seem to aid the speed of the convergence in this test case*  **Problem5. InvertedPendelum**  **Finding Learning Rates - Small Batch**<jupyter_code>image = mpimg.imread('./plot/InvertedPendulum-v2/sb_(lr).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Finding Learning Rates - Large Batch**<jupyter_code>image = mpimg.imread('./plot/InvertedPendulum-v2/lb_(lr).png')
plt.imshow(image)
plt.show()
image = mpimg.imread('./plot/InvertedPendulum-v2/lb_(batch).png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>***Brief Analysis****lr = 0.03, batch_size = 500* *cmd: python train_pg_f18.py InvertedPendulum-v2 -ep 1000 --discount 0.9 -n 100 -e 3-l 2 -s 64 -b 1000 -lr 0.03 -rtg*  **Problem7. LunarLander**<jupyter_code>image = mpimg.imread('./plot/LunarLanderContinuous-v2/lunar_lander.png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output><jupyter_text>**Problem8.Half Cheetah-v2**<jupyter_code>image = mpimg.imread('./plot/HalfCheetah-v2/batch10000.png')
plt.imshow(image)
plt.show()
image = mpimg.imread('./plot/HalfCheetah-v2/batch30000.png')
plt.imshow(image)
plt.show()
image = mpimg.imread('./plot/HalfCheetah-v2/batch50000.png')
plt.imshow(image)
plt.show()
image = mpimg.imread('./plot/HalfCheetah-v2/batch50000_baseline.png')
plt.imshow(image)
plt.show()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup