How to Solve reinforcement learning Grid world examples using value iteration?

Question

I find either theories or python example which is not satisfactory as a beginner. I just need to understand a simple example for understanding the step by step iterations. Could anyone please show me the 1st and 2nd iterations for the Image that I have uploaded for value iteration? Grid world problem

Answer 1

I recommend this PDF: http://www.cis.upenn.edu/~cis519/fall2015/lectures/14_ReinforcementLearning.pdf, which is very clear about the grid world problem. And there are codes on github:

https://github.com/kevlar1818/grid-world-rl

https://github.com/dennybritz/reinforcement-learning/blob/master/DP/Policy%20Evaluation%20Solution.ipynb

Hope those help.

Answer 2

Besides @holibut's links, which are very useful, I also recommend: https://github.com/JaeDukSeo/reinforcement-learning-an-introduction/blob/master/chapter03/GridWorld.py

The author implemented the full grid generation presented in the book. IMHO it is a simpler implementation, and one can debug the grid generation loops to clearly see step by step how the values are computed, and how the bellman equation is applied.