I have a problem that I would like to represent as a large scale Markov Decision Process. I expect to have a model which can predict state transitions and rewards, but I want to infer the value function and policy function. Approximately inference of value functions and "Q" functions and via a linear model or neural network will do nicely.
A key requirement: I am going to grow up my state-space so that it can't be represented in memory. I will need to represent states by feature values that characterize them.
Are there libraries that can help me infer the value and policy function in this large scale setting? Most of the Markov decision process libraries I have encountered seem oriented towards very small scale problems and exact inference.
If there are no turnkey solutions, does anyone have suggestions on how to architect such a system?
You could use TF-Agents which is supported by TensorFlow devs and has pre-implemented DQN agents for Q value function approximation. The main thing you would have to do is to write down the code for your specific envrionment which, together with the agent, constitutes your MDP.