I started in the last days of 2023 a personal project just for fun - I recreated the classical "Snake Game" (from old Nokia mobiles) in Python and I wanted to integrate it with some Machine Learning methods so the ML algorithm would learn how to play and excel at it.
My expectation was do to something like a genetic algorithm + neural network, where I instantiate many games in "parallel", let them play until all games are over, and finally pick the best ones and try to evolve them (applying some random mutations and crossing-over eventually).
In terms of implementation, it was an easy deal, but in terms of performance, it started to get terribly slow once I integrated Keras to the project - but probably it was my fault, maybe there's some other optimal way to do it.
Basically, I did the following:
- Created "N" instances of the snake game
- Created "N" instances of a simple Keras sequential model (this step strongly hurts performance)
- Created a for loop to recalculate the "next move" for all the "N" games, where the next move for the "Nth" game is based on its current state being provided as an input for the "Nth" neural network, which provides the optimal "next move" as its output;
- Updated the game screen containing all the snake games (just for visual feedback / fun, using pygame)
- Repeated these steps until all games are over, then evolved all the neural networks based on mutations the best one(s), and restarted the game loop (started a "new generation").
The fact is that using all these "N" Keras instances seems to be a terrible choice in terms of performance, although the topology of the models is very simple.
I was thinking about implementing an ANN model manually, doing all the matrix multiplication, so it would become much lighter, but before doing it I wanted to ask here in SO - is there any other Python resource available that would fit my need?
Altough fun, combining genetic algorithms with neural networks is just very slow. Genetic algorithms work nicely when the individuals of the population can be tested/validated within a ms. Whenever training a network is involved things get much slower than that.
Where in the steps do you backpropogate the loss to the neural networks? If done at every step, it will speed things up if you do it every x steps.
I am not sure about any resources for these situations. Keeping the models simple, running them on GPU and propogating loss at every x steps will improve things but I am not expecting miracles.