Demo Vids: https://www.youtube.com/watch?v=wT0GU41JocQ, https://m.youtube.com/watch?v=M9TGqmvseLw
Github Repository: https://github.com/mbird1258/Dino-Game
Premise
My plan for this project was to replicate the iconic dinosaur game and train a ML model to beat it. My main goal was to use the minimal amount of libraries, meaning no ML libraries, while keeping the code relatively short and simple.
How it works
More details in GitHub repo
The first ML algorithm is a feed forward neural network that takes in the player’s height above ground, next obstacle’s distance, screen scroll speed, the next obstacle's width, and the next obstacle's height. After two hidden layers with 5 nodes each The player decides to jump, drop, or do nothing based on the output node with the highest value.
After each generation of 3 trials, the top 20% of models experience no changes. 79% of the remaining models are completely redone, taking 2 models from the top 20%, taking the average weights, and making slight changes to them. The last 1% is a completely randomly created model. The idea is that the top 20% of models can only get better, the 79% will eventually find better solutions and enter the top 20%, and the last 1% will eventually lead to finding the best neural network given infinite time.
My aim for the second ML algorithm was for it to be a bit more complex and hopefully be more widely applicable. This neural network is also a feed forward neural network that takes in the player’s height above ground, next obstacle’s distance, screen scroll speed, and the next obstacle upper y value, with two hidden layers of 250 nodes each, and the player deciding to jump, drop, duck, or do nothing based on the output node with the highest value.
The backpropagation algorithm I used was to find the derivatives iteratively by creating a function for each function in the chain rule breakdown of the function and performing gradient descent with the values found. For my goal values/labels, I used Q learning, a common reinforcement learning algorithm.
Conclusion
This first neural network worked surprisingly well, probably because of the simplicity of the problem and lack of nodes/weights in the neural network, easily beating the game in a few iterations. However, due to the primitive nature of the backpropagation algorithm, I don’t think it would scale well into harder problems.
The second neural network seems to have worked pretty well, learning to jump over cacti and duck under birds relatively consistently, but it seems to struggle learning to deal with the increasing scroll speed of the game. If I were to do this again I would probably aim to use a more complex algorithm and find some way of automatically finding the rough estimate of the derivative at any given point instead of calculating the exact derivative with manually derived functions.