How to use Python to make Minecraft Java Deep Learning bots

Question

Okay I have an idea that make be theoretical. I was looking at this paper, and I saw that they used something called Malmo to interact with Minecraft. I took a quick look at this and I saw this was a offline thing. I want to make a program to directly interact with Minecraft Java. I thought about using some library to read pixel data and using the keyboard library for input. This would work of course but I want to also fetch things like player position, inventory, and other things. I would ideally want this to work client side. So my final question is how would I be able to test neural network Minecraft bots with python on pure Minecraft Java? Would I need to make a Forge or Fabric client?

Answer 1

I don't think this is possible in Python but maybe in JavaScript/ Node.js with a library called Mineflayer. you can't directly send keystrokes and mouse input from Python because Minecraft will receive it as a digital input and simply won't respond properly. here is the Mineflayer Github where you can find how to use it and various addons to MineFlayer such as MineFlayer-pathfinder or MineFlayer-navigate.