Beating a Flash game with an AI
After reading this article on Programming a Bot to Play the "Sushi Go Round" Flash Game, I wanted to follow the example and make my own Flash game bot. I chose the Flash game Cardinal because its mechanics are simple: you are prompted with a direction to move (by pressing one of the directional keys), one after the other, until you fail because of the response time getting shorter. In fact you are a cube surrounded by four wall presses, and only one them is absent each time: your exit in other words. If you fail to escape in time, the walls crush you. My mere human skills escaped death 86 times in a row, whereas my bot did, well… a lot better.
First step - Making the bot play correctly
Here is the process the bot needs to follow to play the game correctly:
- Locate the game area on screen
- Click the Play button so the game starts
- Look for the missing wall in the current setup
- Press the corresponding direction and wait for the next round
To this end, we can take screenshots, look for given images on the screen, move the cursor and input keys. These fundamentals actions are made available through the pyautogui module.
Locating the game area on the screen is easy as long as there is a part of the game startup screen that is constant. We can just locate that sub-image on the whole screen and deduce from that the relative coordinates of the game screen and all the elements whose positions are fixed. We can then safely move the cursor to the Play button position and click.
As the walls' positions are fixed, looking for the missing wall consist in taking a screenshot and checking the color value of one pixel from each wall's known position. If one of these pixels doesn't match the red color of a wall, then the corresponding way is clear. We can then make a move and input the appropriate direction!
After moving, we must wait until the animation of the escaping square is over and a new walls setup is displayed. The red square is always put back to the center of the screen before every move. So we can watch for that moment to come before continuing, by repeatedly taking screenshots and checking the presence of the square. However, pyautogui is slow at taking screenshots (>100ms). I suspect it is because it saves images in a temporary file. So if we don't want to be late for the next round, we must time our screenshot so it happens short after the square is back to the center. By timing the screenshot well, we can be just in time to make the next move. The whole performance of the bot lies on this precise timing.
Unfortunately, a screenshot duration varies, so we can only assume it has an upper bound. Practically, this means that we must time it so the result is available some time after the next round, but still keep some room for error. In the worst case, we get a screenshot just before the next round and we have to retake a screenshot, which takes a lot of time, usually resulting in our time being up. The first version of the bot would not go higher than a 80 score.
Second step - Gotta go fast
In order to improve the bot's performance, we must make faster screenshots. I chose a simple way to achieve that. Instead of using the built-in screenshot feature of pyautogui, I would create a small C API for taking screenshot, using the Xlib library and use this API in Python with the ctypes module. The C code just need to be compiled as a shared library to be loaded by Python. The C code does not save pictures in temporary files but store them in memory as XImage structures. The only downside is that you now have to handle memory management in Python too. But now we can also take screenshots of the game region in about 12ms, which is inferior to a frame duration! That means enough speed to actually watch repeatedly for the next round to come, as we wanted to do in the first place.
While the reaction time window shrinks as the game progresses, it seems it will not fall under the duration of a frame. This means in practice that this second version of the bot is able to play the game indefinetely. I left it run for two hours one day, and it was still playing past the 10,000 score mark. I had to kill it, sadly, as its power was becoming a threat for mankind. It also took away all the fun from this game in the process…
Now I need to find a new game —sigh—.
