Automatic Play based on LR and SVM Algorithms

: With the progress of society, people's quality of life significantly improved, this gives us more free time to do things we enjoy, so our life has become colorful, countless kinds of entertainment for people to choose, video games is one of the important, with the level of technology to improve the video game is also getting richer, when the video game into the artificial intelligence, but also to improve the fun of the game, the challenging. In this project, we will use logistic regression algorithm and support vector machine to automate the machine to play the dinosaur parkour game in Google Chrome, through the training of logistic regression and SVM, so that the machine can operate and control the dinosaur to jump and crouch to achieve high score.


Background and Significance of the Study
Nowadays, artificial intelligence is a hot topic in our daily life, and in recent years, with the outbreak of the third artificial intelligence boom more and more scientists and researchers are involved in it, so that the development of artificial intelligence technology is advancing day by day, although the current artificial intelligence is still not able to show the general wisdom of human beings, but unknowingly our life has long been inseparable from the help of artificial intelligence. Besides that, people's material living standards improve, more needs come from entertainment, games and other aspects, data show that after entering the twenty-first century, people's demand for video games is growing. In order to meet the needs of the majority of players, and increase the playability of the game as well as fun, to improve the playability and fun of the game mainly two points, one is the innovation of the game, innovation is the core of our development, novel game content, different game operation, in the game to experience a different life, play different characters, is the main original video game to attract players, followed by the challenge of the game. The sense of achievement after the difficult completion of the task, "Elden Ring" is a very difficult RPG (Role-playing game) released in 2022, the main gameplay is to explore the open world to complete the task, challenge the powerful enemies, the game's enemies will also be strengthened according to the growth of the player, in the open world, the enemy will be based on the player's behavior to block or counterattack and other behaviors, there is no fixed template of the enemy according to the level of the player Adaptive to improve the difficulty of the game and improve the challenge of the game. This is also true for NPCs in shooter games including Call of Duty, where the novel play patterns of these more intelligent NPC characters have improved the playability and interest of the game [1]. Therefore, the application of AI technology in video games has become a trend in the development of video games, and major game makers have introduced AI technology to improve the competitiveness of their games, with Japanese game maker Nintendo cooperating with Japan's largest AI development company HEROZ, China's Tencent creating the "Kaiwu" AI game platform, and NetEase creating the first game in China. NetEase also created the first domestic game AI development studio "Fuxi Lab", the United States Blizzard Entertainment also cooperated with Google's DeepMind, and recently Ubisoft, a multinational game company, independently developed an AI tool for automatically generating video game NPC dialogues. First of all, for developers, it can assist game development, reduce game production costs, make game development convenient, produce higher quality games in a shorter time, reduce the workload of developers, and make the game realistic in the player's game experience, improve playability and fun.

Current Status of Domestic and International Research
When artificial intelligence technology was not introduced into the development of video games, the behavior of NPC characters interacting with players in the game was designed in advance and would not change according to the player's operation or environmental changes caused by the player. NPCs also can make corresponding responses based on the player's actions. The most prominent form is the fighting game in which the AI controls the NPCs as the player's enemies to fight [1].
The most classic and familiar application of artificial intelligence in games is the application in chess games, the best of which is AlphaGO, the AI that defeated the human Go champion. was introduced, which defeated AlphaGo, and AlphaZero outperformed other super-level AI in Go, chess, and chess. However, AlphaZero is not suitable as an opponent for human players because AlphaZero is too powerful for almost all human players. If a game is too easy or too difficult, then players will give up on the game. Initially, people used a kind of static level distinction between easy, standard, difficult, and similar levels in order to differentiate game levels, and with the development of artificial intelligence technology, people began to consider how to use artificial intelligence to replace the static division. As Google who developed AlphaGo zero, they later launched AlphaDDA which is an algorithm that can automatically adjust the difficulty of the game by analyzing the level of the player [2]. The recent development of Ghostwriter by Ubisoft has also turned game production upside down. This AI tool can automatically generate dialogue for video game NPCs, which cannot replace the video game developer but relieves the author of the onerous task. In fact, automatic game play is the basis for training AI game AI, 2019 OpenAI through reinforcement learning methods to train a set of team-based hide-and-seek game AI, they are red seekers and blue hiders, the game environment is scattered with some objects, both sides can grab and lock in place, used to stop each other's line, in many experiments appeared a surprising effects, not only showing superb dexterity to collar tools for dodging or blocking, the AI even learned to work in teams [3]. DeepMind, the company that made AlphaGo, partnered with Blizzard's StarCraft 2 to anonymously enter the game's ladder matchmaking mode with a new AI, AlphaStar, to play against real players and collect data.
At present, the commonly used game artificial intelligence technology mainly includes this kind, first of all, the finite state machine, and then there is the most commonly used technology when building a game decision-making system. It is the simplest, most effective, and most commonly used method in AI systems, such as Red Alert, Mount &Blade and other games that require players to give logic, movement and other action commands to the characters in the game. Randomness and probability: increase the uncertainty of the game and bring changes to the game. The most typical example is that in FIFA, when the player controls the character to shoot, it is not only related to the player's operation, but also adds a certain amount of randomness, avoiding Formulated and fixed games. Sensor system: Take Mount &Blade as an example. Each lord in the game will judge whether to attack or flee based on the strength of his own power and the strength of other lords. This function is realized through the sensor system. There is also the A* pathfinding algorithm: this is a pathfinding algorithm that guides the NPS to find the path. It has the characteristics of real-time, stability, and intelligence, such as AI characters in some shooting games Call of Duty, etc. [4] [5].

Research Objectives
This subject research will use the logic algorithm and the svm algorithm to realize Google's dinosaur parkour game. It is a simple parkour game that comes with the Google browser. When the network connection is disconnected, this will automatically pop up. You can also use the website to enter the game. This is a parkour game. The gameplay is very simple. When there is an obstacle (cactus, bird) in front of you, press the space to control the little dinosaur to jump and avoid obstacles. In the game, the speed of the little dinosaur will become faster and faster. After reaching a certain score, it will enter the night mode, and the difficulty will be greatly increased. Through the logistic regression algorithm, the machine will automatically operate according to the situation. For the convenience of research, ignore in the case of squatting, all operations are avoided by jumping.

Research Methods
This topic uses the logistic regression algorithm and support vector machine algorithm. The linear regression, its mathematical model is linear regression plus a special function, this special function is called the activation function. y = wx + b is the basis of linear regression. The algorithm of linear regression is to find a set of optimal w, b, which is the closest to the correct result obtained by the function. After obtaining the optimal solution w, b, we only need to If you want to divide the results into two categories, you can use 1 and -1 to represent it. At this time, you need an activation function to direct all the results to 1 and -1, so that you can successfully classify the results. [6]. The basic model definition of the SVM algorithm is that its basic model is a linear classifier with the largest interval defined in the feature space. The largest interval makes it different from the perceptron; the learning strategy of SVM is to maximize the interval, which can be formalized as the problem of solving a convex quadratic programming is also equivalent to the problem of minimizing the regularized hinge loss function. It shows many unique advantages in solving small sample, nonlinear and high-dimensional pattern recognition, and can be extended to other machine learning problems such as function fitting. SVM also includes kernel tricks which allows it to handle different situations, so it is important to choose the correct kernel function when using the SVM algorithm. linear kernel function is K(v1,v2)=<v1,v2> is the kernel function to handle linear problems, poly kernel function K(v1,v2)= (γ<v1,v2> +c)ⁿ is the polynomial kernel function, rbf kernel K (v1,v2) =exp(-γ||v1-v2||²) is the path like kernel function also called Gaussian kernel, sigmoid kernel function, and there is a precomputed kernel matrix, which is the kernel matrix given when the kernel function matrix is calculated in advance and will be selected for the algorithm. SVM was originally used to deal with binary classification problems, but it can also be used to implement multiple classification, there are two methods one is the one-to-many method (one-versus-rest) in the training of one category as the main category, the other grouped into one category, each category should be used as the main category, so as to do n times SVM, in n times to choose the best test results of one, the second category is the one-to-one method (one--versus-one) one SVM is designed for each two classes, and there is n (n-1)/2 SVMs, in which the classification of unknown data is the highest score is the result [7].

Experimental Process
In the Python environment imported sklearn, os, cv2, joblib, PIL library, etc. We can view the dinosaur game as a classification problem, that is, the image of the current game, the output is 0, 1 of a binary classification problem, where 0 means jump, 1 means not jump project is divided into three main steps, the first is data collection, manual game, through the PIL library in the game process continuously intercepts images to collect game data, and the intercepted images are stored in a folder named images. For the processing part of the images, firstly, we need to classify the picked images into folders named jump and none by manual judgment, jump is the case of jumping, the data size is 129 images, none is the case of no jumping, the data size is 253 images. Next, the images are read in through the cv2 library and the os library, and those that need to jump are marked as 0 and those that do not need to jump are marked as 1. Then the images are coded and transformed into a one-dimensional array. The final step of data processing is to split the data set into a test set and a training set, with the test set accounting for 20% of the total data. The second step is to train the model, through the logistic regression algorithm on the training, will be trained to play the model to save, the number of iterations is 500, the accuracy of 0.94.The last step is to use the trained model is the machine automatically play the game, here used the pynput library, created a space bar to control the small dinosaur jump, run the code, intercept the current game screen, through the training model prediction results, determine whether the need to jump, if necessary, then use the keyboard control through pynput, press the jump button. The results trained by logistic regression can make the little dinosaur run to about 700 points, because after 700 points the game will enter the black sky mode, and logistic regression is more suitable for training the second classification of the model after the black sky cannot automatically carry out the game, which is the limitation of logistic regression.
Therefore, this project uses the SVM algorithm and logistic regression to compare the experiment, and the difference with the logistic regression algorithm is that we divide the intercepted image data into four groups. The SVC function is used for classification in SVM. Since SVC is mainly used for binary classification, we add parameters to the function, firstly, the kernel function. Therefore, the linear kernel function was chosen for this project, and the one-to-one method was chosen for the multi classification method, and the final result reached 2550 points.

Experimental Summary
After the experiment, the logistic regression algorithm, to train the model, can exclude some complex situations in the game, the game has been simplified to achieve the training effect but as mentioned above, logistic regression has a great limitation, can only run to about 700 points, into the dark cannot continue the game. The SVM algorithm, on the other hand, is faster to train and more accurate than the logistic regression algorithm, and the dinosaur performance is much better to reach about 2550 points. Compared to the method of determining whether to jump by the pixels in front of the dinosaur, the use of machine learning not only has a greater improvement in accuracy, but also a faster image processing speed.
However, there are still deficiencies in the current experiment. In the data processing part, because the images used for training are manually classified, the trained small dinosaurs are very unstable. The speed of the game will continue to speed up as the game progresses, and the height of obstacles, the width is different, so even the same obstacle in the game will have different best timings for jumping due to different speeds, so the final score is slightly unstable.

Conclusion and Outlook
This article uses the logistic regression algorithm and the support vector machine algorithm to train the machine to automatically play the game of Google Dinosaur Parkour, analyzes the difference between the logistic regression and the support vector machine application, and compares the use of various kernel functions of SVM, and finally makes our small Dinosaurs can achieve higher scores.
The use of artificial intelligence technology in games will undoubtedly be a groundbreaking technology for video games. The gameplay and fun will be richer, allowing players to have a richer experience. With the development of artificial intelligence technology, more and more the novel game experience will also be presented in front of the players, and the game can also become the touchstone of AI for AI. The birth of most groundbreaking AI is presented by the game in our public view. Powerful AI Defeating human players in one game after another is declaring that AI capabilities are getting stronger and stronger, but the role of AI is not just to defeat humans. In the future, AI that plays automatically like this topic will play against humans at the same time, it is also necessary to be able to judge the rationality of the game, adjust the difficulty of the game, and give players a better experience. For game development companies, the first thing is to look for market opportunities, develop more AI application areas and even create game types that could not exist before, allowing games to incorporate more AI technologies, even like "Star Trek" in science fiction movies Create a personalized game according to the needs of the players like the holographic deck in the game.