Software Framework for Agent-Based Games and Simulations

Distributed AI Framework for Agent-based Games and Simulations Sacerdotianu George line 1 (of Affiliation): dept. name of organization line 2: name of organization, acronyms acceptable line 3: City, Country george_sacerdotianu@yahoo.com I. Ilie Sorin line 1 (of Affiliation): dept. name of organization line 2: name of organization, acronyms acceptable line 3: City, Country line 4: e-mail address if desired INTRODUCTION Abstract 1.1 Motivation Generally, multi agent systems are used in the military, aircraft control or in business models. As we can see in paper [1], some form of artificial intelligence can be implemented in agents in order to make them play in AI games. So the development of agent based games has been possible The inherit distributed nature of agent should facilitate the creation of agent based multiplayer games. Another advantage is the high level of abstractization which offers the framework generality, generality which will be demonstrated with the variety of games simulated with the framework. Moreover the autonomy of the agents facilitates the creation of a modular framework. 1.2 Agent usage advantage Nowadays, agent-based technologies are promising to applications that often must operate across continents and intersystems. It is because they offer the high-level software abstractions needed to manage complex applications and Badica Costin line 1 (of Affiliation): dept. name of organization line 2: name of organization, acronyms acceptable line 3: City, Country line 4: e-mail address if desired because they were invented to cope with distribution and interoperability. Given the fact that the functions of sending and receiving messages, the communication protocols and the message structures were already implemented, we considered that this will make our work easier. The use of agents is mainly justified by the fact that they are a solution for managing complex systems. Another reason is because of their autonomy, meaning that they can operate without the direct intervention of humans or others. They also have some kind of control over their actions and internal state, without having only the role of a simple interface. So we implemented two types of Agents , one the Environment that validates the movements and has the game rules and the other one the Player that connects (join) to the Environment in order to enter the game and then establish a communication protocol so as to move on the map. We have simulated three types of games, Football, Battle , and a Labyrinth where we used a pathfinding algorithm. The interesting thing is the fact that after joining the environment, the player doesn’t need input, it takes decisions by itself acting only by the information received from the Environment and the experience that it accumulated during the game. Another advantage is the reactivity property of the agents, meaning that they perceive their environment and respond in a timely fashion to changes that occur in it. For developing this game framework we used JADE (Java Agent DEvelopment Framework), a software framework to write agent applications in compliance with the FIPA specifications for interoperable intelligent multi-agent systems, as programming language we use JAVA, and Eclipse as a IDE. I.3 Overview of the paper This paper is divided into four sections each of them containing specific information about the architecture and tries to make the user understand the it better. Section I is an introduction that tries to accommodate the reader with the structure of the paper, and with what we are talking about in each section, providing an overview. Section II contains the architecture of this project which includes the join protocol, the communication protocol, description of objects that are sent between the Environment and the Players(both of them being software agents [3]) in order to be recognized and analyzed at the destination. In section III will sustain the generality of the framework with different games simulated and provide the needed configuration details. IV - related work Fig. 1 Architecture In figure 1we described the main components of the system, Environment, Player and interface and the connections between them. As we can see the Environment and the Player are connected with a double sided line because they are exchanging messages and the interface is connected with a simple line meaning that only the Environment has control over the interface. V - Conclusions II. FRAMEWORK / SYSTEM DESIGN Our current proposal is based on two main classes of agents: 1. Environment agent the central entity of the game that moves the players around the map at their request 2. Bot Engine an abstract agent class that implements common interactions with the Environment. To create a Player Agent that can participate in a game this class must be extended and its abstract methods must be implemented. In the following we will present the interactions between these two entities. Fig 2. Distributed Architecture Figure 2 presents an image of the distributed architecture. The Environment and Players can all be on the same computer, or the Environment on one Computer and players on other, or they could be one on every computer. Those computers could be on a LAN, or they could be connected on the internet. Those games are running on a JADE platform, and on the same platform could be one or more games at the same time. The Framework configuration as shown in figure 3 is very general, that any AI or AI assisted game can be played/ simulated. All we have to do is to put the game rules in the Environment, and implement the corresponding player AI . The rules must be implemented inside a behaviour of the Environment. Fig 3. Framework configuration if (iHaveBall) Shoot() else if(FOV.myTeamIsDefending) Defend() else if(FOV.containsBall()) GoToBall() } Algorithm 2. Football player rules After the Environment and Player are created a protocol named Join Protocol is established between them. The protocol is described in figure 3. The Player sends a join message followed by it’s nickname to the Environment and the Environment accepts or rejects it according to the slots available. After we click the start button in the interface the Environment broadcast a start message to all the Players connected and the game is ready to begin. The Players AI is pure Java code. Each Player extends an engine where the basic functions for moving (left, right, forward, backward, hold_position), rotation, shoot are implemented and an abstract one containing the AI, that every Player will implement. The algorithm of every player is detailed below : handleFov(FOV) { determineNextMove (FOV) sendMove() } Algorithm 1. Player handling FOV contains the Player AI that is applied over the FOV received as parameter. So to create a new type of player al we have to do is to implemed the AI in this function. The example below is the AI of a football player . determineNextMove(FOV){ if(FOV.myTeamIsAttacking) Atack() else Fig 3 Join Protocol The interaction between this entities is done trough message exchange. The Protocol on witch the exchange is done is established when the game starts and remains active until it ends. The Environment send an update message to all players connected , containing the player FOV (Field Of View). This is done with the Environment tickerbehaviour. When player receives the message, it analyses it and send the movement back to Environment witch validates it. If the movement is valid the player position is updated on the map, otherwise same FOV is sent to the Player. Fig 4 Comunication Protocol Generally the FOV object is the field of view of each player , represented visually as a matrix around a player, and it contains the x and y coordinate of the center of the matrix where our player is, the objects, obstacles, special zones, players near him. Particularly the FOV object is different for every game played. We will describe the components of the FOV object for every type of game. Before presenting the components we must see how a FOV object is created, and how a map is loaded. When the environment starts, it loads the map. The map is represented as a XML file. Having all the information available in the XML we use a XML parser to move this information to a Map object. We choose for this a DOM Parser because is easier to use, and is well fitted for small XML files. A Map object looks just like the one in ​figure 5​. Also in this figure is presented the entire class diagram of class Map with all its dependencies. Fig 5 Map class diagram After reading and parsing the XML there are created objects after the main categories in the file. Every object has fields corresponding to the XML fields: object, Obstacle, Special Zone, Characteristics, Players. Then the objects are unified in a Map object. So that’s how the Map is represented in the environment. In the next section we will describe the types of FOV for every game showing some differences between them. Having the object Map, the Environment creates and sends parts of it, of different dimensions given by the size of FOV in the characteristics field to the players connected in order to give them the object they need for taking decisions. If the dimension is 0 it sends the entire FOV. Also the information in FOV is customized for every type of game, because for example we don’t need to send a ball position to a player that play’s a Battle or a Labyrinth games as well as we don’t send a vector of players in FOV to a Labyrinth player. So in order to send lower size objects wecustomized the FOV object for every game played. As we will see FOV’s can be all generated at the beginning in case the map has a limited FOV or it can be created every time for the games when all players receive the same thing, containing only the information that has been changed from the previous FOV. If the FOV object to be sent is the same for the all players, the environment makes a broadcast to them, and if not it sends particular objects to everyone connected. The environment doesn’t only send the “update” string followed by the FOV object, it also receives the Movement object from the Players connected (see figure ​4​). All type of players send the movement into a given structure and serializable, that the Environment handles it in the same way for all users. The movement class is described below. Figure 8The Interface class players​ - contains the PlayerEntities objects on the map Fig.7The Movement class The components of the class as shown in figure ​7​are: move type - ​represents the type of movement coded with 1 if there is a forward movement and with 0 if the player wants to hold its position rotation Angle - represents the angle of rotation for the player. This could be any given value between 0 and 360 degrees shoot - represents the player choice, it is true if the player wants to shoot and false if it doesn’t want to For example if the player wants to rotate 90 degrees, move forward and shoot it creates a new Movement (1,90,true) and send it to the environment. Having their actions coordinated by the following finite automatons, the player creates an object of type Movement and sends it to the environment. Receiving the object the Environment test if the player move type and rotation are valid, and only if there are it moves it on the Map meaning that the rotation is ok, but if the move type isn’t it executes only the rotation. This is what is happening to all fields in Movement class. If the moves aren’t valid the player receives the same FOV as previous, and if there are it receives the new FOV. shoots - contains the ShootEntities objects on the map. This Entities are either projectiles or a football ball map - the object that contains all the things there are in the map bufStr​ - the BufferStrategy used playersCon​ - the players that are conected to the Environment PlayerEntity represents the class that draws a specific shape onthe screen in our case a triangle representing the player. There is also the ShootEntity class that draws a circle on the screen. This class is used to represent the ball in a Football game and a projectile in a Battle. Both classes PlayerEntity and ShootEntity extend Entity class that contains functions for moving an entity, rotating it, moving it after a specific angle. Other components of the class are described in the figure 9. Game entities The actions that are validated are visible through an interface where entities are moved by the Environment commands. The interface class is presented in figure 8. This class extends Canvas that allows us to draw on the screen and implements Runnable Figure 9The Entity class PlayerEntity class contains information about the Interface where the player is moving, the name, life and its team. The ShootEntity has information about the Interface where the projectile or the ball is, and the damage it takes when it collides with a player. These entities also helped me with the collisions on the screen. We have created a method that is returning true or false according to the fact that a pixel of an entity is inside the bounds of the current entity. For moving those entities along the map we have used the (1) and .2) formulas, that translate the entities with a “d” distance rotated by α degrees. x’ = x + d*cos(α); (1) y’ = y + d*sin(α); (2) To rotate the entities we did a translation in the origin first, that we rotated using formulas (3.3) and (3.4), and then again a translation to the original position. x’ = x* cos(θ) – y* sin(θ) (3) y’ = x* sin(θ) – y* cos(θ) (4) In order to test these scenarios we have created some demonstrative bots with a basic AI that highlight/mark the good functionality of the game. So we created a labyrinth player, a dummy one, and one that act by the A star algorithm, for the football game we have created a striker , a defender, a midfielder and a goalkeeper player, and for the battle, two types of player, a dummy one , and a normal one. In the followingsI will describe the AI of players as finite automatons because they move to different stages as consequences of their actions. Their initial state will be in the moment they enter the game, and the final state will be dictated by the environment, meaning that they don’t know exactly this state except the Labyrinth player, because it knows that if it finds the key, it won the game. III. EVALUATION / SAMPLE GAMES - printr-oimplementare JADE - trebuienisteconcluzii ale evaluarii To prove the generality and flexibility of the framework we have simulated three types of games, a football game, a labyrinth game and a battle game, each of them having their rules defined in the Environment and the AI implemented in the Player. The Player’s AI is represented as finite automatons.In our case the states are the actions the player takes (move, shoot, defend, attack, pick object), and the transitions are the conditions the player needs to meet. A. The football game Rules in the environment - The player movement is restricted at the footbal field The ball can’t get out of the field If the ball enters the gate the score is updated and the player positions are reseted to the inital ones. If the score is equal than a draw is declared The player acts according to it’s position The game ends when the time is epleased Player AI In a football game there are 4 types of players : Strikers, Midfielders, Defenders and Golkeepers each of them acting according to the position in the field Strikers and ​Midfielders​are acting in the same way.So when it receives the FOV ifit doesn’t have the ball,itsearches the FOV to see which team has the ball. If its team is in possession, it goes to its attack position and if the enemy team is in possession it goes to its defense position. If neither its team nor the enemy team has the ball, it moves to it. When it reached the ball, it takes it and move in the initial state. If it has the ball it checks around him to see if there is any opponent in front of him If there is no enemy it check to see if it is in the attack position or not, and if it isn’t it moves forward to the attack position. If it is in attack position it shoots to the gate If there is an enemy player in front of him it passes the ball to a teammate. The ​Defenders act almost as the ​Strikers and Midfielders with a few changes. So when it receives the FOV. If it doesn’t have the ball, it searches the FOV to see which team has the ball. If its team is in possession, it goes to its attack position and if the enemy team is in possession it goes to its defense position. If neither its team nor the enemy team has the ball, it moves to it. When it reached the ball, it takes it and move in the initial state. If it has the ball it checks around him to see if there is any opponent in front of him. If there is no enemy it check to see if it is in the attack position or not, and if it isn’t it moves forward to the attack position. If it is in attack position it shoots to the gate If there is an enemy player in front of him it passes the ball to a teammate. And here comes the difference. When the player go to the defense position it tests if it reached it and if it does it started to move left right patrolling the area in order to intercept the ball. If it doesn’t reached the defense position it enters back into initial state where the process starts again The ​Goalkeeper has its actions based on the following finite automaton and its main role is to prevent the ball from entering the goalpost. As all others players in the game it has no final state. When the game begins the player save the map received from the environment in its memory (in this case the player receives the whole map) then it applies the ​A star algorithm on the given map. The algorithm returns a list of squares that, if the player follows them in the reverse order, finds the way to the key (exit). A square represents a structure that contains the x and y coordinate, and the cost of the move. Fig10 Goal keeper finite automaton A goal keeper is a lower complexity player because all it has to do is to move left right when the enemy team has the ball and when it catches the ball it throws it back in the field. Figure 12 Dummy Player finite Automaton C. The battle game B. The labyrinth game Rules in the environment ● ● The player can’t walk trough walls The game ends when the player reaches the exit position Player AI In figure 11 we described the states a labyrinth player passes through, and the transitions between this states. Rules in the environment ● ● ● ● ● ● ● The player movement is restricted at the map size The players can -shot with projectiles at their moving direction -pick and drop objects There are special zones that heal or damage the player Player have life Projectiles decrease life The obstacles restrict the movement of the player and stop the projectiles The game end when the life of a player reached 0 Player AI Battle player Finite Automaton is described below. It has an initial state , multiple intermediate states , no final state, because it’s role is to play until there is no enemy on the map, and the only way to know this is by the Environment which tell him if it lose or that there are no enemies on the map meaning that it wins. Figure 13 Figure 11 Labyrinth player finite automaton agents can stay in different locations (pc, pda, and cellphones). This distribution comes from the fact that the JADE arhitecture is distribute. To test this, we simulated three games to sustain that the framework is general, and observed how agents are acting in different scenarios. For every game we defined the set of rules in the Environment and implemented the AI of different players. In the future we would like: ● to build more complex agent system, the AI of the robots to be more powerful, the algorithms used to have a larger complexity, swarm alghoritms. ● to use this framework to simulate a disaster management and a resque scenario ● the interface, design of the robots to be more user friendly and attractive. ● the robots will be able to accumulate more experience during the game, and the map will be saved more efficiently in the agent memory. ● to implement a distributive map discovery, every agent to handle a part of the map and share the information with other agent by simple messages or trough a central point where everyone will put the information it know. ● the users will also have the option to modify some parameters of the player agents in order to increase or decrease the player’s life, damage, armor and other fields, making that player customizable and unique in the game. ● to do is a map editor that allows the user to make his own maps. ● to upgrade the system scalability by eliminating the central environment, and distributing it. ● if it will be possible to use real robots with suitable sensors to play some parts of game for the beginning, and finally the entire game .Those robots to have wireless transmiters and receptors in order to communicate with the Environment and with other players ● upgrading the functionality needed to organise a local contest or a national contest. Figure 13 Battleplayers Finite Automaton First of all the player checks in the information received from the Environment if there are objects on the map and next if there is any enemy on its FOV. . If there is an enemy, the player rotates the turret to the enemy and shots at him, next it does a random move. If there is an object in FOV, and the inventory is not full, it picks it up, otherwise it left it untouched. If there is no enemy on the map, then it starts to search for him so the player does a V - > N -> E -> S. movement, then returns to the decision state. If there is no enemy remaining the Player wins the battle and if the player’s life is 0 or below 0 it loses the game. All this rules are known by the environment which will stop the battle and declare a winner. Forsome interactivity in the battle game we also created a dummy bot that acts almost as a normal bot with a little difference. III. Related Work Searching over the internet we found some attempts to create agent-based games [1], and some attempts to improve their AI by improving the algorithms used or by tuning them using Q-Learning [1]. V. CONCLUSIONS AND FUTURE WORK References I developed a configurable distributed framework for AI agent-based games. Being distributed it means that the [1] Purvag G. Patel, Norman Carver, ShahramRahimi. Tuning Computer Gaming Agents using Q-Learning​(lucrarea trimisa de tine) [2] Andrea Omicini, Alessandro Ricci, Mirko Viroli. Artifacts in the A&A meta-model for multi-agent system [3] M. Wooldridge: An Introduction to MultiAgent Systems, John Wiley & Sons, 2002.