Hybrid Gaming

In recent years, there has been much talk that machine learning will fundamentally change the gaming industry, as this technology has already become a breakthrough in many other digital applications. But do not forget that the game is much more complex than a driving simulator, drone control program or facial recognition algorithms in the image.

So far, the gaming industry still uses traditional AI methods, such as the finite machine method, behavioral trees and – more recently – Utility-based AI (utility-based systems). Such AI is also called design-based (artificial design intelligence) or expert systems.

But it is becoming increasingly clear – and to players in the first place – that these systems are less and less suitable for creating truly advanced opponents who could mimic player behavior. This is especially true for creative solutions.

This can be explained by the fact that the developers of artificial intelligence is not able to take into account all possible tactics and behavioral strategies and successfully implement them in traditional AI systems. For players, this often turns out to be boring and predictable to play with an opponent whose line of behavior is easy to remember.

As things stand now.

The British Artificial Intelligence company DeepMind recently showed how AI can learn to play games on their own, learn their rules and find ways to pass or win a game – so far, however, only on the example of simple games such as early Atari games – such as chess and Japanese logic game go.

The results obtained for them show that artificial intelligence is able to form for itself an adequate assessment of what is happening on the field. If we talk about adapting the AI to different styles of play of the opponent, the results are not yet so impressive.

The Gaming AI

As for the application of machine learning in the gaming industry, there are a number of limitations due to which it is not always possible to use this type of architecture. These include system requirements, in particular relating to the CPU, which determine the ability of the computer to process a complex game structure and its suitability for gaming prioritization and gameplay.

Thus, it turns out that in many games to implement a complex system of artificial intelligence is not possible to organize the necessary hardware, and even more so the cluster of servers that exist. Sometimes and should work simultaneously several AI – and not only on computers, but also on mobile devices, and other less productive platforms.

All of this imposes limitations on the size and complexity of the machine learning architecture, because all calculations must also be performed with a frame length of about 1 or 2 milliseconds. Of course, you can use different technologies of optimization and load balancing between frames, but still can not get rid of these limitations at all.

Serious problems for AI can cause difficulty in the game. After all, in games such as StarCraft II, the mechanics of the game is much more complicated than in Atari games. Therefore, you should not expect that at a given frame rate and with known system requirements, machine learning will certainly cope with the study of the entire state of the game and will be able to interact with it.

As a player is often guided by intuition in the early stages of the game, so the AI must learn how to process the initial state of the game to facilitate its further passage.

This is a particularly relevant aspect in the case of games. Often, generally accepted methods of solving machine learning problems do not apply to game artificial intelligence. More often than not, the role of AI is to make the passing of the game more exciting.

Practical problems of machine learning

These problems came up in the development of artificial intelligence based on machine learning for Unleash, where AI must behave like ordinary players – that is, be as flexible and resourceful.

Like Starcraft II, Unleashed is much more complex than chess and go for Atari. The gameplay is intuitive and easy to learn, but in order to really succeed in it, you need certain meth control skills.

To do this, he must bluff and calculate other people’s moves in advance, as well as to manage the psychological meth – it is she makes of poker more than just a statistical game.

It quickly became clear that there are many global problems that Unleashed has to solve to which it is difficult to adapt machine learning.

One of them is building an efficient labyrinth. As in many games, whose purpose is to protect the tower, here players need to build a maze around it, through which monsters will break through. They, in turn, must be eliminated with weapons placed on the maze.

The need for parallel approaches

While machine-based AI training has been slow and not particularly successful, the other stages of testing and development have required better AI and more reliable rival AI. To implement them, the Utility architecture was used to create special AIs for testing and verifying game quality, in-game tests and balancing weapons and monsters, and creating specific mazes and monsters.

However, during the development of Unleash, the creators perfected their skills in the game, and then decided to use this knowledge to create a more complex Utility AI. So it became clear that many problems arising in systems of artificial intelligence, based on machine learning, can be easily solved with the help of Utility systems that use the knowledge embedded in them, and vice versa.

You can easily describe and program an algorithm to build a maze and the location of weapons in it in a way that would be easier to defend the tower from specific monsters to the living player.

But the creation of an army of monsters based on knowledge of the enemy base was a difficult task for such AI, because the number of different conditions and combinations that must be taken into account, struck the imagination.

The creation of hybrid AI

So, it was decided to combine the two approaches and thus to create a hybrid system of artificial intelligence based on machine learning and Utility. The idea was that where it was necessary to process a huge number of combinations and states of the game, or where it was necessary to teach it something, used machine learning.

For other tasks, where it was better to rely on personal experience of developers, Utility systems were used. This approach also has the advantage that you can better control the AI’s behavior if necessary to ensure that it is more accurate in achieving its intended purpose.

You can also assign neural networks specific value systems to form different preferences when recruiting air or ground monsters and thus add individual AI personality. There are many other options for implementing design solutions, and they all emphasize the strengths of one or the other AI architecture.

The hybrid approach was also the answer to another question the development team faced during the development of artificial intelligence for Unleash: is it worth applying a single global deep neural network based on machine learning to account for all input and output data, or is it better to design an AI with a hierarchical structure?