Introduction

Games teach lessons. Research has already proven it with students. For Machine Learning (ML) algorithms, game-based learning is no exception. Any machine learning algorithm’s success relies on its ability to learn. To do this, it needs data. Games offer a unique way to instill skills and knowledge. By providing a playful, yet challenging environment, you can make machine learn and benefit from the experience.

Here is an example of a game-based learning algorithm:

```
import numpy as np
import matplotlib.pyplot as plt

# Create a dataset
X = np.array([[0,0,1],
              [0,1,1],
              [1,0,1],
              [1,1,1]])

y = np.array([[0],
              [1],
              [1],
              [0]])

# Define the sigmoid function
def sigmoid(x):
    return 1/(1+np.exp(-x))

# Derivative of the sigmoid function
def sigmoid_derivative(x):
    return x * (1 - x)

# Define the loss function
def loss(y, y_hat):
    return -np.mean(y*np.log(y_hat) + (1-y)*np.log(1-y_hat))

# Define the neural network
class NeuralNetwork:
    def __init__(self, x, y):
        self.input = x
        self.weights1 = np.random.rand(self.input.shape[1],4)
        self.weights2 = np.random.rand(4,1)
        self.y = y
        self.output = np.zeros(self.y.shape)
        
    def feedforward(self):
        self.layer1 = sigmoid(np.dot(self.input, self.weights1))
        self.output = sigmoid(np.dot(self.layer1, self.weights2))
        
    def backprop(self):
        # application of the chain rule to find derivative of the loss function with respect to weights2 and weights1
        d_weights2 = np.dot(self.layer1.T, (2*(self.y - self.output) * sigmoid_derivative(self.output)))
        d_weights1 = np.dot(self.input.T,  (np.dot(2*(self.y - self.output) * sigmoid_derivative(self.output), self.weights2.T) * sigmoid_derivative(self.layer1)))

        # update the weights with the derivative (slope) of the loss function
        self.weights1 += d_weights1
        self.weights2 += d_weights2

if __name__ == "__main__":
    nn = NeuralNetwork(X,y)

    for i in range(1500):
        nn.feedforward()
        nn.backprop()

    print(nn.output)
```

The above neural network learns to play a game. The game is a simple one; a game of logic. How the neural network learns to play the game is by simply playing it. Yes, it plays the game by making guesses; it makes guesses by using the sigmoid function. The sigmoid function is a mathematical function that takes a number and returns a number between 0 and 1. In simpler terms, it involves a neural network, fed with with weights, input, and output variables. As it is trained, it uses sigmoid and backpropagation functions to adjust the weights. In the end, it returns the predicted output.

Game-Based Reinforcement Learning

That was just one simple example of how game-based learning can be used for ML algorithms. Developers can also create games that teach reinforcement learning algorithms. By providing a stimulating environment, you can make the machine learn more effectively. With reinforcement learning, the machine must learn to optimize its actions in order to gain rewards. For example:


```
import numpy as np

# Initialize the environment
env = Environment()

# Define the reward function
def reward(s, a, s_):
    return -1 if s_ == 'terminal' else 0

# Initialize the Q-Table
Q = np.zeros([env.observation_space, env.action_space])

# Iterate through episodes
for episode in range(1000):
    # Reset the environment
    s = env.reset()

    # Iterate through time steps
    while True:
        # Choose an action
        a = np.argmax(Q[s,:] + np.random.randn(1, env.action_space)*(1./(episode+1)))

        # Take the action
        s_, r, done = env.step(a)

        # Update the Q-Table
        Q[s, a] = Q[s, a] + learning_rate * (r + gamma * np.max(Q[s_,:]) - Q[s,a])

        # Check if the episode is done
        if done:
            break

        # Set the state
        s = s_
```

The above example was of Q-learning, a type of reinforcement learning algorithm.

Here is another example of reinforcement learning from Q-learning algorithm for the Pong game:

import gym
import numpy as np

# Create the environment
env = gym.make('Pong-v0')

# Initialize the Q-Table
Q = np.zeros([env.observation_space.n,env.action_space.n])

# Parameters
alpha = 0.8
gamma = 0.95
num_episodes = 2000

# List to keep track of rewards
rList = []

# Training Loop
for i in range(num_episodes):
    # Reset the environment
    s = env.reset()
    rAll = 0
    done = False
    j = 0

    # The Q-Table learning algorithm
    while j < 99:
        j+=1
        # Choose an action by greedily (with noise) picking from Q table
        a = np.argmax(Q[s,:] + np.random.randn(1,env.action_space.n)*(1./(i+1)))
        
        # Get new state and reward from environment
        s1,r,done,_ = env.step(a)
        
        # Update Q-Table with new knowledge
        Q[s,a] = Q[s,a] + alpha*(r + gamma*np.max(Q[s1,:]) - Q[s,a])
        
        rAll += r
        s = s1
        if done == True:
            break
    
    # Reduce chance of random action as we train the model.
    e = 1./((i/50) + 10)
    rList.append(rAll)

print("Score over time: " +  str(sum(rList)/num_episodes))
print("Final Q-Table Values")
print(Q)

Below is an example of game-based reinforcement learning, where you use an actual game to train a ML algorithm: (Using OpenAI’s library; works with the OpenAI Gym environment.) In the below example, the algorithm is trained using the FrozenLake game. The algorithm learns to take the most optimal path to the goal.

import gym
import numpy as np

# Initialize the environment
env = gym.make('FrozenLake-v0')

# Initialize the Q-Table
Q = np.zeros([env.observation_space.n, env.action_space.n])

# Set the learning parameters
lr = .8
y = .95
num_episodes = 2000

# Create lists to contain total rewards and steps per episode
rList = []
for i in range(num_episodes):
    # Reset the environment and get the first new observation
    s = env.reset()
    rAll = 0
    d = False
    j = 0
    # The Q-Table learning algorithm
    while j < 99:
        j+=1
        # Choose an action by greedily (with noise) picking from Q table
        a = np.argmax(Q[s,:] + np.random.randn(1,env.action_space.n)*(1./(i+1)))
        # Get new state and reward from environment
        s1,r,d,_ = env.step(a)
        # Update Q-Table with new knowledge
        Q[s,a] = Q[s,a] + lr*(r + y*np.max(Q[s1,:]) - Q[s,a])
        rAll += r
        s = s1
        if d == True:
            break
    rList.append(rAll)

print("Score over time: " +  str(sum(rList)/num_episodes))
print("Final Q-Table Values")
print(Q)

Through a game, reinforcement learning algorithms can learn to take the most optimal path to a goal. That’s because the game provides the right kind of environment; for example, a maze-like game or an environment with obstacles. Real-world games can also be used to teach ML algorithms. Such games often contribute to teaching physical robots how to interact with real environments.

Take this one for an example:

As you see in the above YouTube video, deepmind’s humanoid robots are learning to play soccer in a virtual environment. And most importantly, they are getting better.

Related Post: Machine Learning(ML) and Infinite Machine Learning(IML)

Broad usage of game-based learning in ML

Game-based learning offers many advantages over traditional learning methods for training algorithms. Traditionally, algorithms were trained through tedious manual processes. For example, manual trial and error, tedious programming, and other labor-intensive processes. Even popular programs from the past used to require long hours of tedious programming. Some examples of those programs were 90s-era adventure games and the first generations of gaming consoles.

In most cases, game-based learning is used to improve in-game environments. For example, using game-based learning, developers can create NPCs that can learn from their successes and failures. As such, NPCs can evolve over time by taking into account the actions taken by players. But game-based learning’s even broader use is to make the algorithm learn from games, to implement in real-world scenarios and tasks. For example, if you have to train an algorithm to recognize objects in images, you can use a game-based learning platform to teach it how to recognize those objects. Similarly, games can also teach ML algorithms to predict crimes. That’s by feeding them data gathered from different in-game crime scenarios so that they can make better decisions.

Game-based learning is not only about reinforcement learning. The definition of reinforcement learning is learning from reward and punishment; other approaches, such as evolutionary computing, are also used to teach algorithms through games. Game-based learning also offers the potential for scalability and repeatability. Algorithms can be tested on a large scale, and the same game-based learning experience can be reused and repeated for greater efficiency.

Conclusion

One interesting thing about the nature of games is that their rules and structure make learning suitable. There is no physical limitation to the number of players, only the imagination of the players. For any Machine learning algorithm, games are like an open playground they can use to learn, just like kids do. Formation of game-based learning helps the ML algorithms to discover new patterns and interpret data, while making the entire process enjoyable. Not only for game developers, but also for real-world scenarios, game-based learning for ML makes a difference.

Latest posts by Britney Foster (see all)
Previous post Disney Plus Free Trial Availability Status Worldwide
Next post Introduction and types of virtual keyboards
Show Buttons
Hide Buttons