An improved Q-learning algorithm using synthetic pheromones

Abstract

In this paper we propose an algorithm for multi-agent Q-learning. The algorithm is inspired by the natural behaviour of ants, which deposit pheromone in the environment to communicate. The benefit besides simulating ant behaviour in a colony is to design complex multi-agent systems. Complex behaviour can emerge from relatively simple interacting agents. The proposed Q-learning update equation includes a belief factor. The belief factor reflects the confidence the agent has in the pheromone detected in its environment. Agents communicate implicitly to co-ordinate and co-operate in learning to solve a problem.