Analysis of Soft Friend or Foe Reinforcement Learning Algorithm in Multiagent Environment

Abstract

This paper evaluates a new off policy multiagent reinforcement learning algorithm called Soft Friend or Foe. The new algorithm is the result of modifying the Friend or Foe [1] algorithm by using the correlation in returns between two agents to soften the distinction between friend and foe. The goal is to achieve results similar to the Nash-Q [3] algorithm without the computational complexity and convergence issues. Comparison of three multiagent reinforcement learning algorithms is performed on three simple grid world environments. The algorithms consist of: Michael Littman's Friend or Foe algorithm[1], Soft Friend or Foe, and the Q-Learning algorithm[6] adjusted to a multiagent environment. The Soft Friend or Foe was shown to converge faster than the other two algorithms and get returns equal to or greater than returns received using Q-Learning. Soft Friend or Foe received returns as good as Friend or Foe in all environments.