Incorporating Online Learning Into MCTS-Based Intention Progression

Agents have been applied to a wide variety of fields, including power systems and spacecraft. Belief-Desire-Intention (BDI) agents, as one of the most widely used and researched architectures, have the advantage of being able to pursue multiple goals in parallel. The problem of deciding “what to do” next at each of the agent's deliberation cycle is therefore critical for BDI agents, which is defined as the intention progression problem (IPP). Among all existing approaches to IPP, the majority of approaches have overlooked the significance of runtime historical data, thereby limiting the adaptability and decision-making capabilities of agents to some extent. In this paper, we propose to incorporate online learning into the current state-of-the- art intention progression approach S_A to overcome the limitation that S_A cannot utilize historical data to guide the decision-making of agents. This approach not only prevents S_A from consuming computational resources on ineffective and inefficient simulations, but also significantly improves the execution efficiency of the agent. Especially when dealing with large-scale problem domains, this improvement significantly enhances the planning capability of the agents. In particular, we have proposed the SA_Q and SA_L schedulers, both of which can learn how to generate “reasonable” rollouts during the simulation phase of MCTS based on historical simulation data at run time.We compare the performance of our approach with the state-of-the-art S_A in a range of scenarios of increasing difficulty. The results demonstrate that our approaches outperform S_A , both in terms of the number of goals achieved and the computational overhead required.