CourIRL: Predicting Couriers' Behavior in Last-Mile Delivery Using Crossed-Attention Inverse Reinforcement Learning

Abstract

Human behavior prediction is an essential AI-based task, which has inspired many real-world applications. In last-mile logistics, predicting couriers' behavior can benefit the couriers' preference learning and workflow optimization. In this paper, we devote to the behavioral prediction of courier workload and quantify their workload by the working time spent at each area of interest (AOI). Considering the behavior interpretability of inverse reinforcement learning (IRL), existing studies have applied IRL to some real-world transportation prediction scenarios. However, in last-mile logistics, the platform assigns multiple orders to each courier, and couriers also receive new tasks in real-time, which additionally influence the couriers' subsequent decisions. The uncertainty in decision spaces and dynamic the workflow distribution make it more challenging to predict the couriers' working time. In this paper, we propose CourIRL, a practical IRL-based framework leveraging cross-attention to integrate Couriers' historical and spatio-temporal features to predict their future working time. CourIRL formulates the couriers' pick-up and delivery tour as a sequential decision-making process and designs a model-free IRL to learn decision-making preference vectors. A multi-head cross-attention mechanism-based deep regression model is proposed for fine-grained working-time prediction. The results of extensive experiments on two real-world datasets demonstrate that the proposed CourIRL surpasses the state-of-the-art baselines by an average of 6.11% across settings, showing the efficacy and potential contributions of CourIRL in last-mile logistics.