Execution-Time State Projection for Dynamic Air-Ground Logistics: A Digital-Twin Cooperative Q-Learning Approach

Abstract

In dynamic air-ground delivery, the operating conditions observed when an order is assigned may no longer hold when the selected resource actually becomes available. This timing gap is especially relevant when congestion, temporary airspace restrictions, or UAV endurance conditions change while tasks are waiting in a queue. To address this issue, we introduce execution-time state projection into a cooperative Q-learning scheduler. For every candidate vehicle or UAV, the scheduler estimates the state expected at the earliest feasible service-start time and uses this information when choosing the transport mode and resource rule. A deterministic execution mask is retained to prevent infeasible UAV actions from being dispatched. The mechanism is examined on a synthetic county-rural network containing 18 service nodes, three ground vehicles, and five UAVs. Six operating conditions are considered, with 50 matched replications for each method-scenario pair. The largest differences appear under the compound disruption. Relative to the rolling-horizon heuristic, mean response time decreases by 15.8%, P95 response time by 18.4%, the modeled energy index by 7.3%, and recovery time by 17.4%. These four differences remain significant after Holm adjustment (p<0.001). By contrast, the change in on-time delivery is only 0.65 percentage points and is not statistically significant (p=0.053). When execution-time projection is removed, mean response increases by 22.0%, and the stale-state policy proposes infeasible UAV actions for 16.9% of orders. Forecast-severity perturbations of ±10% and ±20% weaken performance gradually rather than abruptly. The results indicate that the main benefit arises from using state information that is closer to the actual execution time, rather than from a general advantage of reinforcement learning over heuristic dispatch.