Abstract

Unmanned aerial vehicle (UAV) swarms generate massive multi-modal sensor data streams from onboard payloads such as RGB cameras, LiDAR, and thermal sensors. Large language models (LLMs) can interpret these data for natural language-based swarm coordination. However, deploying LLMs directly on resource-constrained UAV nodes faces a critical bottleneck. Long-context textual sensor logs (e.g., continuous status reports with GPS, altitude, and detection events) lead to high prefill latency. Existing distributed inference frameworks suffer from load imbalance and pipeline bubbles, violating real-time mission requirements. To address these issues, we propose Orion, an edge-only collaborative inference framework for LLM-based sensor data processing in heterogeneous UAV swarms. Orion incorporates three innovations: (1) optimal model partitioning via dynamic programming, (2) adaptive sequence partitioning that balances causal attention load across pipeline stages, and (3) a predictive decoding mechanism that speculatively generates the first token during idle intervals. Experiments on a comprehensive simulation framework ((using Meta’s Llama-2 (Large Language Model Meta AI)) 7B/13B/70B and simulated UAV swarm sensor traces) show that Orion reduces prefill latency by 81% (7B) and 78% (13B) compared to the best cloud–UAV baseline. Among the evaluated frameworks, Orion is the only framework capable of running the full 70B model on memory-constrained UAV nodes, enabling real-time sensor-aware LLM inference.

IPC Classification

Keywords