Abstract

Temporal Action Detection (TAD) localizes and classifies action instances within long videos and underlies many downstream video understanding applications. Transformer-based detectors scale poorly to long sequences due to quadratic self-attention, while existing SSM-based variants tend to dilute fine-grained boundary cues during global modeling. To address these limitations, we propose DST-Mamba, a Decoupled Spatial–Temporal Mamba Adapter inserted into a frozen video backbone for parameter-efficient end-to-end TAD. DST-Mamba decouples spatial and temporal modeling into two cooperating branches and explicitly fuses them through cross-branch interaction. Within the temporal branch, we introduce a Temporal Boundary-aware SSM (TB-SSM) with direction-specific forward/backward state-transition matrices, providing a stronger inductive bias for asymmetric action boundaries. Across multiple benchmarks, DST-Mamba consistently outperforms competitive Transformer- and SSM-based baselines while being more computationally efficient.

IPC Classification

Keywords