Balancing Performance and Feasibility in Selecting VLMs for Temporal Motion Understanding

Kurzfassung

Selecting a video-capable vision language model for edge deployment requires balancing temporal reasoning capability against hardware constraints that eliminate most otherwise capable architectures. This paper presents a feasibility-first methodology for identifying video VLMs suitable for memory-constrained devices with native temporal motion understanding. Candidates from three architectural families are filtered through sequential elimination stages: for an 8GB edge device (NVIDIA Jetson Orin Nano Super), native video temporal encoding, and architectural currency. Four models survive: Mobile-VideoGPT-1.5B, VideoLLaMA3-2B, NVILA-Lite-2B, and Gemma3n-E2B. Each is fine-tuned using Low-Rank Adaptation on an exercise video corpus and evaluated with BERTScore, METEOR, and ROUGE-L. Mobile-VideoGPT-1.5B achieves the highest scores across all three metrics while requiring the fewest parameters. The filtering framework generalizes to any deployment that combines multimodal video understanding with resource-constrained hardware.