Systematic Benchmarking and Edge-Aware Selection of Compact Vision-Language Models for Blind and Low-Vision Navigation Assistance

Kurzfassung

Visual impairment poses significant challenges for independent navigation, as traditional assistive tools lack semantic scene understanding and human-assisted services introduce latency and privacy concerns. While Vision-Language Models (VLMs) offer robust spatial reasoning, their substantial memory footprints and high inference latency typically restrict real-time, localized deployment. This paper investigates the optimization of compact, edge-deployable VLMs (at the 2B parameter scale) to provide proactive, safety-critical pedestrian navigation assistance for blind and low-vision (BLV) individuals. Operating within the strict hardware constraints of the NVIDIA Jetson Orin Nano, we systematically evaluate seven candidate models using a curated, navigation-specific subset of the Walking Awareness Dataset. Based on baseline evaluations of spatial reasoning and inference speed, Gemma-3n-2B and InternVL3-2B were selected for targeted Low-Rank Adaptation (LoRA) fine-tuning. This alignment phase successfully shifted the models’ zero-shot behavior from generating verbose environmental descriptions to producing concise, actionable alerts using strict clock-face directional formatting. Empirical results demonstrate profound improvements across both quantitative metrics and deployment feasibility; fine-tuning increased the CIDEr score of Gemma-3n-2B from 0.263 to 1.137 (reducing latency to 2.05s), and improved InternVL3-2B’s CIDEr score from 0.388 to 0.832 while achieving an optimal inference time of 0.62 seconds. Ultimately, this work establishes that task-specifically fine-tuned compact VLMs can deliver robust, low-latency, and privacy-preserving navigational guidance entirely on edge hardware.