MS Thesis Project
Vision-and-Language Navigation for Autonomous Drone Search-and-Return in Urban Environments
Department of Computer Science and Engineering, University of California, Riverside
Abstract
This thesis studies a search-and-return extension of aerial vision-and-language navigation (VLN), in which a drone follows a language instruction to a target and then returns to its starting point. The main challenge is that the return phase must reuse information collected during the search. We extend OpenFly-Agent with a return trigger, a landmark memory module, and a return policy with an optional LoRA adapter, and construct SAR-Drone-VLN-3K, a paired search-and-return dataset for training and analysis. Experiments show that the base OpenFly-Agent returns only weakly even with landmark prompts, while the LoRA adapter often reaches the start area but fails to stop reliably. These results suggest that search-and-return navigation is harder than one-way aerial VLN and demands stronger memory and stopping decisions. To our knowledge, this is the first work to explicitly model the return phase as a separate component in language-guided aerial navigation.
Search-and-Return Replays
Overview
The project extends one-way aerial VLN into a two-phase task. The agent searches for the target, records compact landmark memory, triggers return, and then navigates back to the original start position.
Key Figures
Dataset Analysis
Quantitative Results
All baselines are evaluated on the first 100 episodes of the SAR evaluation set. Search, return, and overall SAR success are reported separately. Bold numbers indicate the best performance among the agent-based methods for each metric.
| Method | Search | Return | Overall | ||||||
|---|---|---|---|---|---|---|---|---|---|
| NE↓ | SR↑ | OSR↑ | SPL↑ | NE↓ | SR↑ | OSR↑ | SPL↑ | SAR-SR↑ | |
| Random return | 5.01 m | 100% | 100% | 91.0% | 251.73 m | 1% | 6% | 1.0% | 1% |
| Base fixed | 237.88 m | 6% | 50% | 4.8% | 275.44 m | 24% | 29% | 22.8% | 1% |
| Base landmark | 213.35 m | 10% | 48% | 9.1% | 237.63 m | 27% | 30% | 24.6% | 4% |
| LoRA fixed | 246.38 m | 6% | 48% | 5.7% | 382.85 m | 5% | 37% | 4.5% | 0% |
| LoRA landmark | 231.42 m | 9% | 52% | 8.8% | 334.43 m | 10% | 36% | 9.0% | 2% |
The unadapted OpenFly-Agent benefits from landmark memory and reaches a return SR of 27%. The LoRA variants achieve higher return OSR (36–37%) but lower SR, suggesting the policy can move toward the start position but often fails to stop in time. The next table inspects this stopping behavior.
| Method | Trajectories | stop% | mean_len | min_len | max_len |
|---|---|---|---|---|---|
| GT (eval 240) | 240 | 100.0% | 17.96 | 2 | 65 |
| GT (first 100) | 100 | 100.0% | 19.27 | 2 | 65 |
| Base fixed | 100 | 94.0% | 4.84 | 1 | 30 |
| Base landmark | 100 | 74.0% | 11.00 | 1 | 30 |
| LoRA fixed | 100 | 44.0% | 20.99 | 1 | 30 |
| LoRA landmark | 100 | 35.0% | 20.74 | 1 | 30 |
The base policy with a fixed prompt collapses to an immediate stop (mean return length 4.84 vs. GT 17.96). Adding the landmark prompt pushes the mean length to 11.0 and lowers the explicit stop rate to 74%, showing route-specific context helps the unadapted policy resist early stopping. The LoRA variants reach around 21 steps, close to the GT distribution, but only 35–44% of episodes terminate via an explicit stop prediction — the remainder are truncated by the 30-step return budget, which directly explains the large OSR–SR gap.
3D Trajectory Visualizations
Nine representative episodes are visualized below. In every subplot, the green dashed curve is the search path, the red solid curve is the return path, the blue star marks the start point, the yellow triangle marks the search endpoint, and the red inverted triangle marks the return endpoint. A return is counted as successful when the return endpoint lies within 20 m of the start point.
