Jin, Pu und Mou, LiChao und Hua, Yuansheng und Xia, Gui-Song und Zhu, Xiao Xiang (2022) FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification. IEEE Transactions on Geoscience and Remote Sensing, 60, Seite 5618913. IEEE - Institute of Electrical and Electronics Engineers. doi: 10.1109/TGRS.2022.3150917. ISSN 0196-2892.
![]() |
PDF
- Verlagsversion (veröffentlichte Fassung)
6MB |
Offizielle URL: https://ieeexplore.ieee.org/document/9709809
Kurzfassung
Unmanned aerial vehicles (UAVs) are now widely applied to data acquisition due to its low cost and fast mobility. With the increasing volume of aerial videos, the demand for automatically parsing these videos is surging. To achieve this, current research mainly focuses on extracting a holistic feature with convolutions along both spatial and temporal dimensions. However, these methods are limited by small temporal receptive fields and cannot adequately capture long-term temporal dependencies that are important for describing complicated dynamics. In this article, we propose a novel deep neural network, termed Fusing Temporal relations and Holistic features for aerial video classification (FuTH-Net), to model not only holistic features but also temporal relations for aerial video classification. Furthermore, the holistic features are refined by the multiscale temporal relations in a novel fusion module for yielding more discriminative video representations. More specially, FuTH-Net employs a two-pathway architecture: 1) a holistic representation pathway to learn a general feature of both frame appearances and short-term temporal variations and 2) a temporal relation pathway to capture multiscale temporal relations across arbitrary frames, providing long-term temporal dependencies. Afterward, a novel fusion module is proposed to spatiotemporally integrate the two features learned from the two pathways. Our model is evaluated on two aerial video classification datasets, ERA and Drone-Action, and achieves the state-of-the-art results. This demonstrates its effectiveness and good generalization capacity across different recognition tasks (event classification and human action recognition). To facilitate further research, we release the code at https://gitlab.lrz.de/ai4eo/reasoning/futh-net .
elib-URL des Eintrags: | https://elib.dlr.de/192765/ | ||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Zeitschriftenbeitrag | ||||||||||||||||||||||||
Titel: | FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification | ||||||||||||||||||||||||
Autoren: |
| ||||||||||||||||||||||||
Datum: | Februar 2022 | ||||||||||||||||||||||||
Erschienen in: | IEEE Transactions on Geoscience and Remote Sensing | ||||||||||||||||||||||||
Referierte Publikation: | Ja | ||||||||||||||||||||||||
Open Access: | Ja | ||||||||||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||||||||||
In SCOPUS: | Ja | ||||||||||||||||||||||||
In ISI Web of Science: | Ja | ||||||||||||||||||||||||
Band: | 60 | ||||||||||||||||||||||||
DOI: | 10.1109/TGRS.2022.3150917 | ||||||||||||||||||||||||
Seitenbereich: | Seite 5618913 | ||||||||||||||||||||||||
Verlag: | IEEE - Institute of Electrical and Electronics Engineers | ||||||||||||||||||||||||
ISSN: | 0196-2892 | ||||||||||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||||||||||
Stichwörter: | Aerial video classification, convolutional neural networks (CNNs), holistic features, temporal relations, two-pathway, unmanned aerial vehicle (UAV) | ||||||||||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||||||||||||||
HGF - Programmthema: | Erdbeobachtung | ||||||||||||||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||||||||||||||
DLR - Forschungsgebiet: | R EO - Erdbeobachtung | ||||||||||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Künstliche Intelligenz | ||||||||||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||||||||||
Institute & Einrichtungen: | Institut für Methodik der Fernerkundung > EO Data Science | ||||||||||||||||||||||||
Hinterlegt von: | Haschberger, Dr.-Ing. Peter | ||||||||||||||||||||||||
Hinterlegt am: | 22 Dez 2022 09:06 | ||||||||||||||||||||||||
Letzte Änderung: | 30 Jan 2024 11:10 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags