DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Intention-Conditioned Long-Term Human Egocentric Action Anticipation

Valls Mascaró, Esteve and Ahn, Hyemin and Lee, Dongheui (2023) Intention-Conditioned Long-Term Human Egocentric Action Anticipation. In: 23rd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023, pp. 6037-6046. IEEE. 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, 2023-01-02 - 2023-01-07, Waikoloa, HI, USA. doi: 10.1109/WACV56688.2023.00599. ISBN 978-166549346-8. ISSN 2472-6737.

Full text not available from this repository.

Official URL: https://ieeexplore.ieee.org/document/10030492


To anticipate how a person would act in the future, it is essential to understand the human intention since it guides the subject towards a certain action. In this paper, we propose a hierarchical architecture which assumes a sequence of human action (low-level) can be driven from the human intention (high-level). Based on this, we deal with long-term action anticipation task in egocentric videos. Our framework first extracts this low- and high-level human information over the observed human actions in a video through a Hierarchical Multi-task Multi-Layer Perceptrons Mixer (H3M). Then, we constrain the uncertainty of the future through an Intention-Conditioned Variational Auto-Encoder (I-CVAE) that generates multiple stable predictions of the next actions that the observed human might perform. By leveraging human intention as high-level information, we claim that our model is able to anticipate more time-consistent actions in the long-term, thus improving the results over the baseline in Ego4D dataset. This work results in the state-of-the-art for Long-Term Anticipation (LTA) task in Ego4D by providing more plausible anticipated sequences, improving the anticipation scores of nouns and actions. Our work ranked first in both CVPR@2022 and ECCV@2022 Ego4D LTA Challenge.

Item URL in elib:https://elib.dlr.de/197483/
Document Type:Conference or Workshop Item (Speech)
Title:Intention-Conditioned Long-Term Human Egocentric Action Anticipation
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Valls Mascaró, EsteveTU Wienhttps://orcid.org/0000-0003-4195-8672UNSPECIFIED
Ahn, HyeminUNSPECIFIEDhttps://orcid.org/0000-0001-8081-6023UNSPECIFIED
Lee, DongheuiUNSPECIFIEDhttps://orcid.org/0000-0003-1897-7664UNSPECIFIED
Date:6 February 2023
Journal or Publication Title:23rd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In ISI Web of Science:Yes
Page Range:pp. 6037-6046
Keywords:Action Anticipation
Event Title:2023 IEEE/CVF Winter Conference on Applications of Computer Vision
Event Location:Waikoloa, HI, USA
Event Type:international Conference
Event Start Date:2 January 2023
Event End Date:7 January 2023
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Basic Technologies [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Deposited By: Strobl, Dr. Klaus H.
Deposited On:22 Sep 2023 12:56
Last Modified:24 Apr 2024 20:57

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.