Diagnosing Error in Temporal Action Detectors

Humam Alwassel; Fabian Caba Heilbron; Victor Escorcia; Bernard Ghanem

doi:10.1007/978-3-030-01219-9_16

Diagnosing Error in Temporal Action Detectors

Humam Alwassel^*, Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

11 Scopus citations

Abstract

Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.

Original language	English (US)
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert, Yair Weiss
Publisher	Springer Verlag
Pages	264-280
Number of pages	17
ISBN (Print)	9783030012182
DOIs	https://doi.org/10.1007/978-3-030-01219-9_16
State	Published - 2018
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: Sep 8 2018 → Sep 14 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11207 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	09/8/18 → 09/14/18

Keywords

Action localization
Diagnosis tool
Error analysis
Temporal action detection

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-01219-9_16

Cite this

Alwassel, H., Caba Heilbron, F., Escorcia, V., & Ghanem, B. (2018). Diagnosing Error in Temporal Action Detectors. In V. Ferrari, C. Sminchisescu, M. Hebert, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 264-280). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11207 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01219-9_16

Alwassel, Humam ; Caba Heilbron, Fabian ; Escorcia, Victor et al. / Diagnosing Error in Temporal Action Detectors. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Vittorio Ferrari ; Cristian Sminchisescu ; Martial Hebert ; Yair Weiss. Springer Verlag, 2018. pp. 264-280 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c60f06a156484ca097165f225a9f0d00,

title = "Diagnosing Error in Temporal Action Detectors",

abstract = "Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.",

keywords = "Action localization, Diagnosis tool, Error analysis, Temporal action detection",

author = "Humam Alwassel and {Caba Heilbron}, Fabian and Victor Escorcia and Bernard Ghanem",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01219-9_16",

language = "English (US)",

isbn = "9783030012182",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "264--280",

editor = "Vittorio Ferrari and Cristian Sminchisescu and Martial Hebert and Yair Weiss",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

address = "Germany",

}

Alwassel, H, Caba Heilbron, F, Escorcia, V & Ghanem, B 2018, Diagnosing Error in Temporal Action Detectors. in V Ferrari, C Sminchisescu, M Hebert & Y Weiss (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11207 LNCS, Springer Verlag, pp. 264-280, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 09/8/18. https://doi.org/10.1007/978-3-030-01219-9_16

Diagnosing Error in Temporal Action Detectors. / Alwassel, Humam; Caba Heilbron, Fabian; Escorcia, Victor et al.
Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert; Yair Weiss. Springer Verlag, 2018. p. 264-280 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11207 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Diagnosing Error in Temporal Action Detectors

AU - Alwassel, Humam

AU - Caba Heilbron, Fabian

AU - Escorcia, Victor

AU - Ghanem, Bernard

PY - 2018

Y1 - 2018

N2 - Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.

AB - Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.

KW - Action localization

KW - Diagnosis tool

KW - Error analysis

KW - Temporal action detection

UR - http://www.scopus.com/inward/record.url?scp=85055132729&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01219-9_16

DO - 10.1007/978-3-030-01219-9_16

M3 - Conference contribution

AN - SCOPUS:85055132729

SN - 9783030012182

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 264

EP - 280

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Hebert, Martial

A2 - Weiss, Yair

PB - Springer Verlag

T2 - 15th European Conference on Computer Vision, ECCV 2018

Y2 - 8 September 2018 through 14 September 2018

ER -

Alwassel H, Caba Heilbron F, Escorcia V, Ghanem B. Diagnosing Error in Temporal Action Detectors. In Ferrari V, Sminchisescu C, Hebert M, Weiss Y, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 264-280. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01219-9_16

Diagnosing Error in Temporal Action Detectors

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this