SAGES consensus recommendations on an annotation framework for surgical video

the SAGES Video Annotation for AI Working Groups

Research output: Contribution to journalArticlepeer-review

34 Scopus citations


Background: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. Methods: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. Results: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. Conclusions: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

Original languageEnglish (US)
Pages (from-to)4918-4929
Number of pages12
JournalSurgical Endoscopy
Issue number9
StatePublished - Sep 2021


  • Annotation
  • Artificial intelligence
  • Computer vision
  • Consensus
  • Minimally invasive surgery
  • Surgical video

ASJC Scopus subject areas

  • Surgery


Dive into the research topics of 'SAGES consensus recommendations on an annotation framework for surgical video'. Together they form a unique fingerprint.

Cite this