Trespassing the Boundaries:
Labelling Temporal Bounds for Object Interactions in Egocentric Video

March 2017

Davide Moltisanti, Michael Wray, Walterio Mayol-Cuevas, Dima Damen


Temporal Boundaries Inconsistencies

Manual annotations of temporal bounds for object interactions (i.e. start and end times) are typical training input to recognition, localisation and detection algorithms.

For three publicly available egocentric datasets, we uncover inconsistencies in ground truth temporal bounds within and across annotators and datasets.
We systematically assess the robustness of state-of-the-art approaches to changes in labelled temporal bounds, for object interaction recognition.
As boundaries are trespassed, a drop of up to 10% is observed for both Improved Dense Trajectories and Two-Stream Convolutional Neural Network.

We demonstrate that such disagreement stems from a limited understanding of the distinct phases of an action, and propose annotating based on the Rubicon Boundaries,
inspired by a similarly named cognitive model, for consistent temporal bounds of object interactions. Evaluated on a public dataset, we report a 4% increase
in overall accuracy, and an increase in accuracy for 55% of classes when Rubicon Boundaries are used for temporal annotations.



Rubicon Boundaries examples

Publication:

Davide Moltisanti, Michael Wray, Walterio Mayol-Cuevas and Dima Damen (2017). Trespassing the Boundaries: Labelling Temporal Bounds for Object Interactions in Egocentric Video. International Conference on Computer Vision (ICCV), Venice, Italy, Oct 2017 | pdf (camera ready) | arXiv (v1)

RB Annotations:

Rubicon Boundary annotations for GTEA Gaze+ can be downloaded here

Software:

Software for annotating Rubicon boundaries in video can be downloaded here

Video: