Supervision Levels Scale (SLS)
This provides a standardized way to compare methods by supervision levels, which is incremental for benchmarking in computer vision tasks.
The authors introduced the Supervision Levels Scale (SLS), a three-dimensional scale to encode supervision levels in model training, and applied it to the EPIC-KITCHENS-100 dataset for leaderboard comparisons.
We propose a three-dimensional discrete and incremental scale to encode a method's level of supervision - i.e. the data and labels used when training a model to achieve a given performance. We capture three aspects of supervision, that are known to give methods an advantage while requiring additional costs: pre-training, training labels and training data. The proposed three-dimensional scale can be included in result tables or leaderboards to handily compare methods not only by their performance, but also by the level of data supervision utilised by each method. The Supervision Levels Scale (SLS) is first presented generally fo any task/dataset/challenge. It is then applied to the EPIC-KITCHENS-100 dataset, to be used for the various leaderboards and challenges associated with this dataset.