Overview of computer vision supervised learning techniques for low-data training

Abstract

This work is an overview of techniques of varying complexity and novelty for supervised, or rather weakly supervised learning for computer vision algorithms. With the advent of deep learning the number of organizations and practitioners who think that they can solve problems using it also grows. Deep learning algorithms normally require vast amounts of labeled data, but depending on the domain it is not always possible to have a well annotated huge dataset, just think about healthcare. This paper starts with giving some background on supervised, weakly-supervised and then self-supervised learning in general, and in computer vision specifically. Then it goes on describing various methods to ease the need for a big labeled dataset. The paper describes the importance of these methods in fields such as medical imaging, autonomous driving, and even drone autonomous navigation. Starting with simple methods like knowledge transfer it also describes a number of knowledge distillation techniques and ends with the latest methods from self- and semi-supervised methods like Unsupervised Data Augmentation (UDA), MixMatch, Snorkel and adding synthetic tasks to the learning model, thus touching the multi-task learning problem. Finally topics/papers not reviewed yet are mentioned with some commentaries and the paper is closed with a discussions section. This paper does not go into few-shot/one-shot learning, because this another huge sub-domain, with a scope a bit different from the one of weaklysupervised and self-supervised learning.