This project aims at investigating sampling-based large-scale machine-learning methods for classification and regression. All the developed techniques will be benchmarked on standard image classification and object-detection data-sets, on pedestrian detection and re-identification, and on controlled data-sets produced over the course of the project.
We structure this proposal in three sub-projects. The first will investigate a general strategy based on importance sampling to deal with very large training sets. While the current common approach to learning is the stochastic gradient descent, very few efforts have been invested in the choice of the ``re-sampling'' strategy. Virtually every state-of-the-art method uses a uniform visit of the samples over the learning, without prioritizing according to computation done previously. Our central objective is to develop a general framework to apply importance-sampling to gradient-descent and other optimization schemes so that they concentrate the computational effort over problematic and informative samples. Preliminary results show that such an approach to learning can be applied to both Boosting and neural networks, and that it can leverage a prior structure over the samples to avoid looking at redundant subsets, and consequently achieve sub-linear training times.
The second sub-project will focus on learning pose-indexed ``stationary features''. Instead of trying to learn features with individual invariance to geometric and photometric perturbations as it is usually done, we want to learn groups of features whose joint behavior exhibit statistical regularities.
The third sub-project will investigate the data-driven optimization of multi-layer perceptron structures, to avoid the requirement for the standard and computational intensive combination of meta-parameter grid-search and hand-tuning. Both of these topics are perfect test-beds for importance-based learning, since they require the use of very large sample set to provide meaningful empirical loss estimates.