Fast rates by transferring from auxiliary hypotheses

Ilja Kuzborskij, Francesco Orabona

Research output: Contribution to journalArticlepeer-review

36 Scopus citations

Abstract

In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer. We establish generalization and excess risk bounds, showing that, if the algorithm is fed with a good combination of source hypotheses, generalization happens at the fast rate O(1 / m) instead of the usual O(1/m). On the other hand, if the source hypotheses combination is a misfit for the target task, we recover the usual learning rate. As a byproduct of our study, we also prove a new bound on the Rademacher complexity of the smooth loss class under weaker assumptions compared to previous works.
Original languageEnglish (US)
Pages (from-to)171-195
Number of pages25
JournalMachine Learning
Volume106
Issue number2
DOIs
StatePublished - Feb 1 2017
Externally publishedYes

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software

Fingerprint

Dive into the research topics of 'Fast rates by transferring from auxiliary hypotheses'. Together they form a unique fingerprint.

Cite this