TY - GEN

T1 - A Precise Performance Analysis of Support Vector Regression

AU - Sifaou, Houssem

AU - Kammoun, Abla

AU - Alouini, Mohamed Slim

N1 - Publisher Copyright:
Copyright © 2021 by the author(s)

PY - 2021

Y1 - 2021

N2 - In this paper, we study the hard and soft support vector regression techniques applied to a set of n linear measurements of the form yi = βT? xi + ni where β? is an unknown vector, (xi)ni=1 are the feature vectors and (ni)ni=1 model the noise. Particularly, under some plausible assumptions on the statistical distribution of the data, we characterize the feasibility condition for the hard support vector regression in the regime of high dimensions and, when feasible, derive an asymptotic approximation for its risk. Similarly, we study the test risk for the soft support vector regression as a function of its parameters. Our results are then used to optimally tune the parameters intervening in the design of hard and soft support vector regression algorithms. Based on our analysis, we illustrate that adding more samples may be harmful to the test performance of support vector regression, while it is always beneficial when the parameters are optimally selected. Such a result reminds a similar phenomenon observed in modern learning architectures according to which optimally tuned architectures present a decreasing test performance curve with respect to the number of samples.

AB - In this paper, we study the hard and soft support vector regression techniques applied to a set of n linear measurements of the form yi = βT? xi + ni where β? is an unknown vector, (xi)ni=1 are the feature vectors and (ni)ni=1 model the noise. Particularly, under some plausible assumptions on the statistical distribution of the data, we characterize the feasibility condition for the hard support vector regression in the regime of high dimensions and, when feasible, derive an asymptotic approximation for its risk. Similarly, we study the test risk for the soft support vector regression as a function of its parameters. Our results are then used to optimally tune the parameters intervening in the design of hard and soft support vector regression algorithms. Based on our analysis, we illustrate that adding more samples may be harmful to the test performance of support vector regression, while it is always beneficial when the parameters are optimally selected. Such a result reminds a similar phenomenon observed in modern learning architectures according to which optimally tuned architectures present a decreasing test performance curve with respect to the number of samples.

UR - http://www.scopus.com/inward/record.url?scp=85150252372&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85150252372

T3 - Proceedings of Machine Learning Research

SP - 9671

EP - 9680

BT - Proceedings of the 38th International Conference on Machine Learning, ICML 2021

PB - Mathematical Research Press

T2 - 38th International Conference on Machine Learning, ICML 2021

Y2 - 18 July 2021 through 24 July 2021

ER -