Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters

Hyperparameter tuning is a mandatory step for building a support vector machine classifier. In this article, the authors study some methods based on metrics of the training set itself, and not the performance of the classifier on a different test set – the usual cross-validation approach. Then, they compare cross-validation (5-fold) with Xi-alpha, radius-margin bound, generalized approximate cross validation, maximum discrepancy and distance between two classes on 110 public binary data sets.

The authors demonstrate that cross validation is the method that resulted in the best selection of the hyper-parameters, but it is also the method with one of the highest execution time. On the other hand, distance between two classes (DBTC) is the fastest and the second best ranked method. The authors also discuss that DBTC is a reasonable alternative to cross validation when training/hyperparameter-selection times are an issue and that the loss in accuracy when using DBTC is reasonably small.


Edson Duarte, Jacques Wainer, Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters, Pattern Recognition Letters, Volume 88, 1 March 2017, Pages 6-11, ISSN 0167-8655, http://dx.doi.org/10.1016/j.patrec.2017.01.007.

Advertisements
This entry was posted in blog, publications, science and tagged , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s