Efficient AUC maximization with regularized least-squares

Tapio Pahikkala, Antti Airola, Hanna Suominen, Jorma Boberg, Tapio I. Salakoski

Research output: A Conference proceeding or a Chapter in BookConference contribution

4 Citations (Scopus)

Abstract

Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing such algorithms in the framework of kernel methods has proven to be challenging. In this paper, we address AUC maximization with the regularized least-squares (RLS) algorithm also known as the least-squares support vector machine. First, we introduce RLStype binary classifier that maximizes an approximation of AUC and has a closedform solution. Second, we show that this AUC-RLS algorithm is computationally as efficient as the standard RLS algorithm that maximizes an approximation of the accuracy. Third, we compare the performance of these two algorithms in the task of assigning topic labels for newswire articles in terms of AUC. Our algorithm outperforms the standard RLS in every classification experiment conducted. The performance gains are most substantial when the distribution of the class labels is unbalanced. In conclusion, modifying the RLS algorithm to maximize the approximation of AUC does not increase the computational complexity, and this alteration enhances the quality of the classifier.

Original languageEnglish
Title of host publicationProceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008
EditorsAnders Holst, Per Kreuger, Peter Funk
Place of PublicationUnited States
PublisherIOS Press
Pages12-19
Number of pages8
Volume173
ISBN (Print)9781586038670
Publication statusPublished - 2008
Externally publishedYes
Event10th Scandinavian Conference on Artificial Intelligence - Stockholm, Stockholm, Sweden
Duration: 26 May 200828 May 2008

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume173
ISSN (Print)09226389

Conference

Conference10th Scandinavian Conference on Artificial Intelligence
Abbreviated titleSCAI 2008
CountrySweden
CityStockholm
Period26/05/0828/05/08

Fingerprint

Classifiers
Labels
Learning algorithms
Support vector machines
Learning systems
Computational complexity
Experiments

Cite this

Pahikkala, T., Airola, A., Suominen, H., Boberg, J., & Salakoski, T. I. (2008). Efficient AUC maximization with regularized least-squares. In A. Holst, P. Kreuger, & P. Funk (Eds.), Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008 (Vol. 173, pp. 12-19). (Frontiers in Artificial Intelligence and Applications; Vol. 173). United States: IOS Press.
Pahikkala, Tapio ; Airola, Antti ; Suominen, Hanna ; Boberg, Jorma ; Salakoski, Tapio I. / Efficient AUC maximization with regularized least-squares. Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008. editor / Anders Holst ; Per Kreuger ; Peter Funk. Vol. 173 United States : IOS Press, 2008. pp. 12-19 (Frontiers in Artificial Intelligence and Applications).
@inproceedings{7b6af960b40c4ad2877ef22fbe25e2fe,
title = "Efficient AUC maximization with regularized least-squares",
abstract = "Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing such algorithms in the framework of kernel methods has proven to be challenging. In this paper, we address AUC maximization with the regularized least-squares (RLS) algorithm also known as the least-squares support vector machine. First, we introduce RLStype binary classifier that maximizes an approximation of AUC and has a closedform solution. Second, we show that this AUC-RLS algorithm is computationally as efficient as the standard RLS algorithm that maximizes an approximation of the accuracy. Third, we compare the performance of these two algorithms in the task of assigning topic labels for newswire articles in terms of AUC. Our algorithm outperforms the standard RLS in every classification experiment conducted. The performance gains are most substantial when the distribution of the class labels is unbalanced. In conclusion, modifying the RLS algorithm to maximize the approximation of AUC does not increase the computational complexity, and this alteration enhances the quality of the classifier.",
keywords = "Area under curve, Binary classifier, Regularized least-squares",
author = "Tapio Pahikkala and Antti Airola and Hanna Suominen and Jorma Boberg and Salakoski, {Tapio I.}",
year = "2008",
language = "English",
isbn = "9781586038670",
volume = "173",
series = "Frontiers in Artificial Intelligence and Applications",
publisher = "IOS Press",
pages = "12--19",
editor = "Anders Holst and Per Kreuger and Peter Funk",
booktitle = "Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008",
address = "Netherlands",

}

Pahikkala, T, Airola, A, Suominen, H, Boberg, J & Salakoski, TI 2008, Efficient AUC maximization with regularized least-squares. in A Holst, P Kreuger & P Funk (eds), Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008. vol. 173, Frontiers in Artificial Intelligence and Applications, vol. 173, IOS Press, United States, pp. 12-19, 10th Scandinavian Conference on Artificial Intelligence, Stockholm, Sweden, 26/05/08.

Efficient AUC maximization with regularized least-squares. / Pahikkala, Tapio; Airola, Antti; Suominen, Hanna; Boberg, Jorma; Salakoski, Tapio I.

Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008. ed. / Anders Holst; Per Kreuger; Peter Funk. Vol. 173 United States : IOS Press, 2008. p. 12-19 (Frontiers in Artificial Intelligence and Applications; Vol. 173).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Efficient AUC maximization with regularized least-squares

AU - Pahikkala, Tapio

AU - Airola, Antti

AU - Suominen, Hanna

AU - Boberg, Jorma

AU - Salakoski, Tapio I.

PY - 2008

Y1 - 2008

N2 - Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing such algorithms in the framework of kernel methods has proven to be challenging. In this paper, we address AUC maximization with the regularized least-squares (RLS) algorithm also known as the least-squares support vector machine. First, we introduce RLStype binary classifier that maximizes an approximation of AUC and has a closedform solution. Second, we show that this AUC-RLS algorithm is computationally as efficient as the standard RLS algorithm that maximizes an approximation of the accuracy. Third, we compare the performance of these two algorithms in the task of assigning topic labels for newswire articles in terms of AUC. Our algorithm outperforms the standard RLS in every classification experiment conducted. The performance gains are most substantial when the distribution of the class labels is unbalanced. In conclusion, modifying the RLS algorithm to maximize the approximation of AUC does not increase the computational complexity, and this alteration enhances the quality of the classifier.

AB - Area under the receiver operating characteristics curve (AUC) is a popular measure for evaluating the quality of binary classifiers, and intuitively, machine learning algorithms that maximize an approximation of AUC should have a good AUC performance when classifying new examples. However, designing such algorithms in the framework of kernel methods has proven to be challenging. In this paper, we address AUC maximization with the regularized least-squares (RLS) algorithm also known as the least-squares support vector machine. First, we introduce RLStype binary classifier that maximizes an approximation of AUC and has a closedform solution. Second, we show that this AUC-RLS algorithm is computationally as efficient as the standard RLS algorithm that maximizes an approximation of the accuracy. Third, we compare the performance of these two algorithms in the task of assigning topic labels for newswire articles in terms of AUC. Our algorithm outperforms the standard RLS in every classification experiment conducted. The performance gains are most substantial when the distribution of the class labels is unbalanced. In conclusion, modifying the RLS algorithm to maximize the approximation of AUC does not increase the computational complexity, and this alteration enhances the quality of the classifier.

KW - Area under curve

KW - Binary classifier

KW - Regularized least-squares

UR - http://www.scopus.com/inward/record.url?scp=84867510815&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781586038670

VL - 173

T3 - Frontiers in Artificial Intelligence and Applications

SP - 12

EP - 19

BT - Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008

A2 - Holst, Anders

A2 - Kreuger, Per

A2 - Funk, Peter

PB - IOS Press

CY - United States

ER -

Pahikkala T, Airola A, Suominen H, Boberg J, Salakoski TI. Efficient AUC maximization with regularized least-squares. In Holst A, Kreuger P, Funk P, editors, Proceedings of the 2008 Conference on 10th Scandinavian Conference on Artificial Intelligence: SCAI 2008. Vol. 173. United States: IOS Press. 2008. p. 12-19. (Frontiers in Artificial Intelligence and Applications).