Fuzzy Kernel Stochastic Gradient Descent Machines

Tuan Nguyen, Phuong Duong, Trung Le, Anh Le, Viet Ngo, Dat TRAN, Wanli MA

Research output: A Conference proceeding or a Chapter in BookConference contribution

1 Downloads (Pure)

Abstract

Stochastic Gradient Descent (SGD) based method offers a viable solution to training large-scale dataset. However, the traditional SGD-based methods cannot get benefit from the distribution or geometry information carried in data. The reason is that these methods make use of the uniform distribution over the entire training set so as to sample the next data point for updating the model. We address this issue by incorporating the distribution or geometry information carried in the data into the sampling procedure. In particular, we utilize the fuzzy-membership evaluation methods which allow transferring the distribution or geometry information carried in the data to the fuzzy memberships. The fuzzy memberships is then normalized to a discrete distribution from which the next data point is sampled. This allows the training staying more focused on the important data points and tending to ignore the less impact data points, e.g., the noises and outliers. We validate the proposed methods on 8 benchmark datasets. The experimental results show that the proposed methods are comparable with the standard SGD-based method in training time while offering a significant improvement in classification accuracy
Original languageEnglish
Title of host publication2016 International Joint Conference on Neural Networks (IJCNN)
EditorsHussein A. Abbass, Huanhuan Chen
Place of PublicationUnited States of America
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages3226-3232
Number of pages7
Volume1
ISBN (Electronic)9781509006205
ISBN (Print)9781509006212
DOIs
Publication statusPublished - 2016
Event2016 International Joint Conference on Neural Networks - Vancouver, Vancouver, Canada
Duration: 24 Jul 201629 Jul 2016

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2016-October

Conference

Conference2016 International Joint Conference on Neural Networks
Abbreviated titleIJCNN 2016
CountryCanada
CityVancouver
Period24/07/1629/07/16

Fingerprint Dive into the research topics of 'Fuzzy Kernel Stochastic Gradient Descent Machines'. Together they form a unique fingerprint.

  • Cite this

    Nguyen, T., Duong, P., Le, T., Le, A., Ngo, V., TRAN, D., & MA, W. (2016). Fuzzy Kernel Stochastic Gradient Descent Machines. In H. A. Abbass, & H. Chen (Eds.), 2016 International Joint Conference on Neural Networks (IJCNN) (Vol. 1, pp. 3226-3232). [7727611] (Proceedings of the International Joint Conference on Neural Networks; Vol. 2016-October). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IJCNN.2016.7727611