Mobile big data analytics using deep learning and apache spark

Mohammad Abu Alsheikh, Dusit Niyato, Shaowei Lin, Hwee Pink Tan, Zhu Han

Research output: Contribution to journalArticle

76 Citations (Scopus)

Abstract

The proliferation of mobile devices, such as smartphones and Internet of Things gadgets, has resulted in the recent mobile big data era. Collecting mobile big data is unprofitable unless suitable analytics and learning methods are utilized to extract meaningful information and hidden patterns from data. This article presents an overview and brief tutorial on deep learning in mobile big data analytics and discusses a scalable learning framework over Apache Spark. Specifically, distributed deep learning is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall mobile, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.

Original languageEnglish
Article number7474340
Pages (from-to)22-29
Number of pages8
JournalIEEE Network
Volume30
Issue number3
DOIs
Publication statusPublished - 20 May 2016
Externally publishedYes

Fingerprint

Electric sparks
Smartphones
Mobile devices
Deep learning
Big data

Cite this

Abu Alsheikh, Mohammad ; Niyato, Dusit ; Lin, Shaowei ; Tan, Hwee Pink ; Han, Zhu. / Mobile big data analytics using deep learning and apache spark. In: IEEE Network. 2016 ; Vol. 30, No. 3. pp. 22-29.
@article{989e0897029b4460aba555d7f86a3f58,
title = "Mobile big data analytics using deep learning and apache spark",
abstract = "The proliferation of mobile devices, such as smartphones and Internet of Things gadgets, has resulted in the recent mobile big data era. Collecting mobile big data is unprofitable unless suitable analytics and learning methods are utilized to extract meaningful information and hidden patterns from data. This article presents an overview and brief tutorial on deep learning in mobile big data analytics and discusses a scalable learning framework over Apache Spark. Specifically, distributed deep learning is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall mobile, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.",
keywords = "Distributed deep learning, big data, Internet of things, context-awareness",
author = "{Abu Alsheikh}, Mohammad and Dusit Niyato and Shaowei Lin and Tan, {Hwee Pink} and Zhu Han",
year = "2016",
month = "5",
day = "20",
doi = "10.1109/MNET.2016.7474340",
language = "English",
volume = "30",
pages = "22--29",
journal = "IEEE Network",
issn = "0890-8044",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "3",

}

Mobile big data analytics using deep learning and apache spark. / Abu Alsheikh, Mohammad; Niyato, Dusit; Lin, Shaowei; Tan, Hwee Pink; Han, Zhu.

In: IEEE Network, Vol. 30, No. 3, 7474340, 20.05.2016, p. 22-29.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Mobile big data analytics using deep learning and apache spark

AU - Abu Alsheikh, Mohammad

AU - Niyato, Dusit

AU - Lin, Shaowei

AU - Tan, Hwee Pink

AU - Han, Zhu

PY - 2016/5/20

Y1 - 2016/5/20

N2 - The proliferation of mobile devices, such as smartphones and Internet of Things gadgets, has resulted in the recent mobile big data era. Collecting mobile big data is unprofitable unless suitable analytics and learning methods are utilized to extract meaningful information and hidden patterns from data. This article presents an overview and brief tutorial on deep learning in mobile big data analytics and discusses a scalable learning framework over Apache Spark. Specifically, distributed deep learning is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall mobile, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.

AB - The proliferation of mobile devices, such as smartphones and Internet of Things gadgets, has resulted in the recent mobile big data era. Collecting mobile big data is unprofitable unless suitable analytics and learning methods are utilized to extract meaningful information and hidden patterns from data. This article presents an overview and brief tutorial on deep learning in mobile big data analytics and discusses a scalable learning framework over Apache Spark. Specifically, distributed deep learning is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall mobile, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.

KW - Distributed deep learning

KW - big data

KW - Internet of things

KW - context-awareness

UR - http://www.scopus.com/inward/record.url?scp=84971373627&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/mobile-big-data-analytics-using-deep-learning-apache-spark

U2 - 10.1109/MNET.2016.7474340

DO - 10.1109/MNET.2016.7474340

M3 - Article

VL - 30

SP - 22

EP - 29

JO - IEEE Network

JF - IEEE Network

SN - 0890-8044

IS - 3

M1 - 7474340

ER -