Markov decision processes with applications in wireless sensor networks

A survey

Mohammad Abu Alsheikh, Dinh Thai Hoang, Dusit Niyato, Hwee Pink Tan, Shaowei Lin

Research output: Contribution to journalArticle

67 Citations (Scopus)

Abstract

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are used to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs.

Original languageEnglish
Article number7080987
Pages (from-to)1239-1267
Number of pages29
JournalIEEE Communications Surveys and Tutorials
Volume17
Issue number3
DOIs
Publication statusPublished - 1 Jul 2015
Externally publishedYes

Fingerprint

Wireless sensor networks
Stochastic systems
Electronic data interchange
Adaptive algorithms
Sensor nodes
Decision making
Topology
Network protocols
Costs

Cite this

Abu Alsheikh, Mohammad ; Hoang, Dinh Thai ; Niyato, Dusit ; Tan, Hwee Pink ; Lin, Shaowei. / Markov decision processes with applications in wireless sensor networks : A survey. In: IEEE Communications Surveys and Tutorials. 2015 ; Vol. 17, No. 3. pp. 1239-1267.
@article{b985f12ec7774f59b9c2b2453db66fad,
title = "Markov decision processes with applications in wireless sensor networks: A survey",
abstract = "Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are used to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs.",
keywords = "decision making tools, Markov decision processes (MDPs), multi-agent systems, optimization methods, stochastic control, Wireless sensor networks",
author = "{Abu Alsheikh}, Mohammad and Hoang, {Dinh Thai} and Dusit Niyato and Tan, {Hwee Pink} and Shaowei Lin",
year = "2015",
month = "7",
day = "1",
doi = "10.1109/COMST.2015.2420686",
language = "English",
volume = "17",
pages = "1239--1267",
journal = "IEEE Communications Surveys and Tutorials",
issn = "1553-877X",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "3",

}

Markov decision processes with applications in wireless sensor networks : A survey. / Abu Alsheikh, Mohammad; Hoang, Dinh Thai; Niyato, Dusit; Tan, Hwee Pink; Lin, Shaowei.

In: IEEE Communications Surveys and Tutorials, Vol. 17, No. 3, 7080987, 01.07.2015, p. 1239-1267.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Markov decision processes with applications in wireless sensor networks

T2 - A survey

AU - Abu Alsheikh, Mohammad

AU - Hoang, Dinh Thai

AU - Niyato, Dusit

AU - Tan, Hwee Pink

AU - Lin, Shaowei

PY - 2015/7/1

Y1 - 2015/7/1

N2 - Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are used to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs.

AB - Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are used to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs.

KW - decision making tools

KW - Markov decision processes (MDPs)

KW - multi-agent systems

KW - optimization methods

KW - stochastic control

KW - Wireless sensor networks

UR - http://www.scopus.com/inward/record.url?scp=84940107302&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/markov-decision-processes-applications-wireless-sensor-networks-survey

U2 - 10.1109/COMST.2015.2420686

DO - 10.1109/COMST.2015.2420686

M3 - Article

VL - 17

SP - 1239

EP - 1267

JO - IEEE Communications Surveys and Tutorials

JF - IEEE Communications Surveys and Tutorials

SN - 1553-877X

IS - 3

M1 - 7080987

ER -