TY - JOUR
T1 - Fast Adaptation of Activity Sensing Policies in Mobile Devices
AU - Abu Alsheikh, Mohammad
AU - Niyato, Dusit
AU - Lin, Shaowei
AU - Tan, Hwee Pink
AU - Kim, Dong In
PY - 2017/11/15
Y1 - 2017/11/15
N2 - With the proliferation of sensors, such as accelerometers, in mobile devices, activity and motion tracking has become a viable technology to understand and create an engaging user experience. This paper proposes a fast adaptation and learning scheme of activity tracking policies when user statistics are unknown a priori, varying with time, and inconsistent for different users. In our stochastic optimization, user activities are required to be synchronized with a backend under a cellular data limit to avoid overcharges from cellular operators. The mobile device is charged intermittently using wireless or wired charging for receiving the required energy for transmission and sensing operations. First, we propose an activity tracking policy by formulating a stochastic optimization as a constrained Markov decision process (CMDP). Second, we prove that the optimal policy of the CMDP has a threshold structure using a Lagrangian relaxation approach and the submodularity concept. We accordingly present a fast Q-learning algorithm by considering the policy structure to improve the convergence speed over that of conventional Q-learning. Finally, simulation examples are presented to support the theoretical findings of this paper.
AB - With the proliferation of sensors, such as accelerometers, in mobile devices, activity and motion tracking has become a viable technology to understand and create an engaging user experience. This paper proposes a fast adaptation and learning scheme of activity tracking policies when user statistics are unknown a priori, varying with time, and inconsistent for different users. In our stochastic optimization, user activities are required to be synchronized with a backend under a cellular data limit to avoid overcharges from cellular operators. The mobile device is charged intermittently using wireless or wired charging for receiving the required energy for transmission and sensing operations. First, we propose an activity tracking policy by formulating a stochastic optimization as a constrained Markov decision process (CMDP). Second, we prove that the optimal policy of the CMDP has a threshold structure using a Lagrangian relaxation approach and the submodularity concept. We accordingly present a fast Q-learning algorithm by considering the policy structure to improve the convergence speed over that of conventional Q-learning. Finally, simulation examples are presented to support the theoretical findings of this paper.
KW - Activity tracking
KW - fast adaptation
KW - Internet of things
KW - Markov decision processes
KW - wireless charging
UR - http://www.scopus.com/inward/record.url?scp=85029611760&partnerID=8YFLogxK
UR - http://www.mendeley.com/research/fast-adaptation-activity-sensing-policies-mobile-devices
U2 - 10.1109/TVT.2016.2628966
DO - 10.1109/TVT.2016.2628966
M3 - Article
AN - SCOPUS:85029611760
SN - 0018-9545
VL - 66
SP - 5995
EP - 6008
JO - IEEE Transactions on Vehicular Technology
JF - IEEE Transactions on Vehicular Technology
IS - 7
M1 - 7744681
ER -