A Transformer Based Approach for Activity Detection

Gulshan Sharma; Abhinav Dhall; Ramanathan Subramanian

doi:10.1145/3503161.3551598

A Transformer Based Approach for Activity Detection

Gulshan Sharma, Abhinav Dhall, Ramanathan Subramanian

Information Systems

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

5 Citations (Scopus)

Abstract

Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

Original language	English
Title of host publication	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia
Publisher	Association for Computing Machinery, Inc
Pages	7155-7159
Number of pages	5
ISBN (Electronic)	9781450392037
DOIs	https://doi.org/10.1145/3503161.3551598
Publication status	Published - 10 Oct 2022
Event	30th ACM International Conference on Multimedia, MM 2022 - Lisboa, Portugal Duration: 10 Oct 2022 → 14 Oct 2022

Publication series

Name	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

Conference

Conference	30th ACM International Conference on Multimedia, MM 2022
Abbreviated title	ACM MM
Country/Territory	Portugal
City	Lisboa
Period	10/10/22 → 14/10/22

Access to Document

10.1145/3503161.3551598

Cite this

@inproceedings{f1dfa5e345924ca9956b8ac3f648f428,

title = "A Transformer Based Approach for Activity Detection",

abstract = "Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.",

keywords = "cnn, human activity recognition, transformers",

author = "Gulshan Sharma and Abhinav Dhall and Ramanathan Subramanian",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 30th ACM International Conference on Multimedia, MM 2022, ACM MM ; Conference date: 10-10-2022 Through 14-10-2022",

year = "2022",

month = oct,

day = "10",

doi = "10.1145/3503161.3551598",

language = "English",

series = "MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia",

publisher = "Association for Computing Machinery, Inc",

pages = "7155--7159",

booktitle = "MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia",

}

Sharma, G, Dhall, A & Subramanian, R 2022, A Transformer Based Approach for Activity Detection. in MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia. MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, pp. 7155-7159, 30th ACM International Conference on Multimedia, MM 2022, Lisboa, Portugal, 10/10/22. https://doi.org/10.1145/3503161.3551598

A Transformer Based Approach for Activity Detection. / Sharma, Gulshan; Dhall, Abhinav; Subramanian, Ramanathan.
MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia. Association for Computing Machinery, Inc, 2022. p. 7155-7159 (MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia).

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

TY - GEN

T1 - A Transformer Based Approach for Activity Detection

AU - Sharma, Gulshan

AU - Dhall, Abhinav

AU - Subramanian, Ramanathan

PY - 2022/10/10

Y1 - 2022/10/10

N2 - Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

AB - Non-invasive physiological sensors allow for the collection of user-specific data in realistic environments. In this paper, using physiological data, we investigate the effectiveness of Convolutional Neural Network (CNN) based feature embeddings and Transformer architecture for the human activity recognition task. 1D-CNN representation is used for the heart rate, and 2D-CNN is used for short-term Fourier transformation of the accelerometer data. Post fusion, the feature is input into a transformer. The experiments are performed on the harAGE dataset. The findings indicate the discriminative ability of the feature-fusion on transformer-based architecture, and the method outperforms the harAGE baseline by an absolute 3.7%.

KW - cnn

KW - human activity recognition

KW - transformers

UR - http://www.scopus.com/inward/record.url?scp=85151158328&partnerID=8YFLogxK

U2 - 10.1145/3503161.3551598

DO - 10.1145/3503161.3551598

M3 - Conference contribution

AN - SCOPUS:85151158328

T3 - MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

SP - 7155

EP - 7159

BT - MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

PB - Association for Computing Machinery, Inc

T2 - 30th ACM International Conference on Multimedia, MM 2022

Y2 - 10 October 2022 through 14 October 2022

ER -

A Transformer Based Approach for Activity Detection

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this