A smart fusion framework for multimodal object, activity and event detection

Girija CHETTY, Mohammad Yamin

Research output: A Conference proceeding or a Chapter in BookConference contribution

Abstract

With an increasing diffusion of wearable technologies and mobile sensor systems, along with entrenchment of social media networks and crowdsourced information systems in every aspect of modern society, an unavoidable reality is that of continuous, pervasive and ubiquitous sensing, monitoring, surveillance and detection of every type of object, activity, event and incident at a global scale. This rapid proliferation has provided immense opportunities to make use of comprehensive information from a diverse array of multimodal, multi-view, and multisensory data streams for developing efficient and robust, automated computer based decision support systems. Further, with the availability of the complementary and the supplementary information in terms of auxiliary meta-data from the social networks, human experts and the crowdsourced communities, it is possible to obtain better actionable intelligence from these systems. In this paper, we propose a novel computational framework for addressing this gap. The proposed smart fusion framework with particular focus on combining heterogeneous, multimodal real-time big data streams-with information from different types of sensor and auxiliary information drawn from human experts and opinion scores in the loop, allows synergistic fusion to be achieved, leading to better actionable intelligence from the computer based decision support systems. The details of this framework implementation with a component based software platform-the msifStudio, and its evaluation for some of the use case application scenarios is presented here.

Original languageEnglish
Title of host publicationProceedings of the 10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development, INDIACom 2016
EditorsM. N. Hoda
Place of PublicationNew Delhi, India
PublisherIEEE
Pages1417-1422
Number of pages6
Volume1
ISBN (Electronic)9789380544199
ISBN (Print)9789380544199
Publication statusPublished - 2016
Event10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development - New Delhi, New Delhi, India
Duration: 16 Mar 201618 Mar 2016

Publication series

NameProceedings of the 10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development, INDIACom 2016

Conference

Conference10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development
Abbreviated titleINDIACom
CountryIndia
CityNew Delhi
Period16/03/1618/03/16

Fingerprint Dive into the research topics of 'A smart fusion framework for multimodal object, activity and event detection'. Together they form a unique fingerprint.

  • Cite this

    CHETTY, G., & Yamin, M. (2016). A smart fusion framework for multimodal object, activity and event detection. In M. N. Hoda (Ed.), Proceedings of the 10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development, INDIACom 2016 (Vol. 1, pp. 1417-1422). [7724498] (Proceedings of the 10th INDIACom; 2016 3rd International Conference on Computing for Sustainable Global Development, INDIACom 2016). IEEE.