Multitask linear discriminant analysis for view invariant action recognition

Yan Yan, Elisa Ricci, Ramanathan Subramanian, Gaowen Liu, Nicu Sebe

Research output: Contribution to journalArticlepeer-review

147 Citations (Scopus)


Robust action recognition under viewpoint changes has received considerable attention recently. To this end, self-similarity matrices (SSMs) have been found to be effective view-invariant action descriptors. To enhance the performance of SSM-based methods, we propose multitask linear discriminant analysis (LDA), a novel multitask learning framework for multiview action recognition that allows for the sharing of discriminative SSM features among different views (i.e., tasks). Inspired by the mathematical connection between multivariate linear regression and LDA, we model multitask multiclass LDA as a single optimization problem by choosing an appropriate class indicator matrix. In particular, we propose two variants of graph-guided multitask LDA: 1) where the graph weights specifying view dependencies are fixed a priori and 2) where graph weights are flexibly learnt from the training data. We evaluate the proposed methods extensively on multiview RGB and RGBD video data sets, and experimental results confirm that the proposed approaches compare favorably with the state-of-the-art.

Original languageEnglish
Article number6939719
Pages (from-to)5599-5611
Number of pages13
JournalIEEE Transactions on Image Processing
Issue number12
Publication statusPublished - 1 Dec 2014
Externally publishedYes


Dive into the research topics of 'Multitask linear discriminant analysis for view invariant action recognition'. Together they form a unique fingerprint.

Cite this