Integrated Representation and Discrimination Models for Functional Data Classification

Abstract

The modus operandi for machine learning is to map functional data to numerical summaries, filter the data, and/or subject it to global signature extractions with the objecive of building robust feature vectors that uniquely characterize each function and then proceed with training algorithms that seek to optimally partition the feature space S ⊂ Rn into labeled regions. This holds true even when the original data are functional in nature, i.e. curves or surfaces that inherently vary over a continuum such as time or space. Functional data are often reduced to summary statistics, locally-sensitive features, and global signatures with the objective of building comprehensive feature vectors that uniquely characterize each function. This dissertation directly addresses representational issues of functional data for supervised learning. We propose novel frameworks for joint representation and discrimination of functional data, where rather than stripping the data of their functional attributes to make feature vectors, we instead use the basis representations of the data to enhance the inherent discriminative information. We first propose the Classification by Discriminative Interpolation (CDI) framework wherein functional data in the same class are adaptively reconstructed to be more similar to each other, while nearest neighbor functional data in other classes are simultaneously repelled. We then extend the CDI framework to further leverage the functional characteristics of the data by applying CDI to different feature function combinations which we term Classification by Discriminative Interpolation with Features (CDIF). Akin to other recent nearest neighbor metric learning paradigms like stochastic k-neighborhood selection and large margin nearest neighbors, both CDI and CDIF use class-specific representations which gerrymander similar functional data in an appropriate parameter space. In our third proposed supervised learning approach, time series in the same class are adaptively reconstructed to be more similar to each other, as done in CDI, but unlike CDI, the optimal Support Vector Machine (SVM) hyperplane that separates the basis expansions of the data is sought simultaneously. This method is termed Classification by Discriminative Reconstruction (CDR). The methodology of CDR explores the use of Wavelets and Radial Basis functions representations, which are further pruned through the use of sparsity inducing norm penalties. Experimental validation on several time series datasets, provided by the UCR Time Series Classification repository, establish the proposed discriminative interpolation frameworks as competitive or better in comparison to recent state-of-the-art techniques which continue to rely on the standard feature vector representation.