Simultaneous feature selection and Gaussian mixture model estimation for supervised classification problems

Abstract

Abstract: Using mixture models to represent univariate and multivariate data has shown to be a very flexible and powerful tool in several areas, for example computer vision, machine learning, pattern recognition and remote sensing. The expectation maximization (EM) algorithm is a widely acknowledged statistical approach in generative model estimation. In this article, a new EM-algorithm for time-critical supervised classification of aerial imagery is proposed. Compared to standard EM and other approaches, the proposed method has the following advantages: 1) No knowledge about the distribution of each thematical class is needed. 2) The number of components for each class is estimated. 3) The algorithm does not require careful initialization. 4) Singular estimates are avoided due to the ability of pruning components. 5) The features that best discriminate between the classes are identified simultaneously. 6) The relevant features are identified by incorporating the separability of the classes in the feature space domain via Mahalanobis distances. Three experiments using artificial and real datasets are carried out in order to demonstrate the relevance and quality of the results obtained by the proposed method. The main findings are: 1) Feature selection is a very important task in terms of prediction quality of models. 2) In the examined experiments the proposed method estimates better models, in terms of classification results and further measurements, than other state-of-the-art methods, e.g. Random swap EM. 3) The Incorporation of Mahalanobis distances is very valuable for the identification of relevant features. 4) The proposed method is more robust than the compared methods. 5) In case of complex data distibutions, the new approach is able to provide better results than the compared methods.