Semantic Annotation and Ontologies for the TerraSAR-X Image Products

Kurzfassung

In this article, we propose a new methodology used for annotating TerraSAR-X products in the data base of the PDGS. Because manual annotation is very difficult and inefficient, we propose a semi-automated procedure in order to annotate TerraSAR-X datasets. Since for high resolution images, pixel-based methods do not capture the contextual information (complex structures are usually a mixture of regions and cover many pixels; different distributions of the same object can have different semantic meanings), and the global features describing the overall properties of images are not accurate enough. Therefore, the general approach adopted is to tile TerraSAR-X images into a number of no-overlapping sub-images (called patches) and to perform the feature extraction associated with these patches. In order to annotate the test dataset the following steps shall be applied to each product: 1) Group the scenes (products) in collections. 2) Select the optimal SAR image descriptors for given resolution, pixel spacing, optimal patch size, incidence angle, and orbit direction that corresponds to the basic detected TerraSAR-X products. 3) Tile the product-image into patches. 4) Generate a quick-look (in “jpg” format without rescaling the data) of each patch and also a quick-look of the full image needed for visualization. 5) Compute the descriptors (primitive features) associated with each patch. Gabor filters are used as primitive features generators computing 4 scale and 6 orientations that gives us feature vectors of 48 components (computing the mean and variance of each scale and orientation). 6) Use a classifier in order to group the features into categories. A Support Vector Machine (SVM) with a relevance feedback (RF) was built. The SVM-RF tool supports users to search patches (the quick-look of these patches) of interest in a large repository having as a support the full image. This classifier is selected based on the state-of-the-art. The performances of this learning machine/classifier, reported in the literature, are very good and the kernel has the capacity to perform highly accurate classification using a very limited number of examples. A new Graphical User Interface (GUI) of this tool allows Human-Machine Interaction (HMI) to rank the automatically suggested images which are expected to be grouped in the class of relevance. The new visual support of the tool allows enhancing the quality of search results by giving positive and negative examples directly for a full image. 7) Annotate semantically each category using as visual support the ground truth of Google Earth and give an appropriate meaning to each category. After an appropriate label is found for each category the quick-looks that are belonging to this category are moved from the dataset into a folder bearing the name of the category. Once the generation of the category is finished, a new classification can start until all the patches from the dataset are annotated. The total number of scenes annotated until now amounts to 82 products, ca. 100 000 image patches for which 688 semantic categories were defined based on the hierarchical scheme. A 2 layers taxonomy is introduced in support of definition of an ontology for common understanding of the TerraSAR-X image products. The algorithms and tools are now under integration process in the TerraSAR-X PDGS for the generation of a new semantic catalogue.