A Case Study for User Evaluation of a CBIR Tool: an Application of Open-Ended Feedback with Comment Clustering and Inductive Categorization

Kurzfassung

In the context of Earth Observation (EO), image information retrieval systems have gained importance as a way to explore terabytes of archive data. Concurrently, evaluation of these systems becomes a topic. Evaluation has typically been conducted in the form of metrics such as Precision Recall measures, with more recent approaches attempting to include the user in the evaluation process. This paper presents a more user centered evaluation of a CBIR tool in an EO context. The evaluation methodology involved open ended user feedback, which was then inductively categorized, and its distribution and content were analyzed. Results are presented, with conclusions indicating certain aspects of the user experience cannot be obtained from metrics alone, but can be complementary to metrics.