Deriving semantics for image clustering from accumulated user feedbacks

Image clustering solely based on visual features without any knowledge or background information suffers from the problem of semantic gap. In this paper, we propose SS-NMF: a semi-supervised non-negative matrix factorization framework for image clustering. Accumulated relevance feedback in a CBIR system is treated as user provided supervision for guiding the image clustering. We consider the set of positive images in the feedback as constraints on the clustering specifying that the images "must" be clustered together. Similarly, negative images provide constraints specifying that they "cannot" be clustered along with the positive images. Through an iterative algorithm, we perform symmetric tri-factorization of the image-image similarity matrix to infer the clustering. Theoretically, we prove the correctness of SS-NMF by showing that the algorithm is guaranteed to converge. Through experiments conducted on general purpose image datasets, we demonstrate the superior performance of SS-NMF for clustering images effectively.