Abstract
In this paper we present a simple, novel method to use state-of-the-art image concept detectors and publicly available image search engines to retrieve images for semantically more complex queries from local databases without re-indexing of the database. Our low-key, data-driven method for associative recognition of unknown, or more elaborate, concepts in images allows user selection of visual examples to tailor query results to the typical preferences of the user. The method is compared with a baseline approach using ConceptNet-based semantic expansion of the query phrase to known concepts, as set by the concepts of the image concept detectors. Using the output of the image concept detector as index for all images in the local image database, a quick nearest-neighbor matching scheme is presented that can match queries swiftly via concept output vectors. We show preliminary results for a number of query phrases followed by a general discussion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
de Boer, M.H.T., Daniele, L., Brandt, P., Sappelli, M.: Applying semantic reasoning in image retrieval. In: ALLDATA 2015, The First International Conference on Big Data, Small Data, Linked Data and Open Data, pp. 69–74. IARIA (2015)
Bouma, H., Eendebak, P.T., Schutte, K., Azzopardi, G., Burghouts, G.J.: Incremental concept learning with few training examples and hierarchical classification. In: Proc. SPIE, vol. 9652 (2015)
Chatfield, K., Zisserman, A.: VISOR: towards on-the-fly large-scale object category retrieval. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part II. LNCS, vol. 7725, pp. 432–446. Springer, Heidelberg (2013)
Chen, X., Shrivastava, A., Gupta, A.: Neil: Extracting visual knowledge from web data. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1409–1416. IEEE (2013)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009 (2009)
Fergus, R., Fei-Fei, L., Perona, P., Zisserman, A.: Learning object categories from google’s image search. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 2, pp. 1816–1823. IEEE (2005)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. arXiv preprint 1408.5093 (2014)
Li, L.J., Fei-Fei, L.: Optimol: automatic online picture collection via incremental model learning. International Journal of Computer Vision 88(2), 147–168 (2010)
Muja, M., Lowe, D.G.: Flann, fast library for approximate nearest neighbors (2009)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge (2014)
Schutte, K., Bouma, H., Schavemaker, J., Daniele, L., Sappelli, M., Koot, G., Eendebak, P., Azzopardi, G., Spitters, M., de Boer, M., Brandt, P.: Interactive detection of incrementally learned concepts in images with ranking and semantic query interpretation. In: Proc. of 13th International Workshop on Content-Based Multimedia Indexing (CBMI) (2015)
Shi, Z., Yang, Y., Hospedales, T.M., Xiang, T.: Weakly supervised learning of objects, attributes and their associations. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 472–487. Springer, Heidelberg (2014)
Snoek, C.G.M., Worring, M., Koelma, D.C., Arnold, W.M., Smeulders, M.: A learned lexicon-driven paradigm for interactive video retrieval. IEEE Transactions on Multimedia 9(2) (2007)
Speer, R., Havasi, C.: Representing general relational knowledge in ConceptNet 5. In: LREC, pp. 3679–3686 (2012)
Torralba, A., Fergus, R., Freeman, W.T.: 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(11), 1958–1970 (2008)
Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: A neural image caption generator. CoRR abs/1411.4555 (2014). http://arxiv.org/abs/1411.4555
Wang, X.J., Zhang, L., Jing, F., Ma, W.Y.: Annosearch: Image auto-annotation by search. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1483–1490. IEEE (2006)
Zhang, R., Zhang, Z., Li, M., Ma, W.Y., Zhang, H.J.: A probabilistic semantic model for image annotation and multimodal image retrieval. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 846–851. IEEE (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Schavemaker, J., Spitters, M., Koot, G., de Boer, M. (2015). Fast Re-ranking of Visual Search Results by Example Selection. In: Azzopardi, G., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2015. Lecture Notes in Computer Science(), vol 9256. Springer, Cham. https://doi.org/10.1007/978-3-319-23192-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-23192-1_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23191-4
Online ISBN: 978-3-319-23192-1
eBook Packages: Computer ScienceComputer Science (R0)