Abstract
Classification of diagnose images and illustrations in the literature is a major challenge towards automated literature review and retrieval. Although being widely recognized as the most successful image classification technique, deep learning models, however, may need to be complemented by traditional visual features to solve this problem, in which there are intra-class variation, inter-class similarity and a small training dataset. In this paper, we propose an approach to classifying diagnose images and biomedical publication illustrations. This algorithm jointly uses the image representations learned by three pre-trained deep convolutional neural network models and ten types of traditional visual features in a stacked support vector machine (SVM) classifier. We have evaluated this algorithm on the ImageCLEF 2016 Subfigure Classification dataset and achieved an accuracy of 85.62%, which is higher than the top performance of purely visual approaches in this challenge.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Liverman, C.T., Fulco, C.E., Kipen, H.M.: Internet access to the national library of medicine’s toxicology and environmental health databases. National Academies Press (US), Washington, D.C (1998). doi:10.17226/6327
Cirujeda, P., Binefa, X.: Medical image classification via 2D color feature based covariance descriptors. In: CLEF (Working Notes) (2015)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference Computer Vision and Pattern Recognition, pp. 2169–2178 (2006)
Kitanovski, I., Dimitrovski, I., Loskovska, S.: FCSE at medical tasks of ImageCLEF 2013. In: CLEF (Working Notes) (2013)
Abedini, M., Cao, L., Codella, N., et al.: IBM research at ImageCLEF 2013 medical tasks. In: American Medical Informatics Association (AMIA) ImageCLEF, Medical Image Retrieval Workshop (2013)
Unay, D., Ekin, A.: Intensity versus texture for medical image search and retrieval. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro (2008)
Valavanis, L., Stathpoulos, S., Kalamboukis, T.: IPL at CLEF 2016 medical task. In: CLEF (Working Notes) (2016)
Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)
Dandil, E., et al.: Artificial neural network-based classification system for lung nodules on computed tomography scans. In: 2014 6th International Conference of Soft Computing and Pattern Recognition (2014)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Kumar, A., et al.: Subfigure and multi–label classification using a fine–tuned convolutional neural network. In: CLEF2016 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Évora (2016)
Lopez, A.R., et al.: Skin lesion classification from dermoscopic images using deep learning techniques. In: 2017 13th IASTED International Conference on Biomedical Engineering, pp. 40–54 (2017)
Koitka, S., Friedrich, C.M.: Traditional feature engineering and deep learning approaches at medical classification task of ImageCLEF 2016. In: CLEF2016 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Évora (2016)
Song, Y., et al.: Large margin local estimate with applications to medical image classification. IEEE Trans. Med. Imaging 34(6), 1362–1377 (2015)
Zhang, C., Bengio, S., et al.: Understanding deep learning requires rethinking generalization. In: International Conference on Learning Representations, Toulon France (2017)
Donahue, J., Jia, Y., Vinyals, O., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. Int. Conf. Mach. Learn. 50, 815–830 (2014)
Müller, H., Kalpathy-cramer, J., Demner-Fushman, D., et al.: Creating a classification of image types in the medical literature for visual categorization. In: SPIE Medical Imaging. International Society for Optics and Photonics, pp. 75–84 (2012)
Gilbert, A., Piras, L., Wang, J., et al.: Overview of the ImageCLEF 2016 scalable concept image annotation task. In: Conference and Labs of the Evaluation Forums (2016)
Chen, C.: Computer Vision in Medical Imaging, vol. 2. World Scientific, Singapore (2014)
Cula, O.G., Dana, K.J.: Compact representation of bidirectional texture functions. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, I-1041–I-1047 (2001)
Novak, C.L., Shafer, S.A.: Anatomy of a color histogram. In: 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 599–605 (2002)
Guo, D., Song, Z.: A study on texture image classifying based on gray-level co-occurrence matrix. For. Mach. Woodworking Equip. 7, 007 (2005)
Ojala, T., PietikaInen, M., et al.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)
Fogel, I., Sagi, D.: Gabor filters as texture discriminator. Biol. Cybern. 61(2), 103–113 (1989)
Wu, J., Rehg, J.M.: CENTRIST: a visual descriptor for scene categorization. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1489–1501 (2011)
Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: ACM International Conference on Image and Video Retrieval, pp. 401–408 (2007). ACM
Sikora, T.: The MPEG-7 visual standard for content description-an overview. IEEE Trans. Circ. Syst. Video Technol. 11(6), 696–702 (2001)
Huang, J., Kumar, S.R., Mitra, M., et al.: Image indexing using color correlograms. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p. 762 (1997)
Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 312–322. Springer, Heidelberg (2008). doi:10.1007/978-3-540-79547-6_30
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Keys, R.G.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Sig. Process. 29(6), 1153–1160 (1981)
Dunteman, G.H.: Principal Components Analysis. Sage, Thousand Oaks (1989)
van der Maaten, L., Postma, E., van den Herik, J.: Dimensionality reduction: a comparative review. J. Mach. Learn. Res. 10(1), 66–71 (2007)
He, X., Niyogi, P.: Locality preserving projections. In: Advances in Neural Information Processing Systems (2004)
Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. Adv. Neural. Inf. Process. Syst. 17, 777–784 (2004)
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China under Grants 61471297 and 61771397, and in part by the Seed Foundation of Innovation and Creation for Graduate Students in Northwestern Polytechnical University. We appreciate the efforts devoted by the organizers of the ImageCLEF2016 Medical Image Classification Challenge to collect and share the data for comparing algorithms of classifying diagnose images and illustrations in the biomedical literature.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wang, H., Zhang, J., Xia, Y. (2017). Jointly Using Deep Model Learned Features and Traditional Visual Features in a Stacked SVM for Medical Subfigure Classification. In: Sun, Y., Lu, H., Zhang, L., Yang, J., Huang, H. (eds) Intelligence Science and Big Data Engineering. IScIDE 2017. Lecture Notes in Computer Science(), vol 10559. Springer, Cham. https://doi.org/10.1007/978-3-319-67777-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-67777-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67776-7
Online ISBN: 978-3-319-67777-4
eBook Packages: Computer ScienceComputer Science (R0)