Abstract
Chinese calligraphic images have important artistic and historical values. However, subjected to hundreds of years of natural weathering, corrosion and man-made destruction, Chinese calligraphic images inevitably contain some special noise, such as dotted noise, flake corrosion noise and scratch noise. How to denoise this special noise is a challenge for digital preservation of Chinese calligraphic. In this paper, we propose an end-to-end calligraphic image denoising algorithm based on a well-designed generative adversarial network. The generator contains a recurrent network and a denoising autoencoder. By introducing an attention mechanism, we use a recurrent network with multiple progressive network units to generate a noise attention map. Through the noise attention map, the denoising autoencoder can restore the noisy calligraphic image into a clean image with reduced noise or even no noise. The extensive experiments results show that the results of our method are better than those of other comparison methods in terms of visual effects, PSNR and SSIM.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Buades, A., Coll, B., Morel, J.M.: Non-local means denoising. Image Process. Line 1, 208–212 (2011). https://doi.org/10.5201/ipol.2011.bcm_nlm
Cao, S., Song, B.: Visual attentional-driven deep learning method for flower recognition. Math. Biosci. Eng. MBE 18(3), 1981–1991 (2021). https://doi.org/10.3934/mbe.2021103
Chen, J., Benesty, J., Huang, Y., Doclo, S.: New insights into the noise reduction Wiener filter. IEEE Trans. Audio Speech Lang. Process. 14(4), 1218–1234 (2006). https://doi.org/10.1109/TSA.2005.860851
Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3155–3164 (2018).
Chen, R., Yu, W., Wang, R., Liu, G., Shao, Y.: Interferometric phase denoising by pyramid nonlocal means filter. IEEE Geosci. Remote Sens. Lett. 10(4), 826–830 (2013). https://doi.org/10.1109/LGRS.2012.2225594
Ding, B., Long, C., Zhang, L., Xiao, C.: ARGAN: Attentive recurrent generative adversarial network for shadow detection and removal. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 10213–10222 (2019).
Friebe, M., Kaup, A.: Low complexity two classes gauss weighting filter for noise reduction in mobile received analog tv signals. In: 2005 13th European signal processing conference, IEEE pp. 1–4 (2005).
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., Lu, H.: Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3146–3154 (2019).
Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. IEEE Trans. Neural Netw. Learn. Syst. 31(6), 1794–1807 (2019). https://doi.org/10.1109/TNNLS.2019.2926481
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural information processing systems, 27 (2014).
Guo, H., Zheng, K., Fan, X., Yu, H., Wang, S.: Visual attention consistency under image transforms for multi-label image classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 729–739 (2019).
Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1712–1722, (2019).
Huang, T., Yang, G.J.T.G.Y., Tang, G.: A fast two-dimensional median filtering algorithm. IEEE Trans. Acoust. Speech Signal Process. 27(1), 13–18 (1979). https://doi.org/10.1109/TASSP.1979.1163188
Kandemir, C., Kalyoncu, C., Toygar, Ö.: A weighted mean filter with spatial-bias elimination for impulse noise removal. Digit. Signal Process. 46, 164–174 (2015). https://doi.org/10.1016/j.dsp.2015.08.012
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690 (2017).
Li, R., Pan, J., Li, Z., Tang, J.: Single image dehazing via conditional generative adversarial network. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 8202–8211 (2018).
Liu, X.W., Liu, C.Y.: An optional gauss filter image denoising method based on difference image fast fuzzy clustering. Appl. Mech. Mater. 411, 1348–1352 (2013). https://doi.org/10.4028/www.scientific.net/AMM.411-414.1348
Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural Inf. Process. Syst. 29, 2802–2810 (2016)
Qin, Y., Jia, L., Feng, J., An, M., Diao, L.: Proceedings of the 2015 International Conference on Electrical and Information Technologies for Rail Transportation: Transportation, vol. 378. Springer, Berlin (2016)
Shi, Z., Xu, B., Zheng, X., Zhao, M.: A Chinese character structure preserved denoising method for Chinese tablet calligraphy document images based on KSVD dictionary learning. Multimed. Tools Appl. 76(13), 14921–14936 (2017). https://doi.org/10.1007/s11042-016-4284-3
Shi, Z., Xu, B., Zheng, X., Zhao, M.: An integrated method for ancient Chinese tablet images de-noising based on assemble of multiple image smoothing filters. Multimed. Tools Appl. 75(19), 12245–12261 (2016). https://doi.org/10.1007/s11042-016-3421-3
Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020). https://doi.org/10.1016/j.neunet.2019.12.024
Wang, C., Xu, C., Wang, C., Tao, D.: Perceptual adversarial networks for image-to-image transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018). https://doi.org/10.1109/TIP.2018.2836316
Wang, Q., Huang, Y., Jia, W., He, X., Blumenstein, M., Lyu, S., Lu, Y.: FACLSTM: ConvLSTM with focused attention for scene text recognition. Sci. China Inf. Sci. 63(2), 1–14 (2020). https://doi.org/10.1007/s11432-019-2713-1
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Change Loy, C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops, pp. 1–16 (2018).
Xiao, L., He, K., Zhou, J.L., Wu, D.: Image noise removal on improvement adaptive medium filter. Laser J. 30(2), 44–46 (2009)
Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 842–850 (2015).
Xingjian, S. H. I., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., Woo, W. C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp. 802–810, (2015).
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:https://arxiv.org/abs/1511.07122 (2015).
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2019). https://doi.org/10.1109/TCSVT.2019.2920407
Zhang, J., Guo, M., Fan, J.: A novel CNN structure for fine-grained classification of Chinese calligraphy styles. Int. J. Doc. Anal. Recogn. (IJDAR) 22(2), 177–188 (2019). https://doi.org/10.1007/s10032-019-00324-1
Zhang, J., Guo, M., Fan, J.: A novel generative adversarial net for calligraphic tablet images denoising. Multimed. Tools Appl. 79(1), 119–140 (2020). https://doi.org/10.1007/s11042-019-08052-8
Zhang, J.S., Yu, J.H., Mao, G.H., Ye, X.Z.: Denoising of Chinese calligraphy tablet images based on run-length statistics and structure characteristic of character strokes. J. Zhejiang Univ.-Sci. A 7(7), 1178–1186 (2006). https://doi.org/10.1631/jzus.2006.A1178
Zhang, K., Zuo, W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018). https://doi.org/10.1109/TIP.2018.2839891
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017). https://doi.org/10.1109/TIP.2017.2662206
Zheng, X., Miao, Q., Shi, Z., Fan, Y., Shui, W.: A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimed. Tools Appl. 75(14), 8719–8744 (2016). https://doi.org/10.1007/s11042-015-2788-x
Liu, Q., Zhang, C., Guo, Q., Xu, H., Zhou, Y.: Adaptive sparse coding on PCA dictionary for image denoising. Vis. Comput. 32(4), 535–549 (2016). https://doi.org/10.1007/s00371-015-1087-x
Jiang, X., Jin, Y., Yao, Y.: Low-dose CT lung images denoising based on multiscale parallel convolution neural network. Vis. Comput. 37(8), 2419–2431 (2021). https://doi.org/10.1007/s00371-020-01996-1
Zhang, L., Long, C., Zhang, X., Xiao, C.: Ris-gan: Explore residual and illumination with generative adversarial networks for shadow removal. Proc. AAAI Conf. Artif. Intell. 34(07), 12829–12836 (2020). https://doi.org/10.1609/aaai.v34i07.6979
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016).
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on medical image computing and computer-assisted intervention (pp. 234–241) (2015) doi: https://doi.org/10.1007/978-3-319-24574-4_28
Chu, J., Guo, Z., Leng, L.: Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access 6, 19959–19967 (2018). https://doi.org/10.1109/ACCESS.2018.2815149
Zhang, Y., Chu, J., Leng, L., Miao, J.: Mask-refined R-CNN: A network for refining object details in instance segmentation. Sensors 20(4), 1010 (2020). https://doi.org/10.3390/s20041010
Chen, Y., Liu, L., Tao, J., Xia, R., Zhang, Q., Yang, K., Xiong, J., Chen, X.: The improved image inpainting algorithm via encoder and similarity constraint. Vis. Comput. 37(7), 1691–1705 (2021). https://doi.org/10.1007/s00371-020-01932-3
Gou, Y., Li, B., Liu, Z., Yang, S., Peng, X.: CLEARER: Multi-scale neural architecture search for image restoration. Advances in Neural Information Processing Systems, 33 (2020).
Funding
This study was funded by the NSFC under Grant No.61972315.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
To the best of our knowledge, the named authors have no conflict of interest, financial or otherwise.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, X., Wu, K., Zhang, Y. et al. A GAN-based Denoising Method for Chinese Stele and Rubbing Calligraphic Image. Vis Comput 39, 1351–1362 (2023). https://doi.org/10.1007/s00371-022-02410-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02410-8