Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Frequency-constrained transferable adversarial attack on image manipulation detection and localization

  • Research
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Recent works have demonstrated the great performance of forgery image forensics based on deep learning, but there is still a risk that detectors could be susceptible to unknown illegal attacks, raising growing security concerns. This paper starts from the perspective of reverse forensics and explores the vulnerabilities of current image manipulation detectors to achieve targeted attacks. We present a novel reverse decision aggregate gradient attack under low-frequency constraints (RevAggAL). Specifically, we first propose a novel pixel reverse content decision-making (PRevCDm) loss to optimize perturbation generation with a specific principle more suitable for segmenting manipulated regions. Then, we introduce the low-frequency component to constrain the perturbation into more imperceptible details, significantly avoiding the degradation of image quality. We also consider aggregating gradients on model-agnostic features to enhance the transferability of adversarial examples in black-box scenarios. We evaluate the effectiveness of our method on three representative detectors (ResFCN, MVSSNet, and OSN) with five widely used forgery datasets (COVERAGE, COLUMBIA, CASIA1, NIST 2016, and Realistic Tampering). Experimental results show that our method improves the attack success rate (ASR) while ensuring better image quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

No datasets were generated or analyzed during the current study.

References

  1. Asnani, V., Yin, X., Hassner, T., Liu, S., Liu, X.: Proactive image manipulation detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15,386–15,395 (2022)

  2. Bammey, Q., Gioi, R.G.v., Morel, J.M.: An adaptive neural network for unsupervised mosaic consistency analysis in image forensics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14,194–14,204 (2020)

  3. Cai, Z., Tan, Y., Asif, M.S.: Ensemble-based blackbox attacks on dense prediction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4045–4055 (2023)

  4. Carlini, N., Farid, H.: Evading deepfake-image detectors with white-and black-box attacks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 658–659 (2020)

  5. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE symposium on security and privacy, pp. 39–57 (2017)

  6. Chen, X., Dong, C., Ji, J., Cao, J., Li, X.: Image manipulation detection by multi-view multi-scale supervision. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 14,185–14,193 (2021)

  7. Dong, C., Chen, X., Hu, R., Cao, J., Li, X.: MVSS-Net: multi-view multi-scale supervised networks for image manipulation detection. IEEE Trans. Pattern Anal. Mach. Intell. 45(3), 3539–3553 (2022)

    Google Scholar 

  8. Dong, J., Wang, W., Tan, T.: CASIA image tampering detection evaluation database. In: 2013 IEEE China summit and international conference on signal and information processing, pp. 422–426 (2013)

  9. Feng, Y., Chen, B., Dai, T., Xia, S.T.: Adversarial attack on deep product quantization network for image retrieval. In: Proceedings of the AAAI conference on artificial intelligence, 34, pp. 10,786–10,793 (2020)

  10. Fontani, M., Bianchi, T., De Rosa, A., Piva, A., Barni, M.: A framework for decision fusion in image forensics based on Dempster-Shafer theory of evidence. IEEE Trans. Inf. Forensics Secur. 8(4), 593–607 (2013)

    Article  Google Scholar 

  11. Fridrich, J., Kodovsky, J.: Rich models for steganalysis of digital images. IEEE Trans. Inf. Forensics Secur. 7(3), 868–882 (2012)

    Article  Google Scholar 

  12. Fridrich, J., Lukas, J., Goljan, M.: Digital camera identification from sensor noise. IEEE Trans. Inf. Secur. Forensics 1(2), 205–214 (2006)

    Article  Google Scholar 

  13. Gallagher, A.C., Chen, T.: Image authentication by detecting traces of demosaicing. In: 2008 IEEE computer society conference on computer vision and pattern recognition workshops, pp. 1–8 (2008)

  14. Gao, Z., Sun, C., Cheng, Z., Guan, W., Liu, A., Wang, M.: TBNet: a two-stream boundary-aware network for generic image manipulation localization. IEEE Trans. Knowl. Data Eng. (2022)

  15. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv:1412.6572 (2014)

  16. Guan, H., Kozak, M., Robertson, E., Lee, Y., Yates, A.N., Delgado, A., Zhou, D., Kheyrkhah, T., Smith, J., Fiscus, J.: MFC datasets: large-scale benchmark datasets for media forensic challenge evaluation. In: 2019 IEEE winter applications of computer vision workshops, pp. 63–72 (2019)

  17. Guillaro, F., Cozzolino, D., Sud, A., Dufour, N., Verdoliva, L.: TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20,606–20,615 (2023)

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)

  19. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local NASH equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017)

  20. Huang, H., Chen, Z., Chen, H., Wang, Y., Zhang, K.: T-SEA: Transfer-based self-ensemble attack on object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20,514–20,523 (2023)

  21. Jia, S., Ma, C., Yao, T., Yin, B., Ding, S., Yang, X.: Exploring frequency adversarial attacks for face forgery detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4103–4112 (2022)

  22. Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., Irani, M.: Imagic: Text-based real image editing with diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6007–6017 (2023)

  23. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)

  24. Korus, P., Huang, J.: Evaluation of random field models in multi-modal unsupervised tampering localization. In: 2016 IEEE international workshop on information forensics and security, pp. 1–6 (2016)

  25. Korus, P., Huang, J.: Multi-scale analysis strategies in PRNU-based tampering localization. IEEE Trans. Inf. Forensics Secur. 12(4), 809–824 (2016)

    Article  Google Scholar 

  26. Kwon, M.J., Nam, S.H., Yu, I.J., Lee, H.K., Kim, C.: Learning jpeg compression artifacts for image manipulation detection and localization. Int. J. Comput. Vision 130(8), 1875–1895 (2022)

    Article  Google Scholar 

  27. Li, B., Qi, X., Lukasiewicz, T., Torr, P.H.: ManiGAN: text-guided image manipulation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7880–7889 (2020)

  28. Li, D., Wang, W., Fan, H., Dong, J.: Exploring adversarial fake images on face manifold. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5789–5798 (2021)

  29. Li, H., Huang, J.: Localization of deep inpainting using high-pass fully convolutional network. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 8301–8310 (2019)

  30. Li, Q., Shen, L., Guo, S., Lai, Z.: Wavelet integrated CNNs for noise-robust image classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7245–7254 (2020)

  31. Liu, X., Liu, Y., Chen, J., Liu, X.: PSCC-Net: progressive spatio-channel correlation network for image manipulation detection and localization. IEEE Trans. Circuits Syst. Video Technol. 32(11), 7505–7517 (2022)

    Article  Google Scholar 

  32. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440 (2015)

  33. Lu, C.S., Liao, H.Y.M.: Structural digital signature for image authentication: an incidental distortion resistant scheme. In: Proceedings of the 2000 ACM workshops on multimedia, pp. 115–118 (2000)

  34. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083 (2017)

  35. Ng, T.T., Chang, S.F., Sun, Q.: A data set of authentic and spliced image blocks. Columbia University, ADVENT Technical Report 4 (2004)

  36. Nikolaidis, N., Pitas, I.: Copyright protection of images using robust digital signatures. In: 1996 IEEE international conference on acoustics, speech, and signal processing conference proceedings, 4, pp. 2168–2171 (1996)

  37. Park, T., Zhu, J.Y., Wang, O., Lu, J., Shechtman, E., Efros, A., Zhang, R.: Swapping autoencoder for deep image manipulation. Adv. Neural Inf. Process. Syst. 33, 7198–7211 (2020)

    Google Scholar 

  38. Qiu, X., Li, H., Luo, W., Huang, J.: A universal image forensic strategy based on steganalytic model. In: Proceedings of the 2nd ACM workshop on information hiding and multimedia security, pp. 165–170 (2014)

  39. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)

    Article  Google Scholar 

  40. Salloum, R., Ren, Y., Kuo, C.C.J.: Image splicing localization using a multi-task fully convolutional network (MFCN). J. Vis. Commun. Image Represent. 51, 201–209 (2018)

    Article  Google Scholar 

  41. Schwarcz, S., Chellappa, R.: Finding facial forgery artifacts with parts-based detectors. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 933–942 (2021)

  42. Swaminathan, A., Wu, M., Liu, K.R.: Component forensics of digital cameras: A non-intrusive approach. In: 2006 40th annual conference on information sciences and systems, pp. 1194–1199 (2006)

  43. Wang, J., Wu, Z., Chen, J., Han, X., Shrivastava, A., Lim, S.N., Jiang, Y.G.: Objectformer for image manipulation detection and localization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2364–2373 (2022)

  44. Wang, Z., Guo, H., Zhang, Z., Liu, W., Qin, Z., Ren, K.: Feature importance-aware transferable adversarial attacks. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 7639–7648 (2021)

  45. Warbhe, A.D., Dharaskar, R., Thakare, V.: Computationally efficient digital image forensic method for image authentication. Proc. Comput. Sci. 78, 464–470 (2016)

    Article  Google Scholar 

  46. Wei, Z., Chen, J., Wu, Z., Jiang, Y.G.: Enhancing the self-universality for transferable targeted attacks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12,281–12,290 (2023)

  47. Wen, B., Zhu, Y., Subramanian, R., Ng, T.T., Shen, X., Winkler, S.: COVERAGE—A novel database for copy-move forgery detection. In: 2016 IEEE international conference on image processing, pp. 161–165 (2016)

  48. Wu, H., Zhou, J., Tian, J., Liu, J., Qiao, Y.: Robust image forgery detection against transmission over online social networks. IEEE Trans. Inf. Forensics Secur. 17, 443–456 (2022)

    Article  Google Scholar 

  49. Zeng, Y., Lin, Z., Patel, V.M.: SketchEdit: Mask-free local image manipulation with partial sketches. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5951–5961 (2022)

  50. Zhang, J., Wu, W., Huang, J.t., Huang, Y., Wang, W., Su, Y., Lyu, M.R.: Improving adversarial transferability via neuron attribution-based attacks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14,993–15,002 (2022)

  51. Zhao, A., Chu, T., Liu, Y., Li, W., Li, J., Duan, L.: Minimizing maximum model discrepancy for transferable black-box targeted attacks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8153–8162 (2023)

  52. Zhao, Z., Liu, Z., Larson, M.: Towards large yet imperceptible adversarial image perturbations with perceptual color distance. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1039–1048 (2020)

  53. Zhou, L., Cui, P., Zhang, X., Jiang, Y., Yang, S.: Adversarial eigen attack on black-box models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15,254–15,262 (2022)

  54. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Learning rich features for image manipulation detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1053–1061 (2018)

  55. Zhu, P., Osada, G., Kataoka, H., Takahashi, T.: Frequency-aware GAN for adversarial manipulation generation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 4315–4324 (2023)

  56. Zou, J., Duan, Y., Li, B., Zhang, W., Pan, Y., Pan, Z.: Making adversarial examples more transferable and indistinguishable. In: Proceedings of the AAAI conference on artificial intelligence, 36, pp. 3662–3670 (2022)

Download references

Acknowledgements

This work was supported in part by the Science and Technology Development Fund, Macau SAR, under Grants 0087/2020/A2, 0141/2023/RIA2 and 0193/2023/RIA3.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau, China

    Yijia Zeng & Chi-Man Pun

Authors
  1. Yijia Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chi-Man Pun
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.Z. wrote the first draft and it is revised by the C.M.P. All authors contributed to and reviewed the manuscript.

Corresponding author

Correspondence to Chi-Man Pun.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, Y., Pun, CM. Frequency-constrained transferable adversarial attack on image manipulation detection and localization. Vis Comput 40, 4817–4828 (2024). https://doi.org/10.1007/s00371-024-03482-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-024-03482-4

Keywords

Search

Navigation