On Comparing Ad Hoc Detectors with Statistical Hypothesis Tests
Pages 37 - 46
Abstract
This paper addresses how to fairly compare ROCs of ad hoc (or data driven) detectors with tests derived from statistical models of digital media. We argue that the ways ROCs are typically drawn for each detector type correspond to different hypothesis testing problems with different optimality criteria, making the ROCs uncomparable. To understand the problem and why it occurs, we model a source of natural images as a mixture of scene oracles and derive optimal detectors for the task of image steganalysis. Our goal is to guarantee that, when the data follows the statistical model adopted for the hypothesis test, the ROC of the optimal detector bounds the ROC of the ad hoc detector. While the results are applicable beyond the field of image steganalysis, we use this setup to point out possible inconsistencies when comparing both types of detectors and explain guidelines for their proper comparison. Experiments on an artificial cover source with a known model with real steganographic algorithms and deep learning detectors are used to confirm our claims.
References
[1]
P. Bas. Steganography via cover-source switching. In 2016 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1--6, December 4-7 2016.
[2]
P. Bas, T. Filler, and T. Pevný. Break our steganographic system - the ins and outs of organizing BOSS. In T. Filler, T. Pevný, A. Ker, and S. Craver, editors, Information Hiding, 13th International Conference, volume 6958 of Lecture Notes in Computer Science, pages 59--70, Prague, Czech Republic, May 18-20, 2011.
[3]
P. Bas and T. Furon. BOWS-2. http://bows2.ec-lille.fr, July 2007.
[4]
R. Böhme. Improved Statistical Steganalysis Using Models of Heterogeneous Cover Signals. PhD thesis, Faculty of Computer Science, Technische Universität Dresden, Germany, 2008.
[5]
M. Boroumand, J. Fridrich, and R. Cogranne. Are we there yet" In A. Alattar and N. D. Memon, editors, Proceedings IS&T, Electronic Imaging, Media Watermarking, Security, and Forensics 2019, San Francisco, CA, January 26-30, 2019.
[6]
J. Butora and J. Fridrich. Reverse JPEG compatibility attack. IEEE Transactions on Information Forensics and Security, 15:1444--1454, 2020.
[7]
J. Butora, Y. Yousfi, and J. Fridrich. How to pretrain for steganalysis. In D. Borghys and P. Bas, editors, The 9th ACM Workshop on Information Hiding and Multimedia Security, Brussels, Belgium, 2021. ACM Press.
[8]
R. Cogranne. Selection-channel-aware reverse JPEG compatibility for highly reliable steganalysis of JPEG images. In Proceedings IEEE, International Conference on Acoustics, Speech, and Signal Processing, pages 2772--2776, Barcelona, Spain, May 4-8, 2020.
[9]
R. Cogranne, Q. Giboulot, and P. Bas. The ALASKA steganalysis challenge: A first step towards steganalysis "Into the wild". In R. Cogranne and L. Verdoliva, editors, The 7th ACM Workshop on Information Hiding and Multimedia Security, Paris, France, July 3-5, 2019. ACM Press.
[10]
R. Cogranne and F. Retraint. Application of hypothesis testing theory for optimal detection of LSB matching data hiding. Signal Processing, 93(7):1724--1737, July, 2013.
[11]
R. Cogranne and F. Retraint. An asymptotically uniformly most powerful test for LSB Matching detection. IEEE Transactions on Information Forensics and Security, 8(3):464--476, 2013.
[12]
R. Cogranne, V. Sedighi, T. Pevný, and J. Fridrich. Is ensemble classifier needed for steganalysis in high-dimensional feature spaces? In IEEE International Workshop on Information Forensics and Security, Rome, Italy, November 16-19, 2015.
[13]
R. Cogranne, C. Zitzmann, L. Fillatre, F. Retraint, I. Nikiforov, and P. Cornu. A cover image model for reliable steganalysis. In T. Filler, T. Pevný, A. Ker, and S. Craver, editors, Information Hiding, 13th International Conference, Lecture Notes in Computer Science, pages 178--192, Prague, Czech Republic, May 18-20, 2011.
[14]
R. Cogranne, C. Zitzmann, F. Retraint, I. Nikiforov, L. Fillatre, and P. Cornu. Statistical detection of LSB Matching using hypothesis testing theory. In M. Kirchner and D. Ghosal, editors, Information Hiding, 14th International Conference, volume 7692 of Lecture Notes in Computer Science, pages 46--62, Berkeley, California, May 15-18, 2012.
[15]
E. Dworetzky, E. Kaziakhmedov, and J. Fridrich. Advancing the JPEG com- patibility attack: Theory, performance, robustness, and practice. In Y. Yousfi, C. Pasquini, and A. Bharati, editors, The 11th ACM Workshop on Information Hiding and Multimedia Security, Chicago, IL, June 28--30, 2023. ACM Press.
[16]
L. Fillatre. Adaptive steganalysis of least significant bit replacement in grayscale images. IEEE Transactions on Signal Processing, 60(2):556--569, 2011.
[17]
T. Filler and J. Fridrich. Fisher information determines capacity of ∈-secure steganography. In S. Katzenbeisser and A.-R. Sadeghi, editors, Information Hiding, 11th International Conference, volume 5806 of Lecture Notes in Computer Science, pages 31--47, Darmstadt, Germany, June 7-10, 2009. Springer-Verlag, New York.
[18]
J. Fridrich and J. Kodovský. Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 7(3):868--882, June 2011.
[19]
Q. Giboulot, P. Bas, and R. Cogranne. Multivariate side-informed Gaussian embedding minimizing statistical detectability. IEEE Transactions on Information Forensics and Security, 17:1841--1854, 2022.
[20]
Q. Giboulot, R. Cogranne, and P. Bas. Detectability-based JPEG steganography modeling the processing pipeline: The noise-content trade-off. IEEE Transactions on Information Forensics and Security, 16:2202--2217, 2021.
[21]
V. Holub, J. Fridrich, and T. Denemark. Universal distortion design for stegano- graphy in an arbitrary domain. EURASIP Journal on Information Security, Special Issue on Revised Selected Papers of the 1st ACM IH and MMS Workshop, 2014:1, 2014.
[22]
J. Indritz. An inequality for Hermite polynomials. 12(6):981--983, 1961.
[23]
S. M. Kay. Fundamentals of Statistical Signal Processing, Volume II: Detection Theory, volume II. Upper Saddle River, NJ: Prentice Hall, 1998.
[24]
A. D. Ker. Steganalysis of LSB matching in grayscale images. IEEE Signal Processing Letters, 12(6):441--444, June 2005.
[25]
A. D. Ker. Estimating steganographic fisher information in real images. In S. Katzenbeisser and A.-R. Sadeghi, editors, Information Hiding, 11th International Conference, volume 5806 of Lecture Notes in Computer Science, pages 73--88, Darmstadt, Germany, June 7-10, 2009. Springer-Verlag, New York.
[26]
A. D. Ker. On the relationship between embedding costs and steganographic capacity. In M. Stamm, M. Kirchner, and S. Voloshynovskiy, editors, The 5th ACM Workshop on Information Hiding and Multimedia Security, Philadelphia, PA, June 20-22, 2017. ACM Press.
[27]
K. Lee and A. Westfeld. Generalized category attack - improving histogram- based attack on JPEG LSB embedding. In T. Furon, F. Cayre, G. Doërr, and P. Bas, editors, Information Hiding, 9th International Workshop, volume 4567 of Lecture Notes in Computer Science, pages 378--392, Saint Malo, France, June 11-13, 2007. Springer-Verlag, Berlin.
[28]
E.L. Lehmann and J.P. Romano. Testing Statistical Hypotheses, Third Edition. Springer, 3rd edition, 2005.
[29]
B. Li, M. Wang, and J. Huang. A new cost function for spatial image steganography. In Proceedings IEEE, International Conference on Image Processing, ICIP, Paris, France, October 27--30, 2014.
[30]
J. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley & Sons, 3rd edition, 2007.
[31]
M. K. Mihcak, I. Kozintsev, K. Ramchandran, and P. Moulin. Low-complexity image denoising based on statistical modeling of wavelet coefficients. IEEE Signal Processing Letters, 6(12):300--303, December 1999.
[32]
V. Sedighi, R. Cogranne, and J. Fridrich. Content-adaptive steganography by minimizing statistical detectability. IEEE Transactions on Information Forensics and Security, 11(2):221--234, 2016.
[33]
T. Taburet, P. Bas, W. Sawaya, and J. Fridrich. A natural steganography embed- ding scheme dedicated to color sensors in the JPEG domain. In A. Alattar and N. D. Memon, editors, Proceedings IS&T, Electronic Imaging, Media Watermarking, Security, and Forensics 2019, San Francisco, CA, January 26-30, 2019.
[34]
T. Thai, R. Cogranne, and F. Retraint. Statistical model of quantized DCT coeffi- cients: Application in the steganalysis of Jsteg algorithm. Image Processing, IEEE Transactions on, 23(5):1--14, May 2014.
[35]
T. H. Thai, R. Cogranne, and F. Retraint. Optimal detection of OutGuess using an accurate model of DCT coefficients. In Sixth IEEE International Workshop on Information Forensics and Security, Atlanta, GA, December 3-5, 2014.
[36]
Thanh Hai Thai, R. Cogranne, and F. Retraint. Camera model identification based on the heteroscedastic noise model. Image Processing, IEEE Transactions on, 23(1):250--263, Jan 2014.
[37]
J. Zhang, I. J. Cox, and G. Doerr. Steganalysis for LSB matching in images with high-frequency noise. In IEEE 9th Workshop on Multimedia Signal Processing, pages 385--388, 2007.
[38]
T. Zhang and X. Ping. A fast and effective steganalytic technique against Jsteg- like algorithms. In Proceedings of the ACM Symposium on Applied Computing, pages 307--311, Melbourne, FL, March 9-12, 2003.
Index Terms
- On Comparing Ad Hoc Detectors with Statistical Hypothesis Tests
Recommendations
A novel solution for achieving anonymity in wireless ad hoc networks
PE-WASUN '04: Proceedings of the 1st ACM international workshop on Performance evaluation of wireless ad hoc, sensor, and ubiquitous networksA mobile ad hoc network consists of mobile nodes that can move freely in an open environment. Communicating nodes in a wireless and mobile ad hoc network usually seek the help of other intermediate nodes to establish communication channels. In such an ...
Enhancing DSR protocol performance in mobile ad hoc network using ACK reply
This paper presents a new method to improve performance of DSR in Ad Hoc Network. We present an analysis of DSR protocol and propose our algorithm to improve the performance of DSR protocol by using ACK reply path as a backup route when an original ...
Link Stability Approach and Scalability Method on ODMRP in Ad Hoc Networks
CNSR '09: Proceedings of the 2009 Seventh Annual Communication Networks and Services Research ConferenceAn ad hoc network is a dynamically reconfigurable wireless network with no fixed wired infrastructure. The primary concerns in ad hoc networks are bandwidth limitation and unpredictable dynamic topology. The On-Demand Multicast Routing Protocol (ODMRP) ...
Comments
Information & Contributors
Information
Published In
June 2023
190 pages
ISBN:9798400700545
DOI:10.1145/3577163
- General Chair:
- Daniel Moreira,
- Program Chairs:
- Aparna Bharati,
- Cecilia Pasquini,
- Yassine Yousfi
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
IH&MMSec '23
Sponsor:
IH&MMSec '23: ACM Workshop on Information Hiding and Multimedia Security
June 28 - 30, 2023
IL, Chicago, USA
Acceptance Rates
Overall Acceptance Rate 128 of 318 submissions, 40%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 67Total Downloads
- Downloads (Last 12 months)44
- Downloads (Last 6 weeks)17
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in