research-article

Detecting Audio Adversarial Examples with Logit Noising

Authors:

Jong KimAuthors Info & Claims

ACSAC '21: Proceedings of the 37th Annual Computer Security Applications Conference

Pages 586 - 595

https://doi.org/10.1145/3485832.3485912

Published: 06 December 2021 Publication History

Abstract

Automatic speech recognition (ASR) systems are vulnerable to audio adversarial examples that attempt to deceive ASR systems by adding perturbations to benign speech signals. Although an adversarial example and the original benign wave are indistinguishable to humans, the former is transcribed as a malicious target sentence by ASR systems. Several methods have been proposed to generate audio adversarial examples and feed them directly into the ASR system (over-line). Furthermore, many researchers have demonstrated the feasibility of robust physical audio adversarial examples (over-air). To defend against the attacks, several studies have been proposed. However, deploying them in a real-world situation is difficult because of accuracy drop or time overhead.

In this paper, we propose a novel method to detect audio adversarial examples by adding noise to the logits before feeding them into the decoder of the ASR. We show that carefully selected noise can significantly impact the transcription results of the audio adversarial examples, whereas it has minimal impact on the transcription results of benign audio waves. Based on this characteristic, we detect audio adversarial examples by comparing the transcription altered by logit noising with its original transcription. The proposed method can be easily applied to ASR systems without any structural changes or additional training. The experimental results show that the proposed method is robust to over-line audio adversarial examples as well as over-air audio adversarial examples compared with state-of-the-art detection methods.

References

[1]

Martín Abadi 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.

[2]

H. Abdullah, M. Rahman, W. Garcia, K. Warren, A. Swarnim Yadav, T. Shrimpton, and P. Traynor. 2021. Hear ”No Evil”, See ”Kenansville”*: Efficient and Transferable Black-Box Attacks on Speech Recognition and Voice Identification Systems. In 2021 2021 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, Los Alamitos, CA, USA, 142–159. https://doi.org/10.1109/SP40001.2021.00009

[3]

Hadi Abdullah, Kevin Warren, Vincent Bindschaedler, Nicolas Papernot, and Patrick Traynor. 2020. SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems. arxiv:2007.06622 [cs.CR]

[4]

Nasir Ahmed, T_ Natarajan, and Kamisetty R Rao. 1974. Discrete cosine transform. IEEE transactions on Computers 100, 1 (1974), 90–93.

[5]

”Amazon”. [n. d.]. ”Amazon Alexa”. https://www.amazon.com.

[6]

”Apple”. [n. d.]. ”Apple Siri”. https://www.apple.com/siri.

[7]

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing Robust Adversarial Examples. In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 284–293. http://proceedings.mlr.press/v80/athalye18b.html

[8]

Nicholas Carlini and David Wagner. 2018. Audio Adversarial Examples: Targeted Attacks on Speech-to-Text. In 2018 IEEE Security and Privacy Workshops (SPW). 1–7. https://doi.org/10.1109/SPW.2018.00009

[9]

Tao Chen, Longfei Shangguan, Zhenjiang Li, and Kyle Jamieson. 2020. Metamorph: Injecting Inaudible Commands into Over-the-air Voice Controlled Systems. In Proceedings of NDSS.

[10]

Yuxuan Chen, Xuejing Yuan, Jiangshan Zhang, Yue Zhao, Shengzhi Zhang, Kai Chen, and XiaoFeng Wang. 2020. Devil’s Whisper: A General Approach for Physical Adversarial Attacks against Commercial Black-box Speech Recognition Devices. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2667–2684. https://www.usenix.org/conference/usenixsecurity20/presentation/chen-yuxuan

[11]

Tom Dörr, Karla Markert, Nicolas M. Müller, and Konstantin Böttinger. 2020. Towards Resistant Audio Adversarial Examples. In Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence (Taipei, Taiwan) (SPAI ’20). Association for Computing Machinery, New York, NY, USA, 3–10. https://doi.org/10.1145/3385003.3410921

Digital Library

[12]

Xia Du, Chi-Man Pun, and Zheng Zhang. 2020. A Unified Framework for Detecting Audio Adversarial Examples. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM ’20). Association for Computing Machinery, New York, NY, USA, 3986–3994. https://doi.org/10.1145/3394171.3413603

Digital Library

[13]

”Google”. [n. d.]. ”Google Assistant”. https://assistant.google.com.

[14]

Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. In Proceedings of the 23rd International Conference on Machine Learning (Pittsburgh, Pennsylvania, USA) (ICML ’06). Association for Computing Machinery, New York, NY, USA, 369–376. https://doi.org/10.1145/1143844.1143891

Digital Library

[15]

Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, and Andrew Y. Ng. 2014. Deep Speech: Scaling up end-to-end speech recognition. arxiv:1412.5567 [cs.CL]

[16]

Hyun Kwon, Hyunsoo Yoon, and Ki-Woong Park. 2019. POSTER: Detecting Audio Adversarial Example through Audio Modification. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (London, United Kingdom) (CCS ’19). Association for Computing Machinery, New York, NY, USA, 2521–2523. https://doi.org/10.1145/3319535.3363246

Digital Library

[17]

Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. Soviet Union, 707–710.

[18]

Juncheng B Li, Shuhui Qu, Xinjian Li, Joseph Szurley, J Zico Kolter, and Florian Metze. 2019. Adversarial music: Real world audio adversary against wake-word detection system. arXiv preprint arXiv:1911.00126(2019).

[19]

Ruirui Li, Jyun-Yu Jiang, Xian Wu, Chu-Cheng Hsieh, and Andreas Stolcke. 2020. Speaker Identification for Household Scenarios with Self-Attention and Adversarial Training. In Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China, 25-29 October 2020, Helen Meng, Bo Xu, and Thomas Fang Zheng (Eds.). ISCA, 2272–2276. https://doi.org/10.21437/Interspeech.2020-3025

[20]

Zhuohang Li, Yi Wu, Jian Liu, Yingying Chen, and Bo Yuan. 2020. AdvPulse: Universal, Synchronization-Free, and Targeted Audio Adversarial Attacks via Subsecond Perturbations. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (Virtual Event, USA) (CCS ’20). Association for Computing Machinery, New York, NY, USA, 1121–1134. https://doi.org/10.1145/3372297.3423348

Digital Library

[21]

Xiaolei Liu, Kun Wan, Yufei Ding, Xiaosong Zhang, and Qingxin Zhu. 2020. Weighted-Sampling Audio Adversarial Example Attack. Proceedings of the AAAI Conference on Artificial Intelligence 34, 04 (Apr. 2020), 4908–4915. https://doi.org/10.1609/aaai.v34i04.5928

[22]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083(2017).

[23]

”Microsoft”. [n. d.]. ”Microsoft Cortana”. https://www.microsoft.com/en-us/cortana.

[24]

Lindasalwa Muda, Mumtaj Begam, and Irraivan Elamvazuthi. 2010. Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. arXiv preprint arXiv:1003.4083(2010).

[25]

Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 5206–5210. https://doi.org/10.1109/ICASSP.2015.7178964

[26]

Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, 2011. The Kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding. IEEE Signal Processing Society.

[27]

Yao Qin, Nicholas Carlini, Garrison Cottrell, Ian Goodfellow, and Colin Raffel. 2019. Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 5231–5240. http://proceedings.mlr.press/v97/qin19a.html

[28]

Lawrence R Rabiner, Ronald W Schafer, 1978. Digital processing of speech signals. Prentice-hall.

[29]

Lea Schönherr, Thorsten Eisenhofer, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. 2020. Imperio: Robust Over-the-Air Adversarial Examples for Automatic Speech Recognition Systems. In Annual Computer Security Applications Conference(Austin, USA) (ACSAC ’20). Association for Computing Machinery, New York, NY, USA, 843–855. https://doi.org/10.1145/3427228.3427276

Digital Library

[30]

Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. 2018. Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding. arXiv preprint arXiv:1808.05665(2018).

[31]

Jonathan Shen and et. al.2019. Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling. CoRR abs/1902.08295(2019). arxiv:1902.08295http://arxiv.org/abs/1902.08295

[32]

Rohan Taori, Amog Kamsetty, Brenton Chu, and Nikita Vemuri. 2019. Targeted Adversarial Examples for Black Box Audio Systems. In 2019 IEEE Security and Privacy Workshops (SPW). 15–20. https://doi.org/10.1109/SPW.2019.00016

[33]

Xiong Wang, Sining Sun, Changhao Shan, Jingyong Hou, Lei Xie, Shen Li, and Xin Lei. 2019. Adversarial Examples for Improving End-to-end Attention-based Small-footprint Keyword Spotting. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019. IEEE, 6366–6370. https://doi.org/10.1109/ICASSP.2019.8683479

[34]

Hiromu Yakura and Jun Sakuma. 2019. Robust Audio Adversarial Example for a Physical Attack. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 5334–5341. https://doi.org/10.24963/ijcai.2019/741

[35]

Zhuolin Yang, Bo Li, Pin-Yu Chen, and Dawn Song. 2018. Characterizing Audio Adversarial Examples Using Temporal Dependency. In International Conference on Learning Representations.

[36]

Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, and Carl A Gunter. 2018. Commandersong: A systematic approach for practical adversarial voice recognition. In 27th {USENIX} Security Symposium ({USENIX} Security 18). 49–64.

Cited By

Park NKim J(2024)Toward Robust ASR System against Audio Adversarial Examples using Agitated LogitACM Transactions on Privacy and Security10.1145/366182227:2(1-26)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3661822
Noureddine KKheddar HMaazouz M(2023)Adversarial Example Detection Techniques in Speech Recognition Systems: A review2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM)10.1109/IC2EM59347.2023.10419688(1-7)Online publication date: 28-Nov-2023
https://doi.org/10.1109/IC2EM59347.2023.10419688
Choi YPark JLee JKim H(2023)Exploring Diverse Feature Extractions for Adversarial Audio DetectionIEEE Access10.1109/ACCESS.2023.323411011(2351-2360)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3234110
Show More Cited By

Index Terms

Detecting Audio Adversarial Examples with Logit Noising

Index terms have been assigned to the content through auto-classification.

Recommendations

A Unified Framework for Detecting Audio Adversarial Examples
MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Adversarial attacks have been widely recognized as the security vulnerability of deep neural networks, especially in deep automatic speech recognition (ASR) systems. The advanced detection methods against adversarial attacks mainly focus on pre-...
Toward Robust ASR System against Audio Adversarial Examples using Agitated Logit
Automatic speech recognition (ASR) systems are vulnerable to audio adversarial examples, which aim at deceiving ASR systems by adding perturbations to benign speech signals. These audio adversarial examples appear indistinguishable from benign audio waves,...
Synthesising Audio Adversarial Examples for Automatic Speech Recognition
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Adversarial examples in automatic speech recognition (ASR) are naturally sounded by humans yet capable of fooling well trained ASR models to transcribe incorrectly. Existing audio adversarial examples are typically constructed by adding constrained ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACSAC '21: Proceedings of the 37th Annual Computer Security Applications Conference

December 2021

1077 pages

ISBN:9781450385794

DOI:10.1145/3485832

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Institute for Information and communications Technology Promotion

Conference

ACSAC '21

ACSAC '21: Annual Computer Security Applications Conference

December 6 - 10, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 104 of 497 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
207
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Park NKim J(2024)Toward Robust ASR System against Audio Adversarial Examples using Agitated LogitACM Transactions on Privacy and Security10.1145/366182227:2(1-26)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3661822
Noureddine KKheddar HMaazouz M(2023)Adversarial Example Detection Techniques in Speech Recognition Systems: A review2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM)10.1109/IC2EM59347.2023.10419688(1-7)Online publication date: 28-Nov-2023
https://doi.org/10.1109/IC2EM59347.2023.10419688
Choi YPark JLee JKim H(2023)Exploring Diverse Feature Extractions for Adversarial Audio DetectionIEEE Access10.1109/ACCESS.2023.323411011(2351-2360)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3234110
Han SXu KGuo SYu MYang B(2022)Evading Logits-Based Detections to Audio Adversarial Examples by Logits-Traction AttackApplied Sciences10.3390/app1218938812:18(9388)Online publication date: 19-Sep-2022
https://doi.org/10.3390/app12189388
Choosaksakunwiboon SPizzi KKao C(2022)Comparing Unsupervised Detection Algorithms for Audio Adversarial ExamplesSpeech and Computer10.1007/978-3-031-20980-2_11(114-127)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-20980-2_11

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten