An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by Speakers
Article No.: 39, Pages 1 - 30
Abstract
The use of speakers in electronic devices has become widespread, but the security risks associated with micro-speakers, such as earphones, are often overlooked. Many assume that soundproof barriers can prevent sound leakage and protect privacy. This article presents the prototype MagEar, an eavesdropping system that exploits magnetic side-channel signals leaked by a micro-speaker to restore intelligible human speech. MagEar outperforms some high-precision magnetometers in detecting magnetic fields at the nanotesla level. Even at a distance of 60 cm, it can recover high-quality audio with a 90% similarity to the original audio. Moreover, the MagEar prototype is portable and can be concealed within a headset housing. We have implemented MagEar as a proof-of-concept system and conducted multiple case studies on the eavesdropping of various speaker-embedded devices, including earphones. The recovered speech can be transcribed using automatic speech recognition techniques, even when obstructed by soundproof walls. It is our aspiration that our work can prompt manufacturers to reconsider the security vulnerabilities of speakers.
References
[1]
2017. Global Micro Speaker Market Driven by the Rise in Adoption of E-commerce Platforms: Technavio. Retrieved March 25, 2021 from https://www.businesswire.com/news/home/20170224005162/en/Global-Micro-Speaker-Market-Driven-by-the-Rise-in-Adoption-of-E-commerce-Platforms-Technavio
[2]
2017. The Market for MEMS Microphones and ECMs, Micro-speakers and Audio ICs Will Be Worth 20B in 2022. Retrieved Marcy 25, 2021 from https://www.i-micronews.com/products/acoustic-mems-and-audio-solutions-2017/
[3]
2019. Apple to Limit Accelerometer and Gyroscope Access in Safari on iOS 12.2 for Privacy Reasons. Retrieved March 25, 2021 from https://www.macrumors.com/2019/02/04/ios-12-2-safari-motion-orientation-access-toggle/
[4]
AbeDavis, MichaelRubinstein, NealWadhwa, GauthamJ.Mysore, FredoDurand, and WilliamT.Freeman. 2014. The visual microphone: Passive recovery of sound from video. ACM Trans. Graph. 33, 4 (2014), 79:1–79:10.
[5]
S. Abhishek Anand and Nitesh Saxena. 2018. Speechless: Analyzing the threat to speech privacy from smartphone motion sensors. In Proceedings of the IEEE Symposium on Security and Privacy (SP’18). IEEE, 1000–1017.
[6]
S. Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, and Yingying Chen. 2021. Spearphone: a lightweight speech privacy exploit via accelerometer-sensed reverberations from smartphone loudspeakers. In Proceedings of the 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks. 288–299.
[7]
Zhongjie Ba, Tianhang Zheng, Xinyu Zhang, Zhan Qin, Baochun Li, Xue Liu, and Kui Ren. 2020. Learning-based practical smartphone eavesdropping with built-in accelerometer. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’20). 23–26.
[8]
Michael Berouti, Richard Schwartz, and John Makhoul. 1979. Enhancement of speech corrupted by acoustic noise. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’79), Vol. 4. IEEE, 208–211.
[9]
John Borwick. 2012. Loudspeaker and Headphone Handbook. CRC Press.
[10]
Yushi Cheng, Xiaoyu Ji, Wenyuan Xu, Hao Pan, Zhuangdi Zhu, Chuang-Wen You, Yi-Chao Chen, and Lili Qiu. 2019. MagAttack: Guessing application launching and operation via smartphone. In Proceedings of the ACM Asia Conference on Computer and Communications Security (Asia CCS’19). Association for Computing Machinery, New York, NY, 283–294. DOI:
[11]
Daeseon Choi and Younho Lee. 2016. Eavesdropping one-time tokens over magnetic secure transmission in samsung pay. In Proceedings of the 10th USENIX Conference on Offensive Technologies (WOOT’16). USENIX Association, 52–58.
[12]
Daeseon Choi and Younho Lee. 2018. Eavesdropping of magnetic secure transmission signals and its security implications for a mobile payment protocol. IEEE Access 6 (07 2018), 1–1. DOI:
[13]
Minhao Cui, Yuda Feng, Qing Wang, and Jie Xiong. 2020. Sniffing visible light communication through walls. MobiCom’20: The 26th Annual International Conference on Mobile Computing and Networking. 30:1–30:14.
[14]
Jun Han, Albert Jin Chung, and Patrick Tague. 2017. Pitchln: Eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion. In Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN’17). Association for Computing Machinery, New York, NY, 181–192. DOI:
[15]
Yuichi Hayashi, Naofumi Homma, Mamoru Miura, Takafumi Aoki, and Hideaki Sone. 2014. A threat for tablet PCs in public space: Remote visualization of screen images using em emanation. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’14). Association for Computing Machinery, New York, NY, 954–965. DOI:
[16]
Maurice Hott, Peter A. Hoeher, and Sebastian F. Reinecke. 2019. Magnetic communication using high-sensitivity magnetic field detectors. Sensors 19, 15 (2019), 3415.
[17]
Hua Huang and Shan Lin. 2020. MET: A magneto-inductive sensing based electric toothbrushing monitoring system. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (MobiCom’20). Association for Computing Machinery, New York, NY, Article 19, 14 pages. DOI:
[18]
Yongzhi Huang, Kaixin Chen, Yandao Huang, Lu Wang, and Kaishun Wu. 2023. A portable and convenient system for unknown liquid identification with smartphone vibration. IEEE Trans. Mobile Comput. 22, 4 (2023), 1894–1911.
[19]
Yongzhi Huang, Kaixin Chen, Yandao Huang, Lu Wang, and Kaishun Wu. 2021. Vi-liquid: Unknown liquid identification with your smartphone vibration. In Proceedings of the Annual International Conference On Mobile Computing And Networking (MobiCom’21). 174–187.
[20]
Yongzhi Huang, Kaixin Chen, Lu Wang, Yinying Dong, Qianyi Huang, and Kaishun Wu. 2021. Lili: Liquor quality monitoring based on light signals. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 256–268.
[21]
Nathan Ida. 2015. Engineering Electromagnetics. Springer.
[22]
Valeriy Korepanov and Vira Pronenko. 2010. Induction magnetometers-design peculiarities. Sens. Transduc. 120, 9 (2010), 92.
[23]
Andrew Kwong, Wenyuan Xu, and Kevin Fu. 2019. Hard drive of hearing: Disks that eavesdrop with a synthesized microphone. In Proceedings of the IEEE Symposium on Security and Privacy (SP’19).
[24]
Qianru Liao, Yongzhi Huang, Yandao Huang, Yuheng Zhong, Huitong Jin, and Kaishun Wu. 2022. MagEar: Eavesdropping via audio recovery using magnetic side channel. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 371–383.
[25]
Yihao Liu, Kai Huang, Xingzhe Song, Boyuan Yang, and Wei Gao. 2020. MagHacker: Eavesdropping on stylus pen writing via magnetic sensing from commodity mobile devices. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 148–160.
[26]
Chen-Chou Lo, Szu-Wei Fu, Wen-Chin Huang, Xin Wang, Junichi Yamagishi, Yu Tsao, and Hsin-Min Wang. 2019. MOSNet: Deep learning based objective assessment for voice conversion. Interspeech 2019, 20th Annual Conference of the International Speech Communication Association (ISCA’19). 1541–1545.
[27]
Yan Michalevsky, Dan Boneh, and Gabi Nakibly. 2014. Gyrophone: Recognizing speech from gyroscope signals. In Proceedings of the 23rd USENIX Security Symposium.
[28]
Ralph P. Muscatell. 1984. Laser microphone. (Oct. 23 1984). US Patent 4,479,265.
[29]
Ben Nassi, Yaron Pirutin, Adi Shamir, Yuval Elovici, and Boris Zadov. 2020. Lamphone: Real-time passive sound recovery from light bulb vibrations. BlackHat USA (2020).
[30]
Nirupam Roy and Romit Roy Choudhury. 2016. Listening through a vibration motor. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’16). Association for Computing Machinery, New York, NY, 57–69. DOI:
[31]
John R. Speciale. 2001. Pulsed laser microphone. (Oct. 9 2001). US Patent 6,301,034.
[32]
Cem Subakan, Mirco Ravanelli, Samuele Cornell, Mirko Bronzi, and Jianyuan Zhong. 2021. Attention is all you need in speech separation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’21). IEEE, 21–25.
[33]
Avery Wang et al. 2003. An industrial strength audio search algorithm. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’03), Vol. 2003. Citeseer, 7–13.
[34]
Chen-Chia Wang, Sudhir Trivedi, Feng Jin, V. Swaminathan, Ponciano Rodriguez, and Narasimha S. Prasad. 2009. High sensitivity pulsed laser vibrometer and its application as a laser microphone. Appl. Phys. Lett. 94, 5 (2009), 051112.
[35]
Guanhua Wang, Yongpan Zou, Zimu Zhou, Kaishun Wu, and Lionel M. Ni. 2014. We can hear you with Wi-Fi!. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (MobiCom’14). Association for Computing Machinery, New York, NY, 593–604. DOI:
[36]
Teng Wei, Shu Wang, Anfu Zhou, and Xinyu Zhang. 2015. Acoustic eavesdropping through wireless vibrometry. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom’15). Association for Computing Machinery, New York, NY, 130–141. DOI:
[37]
Jerry Whitaker and Blair Benson. 2001. Standard Handbook of Audio Engineering. McGraw–Hill Education.
[38]
Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. LibriTTS: A corpus derived from librispeech for text-to-speech. arXiv:1904.02882. Retrieved from https://arxiv.org/abs/1904.02882
[39]
Li Zhang, Parth H. Pathak, Muchen Wu, Yixin Zhao, and Prasant Mohapatra. 2015. AccelWord: Energy efficient hotword detection through accelerometer. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’15). Association for Computing Machinery, New York, NY, 301–315. DOI:
[40]
Renjie Zhao, Purui Wang, Yunfei Ma, Pengyu Zhang, Hongqiang Harry Liu, Xianshang Lin, Xinyu Zhang, Chenren Xu, and Ming Zhang. 2020. NFC+: Breaking NFC networking limits through resonance engineering. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM’20). Association for Computing Machinery, New York, NY, 694–707. DOI:
Index Terms
- An Eavesdropping System Based on Magnetic Side-Channel Signals Leaked by Speakers
Recommendations
MagEar: eavesdropping via audio recovery using magnetic side channel
MobiSys '22: Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and ServicesSpeakers have been widely embedded in various electronic devices as a standard configuration. The security vulnerability of microspeakers (such as earphones) is commonly overlooked because it is often assumed that soundproof boundaries, such as walls, ...
Evaluating the Effectiveness of Protection Jamming Devices in Mitigating Smart Speaker Eavesdropping Attacks Using Gaussian White Noise
ACSAC '21: Proceedings of the 37th Annual Computer Security Applications ConferenceProtection Jamming Devices (PJD) are specialized tools designed to sit on top of virtual assistant (VA) smart speakers and hinder them from “hearing” nearby user speech. PJDs aim to protect you from eavesdropping attacks by injecting a jamming signal ...
Prefetch Side-Channel Attacks: Bypassing SMAP and Kernel ASLR
CCS '16: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications SecurityModern operating systems use hardware support to protect against control-flow hijacking attacks such as code-injection attacks. Typically, write access to executable pages is prevented and kernel mode execution is restricted to kernel code pages only. ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Journal Family
Publication History
Published: 10 January 2024
Online AM: 11 December 2023
Accepted: 01 December 2023
Revised: 12 November 2023
Received: 05 January 2023
Published in TOSN Volume 20, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- China NSFC
- Guangdong Provincial Key Lab of Integrated Communication, Sensing and Computation for Ubiquitous Internet of Things
- Project of DEGP
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 369Total Downloads
- Downloads (Last 12 months)369
- Downloads (Last 6 weeks)24
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in