Article

Gyrophone: recognizing speech from gyroscope signals

Authors:

Yan Michalevsky,

Gabi NakiblyAuthors Info & Claims

SEC'14: Proceedings of the 23rd USENIX conference on Security Symposium

Pages 1053 - 1067

Published: 20 August 2014 Publication History

Abstract

We show that the MEMS gyroscopes found on modern smart phones are sufficiently sensitive to measure acoustic signals in the vicinity of the phone. The resulting signals contain only very low-frequency information (<200Hz). Nevertheless we show, using signal processing and machine learning, that this information is sufficient to identify speaker information and even parse speech. Since iOS and Android require no special permissions to access the gyro, our results show that apps and active web content that cannot access the microphone can nevertheless eavesdrop on speech in the vicinity of the phone.

References

[1]

3-axis digital gyroscopes. http://www.st.com/st-web-ui/ static/active/en/resource/sales_and_marketing/ promotional_material/flyer/fl3axdigitalgyro.pdf.

[2]

Corona SDK API reference. http://docs.coronalabs.com/ api/library/system/setGyroscopeInterval.html.

[3]

Galaxy Tab 7.7. http://www.techrepublic.com/blog/ cracking-open/galaxy-tab-77-teardown-reveals-lots-of-samsungs-homegrown-hardware/588/.

[4]

Inside the Latest Galaxy Note 3. http://www.chipworks.com/ en/technical-competitive-analysis/resources/ blog/inside-the-galaxy-note-3/.

[5]

Inside the Samsung Galaxy S4. http://www.chipworks.com/ en/technical-competitive-analysis/resources/ blog/inside-the-samsung-galaxy-s4/.

[6]

Inside the Samsung Galaxy SIII. http://www.chipworks.com/ en/technical-competitive-analysis/resources/ blog/inside-the-samsung-galaxy-siii/.

[7]

InvenSense Inc. http://www.invensense.com/.

[8]

iPad Mini Retina Display Teardown. http://www.ifixit.com/ Teardown/iPad+Mini+Retina+Display+Teardown/19374.

[9]

L3G4200D data sheet. http://www.st.com/st-web-ui/ static/active/en/resource/technical/document/ datasheet/CD00265057.pdf.

[10]

LSM330DLC data sheet. http://www.st.com/st-web-ui/ static/active/en/resource/technical/document/ datasheet/DM00037200.pdf.

[11]

MPU-6000 and MPU-6050 Register Map and Descriptions. http://invensense.com/mems/gyro/documents/RM-MPU- 6000A.pdf.

[12]

MPU-6050 product specification. http://www.invensense.com/mems/gyro/documents/PS-MPU- 6000A-00v3.4.pdf.

[13]

Nexus 4 Teardown. http://www.ifixit.com/Teardown/ Nexus+4+Teardown/11781.

[14]

Nexus 7 Teardown. http://www.ifixit.com/Teardown/ Nexus+7+Teardown/9623.

[15]

STMicroelectronics Inc. http://www.st.com/.

[16]

Everything about STMicroelectronics 3-axis digital MEMS gyroscopes. http://www.st.com/web/en/ resource/technical/document/technical_article/ DM00034730.pdf, July 2011.

[17]

iPhone 5S MEMS Gyroscope STMicroelectronics 3×3mm - Reverse Costing Analysis. http://www.researchandmarkets.com/research/lxrnrn/ iphone_5s_mems, October 2013.

[18]

MEMS for Cell Phones and Tablets. http://www.imicronews. com/upload/Rapports/Yole_MEMS_for_ Mobile_June_2013_Report_Sample.pdf, July 2013.

[19]

AL-HAIQI, A., ISMAIL, M., AND NORDIN, R. On the best sensor for keystrokes inference attack on android. Procedia Technology 11 (2013), 989-995.

[20]

APPELMAN, D. The Science of Vocal Pedagogy: Theory and Application. Midland book. Indiana University Press, 1967.

[21]

CAI, L., AND CHEN, H. Touchlogger: inferring keystrokes on touch screen from smartphone motion. In Proceedings of the 6th USENIX conference on Hot topics in security (2011), USENIX Association, pp. 9-9.

Digital Library

[22]

CASTRO, S., DEAN, R., ROTH, G., FLOWERS, G. T., AND GRANTHAM, B. Influence of acoustic noise on the dynamic performance of mems gyroscopes. In ASME 2007 International Mechanical Engineering Congress and Exposition (2007), pp. 1825-1831.

[23]

CRAMMER, K., KEARNS, M., AND WORTMAN, J. Learning from multiple sources. The Journal of Machine Learning Research 9 (2008), 1757-1774.

Digital Library

[24]

DEAN, R. N., CASTRO, S. T., FLOWERS, G. T., ROTH, G., AHMED, A., HODEL, A. S., GRANTHAM, B. E., BITTLE, D. A., AND BRUNSCH, J. P. A characterization of the performance of a mems gyroscope in acoustically harsh environments. Industrial Electronics, IEEE Transactions on 58, 7 (2011), 2591-2596.

[25]

DEAN, R. N., FLOWERS, G. T., HODEL, A. S., ROTH, G., CASTRO, S., ZHOU, R., MOREIRA, A., AHMED, A., RIFKI, R., GRANTHAM, B. E., ET AL. On the degradation of mems gyroscope performance in the presence of high power acoustic noise. In Industrial Electronics, 2007. ISIE 2007. IEEE International Symposium on (2007), IEEE, pp. 1435-1440.

[26]

EL-CHAMMAS, M., AND MURMANN, B. Background calibration of time-interleaved data converters. Springer, 2012.

[27]

EL-CHAMMAS, M. I. Background Calibration of Timing Skew in Time-Interleaved A/D Converters. Stanford University, 2010.

[28]

ELDAR, Y. C., AND OPPENHEIM, A. V. Filterbank reconstruction of bandlimited signals from nonuniform and generalized samples. Signal Processing, IEEE Transactions on 48, 10 (2000), 2864-2875.

Digital Library

[29]

GIANNAKOPOULOS, T. A method for silence removal and segmentation of speech signals, implemented in matlab.

[30]

HASAN, M. R., JAMIL, M., AND RAHMAN, M. G. R. M. S. Speaker identification using mel frequency cepstral coefficients. variations 1 (2004), 4.

[31]

JOHANSSON, H. K., AND LÖWENBORG, P. Reconstruction of periodically nonuniformly sampled bandlimited signals using time-varying FIR filters. 2005.

[32]

LARTILLOT, O., TOIVIAINEN, P., AND EEROLA, T. A matlab toolbox for music information retrieval. In Data analysis, machine learning and applications. Springer, 2008, pp. 261-268.

[33]

LEONARD, R. G., AND DODDINGTON, G. TIDIGITS. http://catalog.ldc.upenn.edu/LDC93S10, 1993.

[34]

MANTYJARVI, J., LINDHOLM, M., VILDJIOUNAITE, E., MAKELA, S.-M., AND AILISTO, H. Identifying users of portable devices from gait pattern with accelerometers. In Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP'05). IEEE International Conference on (2005), vol. 2, IEEE, pp. ii-973.

[35]

MARQUARDT, P., VERMA, A., CARTER, H., AND TRAYNOR, P. (sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In Proceedings of the 18th ACM conference on Computer and communications security (2011), ACM, pp. 551-562.

Digital Library

[36]

MIET, G. Towards wideband speech by narrowband speech bandwidth extension: magic effect or wideband recovery? These de doctorat, University of Maine (2001).

[37]

MÜLLER, M. Dynamic time warping. Information retrieval for music and motion (2007), 69-84.

[38]

PAPOULIS, A. Generalized sampling expansion. Circuits and Systems, IEEE Transactions on 24, 11 (1977), 652-654.

[39]

PRENDERGAST, R. S., LEVY, B. C., AND HURST, P. J. Reconstruction of band-limited periodic nonuniformly sampled signals through multirate filter banks. Circuits and Systems I: Regular Papers, IEEE Transactions on 51, 8 (2004), 1612-1622.

[40]

RABINER, L. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE (1989).

[41]

REYNOLDS, D. A., AND ROSE, R. C. Robust text-independent speaker identification using gaussian mixture speaker models. Speech and Audio Processing, IEEE Transactions on 3, 1 (1995), 72-83.

[42]

ROSSI, M., FEESE, S., AMFT, O., BRAUNE, N., MARTIS, S., AND TROSTER, G. Ambientsense: A real-time ambient sound recognition system for smartphones. In Pervasive Computing and Communications Workshops (PERCOM Workshops), 2013 IEEE International Conference on (2013), IEEE, pp. 230-235.

[43]

SEEGER, J., LIM, M., AND NASIRI, S. Development of high-performance high-volume consumer MEMS gyroscopes. http://www.invensense.com/mems/gyro/documents/ whitepapers/Development-of-High-Performance-High-Volume-Consumer-MEMS-Gyroscopes.pdf.

[44]

SHANNON, C. E. Communication in the presence of noise. Proceedings of the IRE 37, 1 (1949), 10-21.

[45]

SINDHI, S., AND PRABHU, K. Reconstruction of N-th Order Nonuniformly Sampled Signals Using Digital Filter Banks. commsp.ee.ic.ac.uk (2012).

[46]

STROHMER, T., AND TANNER, J. Fast reconstruction algorithms for periodic nonuniform sampling with applications to time-interleaved adcs. In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on (2007), vol. 3, IEEE, pp. III-881.

[47]

WALKER, W., LAMERE, P., KWOK, P., RAJ, B., SINGH, R., GOUVEA, E., WOLF, P., AND WOELFEL, J. Sphinx-4: A flexible open source framework for speech recognition. Tech. rep., 2004.

Digital Library

[48]

YEE, L., AND AHMAD, A. Comparative Study of Speaker Recognition Methods: DTW, GMM and SVM. comp.utm.my.

Cited By

Su YHuang SLiu HChen YDu YWang YRen YChen Y(2024)PPG-Hear: A Practical Eavesdropping Attack with Photoplethysmography SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596038:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659603
Sabra MVinayaga-Sureshkanth NSharma AMaiti AJadliwala MKim YKim JKoushanfar FRasmussen K(2024)De-anonymizing VR Avatars using Non-VR Motion Side-channelsProceedings of the 17th ACM Conference on Security and Privacy in Wireless and Mobile Networks10.1145/3643833.3656135(54-65)Online publication date: 27-May-2024
https://dl.acm.org/doi/10.1145/3643833.3656135
Zhao RYu JZhao HNgai E(2023)Radio2TextProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108737:3(1-28)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610873
Show More Cited By

Gyrophone: recognizing speech from gyroscope signals
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Hardware
  1. Communication hardware, interfaces and storage

Recommendations

An Explorative Study of the Mobile App Ecosystem from App Developers' Perspective
WWW '17: Proceedings of the 26th International Conference on World Wide Web

With the prevalence of smartphones, app markets such as Apple App Store and Google Play has become the center stage in the mobile app ecosystem, with millions of apps developed by tens of thousands of app developers in each major market. This paper ...
A Measurement-based Study on Application Popularity in Android and iOS App Stores
Mobidata '15: Proceedings of the 2015 Workshop on Mobile Big Data

Mobile application stores (appstores) are emerging digital distribution platforms with explosive growth. Although there have been some observations on the mobile application (app) popularity in Android appstores, there is no report on the app popularity ...
RubyMotion iOS Development Essentials

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

SEC'14: Proceedings of the 23rd USENIX conference on Security Symposium

August 2014

1067 pages

ISBN:9781931971157

Program Chair:
Kevin Fu
University of Michigan

Sponsors

Akamai: Akamai
Google Inc.
IBMR: IBM Research
NSF
Microsoft Reasearch: Microsoft Reasearch
USENIX Assoc: USENIX Assoc

Publisher

USENIX Association

United States

Publication History

Published: 20 August 2014

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Su YHuang SLiu HChen YDu YWang YRen YChen Y(2024)PPG-Hear: A Practical Eavesdropping Attack with Photoplethysmography SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596038:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659603
Sabra MVinayaga-Sureshkanth NSharma AMaiti AJadliwala MKim YKim JKoushanfar FRasmussen K(2024)De-anonymizing VR Avatars using Non-VR Motion Side-channelsProceedings of the 17th ACM Conference on Security and Privacy in Wireless and Mobile Networks10.1145/3643833.3656135(54-65)Online publication date: 27-May-2024
https://dl.acm.org/doi/10.1145/3643833.3656135
Zhao RYu JZhao HNgai E(2023)Radio2TextProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108737:3(1-28)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610873
Bolton CLong YHan JHester JFu K(2023)Characterizing and Mitigating Touchtone Eavesdropping in Smartphone Motion SensorsProceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3607199.3607203(164-178)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3607199.3607203
Long YFu K(2022)Side Auth: Synthesizing Virtual Sensors for AuthenticationProceedings of the 2022 New Security Paradigms Workshop10.1145/3584318.3584321(35-44)Online publication date: 24-Oct-2022
https://dl.acm.org/doi/10.1145/3584318.3584321
Walker PSaini SAnand SHalevi TSaxena NSuga YSakurai KDing XSako K(2022)Hearing Check FailedProceedings of the 2022 ACM on Asia Conference on Computer and Communications Security10.1145/3488932.3517399(67-81)Online publication date: 30-May-2022
https://dl.acm.org/doi/10.1145/3488932.3517399
Diamantaris MMoustakas SSun LIoannidis SPolakis JKim YKim JVigna GShi E(2021)This Sneaky Piggy Went to the Android Ad Market: Misusing Mobile Sensors for Stealthy Data ExfiltrationProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3485366(1065-1081)Online publication date: 12-Nov-2021
https://dl.acm.org/doi/10.1145/3460120.3485366
Nassi BPirutin YGalor TElovici YZadov BKim YKim JVigna GShi E(2021)Glowworm Attack: Optical TEMPEST Sound Recovery via a Device's Power Indicator LEDProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3484775(1900-1914)Online publication date: 12-Nov-2021
https://dl.acm.org/doi/10.1145/3460120.3484775
Anand SWang CLiu JSaxena NChen YPöpper CVanhoef MBatina LMayrhofer R(2021)SpearphoneProceedings of the 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks10.1145/3448300.3468499(288-299)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3448300.3468499
Farrukh HYang TXu HYin YWang HCelik Z(2021)S3Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34480855:1(1-25)Online publication date: 30-Mar-2021
https://dl.acm.org/doi/10.1145/3448085
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten