poster

SipBit: A Sensing Platform to Recognize Beverage Type, Volume, and Sugar Content Using Electrical Impedance Spectroscopy and Deep Learning

Authors:

Chamath Amarasinghe,

Nimesha RanasingheAuthors Info & Claims

CHI EA '22: CHI Conference on Human Factors in Computing Systems Extended Abstracts

Article No.: 371, Pages 1 - 8

https://doi.org/10.1145/3491101.3519713

Published: 28 April 2022 Publication History

Abstract

We present SipBit, a sensing platform that digitally recognizes beverages and their attributes, an essential component in facilitating novel human-food interactions. SipBit consists of an electrical impedance measurement unit and a recognition method based on deep learning techniques. First, impedance measurements of a beverage are acquired using Electrical Impedance Spectroscopy. Then, a multi-task network cascade algorithm was employed to identify eight different beverage types in various volume levels and sugar concentrations. Results show that the multi-task network cascade discriminates beverage types with an accuracy of 96.32%, and estimates volumes with a root mean square error of 13.74ml and sugar content with a root mean square error of 7.99gdm− 3. Future work will include: 1) developing utensils embedded with SipBit for automatic beverage and attribute recognition, and 2) further developing SipBit to recognize additional beverage types and their attributes, thus enabling a new avenue for designing human-food interactive technologies.

Supplementary Material

MP4 File (3491101.3519713-video-preview.mp4)

Video Preview

Download
35.11 MB

MP4 File (3491101.3519713-talk-video.mp4)

Talk Video

Download
74.13 MB

References

[1]

Ferran Altarriba Bertran, Danielle Wilde, Ernő Berezvay, and Katherine Isbister. 2019. Playful Human-Food Interaction Research: State of the Art and Future Directions. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY ’19). Association for Computing Machinery, New York, NY, USA, 225–237. https://doi.org/10.1145/3311350.3347155

Digital Library

[2]

Felicia Cordeiro, Daniel A. Epstein, Edison Thomaz, Elizabeth Bales, Arvind K. Jagannathan, Gregory D. Abowd, and James Fogarty. 2015. Barriers and Negative Nudges: Exploring Challenges in Food Journaling. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1159–1162. https://doi.org/10.1145/2702123.2702155

Digital Library

[3]

Jifeng Dai, Kaiming He, and Jian Sun. 2016. Instance-Aware Semantic Segmentation via Multi-task Network Cascades. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, New York, NY, USA, 3150–3158. https://doi.org/10.1109/CVPR.2016.343

[4]

Chirantan Das, Subhadip Chakraborty, Anupam Karmakar, and Sanatan Chattopadhyay. 2018. On-chip detection and quantification of soap as an adulterant in milk employing electrical impedance spectroscopy. In 2018 International Symposium on Devices, Circuits and Systems (ISDCS). IEEE, New York, NY, USA, 1–4. https://doi.org/10.1109/ISDCS.2018.8379634

[5]

Ashutosh Dhekne, Mahanth Gowda, Yixuan Zhao, Haitham Hassanieh, and Romit Roy Choudhury. 2018. LiquID: A Wireless Liquid IDentifier. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (Munich, Germany) (MobiSys ’18). Association for Computing Machinery, New York, NY, USA, 442–454. https://doi.org/10.1145/3210240.3210345

Digital Library

[6]

Marco Grossi and Bruno Riccò. 2017. Electrical impedance spectroscopy (EIS) for biological analysis and food characterization: A review. Journal of sensors and sensor systems 6, 2 (2017), 303–325.

[7]

Yongzhi Huang, Kaixin Chen, Yandao Huang, Lu Wang, and Kaishun Wu. 2021. Vi-Liquid: Unknown Liquid Identification with Your Smartphone Vibration. Association for Computing Machinery, New York, NY, USA, 174–187. https://doi.org/10.1145/3447993.3448621

[8]

Ian T Jolliffe and Jorge Cadima. 2016. Principal component analysis: a review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 374, 2065 (2016), 20150202.

[9]

Yoshiyuki Kawano and Keiji Yanai. 2015. Foodcam: A real-time food recognition system on a smartphone. Multimedia Tools and Applications 74, 14 (2015), 5263–5287.

Digital Library

[10]

Fanyu Kong and Jindong Tan. 2012. DietCam: Automatic dietary assessment with mobile camera phones. Pervasive and Mobile Computing 8, 1 (2012), 147–163.

Digital Library

[11]

Chang Liu, Yu Cao, Yan Luo, Guanling Chen, Vinod Vokkarane, and Yunsheng Ma. 2016. DeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment. In Inclusive Smart Cities and Digital Health, Carl K. Chang, Lorenzo Chiari, Yu Cao, Hai Jin, Mounir Mokhtari, and Hamdi Aloulou (Eds.). Springer International Publishing, Cham, 37–48.

[12]

António M Lopes, JA Tenreiro Machado, Elisa Ramalho, and Vânia Silva. 2018. Milk characterization using electrical impedance spectroscopy and fractional models. Food Analytical Methods 11, 3 (2018), 901–912.

[13]

Simon Mezgec and Barbara Koroušić Seljak. 2017. NutriNet: a deep learning food and drink image recognition system for dietary assessment. Nutrients 9, 7 (2017), 657.

[14]

Paritosh Pandey, Akella Deepthi, Bappaditya Mandal, and Niladri B Puhan. 2017. FoodNet: Recognizing foods using ensemble of deep networks. IEEE Signal Processing Letters 24, 12 (2017), 1758–1762.

[15]

Uwe Pliquett. 2010. Bioimpedance: a review for food processing. Food engineering reviews 2, 2 (2010), 74–94.

[16]

Parisa Pouladzadeh, Pallavi Kuhad, Sri Vijay Bharat Peddi, Abdulsalam Yassine, and Shervin Shirmohammadi. 2016. Food calorie measurement using deep learning neural network. In 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings. IEEE, New York, NY, USA, 1–6. https://doi.org/10.1109/I2MTC.2016.7520547

[17]

Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, and Ellen Yi-Luen Do. 2017. Virtual Lemonade: Let’s Teleport Your Lemonade!. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (Yokohama, Japan) (TEI ’17). Association for Computing Machinery, New York, NY, USA, 183–190. https://doi.org/10.1145/3024969.3024977

Digital Library

[18]

Nimesha Ranasinghe, Meetha Nesam James, Michael Gecawicz, Jonathan Bland, and David Smith. 2020. Influence of Electric Taste, Smell, Color, and Thermal Sensory Modalities on the Liking and Mediated Emotions of Virtual Flavor Perception. Association for Computing Machinery, New York, NY, USA, 296–304. https://doi.org/10.1145/3382507.3418862

[19]

Nimesha Ranasinghe, Thi Ngoc Tram Nguyen, Yan Liangkun, Lien-Ya Lin, David Tolley, and Ellen Yi-Luen Do. 2017. Vocktail: A Virtual Cocktail for Pairing Digital Taste, Smell, and Color Sensations. In Proceedings of the 25th ACM International Conference on Multimedia (Mountain View, California, USA) (MM ’17). Association for Computing Machinery, New York, NY, USA, 1139–1147. https://doi.org/10.1145/3123266.3123440

Digital Library

[20]

Rigol technologies, Inc 2017. DSA800/E Series Spectrum Analyzer (Datasheet). Rigol technologies, Inc. $https://www.rigolna.com/products/spectrum-analyzers/dsa800/$N/A.

[21]

Mrinmoy Roy and BK Yadav. 2021. Electronic nose for detection of food adulteration: A review. Journal of Food Science and Technology 59, 4 (2021), 1–13.

[22]

Shota Sasano, Xian-Hua Han, and Yen-Wei Chen. 2016. Food recognition by combined bags of color features and texture features. In 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). IEEE, New York, NY, USA, 815–819. https://doi.org/10.1109/CISP-BMEI.2016.7852822

[23]

C Soares, JA Tenreiro Machado, António M Lopes, E Vieira, and C Delerue-Matos. 2020. Electrochemical impedance spectroscopy characterization of beverages. Food chemistry 302(2020), 125345.

[24]

Yusuke Tahara and Kiyoshi Toko. 2013. Electronic tongues–a review. IEEE Sensors Journal 13, 8 (2013), 3001–3011.

[25]

Juzhong Tan and Jie Xu. 2020. Applications of electronic nose (e-nose) and electronic tongue (e-tongue) in food quality-related properties determination: A review. Artificial Intelligence in Agriculture 4 (2020), 104–115. https://doi.org/10.1016/j.aiia.2020.06.003

[26]

Ryosuke Tanno, Koichi Okamoto, and Keiji Yanai. 2016. DeepFoodCam: A DCNN-Based Real-Time Mobile Food Recognition System. In Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management(Amsterdam, The Netherlands) (MADiMa ’16). Association for Computing Machinery, New York, NY, USA, 89. https://doi.org/10.1145/2986035.2986044

Digital Library

[27]

Ju Wang, Jie Xiong, Xiaojiang Chen, Hongbo Jiang, Rajesh Krishna Balan, and Dingyi Fang. 2017. TagScan: Simultaneous Target Imaging and Material Identification with Commodity RFID Devices. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (Snowbird, Utah, USA) (MobiCom ’17). Association for Computing Machinery, New York, NY, USA, 288–300. https://doi.org/10.1145/3117811.3117830

Digital Library

[28]

Binbin Xie, Jie Xiong, Xiaojiang Chen, Eugene Chai, Liyao Li, Zhanyong Tang, and Dingyi Fang. 2019. Tagtag: Material Sensing with Commodity RFID. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems (New York, New York) (SenSys ’19). Association for Computing Machinery, New York, NY, USA, 338–350. https://doi.org/10.1145/3356250.3360027

Digital Library

[29]

Shulin Yang, Mei Chen, Dean Pomerleau, and Rahul Sukthankar. 2010. Food recognition using statistics of pairwise local features. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, USA, 2249–2256. https://doi.org/10.1109/CVPR.2010.5539907

[30]

Xue Ying. 2019. An Overview of Overfitting and its Solutions. Journal of Physics: Conference Series 1168 (feb 2019), 022022. https://doi.org/10.1088/1742-6596/1168/2/022022

[31]

Yixuan Zhang and Andrea G. Parker. 2020. Eat4Thought: A Design of Food Journaling. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3383044

Digital Library

[32]

Lei Zhou, Chu Zhang, Fei Liu, Zhengjun Qiu, and Yong He. 2019. Application of deep learning in food: a review. Comprehensive Reviews in Food Science and Food Safety 18, 6(2019), 1793–1811.

Cited By

Amarasinghe CRanasinghe N(2024)Digital Food Sensing and Ingredient Analysis Techniques to Facilitate Human-Food Interface DesignsACM Computing Surveys10.1145/368567557:1(1-39)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3685675
Mayumi DNakamura YMatsuda YMisaki SYasumoto K(2024)Aromug: A Mug-Type Olfactory Interface to Enhance the Sweetness Perception of BeveragesIEEE Access10.1109/ACCESS.2024.340139212(78366-78378)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3401392
Liu MZhou BRey VBian SLukowicz P(2024)iEat: automatic wearable dietary monitoring with bio-impedance sensingScientific Reports10.1038/s41598-024-67765-514:1Online publication date: 2-Aug-2024
https://doi.org/10.1038/s41598-024-67765-5

Recommendations

A Pulsed Approach for Electrical Impedance Spectroscopy Measurement
ISDEA '10: Proceedings of the 2010 International Conference on Intelligent System Design and Engineering Application - Volume 01

A pulsed method for impedance spectroscopy measurement, different from the traditional swept frequency methods, is presented. A string of square-wave pulses is digitally generated as the excitation signal to the sensing electrodes. A pulsed measurement ...
iEat: Human-food interaction with bio-impedance sensing
UbiComp/ISWC '23 Adjunct: Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing

We explore an atypical use of bio-impedance by leveraging the unique temporal impedance patterns caused by the dynamic circuit changes between a pair of electrodes due to the body motions, and interactions with metal utensils and food during dining ...
Controlling Temporal Change of a Beverage's Taste Using Electrical Stimulation
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

In this paper, we discuss concepts for improving beverage experience using electrical stimulation in terms of the requirements for the procedure (design space), previous study (completion), and limitations of conventional technologies. To improve ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

3066 pages

ISBN:9781450391566

DOI:10.1145/3491101

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster
Research
Refereed limited

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
250
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)15

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Amarasinghe CRanasinghe N(2024)Digital Food Sensing and Ingredient Analysis Techniques to Facilitate Human-Food Interface DesignsACM Computing Surveys10.1145/368567557:1(1-39)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3685675
Mayumi DNakamura YMatsuda YMisaki SYasumoto K(2024)Aromug: A Mug-Type Olfactory Interface to Enhance the Sweetness Perception of BeveragesIEEE Access10.1109/ACCESS.2024.340139212(78366-78378)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3401392
Liu MZhou BRey VBian SLukowicz P(2024)iEat: automatic wearable dietary monitoring with bio-impedance sensingScientific Reports10.1038/s41598-024-67765-514:1Online publication date: 2-Aug-2024
https://doi.org/10.1038/s41598-024-67765-5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents