Cited By
View all- Hiraguchi HPerone PToet ACamps GBrouwer A(2023)Technology to Automatically Record Eating Behavior in Real Life: A Systematic ReviewSensors10.3390/s2318775723:18(7757)Online publication date: 8-Sep-2023
DepthGrillCam: A Mobile Application for Real-time Eating Action Recording Using RGB-D Images
Authors: 
Kento Adachi, Keiji YanaiAuthors Info & Claims
Kento Adachi, Keiji YanaiAuthors Info & ClaimsPages 55 - 59
Abstract
An automatic meal recording is one of typical applications of image recognition technology. In fact, some mobile apps on meal recording have been released so far. Most of the apps assume that a user takes a meal photo before start eating. However, this approach is not appropriate for the meals in which foods are served while taking meals such as food buffets, shared large plates and hot pots. In this study, we propose a mobile meal recording system that estimates food calories during eating in the real-time way by eating action recognition with RGB-D images obtained by a front-mounted depth sensor on a smartphone. % which is used for face recognition. In the experiments with the mobile app implemented for an iPhone, in the situation of eating grilled meat, the proposed system improved the accuracy of calorie estimation by up to 28% and recognized the correct meal category with 6.67 times higher accuracy in eating action recognition compared to the baseline system.
References
[1]
Koichi Okamoto and Keiji Yanai. Realtime eating action recognition system on a smartphone. In Proc. of IEEE International Conference on Multimedia and Expo Workshops, 2014.
[2]
Koichi. Okamoto and Keiji Yanai. GrillCam: A real-time eating action recognition system. In Proc. of International Conference on Multimedia Modeling (MMM), 2016.
[3]
Koichi Okamoto and Keiji Yanai. An automatic calorie estimation system of food images on a smartphone. In Proc. of ACM Multimedia Workshop on Multimedia Assisted Dietary Management, 2016.
[4]
Takumi Ege, Wataru Shimoda, and Keiji Yanai. A new large-scale food image segmentation dataset and its application to food calorie estimation based on grains of rice. In Proc. of the 5th International Workshop on Multimedia Assisted Dietary Management, 2019.
[5]
Elder Akpro Hippocrate Akpa, Hirohiko Suwa, Yutaka Arakawa, and Keiichi Yasumoto. Smartphone-based food weight and calorie estimation method for effective food journaling. SICE Journal of Control, Measurement, and System Integration, 10(5):360--369, 2017.
[6]
Shaobo Fang, Chang Liu, Fengqing Zhu, Edward J. Delp, and Carol J. Boushey. Single-view food portion estimation based on geometric models. In Proc. of IEEE International Symposium on Multimedia, pages 385--390, 2015.
[7]
Austin Myers, Nick Johnston, Vivek Rathod, Anoop Korattikara, Alex Gorban, Nathan Silberman, Sergio Guadarrama, George Papandreou, Jonathan Huang, and Kevin Murphy. Im2calories: towards an automated mobile vision food diary. In Proc.of IEEE International Conference on Computer Vision, 2015.
[8]
Alexandros Graikos, Vasileios Charisis, Dimitrios Iakovakis, Stelios Hadjidimitriou, and Leontios Hadjileontiadis. Single image-based food volume estimation using monocular depth-prediction networks. In Proc. of International Conference on Universal Access in Human-Computer Interaction. Applications and Practice, pages 532--543, 2020.
[9]
Ya Lu, T. Stathopoulou, and S. Mougiakakou. Partially supervised multi-task network for single-view dietary assessment. In Proc. of International Conference on Pattern Recognition (ICPR), pages 8156--8163, 2021.
[10]
Shu Naritomi and Keiji Yanai. Hungry networks: 3d mesh reconstruction of a dish and a plate from a single dish image for estimating food volume. In Proc. of ACM Multimedia Asia, 2020.
[11]
Shaobo Fang, Zeman Shao, Runyu Mao, Chichen Fu, Deborah A. Kerr, Carol J. Boushey, Edward J. Delp, and Fengqing Zhu. Single-view food portion estimation: Learning image-to-energy mappings using generative adversarial networks, arXiv:1802.09670, 2018.
[12]
Zeman Shao, Shaobo Fang, Runyu Mao, Jiangpeng He, Janine Wright, Deborah Kerr, Carol Jo Boushey, and Fengqing Zhu. Towards learning food portion from monocular images with cross-domain feature adaptation, arXiv:2103.07562, 2021.
[13]
Yoshikazu Ando, Takumi Ege, Jaehyeong Cho, and Keiji Yanai. Depthcaloriecam: A mobile application for volume-based food calorie estimation using depth cameras. In Proc. of ACM Multimedia Workshop on Multimedia Assisted Dietary Management, 2019.
[14]
Quin Thames, Arjun Karpur, Wade Norris, Fangting Xia, Liviu Panait, Tobias Weyand, and Jack Sim. Nutrition5k: Towards automatic nutritional understanding of generic food. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8903--8911, 2021.
[15]
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation, arXiv:1706.05587, 2017.
[16]
Kaimu Okamoto and Keiji Yanai. UEC-FoodPIX Complete: A large-scale food image segmentation dataset. In Proc.of ICPR Workshop on Multimedia Assisted Dietary Management, 2021
Index Terms
- DepthGrillCam: A Mobile Application for Real-time Eating Action Recording Using RGB-D Images
Recommendations
DietCam: Automatic dietary assessment with mobile camera phones
Obesity has become a severe health problem in developed countries, and a healthy food intake has been recognized as the key factor for obesity prevention. This paper presents a mobile phone based system, DietCam, to help assess food intakes with few ...
Image-Based Food Calorie Estimation Using Knowledge on Food Categories, Ingredients and Cooking Directions
Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017Image-based food calorie estimation is crucial to diverse mobile applications for recording everyday meal. However, some of them need human help for calorie estimation, and even if it is automatic, food categories are often limited or images from ...
Saliency-Aware Class-Agnostic Food Image Segmentation
Survey PaperAdvances in image-based dietary assessment methods have allowed nutrition professionals and researchers to improve the accuracy of dietary assessment, where images of food consumed are captured using smartphones or wearable devices. These images are ...
Comments
Information & Contributors
Information
Published In
October 2022
97 pages
ISBN:9781450395021
DOI:10.1145/3552484
- General Chairs:
- Stavroula Mougiakakou,
- Giovanni Maria Farinella,
- Keiji Yanai,
- Dario Allegra
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- JSPS
Conference
MM '22
Sponsor:
Acceptance Rates
MADiMa '22 Paper Acceptance Rate 9 of 10 submissions, 90%;
Overall Acceptance Rate 16 of 24 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 109Total Downloads
- Downloads (Last 12 months)45
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
Cited By
View all- Hiraguchi HPerone PToet ACamps GBrouwer A(2023)Technology to Automatically Record Eating Behavior in Real Life: A Systematic ReviewSensors10.3390/s2318775723:18(7757)Online publication date: 8-Sep-2023
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in