Smart-Pikachu: Extending Interactivity of Stuffed Animals with Large Language Models
Authors: 
Toma Itagaki, Richard LiAuthors Info & Claims
Toma Itagaki, Richard LiAuthors Info & ClaimsArticle No.: 114, Pages 1 - 2
Abstract
We propose Smart-Pikachu, a stuffed animal equipped with sensing and actuation to explore the use of large language models (LLM’s) with sensor data inputs. The augmentation of pressure sensing will allow for the LLM to interpret various interactions such as hugs and handshakes with the user. Furthermore, the actuation capabilities will extend our system’s interactivity by providing physical feedback to the user. We will also incorporate text-to-speech output from the LLM to add another mode of interaction between the system and user. In this Student Innovation Challenge, we intend to explore applications at the intersection of sensing and interaction through LLM’s and demonstrate an extension of LLMs’ multimodal capabilities.
References
[1]
Mike Binnard and Mark R Cutkosky. 2000. Design by composition for layered manufacturing. J. Mech. Des. 122, 1 (2000), 91–101.
[2]
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023).
[3]
Aaron M Dollar and Robert D Howe. 2006. A robust compliant grasper via shape deposition manufacturing. IEEE/ASME transactions on mechatronics 11, 2 (2006), 154–161.
[4]
Ali Kiaghadi, Jin Huang, Seyedeh Zohreh Homayounfar, Trisha Andrew, and Deepak Ganesan. 2022. FabToys: plush toys with large arrays of fabric-based pressure sensors to enable fine-grained interaction detection. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 1–13.
[5]
Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy, Jacob Sunshine, Jiening Zhan, Ming-Zher Poh, Shun Liao, Paolo Di Achille, and Shwetak Patel. 2023. Large Language Models are Few-Shot Health Learners. arXiv preprint arXiv:2305.15525 (2023).
[6]
Yue Wu, So Yeon Min, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Yuanzhi Li, Tom Mitchell, and Shrimai Prabhumoye. 2023. Plan, Eliminate, and Track–Language Models are Good Teachers for Embodied Agents. arXiv preprint arXiv:2305.02412 (2023).
Index Terms
- Smart-Pikachu: Extending Interactivity of Stuffed Animals with Large Language Models
Recommendations
Extending interaction for smart watches: enabling bimanual around device control
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing SystemsThe size of a smart watch limits the available interactive surface for the user. Most current smart watches use a combination of a touch screen and physical buttons. Unfortunately, a small touch screen's usability is limited when it can be easily ...
Interactivity: Using Expressy to Demonstrate Expressiveness in Touch-based Interactions
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsWe present an interactivity demonstration of Expressy. A system that augments existing touchscreen devices with a variety of continuous expressive interaction capabilities, using movement data from a wrist-worn IMU. Our demonstration comprises a set of ...
On Implementing Case-Based Reasoning with Large Language Models
Case-Based Reasoning Research and DevelopmentAbstractSystems based on Large Language Models (LLMs), such as ChatGPT, have impressive performance but also well-known issues with erroneous output. Retrieval Augmented Generation (RAG), which typically presents the LLM with text snippets of additional ...
Comments
Information & Contributors
Information
Published In
October 2023
424 pages
ISBN:9798400700965
DOI:10.1145/3586182
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 October 2023
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
UIST '23: The 36th Annual ACM Symposium on User Interface Software and Technology
October 29 - November 1, 2023
CA, San Francisco, USA
Acceptance Rates
Overall Acceptance Rate 842 of 3,967 submissions, 21%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 154Total Downloads
- Downloads (Last 12 months)154
- Downloads (Last 6 weeks)24
Reflects downloads up to 29 Jul 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format