poster

Automatically generating stories from sensor data

Authors:

Joseph Reddington,

Nava TintarevAuthors Info & Claims

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

Pages 407 - 410

https://doi.org/10.1145/1943403.1943477

Published: 13 February 2011 Publication History

Abstract

Recent research in Augmented and Alternative Communication (AAC) has begun to make use of Natural Language Generation (NLG) techniques. This creates an opportunity for constructing stories from sensor data, akin to existing work in life-logging. This paper examines the potential of using NLG to merge the AAC and life-logging domains. It proposes a four stage hierarchy that categorises levels of complexity of output text. It formulates a key subproblem of clustering sensor data into narrative events and describes three potential approaches for resolving this subproblem.

References

[1]

S. Ashraf, A. Judson, I. W. Ricketts, A. Waller, N. Alm, B. Gordon, F. MacAulay, J. K. Brodie, M. Etchels, A. Warden, and A. J. Shearer. Capturing phrases for ICU-Talk, a communication aid for intubated intensive care patients. In ACM Conference on Assistive technologies, pp. 213--217, New York, NY, USA, 2002.

Digital Library

[2]

R. Black, R. Turner, E. Reiter, and A. Waller. A new tool to support interactive narrative experience for children with communication disorders. Conference of the International Society for Augmentative and Alternative Communication, Barcelona, July 2010.

[3]

M. Dempster, N. Alm, and E. Reiter. Automatic generation of conversational utterances and narrative for autmentative and alternative communication: a prototype system. NAACL HLT SLPAT Workshop, pp. 1--9, Los Angeles, California, June 2010.

Digital Library

[4]

N. Eagle and A. S. Pentland. Eigenbehaviors: Identifying structure in routine. Behavioral Ecology and Sociobiology, 63:7:1057--1066, 2009.

[5]

D. Ellis and K. Lee. Minimal-impact audio-based personal archives. Workshop on continuous archival and retrieval of personal experiences in Multimedia, 2004.

Digital Library

[6]

J. Gemmell, L. Williams, K. Wood, R. Lueder, and G. Bell. Passive capture and ensuing issues for a personal lifetime store. In CARPE Workshop in Assoc. with the ACM Multimedia Con, pp. 48--55. 2004.

Digital Library

[7]

D. J. Higginbotham, H. Shane, S. Russell, and K. Caves. Access to AAC: Present, past, and future. Augmentative and Alternative Communication, 23(3):243--257, January 2007.

[8]

S. Mann. Continuous lifelong capture of personal experience with eyetap. In CARPE Workshop in Assoc. with the ACM Multimedia Conference, 2004.

Digital Library

[9]

K. F. Mccoy and P. Demasco. Some applications of natural language processing to the field of augmentative and alternative communication. In Proceedings of the IJCAI-95 Workshop on Developing AI Applications for People with Disabilities, pp. 97--112. Press, 1995.

[10]

M. Raento, A. Oulasvirta, R. Petit, and H. Toivonen. Contextphone: A prototyping platform for context-aware mobile applications. IEEE Pervasive Computing, 4:51--59, 2005.

Digital Library

[11]

E. Reiter and R. Dale. Building natural language generation systems, 2000.

Digital Library

[12]

E. Reiter, A. Gatt, F. Portet, and M. van der Meulen. The importance of narrative and other lessons from an evaluation of an NLG system that summarises clinical data. INLG '08, pp. 147--156, Morristown, NJ, USA, 2008. Association for Computational Linguistics.

Digital Library

[13]

E. Reiter, R. Turner, N. Alm, R. Black, M. Dempster, and A. Waller. Using NLG to help language-impaired users tell stories and participate in social dialogues. European Workshop on Natural Language Generation (ENLG-09), 2009.

Digital Library

[14]

G. Soto, E. Hartmann, and D. Wilkins. Exploring the elements of narrative that emerge in the interactions between an 8-year-old child who uses an AAC device and her teacher. Augmentative and Alternative Communication 22(4), 2006.

[15]

E. Goldberg, N. Driedger, and R. I. Kittredge. Using natural-language processing to produce weather forecasts. IEEE Expert: Intelligent Systems and Their Applications, 9(2):45--53, 1994.

Digital Library

Cited By

Curtis HNeate TVazquez Gonzalez C(2022)State of the Art in AAC: A Systematic Review and TaxonomyProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544810(1-22)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544810
You CYuan CBi NHung MHuang PWang HKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Go Gig or Go Home: Enabling Social Sensing to Share Personal Data with Intimate Partner for the Health and Wellbeing of Long-Hour workersProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445278(1-16)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445278
Waller A(2018)Telling tales: unlocking the potential of AAC technologiesInternational Journal of Language & Communication Disorders10.1111/1460-6984.1244954:2(159-169)Online publication date: 30-Dec-2018
https://doi.org/10.1111/1460-6984.12449
Show More Cited By

Index Terms

Automatically generating stories from sensor data
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

The effect of narrative element incorporation on physical activity and game experience in active and sedentary virtual reality games
Abstract
Narratives are pervasive in video games and have been found to increase physical activity in active video games. However, the effect of incorporating narrative elements has seldom been examined in fully immersive virtual reality games. We ...
Storyspace: a story-driven approach for creating museum narratives
HT '12: Proceedings of the 23rd ACM conference on Hypertext and social media

In a curated exhibition of a museum or art gallery, a selection of heritage objects and associated information is presented to a visitor for the purpose of telling a story about them. The same underlying story can be presented in a number of different ...
A Case Study in the Design of Interactive Narrative: The Subversion of the Interface

There is a potential conflict in the design of interactive narratives. The exercise of interaction in digital environments, including games, may interfere with the experience of story. The article uses the interactive CDROMCEREMONYOFINNOCENCE as a case ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

February 2011

504 pages

ISBN:9781450304191

DOI:10.1145/1943403

General Chairs:
Pearl Pu
EPFL, Switzerland
,
Michael Pazzani
Rutgers University, USA
,
Program Chairs:
Elisabeth André
Augsburg University, Germany
,
Doug Riecken
CCLS Columbia University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 February 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

IUI '11

Sponsor:

IUI '11: 16th International Conference on Intelligent User Interfaces

February 13 - 16, 2011

CA, Palo Alto, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
355
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Curtis HNeate TVazquez Gonzalez C(2022)State of the Art in AAC: A Systematic Review and TaxonomyProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544810(1-22)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544810
You CYuan CBi NHung MHuang PWang HKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Go Gig or Go Home: Enabling Social Sensing to Share Personal Data with Intimate Partner for the Health and Wellbeing of Long-Hour workersProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445278(1-16)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445278
Waller A(2018)Telling tales: unlocking the potential of AAC technologiesInternational Journal of Language & Communication Disorders10.1111/1460-6984.1244954:2(159-169)Online publication date: 30-Dec-2018
https://doi.org/10.1111/1460-6984.12449
(2018)Personal storytellingInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2016.04.00592:C(1-16)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1016/j.ijhcs.2016.04.005
Albayrak MTurhan K(2017)E-survey based approach for pervasive healthcare2017 Medical Technologies National Congress (TIPTEKNO)10.1109/TIPTEKNO.2017.8238077(1-4)Online publication date: Oct-2017
https://doi.org/10.1109/TIPTEKNO.2017.8238077
Triantafyllidis AVelardo CSalvi DShah SKoutkias VTarassenko L(2017)A Survey of Mobile Phone Sensing, Self-Reporting, and Social Sharing for Pervasive HealthcareIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2015.248390221:1(218-227)Online publication date: Jan-2017
https://doi.org/10.1109/JBHI.2015.2483902
Shin SUm JSeo DChoi SLee SJung HYi M(2014)Platform to Build the Knowledge Base by Combining Sensor Data and Context DataInternational Journal of Distributed Sensor Networks10.1155/2014/54276410:1(542764)Online publication date: Jan-2014
https://doi.org/10.1155/2014/542764
Curry JZhu WCraig BTurpin LBokhari MMhasavekar PHill RKuhl M(2013)Using a natural language generation approach to document simulation resultsProceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World10.5555/2675983.2676246(2116-2126)Online publication date: 8-Dec-2013
https://dl.acm.org/doi/10.5555/2675983.2676246
Reddington J(2013)The Domesday datasetJournal of Intellectual Disabilities10.1177/174462951348265017:2(107-121)Online publication date: 22-Mar-2013
https://doi.org/10.1177/1744629513482650
Samaras CVakali AGiatsoglou MChatzakou DAngelis LKetikidis PMargaritis KVlahavas IChatzigeorgiou AEleftherakis GStamelos I(2013)Requirements and architecture design principles for a smart city experiment with sensor and social networks integrationProceedings of the 17th Panhellenic Conference on Informatics10.1145/2491845.2491887(327-334)Online publication date: 19-Sep-2013
https://dl.acm.org/doi/10.1145/2491845.2491887
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten