research-article

Free access

Report on the second NLG challenge on generating instructions in virtual environments (GIVE-2)

Authors:

Jon OberlanderAuthors Info & Claims

INLG '10: Proceedings of the 6th International Natural Language Generation Conference

Pages 243 - 250

Published: 07 July 2010 Publication History

PDF eReader

Abstract

We describe the second installment of the Challenge on Generating Instructions in Virtual Environments (GIVE-2), a shared task for the NLG community which took place in 2009--10. We evaluated seven NLG systems by connecting them to 1825 users over the Internet, and report the results of this evaluation in terms of objective and subjective measures.

References

[1]

Andrew Gargett, Konstantina Garoufi, Alexander Koller, and Kristina Striegnitz. 2010. The GIVE-2 corpus of giving instructions in virtual environments. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC), Malta.

Google Scholar

[2]

S. D. Gosling, S. Vazire, S. Srivastava, and O. P. John. 2004. Should we trust Web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59:93--104.

Crossref

Google Scholar

[3]

A. Koller, D. Byron, J. Cassell, R. Dale, J. Moore, J. Oberlander, and K. Striegnitz. 2009a. The software architecture for the first challenge on generating instructions in virtual environments. In Proceedings of the EACL-09 Demo Session.

Digital Library

Google Scholar

[4]

Alexander Koller, Kristina Striegnitz, Donna Byron, Justine Cassell, Robert Dale, Sara Dalzel-Job, Johanna Moore, and Jon Oberlander. 2009b. Validating the web-based evaluation of NLG systems. In Proceedings of ACL-IJCNLP 2009 (Short Papers), Singapore.

Digital Library

Google Scholar

[5]

Alexander Koller, Kristina Striegnitz, Donna Byron, Justine Cassell, Robert Dale, Johanna Moore, and Jon Oberlander. 2010. The first challenge on generating instructions in virtual environments. In E. Krahmer and M. Theune, editors, Empirical Methods in Natural Language Generation, volume 5790 of LNCS, pages 337--361. Springer.

Digital Library

Google Scholar

Cited By

View all

Williams TYazdani FSuresh PScheutz MBeetz M(2019)Dempster-Shafer theoretic resolution of referential ambiguityAutonomous Robots10.1007/s10514-018-9795-543:2(389-414)Online publication date: 16-Mar-2019
https://dl.acm.org/doi/10.1007/s10514-018-9795-5
Benotti LLau TVillalba M(2014)Interpreting Natural Language Instructions Using Language, Vision, and BehaviorACM Transactions on Interactive Intelligent Systems (TiiS)10.1145/26296324:3(1-22)Online publication date: 11-Aug-2014
https://dl.acm.org/doi/10.1145/2629632
Poesio MChamberlain JKruschwitz URobaldo LDucceschi L(2013)Phrase detectivesACM Transactions on Interactive Intelligent Systems (TiiS)10.1145/2448116.24481193:1(1-44)Online publication date: 24-Apr-2013
https://dl.acm.org/doi/10.1145/2448116.2448119
Show More Cited By

Recommendations

Report on the first NLG Challenge on Generating Instructions in Virtual Environments (GIVE)
ENLG '09: Proceedings of the 12th European Workshop on Natural Language Generation

We describe the first installment of the Challenge on Generating Instructions in Virtual Environments (GIVE), a new shared task for the NLG community. We motivate the design of the challenge, describe how we carried it out, and discuss the results of ...
The first challenge on generating instructions in virtual environments
Empirical methods in natural language generation

This paper describes the First Challenge on Generating Instructions in Virtual Environments (GIVE-1). GIVE is a shared task for generation systems which give real-time natural-language instructions to users in a virtual 3D world. These systems are ...
Report on the second second challenge on generating instructions in virtual environments (GIVE-2.5)
ENLG '11: Proceedings of the 13th European Workshop on Natural Language Generation

GIVE-2.5 evaluates eight natural language generation (NLG) systems that guide human users through solving a task in a virtual environment. The data is collected via the Internet, and to date, 536 interactions of subjects with one of the NLG systems have ...

Comments

Information & Contributors

Information

Published In

INLG '10: Proceedings of the 6th International Natural Language Generation Conference

July 2010

266 pages

Program Chairs:
John Kelleher
Dublin Institute of Technology, Ireland
,
Brian Mac Namee
Dublin Institute of Technology, Ireland
,
Ielka van der Sluis
Trinity College Dublin, Ireland

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 07 July 2010

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
173
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)13

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Williams TYazdani FSuresh PScheutz MBeetz M(2019)Dempster-Shafer theoretic resolution of referential ambiguityAutonomous Robots10.1007/s10514-018-9795-543:2(389-414)Online publication date: 16-Mar-2019
https://dl.acm.org/doi/10.1007/s10514-018-9795-5
Benotti LLau TVillalba M(2014)Interpreting Natural Language Instructions Using Language, Vision, and BehaviorACM Transactions on Interactive Intelligent Systems (TiiS)10.1145/26296324:3(1-22)Online publication date: 11-Aug-2014
https://dl.acm.org/doi/10.1145/2629632
Poesio MChamberlain JKruschwitz URobaldo LDucceschi L(2013)Phrase detectivesACM Transactions on Interactive Intelligent Systems (TiiS)10.1145/2448116.24481193:1(1-44)Online publication date: 24-Apr-2013
https://dl.acm.org/doi/10.1145/2448116.2448119
Benotti LVillalba MLau TCerruti JLi HLin COsborne M(2012)Corpus-based interpretation of instructions in virtual environmentsProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 210.5555/2390665.2390711(181-186)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390665.2390711
Sabou MBontcheva KScharl ALindstaedt SGranitzer M(2012)Crowdsourcing research opportunitiesProceedings of the 12th International Conference on Knowledge Management and Knowledge Technologies10.1145/2362456.2362479(1-8)Online publication date: 5-Sep-2012
https://dl.acm.org/doi/10.1145/2362456.2362479
Akkersdijk SLangenbach MLoch FTheune MGardent CStriegnitz K(2011)The Thumbs Up!Proceedings of the 13th European Workshop on Natural Language Generation10.5555/2187681.2187739(312-317)Online publication date: 28-Sep-2011
https://dl.acm.org/doi/10.5555/2187681.2187739
Denis AGardent CStriegnitz K(2011)The Loria instruction generation system L in GIVE 2.5Proceedings of the 13th European Workshop on Natural Language Generation10.5555/2187681.2187737(302-306)Online publication date: 28-Sep-2011
https://dl.acm.org/doi/10.5555/2187681.2187737
Striegnitz KDenis AGargett AGaroufi KKoller ATheune MGardent CStriegnitz K(2011)Report on the second second challenge on generating instructions in virtual environments (GIVE-2.5)Proceedings of the 13th European Workshop on Natural Language Generation10.5555/2187681.2187732(270-279)Online publication date: 28-Sep-2011
https://dl.acm.org/doi/10.5555/2187681.2187732
Benotti LDenis AMoore JTraum DChai JPassonneau R(2011)Giving instructions in virtual environments by corpus based selectionProceedings of the SIGDIAL 2011 Conference10.5555/2132890.2132900(68-77)Online publication date: 17-Jun-2011
https://dl.acm.org/doi/10.5555/2132890.2132900
Benotti LDenis AKurohashi S(2011)Prototyping virtual instructors from human-human corporaProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Systems Demonstrations10.5555/2002440.2002451(62-67)Online publication date: 21-Jun-2011
https://dl.acm.org/doi/10.5555/2002440.2002451
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Recommendations

Report on the first NLG Challenge on Generating Instructions in Virtual Environments (GIVE)

The first challenge on generating instructions in virtual environments

Report on the second second challenge on generating instructions in virtual environments (GIVE-2.5)

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations