short-paper

Towards context-aware automated writing evaluation systems

Authors:

Pierre-André Patout,

Maxime CordyAuthors Info & Claims

EASEAI 2019: Proceedings of the 1st ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence

Pages 17 - 20

https://doi.org/10.1145/3340435.3342722

Published: 26 August 2019 Publication History

Abstract

Writing is a crucial skill in our society, which is regularly exerted by students across all disciplines. Automated essay scoring and automatic writing evaluation systems can support professors in the evaluation of written texts and, conversely, help students improving their writing. However, most of those systems fail to consider the context of the writing, such as the targeted audience and the genre. In this paper, we depict our vision towards new-generation AES systems that could evaluate written products while considering their specific context. In education, such tools could support students not only in adapting their written product to their particular context, but also in identifying points for improvement and situational settings where their writing is less proficient.

References

[1]

Scott A Crossley, Danielle McNamara, and Com. 2010. Cohesion, coherence, and expert evaluations of writing proficiency. Proceedings of the 32nd Annual Conference of the Cognitive Science Society.

[2]

Scott A Crossley and Danielle McNamara. 2011. Text Coherence and Judgments of Essay Quality: Models of Quality and Coherence. Proceedings of the 29th Annual Conference of the Cognitive Science Society.

[3]

Jean-Michel Adam. 2011. La linguistique textuelle. A. Colin Eds.

[4]

Evelin Amorim, Marcia Cançado, and Adriano Veloso. 2018. Automated Essay Scoring in the Presence of Biased Ratings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana.

[5]

Scott Crossley, Laura K. Allen, Erica L. Snow, and Danielle S. McNamara. 2015. Pssst... Textual Features... There is More to Automatic Essay Scoring Than Just You!. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (LAK ’15). 203–207.

Digital Library

[6]

Scott A. Crossley, Kristopher Kyle, and Danielle S. McNamara. 2016. The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods 48, 4 (2016), 1227–1237.

[7]

A.E. Dalcq. 1999. Mettre de l’ordre dans ses idées: classification des articulations logiques pour structurer son texte. Duculot.

[8]

Fei Dong and Yue Zhang. 2016. Automatic Features for Essay Scoring - An Empirical Study. In EMNLP.

[9]

Michael A.K. Halliday and Ruqaiya Hasan. 1976. Cohesion in English. Longman, London.

[10]

Judith Kamalski, Ted Sanders, and L.R. Lentz. 2008. Coherence Marking, Prior Knowledge, and Comprehension of Informative and Persuasive Texts: Sorting Things Out. Discourse Processes 45 (07 2008).

[11]

Mirella Lapata and Regina Barzilay. 2005. Automatic Evaluation of Text Coherence: Models and Representations. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI’05). 1085–1090.

Digital Library

[12]

Yong-Won Lee, Claudia Gentile, and R Kantor. 2010. Toward Automated Multitrait Scoring of Essays: Investigating Links among Holistic, Analytic, and Text Feature Scores. Applied Linguistics - APPL LINGUIST 31 (07 2010), 391–417.

[13]

Danielle McNamara. 2001. Reading both high and low coherence texts: Effects of text sequence and prior knowledge. Canadian journal of experimental psychology = Revue canadienne de psychologie expérimentale 55 (03 2001), 51–62.

[14]

Danielle McNamara, Scott A. Crossley, Rod Roscoe, Laura K. Allen, and Jianmin Dai. 2015. A hierarchical classification approach to automated essay scoring. Assessing Writing 23 (1 1 2015), 35–59.

[15]

Danielle McNamara and Walter Kintsch. 1996. Learning From Texts: Effects of Prior Knowledge and Text Coherence. Discourse Processes 22 (10 1996), 247–288.

[16]

Danielle S. McNamara, Arthur C. Graesser, Philip M. McCarthy, and Zhiqiang Cai. 2014. Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press, New York, NY, USA.

Digital Library

[17]

Fayol Michel. 2000. Des idées au texte, psychologie cognitive de la production verbale, orale et écrite. Pratiques 105 (01 2000), 247–250.

[18]

Marie-Francine Moens. 2008. Using patterns of thematic progression for building a table of contents of a text. Natural Language Engineering 14 (04 2008), 145–172.

Digital Library

[19]

Kaveh Taghipour and Hwee Tou Ng. 2016. A Neural Approach to Automated Essay Scoring. In Proceedings of EMNLP. 1882–1891.

[20]

Elena Tribushinina, Elena Dubinkina, and Ted Sanders. 2015. Can connective use differentiate between children with and without specific language impairment? First Language 35, 1 (2015), 3–26.

Cited By

França AReategui EMintz JMeira RMotz R(2024)Writing Analytics and AI for Special Education: Preliminary Results on Students with Autism Spectrum DisorderArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-64312-5_23(192-199)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64312-5_23
Alharbi W(2023)AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance ToolsEducation Research International10.1155/2023/42533312023(1-15)Online publication date: 8-Feb-2023
https://doi.org/10.1155/2023/4253331
Vanderose BFrenay BHenry JDevroey X(2020)Report from the 1st Int. Workshop on Education through Advanced Software Engineering and Artificial Intelligence (EASEAI '19)ACM SIGSOFT Software Engineering Notes10.1145/3375572.337557945:1(25-27)Online publication date: 22-Jan-2020
https://dl.acm.org/doi/10.1145/3375572.3375579
Show More Cited By

Index Terms

Towards context-aware automated writing evaluation systems
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Student assessment

Recommendations

The Effectiveness of Using Grammarly to Improve Students' Writing Skills
ICDEL '20: Proceedings of the 5th International Conference on Distance Education and Learning

The purpose of this case study was to investigate the effectiveness of using Grammarly in EFL writing during a 16-week experimental period at a four-year university in China. A total of 43 English-major sophomores participated in this study. Students' ...
Contextualizing automated writing evaluation: a case of english for specific purposes writings
Abstract
With the developments of technology, writing evaluation and feedback are supported by Natural Language Processing (NLP) and Latent Semantic Analysis (LSA). The immersion of NLP and LSA in Writing contributes to the development of Automatic Writing ...
Predictors of middle school students’ perceptions of automated writing evaluation
Abstract
This study examined middle school students' perceptions of an automated writing evaluation (AWE) system, MI Write. We summarize students' perceptions of MI Write's usability, usefulness, and desirability both quantitatively and qualitatively. We ...
Highlights
- We study middle school students' perceptions of an AWE system called MI Write.
- Students with LEP and lower SES perceived MI Write more useable/useful.
- So too did students who liked writing and believed that revising was important.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EASEAI 2019: Proceedings of the 1st ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence

August 2019

45 pages

ISBN:9781450368520

DOI:10.1145/3340435

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ESEC/FSE '19

Sponsor:

SIGSOFT

ESEC/FSE '19: 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 26, 2019

Tallinn, Estonia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
129
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

França AReategui EMintz JMeira RMotz R(2024)Writing Analytics and AI for Special Education: Preliminary Results on Students with Autism Spectrum DisorderArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-64312-5_23(192-199)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64312-5_23
Alharbi W(2023)AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance ToolsEducation Research International10.1155/2023/42533312023(1-15)Online publication date: 8-Feb-2023
https://doi.org/10.1155/2023/4253331
Vanderose BFrenay BHenry JDevroey X(2020)Report from the 1st Int. Workshop on Education through Advanced Software Engineering and Artificial Intelligence (EASEAI '19)ACM SIGSOFT Software Engineering Notes10.1145/3375572.337557945:1(25-27)Online publication date: 22-Jan-2020
https://dl.acm.org/doi/10.1145/3375572.3375579
Wang Z(2020)Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing courseLibrary Hi Tech10.1108/LHT-05-2020-0113ahead-of-print:ahead-of-printOnline publication date: 14-Oct-2020
https://doi.org/10.1108/LHT-05-2020-0113

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents