Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2723576.2723595acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
short-paper

Pssst... textual features... there is more to automatic essay scoring than just you!

Published: 16 March 2015 Publication History

Abstract

This study investigates a new approach to automatically assessing essay quality that combines traditional approaches based on assessing textual features with new approaches that measure student attributes such as demographic information, standardized test scores, and survey results. The results demonstrate that combining both text features and student attributes leads to essay scoring models that are on par with state-of-the-art scoring models. Such findings expand our knowledge of textual and non-textual features that are predictive of writing success.

References

[1]
Allen, L. K., Snow, E. L., and Crossley, S. A., Jackson, G. T., & McNamara, D. S. 2014. Reading comprehension components and their relation to the writing process. L'année Psychologique,/ Topics in Cognitive Psychology, 114, 663--691.
[2]
Attali, Y. and Burstein, J. 2006. Automated essay scoring with e-rater V.2. Journal of Technology, Learning, and Assessment. 4, 3
[3]
Biber, D. 1988. Variation across speech and writing. Cambridge: Cambridge University Press.
[4]
Crossley, S. A., Allen, L. K., and McNamara, D. S. 2014. Analyzing discourse processing using a simple natural language processing tool (SiNLP). Discourse Processes, 51, 511--534.
[5]
Crossley, S. A., Roscoe, R., and McNamara, D. S. 2014. What is quality writing? An investigation into the multiple ways writers can write high quality essays. Written Communication, 31 (2), 184--214.
[6]
Daly, J. A., and Miller, M. D. 1975. Apprehension of writing as a predictor of message intensity. Journal of Psychology, 89, 175--177.
[7]
Gates-MacGinitie Reading Tests. 1989. Technical Report for Gates-MacGinitie Reading Tests form S. Chicago, Illinois: Riverside Publishing.
[8]
Graham, S. 2006. Writing. In P. Alexander & P. Winne (Eds.), Handbook of educational psychology (pp. 457--477). Mahwah, NJ: Erlbaum.
[9]
Hearst, M. 2002. The debate on automated essay scoring. Intelligent Systems and their Applications, IEEE, 15, 22--37.
[10]
McNamara, D., Crossley, S., and Roscoe, R. 2013. Natural language processing in an intelligent writing strategy tutoring system. Behavioral Research Methods, Instruments and Computers, 45, 499--515.
[11]
McNamara, D. S., Graesser, A. C., McCarthy, P., and Cai, Z. 2014. Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press.
[12]
McNamara, D., Raine, R., Roscoe, R., Crossley, S., Jackson, G., Dai, J., Cai, Z., Renner, A., Brandon, R., Weston, J., Dempsey, K., Carney, D., Sullivan, S., Kim, L., Rus, V., Floyd, R., McCarthy, P., and Graesser, A. 2012. The Writing-Pal: Natural language algorithms to support intelligent tutoring on writing strategies. In P. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing and content analysis: Identification, investigation, and resolution (pp. 298--311). Hershey, P. A.: IGI Global.
[13]
O'Reilly, T. and McNamara, D. S. 2007. The impact of science knowledge, reading strategy knowledge on more traditional "High-Stakes" measures of high school students' science achievement. American Educational Research Journal, 44, 161--196.
[14]
O'Reilly, T., Best, R., and McNamara, D. S. 2004. Self-explanation reading training: Effects for low-knowledge readers. In K. Forbus, D. Gentner, & T. Regier (Eds.), Proceedings of the 26th Annual Cognitive Science Society (pp. 1053--1058). Mahwah, NJ: Erlbaum.
[15]
Rudner, L., Garcia, V., and Welch, C. 2006. An evaluation of the IntelliMetric essay scoring system. Journal of Technology, Learning, and Assessment, 4, 4.
[16]
Saddler, B., and Graham, S. 2007. The relationship between writing knowledge and writing performance among more and less skilled writers. Reading and Writing Quarterly, 23, 231--247.
[17]
Scardamalia, M., and Bereiter, C. 1987. Knowledge telling and knowledge transforming in written composition. In S. Rosenberg (Ed.), Advances in applied psycholinguistics: Reading, writing, and language learning (vol. 2, pp. 142--175). New York: Cambridge University Press.
[18]
Stæhr, L. S. 2008. Vocabulary size and the skills of listening, reading and writing. Language Learning Journal, 36(2), 139--152.
[19]
Tierney, R. J., and Shanahan, T. 1991. Research on the reading-writing relationship: Interactions, transactions, and outcomes. In R. Barr, M. L. Kamil, P. Mosenthal, & P. D. Pearson (Eds.), Handbook of reading research (Vol. 2, ppp. 246--280). New York: Longman
[20]
Warschauer, M., and Ware, P. 2006. Automated writing evaluation: defining the classroom research agenda. Language Teaching Research, 10, 1--24.

Cited By

View all
  • (2024)Is the variation in syntactic complexity features observed in argumentative essays produced by B1 level EFL learners in Finland and Pakistan attributable exclusively to their L1?Assessing Writing10.1016/j.asw.2024.10083960(100839)Online publication date: Apr-2024
  • (2024)From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case StudyInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00387-6Online publication date: 25-Jan-2024
  • (2023)Quantification of students’ active learning in design, build, and test engineering modulesDeveloping Academic Practice10.3828/dap.2023.22023:Special(17-37)Online publication date: Jan-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge
March 2015
448 pages
ISBN:9781450334174
DOI:10.1145/2723576
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automatic essay scoring
  2. corpus linguistics
  3. data mining
  4. individual differences
  5. intelligent tutoring systems
  6. natural language processing

Qualifiers

  • Short-paper

Conference

LAK '15

Acceptance Rates

LAK '15 Paper Acceptance Rate 20 of 74 submissions, 27%;
Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)2
Reflects downloads up to 15 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Is the variation in syntactic complexity features observed in argumentative essays produced by B1 level EFL learners in Finland and Pakistan attributable exclusively to their L1?Assessing Writing10.1016/j.asw.2024.10083960(100839)Online publication date: Apr-2024
  • (2024)From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case StudyInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00387-6Online publication date: 25-Jan-2024
  • (2023)Quantification of students’ active learning in design, build, and test engineering modulesDeveloping Academic Practice10.3828/dap.2023.22023:Special(17-37)Online publication date: Jan-2023
  • (2023)Style Augmented Transformer Architecture for Automatic Essay Assessment2023 IEEE International Conference on Advanced Learning Technologies (ICALT)10.1109/ICALT58122.2023.00105(339-343)Online publication date: Jul-2023
  • (2022)Construcción de contenido para un Sistema Tutor Inteligente en idiomas: un estudio piloto con el corpus OneStopEnglishRevista Colombiana de Computación10.29375/25392115.448423:1(53-60)Online publication date: 21-Jun-2022
  • (2020)STP:Suicidal Tendency Prediction Among the Youth Using Social Network DataNext Generation Information Processing System10.1007/978-981-15-4851-2_17(161-169)Online publication date: 14-Jun-2020
  • (2019)Towards context-aware automated writing evaluation systemsProceedings of the 1st ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence10.1145/3340435.3342722(17-20)Online publication date: 26-Aug-2019
  • (2019)Text mining in educationWIREs Data Mining and Knowledge Discovery10.1002/widm.13329:6Online publication date: 4-Aug-2019
  • (2018)Using Natural Language Processing and Qualitative Thematic Coding to Explore Math Learning and Critical ThinkingProceedings of the 2018 International Conference on Big Data and Education10.1145/3206157.3206178(38-43)Online publication date: 9-Mar-2018
  • (2017)Learning Analytics: at the Nexus of Big Data, Digital Innovation, and Social Justice in EducationTechTrends10.1007/s11528-017-0226-962:1(37-45)Online publication date: 10-Oct-2017
  • Show More Cited By

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media