Keywords
natural language processing
automated essay scoring
AES
history of AES
critique of AES
humanistic values of pedagogy
humanistic implementation approach
How to Cite
Abstract
Writing essays plays a crucial role in developing essential competencies at various levels of education, especially in the humanities. At the same time, essays constitute an indispensable element of educational assessment. Modern Natural Language Processing (NLP) technologies powered by Artificial Intelligence (AI) offer teachers the opportunity for support and even automated grading of written assignments (Automated Essay Scoring). Due to significant pedagogical and socio-cultural implications, these technologies are the subject of numerous analyses today. The article provides a comprehensive overview of AES (Automated Essay Scoring) technologies, encompassing their history and initial concepts, current usage, operation models, and key areas of critique. Additionally, a humanistic approach to integrating AES into pedagogical practice is outlined, focusing on optimizing the benefits derived from its utilization and mitigating the threats it may pose to humanistic values of pedagogy.
References
Amorim, E., Cançado, M., Veloso, A. (2018). „Automated essay scoring in the presence of biased ratings”. W: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, 229-237. New Orleans: Association for Computational Linguistics. DOI: https://doi.org/10.18653/v1/N18-1021
Bai, X., Stede, M. (2022). „A survey of current machine learning approaches to student free-text evaluation for intelligent tutoring”. International Journal of Artificial Intelligence in Education 28, 1–39. DOI: https://doi.org/10.1007/s40593-022-00323-0
Barshay, J. (2022). PROOF POINTS: A smarter robo-grader. The Hechinger report, https://hechingerreport.org/proof-points-a-smarter-robo-grader.
Christensen, C.M. (1997). The innovator’s dilemma: how new technologies cause great firms to fail. Harvard Business School Press.
Chung, G.K.W.K., Baker, E.L. (2003). „Issues in the reliability and validity of automated scoring of constructed responses”. W: M.D. Shermis & J. Burstein (red.), Automated essay scoring: A cross-disciplinary perspective. 23–40. Lawrence Erlbaum Associates Publishers.
Dikli, S. (2006). „An overview of automated scoring of essays”. Journal of Technology, Learning, and Assessment, 5(1), 1-36.
Feathers, T. (2019). „Flawed Algorithms Are Grading Millions of Students’ Essays”. Vice, https://www.vice.com/en/article/pa7dj9/flawed-algorithms-are-grading-millions-of-students-essays.
Graham, M., Milanowski, A., Miller, J. (2012). „Measuring and promoting inter-rater agreement of teacher and principal performance ratings”. The Center for Educator Compensation and Reform, 1-37.
Gulson, K., Thompson, G., Swist, T., Kitto, K., Rutkowski, L., Rutkowski, D., Hogan, A., Zhang, V., Knight, S. (2022). Automated essay scoring in Australian schools: key issues and recommendations. White Paper. University of Sydney, Australia.
Horbach, A., Ding, Y., Zesch, T. (2017). „The influence of spelling errors on content scoring performance”. W: Proceedings of the 4th workshop on natural language processing techniques for educational applications, 45–53. Taipei: Asian Federation of Natural Language Processing.
Ke, Z., Ng, V. (2019). „Automated essay scoring: a survey of the state of the art”. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence Survey track. 6300-6308, https://doi.org/10.24963/ijcai.2019/879. DOI: https://doi.org/10.24963/ijcai.2019/879
Kolowich, S. (2012). „A win for the Robo-Readers”, Inside Higher Ed, https://www.insidehighered.com/news/2012/04/13/large-study-shows-little-difference-between-human-and-robotessay-graders.
Koziol, M., Singhal, P., Cook, H. (2018). „Computer says no: governments scrap plan for ‘robot marking’ of NAPLAN essays”. The Sydney Morning Herald, https://www.smh.com.au/politics/federal/computer-says-no-governments-scrap-plan-for-robot-marking-of-naplan-essays-20180129-h0py6v.html.
Li, F, Xi, X, Cui, Z, Li, D, Zeng, W. (2023). „Automatic essay scoring method based on multi-scale features”. Applied Sciences 13(11), 6775. DOI: https://doi.org/10.3390/app13116775
Markoff, J. (2013). „Essay-grading software offers professors a break”. The New York Times, https://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-essays-atcollege-level.html?pagewanted=all&_r=0.
Marquard, O. (1994). Apologia przypadkowości. Warszawa: Oficyna Naukowa.
Measurement Inc. (n.d.). Pobrane z https://www.measurementinc.com/services#scoring-services.
Naisbitt, J., Naisbitt, M., Philips, D. (1999). High tech high touch. New York: Broadway Books.
Nguyen, H., Dery, L. (2016). „Neural networks for automated essay grading”. Report for CS224d: Deep Learning for Natural Language Processing. https://cs224d.stanford.edu/reports/huyenn.pdf.
Page, E.B. (1966). „The imminence of... grading essays by computer”. The Phi Delta Kappan 47(5), 238-243.
Page, E.B. (1968). „The use of the computer in analyzing student essays”. International Review of Education 14(3), 253-263. DOI: https://doi.org/10.1007/BF01419938
V. Paruchuri (2013). On the automated scoring of essays and the lessons learned along the way, https://www.vikas.sh/post/on-the-automated-scoring-of-essays.
Perelman, L.C. (2013). „Critique of Mark D. Shermis & Ben Hamner. Contrasting state-of-the-art automated scoring of essays: analysis”. The Journal of Writing Assessment 6(1).
Perelman, L.C. (2020). „The BABEL Generator and E-Rater: 21st Century Writing Constructs and Automated Essay Scoring (AES)”. Journal of Writing Assessment 13(1).
Potts, M. (2005). „Ellis Page, 81, a developer of computerized grading, dies”. The New York Times, https://www.nytimes.com/2005/05/23/us/ellis-page-81-a-developer-of-computerized-gradingdies.html.
Shermis, M.D. (2014). „State-of-the-art automated essay scoring: competition, results, and future DOI: https://doi.org/10.1016/j.asw.2013.04.001
directions from a United States demonstration”. Assessing Writing 20, 53-76.
Mathias, S., & Bhattacharyya, P. (2020). „Can neural networks automatically score essay traits? W: Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications. 85–91. Seattle: Association for Computational Linguistics. DOI: https://doi.org/10.18653/v1/2020.bea-1.8
Ramesh, D., Sanampudi, S.K. (2022). „An automated essay scoring systems: a systematic literature review”. Artificial Intelligence Review 55, 2495–2527. DOI: https://doi.org/10.1007/s10462-021-10068-2
Rutner, S.M., Scott, R.A. (2022). „Use of artificial intelligence to grade student discussion boards: an exploratory study”. Information Systems Education Journal 20(4), 4-18.
Shehab, A., Elhoseny, M., Hassanien, A.E. (2016). „A hybrid scheme for automated essay grading based on LVQ and NLP techniques. W: 12th International Computer Engineering Conference (ICENCO). Kair, 65-70. DOI: https://doi.org/10.1109/ICENCO.2016.7856447
Smith, T. (2018). „More states opting to ‘Robo-Grade’ student essays by computer”. National Public Radio, https://www.npr.org/2018/06/30/624373367/more-states-opting-to-robo-gradestudent-essays-by-computer?t=1603620181809.
Süzen, N., Gorban, A.N., Levesley, J., & Mirkes, E.M. (2020). „Automatic short answer grading and feedback using text mining methods”. Procedia Computer Science 169, 726-743. DOI: https://doi.org/10.1016/j.procs.2020.02.171
Taghipour, K., & H.T. Ng (2016). „A neural approach to automated essay scoring. W: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing 1882–1891. DOI: https://doi.org/10.18653/v1/D16-1193
Thomas, E. (2023). „Grading can overwhelm teachers as the semester ends. Does AI present a better way?”. Kansas Reflector, https://kansasreflector.com/2023/04/14/grading-can-overwhelmteachers-as-the-semester-ends-does-ai-present-a-better-way.
UoA. (2012). Man and Machine: better writers, better grades. https://cacm.acm.org/news/148670-man-and-machine-better-writers-better-grades.
Woods, B., Adamson, D., Miel, S., & Mayfield, E. (2017). „Formative essay feedback using predictive scoring models”. W: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. DOI: https://doi.org/10.1145/3097983.3098160
License
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.