Keywords
large language models
history education
digital humanities
retrieval-augmented generation (RAG)
agentic AI
prompting
narrative analysis; readability
hallucinations
How to Cite
Number of views: 0
Number of downloads: 0
Abstract
This article offers a comparative, methodologically transparent pilot study of how different large-language-model configurations shape school-type historical narratives. Using a fixed prompt set (P1–P5) drawn from the textbook Europa. Nasza historia and four conditions - default ChatGPT baseline (C0), a ‘scientist/analytical researcher’ prompted ChatGPT (C1), NotebookLM with retrieval-augmented generation over a shared textbook source (C2), and an agentic system configured on the same knowledge base (C3) - we analyze outputs as discursive artefacts. The study combines four analytic perspectives: substantive accuracy and source anchoring, rhetorical profile and evaluative stance, formal readability and lexical texture, and logical/causal coherence. Quantitative proxies (sentence length, Polish-adapted Fog-type readability, lexical diversity via TTR and MTLD, nominalization and structural markers) are computed on the original Polish outputs without translation. Results reveal systematic trade-offs between anchoring and interpretive depth: retrieval-based and agentic configurations tend to produce a more textbook-like voice and stronger didactic scaffolding, while persona prompting increases meta-argumentation and criteria-driven evaluation. A critical-incident analysis further demonstrates that the appearance of grounding can coexist with hinge-fact failure, where a single erroneous historical anchor remains embedded in an otherwise coherent narrative. The article concludes with implications for history education, suggesting configuration-sensitive uses of LLMs and practical guardrails for classroom deployment.
References
Cotton D.R.E., Cotton P.A., Shipway J.R., Chatting and cheating: Ensuring academic integrity in the age of ChatGPT, “Assessment & Evaluation in Higher Education” 2024.
Europa. Nasza historia, Wydawnictwa Szkolne i Pedagogiczne - Eduversum, Warszawa 2022.
Gunning R., The Technique of Clear Writing, McGraw-Hill, New York 1952.
Holmes W., Bialik M., Fadel C., Artificial Intelligence in Education: Promises and Implications for Teaching and Learning, Center for Curriculum Redesign, Boston 2019.
Huang L., Yu W., Ma W. et al., A Survey on Hallucination in Large Language Models, “ACM Computing Surveys” 2025.
Imran M., Almusharraf N., Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature, “Contemporary Educational Technology” 2023, 15(4), ep464.
Kasneci E., Sessler K., Küchemann S. et al., ChatGPT for good? On opportunities and challenges of large language models for education, “Learning and Individual Differences” 2023, 103, 102274.
Kincaid J.P., Fishburne R.P., Rogers R.L., Chissom B.S., Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel, Research Branch Report 8-75, Naval Technical Training Command, 1975.
Labadze L., Grigolia M., Machaidze L., Role of AI chatbots in education: systematic literature review, “International Journal of Educational Technology in Higher Education” 2023, 20, 56.
Lewis P., Perez E., Piktus A. et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, “Advances in Neural Information Processing Systems (NeurIPS)” 2020.
McLuhan M., “The Medium is the Message,” in: Durham M. G., Kellner D. M. (eds.), Media and Cultural Studies: KeyWorks, Wiley-Blackwell, Malden-Oxford-Chichester 2012, pp. 100–107.
Miao F., Holmes W. (eds.), Guidance for generative AI in education and research, UNESCO, Paris 2023.
Munaye Y.Y., Admass W., Belayneh Y. et al., ChatGPT in Education: A Systematic Review on Opportunities, Challenges, and Future Directions, “Algorithms” (MDPI) 2025, 18(6), 352.
OECD; Education International, Opportunities, guidelines and guardrails for effective and equitable use of AI in education, OECD Publishing, 2023 (raport/ wytyczne).
OpenAI, GPT-5 System Card, arXiv 2025.
Schick T., Dwivedi-Yu J., Dessì R. et al., Toolformer: Language Models Can Teach Themselves to Use Tools, “Advances in Neural Information Processing Systems (NeurIPS)” 2023.
Sullivan M., Kelly A., McLaughlan P., ChatGPT in higher education: Considerations for academic integrity and student learning, “Journal of Applied Learning & Teaching” 2023, 6(1).
Wang X., Wei J., Schuurmans D. et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models, arXiv 2022.
Wei J., Wang X., Schuurmans D. et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, “Advances in Neural Information Processing Systems (NeurIPS)” 2022.
Werner W., Gralik D., Trzoss A., Media społecznościowe a funkcjonowanie wiedzy historycznej w Polsce. Raport z badañ, „Przegląd Archiwalno-Historyczny” 6 (2019), pp. 211-235.
Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser £., Polosukhin I., Attention Is All You Need, “Advances in Neural Information Processing Systems” 30 (2017), pp. 5998-6008
Yao S., Zhao J., Yu D. et al., ReAct: Synergizing Reasoning and Acting in Language Models, “International Conference on Learning Representations (ICLR)” 2023.
Zawacki-Richter O., Marín V.I., Bond M., Gouverneur F., Systematic review of research on artificial intelligence applications in higher education - where are the educators?, “International Journal of Educational Technology in Higher Education” 2019, 16, 39.
License
Copyright (c) 2026 Wiktor Werner
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.