Implementation of Automated Feedback System for Japanese Essays in Intermediate Education

Abstract

Writing is a fundamental skill that plays an important role in student success in both academic and professional settings. Through writing, students can express their thoughts, ideas and knowledge and communicate effectively with others. This skill is essential for success in any field as it enables individuals to articulate their ideas clearly and persuasively. One of the most important reasons for writing is that it helps students develop critical thinking skills. Writing requires students to analyze, evaluate, and synthesize information, which helps develop analytical and problem-solving skills. Additionally, writing is critical to a student's academic success. At American school, students are required to write essays, and other written assignments as part of the curriculum. These assignments provide students with an opportunity to demonstrate their understanding of specific topics and develop their writing skills. It can help students organize their thoughts and present their ideas in a logical and coherent way. But Japanese exams are geared toward non open answer tasks as they are easier to score and fairer to grade. In these exams are homework, AES (Automated Essay Scoring) is a common method of scoring written answer using computer software. It is commonly used in education for grading student essays but can also be used in other areas. To analyze and evaluate written text, the AES system uses natural language processing and machine learning algorithms. It can be trained on a large dataset of previously graded essays to learn the characteristics of good essays and apply what it learns to grade new essays. AES can be used to provide student feedback, assist teacher. But AES can only provide students with a holistic score, unable to provide meaningful feedback on students writing. Automated Essay Feedback (AEF) can help provide those important feedback and be the important component of the learning process. Using those feedback, it can help students understand their performance, provides personalized and detailed information about their work, increases engagement and motivation, and aids in the development of critical thinking skills. But the feedback also needs to be chosen well before given to the students. Our research utilizes the 6+1 writing traits theory, which is widely used in American schools. The theory is widely accepted in teaching writing, assessing student writing, and providing feedback to students. The feedback from the theory is intended to be used as the comprehensive method for assessing student writing and can be used to provide feedback on the writing of students of all proficiency levels, from beginner to intermediate. Idea, structure, style, word, convention, and readability are chosen from the 6+1 writing-trait theory to create our AEF systems, that are suitable for Japanese L1 students. Idea trait is related to the content of the text and the quality of the ideas presented. This includes the relevance, originality, and development of ideas within the essay. Structure trait is related to sentence structure and coherence. This includes the logical flow of ideas, the use of transitions, and the overall coherence of writing. Style trait refers to the students’ voice and tone when writing. Word trait is related to the students’ use of vocabulary and language. This includes the accuracy, meaning and appropriateness of the words used in the writing. Convention trait refers to the students’ adherence to the rules of grammar, spelling, and punctuation. It includes the students’ ability to use capitalization, punctuation, and grammar correctly and consistently. And the final Readability trait refers to the general ease of reading and understanding what is written. This includes the ability of how students can communicate their ideas clearly and concisely, and the use of formatting and visuals to enhance readability. By combining these traits with a data-driven model, we created a system that can automatically grade and give feedback to students. The system automatically identifies parts of student writing that need improvement, then recommends feedback to the student. The feedback can come into two form, corrective and suggestive feedback. Suggestive feedback is a type of feedback that gives students advice and suggestions on how to improve their writing. It is different from corrective feedback, which simply tells the students whether their writing is good or bad. While suggestive feedback is preferable, it is not always easy to give or receive. It takes time, effort, and practice to provide feedback that is both specific and actionable while remaining non-obtrusive. Corrective feedback has both advantages and disadvantages. The advantage is that it provides a clear picture of a student's overall achievement and is easy to understand for students, parents, and teachers. The downside is that it doesn't provide specific information about what students need to do. It also does not give students the opportunity to take control of their own learning process. While it can provide a judgment or assessment of the quality of a student's writing and give students a general idea of how well they are doing in their writing, corrective feedback is best to be used in conjunction with other types of feedback such as suggestive feedback, to give students specific information on what they need to improve. Our contributions in this research are twofold: design a 6+1 writing-trait AEF for Japanese L1 students and evaluate a new feedback type using peer answer as the feedback. We propose that every AEF system should partially relies on AES system, so the feedback can be measured based on the student’s score metric. The result from our system shows that using peer answer as feedback can lead to the improvement of student writing and students prefer human-like feedback more than peer-answer feedback as the latter often lack the context and explanation.