In this study, we evaluated the performance of the state-of-the-art sequence tagging grammar error detection and correction model (SeqTagger) using Japanese university students' writing samples. With an automatic annotation toolkit, ERRANT, we first evaluated SeqTagger's performance on error correction with human expert correction as the benchmark. Then a human-annotated approach was adopted to evaluate Seqtagger's performance in error detection using a subset of the writing dataset. Results indicated a precision of 63.66% and a recall of 20.19% for error correction in the full dataset. For the subset, after manual exclusion of irrelevant errors such as semantic and mechanical ones, the model shows an adjusted precision of 97.98\% and an adjusted recall of 42.98% for error detection, indicating the model’s high accuracy but also its conservativeness. Thematic analysis on errors undetected by the model revealed that determiners and articles, especially the latter, were predominant. Specifically, in terms of context-independent errors, the model occasionally overlooked basic ones and faced challenges with overly erroneous or complex structures. Meanwhile, context-dependent errors, notably those related to tense and noun number, as well as those possibly influenced by the students' first language (L1), remained particularly challenging.

A web-based testing system including the most important grammar points has been created, which can be accessed via https://grammar-diagnosis.herokuapp.com/

The research has yield fruitful results in the evaluation of open-source semantic similarity tools and in developing a web-based application for automated grading of fact-based writing. The evaluation part has been published in a high-impact journal, Education and Information Technologies, while the application is under beta test right now.

This study explores the role of story-rich computer games in narrative writing of Chinese university students. In a two-month writing class with the game Sim4, a participant received instruction on gameplay techniques and language, completed game quests set by the teacher, wrote 14 narratives based on gameplay events and received all-encompassing corrective feedback to her writing samples. A pre-test on game vocabulary and two writing pre- and post- tests based on non-game videos were also administered. The researchers evaluated the participant’s narrative writing performance with in-depth analysis of linguistic features in the writing samples, which included syntax, lexicon, cohesion and content. The analysis was conducted through NLP technology and inductive coding on revisions in the corrective feedback. A follow-up interview was conducted to complement results on writing evaluation and to provide information on the participant’s perception and attitude towards the GBW class. Results show that while writing evaluation did not indicate any consistent improvement in the participant’s narrative writing performance throughout the GBW class, the rich stories and contextualized vocabulary in the game contributed to her content and lexicon and helped her to better understand how to write narratives. The study further indicates that the challenges facing similar medium-level students in narrative writing are likely to be caused by inadequate grammatical competence and the inability to express ideas clearly in English, rather than by specific features of the narrative genre. Moreover, the all-encompassing feedback in the study was found to be cost-inefficient and thus is not recommended in future studies. 

This research aims to create a gamified application for students to learn English grammar. The researcher first referred to the CEFR and CEFR-J for well-stablished grammatical categories and wrote 180 questions corresponding to 180 grammar points under the categories. A group of 5 experts reviewed and finalized the questions to ensure they cover the most important grammar points in university students' exposure to English reading and writing. After that, a web-based app was developed for this research, where the users can take the grammar quizzes and get feedback. For gamification of the app, the researcher will add leaderboard features to encourage engagement from students. When this is done, the researcher will recruit students to try the app and iterate the app based on student feedback. Then, the researcher will make the app adaptive, which means the questions for each individual student will be based on his or her prior performance in the app.