Abstract

There has long been interest in the effect of stay abroad on pragmatic development in second language (L2) pragmatics research. However, few testing instruments exist to assess pragmatics learning in stay abroad and provide information about learners’ development to help institutions evaluate the success of stay abroad programs. In this study, we describe the adaptation and validation of Roever’s (2005) test of second language routine formulae to be used large-scale with Japanese stay-abroad learners. We follow Kane’s (2006) argument-based approach to validation, evaluating the first four inferences: Domain description, Evaluation, Generalization, and Explanation. We also investigate the test’s practicality within the institutional setting of intended use. We find evidence supporting the use of the test for providing information on pragmatics learning during stay abroad. The test can differentiate between low- and high-ability learners, shows adequate reliability, and reflects the roles of proficiency and exposure in pragmatics learning in stay abroad contexts. The test is also practical and requires few resources. Further steps, such as the investigation of the Extrapolation and Decision inference, are discussed.

<title>Abstract</title>
This article reports on two quasi-experimental studies that investigated the possible development and transfer of
audience awareness in novice EFL writers as they engaged in online writing tasks through a Social Networking Service (SNS).
Japanese students from two universities were asked to write, read, and comment on other students’ writing once a week. The two
studies were arranged sequentially so as to capture in an exploratory but jointly illuminating manner whether and how the
“elusive” (<xref>Hyland, 2005</xref>) construct of “sense of audience” can develop and transfer
across genres. The results of both studies suggest that the SNS environment can help L2 writers develop audience awareness and
transfer that awareness across genres when two conditions are met: (1) the genre of the SNS tasks should be perceived as similar
to that for which transfer was expected; and (2) the students did not develop a sense of audience in previous writing
instruction.

We conducted a randomized experiment targeting 322 Japanese high school students to examine the impacts of a newly developed English-language learning program. The treated students were offered an opportunity to communicate for 25 minutes with English-speaking Filipino teachers via Skype several times a week over a 5-month period as an extracurricular activity. The results show that the Skype program increased the interest of the treated students in an international vocation and in foreign affairs. However, the students did not improve their English communication abilities, as measured by standardized tests, probably because of the program's low utilization rate. Further investigation showed that the utilization rate was particularly low among students demonstrating a tendency to procrastinate. These results suggest the importance of maintaining students’ motivation to keep using such information and communication technology-assisted learning programs if they are not already incorporated into the existing curriculum. Having procrastinators self-regulate may be especially crucial.

This study explores the effects of various motivational variables operating within 44 English classes on 1-year-long gains in the English reading proficiency of 1,149 Japanese university students. The study adds new knowledge to the recent outcomes of second language (L2) motivational studies in 3 major ways. First, the explanatory variables include the participants’ own motivation as well as the class norms (ethos) shared by class members, some of which (e.g., career aspirations) were not directly related to their L2 learning. Second, multi-level modeling enabled a simultaneous analysis of L2 proficiency growth over 1 year at both the individual and the class levels. Third, its explanatory sequential mixed methods design (Cresswell,) demonstrated how the participants’ own accounts help to explain the initial quantitative results in ways the researchers’ outsider speculations could not. The statistical results indicate that the 44 classes grew in L2 reading abilities at significantly different rates over the year and that these differences can be explained by the students’ perceptions of their classmates’ normative career aspirations (among other predictors). These results suggest that being surrounded by classmates with high career aspirations may lead to higher motivation to study the L2 even if students are not highly motivated in the beginning.

The present study investigated the effects of varying lengths of overseas experiences on 37 Japanese students&apos; English writing ability and motivation over 3.5 years. The students were observed at the beginning of their first year and in the middle of their second, third, and fourth years at their university. During the 3.5-year observation period, 28 of the 37 students spent 1.5 to 11 months in English-speaking countries. The results revealed that (1) students&apos; second language (L2) writing ability did not change in a linear way; (2) over the 3.5 years, students who spent some time abroad significantly improved their L2 writing ability whereas those who stayed in Japan did not; (3) many of those students who went abroad formed L2-related imagined communities that possibly motivated them to improve their L2 writing ability; (4) those students who spent more than 4 months abroad improved their L2 writing ability significantly more than the other students; and (5) only those students who spent more than 8 months abroad became intrinsically motivated and voluntarily practiced to improve their L2 writing. doi: 10.5054/tq.2011.240861

In the present study I describe the 150-year history of school-based English language assessment in Japan. The history is divided into four major periods according to the purposes of English language education set by the government in the different periods: (1) 1860 to 1945, when English was first introduced and taught in schools mainly for elite classes; (2) 1945 to 1970, when English became part of the compulsory education for the first time; (3) 1970 to 1990 when English began to be regarded as the most effective means to communicate with foreign people in the rapidly shrinking world; and (4) 1990 to the present, when several innovative policies have been introduced into classroom measurement systems. I describe how assessment practices for English education at schools ill each of these periods were and still are affected by various factors, including political, economic, and demographic changes in society, as well as academic paradigm shifts in the fields of education and applied linguistics.

This study was a comparison of the changes in English writing behavior of 7 Japanese university students (the study-abroad group) who spent 4 to 9 months in English-speaking countries with those of 6 counterparts majoring in British and American studies (the at-home group) who remained in Japan. The study aimed at confirming the results of Sasaki (2004), who investigated similar students' English-writing behavior over 3.5 years. After a 1-year observation period, (a) both groups improved their general English proficiency
(b) only those in the study-abroad group improved their second language (L2) writing ability and fluency
(c) unlike Sasaki's participants, the study-abroad group made more local plans and the at-home group fewer
(d) at the end of the study, both groups translated their ideas into the L2 as often as they did at the beginning of the study
and (e) only the study-abroad group became more motivated to write better L2 compositions. © 2007 The Modern Language Journal.

The present study investigated the changes in Japanese students' English writing behaviors over a 3.5-year period using multiple data sources including written texts, videotaped writing behaviors, and stimulated-recall protocols. Data from student interviews supplemented the analyses. Because 6 (henceforth, English as a second language [ESL] students) out of the 11 participants spent 2 to 8 months in English-speaking environments, the study was also able to examine the effects of such overseas experiences. The results revealed that over the observation period (a) both the English as a foreign language (EFL, remaining in Japan) and the ESL students improved their English proficiency, English composition quality/fluency, and confidence in English writing; (b) the ESL students' overseas experiences were helpful for improving their writing strategy use and motivation to write better compositions; and (c) neither group's abilities or skills became like those of the EFL experts studied in Sasaki (2002).

The present study investigated how schemata activated by culturally familiar words might have influenced students” cloze test-taking processes. Sixty Japanese EFL students were divided into two groups with equivalent English reading proficiency. They completed either a culturally familiar or an unfamiliar version of a cloze test. Partially replicating Chihara et al.'s (1989) experiment, several unfamiliar words in the original cloze test passage were changed to more familiar ones in the modified version. Unlike in Chihara et al. (1989), however, students were asked to give verbal reports of their test-taking processes, and to recall the passage after they had completed the tests. Thus, the two groups” test-taking activities were compared in terms of: 1) item performance
2) expressing correct understanding of the key terms while solving the items and recalling
3) the amount of text information they used to complete the items
and 4) the quantities and qualities of the final recalls. Results demonstrated that those who read the culturally familiar cloze text tried to solve more items and generally understood the text better, which resulted in better performances than those of the students who read the original text. These results also support the claim that cloze tests can measure higher-order processing abilities. © 2000, Sage Publications. All rights reserved.

The present study investigated EFL learners' writing processes using multiple data sources including their written texts, videotaped pausing behaviors while writing, stimulated recall protocols, and analytic scores given to the written texts. Methodologically, the study adopted a research scheme that has been successfully used for building models of Japanese L1 writing. Three paired groups of Japanese EFL writers (experts vs. novices, more- vs. less-skilled student writers, novices before and after 6 months of instruction) were compared in terms of writing fluency, quality/complexity of their written texts, their pausing behaviors while writing, and their strategy use. The results revealed that (a) before starting to write, the experts spent a longer time planning a detailed overall organization, whereas the novices spent a shorter time, making a less global plan
(b) once the experts had made their global plan, they did not stop and think as frequently as the novices
(c) L2 proficiency appeared to explain part of the difference in strategy use between the experts and novices
and (d) after 6 months of instruction, novices had begun to use some of the expert writers' strategies. It was also speculated that the experts' global planning was a manifestation of writing expertise that cannot be acquired over a short period of time.

The present study developed an analytic rating scale for Japanese university students' first language (L1) expository writing. Research proceeded in four stages. In Stage 1, we devised a questionnaire to investigate Japanese L1 teachers' criteria for evaluating expository writing. A total of 102 teachers responded to the questionnaire, and rated the 35 related descriptions according to their importance. In Stage 2, based on the questionnaire results, we either eliminated or reorganized these 35 descriptions under six analytic criteria: 1) Clarity of the theme
2) Appeal to the readers
3) Expression
4) Organization
5) Knowledge of language forms
and 6) Social awareness. Then, in Stage 3, we attempted to investigate possible weighting of these six criteria. We asked 106 teachers to rate two to four compositions with varied profiles using both a holistic scale and the obtained analytic scale. The results showed that the explanatory power of each criterion can vary from composition to composition, and thus we concluded that the six criteria should have equal weighting. Finally, in Stage 4, we pilot-tested the obtained scale using a new set of 69 compositions. The results indicate that the present scale is both valid and reliable, and that it is superior to a traditional analytic scale in capturing such composition qualities as appeal to the readers and social awareness. © 1999 Arnold.

The present study compares two popular measures of second language pragmatic competence: production questionnaires and role plays. Twelve Japanese university students representing three different English proficiency levels responded to both measures for the same four request and four refusal situations. Response length, range and content of the expressions, and native speaker evaluations of these responses were analyzed.
The production questionnaire and role play elicited somewhat different production samples from the students. Role plays induced longer responses, and a larger number and greater variety of strategies/formulas, than production questionnaires. These differences appear to be caused by the interactive nature of role plays. Students often switched strategies for the same situations across different methods. Such intra-participant variations could have been missed if different participants had responded to different methods as in many previous studies. In addition, the correlation between the appropriateness scores of the two methods was not high enough to support the claim that they measured exactly the same trait. The low correlation probably resulted not only because the two methods produced different responses, but also because the role play responses provided additional audio-visual information, which might have affected the raters' evaluations. These findings suggest that production questionnaire scores cannot be simply substituted for role play scores.

The present study examined how a new topic was introduced, maintained, and changed in the Japanese-English interlanguage data of a 45 minute interview between a native and a non-native speaker of English. Retrospective accounts recorded after the interview were used to complement the interlanguage data. In order to achieve a more comprehensive perspective of the interlanguage, the function and distribution of certain topic marking devices in the present data were compared with those of equivalent devices in both native Japanese and English data. Results indicate that, although the topic marking system of the interlanguage shared some features with the first and second languages, it maintained features independent of these languages. Furthermore, the interlanguage data confirmed to two language universal hypotheses of Givon's (1984) topic continuity hierarchy and Du Bois' (1987) preferred argument structure. Possible causes for some unique features observed in the interlanguage were also discussed.

This study investigated factors that might influence Japanese university students' expository writing in English. We examined 70 students of low- to high-intermediate English proficiency along a variety of dimensions, namely, second language (L2) proficiency, first language (L1) writing ability, writing strategies in L1 and L2, metaknowledge of L2 expository writing, past writing experiences, and instructional background. We considered these multiple factors as possible explanatory variables for L2 writing.
Quantitative analysis revealed that (a) students' L2 proficiency, L1 writing ability, and metaknowledge were all significant in explaining the L2 writing ability variance; (b) among these 3 independent variables, L2 proficiency explained the largest portion (52%) of the L2 writing ability variance, L1 writing ability the second largest (18%), and metaknowledge the smallest (11%); and (c) there mere significant-correlations among these independent variables. Qualitative analysis indicated that good writers were significantly different from weak writers in that good writers (a) paid more attention to overall organization while writing in L1 and L2; (b) wrote more fluently in LI and L2; (c) exhibited greater confidence in L2 writing for academic purposes; and (d) had regularly written more than one English paragraph while in high school. There was no significant difference between good and weak writers for other writing strategies and experiences. On the basis of these results, we propose an explanatory model for EFL writing ability.

The present study investigated the relationship between Japanese students' English L2 expository writing and several factors that might influence the quality of the writing product. Nineteen Japanese university students provided both quantitative and qualitative data. Quantitative analysis showed that the students' L2 proficiency and L1 writing ability accounted for a large proportion of variances in L2 writing quality. The finding that L1 writing ability was highly correlated with L2 writing ability is important because it suggests the existence of composing competence across L1 and L2 even for EFL students. There was also a significant interaction between this composing competence and L2 proficiency. Qualitative analysis suggested that the students' composing competence was related to: (a) use of several good writers' strategies, (b) writing fluency, and (c) confidence in writing. Furthermore, probably due to the input-poor EFL environment, the amount of self-initiated L2 writing experiences seemed to ploy an important role in determining students' L2 writing quality. © 1994.

The present protocol analysis supplements Sasaki's (1993) study, which used structural equation modeling to investigate the relationships among measures of second language proficiency (SLP), foreign language aptitude, and two types of intelligence (verbal intelligence and reasoning). In the present study, 6 participants randomly selected from the same subject pool took different types of SLP tests, a foreign language aptitude battery, and an intelligence test. The researcher attempted (a) to test Bachman and Palmer's (1982) hypothesis that a general SLP factor is related to the amount of information processing required for solving a given item, (b) to compare cognitive processes and strategies employed for different types of aptitude/intelligence tests, and (c) to compare the quality and quantity of test-taking processes employed by students with different levels of SLP.

The present study investigated the relationships among measures of second language proficiency (SLP), foreign language aptitude, and two types of intelligence (verbal intelligence and reasoning). The research had two objectives: (a) to examine the factor structure of several different SLP test scores; and (b) to investigate the relationship between a general SLP factor (G-SLP) and a hypothetical general cognitive factor (G-COG) that was assumed to influence foreign language aptitude, verbal intelligence, and reasoning. Several competing hypotheses were tested using structural equation modeling. Participants consisted of 160 Japanese college students studying English as a foreign language.
The results supported two models of SLP: one in which several specific trait factors were highly correlated with each other, and another in which these specific trait factors were connected to a higher-order G-SLP factor. Because there were only three first-order specific factors, these two models could not be distinguished from each other. Further investigation suggested that G-SLP and G-COG were not identical, but mutually correlated (r=0.65). This relationship implies that students' SLP and cognitive abilities were influenced by two distinct, but mutually correlated factors. More than half the G-SLP variance (58%) could not be explained by the G-COG factor. Several suggestions for further studies are presented.

This paper compares two approximation techniques for detecting differential item functioning (DIF) in an English as a second language (ESL) placement test when the group sizes are too small to use other possible methods (e.g., the three parameter item response theory method). An application of the Angoff delta- plot method (Angoff and Ford, 1973) utilizing the one parameter Rasch model adopted in Chen and Henning (1985), and Scheuneman's chi-square method (Scheuneman, 1979) were chosen because they are among the few methods appropriate for a sample size smaller than 100. Two linguistically and culturally diverse groups (Chinese and Spanish speaking) served as the subjects of this study. The results reveal that there was only marginal overlap between DIF items detected by Chen and Henning's method and Scheuneman's method
the former detected fewer DIF items with less variety than the latter. Moreover, Chen and Henning's method tended to detect easier items with smaller differ ences in p-value between the two groups whereas Scheuneman's method tended to detect items with the opposite features. Implications of these results are discussed. © 1991, Sage Publications. All rights reserved.