Updated on 2024/07/18

写真a

 
SAWAKI, Yasuyo
 
Affiliation
Faculty of Education and Integrated Arts and Sciences, School of Education
Job title
Professor
Degree
B.A. in Education ( Kumamoto University )
M.A. in teaching English as a second language ( United States of America )
Ph.D. in applied linguistics ( United States of America )

Research Experience

  • 2014
    -
     

    Professor, Department of Education and Integrated Arts and Sciences, Waseda University, Tokyo, Japan (Since April to Present)

  • 2009
    -
     

    Associate Professor, Department of Education and Integrated Arts and Sciences, Waseda University, Tokyo, Japan (April to March 2014)

  • 2006.09
    -
     

    Research Scientist, Center for Validity Research, Educational Testing Service, Princeton, New Jersey, U.S.A. (September February 2009)

  • 2003.08
    -
     

    Associate Research Scientist, Center for Validity Research, Educational Testing Service, Princeton, New Jersey, U.S.A. (August August 2006)

  • 1996
    -
     

    Research Assistant, Department of American and English Literature, Showa Women's University, Tokyo, Japan (April March 1998)

Education Background

  •  
    -
    2003

    University of California, Los Angeles   Graduate School, Department of Applied Linguistics   applied linguistics, language assessment  

  •  
    -
    2003

    University of California, Los Angeles   Graduate School, Department of Applied Linguistics   applied linguistics, language assessment  

  •  
    -
    1995

    University of Illinois at Urbana-Champaign   Division of English as an International Language   Teaching English as a second language  

Committee Memberships

  • 2022.01
    -
    Now

    Asian Association for Language Assessment (AALA)  First Vice President

  • 2014.06
    -
    2021.12

    Asian Association for Language Assessment (AALA)  Secretary/Treasurer

  • 2016.04
    -
     

    Japan Language Testing Association (JLTA)  Board Member

  • 2016.04
    -
     

    日本言語テスト学会 (JLTA)  理事

  • 2014.06
    -
     

    アジア言語テスト学会 (AALA)  事務局長

  • 2012.04
    -
    2014.03

    Research Meeting Organizing Committee, Japan Language Testing Association (JLTA)  Chair

  • 2012.04
    -
    2014.03

    日本言語テスト学会(JLTA: Japan Language Testing Association) 研究会運営委員会  委員長

  • 2011.01
    -
    2013.12

    International Language Testing Association (ILTA)  Member-at-Large

  • 2011.01
    -
    2013.12

    国際言語テスト学会 (ILTA)  Member-at-Large

  • 2010.04
    -
    2012.03

    Japan Language Testing Association (JLTA)  Vice Chair, Research Meeting Organizing Committee

  • 2010.04
    -
    2012.03

    日本言語テスト学会 (JLTA)  研究会運営委員会副委員長

  • 2010.04
    -
    2012.03

    National Center for University Entrance Examinations  Member, Subject Committee 1

  • 2010.04
    -
    2012.03

    独立行政法人大学入試センター  教科科目第一委員会委員

▼display all

Professional Memberships

  •  
     
     

    Asian Association for Language Assessment

  •  
     
     

    International Language Testing Association (ILTA)

  •  
     
     

    Japan Language Testing Association

Research Areas

  • Foreign language education

Research Interests

  • language assessment, applied linguistics, English education

Awards

  • 2021 Language Testing Reviewer of the Year Award

    2021   Language Testing (Sage)  

    Winner: Yasuyo Sawaki

 

Papers

  • Book review: C. A. Chapelle and E. Voss (Eds.), Validity Argument in Language Testing: Case Studies of Validation Research

    Yasuyo Sawaki

    Language Testing    2023.08  [Invited]

    DOI

  • Do the TOEFL iBT® section scores provide value-added information to stakeholders?

    Sawaki, Y

    Langauge Testing   35 ( 4 ) 529 - 556  2018.10  [Refereed]

    DOI

    Scopus

    15
    Citation
    (Scopus)
  • Foreign Language Learning Motivation in the Japanese Context: Social and Political Influences on Self

    Maya Sugita McEown, Yasuyo Sawaki, Tetsuo Harada

    Modern Language Journal   101 ( 3 ) 533 - 547  2017.09  [Refereed]

     View Summary

    The study focuses on the role of different theories when considered together in a foreign language other than English (LOTE) context. Specifically, the study examines (a) to what extent influential second language (L2) motivational theories, when integrated, explain motivation to learn LOTEs, and (b) how the powerful status of English in Japan affects learners’ self- and identity-related motivation to learn LOTEs. Survey responses of 250 Japanese learners, who simultaneously learned a foreign LOTE and English as a required language, were analyzed using a structural equation modeling (SEM) approach. The results offer insight into how various coexisting social factors are connected to learners’ multiple self- and identity-related orientations, which in turn predict several varied academic consequences (e.g., effort, attitude, and L2 ability). We also confirmed positive and negative interplay of English- and LOTE-related orientations such that the self- and identity-related orientations of the languages will play a competing role (e.g., Csizér &amp
    Lukács, 2010). This finding highlights the importance of taking sociopolitical perspectives into consideration in a context where learners learn two languages and one has a specific political presence.

    DOI

    Scopus

    47
    Citation
    (Scopus)
  • The effects of different levels of performance feedback on TOEFL iBT ® Reading practice test performance.

    Sawaki, Y

    TOEFL iBT Research Report No. TOEFLiBT-29    2017  [Refereed]

    DOI

  • University faculty members’ perspectives on English language demands in content courses and a reform of university entrance examinations in Japan: a needs analysis

    Language Testing in Asia   7 ( 13 )  2017  [Refereed]

    DOI

    Scopus

    7
    Citation
    (Scopus)
  • Providing Test Performance Feedback That Bridges Assessment and Instruction: The Case of Two Standardized English Language Tests in Japan

    Yasuyo Sawaki, Rie Koizumi

    LANGUAGE ASSESSMENT QUARTERLY   14 ( 3 ) 234 - 256  2017  [Refereed]

     View Summary

    This small-scale qualitative study considers feedback and results reported for two major large-scale English language tests administered in Japan: the Global Test of English Communication for Students (GTECfS) and the Eiken Test in Practical English Proficiency (Eiken). Specifically, it examines current score-reporting practices in student and teacher score reports and supplementary materials as well as students' and teachers' perceptions and uses of reported information. First, to summarize key features of performance feedback provided, two independent judges coded the information that was included as feedback in the score reports and supplementary materials as well as how the feedback was presented for both tests, based on Roberts and Gierl's (2010) test score report analysis framework. Second, 16 students and five teachers at three upper secondary schools (Grades 10-12) in Japan were interviewed to explore how they perceived the assessment results and used them in their learning and teaching. Key study results showed that the documents generally followed principles of good score-reporting practice suggested in previous studies, while student and teacher interviews identified various issues to be considered to improve the communication of test performance feedback to its intended audiences.

    DOI

    Scopus

    7
    Citation
    (Scopus)
  • Issues of Language Assessment in Japan: Past, Present and Future: Editorial by Guest Editors of the Special Issue INTRODUCTION

    Yo In'nami, Rie Koizumi, Yasuyo Sawaki, Yoshinori Watanabe

    LANGUAGE ASSESSMENT QUARTERLY   14 ( 3 ) 189 - 191  2017  [Refereed]

    DOI

    Scopus

  • Understanding learner strengths and weaknesses: Assessing performance on an integrated writing task

    Yasuyo Sawaki, Thomas Quinlan, Yong-Won Lee

    Language Assessment Quarterly   10 ( 1 ) 73 - 95  2013.01  [Refereed]

     View Summary

    The present study examined the factor structures across features of 446 examinees' responses to a writing task that integrates reading and listening modalities as well as reading and listening comprehension items of the TOEFL iBT® (Internet-based test). Both human and automated scores obtained for the integrated essays were utilized. Based on a series of preliminary factor analyses, a confirmatory factor analysis (CFA) identified a model that specified a higher order factor for comprehension. In the model, the Comprehension factor underlay factors representing content of the written essay as well as reading and listening comprehension. The Comprehension factor correlated with two writing factors-Productive Vocabulary and Sentence Conventions. Furthermore, follow-up CFA models with covariates (multiple indicators multiple causes models, or MIMIC models) were tested to compare performance between a group of 190 examinees scoring above a frequently used TOEFL iBT Total score requirement for international student admission and the other group of 128 examinees scoring below the requirement. The higher ability group performed significantly better than the lower ability group on all three constructs: Comprehension, Productive Vocabulary, and Sentence Conventions. The identification of the multiple distinct factors in this study may hold promise for obtaining writing profiles that inform instruction in contexts such as test preparation. © 2013 Taylor &amp
    Francis Group, LLC.

    DOI

    Scopus

    46
    Citation
    (Scopus)
  • Word reading efficiency, text reading fluency, and reading comprehension among Chinese learners of English

    Xiangying Jiang, Yasuyo Sawaki, John Sabatini

    Reading Psychology   33 ( 4 ) 323 - 349  2012  [Refereed]

     View Summary

    This study examined the relationship among word reading efficiency, text reading fluency, and reading comprehension for adult English as a Foreign Language (EFL) learners. Data from 185 adult Chinese EFL learners preparing to take the Test-of-English-as-a-Foreign-Language™(TOEFL®) were analyzed in this study. The participants completed a computer-based test battery of word reading efficiency and text reading fluency in addition to TOEFL iBT (Internet-Based Test) reading comprehension. The findings showed a stronger relationship between text oral reading fluency and reading comprehension than between word reading efficiency and reading comprehension. The authors recommended that attention be paid to text oral reading fluency in adult EFL reading research and instruction. © Taylor &amp
    Francis Group, LLC.

    DOI

    Scopus

    23
    Citation
    (Scopus)
  • Examining the factor structure of a state standards-based science assessment for students with learning disabilities

    Jonathan Steinberg, Frederick Cline, &amp, Yasuyo Sawaki

    ETS Research Report   ETS RR-11-38  2011.10  [Refereed]

  • 大規模言語テストの妥当性・有用性検討に関する近年の動向

    澤木 泰代

    『言語教育評価研究』   2   54 - 63  2011.08

    CiNii

  • Relationships Among Reading Skills of Adults With Low Literacy

    John P. Sabatini, Yasuyo Sawaki, Jane R. Shore, Hollis S. Scarborough

    JOURNAL OF LEARNING DISABILITIES   43 ( 2 ) 122 - 138  2010.03  [Refereed]

     View Summary

    In this study, confirmatory factor analyses were used to examine the interrelationships among latent factors of the simple view model of reading comprehension (word recognition and language comprehension) and hypothesized additional factors (vocabulary and reading fluency) in a sample of 476 adult learners with low literacy levels. The results provided evidence for reliable distinctions between word recognition, fluency, language comprehension, and vocabulary skills as components of reading. Even so, the data did not support the hypothesis that the simple view needs to be expanded to include vocabulary or fluency factors, as has been posited in a few prior studies of younger and more able readers. Rather, word recognition and language comprehension alone were found to account adequately for variation in reading comprehension in adults with low literacy.

    DOI

    Scopus

    79
    Citation
    (Scopus)
  • Using Factor Analysis to Investigate Accommodations Used by Students with Disabilities on an English-Language Arts Assessment

    Linda Cook, Daniel Eignor, Yasuyo Sawaki, Jonathan Steinberg, Frederick Cline

    APPLIED MEASUREMENT IN EDUCATION   23 ( 2 ) 187 - 208  2010  [Refereed]

     View Summary

    This study compared the underlying factors measured by a state standards-based grade 4 English-Language Arts (ELA) assessment given to several groups of students. The focus of the research was to gather evidence regarding whether or not the tests measured the same construct or constructs for students without disabilities who took the test under standard conditions, students with learning disabilities who took the test under standard conditions, students with learning disabilities who took the test with accommodations as specified in their Individualized Educational Program (IEP) or 504 plan, and students with learning disabilities who took the test with a read-aloud accommodation/modification. The ELA assessment contained both reading and writing portions. A total of 75 multiple-choice items were analyzed. A series of nested hypotheses were tested to determine if the ELA measured the same factors for students with disabilities who took the assessment with and without accommodations and students without disabilities who took the test without accommodations. The results of these analyses, although not conclusive, indicated that the assessment had a similar factor structure for all groups included in the study.

    DOI

    Scopus

    17
    Citation
    (Scopus)
  • Cognitive Diagnosis and Q-matrices in Language Assessment: The Authors Respond

    Yong-Won Lee, Yasuyo Sawaki

    LANGUAGE ASSESSMENT QUARTERLY   7 ( 1 ) 108 - 112  2010  [Refereed]

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Q-Matrix Construction: Defining the Link Between Constructs and Test Items in Large-Scale Reading and Listening Comprehension Assessments The Authors Respond

    Yasuyo Sawaki, Hae-Jin Kim, Claudia Gentile

    LANGUAGE ASSESSMENT QUARTERLY   7 ( 1 ) 113 - 115  2010  [Refereed]

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Using factor analysis to investigate the impact of accommodations on the scores of students with disabilities on a reading comprehension assessment

    Cook, L. L, Eignor, D, Steinberg, J, Sawaki, Y, Cline, F

    Journal of Applied Testing Technology   10 ( 2 ) 1 - 33  2009  [Refereed]

  • Factor structure of the TOEFL Internet-based test

    Yasuyo Sawaki, Lawrence J. Stricker, Andreas H. Oranje

    LANGUAGE TESTING   26 ( 1 ) 5 - 30  2009.01  [Refereed]

     View Summary

    This construct validation study investigated the factor structure of the Test of English as a Foreign Language (TM) Internet-based test (TOEFL (R) iBT). An item-level confirmatory factor analysis was conducted for a test form completed by participants in a field study. A higher-order factor model was identified, with a higher-order general factor (ESL/EFL ability) and four first-order factors for reading, listening, speaking and writing. Integrated Speaking and Writing tasks, which require language processing in multiple modalities, defined the target modalities (speaking and writing). These results broadly supported the current practice of reporting a total score and four scores corresponding to the modalities for the test, as well as the test design that permits the integrated tasks to contribute only to the scores of the target modalities.

    DOI

    Scopus

    83
    Citation
    (Scopus)
  • Criterion-related Validity of the TOEFL@ iBT Listening Section.

    Sawaki, Y, Nissan, S

    TOEFL iBT Research Report   TOEFLiBT No. 8  2009  [Refereed]

  • Using factor analysis to investigate the impact of accommodations on the scores of students with disabilities on a reading comprehension assessment.

    Cook, L, Eignor, D, Steinberg, J, Sawaki, Y, Cline, F

    Applied Language Testing and Technology   10 ( 2 )  2009

  • Cognitive Diagnosis and Q-Matrices in Language Assessment

    Yong-Won Lee, Yasuyo Sawaki

    LANGUAGE ASSESSMENT QUARTERLY   6 ( 3 ) 169 - 171  2009  [Refereed]

    DOI

    Scopus

    21
    Citation
    (Scopus)
  • Cognitive Diagnosis Approaches to Language Assessment: An Overview

    Yong-Won Lee, Yasuyo Sawaki

    LANGUAGE ASSESSMENT QUARTERLY   6 ( 3 ) 172 - 189  2009  [Refereed]

     View Summary

    A variety of cognitive diagnosis approaches (CDAs) have been used by researchers to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Although some of these approaches have been in use in the psychometric community for quite some time, many of these are relatively new to language testers and have not been widely researched or discussed in the field. The goal of this article is to describe how a CDA framework could be applied to language assessment. We start with a brief history and an overview of the major components of CDAs and then compare the characteristics of psychometric models for CDAs and discuss score interpretation and reporting issues. We conclude this article by outlining some of the challenges ahead and future research avenues for the application of CDAs in language assessment.

    DOI

    Scopus

    72
    Citation
    (Scopus)
  • Q-Matrix Construction: Defining the Link Between Constructs and Test Items in Large-Scale Reading and Listening Comprehension Assessments

    Yasuyo Sawaki, Hae-Jin Kim, Claudia Gentile

    LANGUAGE ASSESSMENT QUARTERLY   6 ( 3 ) 190 - 209  2009  [Refereed]

     View Summary

    In cognitive diagnosis a Q-matrix (Tatsuoka, 1983, 1990), which is an incidence matrix that defines the relationships between test items and constructs of interest, has great impact on the nature of performance feedback that can be provided to score users. The purpose of the present study was to identify meaningful skill coding categories that reflect core language skills and processes assessed in the Reading and Listening sections of the Test of English as a Foreign Language (TM) Internet-based Test (TOEFL (R) iBT). The study was conducted as part of a research activity to explore the possibility of developing a detailed score report for low-stakes use by taking a cognitive diagnosis approach. Content experts conducted a test content analysis to develop draft Q-matrices, while measurement experts empirically analyzed the draft Q-matrices with examinee performance data using a cognitively diagnostic psychometric model called the fusion model (DiBello, Stout, & Roussos, 1995; Hartz, 2002). The draft Q-matrices were refined by repeating fusion model analysis and revision of skill definitions and item coding. This resulted in a set of Q-matrices that represented substantively meaningful score reporting categories of a suitable grain size for score reporting, while maintaining an acceptable level of examinee classification consistency.

    DOI

    Scopus

    60
    Citation
    (Scopus)
  • Application of Three Cognitive Diagnosis Models to ESL Reading and Listening Assessments

    Yong-Won Lee, Yasuyo Sawaki

    LANGUAGE ASSESSMENT QUARTERLY   6 ( 3 ) 239 - 263  2009  [Refereed]

     View Summary

    The present study investigated the functioning of three psychometric models for cognitive diagnosis-the general diagnostic model, the fusion model, and latent class analysis-when applied to large-scale English as a second language listening and reading comprehension assessments. Data used in this study were scored item responses and incidence matrices, or Q-matrices (Tatsuoka, 1983, 1990), for the Listening and Reading sections of two field test forms of the TOEFL (R) iBT (Test of English as a Foreign Language T). Results of empirical data analyses based on the three cognitive diagnosis models above revealed that (a) all three models adequately separated examinees into two mastery levels on most of the reading and listening skills, (b) a moderate level of across-form consistency of examinee skill mastery classification was achieved for most of the reading and listening skills as defined in this study, but (c) a large proportion of examinees were classified into "flat" profiles (i.e., "nonmasters of all skills" and "masters of all skills"). Overall, the three models produced similar results in terms of examinee classification, but some subtle differences across them were identified as well.

    DOI

    Scopus

    84
    Citation
    (Scopus)
  • Factor structure of the TOEFL Internet-based Test

    Sawaki, Y, Stricker, L, a, Oranje, A

    TOEFL iBT Research Report   TOEFLiBT No. 4  2008  [Refereed]

  • Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile.

    Sawaki, Y

    Language Testing   24   355 - 390  2007  [Refereed]

    DOI

    Scopus

    59
    Citation
    (Scopus)
  • The generalizability of summarization and free recall ratings in L2 reading assessment.

    Sawaki, Y

    JLTA Journal   7   21 - 44  2005  [Refereed]

    DOI CiNii

  • The development of a web-based placement test of Japanese as a foreign language.

    Hayshi, A, Farnsworth, T, Haga, Y, Kamei, S.G, Sawaki, Y, am, Yoshizawa, K

    Proceedings of the Princeton Japanese Pedagogy Workshop.    2003

  • Developing a web-based language proficiency test for college-level learners of Japanese as a second/foreign language.

    Matsunaga, S., &am, Sawaki, Y

    Proceedings of the Teachers of Japanese in Southern California Workshops.    2001

  • Comparability of conventional and computerized tests of reading in a second language.

    Sawaki, Y

    Language Learning and Technology   5 ( 2 ) 38 - 59  2001  [Refereed]

  • Language learning orientations of Showa Students and the Boston Study-abroad Program.

    Sawaki, Y

    Gakuen   698   51 - 63  1998

  • Assessing language growth with the TOEFL Institutional Testing Program.

    Sawaki, Y

    Gakuen   674   55 - 65  1997

  • Japanese learners' language learning motivation: A preliminary study.

    Sawaki, Y

    JACET Bulletin   28 ( 28 ) 83 - 96  1997

    CiNii

  • Phonological analysis of vowels in Montana dialect.

    Nishikawa, M, Moriyama (Sawaki), Y

    The Memoirs of the Faculty of Education, Kumamoto University, 40, The Humanities   40   149 - 186  1991

▼display all

Books and Other Publications

  • Computer-based testing. In G. Fulcher & L. Harding (Eds.), The Routledge handbook of language testing (2nd ed., pp. 530-544)

    G. Fulcher, L. Harding

    2022

  • Generalizabiilty theory. In B. B. Frey (Ed.), The Sage encyclopedia of research design (2nd ed.. pp. 618-620)

    B. B. Frey

    Thousand Oaks, CA: Sage  2022

  • Developing summary content scoring criteria for university L2 writing instruction in Japan. In G. J. Ockey, & B. Green (Eds.), Another generation of fundamental considerations in language assessment: A festschrift in honor of Lyle F. Bachman (pp. 153-171)

    Gary J. Ockey, B. Green(pp. 153-171)

    New York: Springer  2020

  • Univariate generalizability theory in language assessment, In V. Aryadoust & M. Raquel (Eds.), Quantitative data analysis for language assessment (Vol 1, pp.30-53)

    V. Aryadoust, M. Raquel

    New York: Routledge  2019

  • Methods of test validation. In E. Shohamy, L. Or, & S. May (Eds.), Encyclopedia of language and education, Volume 7, Language testing and assessment (pp. 193-209)

    Xi, X, Sawaki, Y( Part: Joint author)

    New York: Springer  2017

  • Norm-referenced vs. criterion-reference assessment. Handbook of second language assessment (pp. 45-60)

    D. Tsagari, J. Banerjee( Part: Sole author)

    Berlin, Germany: De Gruyter  2016

  • Large-scale assessments of English for academic purposes from the perspective of English as a lingua franca. Exploring ELF in Japanese academic and business contexts: Conceptualization, research and pedagogic implications (pp. 224-239)

    K. Murata( Part: Sole author)

    New York: Routledge  2015

  • Classical test theory. The companion to language assessment (pp. 1147–1164)

    A. Kunnan( Part: Sole author)

    New York: Wiley  2013

  • The Routledge Encyclopedia of Second Language Acquisition (担当:pp. 616-617, "Structural equation modeling (SEM)"

    Peter Robinson

    Routledge  2013 ISBN: 9780415877510

  • The Routledge Handbook of Language Testing (担当:Chapter 29, "Technology in language testing")

    G. Fulcher, F. Davidson( Part: Sole author)

    Routledge  2012.03 ISBN: 9780415570633

  • The encyclopedia of applied linguistics (担当:pp. 2073-2077, "Factor analysis").

    C. A. Chapelle

    New York: Wiley  2012

  • The encyclopedia of applied linguistics (担当:pp. 5422-5427, "Structural equation modeling in language assessment").

    C. A. Chapelle

    New York: Wiley  2012

  • Encyclopedia of research design (担当:pp. 533-538, "Generalizability theory")

    Neil J. Salkin

    Sage Publications  2010.06 ISBN: 9781412961271

▼display all

Presentations

  • Assessing content for enhancing construct representation in source-based writing performance assessment

    Sawaki, Y

    40th Language Testing Research Colloquium  (Auckland)  International Language Testing Association

    Presentation date: 2018.07

  • Generalizability of content analytic rating scales for assessing university-level Japanese EFL learners’ summarization performance

    Sawaki, Y  [Invited]

    Fundamental Considerations in Language Testing: An International Conference in Honor of Lyle Bachman  (Salt Lake City) 

    Presentation date: 2017.06

  • English language demands at universities; What is the construct of “academic English” in the Japanese context? In L. Cheng (Chair), How does context mediate construct and content? A political, social, and pedagogical analysis of English language testing in Asia

    Sawaki, Y

    38th Language Testing Research Colloquium  (Palermo)  International Language Testing Association

    Presentation date: 2016.06

  • English language demands at universities; What is the construct of “academic English” in the Japanese context? In L. Cheng (Chair), How does context mediate construct and content? A political, social, and pedagogical analysis of English language testing i

    Sawaki, Y

    38th Language Testing Research Colloquium  (Palermo)  International Language Testing Association

    Presentation date: 2016.06

  • Japanese students’ and teachers’ perception and use of score reports for two large-scale EFL tests

    Sawaki, Y, Koizumi, R

    37th Language Testing Research Colloquium  (Toronto)  International Language Testing Association

    Presentation date: 2015.03

  • The effects of feedback generated by an automated essay scoring program on Japanese EFL learners’ writing performance

    Sawaki, Y

    1st Annual Conference of the Asian Association for Language Assessment  (Hangzhou)  Asian Association for Language Assessment

    Presentation date: 2014.10

  • A review of large-scale academic English tests from an ELF perspective

    Sawaki, Y

    2nd Waseda English as a Lingua Franca [ELF] International Workshop  (Tokyo) 

    Presentation date: 2013.03

  • TOEFL iBT 教育者セミナー

    TOEFL iBT 教育者セミナー(新潟県立国際情報高等学校) 

    Presentation date: 2012.09

  • 教室での英語コミュニケーション能力の評価(講習記号:8224)

    平成24年度早稲田大学教員免許更新講習 

    Presentation date: 2012.08

  • 教育現場での英語テスト: より妥当性の高い英語テストの実践を目指して

    2012年度第3回TALK (田辺英語教育学研究会)研究例会(早稲田大学) 

    Presentation date: 2012.07

  • 「探索的因子分析」

    日本言語テスト学会全国大会(桃山学院大学) 

    Presentation date: 2011.10

  • TOEFL iBTの得点アップを目指して

    2011スキルアップセミナー 

    Presentation date: 2011.07

  • TOEFL iBTの得点アップを目指して

    2010スキルアップセミナー 

    Presentation date: 2010.11

  • The value of using TOEFL iBT in university-level English education

    Presentation date: 2010.10

  • Recent developments of research on examining the validity and usefulness of large-scale language assessments

    Presentation date: 2010.09

  • Current research on diagnosing second language ability: an overview

    Annual Meeting of the Japan Language Testing Association 

    Presentation date: 2010.09

  • Investigating the value of reporting TOEFL iBT section scores

    Sawaki, Y  [Invited]

    TOEFL Research Symposium  (Philadelphia)  Educational Testing Service

    Presentation date: 2009.09

  • Relationship of the TOEFL iBT Listening section to academic lecture listening

    Sawaki, Y  [Invited]

    TESOL Program, Temple University (invited presentation) 

    Presentation date: 2009.09

  • 統計セミナー (ワークショップ)

    JACET第二言語習得研究会 

    Presentation date: 2009.08

  • Understanding learner strengths and weaknesses: measuring performance on an integrated writing task.

    31st Language Testing Research Colloquium (LTRC) 

    Presentation date: 2009.03

  • Criterion-related validity of the TOEFL? iBT Listening section.

    30th Language Testing Research Colloquium (LTRC) 

    Presentation date: 2008.06

  • Application of generalizability theory in language assessment.

    30th Language Testing Research Colloquium (LTRC) 

    Presentation date: 2008.06

  • Component reading skill efficiency and reading comprehension for adult ESL/EFL learners.

    American Association for Applied Linguistics (AAAL) 

    Presentation date: 2008.03

  • Application of three cognitive diagnosis models in ESL reading and listening tests.

    29th Language Testing Research Colloquium (LTRC) 

    Presentation date: 2007.06

  • Q-matrix construction: Defining the link between constructs and test items in cognitive diagnosis approaches.

    29th Language Testing Research Colloquium (LTRC) 

    Presentation date: 2007.06

  • Cognitive diagnosis in language assessment: An overview.

    29th Language Testing Research Colloquium (LTRC) 

    Presentation date: 2007.06

  • The factor structure of an ESL test with tasks that integrate modalities.

    Annual Meeting of the National Council on Measurement in Education (NCME) 

    Presentation date: 2007.04

  • Efficiency and fluency in ELL reading comprehension.

    Annual Meeting of the American Educational Research Association (AERA) 

    Presentation date: 2007.04

  • Using factor analysis to investigate the impact of accommodations on the sores of students with disabilities on English-Language Arts assessments.

    National Council on Measurement in Education 2006 Annual Meeting and Training Sessions 

    Presentation date: 2006.04

  • Investigating the dimensionality of an English-Language Arts assessment administered to English-language learners with and without accommodations.

    Annual Meeting of the American Educational Research Association 

    Presentation date: 2006.04

  • Examining the structure of TOEFL iBT: Implications for validity.

    9th Annual Conference of Southern California Association for Language Assessment Research 

    Presentation date: 2006.04

  • TOEFL iBT: Improving the link between English language assessment and instruction.

    9th Annual Conference of Southern California Association for Language Assessment Research 

    Presentation date: 2006.04

  • Sharing TOEFL iBT standard setting experiences.

    40th Annual TESOL Convention and Exhibit 

    Presentation date: 2006.03

  • TOEFL and Arpeggio help with classroom harmony?

    39th TESOL Convention and Exhibit 

    Presentation date: 2005.03

  • Standard setting for Next Generation TOEFL.

    39th TESOL Convention and Exhibit. 

    Presentation date: 2005.03

  • A comparison of summarization and free recall as L2 reading test tasks.

    The 8th Annual Conference of the Japan Language Testing Association. 

    Presentation date: 2004.09

  • Multivariate analytic approaches to the validation of a speaking test.

    24th Language Testing Research Colloquium 

    Presentation date: 2002.12

  • A multivariate generalizability analysis of the dependability of ratings in a Spanish speaking test.

    4th Southern California Association for Language Assessment Research 

    Presentation date: 2002.05

  • How examinees take conventional vs. web-based Japanese reading tests.

    23rd Language Testing Research Colloquium 

    Presentation date: 2001.02

  • Developing a web-based language proficiency test for college-level learners of Japanese as a second/foreign language.

    Teachers of Japanese in Southern California 

    Presentation date: 2000.11

  • A literature review of comparability of paper-and-pencil vs. computer-administered reading tests.

    3rd Annual Conference of the Southern California Association for Language Assessment Research 

    Presentation date: 2000.05

  • Test takers' performances on and affective responses to computer-administered Japanese reading tests.

    22nd Language Testing Research Colloquium 

    Presentation date: 2000.03

  • Developing a web-based language placement examination system.

    22nd Language Testing Research Colloquium 

    Presentation date: 2000.03

  • The learning style preferences of Japanese EFL students.

    31st Annual Teachers of English to Speakers of Other Languages Convention 

    Presentation date: 1997.03

  • Construct validation of the United Nations Association Test of English (UNATE) Level A Test and its implications for further development of the test.

    17th Language Testing Research Colloquium 

    Presentation date: 1995.03

  • Construct validation of the United Nations Association Test of English.

    29th Annual Teachers of English to Speakers of Other Languages Convention 

    Presentation date: 1995.03

▼display all

Research Projects

  • Developing and validating an online tool for learning and teaching how to write summaries in English

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2020.04
    -
    2024.03
     

  • Examining Japanese university students' ability to write summaries in English and developing summarization instruction materials for university-level L2 writing education in Japan

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2016.04
    -
    2019.03
     

    Sawaki Yasuyo

     View Summary

    The primary purposes of the present study were to examine Japanese university students' ability to read a text in English and write a summary of it in English and to develop summarization instruction materials for university-level English writing education in Japan. Through combining a variety of quantitative and qualitative research methods, this study examined the validity of analytic rating scales for assessing summary writing performance as well as the consistency of scores assigned by raters to learner-produced summaries based on those rating scales. The study also identified characteristics of the process and product of Japanese university students’ summary writing performance. Summarization instruction materials developed as part of this study were compared against an existing second language academic writing course textbook in terms of the effects on learners' summary writing performance as well

  • Examining Japanese university students' summary writing ability and developing materials for summarization instruction in university English courses

    Waseda University  Grants-in-Aid for Scientific Research

    Project Year :

    2016.04
    -
    2019.03
     

  • Investigating the relative effectivenes of various types of performance feedback on reading performance for TOEFL iBT test preparation

    Educational Testing Service  TOEFL Committee of Examiners Research Grant

    Project Year :

    2010.10
    -
    2016.03
     

  • The Use of English as a Lingua Franca (ELF) in Academic and Business Contexts in Japan – An analysis of its use and implications for language education

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2011.04
    -
    2015.03
     

    MURATA KUMIKO, YANO Yasukata, IINO Masakazu, TERAUCHI Hajime, TUCHIYA Keiko, BACKHAUS Peter, HARADA Tetsuo, SAWAKI Yasuyo, HOUSE Juliane, KUBOTA Ryuko, SEIDLHOFER Barbara, WIDDOWSON Henry

     View Summary

    The contribution of the current research is classfied into the following three:① the deepening of understanding of the concept of English as a lingua franca (ELF) ② detailed description and understanding of the use of ELF in actual situations and ③ heightening awareness of the use of ELF and impliacations for language pedagogy. The first has been mainly achieved by holding various workshops and publishing working papers based on them. The second has been made possible by collecting and analysing various types of data from diverse ELF users in their use of ELF and by showing how cooperatively and successfully they achieve their communicative needs. Thirdly, disseminating these results and their implications by way of conference presentations and publications has contributed to heightening the awarness of people about ELF and to furthering pedagogical implications

 

Syllabus

▼display all

 

Overseas Activities

  • 日本人大学生の英語要約力検証と大学英語ライティング教育用要約教材の開発

    2017.08
    -
    2018.08

    米国   Educational Testing Service

Sub-affiliation

  • Faculty of Education and Integrated Arts and Sciences   Graduate School of Education

  • Affiliated organization   Global Education Center

Internal Special Research Projects

  • 日本人大学生を対象とする英語要約ライティング指導用フィードバックの作成

    2019   徳永 健伸

     View Summary

    本研究では、英文を読み、内容を英語で要約する力の育成をはかる大学アカデミック・ライティング指導用オンライン教材作成の準備を行った。第一に、早稲田大学教育学部英語英文学科設置アカデミック・ライティング科目授業で教材原案の一部を試行し、授業担当教員・履修者から得た意見や感想を基に教材内容を吟味した。第二に、教材で提供するフィードバック方法を検討するため、上記授業の履修者90名の要約解答データを質的に分析した。複数の評定者による分析と討議によって要約内容やパラフレーズの適切性を評価する方法を検討すると共に、これらの分析を一部自動化する方法について計算言語学研究者と意見交換した。

  • 日本の英語標準化テストにおけるテスト結果フィードバックの現状に関する考察

    2013  

     View Summary

    1.本研究の背景と目的現在主流となっている言語テスト妥当性理論においては、テスト得点だけでなく、テスト結果についてより詳細な情報を含むフィードバックを、語学学習者や教師に対してスコア・レポートの形で提供することの重要性を主張している(例:Bachman & Palmer, 2010; Chapelle, Enright, & Jamieson, 2008)。このようなフィードバックは、学習者の長所や弱点などが容易に解釈でき、その後の指導や学習者の自主学習への橋渡しになるような具体的な内容であることが求められる。しかし、様々なテスト機関や出版社等によって実施されている言語テストのスコア・レポートがどのような方針で作られ、またそのそれぞれにどのような特徴があるか、またスコア・レポートを受け取る側の学習者や教師がその内容をどのように理解し、その後の学習や指導に活かしているかに関する系統的な分析は、まだ十分に行われていない。 本研究では、今後言語テストのスコア・レポートの内容を改善し、適切な使用を促進していく一助として、現在日本で広く使用されている英語標準化テストの運営元が発行するスコア・レポートやその付属資料として提供しているフィードバックの特徴と、教育活動の一貫としての英語学習者・教師によるスコア・レポート使用の実態を調査する。本研究では、日本での学校英語教育において広く使用されている2つの標準化テスト、the Global Test of English Communication (GTEC) for STUDENTS(GTECfS:株式会社ベネッセ・コーポレーション)とthe Test in Practical English Proficiency(英検:日本英語検定協会)の2つに焦点をあて、テスト結果がどのように学習者や教師に報告され、それを受けた学習者や教師がその情報をどのように解釈し、利用しているかを検証した。2.手法(1)参加者本研究には、首都圏の3つの高等学校に所属する高校1年生から3年生16名と英語科教員5名が参加した。参加した高校生のGTECfSのグレードは、7レベルのうちグレード4からグレード7(中央値、最頻値共にグレード6)であった。ベネッセ・コーポレーションが高校卒業時に望まれる英語力レベルとして推奨しているのがグレード5であるため、比較的英語が得意な生徒が本研究に参加したと考えられる。また、教員参加者は、勤続年数が7年から27年の中・高等学校での教歴を有する日本人英語教員であった。参加した3つの高等学校のうち二校(高校A、高校B)はそれぞれ別の私立大学の附属高校であり、残る一校(高校C)は以前文科省によりSELHi(Super English Language High School)に指定された都立高校である。いずれの高校も、2013年秋に本研究を開始するにあたって、今回の研究対象となるGTECfS、あるいは英検を学校で実施している高校に対して本研究の研究代表者・研究協力者が個別に連絡し、協力を依頼した。高校Aと高校Bの二校はいずれもGTECfSのみを学校で定期的に実施し、英検は興味のある生徒が自主的に申し込んで受験している。高校Aでは生徒の英語力の把握や選択科目でのクラス分け、また学内推薦の際の英語力の指標としてGTECfSスコアを用い、生徒の必修英語科目の成績の一部に加味している。これに対し、高校Bでは英語科目の成績にGTECfSを加味することはなく、近年実施された英語カリキュラム改革の効果測定の一環として、生徒の英語力の伸びの確認のために用いている。一方高校Cでは、GTECfSと英検の両方を学校で定期的に実施し、GTECfSは生徒の英語学習の動機付けや英語力の伸びの確認、英語による校外学習の際のクラス分けに用いている。また英検準2級を高校Cの入試の一部で導入しており、準2級と2級は大学への推薦入学等を希望する学生に必要な英語力の指標と位置づけ、学校でも英検受験者対象の事前指導などを積極的に行なっている。(2)調査対象資料今回調査対象となったGTECfSと英検については、テスト運営元が発行する情報以外に、他の出版社の出版物やウェブサイト、ブログなどで様々な情報が出回っているが、本研究では各テストの運営元が発行しているスコア・レポートと関連資料のみを調査対象とした。具体的には、GTECfSについては生徒用のスコア・レポート、教師用帳票、問題冊子、解答解説冊子とテスト復習用教材の「STEP UPノート」(全て印刷物として試験実施校に配布)、また英検については1次・2次試験の受験者用スコア・レポートと、英検ウェブサイトを通じて公開されている問題冊子、解答速報、リスニングの音源とスクリプトを調査対象とした。(3)調査方法と分析本研究では2種類の調査を行なった。まず、調査対象となったスコア・レポートや資料で報告される情報の内容や形式について把握するために、認知診断とインフォメーション・デザインの枠組みに基づいた、Roberts and Gierl (2010) のスコア・レポートに含まれる情報の分類に従って、研究代表者・研究協力者が判定者となり、独立にそれぞれの資料を質的に分析した。その後判定者同士でその結果を話し合い、各テストについて報告されるフィードバックの内容や形式の特徴をまとめた。次に、テスト結果のフィードバックを受け取る側の学習者や教師がどのようにその情報を解釈し、日々の英語学習や指導に利用しているかを検証するために、参加校それぞれにおいて研究代表者・研究協力者が参加者のグループ・インタビューを行なった。高校Aと高校Bでは、それぞれ生徒を3名ずつの2グループに分け、また高校Cでは生徒参加者4名を1グループとしてインタビューを行なった。教員のインタビューは、高校Aは1名の参加者の単独インタビュー、高校BとCでは各2名の参加者によるグループでインタビューした。生徒のインタビューにおいては、それぞれのテストについて (a) スコアレポートを受け取った時に考えたこと、(b) スコアレポートの内容の理解度やわかりやすさ・見やすさについて、(c) スコア・レポートやその関連資料をどのように学校での学習や自主学習に役立てたか、また (d) スコア・レポートと関連資料の改善アドバイスについてディスカッションを行なった。教員インタビューについても上記 (b)、(c)、(d)を扱ったが、それに加えて学校での英語教育における各テストの利用の位置づけについても参加者に話を聴いた。各インタビューは日本語で実施した。生徒のインタビューは各グループにおいて92~124分、教員インタビューはそれぞれ67分~77分であった。ビデオによる録画とICレコーダーによる録音を行い、生徒インタビューはテープ起こしされた原稿、また教員インタビューについてはビデオ録画と録音に基づいて研究代表者・研究協力者が取った詳細なノートを分析対象とした。生徒のインタビュー・データは、上記のRoberts and Gierl (2010)のテストのスコア・レポートで報告される情報の分類に従って、研究代表者・研究協力者が判定者となり、独立してその内容を質的に分析した。一方、教師のインタビュー・データについては話の内容が多岐にわたりRoberts and Gierlの枠組みでの分析が難しかったため、この枠組みには従わず、インタビュー中に出てきたテーマごとに質的な分析を行った。生徒・教師のインタビュー結果それぞれについて、その後判定者同士でその結果を話し合い、本研究に参加した生徒と教師がどのようにテスト結果のフィードバックを解釈し、利用しているかについてまとめた。3.主な結果(1)スコア・レポートと関連資料で報告される情報の内容と形式について主な結果は大きく分けて3点にまとめられる。第一に、いずれのテストも、主なテスト結果を複数の方法を用いて報告している。例えばGTECfSでは総合と各セクションに関するスコアを尺度点と7段階のグレードで、また英検では「合格・不合格A・不合格B」と総合得点(素点)で主なテスト結果を示し、その解釈を容易にするために様々な統計(例:GTECfSの校内・全国平均点、また英検の合格者平均点)やCan-Doリストを添えている。第二に、主なテスト結果だけでなく、より詳細にテスト結果を分析し、大問別得点やスピーキングやライティング・タスクの評価基準と得点、各項目に対する受験者解答の正誤等を掲載し、受験者の長所や弱点、また誤答した項目などが一覧できる。また問題冊子他の資料(例:GTECfSの解答・解説冊子、リスニングのスクリプトと音源)を併用することにより、事後の詳細な復習も可能になる。最後に、両テストともに統計、文字、図表など様々な形式の情報を組み合わせて、読む側にとって見やすい構成・レイアウトの資料を作成している。(2)高校生と教員によるスコア・レポートと関連資料の情報の解釈と英語教育・学習におけるその活用についてまず、生徒がスコア・レポートを受け取った際に考えたことについて注目すべき結果は、多くの生徒が実際に目を通したスコア・レポートの箇所は、GTECfSの尺度点や前回からのスコアの伸び、英検の合否結果など、生徒個人が興味がある部分、あるいは図表などがあり目立つ部分などごく一部に限られており、スコア・レポートを受け取った際はその他の情報には注意が向いていなかったということである。この結果、生徒のインタビューにおいては、参加者にスコア・レポートの各ページやセクション、あるいは関連資料を一つ一つ示し、改めて目を通してもらい、内容の理解度やわかりやすさ等についてコメントしてもらった。その結果の主なポイントは3点で、まずスコア・レポートの各ページやセクションにおいて、生徒参加者が理解できない箇所は殆んどなく、特に統計データの表やグラフ等についてはわかりやすいという評価であった。その一方で、文章による説明には様々な意見が寄せられた。例えば学習に関するアドバイスはパラグラフ形式ではなく箇条書きにした方が読みやすいといった形式に関するものもあれば、あるレベルの英語学習者が一般的に英語でできることを記したCan-Doリストの内容の妥当性や一般化可能性、またGTECfSのライティングについては、受験者のエッセーに対する採点者のコメントや添削記号の理解しづらい点などが挙げられた。一方、教員参加者も、様々な形で提供される情報はよく練られておりわかりやすいという全体評価であったが、その一方で文章によるコメントについては、例えば英検の「学習アドバイス」については抽象的な表現をより具体化したり、高校生にもわかりやすくするためにやさしい表現を使うなど、フィードバックの内容をより明瞭にするための様々な提案があった。また、スコアレポートや関連資料を受け取った後、実際にどのようにその情報を英語指導や学習に役立てているかについては、数名の生徒を除いては、テスト結果を簡単に確認するのみに終わっており、テスト結果についてのフィードバックが復習やさらなる英語学習に活かされている例はごく少数であった。また、教員については、例えばGTECfSのライティング評価基準やエッセー・トピック、生徒のエッセー解答例をライティングの授業計画の参考にする例は複数報告された一方、詳細なデータ分析のもとに生徒の長所と弱点を分析し、事後の指導で弱点を重点的に扱うなどの指導の工夫例は報告されなかった。4.結論今回調査対象となったGTECfSと英検については、スコア・レポートや関連資料で報告される情報には、概ね学習者や教員が十分にその内容を理解できる程度の具体性があり、その提示方法も見やすいように工夫されているといえる。これは過去に主に教育評価の分野において海外で実施された研究で特定された、望ましいテスト結果報告方法の方針に概ね沿う内容であった(例:Zenisky & Hambleton, 2012)。また、高校生と英語教員によるその内容理解と英語指導・学習への使用については、内容的には理解できるが、これらの資料の限られた部分にしか読み手の注意が向いておらず、またその情報の事後使用も限られていた。今後検討すべき課題としては、海外の研究ではテスト結果フィードバックに不可欠と考えられていながら、GTECfSと英検のいずれでも報告されていなかった測定誤差や測定の信頼性など、その理解に一定の統計的知識を要する情報を、どのように学習者や教員に伝えていくか、また様々な情報がわかりやすく提示されていると今回評価された各資料について、なぜ読み手(特に英語学習者である生徒)の注意がその一部にしか向かず、事後の英語指導・学習での利用も限られているのかについては、今後更なる検証が待たれる。今回調査対象としたGTECfSと英検については、テスト結果をわかりやすく読み手に伝えるような様々な工夫が運営元によってなされてはいるが、使う側となる学習者や教員の視点も考慮しながら、今後テスト結果のコミュニケーションをさらに効率化していく余地は十分残されていると言えるであろう。参考文献Bachman, L. F., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (2008). Building a validity argument for the Test of English as a Foreign Language™ . New York: Routledge.Roberts, M. R., & Gierl, M. J. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29(3), 25-38. doi:10.1111/j.1745-3992.2010.00181.xZenisky, A. L., & Hambleton, R. K. (2012). Developing test score reports that work: The process and best practices for effective communication. Educational Measurement: Issues and Practice, 31(2), 21-26.