Lee, Y., Gentile, C., & Kantor, R. (2010). Toward automated multi-trait scoring of essaysInvestigating links among holistic, analytic, and text feature scores. Applied Linguistics, 31, 391-417.
摘要：The main purpose of the study was to investigate the distinctness and reliability of analytic (or multi-trait) rating dimensions and their relationships to holistic scores and e-rater essay feature variables in the context of the TOEFL computer-based test (TOEFL CBT) writing assessment. Data analyzed in the study were holistic and multi-trait essay scores provided by human raters and essay feature variable scores computed by e-rater (version 2.0) for two TOEFL CBT writing prompts. It was found that (i) all of the six multi-trait scores were not only correlated among themselves but also correlated with the holistic score, (ii) high correlations obtained among holistic and multi-trait scores were largely attributable to the impact of essay length on both holistic and multi-trait scoring, and (iii) some strong associations were confirmed between several e-rater variables and multi-trait rating dimensions. Implications are discussed for improving the multi-trait scoring of essays, refining e-rater essay feature variables, and validating automated essay scores.
关键词：second language essay scoring, holistic scoring, analytic scoring, automated scoring
Bacha, N. (2001). Writing evaluation: What can analytic versus holistic essay scoring tell us?. System, 29(1), 371-383.
Phillips, S. M. (2007). Automated essay scoring: A literature review. Society for the Advancement of Excellence in Education.