Vocabulary Profile as a Measure of Vocabulary Sophistication

This study presents a method that assesses ESL learners ’ vocabulary usage to improve an automated scoring system of sponta-neous speech responses by non-native English speakers. Focusing on vocabulary sophistica-tion, we estimate the difficulty of each word in the vocabulary based on its frequency in a reference corpus and assess the mean diffi-culty level of the vocabulary usage across the responses (vocabulary profile). Three different classes of features were gen-erated based on the words in a spoken re-sponse: coverage-related, average word rank and the average word frequency and the extent to which they influence human-assigned lan-guage proficiency scores was studied. Among these three types of features, the average word frequency showed the most predictive power. We then explored the impact of vocabulary profile features in an automated speech scor-ing context, with particular focus on the im-pact of two factors: genre of reference corpora and the characteristics of item-types. The contribution of the current study lies in the use of vocabulary profile as a measure of lexical sophistication for spoken language as-sessment, an aspect heretofore unexplored in the context of automated speech scoring.