From Keystrokes to Achievement Scores: The Moderating Effects of Computer Use on Writing

Mark Warschauer (University of California, Irvine, School of Education)Computer_keyboard
Jamal Abedi (University of California, Davis, School of Education)

Tamara Tate (University of California, Irvine School of Education)

Project Funding:
Spencer Foundation, Grant No. 201500153, $49,882

Writing is difficult and changes in modality matter:  Students perform better in whichever mode (e.g., pen and paper versus computer) they are most proficient (Horkay, Bennett, Allen, Kaplan, & Yan, 2006). Thus, whenever a singular mode of delivery is chosen for an assessment, some students are disadvantaged.

Concerns about the effect of computer-based assessments on students without regular access to technology abound. As educators implement computer-based assessments for NAEP and the Common Core State Standards, the impact of prior computer use becomes increasingly relevant to achievement scores.

This secondary data analysis will look at the process of digital writing as represented in the 2011 National Assessment of Educational Progress (NAEP). This assessment is the first large, national writing test given on computer to secondary students (see discussion in NAGB, 2010), allowing the collection of keystroke-level data to investigate the details of the process of digital writing. Students had access to common word-processing tools for editing (e.g., cut, copy, paste); formatting (e.g., indenting); spelling, grammar, and reference (spell check, thesaurus, dictionary); viewing and reviewing during the assessment (NAGB, 2010). In addition, the assessment included student and teacher questionnaires that gathered information about students’ prior use of computers. While descriptive relationships about this data have been published, inferential statistics have not been used to validate an accurate model of the impact of prior computer access and computer use during an assessment on writing achievement.

This research will analyze the NAEP data to determine the appropriate model of the impact of prior access to and use of computers (prior use) and use of the computer during the assessment (use during) on writing achievement. Current statistical methods allow us to model keystrokes and functions across groups of students, to understand the relationships between achievement and reported prior use of computers, and to deconstruct the differences between successful and less successful writers by gender, ethnicity, status as a current or former English language learner, socioeconomic status, and disability.

This study will examine the following research questions:

1.     What is the effect of prior computer use on writing achievement?
2.     What is the effect of computer use during the assessment on writing achievement?
3.     What is the relationship, if any, between prior computer use and computer use during the assessment in impacting achievement?
4.     Are there heterogeneous effects across demographic groups?

We draw on a sociocultural perspective that views human activity as mediated by tools (Vygotsky, 1981; Wertsch, 1991). From this perspective, what is significant about various tools is not their abstract properties, but rather, when included in the process of behavior, how they may potentially alter the flow and structure of mental functions. Furthermore, this alteration unfolds developmentally, so that analysis of individual events needs to be paired with examination of the development of the individual (Vygotsky, 1962). This leads us to query how the introduction of a powerful tool such as a computer may transform the writing process and how that transformation may be shaped by prior experiences in students’ lives.

Information of ongoing activities.
Tamara has completed the analysis of the impact of prior use and found a small correlation between computer-based writing for school-related purposes.  More casual use of technology did not show a significant correlation.  The results are currently being submitted for publication.  Work during fall 2015 will focus on the keystroke analysis, to better understand the keyboarding that lead to better writing achievement on the assessment.  For example, did higher use of the cut and paste function correlate with higher overall achievement?

American Educational Research Association (AERA) Annual Meeting 2015 Graduate Student Roundtable Presentation.

For additional information:


Back to projects