Investigating Virtual Learning Environments

Mark Warschauer, University of California, Irvine, School of Education
Di Xu, University of California, Irvine, School of Education
Padhraic Smyth, University of California, Irvine, UCI Donald Bren School of Information and Computer Sciences
Teomara Rutherford, North Carolina State University, Raleigh, Department of Curriculum, Instruction and Counselor Education
Rachel Baker, University of California, Irvine, School of Education
Brian Sato, University of California, Irvine, School of Biological Sciences

Postdoctoral Scholar
Fernando Rodriguez, University of California, Irvine, School of Education
Christian Fischer, University of California, Irvine, Teaching and Learning Research Center

Research Staff
Kameryn Denaro, University of California, Irvine, Teaching and Learning Research Center

Graduate Students
Bianca Cung (Education)
Qiujie Li (Education)
Jiyhun Park (Computer Science)
Peter McPartlan (Education)
Mariela Rivas (Education)
Renzhe Yu (Education)

Funding Source: NSF Grant Number 1535300, 2015-2020


With 69% of higher education institutions stating that online learning is a critical part of their long-term strategy and 32% of higher education students taking at least one course online (Allen & Seaman, 2013), new forms of virtual learning are raising the specter of radically transformed undergraduate education. This is particularly true for lower-division courses in science, technology, engineering, and mathematics (STEM), which disproportionately rely on large lectures. The possibilities of providing lecture material online, as is done through the popular Khan Academy and new massive online open courses (MOOCs), suggest to many that virtual learning may have the potential to serve as a comparable or even advantageous alternative to traditional in person lectures. Debates on this issue highlight both the high hopes and potential pitfalls. To date, few rigorous studies have convincingly demonstrated the effects of virtual learning. There have been a number of studies, including some meta-analyses, that have found benefits linked with virtual learning; however, only a handful have used rigorous experimental or quasi-experimental methods to identify causal effects (e.g., Figlio, Rush, & Yin, 2010; Xu & Jaggars, 2013).

In sum, despite their increasingly important role and fast growth in the national education landscape, the field has limited information about the impacts of different course delivery formats on student learning outcomes and on potential strategies to improve their effectiveness. To fill this research gap, the project intends to systematically explore the effectiveness of virtual learning in the particular setting of STEM courses in large research universities. This setting offers the means to test moderation hypotheses regarding student populations, course content, and instructional practices within a context that plays a special role in the STEM pipeline to graduate study and advanced careers. In keeping with the theory of affordances (see Warschauer, 2003) in which we set our work, by conducting both experiments and observations across a broad swath of courses, we aim to flesh out the heterogeneous effects of virtual learning environments and identify what kinds of environments and instructional techniques are most potent for aiding diverse learners.

This project thus has three major goals:

1) to explore how different types of virtual learning environments affect teaching and learning processes, outcomes, and attitudes toward STEM, both in general and for particular groups of underrepresented learners

2) to examine how variation in instructional practices is associated with student learning gains for the purposes of distilling potential strategies to design high-quality virtual learning environments and improve student learning;

3) to understand the student experience of learning within virtual learning environments through investigation of learning behaviors and the ways in which these behaviors relate to student outcomes

For more information contact Mark Warschauer at


Refereed Conference Proceedings

Park, J., Yu, R., Rodriguez, F., Baker, R., Smyth, P., & Warschauer, M. (2018). Understanding student procrastination via mixture models. In Proceedings of the 11th International Conference on Educational Data Mining. Buffalo, NY.  Best Paper Award. [Paper]

Li, Q., Baker, R., & Warschauer, M. (2018). Measuring student self-regulated learning in an online class. In Proceedings of the Eight International Learning Analytics & Knowledge Conference[Paper]

Yu, R., Jiang, D., & Warschauer, M. (2018). Representing and Predicting Student Navigational Pathways in Online College Courses. In Proceedings of the 5th ACM Conference on Learning at Scale. London, United Kingdom. [Paper]

Park, J., Denaro, K., Rodriguez, F., Smyth, P., & Warschauer, M. (2017). Detecting changes in student behavior from clickstream data. In Proceedings of the seventh International Learning Analytics and Knowledge Conference. Vancouver, BC, Canada. Best Paper: Honorable Mention[Paper]

Journal Articles

Baker, R., Evans, B., Li, Q., & Cung, B. (2018). Does inducing students to schedule lecture watching in online classes improve their academic performance? An experimental analysis of a time management intervention. Research in Higher Education. [Paper]

Rodriguez, F., Rivas, M. J., Matsumura, L., Warshauer, M., & Sato, B. (2018). How do students study in STEM courses? Findings from a light touch intervention and its relevance for underrepresented students. PLoS ONE13(7), 1-20. [Paper]

Rodriguez, F., Kataoka, S., Rivas, M., Kandandale, P., Nili, A., & Warschauer, M. (2018). Do spacing and self-testing predict learning? Active Learning in Higher Education, 1-15. [Paper]

Casasola, T., Nguyen, T., Warschauer, M., & Schenke, K. (2017). Can flipping the classroom work? Evidence from undergraduate Chemistry. The International Journal of Teaching and Learning in Higher Education, 29(3), 421-435. [Paper]

He, W., Holton, A., Farkas, G., & Warschauer, M. (2016). The effects of flipped instruction on out-of-class study time, exam performance, and student perceptions. Learning and Instruction, 45, 61-71. [Paper]

Conference Presentations

Baker, R., Evans, B., Li, Q., & Cung, B.  (2017). Nudging students to better academic outcomes: A randomized control trial of a scheduling intervention to improve persistence and performance in online postsecondary courses. Paper presented at annual Association for Education Finance Conference. Washington, DC.

McPartlan, P., Rutherford, T., Rodriguez, F., Shaffer, J. (2017, August). Modality motivation: Assessing motivational differences in online and face-to-face students. Awarded best graduate student poster at the annual meeting of the American Psychological Association Conference. Washington, DC.

Rivas, M., Rodriguez, F., Rutherford, T., & Warschauer, M. (2017). Comparing student motivation and performance between a flipped and traditional college class. Poster presented at the annual meeting of the American Education Research Association. San Antonio, TX.

Kunze, A. & Rutherford, T. (2017, August). Listening to their views: Student perceptions of instruction in online and face-to-face environments. Poster presented at the annual meeting of the American Psychological Association. Washington, D.C.

Training and Software Products

Intro to Python for Data Science

Python for Clickstream Analysis

Canvas API Crawler for Accessing Learning Analytic Data