Work in Progress

Azevedo, R., Moos, D. C., Johnson, A. M., & Chauncey, A. D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues & challenges. Educational Psychologist, 45(4), 210–223. https://doi.org/10.1080/00461520.2010.515934

Azevedo et al. (2010) discussed methods of measuring how students regulate their own learning activities in a hypermedia environment. The authors explain that self-regulated learning is not a single skill that can be measured either prior to or after a learning activity. Self-regulated learning is an ongoing process that involves setting goals, using prior knowledge, selecting strategies, monitoring progress, and adjusting based on the results of that monitoring. The article focused on online trace methods, including verbalizations during problem-solving activities (think-aloud), tracking how the learner navigates through the materials, mouse clicks and movements, time spent assessing resources, and interaction with pedagogical agents. To show how technology can help identify and support self-regulated learning processes, the authors use MetaTutor, a computer-based learning system designed for teaching science. The authors also identified some challenges, including correctly interpreting student behavior, evaluating the quality of students’ notes or goals, and providing timely feedback.
This article does not assume that every click or action represents evidence of learning. The authors state that researchers need to use several different types of data to understand what a student thinks or does while engaged in a learning activity. Examples from MetaTutor and data from 44 participants illustrate the complexity of collecting and analyzing data related to self-regulated learning processes. Authors acknowledge the limitations of using think-aloud protocols. Think-aloud protocols provide very detailed descriptions of how learners complete tasks; however, they can take a long time to administer, transcribe, and analyze by trained personnel. This article is focused on the conceptual framework and methodological approaches to instructional technologies instead of an evaluation of an instructional technology. As a result, this article raises a number of potential research questions; however, very few definitive answers concerning which technologies assist in supporting self-regulated learning.
I found this article relevant to my interest in online learning in PCBA manufacturing. For instance, if a PCBA operator has been trained on an LMS, the operator’s completion of ESD, equipment operations, safety and quality standard training would be reported to the LMS. Reports may include only pass/fail results or test scores and will not demonstrate whether the operator can identify a defective product and resolve the issue on the production line. A more effective LMS would evaluate how an operator responds to realistic scenarios using Augmented Reality (AR) or Virtual Reality (VR) technology; whether operators know what resources to find, when and whom they report to and request assistance from.