An Instrument for Observing Teams to Explicate Regulation Strategies in Computer Science

Wortmann, Carolin; Vahrenhold, Jan

Zusammenfassung

Motivation and Objectives. Informed by a long tradition of studying regulation strategies in general education, recent work in Computing Education Research has highlighted the importance and effects of such strategies in Computer Science classrooms as well. Most of this work has focused on self-regulation of individual or dyads, e.g., in pair programming. Little, however, is known about regulation in larger groups which naturally occur in, e.g., active-learning classrooms, Software Engineering lab courses, or capstone projects. Moreover, research involving regulation strategies, including, but not limited to Computer Science contexts predominantly relies on self-reported data. While recent work has advocated using trace data from learning management systems to assess self-regulation strategies more objectively, such systems cannot capture regulation as it occurs in many version of group work, e.g., in group discussions and interactions in a classroom. We thus aim to develop an observation instrument that can help researchers to explicate regulation strategies in Computer Science group work. Together with self-reported data and – where applicable – trace data, such an instrument would facilitate obtaining a more complete picture of where and how regulation strategies emerge and develop. Methods. Using data from semi-structured interviews with students from two capstone projects, we conducted a deductive qualitative analysis to add concrete descriptors to Miller and Hadwin’s framework for regulated learning. We refined the resulting coding scheme into an observation instrument that was field-tested for saturation, ease-of-use, and reliability in a Software Engineering lab course. Results. Our main result is OTTERS, an instrument for observing teams to explicate regulation strategies in Computer Science. Initial field-testing indicate that this instrument, despite its granularity, is easy to use and reliable. Furthermore, first proof-of-concept observations lead to patterns of regulation activities that are clearly distinguishable thus suggesting construct validity. Discussion. Whereas previous work focused on self-reported data or trace-data obtained from learning management systems, our observation instrument adds another facet to assessing regulation strategies. Compared to self-reported data, it trades off the bias of possibly unreliable self-assessment and the limitations of external observations, compared to analyses based on high-resolution, objective log data it yields observations on a coarser granularity but is applicable not only to self-regulation but to socially shared regulation and co-regulation as well. Our observation instrument thus complements the current methodological portfolio of assessing regulation strategies in Computer Science contexts.