Computational Thinking in ICILS 2023: Analyzing the Construct and Its Antecedent- and Process-Level Predictors

Vahrenhold, Jan; Niemann, Jan; Drossel, Kerstin

Abstract

Motivation and Objectives. Computational Thinking (CT) has become a central theme in K–12 Computer Science education. Over the past twenty years, multiple conceptualizations of CT have emerged, many forming the basis for assessment instruments. One such conceptualization was developed for the large-scale International Computer and Information Literacy Study (ICILS), which assessed CT across 24 countries using representative sampling. The size and sampling quality of the ICILS data set allow for robust statistical analyses which in turn will be of interest to researchers and policy-makers alike. This study situates the ICILS 2023 conceptualization of CT within other established frameworks and conducts a secondary analysis of the ICILS 2023 CT data on non-cognitive antecedents and processes. Methods. Structured deductive content analyses compare the ICILS 2023 items with those from the Bebras Challenge on Informatics and Computational Thinking [13] (Bebras) and the Computational Thinking Test [55]) (CTt), mapped across three CT frameworks—ICILS [28], Shute et al. [65] and Weintrop et al. [71]—and aligned with Bloom’s revised taxonomy [2]. Linear regression analyses on the data of the 20 educational contexts that provided not only CT performance data but also a complete coverage of student data relative to the predictors of CT performance studied in prior work examine the predictive effect of non-cognitive factors on CT performance. Results. The qualitative analyses showed that the ICILS 2023 CT items can be mapped to existing frameworks. Conversely, items from both Bebras and CTt can be mapped to the ICILS framework. The distinct, partially overlapping profiles of the instruments across the frameworks as well as Bloom’s taxonomy indicate that they are complementary in assessing CT, confirming and expanding prior comparisons of Bebras and CTt. The regression analyses indicate no single dominant predictor of CT performance. The association of socioeconomic status, gender, or the home language was consistent with prior findings, predictors related to learning processes, however, vary across educational contexts. Discussion. Our results demonstrate that ICILS 2023 items can be mapped onto multiple established CT frameworks, supporting their broader validity and utility for comparative research. The findings of the regression analysis underscore the complex interplay of non-cognitive factors affecting CT and illustrate the significance of contextual interpretation within educational systems.