The PISA 2022 Creative Thinking Assessment: A welcome opportunity to explore the mechanics of flexibility scoring.

Forthmann, B.

Abstract

The PISA 2022 Creative Thinking Assessment measures the ability to generate diverse ideas, that is, flexibility. However, since test-takers only provide up to three responses, scoring flexibility is challenging. Employed scoring systems rely on dimensions such as linguistic diversity or category systems to quantify how dissimilar a test-taker's responses are. In this work, I will revisit the flexibility scoring in the PISA 2022 Creative Thinking Assessment from the perspective of general diversity measure frameworks, which distinguish variety, balance, and disparity, in the hope of identifying ways to gain more information on test-takers' performances. An empirical illustration demonstrates the potential of diversity scores to better differentiate between different response patterns.