Measurements of susceptibility to anchoring are unreliable: Meta-analytic evidence from more than 50,000 anchored estimates.

Röseler, L., Weber, L., Helgerth, K. A., Stich, E., Günther, M., Tegethoff, P., Wagner, F.S., Schütz, A.

Abstract

Theories on anchoring effects—the assimilation of numerical estimates toward previously considered numbers—have been used to derive hypotheses that susceptibility to anchoring is correlated with certain personality traits. Thus, for the last decade, a considerable amount of research has investigated relationships between people’s susceptibility to anchoring and personality traits (e.g., intelligence, the Big Five, narcissism). However, many of the findings are contradictory. We suspect that this inconsistency is grounded in imprecise measurements. Unfortunately, few reports have disclosed estimates of the susceptibility scores’ reliability (e.g., Cronbach’s Alpha or interitem correlations). We created a large and open data set of anchoring susceptibility scores and conducted a meta-analysis to test how extensive the reliability problem is. Results suggest that the reliability of most tasks is very low. In the few cases in which the reliability is acceptable, the validity of the anchoring scores is questionable. We discuss requirements for further attempts to solve the reliability problem.