Reliable individual differences in researcher performance capacity estimates: Evaluating productivity as explanatory variable.

Forthmann, B., Beisemann, M., Doebler, P., & Mutz, R.

Abstract

Are latent variables of researcher performance capacity merely elaborate proxies of pro- ductivity? To investigate this research question, we propose extensions of recently used item-response theory models for the estimation of researcher performance capacity. We argue that productivity should be considered as a potential explanatory variable of reliable individual differences between researchers. Thus, we extend the Conway-Maxwell Pois- son counts model and a negative binomial counts model by incorporating productivity as a person-covariate. We estimated six different models: a model without productivity as item and person-covariate, a model with raw productivity as person-covariate, a model with log- productivity as person covariate, a model that treats log-productivity as a known offset, a model with item-specific influences of productivity, and a model with item-specific influ- ences of productivity as well as academic age as person-covariate. We found that the model with item-specific influences of productivity fitted two samples of social science research- ers best. In the first dataset, reliable individual differences decreased substantially from excellent reliability when productivity is not modeled at all to inacceptable levels of reli- ability when productivity is controlled as a person-covariate, while in the second dataset reliability decreased only negligibly. This all emphasizes the critical role of productivity in researcher performance capacity estimation.